I'm like you... I think that's pretty even across the board. I think it's obvious that the more you know about a sport the more you'll think it's 'more' cerebral than others.
Used to know a couple of horse trainers/owners, who raced their horses up and down California, nearby states. That is a 'sport' where you'd think there would be little 'game planning'... other than "Go Fast" what's there to really plan?
BUT... listening to them plan for me was like listening to Brian Greene talk astro-physics.
I have mentioned several times that football, today, has passed me by. And it has. But it's NOT because I don't understand the game, even the 'new' game, or the strategy... because that has NOT actually changed that much.. it's the 'new' terminology - and I haven't kept up with it. So if I bother the hell out of people asking "Ok, what does that term mean?"... I can quickly understand and realize it's almost always just something 'old' made 'new again' by new terminology. Problem is: Old now, don't really care. Just wanna watch.