I'm the same. For a few years, I'd watch NCAA basketball beginning with the Sweet Sixteen through the National Championship but I think the last time I did that was 2014 when Wichita State (my alma mater) was in the final four (Louisville, Michigan, Syracuse, Wichita State). I have exactly 3 basketball games in my digital sports stash and they were all games with Wichita State in that magical run towards the 2013 Championship. Louisville won the Championship that year.
Anyhow, I can't explain it either. My dad, sister, and brothers all love basketball but it just doesn't do it for me. Not even close to the thrill I get from watching football. My interest is usually piqued, however, if there's some team in the tournament from Kansas or St Louis but that's as far as I go.