I'm going off memory here admittedly but it really seems like a weakness of his. And what's irritating is he could fix that by hiring someone who has one job on gameday which is watching replay and yelling for them to challenge the call when appropriate.
Currently he will challenge at times just because it was a big play, as if hoping to find something. It's crazy and something he needs to improve at going forward. Great as he is he's like anyone else where there's always things to improve on. I'm sure he's aware of it too. He has to be.