An experienced, retired ref at each stadium with access to replay - can call down to officials on the field and say 'you missed one' or' he never hit the QB's head'. Calls would match up to what people can see at home in hi-def with multiple super slow mo angles.
let coaches challenge everything - just keep the same number of challenges as now - game won't be any longer. Under 2 minutes all review goes to sky judge. Sky judge can also rule on things he / she catches that's egregious that the field refs miss. Kind of like how plays are often reviewed by the NHL in Toronto.
If you keep the same sky judge with the same field crew - you build up a continuity / trust.
The technology available to viewers at home is not going backwards. The genie is out of the bottle - we're not going back to the no-replay days. It's just not happening.
