Not all types of 'cold' are the same.. not even CLOSE to the same.
On my backpacking trips I spent numerous nights sleeping in a sleeping bad in sub-freezing weather... on a couple of occasions below 0º. NONE of the times was as miserable as being at a football game here in my home town when the weather is in the high 40's and the fog was as thick as pea soup.
I've skied many times when the temp never rose above freezing. Cold? Yes... but NOTHING like being in San Francisco on a 50º day with the fog rolling in.
They say 'dry' colds are more tolerable... I tend to agree with that... IF I understand it correctly. We only have 'wet' colds where I live so I'm not a good judge of that.
