My grandfather had some... small dairy along with the farm. We may have had some the first few years after I was born but I don't remember it. I did raise a dairy cow once for 4-H.
However, there were several dairies around me until I was about 10 or so, so yes... I drank a lot of fresh milk. good comparison by the way - using German Beer.