Milk out of fridge.......?

I keep a small quarter pint of milk in my mini fridge which is on all day. I turn the fridge off at night and the milk is ok for tea in the morning then I usually throw it out. However, I have started buying a few of those little quarter pints at a time and freezing them, and putting one in my mini fridge as i need it, and its still very fresh the next day, but this time last week, I put some frozen milk in the fridge with the fridge on, turned it off at night and in the morning it was still cool and fresh, but that day I forgot to turn my fridge on in the morning, and went to throw the milk out the next morning when I remembered, and it didn't smell at all, a whole day unrefrigerated. I threw it out and didn't chance it, but for future reference, would it have been safe if it smelled ok? i even took one sip before chucking it and it tasted ok. I didn't ingest enough to make me ill, but would it be ok?