I would think that when it is hot you're getting hit with more "rays" from the sun. When it's colder, for whatever reason, there isn't the same volume of rays available for you to soak up.

Your question could also be worded as why is it hot some days and cold others even when the distance from the sun is equal, there is an equal number of clouds, et cetera.

Dunno though. >_>