Have you looked at your windows lately?  Have you felt them when it's cold or hot?  

They are also a source of energy loss.