Brown adipose tissue (BAT) contributes to cardiometabolic health by taking up glucose and lipids for oxidation, a process that displays a strong diurnal rhythm. While aging has been shown to reduce thermogenic characteristics of BAT, it is as yet unknown whether this reduction is specific to the time of day. Therefore, we assessed whole-body and BAT energy metabolism in young and middle-aged male and female C57BL/6J mice and studied the consequences for lipid metabolism in humanized APOE*3-Leiden.CETP mice (also on a C57BL/6J background). We demonstrate that in middle-aged versus young mice body temperature is lower in both male and female mice, while uptake of triglyceride (TG)-derived fatty acids (FAs) by BAT, reflecting metabolic activity, is attenuated at its peak at the onset of the dark (wakeful) phase in female mice. This coincided with delayed plasma clearance of TG-rich lipoproteins and TG-depleted lipoprotein core remnants, and elevated plasma TGs at the same time point. Furthermore, middle-aged female mice showed increased adiposity, accompanied by lipid accumulation, increased expression of genes involved in lipogenesis, and reduced expression of genes involved in fat oxidation and the intracellular clock machinery in BAT. Peak abundance of lipoprotein lipase (LPL), a crucial regulator of FA uptake, was attenuated in BAT. Our findings suggest that LPL is a potential therapeutic target for restoring diurnal metabolic BAT activity, and that efficiency of strategies targeting BAT may be improved by including time of day as an important factor.