Car or backpacking I bring a pair of clothes to sleep in. Light weight in summer and heavier for winter. Having a clean pair of clothes for sleeping in is so much more comfortable. Even when I think I am relatively clean it is noticeable to put on those clean sleeping clothes.
Our Scout group is going to Philmont next year and they require separate clothing for sleeping in and that you must not cook in those clothes. The bears are quite active in the area.