If you want to experiment try wearing a warm sweater or fleece shirt over a cotton T neck and then over a poly propylene, silk, wool, or treated polyester T neck at the same temperatures.
I said treated polyester because it is the same as cotton unless it is treated to be hydrophobic. Cheap polyester is the same as cotton. It can be made similar to wool or polypropylene but then it is not cheap.
BTW, the cotton T neck is an easy way to wear fancy ski sweaters in warmish rooms. ;-)