this pattern for an idea thanks to JohnSawyer.
I added a couple inches to the perimeter to match my foot size(11) and it worked out great.
Also, I made double of everything so I could make the pattern work with down. Overall I found it a lot easier than I thought and I plan on making another pair for winter camping.
I made this to go with three season because they weigh less then my normal sleep socks and are more comfy.