Just adding to the brainstorming here...
Perhaps you could skip the plastic all together. Instead, maybe you could apply JBWeld or epoxy to the aluminum, and let it cure. Then sand it down to the OD you need to fit the female end of the other pole.
I've been thinking of a way to fabricate lightweight telescoping poles myself and found these:
Probably don't even need them if the aluminum poles fit tightly enough together, but thought I'd share.