Fitting your GPS to your goals is fine advice. I would add that getting one with an internal compass is WELL worth the cost. Without it, you have to be moving to get proper bearings, and this can be difficult at times.
I personally have two of the Garmin Rino series, which is both GPS and two-way radio. It has an alarm, altimeter, weather radio, and best of all it can transmit your location to another Rino user. (It also works with other radio types,without position transmit capability)
This is certainly not the least expensive model, but if you're in a situation where you'd be carrying a two-way or weather radio anyway, it saves weight by combining two items you'd have.
The harder I work, the luckier I get.