Im not saying get rid of waypoints. Just limit them when in dangerous places like the gobbo camp.
Think of it like you can't fast travel until you clear out all the bad guys. That's what Im now suggesting. If no random encounters, at least limit fast travel if the way to a Waypoint isn't clear.
As for gear limit, Im just talking real weight limits according to D&D 5e rules. At this point seems people in general don't want too many limits, but 5e rules are reasonable.