I don't remember exactly when you can get into the swamp, but the fights in there are pretty difficult for level 5-6 characters... unless you're playing in Explorer mode, in which case no difficulty whatsoever should be expected :P