You are a Lone Wolf Elf with Adrenaline. That's 6 starting AP plus 1 from flesh sacrifice and +1 from Adrenaline. You're more than equipped to handle switching weapons.

That's the price you pay for flexibility. Plan your approach so that you have to switch as little as possible.