You don't have any saves before entering the temple, that you can reload? Even saves inside may help, if you can avoid using as many potions early.
In the bandit camp temple, you can stay on the blood trail to avoid being poisoned, other than the raft section you have to jump across. If you just stand and wait for your health to recover (you should be able to move out of the way of the path of the fireballs from the walls, without getting into the poisoned area), you should hopefully be able to get across without dying.
