Just do all the side quest and clear out all the scattered pockets of creatures and you should be level 10 or so before you get there. Plenty of creatures all around to get some needed EXP.
Find a good bow with magic damage and build up your intelligence a little to boost the magic damage.
Most of those guys on Lovis tower are archers so get up high or behind something if you can and peek out and pick them off one by one. Plus you can time when they are going to shoot and either jump or move to the side a little and they'll miss.
If you can get up high the melee guys will not be able to get to you at all so they are out of the picture till the archers are gone. Summon your creature as well as he will distract them by getting in close and forcing them to go to melee weapons.
I'm a ranger as well and only have 1 set point in poison arrow and splitting arrow. I have one in ranger stealth and ranger strength as well but those are from enchantments or charms. Ranger Strength would be good I'd think as it adds to both magic and regular damage I think.
Oh the poison and splitting arrow I never even actually use. I just upped my dexterity and intelligence and that more then compensates for those skills I think.
I use my summoned creatures to run interference mostly while I pick away right at the edge of my range. A good bow with good damage goes a long way I feel. Enchant that baby up if you can with extra magic or normal damage. The summoned creatures work extremely well in tunnels and such as the bad guys will not be able to get by them till their dead. You just stand back and pick them off before that even happens. Little bit cheatingish by using the guys to block the way but it works.