If you're only in the Maxos Temple, you are terribly overleveled for the main plot. If you're having trouble in Orobas Fjords now, go do the main plot, you should be able to breeze through that pretty easily. You are not meant to be doing the fjords that early, and there's only so much you can do there anyway right now.
***
Defense doesn't matter all as much as other stats. Enemies hit so hard that defense is less useful than offense. Sets are good for the cumulative bonuses you get for wearing more and more pieces, and those tend to outweigh the downsides.
Bonuses do not exceed the attribute cap. I would definitely stop putting points into Intelligence around 60-70 points or so, until your end-game gear is more finalized. There are a couple extra bonuses to intelligence you can get later on. If you're worried, hold on to any +5 Intelligence Charms and keep some attribute points in reserve.
I carefully planned my most recent character (a warrior), and I got a lot of use from attribute bonuses, enough to max out strength and vitality, and get middling scores in Intelligence and Spirit as well (I probably put too much into Spirit, but I was thinking of Thousand Strikes).
***
I'm currently trying out a priest build, and while the Ghost does work well, it will fall behind and become worthless very quickly if you neglect to upgrade it. Keep the Ghost skill as high as you can for your current level and it should do fine. The healing scales well, too, even towards the end. I specced a build using a maxed ghost on a character entering the expansion, and a properly leveled ghost can heal several hundred HP instantly.
But even properly leveled, the Ghost is NOT a worthwhile attacker, it does negligible damage.
