Damage directly caused by you is affected by your talents, stats, skills, build, etc. Damage that comes from a surface interaction is based upon your level. I have tried and tried and tried to see if there is anything to increase surface damage and nothing I've done can reliably show any difference. It may also be that we don't have enough skill points to actually make a difference at low level, but there will be at higher level. I can 100% say that your level effects how much damage is taken from surfaces.