This necessarily doesn't have to do with his trauma from the underdark but I remember when Halsin shares that his line perished a long time ago, he even says the wounds don't heal but over time they get a bit more bearable. So I like to think it's still there his sadness about it and whatever else he's dealing with.

Also maybe off topic but last night I decided to visit the drow twins on a different save (completely avoided the brothel in general at all costs in my actual playthrough) and when you probe him a bit about his trauma in the underdark, he says something that bad/traumatic experiences can be turned into cherished memories??? I can't remember what he exactly said word per word but it really rubbed me the wrong way. In Act 2 with his family/line perished, he seemed to be vulnerable about it and accepting he hasn't healed from it, but with this in Act 3, he's seeing what he went through as something to be a happy about to reminisce from time to time?? Make it make sense because I can't at this point.