Only if you expect money to be an issue - which it never is for me.
Meanwhile those charms will help keep you alive to get to act 4.
And I could argue that it is a waste to spend a fortune for good gear near the end of the game when you won't have many fights left to use it in.
That being said, I don't think I have ever bought a charm - I'm far too cheap LOL even with hundreds of thousands of cash piling up.
