Well, in many cases if a person notices you casting a spell in a conversation, they might start being hostile right away. It's up to DM.
That can be applied even here ...
For one, (lets put aside for a second that EVERYTHING is up to DM ... including rules that are written) right now Larian is our DM and as it seems thei alowed it.
And for two, i would honestly dislike such approach from my DM ... second person next to the charmed can notice, and become unfriendly (if not exactly hostile) why not, that would make sense ... but not that person who is charmed, yes he will be aware that i charmed him once the spell goes off, but right now he is under the effect of spell that litteraly makes him believe that we are friends ... there is no way he could be hostile to me, UNLESS he is also caster and used counterspell.
