Withdrawing usually deals with a sneak and walking to break the tether or a transition into another area. Just killing LOS doesnt stop them "knowing" - which is essentially "I saw them here literally 6 seconds ago, Im going to keep walking in this direction"
The thing is - i was hidden for five turns sneaking away from battle. All enemies were inactive, just standing and not even tried to investigate (but that's another issue).