This happened to me around the hag's lair. It was definitely connected with stealth, but not necessarily the group stealth toggle. It happened once when I hid to pick a lock on the teahouse and had Shadowheart (not stealthed) cast guidance in the check. After this, when I unstealthed, she wouldn't follow as described in other comments.
