Pinning down your specific problem is probably not helped by the continuing click bugs with selection and targeting that Larian have continued to refuse to acknowledge; any number of bugs exist currently with targeting portraits, selecting things, moving skills on and off your bars, selecting objects from menus, movement not registering, or requiring a forceful deselect while nothing is selected or targeted at all to reset itself... etc... all of these bugs continue to persist, with no acknowledgement despite numerous reports. If you aren't having any issue with you mouse in any other game or situation on your computer, then please file a bug report with Larian's formal form and hope for the best.
