When does this happen? (during a turn, when changing to your character's turn, or an opponent's turn, etc)
Can you try with a USB connection, rather than wireless?
Do you have a second controller you could use to start a split screen co-op game, and see if one character remains responsive out of combat, if the other gets stuck in combat?
Try clearing the PS4 cache: press and hold the PlayStation button on the controller, select 'Turn Off PS4', wait another 30 second after it finishes shutting down, and start it again.