In the controller UI, once you get into the puzzle use the left stick to highlight blocks with the pointer, and then the X button to rotate the selected block (A with an Xbox compatible controller).