The problem is that there are many similar games that did this very effectively (Skyrim, DA:O), so it absolutely can be done in a way that does not take away from the experience.
I had DA:O very much in mind with my post above. It restricts you to just 3 companions which is a significant irritation given the prolific amount of inter-companion banter there is, and Bioware tried to compensate by allowing you to swap companions at almost any time, with a fourth-wall breaking teleport rather than requiring you to actually find and meet like the Infinity Engine series did. Now this could be to keep pathfinding simple (something Bioware seems to have had long-standing issues with during the Infinity Engine and Aurora games) but the link with 4-button console controllers seems hard to overlook.
You also have the lack of environment effects with spells. DA:O is capable of environmental interactions much like D:OS is, with fireballs setting fire to pools of oil. Yet I only recall one location (inside a building) that made major use of this. Why? Well the console version lacks the zoomed-out top-down view needed to target spells to make use of such features. So a limitation of the console interface seems to have had an impact on the game environment for all versions.
Skyrim, I can't comment on (boycotted it due to Steamworks DRM).
If the game's distinctiveness and individuality would be compromised by adding support for a peripheral that MANY gamers prefer, it has a lot more problems than anyone has realized.
Then add gamepad support yourself using XPadder
. If it turns out easy and playable, by all means provide the details here. If not, you'll at least have gained UI experience.