<img src="/ubbthreads/images/graemlins/exclamation.gif" alt="" /> If you talked to Janus after using the teleporter stones to get into the castle and accepted the quest to look for the cat, you have broken a main plot quest, and will not be able to complete it, or advance farther, when you eventually enter by invitation.
If that is the case, reload an earlier save if you wish to complete the game.
The duke can not be targeted to attack, magic is suppressed in the castle (except for one cellar) and he is unaffected by the warrior's spin attack or thrown poison bottles. I suspect even if you tried the mage's special move and were able to swap places with Janus until he was out of the castle, area effect spells that do not need a target would most likely not effect him either.
