My problem with their implementation of Oathbreaker's has nothing to do with Rules as Written - it relates to game content that gets broken when you accidentally or purposefully break your oath.
You basically, In my experience, lose ALL access to Withers functions. You can't resurrect any NPC's or yourself, you can't hire mercenaries, etc. He states he doesn't know you and you need to resume your original oath before accessing him. This happens whether you are an Oathbreaker before or after you encounter Withers in the temple.
Which ever way you want to play an Oathbreaker or get stuck with it as a consequence of your actions, the game shouldn't break for you for taking a legitimate game choice as offered by the devs.
Exactly this. I love playing oathbreaker, but I just want to adjust my ability scores, feats, etc. I absolutely do not want to reclaim my oath, but basic game functions are being blocked behind a huge gold sink. Even with that, I took the gold sink and reclaimed and respecced...but now there does not appear to be a way to break my oath again...without potentially breaking end game quest lines and companions potentially. I went back and killed all the remaining goblins from now neutral blighted village. I fnished off all the npcs at the creche. I'm looking hard at killing off any remaining druids at emerald, but afraid it will break something end game if I do. It shouldn't be this problematic to just want to respec ability scores/feats. No other class has this much of an over the top struggle for a basic game function. Just let withers respec everything except the oath already.