You are correct that the only difference between the game you get to play from the CE versus the Digital Deluxe edition you'd get a free upgrade to as an Early Access player is the CE-specific dice skin. The bulk of the value (and presumably cost) of the CE is all the physical stuff it comes with.
As far as I'm aware, it's not possible to buy all that physical stuff without also getting a Steam (or PS5) key for the game along with it, and correspondingly there's no discount on the CE for EA purchasers. That is, there is no upgrade path from early access to CE, so if you want the latter you'll just have to purchase it and end up with two copies of the game.
It has been a bone of contention for some, I know, but I believe that remains Larian's position so we just need to decide whether or not to buy the CE on the basis of it. You can always contact them directly (details of how to contact their support team can be found on their website) if you have any remaining uncertainty.
