Are you talking of the cellar accessible by a wooden hatch (which leads to a sort of lab) *or* the one you either leap into through an old web or by unlocking a huge door ? (which leads to a bunch of artisan workstations)
If you are talking about the one under the wooden hatch, with a large cupboard behind scratch marks on the floor ...
Here is a spoiler about the cupboard :
there is a hidden switch : you may need to move some smaller items to see it.
I failed to find it myself, but a friend in multiplayer showed me.