Genuine question. As both games share the same engine. Could it be possible to make mods to play the D:OE and D:OE2 using BG3 classes and monsters?
Nope. Dos1 isn't compatible with Dos2 either, and they are far closer in terms of differences in engine (It's not technically the same engine, as it's just an in-house engine that tools developers iterate on and chance as the project requires. It's not a 1:1 "same engine", sadly. The model meshes and textures from Dos2 would look terrible in BG3 too.
There are more technical reasons why it doesn't work like that, but no real good way of simplifying it in a concise fashion. Basically not a chance, not in a long shot.
