[SPOILERS AHEAD - both for BG3 and DOS2] Before reading my reply, note the following: I am completely green to this type of RPG games, even more so with DnD. I've played (and very much enjoyed) DOS2 which is why I am here. So do consider that this is from my newbie perspective.
Answer to Part One - Are the story and characters really the problem?
- Before I bought the game in EA I read the feedback on Steam. The most convincing negative feedbacks were mostly regarding followers - the one I was most convinced by basically said "all followers are evil and will kill you in your sleep" - so when I finally bought the game anyways, this was the standard I had when I met them.
I was shocked when I actually started talking to the companions. Sure, I didn't exactly find any character as instantly intriguing as Ifan in DOS2 (I adore him, he is AWESOME - love at first sight... but to be fair - I love all DOS origin characters <3) - and the characters were so much not-befitting-my-character that I even chose to not even attempt to romance anyone - but they were seriously not "evil" at all. The most evil option is the lady you meet at the beginning (sorry, I do not recall her name) - but I wouldn't really classify her as evil. She's more like a dedicated warrior (results > moral), imo. And the vampire dude even apologizes after getting caught trying to suck your blood - that's not very evil... And to be fair, their personalities are not bad (as in poorly written) - I actually think they are rather fine, especially since Larian apparently said that they'd introduce the "least picked" characters first (- don't quote me on this, I just saw someone claiming this on the forums). And the claims that they treat the player poorly? I just disagree - they behave rather naturally considering they don't know us, and we don't know them. I think most of them softened up very quickly, all things considered (like vampire dude). Some of them come off as arrogant, but tbh - so did Sebille in the beginning (I even killed her during my first playthrough :x I didn't notice she was a companion until much later). And in Sebille's case, I thought much worse of her in the beginning than any current companion in BG3.
Answer to Part Two - The Problem In a Nutshell
I definitely think you're on to something about the flag-system. Especially regarding your companions being completely interactive during many conversations (like Red Prince / Sebille in DOS2). But I don't think it is about not caring, rather than Larian just wanting too much without being able to finish it - I mean, in DOS2 the wiki recommends people not to have certain characters at the same time cause their story conflicts with each other etc - that's just not okay.

There are so few origin characters, the LEAST they could do is making sure the game experience is playable with them all.
But, moving on to my personal experiences - I've noticed the scripped/flag issue behaviour both in DOS2 and BG3, and here are a few examples:
BG3 - after the seductive dream:
* After the seductive dream, I talked to all of my followers to see if they had the same dream and what they said about it. I spoke to the yellow lady first and she read my thoughts and saw that I declined the man in the dream (purple robes of that kind on men is not my thing - I do hope they let us alter their clothing later on) and praised me for it which she and the cleric lady approved of. But when I spoke to the cleric lady, she seemed to just assume that I slept with the man in the dream even though she should know that I didn't (cause she reacted to it). Same with all other characters, really. But I didn't find the other characters response as weird as the cleric lady cause she actually reacted to me rejecting him.
BG3 - After killing the first goblin that Wyll is looking for (the one with the gnome at the windmill)
* the goblin gave up, Wyll got his information (where to find Spike etc) and instructed me to kill it. I killed it (cause muck that goblin :x) and when I talk to Wyll afterwards he says something like "Damn it all - I wanted him alive not dead." and I am sitting there like "????".
* Same encounter as above, Wyll was (during combat) ambushing their backline and the vampire dude was next to the goblin in question. The goblin gave up, and I was only presented with two choices: let him run, or kill him. Wyll was standing right next to me, but did not interact with the goblin at all. I got the result I stated above after letting vampire dude attack the backline and have Wyll go for the goblin boss.
( I retried the encounter like 10 times, I either got the first or second result stated above. )
BG3 - right after the celebration thrown after saving the druid grove and the thieflings (? spellcheck)
* Gale told me during the celebration that he'd explain everything (long bedtime story?) the next evening... The next evening, however, I was visited by the half-devil. Next evening after that - nothing and Gale seems to have forgotten about the conversation now... >.<
* I am not sure if this is intended or not - but literally *ALL* characters that I interacted with had something sexual to say to my character during this evening. And I have not used one flirt option on ANYONE (as far as I know). Yellow lady went on about how I'd wonder what her lips would taste like and how I could have earned a place alongside her in bed, Wyll admitted to be jealous cause vampire dude asked me out (????), Gale said that I already had company for the evening and that he also felt rather jealous (?????), the vampire dude said the wine was dissatisfying and that he wanted some quality time when everyone else was sleeping and the cleric lady said something about that it could have been us, but I neglected her (?????) - are they all thirsty drinkers or wtf? xD
DOS2 (skipping the stuff about Red Prince and Sebille interactions crashing together, most people who played DOS2 knows about it):
* In certain conversations, such as the one with the Lohar dude (Beast > player), the Doctor (Beast > player), Alexander (Ifan - unless resorted to combat - > Player) and the Shadow Prince (Sebille > Player), you gotta interact with them TWICE to actually get somewhere in the conversation. First it has a follower dialogue and then it has a casual dialogue with you like the previous one never happened. Having a proper conversation that actually includes all part at once according to flags (like you described Bioware having?) would seem much better.
* After certain events during DOS2 I interacted with my companions to get their thoughts on the matter - more often than not, though, I couldn't even ask them about it. Even if it was rather big stuff involving them personally. Generally, I found all characters lacking in response to actions and consequences, but I felt like I had more options to talk to Ifan (might been due to romance run though) regarding current events than the other ones. As an example, Fane remained depressed for pretty much entire act 2 ("do whatever, what's the point"), even after some crucial discovers and in act 4 when I wanted to hear his thought about the deathfog in the sewers he just said "Arx really isn't that bad!"... Like what! And Beast didn't have much at all to say regarding all the deathfog during act 4 either. BG3 does this much better, to be honest. Even when there isn't an " ! " above the companions head, sometimes they'll have something interesting to say anyways! (or not, if you declined the vampires romantic advances he'll keep spamming "if you're looking for second chances, am afraid you missed your opportunity..." - like what? xD Idc about that, what do you think of the god damn hag in the swamp!?! )
Answer to Part Three - A Bug or Design Philosophy?- about the question of whether this would be fixed in the main release
I honestly don't think it is a design philosophy - but rather just a case of being in over their heads. Like I said, I am completely new to these kinds of games - but I am not (as) new to the concept of programming games. I do agree they seem to prefer maximum character freedom before functionality and depth but I dismiss the thought of Larian doing this on purpose. I doubt we will see all of these things fixed (except for crucial stuff like the Wyll conversation regarding the goblin) but I'd assume they'd try to fix most of them or at least as much as they can. They kind of have to if they want the game to be as big as they want/need it to be.
Answer to Epilogue - Moving Foreward
I could see them do any of these options, to be honest. And no matter which one they choose, I'll be okay with it. Might not become my favorite game in the universe and depending on how much effort they put into it, I might turn to new developers in the future for my new thirst for RPG games - but I'll play BG3 and enjoy my "harmless fun" since I already paid for it. >.< To be honest, if they put the bar at mediocre, then they'll fall out of the business soon. Not even the giants among companies like Electronic Arts or Blizzard is sitting safe with the industry growing so rapidly - laziness, unwillingness to improve and not being willing to go just a little bit further is going to be the end of companies soon enough.
All in all, I agree with most things that you said. Thank you for taking the time to write your post!