Larian Studios Forums Baldur's Gate III Baldur's Gate III - General Slow Forum Performance

no wait, THANK YOU WITHERS

I think I know why the forums can sometimes get slow, if you take a look at this page :
https://forums.larian.com/ubbthreads.php?ubb=online
You can see that there are robots crawling the forum. This is expected since those are used to index content for search engine. However, at the time that I write this post, there are 86 instances of the Aspiegel PetalBot crawler currently indexing pages in this forum. This one is known as a very aggressive Chinese crawler that can hog a considerable amount of bandwidth of a server, see : https://www.grantforrest.me.uk/index.php/software/16-petalbot-another-pain-in-the-bot
If the forums slow down to a crawl again, Larian might consider to blacklist some misbehaving bots.

That was our theory. Unfortunately badly behaved bots are the ones that tend to ignore the robots.txt files, so they can be hard to get rid of, but our admin vometia had requested a number of steps that should help. Unfortunately she's off at the moment so I'm not sure exactly where we are on that.

I've also been (somewhat casually, I admit) tracking forum performance against bot presence and though there's some indication that it could be a contributing factor, the correlation isn't really strong enough to blame it for all our woes. Aspiegel PetalBot is my top suspect, with Yandex a runner up, but there are times we've had 60 instances of it crawling around the site and performance has been fine, and other times when there have been 20-30 and it's been terrible. But I agree bots could be behind at least some slowdowns, so protecting against them as best we can could help.

I'm not sure when vometia will be back, but I'll see if there's a way to get a progress update from someone else.

Apologies for OT, I didn't want to make a new thread just for this, and it seems the least OT here vs other threads.

When did the banner get updated with the right date?

Also, yay! The banner is updated!

I'm still experiencing the same slowdowns that I reported more than 2 years ago, even when the US are still asleep.

More precisely, I'm still noticing the same delay the first time I access the forums (new session); it may be when the server has to create the 'ubbt_mysess' cookie. For example, accessing the site in anonymous doesn't have this particular problem.

It takes something like 10-20 seconds before the page loads, but sometimes there's no delay. This particular delay seems to point at the authentication code, which is usually heavier because of asymmetrical key algorithms, but it shouldn't be that much of an overhead (and the verification part is usually optimized). Perhaps it needs to create a key for the TLS session, too - Is this part performed by a different server? I doubt it but I'm not a specialist. Are bots registered (impacting the authentication system)?

Other than that, I occasionally have very long access delays for every page and time-outs when the US timezone is active and more people are there. I'm not visiting the forums regularly enough to tell whether this is always the case, or only around particular events like a PfH or a release. So that's probably a balancing issue or simply a capacity issue. I've seen a clear difference between 6:00 UTC and 18:00 UTC, several days. Are bots following a specific hourly pattern (impacting the servers in general)?

EDIT: interestingly enough, it just timed out a few times when I tried to post this.

You must gather your servers before venturing forth

Ugh, here we go again. Just checked back and see the forum is crawling.

Yea, it's been overall better lately, but today is grinding again

Servers just failed their saving throw.

its not just one bot... lots hit the wall and get blocked, other get through to 'index' and some just plant spam because with game about to drop more people are here

even a bot that gets blocked still puts load on the cms and once it hits peak any other traffic can't get in

Bot you didn't see that coming.

Is it weird that a company this big has such a rickety forum platform? It feels weird.

The devs can make an elaborate game but can't keep the forums running smoothly? That's odd, right?

It's a difficult situation, if you close the forum to anyone but registered members then you won't get search traffic if people are looking for answers to specific questions. However it does block the bots.

They are crawling all over our messages digesting all the info they can with their filthy little robot hands.

I guess. I just don't understand, though. This doesn't happen on other forums, and don't bots go around other forums, indexing and doing whatever bots do?

Larian has proficiency in game development, but the attribute related to forum performance management was their dump stat.

The forums are run on third party software UBB Threads, and I think might even be hosted by that third party (though I might have got the wrong end of the stick there). It’s been the platform for ages now and is clearly creaking and performance levels are unacceptable, and what is needed is a complete upgrade probably to a new package that is run on infrastructure able to cope with the modern world. But that’s not a small job, especially if there’s a desire to migrate all the historic or in-progress discussions to the new platform.

I don’t know how likely it is to happen soon, but I’m hoping that once BG3 is out, Larian might have a bit more attention to spare for us suffering forumites!

I really don't see this forum being usable at all after release, the sheer volume of new posters will make is basically inaccessible because of the infuriating slowness.

The problem seems to be a combination of antiquated software and bots rather than user load but certainly the release of the game will not help things.

The forum became extremely rickety due to the onslaught of new users since Original Sin.