Server Error Reports

[PlainName]

You probably already know this, but just in case.... I am getting database errors:

Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.

In Buy&Sell and General Technical Chat. Obviously, it cleared up to post this.

[gnif]

Thanks, I will have a look.
I do have an alert for this instance but as it was fleeting I am unsure as to the cause at this time.

Edit: There was an issue with the database load balancer that has now been resolved.

[gnif]

Seems we had an outage during the night, one of the database servers had crashed out in a strange way. Cause is still yet unknown but being investigated.

[PlainName]

I came across that as it was going wrong, if it might help figure it out. I was trying to post and got:

The following error or errors occurred while posting this message:
Your session timed out while posting. Please try to re-submit your message.

Happened in the crowd funding section, and then when I tried to post here (of course). A the time I could read messages, just not post.

[gnif]

Yeah, thanks, I saw this also

Edit: Please be aware that I am still investigating this, we are seeing the database service get into a unrecoverable deadlock state and the cause is still unknown.

[gnif]

I have made some infrastructure changes that should not only resolve the crashing that has been occurring over the last few days, but also improve overall performance of the website quite considerably.
If you are noticing any improvement/degredation please provide feedback here:

https://www.eevblog.com/forum/chat/server-upgrade-feedback/

[tautech]

This after selecting to view new replies is a PITA:
Please try again. If you come back to this error screen, report the error to an administrator.

[gnif]

Already reported in the other thread, investigating

Edit: This is fixed

[m k]

I've recently had few hiccups.
First one was few minutes where page was finally loaded but content was database error.
Possibly just a database restart.

Now login and post were acting up, maybe timed out somehow but without updated Firefox window.
Like where post reply page is still present but new post is also live.
Maybe database machine has some issues.

I'm also quite sure I saw gnif's edit of a post about Steve Bannon and law where sub forum of deleted messages was mentioned.
Search can't find it anymore.

[gnif]

I've recently had few hiccups.
First one was few minutes where page was finally loaded but content was database error.
Possibly just a database restart.

We do not do database restarts, it was likely a peak load period, we get them from time to time.

Now login and post were acting up, maybe timed out somehow but without updated Firefox window.
Like where post reply page is still present but new post is also live.
Maybe database machine has some issues.

This is just latency in the server responding to you after posting, the new data was clearly inserted into the database and some kind of latency caused you to hang on the post page while waiting. There are no DB issues.

I'm also quite sure I saw gnif's edit of a post about Steve Bannon and law where sub forum of deleted messages was mentioned.
Search can't find it anymore.

When someone replies to a topic we can't just delete it, it must be manually moved to a "Deleted" area of the forums. We do not actually delete posts fully so that we have a record of why people were banned, etc. SMF by default has set to post a redirect link when you move a thread, I accidentally left this enabled. This area of the forum is not available to the public.

[RoGeorge]

Can log and browse but can not post from Ro, when using usual browsers (e.g. firefox/ubuntu).  This post was made through a VPN.  The error is session timeout, please try to logoff/login again, but whatever I do it does not fix the err.

Any advice how to fix this, please?

[gnif]

Right in the middle of some minor changes, that might be the cause. Please hold.

[gnif]

Work completed for tonight, still a few loose ends to sort out, but in no worse state for it

[gnif]

Just an update, one of the packages upgraded seems to be causing an issue with the database backup procedure which runs each night at 22:00 UTC. The database is being locked during the backup which should not occur, which is causing a brief outage while the backup happens.

I have implemented what I hope is a fix to this issue and running another backup to see if it's resolved.

[RoGeorge]

For me still not working.  Managed to post once from Ro, then timeout err again.  This one is from VPN.  I wonder if the cookies were tied to the IP, because I could post from Ro, then I couldn't edit it one minute later (with Modify button), then from there only session expired try to resubmit err.  Checking if it works from new IP only, and in the first minutes only.

Later edit
Worked from VPN (which implies new IP for me

Later later edit
Will stay here a little longer to test if I can only post in the very first minutes after a login from a new IP

[gnif]

What IP are you coming from when it times out (PM me)

[RoGeorge]

Will check now and PM you in a minute, from this login

[magic]

I noticed that my /unread/ list gets cleared randomly since maybe yesterday. I think it first happened after some outage (database connection error, bad gateway), but now it happened again without any outage that I noticed.

The attachment substitution bug is still present, like here.

[gnif]

I noticed that my /unread/ list gets cleared randomly since maybe yesterday. I think it first happened after some outage (database connection error, bad gateway), but now it happened again without any outage that I noticed.

This is not related to any server issues, perhaps a SMF bug, but there is nothing in the server deployment that can explain this.

The attachment substitution bug is still present, like here.

I am still tracing this, it's rare to catch it in action among all the noise of other posts going on so the cause is still under investigation.

[RoGeorge]

Trying now from Ro, with Brave browser.  Will try to attach a printscreen.

The following error or errors occurred while posting this message:
Your session timed out while posting. Please try to re-submit your message.

later edit
Strange that I can post now (once) from a new browser (Brave) I've never logged into eevblog before, but couldn't from LibreWolf or Firefox which I use usually, all same Ro IP.

[gnif]

Thanks, this is not a HTTP timeout as I was looking for, but rather a session tracking issue. Let me have a dig through the SMF source and see if I can spot anything obvious that we might be doing to trigger this.

[gnif]

I have made a change to the code to log additional information, can you please re-try the PM post. I need it to generate the error so I can see what is going on, feel free to spam me until it occurs again.

[RoGeorge]

I've just tried a post from LibreWolf, and didn't work.
Then alt+tab to this brand new Brave and it worked.  This was posted with Brave.

Since nobody else sees the same error, maybe it's something on my side.
Will try in the next days from other live OS's, more browsers and more addresses, and let you know the results.

Thank you for the help. 

[gnif]

Sorry but it needs to be a PM as this is the code I have patched

[RoGeorge]

I have made a change to the code to log additional information, can you please re-try the PM post. I need it to generate the error so I can see what is going on, feel free to spam me until it occurs again.

Sent you this

PM sent to regenerate the error, as you requested

 from LibreWolf (didn't work) same " The following error or errors occurred while posting this message:

    Your session timed out while posting. Please try to re-submit your message.

"
I hope you can see it in the logs.  For now this Brave login seems 100% reliable.