Server Error Reports

[gnif]

Not an open invitation to just spam, but if the site is truly down and you can't post on here and you're certain it's not an issue on your end, you can DM HostFission on Twitter (link in signature)

[gnif]

The outage today was due to a disgruntled nutjob abusing AbuseIPDB to report Dave's youtube videos as "Web Spam" and then using this to report Dave's IPs to GorillaServers as being abusive, which caused them to null route our IPs.

I'm sure Greater Minds than mine know what to do, but if the guy is abusing the reporting system then seems he/she should be reported ... 

Indeed, we are already pursuing avenues with regard to this. We know who the person is, and have enough evidence (The abuse report was even PGP signed, lol) to take things further if needed.

[rsjsouza]

If the disgruntled user was from a few years ago, it is quite ingenuous given that normally all defenses are lowered as time goes by.

In any case, congratulations to gnif, Dave and the other hidden actors for maintaining such incredible website running with great availability for the past several years.

[sandalcandal]

Dude, justice existed for millennia without idiocracy and it better stay that way. The last thing we need is disputes between websites and mad Chinese users resolved by some international UN body filled with "experts" from CCP and all the other crazies out there.

Not necessarily defending the actions of the commercial players currently in charge, but really, be careful what you wish for.

I'm not sure the full details are to be made public, so I'm surprised gnif has said as much however this particular case has nothing to do with any Chinese users disgruntled or otherwise. As gnif said, in this case it was a (non-Chinese) nutjob not some CCP conspiracy.

[PlainName]

you can DM HostFission on Twitter

I'll try and remember that, but you're up so reliably there's usually no need to bother. Maybe if you threw a wobbly a bit more often that'd get written down somewhere...

[DrG]

The outage today was due to a disgruntled nutjob abusing AbuseIPDB to report Dave's youtube videos as "Web Spam" and then using this to report Dave's IPs to GorillaServers as being abusive, which caused them to null route our IPs.

I'm sure Greater Minds than mine know what to do, but if the guy is abusing the reporting system then seems he/she should be reported ... 

Indeed, we are already pursuing avenues with regard to this. We know who the person is, and have enough evidence (The abuse report was even PGP signed, lol) to take things further if needed.

It seems bizarre. I looked at those "reports" just now and nothing is clear to me at all. The abuse site says "Confidence of Abuse is 4%". Looks like once or twice a week a report was made. HostFission  just posted (an hour ago) that ..."This information was cited as a reason to have the @eevblog
 server's IPs null routed by the datacenter who obliged."

It seems bizarre because there does not seem to be any "due process" or "due diligence". Granted, I may be missing more than I am getting, but it still seems bizarre. If you can say...is this a technicality, an absurd process with little or no false reporting consequences or something else?

[gnif]

Dave did get a copy of the abuse report which got lost among other emails, there was an opportunity to contest it. The thing is that it shouldn't have even got that far, and I should have also got a copy as I am Dave's Technical contact. At the end of the day, there were mistakes made by all parties involved.

If you have a dig through the reports listed as "Web Spam" by that user you will find he is attacking multiple youtubers, including Fran. He is an utter scumbag.

[bd139]

Yes definitely shouldn’t have got that far.

Well so far they tried to burn the hardware, then wet it then shoot it after a malicious report.

They don’t deserve to have customers.

This is one thing I love about AWS. They call us if there’s an issue and talk to someone before doing anything.

[wilfred]

Dave did get a copy of the abuse report which got lost among other emails, there was an opportunity to contest it. The thing is that it shouldn't have even got that far, and I should have also got a copy as I am Dave's Technical contact. At the end of the day, there were mistakes made by all parties involved.

If you have a dig through the reports listed as "Web Spam" by that user you will find he is attacking multiple youtubers, including Fran. He is an utter scumbag.

I tried to look at what you're referring to. It looked to me like that person had made over 26,000 reports (I could be wrong) which is a lot of work. It seemed like it might be automated.

What is "Web Spam"? Most of the reports listed "email spam".

[gnif]

It is very likely automated to an extent. The user clearly has a grudge though and is also making manual submissions for youtubers. Normally "Web Spam" would be a bot that was caught posting spam to a website.

[DrG]

Enter eevblog.com and I see 28 reports (all from the same source). All are categorized as "web spam" except for the last six which are listed in the email spam category. I did not look at every one, but the ones that I did see (expand the comment section and it ends with a link that I assume is the object of the report) link to YouTube videos and just normal eevblog videos...nothing looks like my definition of spam. Even the email spam category entries that I looked at, link to youtube vids.

Additionally, while from other threads, I know that the YouTube "regs" are "artistic" to be kind, I am not sure what is being reported there that should not be reported to YouTube, if there is anything to report - which again, is not at all clear to me...or to put it simply..."what??? why is that spam???".

[gnif]

Exactly the issue, the user is abusing the submission API to report abuse based on YouTube videos. If there is a problem with YouTube video it must be reported to YouTube, not a third party IP that has nothing to do with it (from a hosting perspective)

[bitseeker]

Ugh, I feel your pain, gnif & Dave. Try to do good in the world and the nut jobs come out in force. What nonsense.

Thanks for your continued perseverance.

[StillTrying]

Some say the site was unavailable for 3 or 4 days, it seemed more like 24 hours to me.

The May stats don't show much of a gap.
https://www.eevblog.com/forum/stats/

[tautech]

Some say the site was unavailable for 3 or 4 days, it seemed more like 24 hours to me.

The May stats don't show much of a gap.
https://www.eevblog.com/forum/stats/

~16 actually if we can go by the most active thread on the forum:
https://www.eevblog.com/forum/testgear/test-equipment-anonymous-(tea)-group-therapy-thread/msg3565691/#msg3565691

[bd139]

Best way of working this out is a tool called AWS CloudWatch Synthetics. You can set it up to do a typical user journey every 10 minutes ie log in, open a thread etc. If anything doesn’t work it’ll keep availability stats and send alert emails etc. $5 a month for a 10 minute cycle approx.

Here’s what poked me yesterday

[gnif]

We keep detailed metrics of the servers, here is the logged outage period.

[bd139]

Nice. That'll give you a machine level outage / traffic outage. The synthetics will tell you the forum is throwing 500 errors as well i.e. what does it look like from the end user's perspective.

[bitseeker]

First time I've gotten this error:

Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.

Occurred about a minute ago. The post I submitted was still saved, though.

[tautech]

First time I've gotten this error:

Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.

Occurred about a minute ago. The post I submitted was still saved, though.

Sporadically had this too since the server was last down. Posting or viewing.

[sandalcandal]

First time I've gotten this error:

Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.

Occurred about a minute ago. The post I submitted was still saved, though.

Sporadically had this too since the server was last down. Posting or viewing.

+1 also having this issue sporadically

[RoGeorge]

First time I've gotten this error:

Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.

Occurred about a minute ago. The post I submitted was still saved, though.

Sporadically had this too since the server was last down. Posting or viewing.

+1 also having this issue sporadically

Just seen this error, too.

[YetAnotherTechie]

Have you seen: https://www.eevblog.com/forum/news/news-forum-reverted-to-backup/125/ ?
I still don't get anything from eevblog on my hotmail account.

[EEVblog]

First time I've gotten this error:

Connection Problems
Sorry, SMF was unable to connect to the database. This may be caused by the server being busy. Please try again later.

Occurred about a minute ago. The post I submitted was still saved, though.

Sporadically had this too since the server was last down. Posting or viewing.

I believe we are still running on a single server box since the fire.

[gnif]

Just an update on the above..

We are fully operational on two servers again and have been for a little bit now.