NEWS: Forum reverted to backup

[beanflying]

There's an argument that if a poster can't tell you their post is missing, then the post was of little consequence.

That's certainly valid for my posts!

In this case if all of Blueskulls 300+ topics for a start have been or are still removed then having other members lose several hundred posts is likely. Forums are about 'content' and the sum of that content is the reason for their ultimate existence. This SUM is of consequence!

[wilfred]

Bulk removing 10k+posts was always a dangerous thing and if that now I gather ex member wanted it done then it should have fallen on them to filter the content they wanted removed or it should have been left in place so the threads in time still read 'sensibly'.

I can't find it now, but I thought I read a post where Dave explained that a member requested Dave's help to remove some posts. What is wrong with that? It is not like they did anything malicious. They did nothing wrong.

I don't know why they wanted it, I don't care. If they ask and Dave is OK with it then as far as I am concerned they have done all they needed to do.

Dave sure as hell isn't trying to shift the blame onto them. Not from anything I've seen. As far as I can see Dave has been completely upfront in accepting responsibility.

I can't help but wonder if it is not possible to journal posts between backups so that no matter how far you must revert the database to get a good  copy for restore, you will be able to roll forward from the journal entries timestamped after the backup. If that was available this would be a much simpler problem to unravel.

I would like to see the changes made to the live forum verified on a clone copy. Whether it is database maintenance, or software upgrades. Then with the confidence of a verified procedure apply them to the alternate server and switch the databases with subsequent updates made to the live server applied from the journals. I'm sure database admins will express it better. Perhaps on these systems it isn't possible. IDK.

Another thing I saw was a comment Dave made about the time lag between a database corruption and the error being reported or noticed. This would cause posts made in the interval vulnerable to loss when the latest backup was applied. Aside from speculating on whether posts can be journaled, can a script be added to the backups to test (and fix) database errors? or at least notify an admin hopefully before users notice in any numbers.

[EEVblog]

I would like to see the changes made to the live forum verified on a clone copy. Whether it is database maintenance, or software upgrades. Then with the confidence of a verified procedure apply them to the alternate server and switch the databases with subsequent updates made to the live server applied from the journals. I'm sure database admins will express it better. Perhaps on these systems it isn't possible. IDK.

Unfortunately in certain circumstances like we've seen her, the only way to "verify" that something worked ok is to wait and crowd source complaints.

Another thing I saw was a comment Dave made about the time lag between a database corruption and the error being reported or noticed. This would cause posts made in the interval vulnerable to loss when the latest backup was applied. Aside from speculating on whether posts can be journaled, can a script be added to the backups to test (and fix) database errors? or at least notify an admin hopefully before users notice in any numbers.

There are just so many variations on what can happen with a database of this size and magntiude, it's likely impossible to cover everything.#Like like I have imagined that deleting indivisual user posts under a specifc circumstance have caused entire threads to be deleted? Not in a million years, or the 15 years I've been using SMF anyway.

[Black Phoenix]

Deleting something from a database in a working live platform is always a problematic endeavour as, if a mistake is made a lot more is lost and tracking back all that happened to recover what was removed by mistake is a PITA and very time consuming.

I've done similar stuff in telecommunications were I just YOLO a config into a NSN MGW (Media Gateway, kinda like a router but for telecommunications that is the size of 3 40U racks) and put half of Portugal out of communications. 10 min for 5 hours of recovery. It was my fault because I should had done a snapshot of the equipment (less than 1 hour) but since it was a small group of commands I thought it was safe. Low and behold, one of the delete commands spectrum was higher that was suppose to be and KAPUT everything gone. Had to recover from the day before snapshot and then get the rest of the changes made that day on the machine via Command Log History.

What Dave done was risky, but he had done the right decisions in terms of recovery he made. Yes, a day of posting was lost. Probably as soon as someone detected the missing posts Dave should had blocked the forum to read only and then made a snapshot of all new posts from the last backup until the moment. Then restore the backup and merge the changes present in the snapshot (don't know how hard it is or even if it is possible with the database and forum software being run).

It wasn't done. Posts were lost, discussions and lines of thought were lost since the last backup. But still Dave, Gnif and the rest of the team done the right choices for the problem in hand. I can't fault them for that. And I will not point the fingers on them because I perfectly know what is to screw up and clean the pile of sh#t you made.

What it makes me perplexing is why causing all this because of removing a user from existence on this forum, per request of said user. Yes he have the right to have his posts deleted. But did he?

I don't know the rules in Australia and if they have something like the Right To Be Forgotten or GDPR. If it is the rules of the country from the community owner, then I don't know the regulations in Australia. If it is the rules of the citizen's country then that's a different beast - In China that right doesn't exist: https://en.wikipedia.org/wiki/Right_to_be_forgotten#China

But the main problem now is that we have a forum now that lost potentially lots of knowledge from that user in posts created by him and lots of quotes from his posts in other threads with line of tough lost in topics that the users didn't quote him for his comment.  Yes he was very extremist in his views regarding his country, call him whatever you want. But from his total amount of posts - 14803 at the time he was banned - I'm sure that only 10% of them was really problematic (and from that 10% a lot were dealt with at the time they were reported for sure). OK let's say 20 to 25% were problematic. It is still 75% of posts with info and knowledge that left the forum.

Again I will say, I'm not against how the forum team reacted in all this situation. On the opposite, if we weight the pros and cons of what was done, it was the right choice for the problem in question. What it saddens me is the info that was lost. Just that.

But again I reiterate - Comparing the problem (the mass deleted posts by mistake), the possible solutions and consequences of the decisions, the forum team done exactly the right choices for the problem in hand with the minimal loss.

It is easy being on this side and judge. But the other side is the one who knows what was the size of the turd that hit the fan.

[magic]

So all of that for blueskull?
Surprising that Dave gave in, but whatever.

One thing that could have been done differently (if it hasn't) was to perhaps make a backup of the state before reversal. On the off chance that something of value had been posted worth recovering manually from the DB.

[thm_w]

But the main problem now is that we have a forum now that lost potentially lots of knowledge from that user in posts created by him and lots of quotes from his posts in other threads with line of tough lost in topics that the users didn't quote him for his comment.  Yes he was very extremist in his views regarding his country, call him whatever you want. But from his total amount of posts - 14803 at the time he was banned - I'm sure that only 10% of them was really problematic (and from that 10% a lot were dealt with at the time they were reported for sure). OK let's say 20 to 25% were problematic. It is still 75% of posts with info and knowledge that left the forum.

Yes this is sad.

Bulk removing 10k+posts was always a dangerous thing and if that now I gather ex member wanted it done then it should have fallen on them to filter the content they wanted removed or it should have been left in place so the threads in time still read 'sensibly'.

Would have been nice if any post mentioning location/personal info etc were deleted and that was it. But it seems like more of a "ok im angry, I'm taking all my stuff back" deal. So to get back at the few idiots provoking him, a wealth of knowledge is lost for thousands of other readers. Poor form on both sides.

How long until we can train an AI to automatically screen posts for political opinions and throw up an error message.

[james_s]

Well that explains what happened at least. There were several threads I could have sworn I had replied to already but my reply wasn't there. I couldn't think of any reason a mod would have deleted what I replied with deliberately.

[rsjsouza]

So all of that for blueskull?
Surprising that Dave gave in, but whatever.

I had no idea that was the reason. Not knowing anything else, this is not a great move (deleting ALL posts). Way too many interactions that will crap the flow of conversations.

OTOH, his technical contributions were good but his rhetoric was extremely toxic. He was given many chances to cool off and, for a 28 year-old guy, I think he should have grown up by then.

[tautech]

So all of that for blueskull?
Surprising that Dave gave in, but whatever.

I had no idea that was the reason. Not knowing anything else, this is not a great move (deleting ALL posts). Way too many interactions that will crap the flow of conversations.

OTOH, his technical contributions were good but his rhetoric was extremely toxic.

Only as toxic as all the China hate on this forum and worldwide.

Blueskull and his parents sacrificed and worked hard for him to gain a doctorate in Power electronics and despite my wishes for him to display such qualifications in his profile he wouldn't flaunt what he had achieved yet the quality of his contributions here over the years grew with his experience.
I for one will miss him here although we do communicate privately so at least I still have that.

RIP Blueskull on EEVblog. 

[JohnnyMalaria]

He sure got thanked a lot...

[gnavigator1007]

I'm mostly a lurker, but I appreciated a great many of his posts. It's a substantial loss for the forum. I was just looking at some of the forum stats and in addition to being one of the most thanked members, he was also one of the top topic starters. Interestingly, 367 topic posts are still viewable via his profile as well as attachments. 

Just noticed looking at the stats again that total page views have passed 1B! Congrats Dave

[james_s]

OTOH, his technical contributions were good but his rhetoric was extremely toxic. He was given many chances to cool off and, for a 28 year-old guy, I think he should have grown up by then.

Frankly I had hardly noticed. Occasionally he would go off on some rant and I would just scroll past it, didn't really seem like THAT big of a deal. There are a few forum members I'm not going to name who have such toxic personalities that I put them on my ignore list a long time ago as they almost never say anything that is not rude and unhelpful but he is not one of them. I would take blueskull anti-western rants all day long over the toxic identity politics and other nonsense that pollute every other corner of the web.

[lowimpedance]

So all of that for blueskull?
Surprising that Dave gave in, but whatever.

I had no idea that was the reason. Not knowing anything else, this is not a great move (deleting ALL posts). Way too many interactions that will crap the flow of conversations.

OTOH, his technical contributions were good but his rhetoric was extremely toxic. He was given many chances to cool off and, for a 28 year-old guy, I think he should have grown up by then.

His strident views certainly shifted after he left the US, however removing all your posts is immature and he's not the first !. It is more sad that the knowledge passed on has been erased and even more so that this exercise has caused collateral damage that cannot be repaired.

[floobydust]

I found his fanaticism refreshing and an insight into that which is at war with democracy.
There are several papers written by Party defectors that give further insight. It's not looking good for us, nor is it all unicorns and rainbows in the Party as his future employer.

I have an issue with allowing an extremist who threatened to report forum members to the CC P to be here. The usual nutbar behaviour gets banned, but not this?
Where's the Forum's rules?

[bitseeker]

I'm pretty sure I have now undeleted almost every thread that was accidently deleted.
I may have missed a couple, and if you are aware of any please let me know.
Everyone's post counts should be back to almost normal, although there will almost certainly be a few casulty posts, sorry about that.
The recovery from my goof was not 100%, but should be pretty darn close.
Thanks to gnif for a late night backup recovery and writing some script to fix some things.
Sorry to everyone for the goof up, it was entirely my fault 

Thanks, Dave and gnif for getting everything mostly back to normal. Post count? I don't know exactly what my post count was. Close enough.

[mnementh]

Wait... what...? This all happened cuz some dingle decided to ragequit; take his ball and go home...? 



I just figured gnif finally had enuf of us all whining aboot the slow pageloads and went a little Widlar on eevBlog's arse... 

mnem
oooh... smoky...

[rsjsouza]

OTOH, his technical contributions were good but his rhetoric was extremely toxic.

Only as toxic as all the China hate on this forum and worldwide.

As I have said in one of many occasions he was the subject of a banning/pardon decision: one can only control oneself's actions and he chose to respond to the provocations. Other members from the same country do a much better job at keeping their emotions at bay.

Blueskull and his parents sacrificed and worked hard...

Yes, just like many others here in this forum and worldwide, but that does not invalidate or justify his actions.

Reiterating: I appreciate a good discussion and the occasional insight about the Chinese culture that he provided, but it was way too often he was involved in imbroglios regarding nationality, political and economical POVs. Other members were banned by much less.

I feel sadened by two things: no more access to his valuable insights and the deletion of the majority of his contributions. May he have success in his future enterprises.

Anyways, my opinion is voiced. I now choose to ignore this thread. Bye y'all.

[Black Phoenix]

OTOH, his technical contributions were good but his rhetoric was extremely toxic.

Only as toxic as all the China hate on this forum and worldwide.

As I have said in one of many occasions he was the subject of a banning/pardon decision: one can only control oneself's actions and he chose to respond to the provocations. Other members from the same country do a much better job at keeping their emotions at bay.

Blueskull and his parents sacrificed and worked hard...

Yes, just like many others here in this forum and worldwide, but that does not invalidate or justify his actions.

Reiterating: I appreciate a good discussion and the occasional insight about the Chinese culture that he provided, but it was way too often he was involved in imbroglios regarding nationality, political and economical POVs. Other members were banned by much less.

I feel sadened by two things: no more access to his valuable insights and the deletion of the majority of his contributions. May he have success in his future enterprises.

Anyways, my opinion is voiced. I now choose to ignore this thread. Bye y'all.

Well let's not idolise the guy and godify him Ok? Rsjsouza is totally right. And as I said his contributions were good and is sad that they were lost because he wanted them removed.

But the toxic behaviour deserved what it arrived at his bay. Specially the last post that got him banned who make accusations to Dave and other members of this forum/companies. There is a say in the web that "Don't feed the troll". He had done that, lots of times.

If people talk bad about my country and whatever do you really think I will care about it? Why me justifying my country choices will change the idea of the person in question or even make any meaningful change in the overall big scope of things? It is a drop in the ocean. It doesn't change a thing, it only makes you feel better, nothing more.

And as a foreigner living in Shenzhen/Hong Kong and having a Chinese family from the side of my wife, both in Mainland and Hong Kong, I have a good view of both sides and different opinions.

[EEVblog]

And as I said his contributions were good and is sad that they were lost because he wanted them removed.

Correct. He wanted every single post removed and would have done it himself if he was given access back.
Everyone has the ability to delete all their posts on this forum, it's just that admins have tools to do it on a mass scale with greater ease.

[Ed.Kloonk]

it's just that admins have tools to do it on a mass scale with greater ease.

!

[Black Phoenix]

Wait... what...? This all happened cuz some dingle decided to ragequit; take his ball and go home...? 



I just figured gnif finally had enuf of us all whining aboot the slow pageloads and went a little Widlar on eevBlog's arse... 

mnem
oooh... smoky...

Lovely mnementh. I was just thinking why did you take so much time to grace us with your well chosen gifs! 5 star, specially for the ones who understand what the command in question do.

To be sincere I've done that plank to a new colleague with the exact same command (but before I made full disk backup of the system on a extra HDD for easy swap and low downtime plus a digital copy on the PC just in case) to show him the main rule from Sysadmins: Don't run commands without fully knowing what kind of implication such commands cause!

[magic]

It's for weeding out Ubuntu users

Code: [Select]

$ sudo rm -rf /*
-bash: sudo: command not found

To be sincere I've done that plank to a new colleague

Not the smartest idea. There was time when it permanently bricked PCs thanks to the genius of UEFI and awesomeness of Linux. Since then I'm somewhat weary of running such things on /sys.

If it wasn't for the technical stuff up/bug, it would have been trivial work and no one would have noticed.

Haha, knowing now what really happened I have some doubts

[mnementh]

Oh, I'll be the first to admit my Kung-Fu is weak; I know just enough to be really dangerous. Versions of that meme were popular aboot the second time I went back to school, and I figured that while it bordered on bad taste, we could maybe use a little frivolous diversion.

Yes, I do know exactly what that command is supposed to do. I assume hope that in the many years since I was in school and had to do that kind of stuff daily, enough people got bitten by it that some err... protections have been put in place, at least in the more common distros...?

mnem
Being the class clown is hard work; but I have a lot of Karma to burn off...   

[bd139]

Gah so much undue vitriol here.

Thanks to Dave and Gnif for managing to get through it. I think we all owe you both some space and a drink for actually giving enough of a crap about the community to respect a former user's request to have their posts removed 

[drussell]

Ah, here it is! 

I knew there should be a thread about the outage somewhere, so this discussion should probably be in here, instead of ataradov's e-mail notification thread:

Something must still be amiss somewhere...

My post count says 1767 but if I view "Show Posts" in my profile, I can go back to page 73 and it shows post #1824...

It is 7455 vs 7406 for me.

Well, they're obviously still doing something behind the scenes on this...

My post count jumped from 1768 yesterday to 1783 today all by itself, and there are now 1841 in my "Show Posts" section in the profile.

They must still be un-munging the database and presumably will have to do some magic SQL incantation to re-populate people's post count field with the actual true value once the repairs are complete...

I'm pretty sure I was right around 1900 posts, as I was closing in on a 1 post-per-day average on the profile page, but I'm currently shown to be back at 0.864 per day.