0
90

[–] pewpewpewmoon 0 points 90 points (+90|-0) ago  (edited ago)

ok

1) Don't run untested scripts on production servers

2) Don't run code you don't understand

3) Airgap your fucking backups

I hope this guy does go out of business. No one should trust someone this reckless and lackadaisical.

0
41

[–] ForgotMyName 0 points 41 points (+41|-0) ago 

I have never seen something this gloriously stupid... and I've seen people delete entire databases by accident. But at least we had on-site backups that required different permissions that we were able to restore. Worst case scenario, we had tape backups hidden away in a vault off site.

I won't even begin enumerating how many errors are simultaneously required in order to be able to completely erase all your servers and all your backups in a single strike.

Pure gold from the article. Just amazing. You have to work to do something that stupid.

0
19

[–] BentAxel 0 points 19 points (+19|-0) ago 

I am a simple person that used work in tech for many years. I am no way to the level of many of you. On my best day, with planning and assistance, I am not sure I could pull this off as glorious as this person has.

0
3

[–] roznak 0 points 3 points (+3|-0) ago 

Not really, inexperience.

As I experienced last year a new drive failed, then it turned out that my all my backups were corrupted. I ended up buying a NAS, only to discover that Windows 10 was snooping on it and did something wrong that basically crashed the complete NAS. By sheer luck I did have an off-line backup.

People do learn by mistake, this is such a mistake.

0
16

[–] multidan 0 points 16 points (+16|-0) ago 

If you are wondering if your backups are not secure enough, they need to be airgapped more. Stick them in a vault in the Vatican's archives, on a RAID, with multiple copies, some in archives with recovery records.

I feel bad for his customers, they had no idea he was acting this stupidly until it's all gone.

0
7

[–] Donbuster 0 points 7 points (+7|-0) ago 

Yup, I get anal about airgapping my backups. For the business I work for, I have a couple systems.

Live NAS for all our storage needs, holds tax data, customer info, and other records.

Nas that automatically creates incremental backup every 10 minutes of the customer order data, and incremental of everything else every 48 hours. Backups last 3 days.

System that has an independent copy of mission critical financial data. This one is updated weekly, and its drives are in a fireproof bombproof .etc safe, with a hole drilled out and caulked shut for sata and power cables.

A pair of drives in a different safe, one has a longterm backup and the other a short term backup of the data above. These drives are not connected to any system. They are never to leave the safe at the same time. I manually clone them from of the disks from the aforementioned safe system.

And if all that fails, an encrypted version of the data in a safe deposit box.

All this might be overkill for a small business that doesn't work in the tech sector. But it sure as hell should be the minimum for a web host. If all copies of your data are on powered up drives at the same time, at ANY time, you are doing something very, very wrong.

0
1

[–] fluffernuttins 0 points 1 points (+1|-0) ago 

And not just that. People forget that backups are worthless unless they actually exist. At least once a month you need to pull something out and try to restore it. You can do all the backups you want but if you're writing out garbage, you might as well not bother.

0
6

[–] TremorAcePV 0 points 6 points (+6|-0) ago  (edited ago)

Having not read the article, I am hoping it somehow came to DROP TABLES; or "sudo rm -rf /"

Edit after reading article: HAHAHAHAHAHAHAHAHAHAHAHAHAHHAAHAHHAAH

0
9

[–] ForgotMyName 0 points 9 points (+9|-0) ago 

Oh of course it was rm -rf /.

Guy wrote a script, didn't test it, threw it out in production on a system he uses to distribute them across all of his boxes, and it just dutifully burned through and deleted everything on every box. Computers are great like that, they generally do exactly what you tell them to do.

0
2

[–] thunder_fuck 0 points 2 points (+2|-0) ago 

We call him little Bobby Tables.

0
2

[–] Gargilius 0 points 2 points (+2|-0) ago 

lackadaisical

TIL a new word. Thanks.

0
14

[–] Mr_Wolf 0 points 14 points (+14|-0) ago 

That piece of code is so famously destructive that it has become a joke within some computing circles.

something tells me he got coding advice from 4chan

0
14

[–] whisky_cat 0 points 14 points (+14|-0) ago 

Press Ctrl+F4 to move faster while playing.

0
0

[–] PlasmaDistortion 0 points 0 points (+0|-0) ago 

That would be an evil April Fools prank, a macro that randomly presses this key combo.

[–] [deleted] 0 points 8 points (+8|-0) ago 

[Deleted]

[–] [deleted] 1 points 8 points (+9|-1) ago 

[Deleted]

0
2

[–] Laurentius_the_pyro 0 points 2 points (+2|-0) ago 

You can charge your phone battery by putting it in the microwave.

0
1

[–] ginx2666 0 points 1 points (+1|-0) ago 

It's common knowledge. Nowadays most systems are 64 bit, so system32 only wastes space. I deleted mine, and not only my PC boots faster, but it has left me with more space to download RAM.

0
2

[–] Laurentius_the_pyro 0 points 2 points (+2|-0) ago 

I remember one post where a bunch of people managed to convince some underage OP to delete system 32 on his dad's computer then when he asked how to fix it they convinced him to rub magnets on his computer.

0
0

[–] DoomMantia 0 points 0 points (+0|-0) ago 

[–] [deleted] 1 points 12 points (+13|-1) ago 

[Deleted]

0
2

[–] Pantsuit 0 points 2 points (+2|-0) ago 

I wish! Alternatively I would love it to be Hillary's campaign's IT admin. Even though that's not a company :</sup>

0
10

[–] 0x5f3759df 0 points 10 points (+10|-0) ago 

I'm almost certain he is just trolling serverfault, but if he's not, he had some seriously bad backup policies.

0
5

[–] LewsTherinTelamon 0 points 5 points (+5|-0) ago  (edited ago)

I'm almost certain it is trolling.

He said:

Last night I accidentally ran, on all servers, a Bash script with a rm -rf {foo}/{bar} with those variables undefined due to a bug in the code above this line.

This command simply doesn't work. On all of my boxes the command either simply exits without doing anything, or throws a syntax error.

Bash variables do not use curly braces. They use dollar signs. The story is bullshit.

1
2

[–] zwy 1 points 2 points (+3|-1) ago 

Try as root

0
5

[–] Mr_Quagmire 0 points 5 points (+5|-0) ago  (edited ago)

He is. This is actually what would happen on CentOS7:

# rm -rf /
rm: it is dangerous to operate recursively on '/'
rm: use --no-preserve-root to override this failsafe

0
0

[–] ShinyVoater 0 points 0 points (+0|-0) ago 

GNU rm has that feature. Bash might summon the system rm, but he'd have to be running a BSD to maybe have one without that catch - in which case my poor enduser self would wonder why he's using bash instead of another shell for scripting.

0
3

[–] whisky_cat 0 points 3 points (+3|-0) ago 

It's odd to me they would identify exactly 1,535 customers. I'm not sure how Stack's timestamps work in regards to timezones, but they posted at 10:25pm and stated "last night I accidentally ran...".

Seems fishy if you nuked thousands of customers data. Why bother posting to a forum the next night when you're likely dealing with 1,535 people [or more] calling you asking "WHAT THE FUCK".

1
9

[–] roznak 1 points 9 points (+10|-1) ago 

Imagine Microsoft hired this engineer and released his script on the Microsoft cloud.

0
8

[–] 51rH0n3y84d93r 0 points 8 points (+8|-0) ago 

I'm sure they airgap at least some of their backups.

[–] [deleted] 0 points 5 points (+5|-0) ago  (edited ago)

[Deleted]

0
3

[–] thrus 0 points 3 points (+3|-0) ago 

I would say at minimum the rm command should not exist anywhere in the backup scripts no matter what you follow it up with.

0
1

[–] zwy 0 points 1 points (+1|-0) ago  (edited ago)

It wasn't in the backup scripts. Rather the backup drives were mounted so they were deleted too. (If this actually happened that is...)

0
4

[–] Nollog 0 points 4 points (+4|-0) ago 

I got to "rm -rf" and face-palmed. Lmao. Linux 101 is to never ever do that, it's like hammered into you from day 1 with all the jokes and people telling you to do it.

0
0

[–] Avnomke 0 points 0 points (+0|-0) ago 

Seriously, always run rm with -i.

0
3

[–] heili 0 points 3 points (+3|-0) ago 

It's not a piece of code, it's a command to execute a utility that is native to every unix based operating system.

Also the r in -rf is recursion, not 'everything in a directory' because unix doesn't let you rm a non-empty directory, so it needs to go all the way to the lowest level, delete, and then get rid of the directory and keep doing that until it reaches the directory from which it was invoked.

So this guy is an idiot and so is the 'journalist'.

1
0

[–] supernatendo 1 points 0 points (+1|-1) ago 

Anything that uses command line is mis-attributed to "code" these days. The journalist is probably in his/her 20's and never used a computer or software that didn't have a gui.

load more comments ▼ (15 remaining)