A minute of electricity black-out that made into news headlines ...

[BravoV]

-> Samsung Electronics chip output at South Korea plant partly halted due to short blackout

Excluding labor cost, curious how much the raw materials cost that will be disposed. 

Lots of cursing for sure. 

[Black Phoenix]

And again the prices of memories will go up...

2018 again???



Big coincidence isn't it (Corsair is known to use Samsung in most of his products).

[NiHaoMike]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

[Cyberdragon]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

There are industrial UPS systems that go up into the megawatts range. A minute is well within the range of operation of the biggest systems. I can't believe Samsung, a tech giant, in Japan, the tech center of the world, didn't think of such things. Let's hope they learned their lesson.

[coppice]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

If the public supply is highly reliable, and you have redundant wiring into the premises to reduce the risk from backhoe loaders, its questionable whether a UPS improves or degrades power reliability. No UPS is without its own problems.

[Cyberdragon]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

If the public supply is highly reliable, and you have redundant wiring into the premises to reduce the risk from backhoe loaders, its questionable whether a UPS improves or degrades power reliability. No UPS is without its own problems.

But they don't necessarily have to go with the fancy flashy ones that have all the power conditioning and whatnot, just a battery backup. Companies of large systems usually have maintenace and checkup programs as well.

[coppice]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

If the public supply is highly reliable, and you have redundant wiring into the premises to reduce the risk from backhoe loaders, its questionable whether a UPS improves or degrades power reliability. No UPS is without its own problems.

But they don't necessarily have to go with the fancy flashy ones that have all the power conditioning and whatnot, just a battery backup. Companies of large systems usually have maintenace and checkup programs as well.

How do you build battery backup that cannot ever fail?

[Gyro]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

There are industrial UPS systems that go up into the megawatts range. A minute is well within the range of operation of the biggest systems. I can't believe Samsung, a tech giant, in Japan, the tech center of the world, didn't think of such things. Let's hope they learned their lesson.

Errr, Samsung is South Korea 

I suspect that it is far too easy to just sit back and criticise a huge smallest geometry fab facility like that for tripping out under power failure conditions without understanding the complexities (or the power consumption) involved.

The lines will be chock full of fail-safes to protect the equipment of the line - rather than the product (which is disposable in this context). Think high vacuum systems, ultra-pure processing stages, hazardous chemicals, the risk of irreversible internal contamination or time consuming equipment cleanup risks etc. It doesn't take much to slash the yield of a fab. It must be quite an achievement to keep parts of a system like that from tripping out every time a mouse farts (not that a mouse is going to make it anywhere close to anything critical).

The story does say "inspecting the production lines for a restart" which means, I suspect, that the protection systems actually worked rather than failed.


P.S. I wonder how you tell, at the start of a 1 minute power outage, that it is going to be just 1 minute long? 

[BravoV]

What interesting is in the report ...

"A half-hour power outage at Samsung’s Pyeongtaek chip plant in 2018 resulted in estimated losses of about 50 billion won ($43.32 million) according to Yonhap. "

They've experienced this, and yet, it happened again, I guess the protection systems needed is much-much more expensive investment, compared to just throwing out <$50 million of dollars ? 

[Gyro]

Yes, I suspect that <$50M doesn't buy you too much additional protection in that context. You still have to plan on the line going down unless you know exactly when power will be restored. It's probably already full of stuff to make the shutdown as graceful and least damaging as possible. That amount of money implies that they didn't actually break anything significant.

[james_s]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

If the public supply is highly reliable, and you have redundant wiring into the premises to reduce the risk from backhoe loaders, its questionable whether a UPS improves or degrades power reliability. No UPS is without its own problems.

At a place I used to work we had outages on two different occasions that were caused by the huge UPS failing. One of those incidents filled the whole building with smoke.

I got rid of the UPS's I had at home after I came home one day and smelled something, then found one of mine with the plastic top panel melting. Stored energy is dangerous and between that and the quiescent draw added by a UPS I decided it wasn't worth it for how rarely my power goes out.

[Gyro]

Back in the '70s (miners and power workers strikes, 4 day week) all the sites in the company I worked for had diesel generators and fuel stores installed. They were still in place in the early '80s when our site started having 'issues' with the mains supply (cable capacity too small and phase load balancing poor) and started popping supply fuses with annoying regularity.

Each time the generator would fire up automatically just fine. The only weakness with the scheme was that it relied on a guy from security going round to open the generator room doors within a couple of minutes to provide cooling (there were small louvres but insufficient). Of course one day there was a new guy on duty. By the time anyone noticed, there were vast clouds of steam and coolant spewing everywhere!

There is always a weakness somewhere.

[Cyberdragon]

There is always a weakness somewhere.

Yes, as you've indentified...humans.

Soon everything will be automated with just a very small skeleton crew that sit around drinking coffee unless an alarm goes off. Also, redundancy is key, just look at how many backups a nuclear plant has.

[Gyro]

There is always a weakness somewhere.

Yes, as you've indentified...humans.

Only in that particular (and trivial) case!

Soon everything will be automated with just a very small skeleton crew that sit around drinking coffee unless an alarm goes off. Also, redundancy is key, just look at how many backups a nuclear plant has.

And yet, they still fail catastrophically (taking Fukushima as the most recent case).

[coppice]

Yes, as you've indentified...humans.

Only in that particular (and trivial) case!

There's usually some human incompetence somewhere when uninterruptible power proves highly interruptible. Redundant wiring that proves to be far less than redundant is a common one. It no use having an uninterruptible power source when minor damage to cables can separate all the sources from the load. Backhoe. Backhoe. Its off the work we go..........

[james_s]

Back in the '70s (miners and power workers strikes, 4 day week) all the sites in the company I worked for had diesel generators and fuel stores installed. They were still in place in the early '80s when our site started having 'issues' with the mains supply (cable capacity too small and phase load balancing poor) and started popping supply fuses with annoying regularity.

Each time the generator would fire up automatically just fine. The only weakness with the scheme was that it relied on a guy from security going round to open the generator room doors within a couple of minutes to provide cooling (there were small louvres but insufficient). Of course one day there was a new guy on duty. By the time anyone noticed, there were vast clouds of steam and coolant spewing everywhere!

There is always a weakness somewhere.

How on earth did that not get noticed during the design phase? It's not that hard to calculate the heat output and determine the amount of airflow needed. Certainly they could have retrofitted fans powered by the generators to exhaust hot air from the room.

[Gyro]

It was a typical CEO 'I'm not going to be held to ransom by striking miners - do it now!' no-brain reflex. They were never properly designed in, just shoehorned into wherever they would fit on each site. At least the exhausts vented outside! After a few years, people pretty much ignored forgot about them, I doubt they even got serviced.

[David Hess]

I wonder how many accountants rejected the plan to include UPSes as "too expensive"?

If the public supply is highly reliable, and you have redundant wiring into the premises to reduce the risk from backhoe loaders, its questionable whether a UPS improves or degrades power reliability. No UPS is without its own problems.

But they don't necessarily have to go with the fancy flashy ones that have all the power conditioning and whatnot, just a battery backup. Companies of large systems usually have maintenace and checkup programs as well.

How do you build battery backup that cannot ever fail?

I think the most common problem is with the transfer switch failing catastrophically.

To be reliable, the UPS needs to be regularly tested and that by itself tend to cause failure unless the designers, builders, and installers were all unusually competent.

[Red Squirrel]

It boggles my mind how lot of critical applications that arn't life critical never bother with generator/UPSes.  Even something like a restaurant can be completely shut down by a 15 minute outage.  Get backup power, it will pay for itself the first time it's ever needed.

In the case of a manufacturing plant, a fly wheel system with stand by generator would probably be the most cost effective, though a UPS with batteries running as a dual conversion setup is the most fail safe.  It's what the telcos do and why land lines tend to be so reliable.

[coppice]

Even something like a restaurant can be completely shut down by a 15 minute outage.  Get backup power, it will pay for itself the first time it's ever needed.

Lots of people haven't seen a single power interruption in a couple of decades. In places like this only life or death systems justify having backup power systems.

[james_s]

Not to mention having a backup system is an ongoing expense. Generators and battery backups both need regular maintenance and for large systems this is not cheap. Then as has already been mentioned the transfer switch adds another potential point of failure. Power outages downtown in many larger cities are virtually unheard of.

[Red Squirrel]

If it's setup right and is of good quality it should not require much upkeep.  In the case of a generator you should run it once in a while and test and exercise it, but other than fuel to run it for a few hours a month it's not that big a deal.  A battery based system is even less maintenance, just need to add water to them once in a while.  Something that is online is running all the time too so if there is a problem you can deal with it before it becomes a big one.  There are no transfer switches either to worry about in such a system.   Even in places with reliable power, they sometimes need to do maintenance on the lines, or a car can hit a hydro pole etc... things happen.

I eventually want to convert my home system to online, right now it's standby, so there is still a chance of the transfer relay failing. 

[james_s]

Don't underestimate the cost of upkeep on a large scale backup system like you'd need for a commercial building. It costs tens of thousands to install in the first place, consumes a significant amount of valuable space for which you pay rent/taxes, in Seattle for example commercial rent is over $41/sq-ft/yr so if we conservatively say an 8'x10' room for the backup generator or UPS that's over $3k/yr just to rent the space. Then you've got to either have a maintenance guy who is qualified to work on it or hire somebody out on a regular basis. Generators need oil changes, cooling system service, and other scheduled maintenance, repairs and regular test runs. UPS systems need battery maintenance and checks, occasionally need new batteries which cost many thousands of dollars. You've also got the additional hazards of stored energy on the premises, either fuel or electricity stored in batteries. Unless you deal with frequent outages, have life critical equipment or valuable inventory that will be ruined by a brief outage, it's more economical to just shut down when power is lost.

We had a large UPS at a former job and it was expensive to maintain and caused at least two outages that I recall. At some point it was deemed inadequate for the load it was supporting and they spent another $50K or so replacing it with a larger unit. I'm not sure why they had it at all, I only remember there being one or two power outages in the more than 10 years I worked there. A UPS only provides power for 10-20 minutes in most cases anyway so it's only there to let you shut down equipment properly, you don't just keep going and hope the power comes back in time.

[Red Squirrel]

Oh yeah if you're renting space I can see it being an issue.  In that case there's way more red tape as to if you can even have such setup in first place since you don't own the building.

[james_s]

The vast majority of businesses rent the space they are in, at least in larger cities in the USA. Commercial buildings are enormously expensive. Even if you own the space you've still paid for it and you pay taxes on it. Space that is consumed by equipment and infrastructure is space that you can't use for other purposes.