Author Topic: Write once, read forever, backup media advice  (Read 4201 times)

0 Members and 1 Guest are viewing this topic.

Offline ebastler

  • Super Contributor
  • ***
  • Posts: 6971
  • Country: de
Re: Write once, read forever, backup media advice
« Reply #50 on: August 20, 2024, 04:30:19 pm »
Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

Are they? Not in my experience, as long as you buy brand name sticks.
 
The following users thanked this post: tooki

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 27923
  • Country: nl
    • NCT Developments
Re: Write once, read forever, backup media advice
« Reply #51 on: August 20, 2024, 04:43:48 pm »
Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

Are they? Not in my experience, as long as you buy brand name sticks.
Agreed. Never had trouble with name brand USB sticks. Typically I buy the ones from Kingston from reputable sources.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 
The following users thanked this post: tooki

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7211
  • Country: va
Re: Write once, read forever, backup media advice
« Reply #52 on: August 20, 2024, 04:51:03 pm »
Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

Are they? Not in my experience, as long as you buy brand name sticks.
They don't seem to have a very good reputation. Possibly not helped by being rather slow too.
 

Offline tooki

  • Super Contributor
  • ***
  • Posts: 12594
  • Country: ch
Re: Write once, read forever, backup media advice
« Reply #53 on: August 20, 2024, 05:43:54 pm »
Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

Are they? Not in my experience, as long as you buy brand name sticks.
They don't seem to have a very good reputation. Possibly not helped by being rather slow too.
That's because there are tons of garbage thumb drives out there, and many people bought those (or got them as promotional gifts, etc).

Thumb drives from top-quality vendors (the same ones one should be choosing for SD cards, too) just work. I've always bought SanDisk thumb drives, and SanDisk and Lexar memory cards (CF, MS, and SD, in my case) and have never had one fail.

More expensive ones are faster. But it does seem that thumb drive Flash controllers were not historically optimized for speed like SD card controllers. That changed with USB 3, and of course a USB SSD is just a high-performance thumb drive.
 
The following users thanked this post: nctnico

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7211
  • Country: va
Re: Write once, read forever, backup media advice
« Reply #54 on: August 20, 2024, 06:47:04 pm »
A lot of USB3 sticks are not really any faster on writes. Read, perhaps, but writes still boringly slow.
 

Offline David Hess

  • Super Contributor
  • ***
  • Posts: 17131
  • Country: us
  • DavidH
Re: Write once, read forever, backup media advice
« Reply #55 on: August 20, 2024, 08:17:00 pm »
Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

Prompted by mysterious incidents of data loss on USB Flash sticks, a couple years ago I ran tests are a bunch of random and almost new USB Flash sticks which I had collected, some generic and some name brand.  All of them lost data within not much longer than a year.  Some of the no-name ones lost data within a month.  Whether the USB stick was powered or not did not seem to make any difference.  These tests convinced me to use SSDs in USB enclosures for routine external storage instead of USB sticks, although this is not always possible.

Thinking about it, I concluded that because USB Flash sticks cannot rely on idle-time-scrubbing or scrub-on-read, because they cannot have power loss protection, they do not retain data well even when powered.  SSDs however may have power loss protection, so can support idle-time-scrubbing.

I have had better results with SD cards, at least when they are in continuous use.

There are some companies which make what should be reliable USB sticks and cards, like Swissbit, but they are also really expensive.

 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15321
  • Country: fr
Re: Write once, read forever, backup media advice
« Reply #56 on: August 20, 2024, 09:10:03 pm »
People often joke about paper or engraved stone as long term means of storage, but for multiple gigabyes this is a non-starter.

In all seriousness, it's not a terrible idea for small amounts of data, but yes, Gigabytes are probably out of the question. This is an interesting project: https://ollydbg.de/Paperbak/

Depending on how compressible your data is, you can store up to 3 MB on a standard A4 sheet of paper.

Except "modern" paper is utter crap and won't survive past a few decades, contrary to some very old paper that is still almost intact centuries later.
And of course, the minimum for the printing itself would be to use a laser printer, but even toner is not going to last longer than a few decades (depending on storage conditions). Maybe that's enough, but that can't be compared with some ancient paper sheets and inks that were made with completely different (and more durable) materials.

Some argue that very long term archival doesn't make sense anyway because people are likely not to have the equipment to read the data, a few centuries from now (or even just a few decades). But as long as the medium/format is documented somewhere, they'll always be able to rebuild equipment to read it if needed (unless humanity has gone through so bad de-evolution that they'll be unable to figure it out, in which case, it doesn't look good anyway).
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7211
  • Country: va
Re: Write once, read forever, backup media advice
« Reply #57 on: August 20, 2024, 10:18:47 pm »
Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

Prompted by mysterious incidents of data loss on USB Flash sticks, a couple years ago I ran tests are a bunch of random and almost new USB Flash sticks which I had collected ...

Thanks.

Now I think about it I do have some USB sticks doing serious duty - they are the boot drives for some servers which run up Xpenology and ESXi. Typically they are read only (copy Linux into RAM to run) and always on. But there have been power cuts and other power-down situations, and so far I've not had one fail to boot. Might just be lucky :)
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 27923
  • Country: nl
    • NCT Developments
Re: Write once, read forever, backup media advice
« Reply #58 on: August 20, 2024, 11:42:36 pm »
People often joke about paper or engraved stone as long term means of storage, but for multiple gigabyes this is a non-starter.

In all seriousness, it's not a terrible idea for small amounts of data, but yes, Gigabytes are probably out of the question. This is an interesting project: https://ollydbg.de/Paperbak/

Depending on how compressible your data is, you can store up to 3 MB on a standard A4 sheet of paper.

Except "modern" paper is utter crap and won't survive past a few decades, contrary to some very old paper that is still almost intact centuries later.
That is why acid-free paper specifically for archiving and preservation purposes exists: https://www.preservationequipment.com/Catalogue/Conservation-Materials/Paper-Board/What-is-acid-free-paper  (random Google find).
Quote

Some argue that very long term archival doesn't make sense anyway because people are likely not to have the equipment to read the data, a few centuries from now (or even just a few decades). But as long as the medium/format is documented somewhere, they'll always be able to rebuild equipment to read it if needed
If needed... but needing to build a contraption becomes expensive quickly which makes it not worthwhile to recover the information. And for documenting the storage method, you'll be back to using paper.
« Last Edit: August 20, 2024, 11:46:22 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline 3roomlab

  • Frequent Contributor
  • **
  • Posts: 852
  • Country: 00
 
The following users thanked this post: nctnico, PlainName

Offline jonovid

  • Super Contributor
  • ***
  • Posts: 1494
  • Country: au
    • JONOVID
Re: Write once, read forever, backup media advice
« Reply #60 on: August 22, 2024, 01:25:11 am »
perpetual generational backups every 10yrs is possible to keep it forever.
this works for linear media as in order to play back video  sound  images or text.
however porting raw unedited code or software between different hardware & media generations can be problematic.
in preserving the essence of the original digital media applications.
so for example porting a video game that is 20 yro software so it will run on new or next generation of hardware.
generational business practices can be a threat. so for example planned obsolescence in all types of digital media.
Hobbyist with a basic knowledge of electronics
 

Offline tooki

  • Super Contributor
  • ***
  • Posts: 12594
  • Country: ch
Re: Write once, read forever, backup media advice
« Reply #61 on: September 07, 2024, 10:57:18 am »
Except "modern" paper is utter crap and won't survive past a few decades, contrary to some very old paper that is still almost intact centuries later.
And of course, the minimum for the printing itself would be to use a laser printer, but even toner is not going to last longer than a few decades (depending on storage conditions). Maybe that's enough, but that can't be compared with some ancient paper sheets and inks that were made with completely different (and more durable) materials.
I think you are somewhat misinformed.

Certainly there is no evidence that “modern” (in the sense of what we can buy now) office papers only last a few decades, and we have papers from every era survive perfectly fine. In fact, papers from the early 20th century age worse than more recent papers, since we learned from the problems with early-20th-century papers.

In the 1980s, all halfway-decent papers were switched to acid-free formulas/processes, eliminating the source of most deterioration: acidity. (And in fact, there are bulk processes to deacidify paper that isn’t acid-free. Archivists use those methods to stabilize entire archives of documents printed on pre-1980s office paper.)

The papers that survive really poorly are the extremely cheap papers used for newspapers and inexpensive paperback novels; those start to yellow after just a few years in suboptimal conditions. But nobody would ever select those as an archival medium! You’d use a quality archival paper.


And it’s not as though all old/ancient papers and inks survive well. Archivists and historians have to deal with deteriorating documents all the time. Many of the inks used in antiquity are really awful, and either fade or damage the paper, or both. The common iron gall ink that was the dominant ink used for over 1400 years in the West is acidic, and sometimes eats through the paper. (Modern iron gall ink is formulated differently so that the acid evaporates quickly, reducing the problems. It’s a niche product now, though.)


As for what printing method to use: toner is good if the paper is kept cool and above all not in contact with soft PVC, whose plasticizers leach out, causing the toner to adhere to the PVC and release from the paper. In particular, monochrome laser is excellent. But honestly, if I were choosing a printing method, I would probably use inkjet. Modern pigment-based inks are very stable, and are waterproof. Since they absorb into the surface of the paper a bit, they can’t easily delaminate. (But one must choose an inkjet with ISO archival compliance, not just any old whatever from Best Buy.) I could also imagine that someone could make (if there isn’t already!) inkjets with specialty inks that adhere extra well and perhaps are also solvent-resistant.

Archival inks are a known technology. (Germany, for example, has specific standards for archival inks in pens, and requires many types of important documents to be signed with pens that have archival inks. There are similar standards for printer inks and toners.)


There are gazillions of toner-based photocopies from the 60s and 70s that survive just fine. Even those stored in mediocre conditions are usually okay. (Other than the aforementioned contact with PVC.) Heat and pressure can cause toner to slowly release to adjacent sheets, especially if the fusing process wasn’t ideal when the print was made. Stored dry, cool, and dark, toner survives well.


So if you’re talking about archiving historical documents and items originally intended for everyday use, then conservation of materials with poor archival characteristics is a big problem. But the context of this thread is deliberate archiving, with careful selection of materials for that purpose. There is no reason to doubt the longevity of these.
 

Online rsjsouza

  • Super Contributor
  • ***
  • Posts: 6058
  • Country: us
  • Eternally curious
    • Vbe - vídeo blog eletrônico
Re: Write once, read forever, backup media advice
« Reply #62 on: September 10, 2024, 12:05:52 am »
Indeed. I have 125+ year-old books that will beg to differ from the misconception that printed paper is crap. Sure, the pages do not exhibit anymore their flexibility and can get quite discolored, but its readability is still quite optimal. My earliest electronics-related material is dated back to 1952 and is in excellent shape until this day - all this is even more impressive as it was stored in quite sub-optimal conditions during its lifetime.
Vbe - vídeo blog eletrônico http://videos.vbeletronico.com

Oh, the "whys" of the datasheets... The information is there not to be an axiomatic truth, but instead each speck of data must be slowly inhaled while carefully performing a deep search inside oneself to find the true metaphysical sense...
 

Online Halcyon

  • Global Moderator
  • *****
  • Posts: 5925
  • Country: au
Re: Write once, read forever, backup media advice
« Reply #63 on: September 10, 2024, 02:08:09 am »
Why are USB sticks so dodgy compared to, say, SDcards or SSDs?

Are they? Not in my experience, as long as you buy brand name sticks.

I wouldn't go as far as saying "so dodgy", but based on my experience, I've had more failures of USB flash drives than any other media type (out of the many hundreds of flash drives I've used).

I generally buy Sandisk, but even then I've had a few of those die (including one of their "Extreme Pro" series). I've also had a few Verbatim drives go bad.
That being said, I also have an old LaCie USB 2.0 drive which has been fine for the last 20 years.

If you buy the cheap Chinese rebranded/no brand crap, then I wouldn't trust those as far as I can throw them.

Either way, I typically treat all USB flash media as a temporary storage only and certainly don't use them to store anything I can't afford to lose.
 

Offline Phil1977

  • Frequent Contributor
  • **
  • Posts: 727
  • Country: de
Re: Write once, read forever, backup media advice
« Reply #64 on: September 10, 2024, 07:03:25 am »

I wouldn't go as far as saying "so dodgy", but based on my experience, I've had more failures of USB flash drives than any other media type (out of the many hundreds of flash drives I've used).

It´s the usual "application rank":

The highest quality memory chips go into embedded systems, server SSDs and fast SSDs. Then come high quality storage cards like CF-Express cards and SATA-SSDs. Then come SD-Cards - USB Sticks are last before the trash bin.

In one of the OEMs specification this was shown in kind of a "quality pyramid".
 

Online Halcyon

  • Global Moderator
  • *****
  • Posts: 5925
  • Country: au
Re: Write once, read forever, backup media advice
« Reply #65 on: September 10, 2024, 07:10:38 am »

I wouldn't go as far as saying "so dodgy", but based on my experience, I've had more failures of USB flash drives than any other media type (out of the many hundreds of flash drives I've used).

It´s the usual "application rank":

The highest quality memory chips go into embedded systems, server SSDs and fast SSDs. Then come high quality storage cards like CF-Express cards and SATA-SSDs. Then come SD-Cards - USB Sticks are last before the trash bin.

In one of the OEMs specification this was shown in kind of a "quality pyramid".

Yep, 100%. And it's that age old "your get what you pay for". Don't expect $500 performance/resilience/reliability out of a $50 device.
 

Offline Phil1977

  • Frequent Contributor
  • **
  • Posts: 727
  • Country: de
Re: Write once, read forever, backup media advice
« Reply #66 on: September 10, 2024, 07:43:18 am »
If I sell you this trashy USB-Stick for $500,- I´ll promise it to be very good  :-+ :-+ :-+
 

Offline Ranayna

  • Frequent Contributor
  • **
  • Posts: 904
  • Country: de
Re: Write once, read forever, backup media advice
« Reply #67 on: September 10, 2024, 07:59:00 am »
Quote
History has proven that texts and drawings on high quality paper can last for hundreds of years.

Yes, they can be. But, equally, they can last half an hour in the wrong circumstance. Also, we only know of the ones that made it, never the ones that didn't (observer bias).

True. But if you put a USB stick, a optical disk, an EPROM and a piece of paper in a safe, I'm pretty sure the thing that remains readable after 100 years is the piece of paper. The rest needs effort and knowledge that might no longer exist IF (big if) the medium hasn't degraded.

The EPROM, optical disk if not organic dye based, and paper will still be readable.  The USB stick, or any current Flash based media, will be toast.  Based on my tests a couple years ago, I would consider it lucky if a USB stick lasted more than a year.
For anything requiring anything special to read - and maybe more important - understand! the archived data, you will run into trouble.
Printed paper may already be iffy if languages change, though this will not happen in a hundred years.

But even if you can still read EPROM, or read a disc with a microscope, can you still understand the encoding?
 

Offline macboy

  • Super Contributor
  • ***
  • Posts: 2287
  • Country: ca
Re: Write once, read forever, backup media advice
« Reply #68 on: September 10, 2024, 02:45:20 pm »
I still have and use some TDK brand Archival DVD+R media. They claim a 100 year lifetime (special dye?) and have the same ultra-hard coating that Blu-ray disks have. Running one's fingernail on the bottom surface of the disc, the feeling is distinctly different than regular DVD or CD media. It feels hard like glass, like a phone screen, rather than plastic. You literally can't scratch these with your fingernail, and they survive a lot of mishandling without damage. These are single layer 4.7 GB discs but use dual-layer-like construction, sandwiching the recording layer in between two substrate layers, rather than having it on the top of a single substrate and coated with a thin "protective" coating like standard CD and DVD media. This physically protects the recording layer and slows the (inevitable) ingress of damaging oxygen. Because they are "dual" layer and DVD+R (not -R) they can't be read on some of the oldest DVD-ROMs which lack those capabilities.  Every so often, I burn some irreplaceable data onto some of these discs and stash them away. TDK no longer makes these but Verbatim has a similar product. Between the construction, the 100-year dye, the superior DVD+R error correction and laser tracking (vs DVD-R), and the scratch resistant coating, these disks are in a different league from regular DVD-R blanks.

I also keep a single special old DVD-ROM drive (actually a "combo" drive: burns CD but only reads DVD). I recognize it as special because I once, several years ago, was trying to copy data from some very old CD-R discs, but had a heck of a time reading them. Neither drive in my main computer would read the discs, nor would the drive in my secondary computer nor laptops (remember when every laptop had an optical drive?). Some just spun and never mounted, some would mount but read slowly then error out. As a last resort, I tried all the drives I had stashed away, and amazingly this drive was able to read every single bad CD-R without any issue, and at full speed! I promptly labelled this drive as special and have kept it ever since.

I wish I had a better off-site backup strategy, but I'm still learning and growing too.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf