Attempting repair of TDS540C - Option 1G - Fail Processor

[shakalnokturn]

This is going to turn out as the longest wound 24C02 replacement on the forum 

[flyte]

Everthing else measure and action without replacing of EEPROM is waste of time. Only one solution: replace EEPROM.

That would be a premature conclusion. There's a ton of options left for this already record-breaking thread:

- decap the EEPROM chip and determine which bond wire broke
- examine it under an electron microscope
- perform a die microprobing attack in order to read it
- get the semicon manufacturer involved and assess the situation
- ...

[Tantratron]

This is going to turn out as the longest wound 24C02 replacement on the forum 

That would be a premature conclusion. There's a ton of options left for this already record-breaking thread:

@shakalnokturn @flyte

The thread is quite long because it covers different topics plus there has been different hypothesis by different members.

On page 1 there was @madao saying the problem was due to NVRAM DS1486 and suggested that I upgrade from 1G to 2G the acquisition board. As I did not have for the moment memory programmer plus I only use Macintosh, it took a while to get the GPIB work and thank to the collecting work of @ragge, I was finally able to upgrade the firmware, to dump and write new NVRAM. Some other members said it was the log file going overflow... many theories where it turned out the problem was not the NVRAM.

I made some mistake understanding the instructions to upgrade to 2G but finally worked per the instructions of @madao plus I was securing the use of my USB-GPIB from my MacBook Air to non-invasive read-write the NVRAM, the flashfile which brings us to the core topic on page 3.

At that moment I decided to install the good Acq-board into the fail TDS540C, it worked so for sure the problem was not a Fail ++Processor as claimed by tektronix but a Fail ++Acquisition https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2860746/#msg2860746

My feeling since the topic was new for me plus it was the 1st time I discover these TDS then I'd prefer to look on internet (EEVBlog, tekforum) for similar reported failure then I've found the threads of @charlyd
https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2868584/#msg2868584
https://forum.tek.com/viewtopic.php?t=138489

For the first time, there was a real repair describing the root cause from @charlyd, namely the EEPROM chip but I was not sure to fully understand the explanation, in fact @ragge was the same https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2877512/#msg2877512

It is quite ironic later on page 4 that @shakalnokturn suggest that I swap the other boards which I hesitated during few days https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2880974/#msg2880974

Finally on page 4 there was the only serious and sound engineer explanation by @flyte on what was happening, the root cause so I took time to verify, understand
https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2886008/#msg2886008

However I got again confused because for some reason the local RAM copy of the default Cal were all zeros so I started to suspect another failure in the processor board
https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2889560/#msg2889560

Later on page 5 there has been some philosophy disagreement between software testing and hardware BUT the key point you need to realize, there was no rush, there is no rush. I have a working TDS540C to use for my little consulting company, I do not have too much spare time due to work and family duties (wife, two children). Furthermore I made clear many times to only use Macintosh, to not have the FAS adequate equipment for that scope, no programming tool for the moment and I wanted to interface my Mac via RS323-USB interface to the J40 debug consol port as a next stage.

I decided to probe finally the I2C bus lines and learning the low level coding of I2C bus which was new to me, finally it was clearly established the EEPROM to not send any data to the ASIC, see page 5
https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2892782/#msg2892782

As a reminder, this TDS540C which was imported from a USA lab to a french lab had a long story of failure (check attached the log file from 2003) where now I suspect the ERROR: 250 mass storage error with READ fail was probably the EEPROM (NV 250) hence my reticence plus its front panel is partially broken
https://www.eevblog.com/forum/repair/tektronix-tds540c-cursor-partial-failure/msg2809344/#msg2809344

Anyway the point, if this thread is too lengthy or not interesting, just don't read it and click UNOTIFY. If you do want to help, just accept that I'm not the employee of anybody, my time is constrained with other activities, I'm not in rush to repair this TDS but rather invest my time to learn this TDS serie totally new for me to decide if I'll keep for my consulting company
https://www.eevblog.com/forum/repair/tds-500-and-tds700-series-a-b-c-or-d-what-subsystem-are-common/msg2831168/#msg2831168

One thing which worries me by the way is the lack of solution about the NVRAM battery failure... What is now certain, we have two similar failures log message from @charlyd and @tantratron which will save time in the future for others even though it seems this EEPROM failure is not very common.

From my point view, this thread and these others had helped me lot learn from scratch the basic of TDS plus keep my MacBook Air to use the GPIB to do many things like dump, write, setting options (FFT)
https://www.eevblog.com/forum/repair/gpib-usb-control-between-macbook-air-(macintosh)-and-tds540c/msg2835780/#msg2835780
https://www.eevblog.com/forum/repair/tekfwtool-for-tds540c-firmware-upgrade/msg2845956/#msg2845956
https://github.com/ragges/tektools
...

Special thanks for @charlyd (first to have repaired this fault) @madao (many tricks and patience) @ragge (GPIB-USB repository valid for Mac) @flyte (floppy stuff and clear explanation).

In the meantime, I'll try as a project to find a method to Macintosh TERMINAL read what the J40 detailed debug says in general with the EEPROM still soldered. After i'll de-solder both X24C02, will build a quick&dirty arduino style programmer (read, write), will replace the EEPROM and will find someone with required tools (PC, DOS, high frequency generator) to do the calibration(the FAS does not run on Linux neither MacOS).

Patience is a virtue...

Thank you and peace, Albert

[charlyd]

feeks like were are back at post #72   ….. 

[madao]

I had a lots of thoughts and times about what to write in this topics.

 

Sorry, albert. I have written, that every action without  replacing  of EEPROM is a waste of time.

 

People of eevblog wait on the result after two very clear answers (EEPROM is dead) and some users have experience with dead EEPROM.

What did you do?  You  wrote a topic with other question (not important for fixing) and didn’t show a result about the EEPROM.  Of course, most people - including me - are annoyed with this „result“. It is similar to repairing a car:  people tell you the wheel  bearing  makes strange noises and you ignore this noise up to the point, when the wheel rolls away and the bearing is broken.  Of course , people don’t  tell you after a few times about this noise and they think: not my problem, if  he doesnt  take this notice about the bad bearing seriously.  It is really not funny for many people, we want help and share experience and are disappointed with this result.

 

You have gotten my offer of fixing this fail, but i wait and wait and wait….. (I know: you wait on an answer from university.)
Indeed, i must cancel my offer to save my time. I have told you, I have much less time since spring (other hobby, car, nissan R34)

Greeting
Matt
 

[Tantratron]

I had a lots of thoughts and times about what to write in this topics.

Sorry, albert. I have written, that every action without  replacing  of EEPROM is a waste of time.

People of eevblog wait on the result after two very clear answers (EEPROM is dead) and some users have experience with dead EEPROM.

I'm sorry Matt but what you said is wrong...

As i've already mentioned, you did first write the problem was NVRAM on processor board (CPU board), you did ask I upload NVRAM's... I had to wait to get my MacBook Air to run with GPIB-USB and tekfwtool but NVRAM upload never solved the problem (check page 1 to page 2). Of course I thank you because you did provide very good help to upgrade from 1G to 2G the acquisition board https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2856924/#msg2856924

Later on page 3, I've solved the problem from a system level by proving it was Acquisition board to be the problem https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2860746/#msg2860746

At this moment we All discover that tektronix troubleshooting display is wrong, namely it says FAIL ++Processor whereas it should state FAIL ++Acquisition. In fact @shakalnokturn wrote about this paradigm or strange situation https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2860990/#msg2860990

After all we're not shaman to discover hidden information so it would have been better for tektronix firmware to correctly troubleshoot by saying Fail ++EEPROM on Acquisition board...

Different members including yourself from this moment suggested it was the EEPROM to be damaged but I hesitated to repair the Acquisition board (replace the EEPROM) because I had another strange failure or effect from the Processor board... see RED sentence in this post https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2889560/#msg2889560

Furthermore I did not have a programmer neither a spare 24C02 chipset and most important, neither had all equipment to perform a full calibration so why spend time to desolder. As a reminder I have another TDS540C which runs very good and I'm using for my daily activities for my company.

What is certain, the only member who did solve the same problem was @charlyd on this post https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2868584/#msg2868584

feeks like were are back at post #72   ….. 

Yeap plus later I finally probed (after many posts) the I2C signals of both EEPROMs to confirm they did not respond to the ASIC read attempt https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2899120/#msg2899120 then https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2899212/#msg2899212

Anyway here is a status of my repair as of february 5th... on january 8th before going through many swapping, uploading and I did DUMP (save) both TDS540C firmware's and NVRAM's as a safety procedure. So on february 5th I dediced to upload the january 8th version of NVRAM's in the failed TDS540C and guess what, now the getcaldata reads the correct default EEPROM values from the firmware. This means that @flyte was right of risk to brick the scope by all these swapping, readings, writings https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2890442/#msg2890442

So I was lucky to have saved NVRAMs early january and now the acquisition board works fine again, namely it sees the EEPROM is wrong using getCalData via GPIB-USB so it stores default calibration values from the firmware as explained by @flyte here https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2886008/#msg2886008

My intent is soon to de-solder both EEPROM, build an arduino style programmer to see if one can be read. Of course I'll buy two EEPROM and solder them back BUT my real problem is no tools in my company for the moment to properly run the FAS.

Best regards, Albert

[charlyd]

Hi Albert, as i mentioned before change both EEproms with new EMPTY onces and start to calibrate…
clear the error log, after soldering and putting things back together. A friend of mine has his own website with some nice TDS tools, here is one you could use.

http://www.hakanh.com/dl/TDSErrLog.htm

and see what comes back your "FATAL: 250 NVstorage too small more bytes requested then available" should be gone.
only a message:"nvlibrarianDiags, libs with crcc failures: , ExtConst" or so will be still visible in the log, which points to the calibration with the FAS.

what and how to do. read back if some of my forst posts.

Did you ever fix the console port adapter which is connected on the cpu board through the RS232 hole. ??

but ok, remember your error log is your reference for having placed working chips 

@Ragge, yes the FAS writes the Eeproms.

But what could be the real reason that these EEProms start failing over time, in a particular range of models?

[shakalnokturn]

But what could be the real reason that these EEProms start failing over time, in a particular range of models?

Something to do with them not using the usual pull-up scheme on the I²C bus?

[Tantratron]

My intent is soon to de-solder both EEPROM, build an arduino style programmer to see if one can be read. Of course I'll buy two EEPROM and solder them back BUT my real problem is no tools in my company for the moment to properly run the FAS.

First I hope most of you are OK with the Covid19 issue wether on health and economical.

On my side with my small consulting and testing company, it is a mess like most companies in France or everywhere worldwide whatever the industry with impossibility to do any business development, do or win new contracts... so I have lot of time due to confinement during april.

Attached some pictures where I've finally un-soldered both X24C02 (Xicor) then re-soldered two new 24C02H (On Semi). I was able via eBay to purchase 20 of them for 10€ last week and they got delivered earlier this week.

It seems this time much less error log messages where as predicted by some of you, the U1052 fail to be read from the ASIC was creating the NV or not enough memory. However there is still the Partial Fail ++Processor and diagnostic error including the wrong CRC.

Maybe this is due to the fact I've not calibrated the scope (no tools for this as of today) neither programmed the 24C02H prior soldering where these are brand new. Even if I had a programmer, I do not have access to the proper content of the EEPROM so I just soldered them to see the result. So I'll try later make an arduino EEPROM reader to see if one of the X24C02 has the philosopher stone data of that specific acquisition board.

I'm fully aware of the option to use @flyte floppy driver method to re-program the EEPROM https://www.eevblog.com/forum/testgear/tektronix-tds500600700-nvram-floppy-dump-tool/

Best wished during this very hard times with the coronavirus pandemic affecting all countries.

Albert

[Tantratron]

P.S. Here is a final update where the TDS540C now pass the self-test as well as the SPC even though no calibration.

I've used the tool set of @flyte https://www.eevblog.com/forum/testgear/tektronix-tds500600700-nvram-floppy-dump-tool/msg2887352/#msg2887352 to read the EEPROM then write them with different EEPROM contents to see the effect from the floppy disk.

Since I did not program the new 24C02's eeprom so they're brand new into the acquisition board, I've first dumped their content which was full of 1's (FFFFFF...). I guess this file content corrupts the CRC check of the TDS540C explaining the self-test partial fail.

Then I've written inside the EEPROM a full content of 0's (000000...) but it still make the self-test to fail even though their CRC is zero. So it seems the TDS operating system knows that a EEPROM content full of zeros is a problem and cannot be a valid acquisition board parameters.

The luck from the start, I do have another TDS540C which was always working so I've copied its EEPROM content into the failed one via the floppy disk method of @flyte. Maybe I've been lucky or miss a technical point but it looks like that I do not need to calibrate because self-test OK and SPC passed all (see attached pictures).

Later I'll still build an arduino EEPROM reader to see if the previous EEPROM can be partially recovered (probably not the first one but maybe yes the 2nd one).

Stay safe and healthy, Albert

[charlyd]

Hi albert this looks good. so at the end.. you went the route i told you and the error is gone now, but with (wrong) cals from the other scope.
i mean to say they are within the borders but a new calibration is for this scope always the best choose.   what do you need just a NI GPIB  card i found mine for $20 and an old PIII pc or so. assuming you have a SG to max 1Ghz . but even those are for a few $$ for sale on ebay.  ( and a few corona hours , nice to do ) and when you finished you can recal the other so there or both ready for the next 20 years.

but ok nice repair. what is the next project gone be …...

[Tantratron]

Hi albert this looks good. so at the end.. you went the route i told you and the error is gone now, but with (wrong) cals from the other scope.
i mean to say they are within the borders but a new calibration is for this scope always the best choose.   what do you need just a NI GPIB  card i found mine for $20 and an old PIII pc or so. assuming you have a SG to max 1Ghz . but even those are for a few $$ for sale on ebay.  ( and a few corona hours , nice to do ) and when you finished you can recal the other so there or both ready for the next 20 years.

but ok nice repair. what is the next project gone be …...

Thank you again @charlyd where indeed you were right from the start, the U1052 eeprom was the culprit and we might have been the only two so far publicly to get that seldom and strange Fail ++Processor.

Everthing else measure and action without replacing of EEPROM is waste of time. Only one solution: replace EEPROM.

That would be a premature conclusion. There's a ton of options left for this already record-breaking thread:

- decap the EEPROM chip and determine which bond wire broke
- examine it under an electron microscope
- perform a die microprobing attack in order to read it
- get the semicon manufacturer involved and assess the situation
- ...

For those of you interested with one solution to repair then dump the EEPROM content, after different trials this week I've fully recovered the 256 octets so the TDS540-1G does not need calibration... see conclusion here https://www.eevblog.com/forum/repair/attempting-repair-eeprom-voltage-increase-method/msg3011456/#msg3011456 or read complete thread for the repair process using an arduino EEPROM reader tool and some soldering

Albert