Attempting repair of TDS540C - Option 1G - Fail Processor

[flyte]

In addition to my previous post:

If you anyhow want to proceed with your current non-invasive diagnostic method only - after all, it's your scope and your decision - then there is not much else you can do but to simply fire up the EEPROM writing script using a good EEPROM dump right away and hope it restores the scope to its former glory.

But no complaints then if it didn't fix it and it's a hardware problem after all, or worse, that the scope would also be way off and you would have lost your original calibration data forever. Meanwhile quite some people here have shared and expressed the same concern.

Admittedly, there is a nice chance those EEPROMs do in fact contain all zeros indeed and it's simply a reprogramming job, but I would never risk to make that assumption at this point without having read the actual values from them externally, certainly knowing the effort is small.

[Tantratron]

If you anyhow want to proceed with your current non-invasive diagnostic method only - after all, it's your scope and your decision - then there is not much else you can do but to simply fire up the EEPROM writing script using a good EEPROM dump right away and hope it restores the scope to its former glory.

But no complaints then if it didn't fix it and it's a hardware problem after all, or worse, that the scope would also be way off and you would have lost your original calibration data forever. Meanwhile quite some people here have shared and expressed the same concern.

Admittedly, there is a nice chance those EEPROMs do in fact contain all zeros indeed and it's simply a reprogramming job, but I would never risk to make that assumption at this point without having read the actual values from them externally, certainly knowing the effort is small.

The reason I'm still hesitating to desolder both EEPROM then check their internal content by another means (i.e. arduino style) is quite simple. As it has been warned by many members including yourself, the philosopher stone or the core heart data of these scopes lies in the calibration content of these EEPROM. One false move like to much heat to desolder or writing wrong data then I'll loose for ever the knowledge of the factory calibration of that specific acquisition board. I've never said to not do it with my heat gun... just I'm cautious plus I've the chance to have a full working operational TDS540C to swap some sub-systems then investigate in non-intrusive manner.

Just as an update, I've now removed the swapped Acq-board then installed the good Acquisition board inside the good TDS540C. As for the bad Acq-board, I've inspect it for bad PCB traces, checked some continuities per the schematics, so far so good. Later i'll try inspect visually how to recover via through holes lines the critical pins (SDA, SCL, WC) from the top side. The idea will be to probe these 3 signals once i'll install later the failed Acq-board inside the failed TDS540C.

Ok now about the good TDS540C with good Processor board and good Acquisition board, back to the start where it works like a charm. So I did use again the Floppy-read script to dump the EEPROM then run the GPIB-getCalData application to dump the EEPROM again. As mentioned before both BIN files match together AND are slightly different that the default EEPROM content saved inside the firmware. So the TDS operation system dumped internally the EEPROM, it concluded correct checksum so did not load the default EEPROM inside the RAM.

Maybe one last question about the firmware management of these TDSxxx-C scope which you described yesterday. Suppose indeed for some reason the EEPROM content is full of zeroes. It happens unfortunately that tektronix did not check that case in their SW design so this ends up with a valid CRC check when the software reads the EEPROM. I guess in that situation the TDS operating system will never attempt to use the EEPROM default values found in the firmware because it concluded good checksum... same old issue as you did find with your first java-checksum application where now you have reinforced the test by excluding all zeroes. However what happens then with calibration constant loaded to all zeroes, the scope thinks everything is OK with calibration constants and so except they are all zeros ?

Furthermore as mentioned before, my intent is to later attempt finding the proper connector, cable and driver driver to actually check via terminal console on my Macintosh the RS232 Console port information via the J40 processor board interface.

Amicalement, Albert

[flyte]

One false move like to much heat to desolder or writing wrong data then I'll loose for ever the knowledge of the factory calibration of that specific acquisition board. I've never said to not do it with my heat gun... just I'm cautious plus I've the chance to have a full working operational TDS540C to swap some sub-systems then investigate in non-intrusive manner.

But software reading leads to a dead end for your bad board, you get either defaults or zeroes, so you'll have to get that soldering iron out. If you do care about the constants as you mention it, that would be your only option at this point.

As mentioned before both BIN files match together AND are slightly different that the default EEPROM content saved inside the firmware.

I've now checked your originally posted EEPROM-TDS540C.zip as well and the two dumps are very different. It's the same structure, sure, but it's like 75% difference just byte-wise. The 2G dump seems normal, the 1G has the exact same values as the firmware defaults and it is the byproduct of the way the GPIB tool reads (i.e. initialized shadow space rather than a fresh hardware access).

Maybe one last question about the firmware management of these TDSxxx-C scope which you described yesterday. Suppose indeed for some reason the EEPROM content is full of zeroes. It happens unfortunately that tektronix did not check that case in their SW design so this ends up with a valid CRC check when the software reads the EEPROM. I guess in that situation the TDS operating system will never attempt to use the EEPROM default values found in the firmware because it concluded good checksum... same old issue as you did find with your first java-checksum application where now you have reinforced the test by excluding all zeroes. However what happens then with calibration constant loaded to all zeroes, the scope thinks everything is OK with calibration constants and so except they are all zeros ?

No that's not the case. It was just a somewhat unfortunate side effect of my checksum tool. The real firmware will verify the checksum as valid but it will also notice due to other parameters the store doesn't contain any values at the indices of interest. So loading all zeroes will get you a CPU FAIL ExtCal. However, whether or not it will proceed with a default value initialization or just leaves the zeroes it read in RAM at certain points in time, depending on when your GPIB read exactly intervenes as this is a concurrent VxWorks task if I'm not mistaken, I'm not sure. It should, in case they've done the job right.   

Alright, good luck with the soldering iron, I hope you do discover some values in there instead of the bunch of zeroes you're faced with for now! 

[Tantratron]

I've now checked your originally posted EEPROM-TDS540C.zip as well and the two dumps are very different. It's the same structure, sure, but it's like 75% difference just byte-wise. The 2G dump seems normal, the 1G has the exact same values as the firmware defaults and it is the byproduct of the way the GPIB tool reads (i.e. initialized shadow space rather than a fresh hardware access).

Ok I've taken the time to inspect both PCB faces of the Bad Acq-Board in the EEPROM zoning in order to be able probing later in the week the key signals (SDA, SCL, GND and +5V). After this I did re-install the bad Acq-board inside the bad TDS-1G then performed some Floppy-dump and GPIB-dump. It seems not possible to probe the WRITE-Enable trace from the PCB-top once re-installed.

See attached a ZIP containing 4 folders, each folder corresponds a scenario with Floppy and GPIB dumps. What is now SUPER wierd, wether installing good Acq-board or bad Acq-board in the failed TDS540C-1G will always return full of ZEROS if dumping via GPIB. Few days ago this was not the case as you saw on my previous ZIP even though their content clearly showed problem. Are we OK that the GPIB only dumps the RAM content which is supposed to be a copy of the EEPROM if checksum correct or copy of the EEPROM default stored in the Firmware ?

Do make a note that unless i'm wrong, wether the EEPROM is still OK or not, wether we dump by Floppy... the dumps will always have to transit through the ADG308 in charge of the I2C management (bottleneck bus situation). My point is that maybe the EEPROM is corrupted and now full of zeros, the Floppy dump returns sometimes all ZEROS, other times all FIVES... however I'm afraid the Acq-Board has multiple issues (maybe ADG308) because it does not make any sense to have correct Floppy dump EEPROM file of good Acq-board when installed inside bad TDS-1G but wrong GPIB dump full of ZEROS (here the Acq-board is the good one)...please folder TDS1G-Good-AcqBoard in the attached ZIP file. Lastly as a reminder, when installing good Acq-board into the failed TDS-1G then the scope passes all self-tests where I did push the exercice to run complete SPC which was fine !

[flyte]

@Tantratron

Don't get me wrong, but you each time you try all sorts of swaps and readouts, you are simply increasing the likelihood of ruining your calibration constants EEPROMs. Why that is? Because you still have no EEPROM backup at this point and you are probably dealing with a hardware failure.

So it could happen - it just could at a certain moment - your persistence in innocent software reading attempts is sending some wrong messages over the I2C bus due to hardware defects, inadvertently issuing write or erase commands, for example. It's like the first rule of computer forensics: don't touch it, leave the data state intact.

Yes, the I2C traffic always transits through the ASIC, it can't go elsewhere, can it?     But the dumps as such, no. If you read via the GPIB tool, as mentioned several times, you are actually reading from RAM and you can hope (or not) the firmware properly initialized that section with the data from the EEPROM. That likely didn't happen or failed on some occasions, as it did put defaults in there. It's very unlikely these defaults actually come from the EEPROM by accident. In case you read with the floppy script, it's a direct hardware call to the ASIC/I2C.

You say some observations don't make sense. They do, always. Unless it's yet another theory on particle physics or the origins of the universe.   It's just we/you can't see the whole picture of the edge cases the firmware is operating in and what it is doing in that exact case.

Your acquisition board may certainly have multiple issues. You could fix all of those, in theory at least. The only thing which you can't fix easily, is getting those cal constants back. Hence you should concentrate your efforts at getting those out of the EEPROMs asap safely. In my humble opinion, which seems shared by other members, this is only possible by removing them and reading them externally, if it's not too late already.

Btw, did you check silly things like the pullup or pulldown resistors on the SDA/SCL bus? Or signal levels? That would be one of my first suspects and a very easy thing to check.

EDIT: just checked your uploaded files. You get 0x0000 and 0x5555 as data result from the floppy dumper. 0x5555 is 0101010101... in binary. This smells like an I2C level or clock feedback problem. I would bet on a bad pullup resistor and a floating bus data line, because if the EEPROMs were bad, it's a bit less likely they're bad two at once. Of course, one could be bad and pull down the bus. I don't even care what the GPIB tool did, it does not matter.

[Tantratron]

@Tantratron

Don't get me wrong, but you each time you try all sorts of swaps and readouts, you are simply increasing the likelihood of ruining your calibration constants EEPROMs. Why that is? Because you still have no EEPROM backup at this point and you are probably dealing with a hardware failure.

So it could happen - it just could at a certain moment - your persistence in innocent software reading attempts is sending some wrong messages over the I2C bus due to hardware defects, inadvertently issuing write or erase commands, for example. It's like the first rule of computer forensics: don't touch it, leave the data state intact.

I see your point but I come from digital signal processing industry as well as radio electronics. My feeling was these TDS are really more digital or software oriented where roughly the only analog part is the acquisition board. So far I was using legacy tektronix 2400 serie so I'm on my learning curve with TDSxxx. However my assumption (maybe I'm wrong) was that if reading whatever memory then there should not be writing access. Usually when I repair stuff in the industry (avionics, radio...), the cannibalization technique is safe or I'd say is the quickest. You replace one suspect sub-system by a working system to isolate which one is faulty. This logic works very well when there is one failure but if multiple failures, there are dependencies hence one can get lost. In fact that is what happened to me with this scope even though I'm very happy because learning lot on this TDS xxx family which I feel investing more to use them, keep them.

What you describe is kind of catastrophic scenario where any test would destroy the machine. Maybe you're right here but usually any good electronic equipment with good software has built-in protection like fail-safe or detect then stop to avoid a runway destructive effect.

As for the EEPROM topic, honestly in this forum and the tek forum only one member @charlyd reported the same type of error log so this EEPROM topic is very seldom. This is why I felt not enough evidence to conclude it was my EEPROM then went on the route to swap and dump from different views. In fact I've learned thanks to you in this thread that getCalData is copy of RAM of either good EEPROM if checksum is OK or copy of default EEPROM stored in Firmware. If I'd have known this before, for sure I'd not have lost my time going along with many swaps, dumpings and false interpretation.

What is certain, I did the test here https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2860746/#msg2860746. The failed TDS-1G once installed a good Acq-Board-2G works fine but what I've discovered later, there seems to be a GPIB issue or something else. The TDS-1G processor-board might be the problem because it does not make any sense with good Acq-Board to GPBIB-getCalData only ZEROS from the RAM copy since the EEPROM. In that test, the Acq board is good, the EEPROM are good so I only kept the 1G-Processor board which pass self-test but does NOT match the BIN files of EEPROM. Your Floppy-script provides the good EEPROM content but the GPIB-dump shows zero. Once I re-install the good Acq-board into the good TDS then every works: your floppy-dump gives exactly the same content of the EEPROM than the GPIB-dump.

So I suspect even though maybe I'm wrong, the failed scope has another partial failure on its Processor-board.

Your acquisition board may certainly have multiple issues. You could fix all of those, in theory at least. The only thing which you can't fix easily, is getting those cal constants back. Hence you should concentrate your efforts at getting those out of the EEPROMs asap safely. In my humble opinion, which seems shared by other members, this is only possible by removing them and reading them externally, if it's not too late already.

Btw, did you check silly things like the pullup or pulldown resistors on the SDA/SCL bus? Or signal levels? That would be one of my first suspects and a very easy thing to check.

EDIT: just checked your uploaded files. You get 0x0000 and 0x5555 as data result from the floppy dumper. 0x5555 is 0101010101... in binary. This smells like an I2C level or clock feedback problem. I would bet on a bad pullup resistor and a floating bus data line, because if the EEPROMs were bad, it's a bit less likely they're bad two at once. Of course, one could be bad and pull down the bus. I don't even care what the GPIB tool did, it does not matter.

Before embarking into de-solder the EEPROM, today I took some of time to try reverse engineer and look patiently of the PCB traces near the EEPROM but also the NVWR protect parts. I did check the electrical continuities of that circuit, so far I did not see any issue then I connected my good old tektronix 2465 with a P6136 probe to check the values of different test signals related to U1052 and U1055 chips (+5V, GND, toggling effect of protection write switch)...

The WC (Write Enable) is always HIGH (5V) whereas NVWR_EN stays LOW so there will be never write access to both EEPROM. Once scope started after a while, I toggle the protect switch, the WC became LOW (0V) while the NVWR_EN becomes HIGH (+5V) so this validates the U1061 chip translation of the NVWR_EN signal to be correct.

The Vss and Vdd of the chips are OK on different parts of the PCB as far as I can tell.

However the SDA (DATA) seems always HIGH and the SCL (Clock) always LOW.

Since I've never probed the SDA and SCL of this TDSxxx serie C, I've no idea what time div frequency scale neither when to look for the CLK or DATA modulation after booting the scope. Maybe you could advise on what best testing strategy to check the data line and clock line once I start the TDS540C... at what moment the ADG308 will engage in the I2C protocol to read the EEPROM, how can I capture the modulation...

Thank you, Albert

[shakalnokturn]

I'd set acquisition for single and trigger for more than one state change, see if that gets you anything on a power-up.

[flyte]

However the SDA (DATA) seems always HIGH and the SCL (Clock) always LOW.

Since I've never probed the SDA and SCL of this TDSxxx serie C, I've no idea what time div frequency scale neither when to look for the CLK or DATA modulation after booting the scope. Maybe you could advise on what best testing strategy to check the data line and clock line once I start the TDS540C... at what moment the ADG308 will engage in the I2C protocol to read the EEPROM, how can I capture the modulation...

The scope will read the I2C at startup and also, obviously, when you run the EEPROM floppy dump script. Set your trigger (I mean, on the equipment you're using to diagnose) to single shot mode on the SCL line, put SDA on the other scope channel, and with the first clock transition you should see whether anything happens on that bus or not. I2C is usually in the 100KHz range, it's slow.

In case you want to know whether your CPU board is good or not, I'm repeating myself here, swap it with the CPU board in the fully working unit. But just the CPU board, as the fault could also be located in the cables, the connectors, some power supply line, whatever, you just don't know. I'm not sure I understand why you keep on insisting to find an explanation for the wrong GetCalData GPIB read, it's irrelevant at this point. It could be a side effect of something you're overlooking at this point. If the CPU board fully works inside the other good unit, then it's good. If not, then it's bad. Pretty straightforward. When testing this, don't forget to load the NVRAM of the "good" unit into the CPU board you're testing/suspecting. Both boards should have the same firmware, same dip switch settings, same NVRAM content ... i.e. they should be identical as possible. Only then you can draw conclusions and isolate the fault.

[Tantratron]

The scope will read the I2C at startup and also, obviously, when you run the EEPROM floppy dump script. Set your trigger (I mean, on the equipment you're using to diagnose) to single shot mode on the SCL line, put SDA on the other scope channel, and with the first clock transition you should see whether anything happens on that bus or not. I2C is usually in the 100KHz range, it's slow.

In case you want to know whether your CPU board is good or not, I'm repeating myself here, swap it with the CPU board in the fully working unit. But just the CPU board, as the fault could also be located in the cables, the connectors, some power supply line, whatever, you just don't know. I'm not sure I understand why you keep on insisting to find an explanation for the wrong GetCalData GPIB read, it's irrelevant at this point. It could be a side effect of something you're overlooking at this point. If the CPU board fully works inside the other good unit, then it's good. If not, then it's bad. Pretty straightforward. When testing this, don't forget to load the NVRAM of the "good" unit into the CPU board you're testing/suspecting. Both boards should have the same firmware, same dip switch settings, same NVRAM content ... i.e. they should be identical as possible. Only then you can draw conclusions and isolate the fault.

This morning I woke up much early to do some comparative test on SCL and SDA single-probed then displayed with my tek 2465 scope before doing my daily job. The 2465 time division set to 100 ms enough to see drastically different behaviors after starting, re-starting multiple times each Acq-Boards ASIC-EEPROM communication.

Attached two new updates pictures showing both faces of the Acq-PCB zoning being focussing on the ADG308 Asic, the X24C02 eeproms, the unique Pull-up resistor (R1065 for SDA) and unique Pull-down resistor (R1060 for NVWR_EN) related to the U1721 chip.

I've re-checked again many times for any PCB trace discontinuity or local bad soldering, it is seems OK.

Make a first note both WR pin (X24C0-7) always show HIGH (+5V) when probed plus the only pull-down resistor is related to that sub-topic via the U1721 XOR chip translating the Protect switch order (there is no way the EEPROM can be written).

The only  PULL-up resistor is the R1065 which works perfectly wether DMM check then later probing SDA (Data). This implies on a side note that the SCL (Clock) from the ASIC does not have any pull-up or pull-down resistor in the TDS540C design.

I did not have time to record, trig the SDA and SCL because I consider the following visual tests with my P6136 and my tek 2465 to be enough proving there is either (1) an ASIC problem (internal or external control from the 68040 or who ever communicates with ADG308 to dump the EEPROM content) or (2) an EEPROM problem.

Tests on good TDS540C:

At start the SCL is HIGH (+5V) then after a while there is clearly a clock modulation (between 0v and 5V) then stays LOW (0V) for few seconds. After a while again clock modulation and finally always stays LOW.

At start the SDA is HIGH then after a while data modulation (much lower freq than SCL clock modulation) then stays HIGH for few seconds. After a while again clock modulation and finally stays HIGH.

It seems like the OS of the TDS will attempt to read again the EEPROM but I could be wrong in this conclusion.

Tests on bad TDS540C:

At start the SCL is HIGH then always stays LOW.

At start the SDA is HIGH then after a while short glitches between 0V and 5V then always stays HIGH.

There is no 2nd attempt to read the EEPROM like in the heathy TDS above test.

Question 1:

Does anyone know if both EEPROM are removed then does the IDG308 would still generate the SCL (Clock) and SDA content attempting to read

Question 2:

What electrical test could be done on the EEPROM still soldered in my Acq-board which could indicate some internal short-circuit or something else implying the SCL output from ASIC to be voided or neutralized or inoperative

Question 3:

What is the role of ADG308-108 pin input which is referred as IC2_DIAG and connected to the output of the U1721 who seems to keep track of Protect switch S1002 status (write protect disable or enable)

In case you want to know whether your CPU board is good or not, I'm repeating myself here, swap it with the CPU board in the fully working unit. But just the CPU board, as the fault could also be located in the cables, the connectors, some power supply line, whatever, you just don't know. I'm not sure I understand why you keep on insisting to find an explanation for the wrong GetCalData GPIB read, it's irrelevant at this point. It could be a side effect of something you're overlooking at this point. If the CPU board fully works inside the other good unit, then it's good. If not, then it's bad. Pretty straightforward. When testing this, don't forget to load the NVRAM of the "good" unit into the CPU board you're testing/suspecting. Both boards should have the same firmware, same dip switch settings, same NVRAM content ... i.e. they should be identical as possible. Only then you can draw conclusions and isolate the fault.

I do not wish to debate on what you have been suggesting here and before because what you describe could create more problems in my opinion. Please do not be offended, I'm not saying you right or wrong but I've repaired much more complex electronic and digital system like Airbus avionics. The method of troubleshooting I prefer to follow: modular test engineering versus component test engineering. The TDS scope can be seen simply as few different main sub-systems, of course inter-connected by busses, cables which dialog via different chips, uC, ASIC... technically speaking, the processor-board of the bad TDS is partially working which is why once connected with health Acq-board then the Scope self-test OK. The only strange thing which i've already mentioned, for some reason the GPIB-GetCalData dumps ALL zeros so maybe the GPIB is partially failed whereas your community-contribution (Floppy-dump and Floppy-Write) set provides the good content of the EEPROM.

Some of the test house companies I know in France when they have a failure and send to repair to tektronix or their OEM repair center, tektronix company changes the boards and not spend to much time as we're all doing here. Plus they charge huge money so their customers would prefer buy new scope to feed electronic obsolescence business model but that is another philosophical topic 

Anyway I prefer to go step by step, after all I'm on my learning curve with this TDSxxx serie C totally new for myself. My today earlier test clearly show there is an issue between ADG308 and the EEPROMs but not necessarily EEPROM itself. Wether the processor boards are good or bad does not change anything, when I did install the bad Acq-board into good TDS o rinto the bad TDS, the self-test Fail ++Processor.

I really want to thank you again to have developed the Floppy-dump and Floppy-write along with your fundamental explanation of checksum failure EEPROM read implies a local RAM copy of CalData from the Firmware. From these different Floppy-dump and GPIB-GetCalData test done on both swap, I did converge to learn then now focus on the EEPROM and ADG308 topic which is clearly failure number one in this oscilloscope.

Later i'll see how to use the possibility to RS232 Console Port the J40 connector which might provide much more refined error log than the synthetic error log provided as standard.

Warm regards, Albert

[flyte]

Does anyone know if both EEPROM are removed then does the IDG308 would still generate the SCL (Clock) and SDA content attempting to read

Sure, why not, as long as the pullup on SDA is there and as you said yourself the SCL must be driven with levels straight from the ASIC. Good to see you're closing in on the idea of EEPROM removal, eventually!

What electrical test could be done on the EEPROM still soldered in my Acq-board which could indicate some internal short-circuit or something else implying the SCL output from ASIC to be voided or neutralized or inoperative

A conductivity test using a multimeter. But that's only gonna show a really bad short circuit, not some unexpected pull down in the circuit. And you still won't know whether look left or right, i.e. ASIC or EEPROM, unless you remove one of them. Odd we're always coming back to that removal suggestion.

What is the role of ADG308-108 pin input which is referred as IC2_DIAG and connected to the output of the U1721 who seems to keep track of Protect switch S1002 status (write protect disable or enable)

No idea, I didn't design it. Perhaps to detect whether you flipped the switch or not, or some additional write safety mechanism fed into the ASIC.

I do not wish to debate on what you have been suggesting here and before because what you describe could create more problems in my opinion. Please do not be offended, I'm not saying you right or wrong but I've repaired much more complex electronic and digital system like Airbus avionics. The method of troubleshooting I prefer to follow: modular test engineering versus component test engineering.

You questioned that CPU board in the first place. So I offered you a very modular testing solution to it: swap it with the CPU in the good unit. Now if that CPU board is suddenly no longer important to check or would create more problems, so be it. If you've repaired more complex systems than this, then you should know how to proceed and I'd assume, given the references you've just made, you know how to remove those EEPROMs in perfect shape. Not to speak personal here let alone how many academic degrees we all have, but bear in mind many people on this board are actually software and/or hardware design engineers. So most of us do know a couple of things about complex matters and systems as such.

My today earlier test clearly show there is an issue between ADG308 and the EEPROMs but not necessarily EEPROM itself.

It does point into that direction and it's very likely indeed, as nearly everyone on this thread has suggested. But clearly show it? Nope. It may very well be there is some communication (bus, cable, whichever other component sits in between or controls it) issue happening before the ASIC which manifests itself in bad or unexpected signalling at the I2C bus.

And, practically speaking, how would you then plan fixing that, other than by removal of the cheapest and easiest component? I'd pick the 50 cent EEPROM to start the removal with and put my suspicion on, not that 200-pin ASIC. But hey, the choice is yours!

If for example that scope is still able to trigger with that bad board, than chances the problem is with the ASIC or before it lower drastically. But I'm not sure it actually can with the bad cal values of course.

Besides, in case you remove the EEPROM and it would indeed be dead, then you had no choice but to replace it. In case it's still working, you should hurry up and get the calibration data out asap. I'm missing any drawbacks there.

I really want to thank you again to have developed the Floppy-dump and Floppy-write

You're very welcome and I'm happy it's useful to you, but in the first place I released the write tool hoping it may serve the community.

[Tantratron]

My today earlier test clearly show there is an issue between ADG308 and the EEPROMs but not necessarily EEPROM itself.

It does point into that direction and it's very likely indeed, as nearly everyone on this thread has suggested. But clearly show it? Nope. It may very well be there is some communication (bus, cable, whichever other component sits in between or controls it) issue happening before the ASIC which manifests itself in bad or unexpected signalling at the I2C bus.

And, practically speaking, how would you then plan fixing that, other than by removal of the cheapest and easiest component? I'd pick the 50 cent EEPROM to start the removal with and put my suspicion on, not that 200-pin ASIC. But hey, the choice is yours!

If for example that scope is still able to trigger with that bad board, than chances the problem is with the ASIC or before it lower drastically. But I'm not sure it actually can with the bad cal values of course.

Tonight I've powered the scope, then after some 30 min warm up time I did run the SPC even though it has the Fail ++Processor. The SPC passes except as reported few weeks ago, the Voltage reference, frequency response and Pulse Trigger stay initialized.

Then I've connected on each channel one P6139A probe which senses the Probe calibrator 1KHz signal output.: each channel will properly trigger.

[Tantratron]

This morning decided to solder some wires into the SDC (CLK green wire) and SDA (DATA red wire) between ADG308 asic and both EEPROMs of the failed Acq-board then partially re-installed the all unit (the black wire is GND)

Attached some pictures of triggered results using the healthy TDS540C with two P6139A probes, please see time stamping where Ch1 (top view) is the CLK whereas Ch2 is the DATA. I did trigger basic but that might not be the best choice or combination.

However we see at time 11:30:45 this is first recorded event after boot (the clock rings a few cycles and I guess some data is sent to request reading). MUCH later at time 11:31:52 there is only one CLK pulse coming after DATA huge pulse...

[SaabFAN]

You might have an issue with probing the i2c-Bus, which gives you false readings: You're measuring a databus that runs at a minimum of 100kHz with a sample-rate of 100kS/s.
Try to use at least 1 Megasample/second. Especially since there's a possibility that the i2c-bus can switch to Fast Mode, which would mean 400kHz Clock-Frequency.

[Tantratron]

You might have an issue with probing the i2c-Bus, which gives you false readings: You're measuring a databus that runs at a minimum of 100kHz with a sample-rate of 100kS/s.
Try to use at least 1 Megasample/second. Especially since there's a possibility that the i2c-bus can switch to Fast Mode, which would mean 400kHz Clock-Frequency.

Thank you @ SaabFAN for the suggestion were in fact both TDS540C do have 1M options so I've lot of margin to trigger, store then zoom. Attached a new updates of the I2C probing sampled at 2.5 MS/s where as a reminder there are two moments or type of events once the scope is booted.

[SaabFAN]

Without reading all the previous posts, I'd hazard a guess and say that there's a chip that does "clock stretching" (a technique by i2c-Slaves to slow down the master in case they are still busy working on the previously received data).
But in this case the clock-stretching goes on forever until the chip times out.
I2C-Busses should always float high when there is no activity.

Since those chips are SMD-Components (I read you were reluctant to desolder them), I'd say there's only a small risk of damaging them or the board with hot air and then testing them with an Arduino. If you're worried about accidentally erasing them, just tie the Write Protect-Pin to the correct level to prevent any write operation.
With the chips desoldered you can also see if there's another chip pulling down the bus. And you can make a dump of the data. If the EEPROMs are damaged and not working correctly, you might be able to read them by increasing the voltage a bit until you finally get the data. That method worked for me with an older parallel EPROM that had started to exhibit bit-errors. With the voltage cranked up to 6,5V I was able to reliably read the chip and re-writing it. After that the device with that chip in it didn't crash any more.
I still replaced the chip though. Just to be sure.
Just make sure you remember which chip goes where

Edit: Just had a more closer look at the pictures and I noticed something.
The Clock-Signal has varying periods. Either a chip is doing clock-stretching on a period-by-period basis OR you have a bad main oscillator / PLL somewhere.
Have you checked other clock-signals for similar behaviour?

[ragge]

It is hard to say since the timing of the individual edges are important, but I think I can read this:

This looks good and bad.

It seems there is an initial transition on CLK that has nothing to do with it; them
A start condition, the address + READ (1 0 1 0 0 0 0 1(R)), then it does one clock cycle to let the EEPROM acknowledge but it doesn't, then a stop condition.
Then five more retries of: start condition, addressing, no ack, stop condition.

So it seems like the ASIC and the I2C bus are OK, but the first EEPROM does not respond.

Maybe the second one still contains good data, and maybe that can be reused with the right knowledge of things.

This would mean that all the expensive parts are good, but the calibration data may be lost.

I would remove the EEPROMs and try to read them with something else, then install new EEPROMs.

Since others have managed to calibrate the scope without the calibration data from the EEPROMs, there is hope.
And thanks to Flyte, you can now program the new EEPROMs when in the scope, and even experiment with different calibration sets.

By the way - you can read about how the protocol works in e.g. almost any 24C02 data sheet, or most other I2C device data sheets, or from some general I2C documentation.

Ragnar

[ragge]

Without reading all the previous posts, I'd hazard a guess and say that there's a chip that does "clock stretching" (a technique by i2c-Slaves to slow down the master in case they are still busy working on the previously received data).
But in this case the clock-stretching goes on forever until the chip times out.
I2C-Busses should always float high when there is no activity.

(FYI: This is not really a normal I2C bus - there is no pullup on SCL, so it must be actively driven both up and down by the ASIC.)

I think the host just gives up and forgets to reset the SCL.

Edit: Just had a more closer look at the pictures and I noticed something.
The Clock-Signal has varying periods. Either a chip is doing clock-stretching on a period-by-period basis OR you have a bad main oscillator / PLL somewhere.
Have you checked other clock-signals for similar behaviour?

Actually it is quite tight except for the start condition, stop condition and the ack cycle, I think it looks entirely normal.
But it is quite common for software driven I2C buses to have varying clock cycle lengths, and it should work anyway.

[Tantratron]

Since those chips are SMD-Components (I read you were reluctant to desolder them), I'd say there's only a small risk of damaging them or the board with hot air and then testing them with an Arduino. If you're worried about accidentally erasing them, just tie the Write Protect-Pin to the correct level to prevent any write operation.
With the chips desoldered you can also see if there's another chip pulling down the bus. And you can make a dump of the data. If the EEPROMs are damaged and not working correctly, you might be able to read them by increasing the voltage a bit until you finally get the data. That method worked for me with an older parallel EPROM that had started to exhibit bit-errors. With the voltage cranked up to 6,5V I was able to reliably read the chip and re-writing it. After that the device with that chip in it didn't crash any more.
I still replaced the chip though. Just to be sure.
Just make sure you remember which chip goes where

Thanks a lot for teaching this tip of slight over-voltage to re-read the content of EEPROM.

I plan as next stage to desolder both EEPROM's then read them arduino style but first I want to make sure the ADG308 asic in charge of the I2C is not at fault since I have another strange effect with this oscilloscope.

When I did test the good TDS540C, there were clearly two attempts to read the EEPROMs as i've reported in this previous post https://www.eevblog.com/forum/repair/attempting-repair-of-tds540c-option-1g-fail-processor/msg2892782/#msg2892782

These two attempts are spaced at least 10-20s each during the boot, no idea if it is a double read check. However I wonder what could be the explanation in the failed TDS540C, see attached again both scopeshots corresponding to the 2nd major event between the ASIC and the EEPROMs. The bottom trace (Ch 2) is SDA, the top trace (Ch 1) the SCL so why it suddenly DATA decides to go LOW and the unique SCL pulse happens much after... wierd

If I had more time and not worried to make a false move, I'd deconstruct the good TDS540C then solder the I2C wires to see exactly what is the complete protocol when everything works OK.

Thank you, Albert
 

[shakalnokturn]

FWIW, I don't have that much experience with I²C other than checking logic levels, reading and sometimes rewriting/blanking corrupt EEPROM's out of the host system.

To backup SaabFAN:
Your waveform captures reminded me of one of my first experiences, (some 25 years ago) with I²C that must have been on a Philips TVC15 with AST (for those who remember...).
Probing with the CRO showed some startup activity, then the typical regular pulses you're seeing.
I remember it was repaired by replacing a 93C46 or 93C56... That taught me one of the basic checks I still use regularly when troubleshooting, in this order when applicable:

Power
Oscillators
Reset
Serial bus
Parallel buses

[Tantratron]

Edit: Just had a more closer look at the pictures and I noticed something.
The Clock-Signal has varying periods. Either a chip is doing clock-stretching on a period-by-period basis OR you have a bad main oscillator / PLL somewhere.
Have you checked other clock-signals for similar behaviour?

I've read bit this morning the I2C norm from https://i2c.info/i2c-bus-specification and http://www.i2c-bus.org/fileadmin/ftp/i2c_bus_specification_1995.pdf where unless i'm wrong, the SDA should always go LOW before the SCL in order to launch a START condition.

Attached new screenshot taken this morning from the failed TDS540C where I've now permuted the signals so upper channel is SDA and lower SCL to be similar to most figures found in the datasheets, documents on I2C.

Unless I'm wrong, the very first activity of the ASIC to communicate with EEPROM seems SCL while the SDA keeps HIGH which cannot validate a START condition. How does the EEPROM will sync or understand the first command from ASIC attempting to read it ?

As for your question Have you checked other clock-signals for similar behaviour, which ADG308-pin would check (see attached page 269 showing the complete circuit around this ASIC and the EEPROM) ?

On a side note, does anyone knows why tektronix decided to not have a pull-up resistor on the SCL line since most literature recommends two pull-ups, namely pull-up on SCL and pull-up on SDA ?

[madao]

Here a pictures with notice of start state.   I2C device  is waked, if  start state come. No start state =  All I²c devices stay inactive.

It choose  U1052 EEPROM with write commando (it means not EEPROM write !)

[madao]

EEPROM IS DEAD.  It doesn't make a ACK !

[ragge]

EEPROM IS DEAD.  It doesn't make a ACK !

You could see that (but not as well) on the earlier pictures, and I said so a few comments up.

Are my comments not visible?

[madao]

Honestly, i haven't read all, because: I haven't desire to read this theard, which we know since long time ago: EEPROM is problem and it isn't replaced. Many talk , less action.
Albert wrote me a mail (Quesition about I²C start state) and i answer it based their mail . What i see : no ACK from EEPROM.

Albert has now two same clear answer (from you and me) Now, he must replace  EEPROM.
Everthing else measure and action without replacing of EEPROM is waste of time. Only one solution: replace EEPROM.

greetings
matt

[ragge]

Honestly, i haven't read all, because: I haven't mood reading theard, which we know since long time ago: EEPROM is problem and it isn't replaced. Many talk , less action.
Albert wrote me a mail (Quesition about I²C start state) and i answer it based their mail . What i see : no ACK from EEPROM.

Albert has now two same clear answer (from you and me) Now, he must replace  EEPROM.
Everthing else measure and action without replacing of EEPROM is waste of time. Only one solution: replace EEPROM.

greetings
matt

Ok, good!