Attempting repair of TDS540C - Option 1G - Fail Processor

[Tantratron]

hello,

I'm still bit confused by the respective answer of Shakalnoktur, madao and Charlyd regarding how to handle the EEPROP story (reading, writing both 2X24C02) versus if it can explain my TSD540C Fail ++Processor (attached again).

Something is stil not clear because some say that we can write data from previous save before corruption of the EEPROM or from another valid TDS540C into a new fresh EEPROM. Some other say it will not work and only running a full calibration FAS onto fresh EEPROM will be valid.

Furthermore my understanding is that FAS will actually write at some point the data into the EEPROM so there should an option if we know the exact mapping address to actually write the EEPROM via the GPIB-USB way. We already know that getcaldata is able to dump or read the content of the EEPROM including CRC check via the GPIB-USB so why not writing as well.

I know that FAS target is to fine calibrate an oscilloscope in particular if some components got replaced but it was suggested before to try replacing the EEPROM with valid data to see if the Fail ++Processor goes away. Later it has said only running FAS into fresh (never written EEPROM) will solve the issue so i'm totally confused.

P.S.1. Since the FAS is run from a DOS computer to control the process via GPIB, maybe someone has designed C source file able to mimic the FAS so we could do from Windows, Linux and MacOS.

P.S.2. Maybe I'm wrong but it seems the souring of X24C02 is only from China or maybe obsolete so we could have similar issue as the sourcing of NVSRAM 1486 and 1650Y

Thank you, Albert

[charlyd]

Hi i had exactly the same error your 24c02 info is already corrupt    ( it says: NvlibrainDiags and libs with CRCC error.)
The storage 250 error is pointing to your broken 24CO2 chips. these are the onces i ordered  https://www.ebay.com/itm/143318363050

In the beginning i also tried to restore my old content which i saved with a programmer and also content from an other scope.. it all doesn t bring you anything... just solder the empty chips on your ACQ board power on the scope ..and you need a CVR-Cal.

            1.  COLD_START      (Initialization)
       2.  SPC                 Signal Path Compensation
       3.  CVR_CAL         Voltage Reference
       4.  POWER_CYCLE      (Reset)
            9.  PROBE_COMP_CAL   Voltage Reference

clear your error log and boot and see

the CPU fail should be gone. After that do the HF_cal and Interleave to finish your complete Calibration.

i hope this clears things up..

[ragge]

Hi i had exactly the same error your 24c02 info is already corrupt    ( it says: NvlibrainDiags and libs with CRCC error.)
The storage 250 error is pointing to your broken 24CO2 chips. these are the onces i ordered  https://www.ebay.com/itm/143318363050

In the beginning i also tried to restore my old content which i saved with a programmer and also content from an other scope.. it all doesn t bring you anything... just solder the empty chips on your ACQ board power on the scope ..and you need a CVR-Cal.

            1.  COLD_START      (Initialization)
       2.  SPC                 Signal Path Compensation
       3.  CVR_CAL         Voltage Reference
       4.  POWER_CYCLE      (Reset)
            9.  PROBE_COMP_CAL   Voltage Reference

clear your error log and boot and see

the CPU fail should be gone. After that do the HF_cal and Interleave to finish your complete Calibration.

i hope this clears things up..

May I ask I there was some specific reason for buying them on Ebay, when there are new ones available much cheaper from about every electronics shop (Farnell, Mouser, Digikey, Distrelec, ...)?

How do you know that the problem is with the EEPROMs, couldn't it just as well be a problem with the NVRAMs?

[charlyd]

of course you may buy them where you want. I just pointed to this link and not the chinese models ;-)

nope it is not the NVram only. he will see even after swapping both Dallas chips. the 250 storage error is NOT gone.

[ragge]

of course you may buy them where you want but. i just pointed to these and not the chinese models ;-)

nope it is not the NVram only. he will see even after swapping both Dallas chips. the 250 storage error is NOT gone.

Sure, I just wonder why anyone would you buy them from some random source on Ebay at all when you can get factory new ones through hopefully controlled channels, and cheaper?

To try make my question clearer:
How can you tell that the EEPROMs and not something else are the problem?

Regards,

Ragnar

[charlyd]

@ragge : i advise to read the above posts...about the storage 250 error. the cal info is stored in these chips.

[ragge]

@ragge : i advise to read the above posts...about the storage 250 error. the cal info in stored in these chips.

Hi Charlyd,

I have read the thread. It is still not clear to me how you can determine that the EEPROMs are broken and needs to be replaced.

After all, broken 24C02:s aren't very common, not random bit flips in them either.
I understand it as it is only written during FAS calibration, so it shouldn't have been corrupted while writing.
They are typically specified for 60-100 years, but some may be worse of course.

The 24C02:s can seemingly can be read so they don't seem entirely broken. The data can be verified by checksumming, but maybe there still is a problem with data that isn't checksummed.

- The error codes indicate NVRAM problems, and several have suggested that it can be in the other NVRAMs as well. Though, the board swap shows that the problem follows the acquisition board, so probably not.

- As others have suggested, it may be some other problem with the I2C bus, maybe some other device on the same bus, if there are any.

You may absolutely be right, I just wonder how you got to your conclusion that the EEPROMs must be replaced.


Regarding where to buy the EEPROMs - so there is no reason not to buy them through the normal electronics suppliers (Farnell, Arrow, Mouser, Digikey, Distrelec ...)?

[Tantratron]

Hello All, this morning I've decided to make some new dumping - reading test of each EEPROM. As a reminder, I have two TDS540C where one is healthy and other partially failed.

Now I've dumped the EEPROM content via two methods (1) FloppyDisk EXE application and (2) getCalData through GPIB. Normally if both applications are OK, the BIN files should be the same.

See attached the ZIP compress hosting the 4 dumpings: 2 dumps for TDS540C-2G (healthy) and 2 dumps for TDS540C-1G (failed).

For some mysterious reason, the 2 dumps of healthy TDS are exactly the same which means wether Floppy Disk utility to read EEPROM content or the GetCalData (C source file compiled) reading EEPROM via GPIB-USB interface, the 24C02 chips have the same content.

However the failed dumps show an extraordinary BIN file content difference... why no idea but now I wonder if the failure is not due to some bus linking the EEPROM to the Display Processor. What are the different protocols or addressing mode when reading the EEPROM via GPIB control versus reading via FloppyDisk

Thank you, Albert

[shakalnokturn]

Does the floppy dump utility check data as it writes it? After looking at your files I'd easily blame that on floppy data errors... Have you tried more than one write on different disks?

If I'm wrong about the floppy and you have more time to spend swapping things around, it would be nice to see the same 4 backups made once you have crossed the acquisition boards.

[ragge]

Does the floppy dump utility check data as it writes it? After looking at your files I'd easily blame that on floppy data errors... Have you tried more than one write on different disks?

I think I can answer this:
No, it doesn't check the write. The script is this:
https://github.com/ragges/tektools/blob/master/tdsNvramFloppyTool/tdsAcqEEPROMFloppyDumper/acqedump.app
(or this, which also dumps the NVRAM at the same time: https://github.com/ragges/tektools/blob/master/tdsNvramFloppyTool/tdsNvramEepromFloppyDumper/dumpall.app)

It can absolutely be a floppy problem - I too would recommend trying again, preferable even with another floppy, and see if you get the same result or not.

The 1G file from "getcaldata" has much more repeated values than the 2G file.
If the firmware can't read the EEPROM, will it make up default values?

getcaldata reads two bytes at a time using the command "WORDCONSTANT:ATOFFSET? 262144,N", where N is 0 ... 251.

Would it maybe be possible to write the EEPROM using some other "WORDCONSTANT:" operator?

Edit: Partly answering my own question:
There is a "WORDCONSTANT:ATOFFSET A,N,X" command that one could guess would set data.
There also is "CALIBRATE:STORE?" that may or may not be interesting.
Does anyone know if these could be used to store data to the EEPROM, and if so how, and if "262144,N" where N is 0 ... 251 is the right way of accessing the data, or if there are other values/addresses/indexes/whatever-they-are that should be used?

Ragnar

[Tantratron]

Does the floppy dump utility check data as it writes it? After looking at your files I'd easily blame that on floppy data errors... Have you tried more than one write on different disks?

I think I can answer this:
No, it doesn't check the write. The script is this:
https://github.com/ragges/tektools/blob/master/tdsNvramFloppyTool/tdsAcqEEPROMFloppyDumper/acqedump.app
(or this, which also dumps the NVRAM at the same time: https://github.com/ragges/tektools/blob/master/tdsNvramFloppyTool/tdsNvramEepromFloppyDumper/dumpall.app)

It can absolutely be a floppy problem - I too would recommend trying again, preferable even with another floppy, and see if you get the same result or not.

The 1G file from "getcaldata" has much more repeated values than the 2G file.
If the firmware can't read the EEPROM, will it make up default values?

Hello ShakalNokturn and Ragnar,

Please find attached a new run done this morning on the failed TDS540C, here i've dumped both EEPROM and NVRAMs via 2 methods (floppy disk driver and GPIB).

Kindly note that I only have one floppy disk (old tektronix 1.44Meg disk which came when I purchased on eBay the healthy TDS540C). Maybe you're right that my floppy disk driver is failed or it would be good to try another floppy disk... no idea but if you display all the BIN files attached, you'll notice that during the All-Dump using my floppy disk and floppy driver the NVRAM content are exactly the same wether done by GPIB or Floppy. Since the Floppy dump method did extract both EEPROM and NVRAM (check screen picture) I conclude that maybe the floppy are OK.

If you have some spare floppy disk maybe you could send me one to my physical address in France. However as mentioned before when I did dump both EEPROM and NVRAM on my healthy TDS540C, all files did match when compared to GPIB dumping.

If for some reason we conclude the floppy disk and floppy driver are OK on both my TDS540C then it brings a serious technical topic to understand why there is failure when dumping EEPROM via Floppy whereas if dumping EEPROM via GPIB it works including correct checksum. This would imply that there are different internal addressing mode (read or write) the EEPROM from the Processor board or Acquisition board. Bad luck would be that when self-test on boot, the TDS540C does attempt reading the EEPROM and fails as does the Floppy. So maybe the EEPROM is valid, no need to solder new ones but this poses the troubleshooting method to apply.

If I'm wrong about the floppy and you have more time to spend swapping things around, it would be nice to see the same 4 backups made once you have crossed the acquisition boards.

If there is serious interest of some members, I might spend some extra time later this week or week-end to actually swap both acquisition board then specifically run the healthy TDS540C connected to the suspect un-healthy Acquisition board. I'd of course re-dump all files via Floppy and GPIB to see if the EEPROM content is visible and not corrupted.

Albert

[ragge]

Please find attached a new run done this morning on the failed TDS540C, here i've dumped both EEPROM and NVRAMs via 2 methods (floppy disk driver and GPIB).

This time the floppy dump only contain zeroes, while the GPIB dump is the same as the previous one.

To hexdump the files, you can for example use:
hexdump file.bin
od -Ax -tx1 file.bin # the flags make the output be all in hex bytewise
To compare files you can use for example:
diff -uw file1.bin file2.bin
cmp file1.bin file2.bin

Kindly note that I only have one floppy disk (old tektronix 1.44Meg disk which came when I purchased on eBay the healthy TDS540C). Maybe you're right that my floppy disk driver is failed or it would be good to try another floppy disk... no idea but if you display all the BIN files attached, you'll notice that during the All-Dump using my floppy disk and floppy driver the NVRAM content are exactly the same wether done by GPIB or Floppy. Since the Floppy dump method did extract both EEPROM and NVRAM (check screen picture) I conclude that maybe the floppy are OK.

If you get an (undetected) floppy error, it could potentially be on just a single bit, but typically on more.

If you had got the same result with both methods, it would be a good indication that the methods worked, but now you didn't, and then it is a good idea to try to determine where the error got injected.

If you have some spare floppy disk maybe you could send me one to my physical address in France. However as mentioned before when I did dump both EEPROM and NVRAM on my healthy TDS540C, all files did match when compared to GPIB dumping.

If for some reason we conclude the floppy disk and floppy driver are OK on both my TDS540C then it brings a serious technical topic to understand why there is failure when dumping EEPROM via Floppy whereas if dumping EEPROM via GPIB it works including correct checksum. This would imply that there are different internal addressing mode (read or write) the EEPROM from the Processor board or Acquisition board. Bad luck would be that when self-test on boot, the TDS540C does attempt reading the EEPROM and fails as does the Floppy. So maybe the EEPROM is valid, no need to solder new ones but this poses the troubleshooting method to apply.

Note that you can get a floppy data error on a otherwise totally working floppy drive and floppy disk - that they have worked before is no guarantee.

Yes, the two program use different methods of accessing the data.
Looking at them, it to me seems likely, but I don't know, that the getcaldata is reading from the copy in RAM, while tdsNvramFloppyTool reads directly from the EEPROMs.

It could be that if the scope can't read the EEPROMs when it boots, it will make up a default calibration data set and put in RAM.

Since you got different results, I would try it even more, or maybe make your own scripts that dumps it five times two five different files in one boot.

Based on what we currently know, there seems to be a problem with reading the EEPROM. It could be broken EEPROM(s), which others say they have had, or some other problem.

You could try replacing the EEPROMs (10-60 cent each for new ones), but you would have to program them too.

Since the old calibration data seemingly can't be reached anymore, my suggestion would to fill them with the data that you currently can read out using GPIB.
But then you haven't really gained anything more than removing the error message.

Another option would be to replace them with empty ones and find someone with all the equipment to run a full calibration using FAS - maybe you have it at the lab.

If you take out the old ones and succeed to read them with something else, the problem probably isn't with the EEPROMs themselves. You then could try new ones with the data from the old ones, and if it still doesn't work, there must be some other problem on the I2C bus.

Or, if it actually is the case that it is now running with a default calibration (which we don't know but seems likely), you could settle with that and ignore the error message. One problem with that is that it may not be as obvious if more errors develop.

Ragnar

[Tantratron]

N.B. Last night I did different EEPROM dumps with Floppy on 2G healthy and 1G failed TDS540's, see attached where the 2G always generates the same EEPROM binary file. However on the 1G, there are different failed dump which can be attributed either to failed Floppy driver installed on 1G or something else on the acquisition board. Since the floppy disk is the same wether I read 2G scope or 1G scope, I'd tend to conclude the disk is OK.

On a side note, the DOS application ACQUEDUMP in charge of EEPROM dump seems to use internal GPIB link, see

Code: [Select]

nulldev = open("/null",3,0666)
taskSpawn "Redirect",1,0,0x4000,ioGlobalStdSet,1,nulldev
taskDelay (600)
GpibInput("WAITICON OPEN")

GpibInput("mess:box 80,80,450,200")
GpibInput("mess:show \"\n\n\n\n ACQ EEPROM dump to floppy started, please wait.\"")

acqEEPROMSize=0x200
tempBuf=malloc(acqEEPROMSize)
_gtlX24c02Bcopy(0x10A00000,tempBuf,acqEEPROMSize)
ls "fd0:/"
fd=open("fd0:/acqeeprm.bin",0x0202,0777)
bytesWritten=write(fd,tempBuf,acqEEPROMSize)
close(fd)

confmsg=malloc(500)
sprintf(confmsg, "mess:show \"\n\n\n\n Dump completed, power off instrument.\n\n")
sprintf(confmsg+strlen(confmsg), " Bytes requested: %d \n Bytes written to disk: %d \n\"", acqEEPROMSize, bytesWritten)

GpibInput(confmsg)
GpibInput("WAITICON CLOSE")
GpibInput("bel")


This code calls _gtlX24c02Bcopy subroutine which can be precisely found inside the firmware when doing a TEXT search

However the getcaldata source code does use another method from the external GPIB interface, see

Code: [Select]

/*
 *
 * CAL EEPROM backup - for TDS520B,TDS540B,TDS724A,TDS744A,TDS754A,TDS782A,TDS784A
 *
 * compiled with MinGW gcc on Windows Vista / Win7
 *  tested with Agilent I/O 16.x and S82357 from [url]http://bmjd.biz/index.php[/url]
 * 
 *
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>

#if defined(WIN32) || defined(_WIN32) || defined(__WIN32__) || defined(__NT__)
  #include <conio.h>
  #include <windows.h>
  /* for NI adapters uncomment following line */

  /* for Agilent adapters uncomment following line */
  #include "ni488.h"
#elif defined(__linux__)
  /* linux with linux-gpib */
  #include <gpib/ib.h>
#elif defined(__APPLE__)
  /* MacOS with NI GPIB drivers */
  #include <ni4882.h>
#else
        #error "Unknown compiler environment!"
#endif


#if !defined (PROG_NAME)
  #define PROG_NAME __FILE__
#endif
#if !defined (GIT_VERSION)
  #define GIT_VERSION "unknown"
#endif
#if !defined (BUILD_TIME)
  #define BUILD_TIME __DATE__ " " __TIME__
#endif



#define ARRAYSIZE 100            // Size of read buffer

int  Dev;                        // Device handle
char ReadBuffer[ARRAYSIZE + 1];  // Read data buffer
char ErrorMnemonic[21][5] = {"EDVR", "ECIC", "ENOL", "EADR", "EARG",
                             "ESAC", "EABO", "ENEB", "EDMA", "",
                             "EOIP", "ECAP", "EFSO", "", "EBUS",
                             "ESTB", "ESRQ", "", "", "", "ETAB"};


void GPIBCleanup(int Dev, char* ErrorMsg);
static int debug;


static char* ident = PROG_NAME "   Version: " GIT_VERSION "   Build time: " BUILD_TIME;
static void print_version(void)
{
fprintf(stderr, "# %s\n", ident);
}


int main(void)  {
/*
 * ========================================================================
 *
 * INITIALIZATION SECTION
 *
 * ========================================================================
 */

/*
 *  Assign a unique identifier to the device and store in the variable
 *  Dev.  If the ERR bit is set in ibsta, call GPIBCleanup with an
 *  error message. Otherwise, the device handle, Dev, is returned and
 *  is used in all subsequent calls to the device.
 */

#define BDINDEX               0     // Board Index
#define PRIMARY_ADDR_OF_DMM   1     // Primary address of device
#define NO_SECONDARY_ADDR     0     // Secondary address of device
#define TIMEOUT               T10s  // Timeout value = 10 seconds
#define EOTMODE               1     // Enable the END message
#define EOSMODE               0     // Disable the EOS mode

FILE *outfile;
int f;
#if 0
unsigned long addr;
// addr = 0x00040000L; /* CAL data base address on TDS5xxB,6xxA,7xxA*/
addr = 262144; /* CAL data base address on TDS5xxB,6xxA,7xxA*/
#endif

    print_version();

    Dev = ibdev (BDINDEX, PRIMARY_ADDR_OF_DMM, NO_SECONDARY_ADDR,
                TIMEOUT, EOTMODE, EOSMODE);
    if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to open device");
       return 1;
    }

/*
 *  Clear the internal or device functions of the device.  If the error
 *  bit ERR is set in ibsta, call GPIBCleanup with an error message.
 */

    ibclr (Dev);
    if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to clear device");
       return 1;
    }

/*
 * ========================================================================
 *
 *  MAIN BODY SECTION
 *
 *  In this application, the Main Body communicates with the instrument
 *  by writing a command to it and reading its response.
 *
 * ========================================================================
 */


        ibwrt (Dev, "*IDN?", 5L);
    if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to write to device");
       return 1;
    }

    ibrd (Dev, ReadBuffer, ARRAYSIZE);
    if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to read data from device");
       return 1;
    }

    ReadBuffer[ibcntl] = '\0';
    printf("DSO IDN:  %s\n", ReadBuffer);
   
    outfile = fopen("U1052.bin","wb");
    printf("dumping U1052.bin\n");
    printf("\nPlease wait ...\n");
   
   
/* the first 8 bytes of the U1052 EEPROM are not mapped and empty (0x00h)
   so we write as well 8 time 0x00h to dump file  */

for(f = 0; f < 8; f++)
{
fputc(0x00, outfile);
}


/* send first TEKTRONIX Password fr TDS5xxB/7xxA models to allow memory dump */
ibwrt (Dev, "PASSWORD PITBULL", 17L);
    if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to write to device");
       return 1;
    }

unsigned char calbuf[255];
int caldata;


for(f = 0; f < 124; f++)
{
        char cmdbuf[64] = "WORDCONSTANT:ATOFFSET? 262144,";
char cmdnr[16];
sprintf(cmdnr,"%d",f);
strcat(cmdbuf, cmdnr);

    if (debug > 1) printf("DSO IDN:  %s\n", cmdbuf);

ibwrt (Dev, cmdbuf, sizeof(cmdbuf));
     if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to write to device");
       return 1;
    }
ibrd(Dev, calbuf, 6);
     if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to write to device");
       return 1;
    }
   
     caldata = atoi((char *) calbuf);
    if (debug > 1) printf("CAL DATA: %X\n", caldata);
 
    unsigned char* abuffer  = (unsigned char *)&caldata;
    fputc(abuffer[1], outfile);
    fputc(abuffer[0], outfile);
}
fclose(outfile);

    outfile = fopen("U1055.bin","wb");
    printf("dumping U1055.bin\n");
    printf("\nPlease wait ...\n");

for(f = 0; f < 128; f++)
{
char cmdbuf[64] = "WORDCONSTANT:ATOFFSET? 262144,";
char cmdnr[16];
sprintf(cmdnr,"%d",f+124);
strcat(cmdbuf, cmdnr);

    if (debug > 1) printf("DSO IDN:  %s\n", cmdbuf);

ibwrt (Dev, cmdbuf, sizeof(cmdbuf));
     if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to write to device");
       return 1;
    }
ibrd(Dev, calbuf, 6);
     if (ibsta & ERR)
    {
       GPIBCleanup(Dev, "Unable to write to device");
       return 1;
    }
   
    caldata = atoi((char *) calbuf);
    if (debug > 1) printf("CAL DATA: %X\n", caldata);
 
    unsigned char* abuffer  = (unsigned char *)&caldata;
    fputc(abuffer[1], outfile);
    fputc(abuffer[0], outfile);
}
fclose(outfile);
   
    printf("\nCalibration data has been successful dumped\n");

/*
 * ========================================================================
 *
 * CLEANUP SECTION
 *
 * ========================================================================
 */

/*  Take the device offline. */

   
    ibonl (Dev, 0);

    return 0;

}


/*
 *  After each GPIB call, the application checks whether the call
 *  succeeded. If an NI-488.2 call fails, the GPIB driver sets the
 *  corresponding bit in the global status variable. If the call
 *  failed, this procedure prints an error message, takes the
 *  device offline and exits.
 */
void GPIBCleanup(int ud, char* ErrorMsg)
{
    printf("Error : %s\nibsta = 0x%x iberr = %d (%s)\n",
            ErrorMsg, ibsta, iberr, ErrorMnemonic[iberr]);
    if (ud != -1)
    {
       printf("Cleanup: Taking device offline\n");
       ibonl (ud, 0);
    }
}


So for sure there are two software and probably hardware method to READ the EEPROM where for some reason, one fails possibly explaining the FAIL of the oscilloscope

[Tantratron]

Based on what we currently know, there seems to be a problem with reading the EEPROM. It could be broken EEPROM(s), which others say they have had, or some other problem.

You could try replacing the EEPROMs (10-60 cent each for new ones), but you would have to program them too.

Since the old calibration data seemingly can't be reached anymore, my suggestion would to fill them with the data that you currently can read out using GPIB.
But then you haven't really gained anything more than removing the error message.

Another option would be to replace them with empty ones and find someone with all the equipment to run a full calibration using FAS - maybe you have it at the lab.

If you take out the old ones and succeed to read them with something else, the problem probably isn't with the EEPROMs themselves. You then could try new ones with the data from the old ones, and if it still doesn't work, there must be some other problem on the I2C bus.

Or, if it actually is the case that it is now running with a default calibration (which we don't know but seems likely), you could settle with that and ignore the error message. One problem with that is that it may not be as obvious if more errors develop.

Ragnar

Please remember that the time-labour to deconstruct, de-solder then re-solder 60 cents EEPROMs then time to run FAS is exponential. It is still not proven the EEPROM to be corrupted so yes new ones cost less than 1 euros but all the effort (solder, program, calibrate) is disproportionate. I do have some calibration tools in my lab like the PG506 necessary for voltage cal but not all test equipment where in the past I only repair, calibration 24xx tektronix oscilloscope.

My gut feeling suspects that GPIB management of EEPROM, firmware and NVRAM to be more stable and safe compared to using floppy disk. However the problem here is to understand the root cause of the Fail ++Processor where I'm still lucky to have a 2nd TDS540C which runs fine.

Albert

[ragge]

Please remember that the time-labour to deconstruct, de-solder then re-solder 60 cents EEPROMs then time to run FAS is exponential. It is still not proven the EEPROM to be corrupted so yes new ones cost less than 1 euros but all the effort (solder, program, calibrate) is disproportionate. I do have some calibration tools in my lab like the PG506 necessary for voltage cal but not all test equipment where in the past I only repair, calibration 24xx tektronix oscilloscope.

My gut feeling suspects that GPIB management of EEPROM, firmware and NVRAM to be more stable and safe compared to using floppy disk. However the problem here is to understand the root cause of the Fail ++Processor where I'm still lucky to have a 2nd TDS540C which runs fine.

Albert

I fully agree.

But if you care about the cost for the time, you should not do this at all - you can buy a working 540C on Ebay for Euro 500 or so, you have already spent much ore than that on this scope. I do this since it may help fixing more scopes, including my own, in the future, and because I like to understand how my gear actually works.

Note, again: That the floppy method has worked before is NO guarantee that it will work without errors again, so you could try it again, though I too suspect that is not the problem here since you have already tried it twice.

One other method could be: Hooking up on the I2C bus with some other I2C bus master device and try to read the EEPROMs that way. You need to have power to the EEPROM:s, so the easiest way would probably be to do it with the scope running and first making sure nothing else uses the I2C bus, since that would garble the data for both you and the other bus master.

You could also check the signals on the I2C bus with a scope and see if they at least look OK.

With my current knowledge of these things, I can't think don't know of anything more to try really.
So, as I wrote in my previous message, I would either.
- just live with the error message, convince yourself that it runs with default calibration, maybe compare performance to some other scope, and see of some better ideas show up
- try reading them in place with something else as suggested above
- take the old ones out and see if they for some strange reason can be read outside of the scope. Then there probably is some other problem, with the I2C bus or the power or something else.
- put new ones in either with the default data (and convince yourself that this is what you currently get with getcaldata), with the data you got from the ones you removed if there is any, with the data from the working one (which would be worse than a default set), or empty and run a full FAS.

Ragnar

[charlyd]

what does the errorlog say?  console port?  mostly the clue can be found there:

something like this:
[my old log part]
nvLibrariansDiag ............... ***FAIL***
..error details:
ERRORID: 163 diagnostic test failure nvLibrariansDiag
Libs with crcc failures:
        ExtConst

what do your cals say: VOLTAGE / FREQ / … show..  PASS or INITIALISE ?

[Tantratron]

This morning I decide to do the suggestion of Shakalnokturn, namely deconstruct both TDS540C's acquisition board then install the failed Acq board into the healthy 2G . So I've kept all subsystems of the healthy TDAS-540C-2G except installing the Acq board taken out from the failed TDS540C-1G.

Please find attached different files where I've GPIB cleared initially the LOG error file so it will show the first error after installing the Acquisition board. Unfortunately we can see that now the healthy TDS540C will indicate FAIL ++ Processor as well as same Error Log content and the SPC will get stuck on INITIALIZED... totally the same situation as before

I've also dumped via GetCalData the content of the EEPROM via GPIB which seems to be HEX same as when dumped from the failed TDS540C few days ago.

On a side note, the only single floppy disk which worked fine with the failed TDS540C cannot work anymore for some reason. The TDS540C will format it OK then I copy from my Mac the AcqDump DOS routine but will never boot in theTDS540C. As a reminder the very similar process successfully worked on the failed TDS540C. For the moment I cannot dump the EEPROM via the Floppy from my working TDS540C whereas few days ago it was working !!!

Anyway the point after installing the failed Acq Board into a healthy TDS540C shows same failure. In order to eventually help members or google search, I'll type in this post the exact Error Log content

Diagnostic Log
FAIL ++ Processor
pass -- Cal Initialization (see error log)

Error Log
FATAL: 250 nv storage too small more bytes requested than available
ERROR: diagnostic test failure nvLibrariansDiag, Libs with crcc failures: , ExtConst
FATAL: 250 nv storage too small more bytes requested than available
ERROR: diagnostic test failure extended cal librarian reset

It seems like the main FATAL issue is not enough bytes to read or write on 250 nv

Key question: who is 250 Non Volatile memory component ?

[charlyd]

the cals as shown in the last picture only gets solved with calibration. I had the same call s pictures posted here on the forum ..my SPC also worked fine. The Voltage you can do seperate to see if you cpu fail is gone.  i explained already above

but first you need to solve:  250 nv storage too small more bytes requested than available
and as i said above this diagnostic test failure nvLibrariansDiag, Libs with crcc failures: , ExtConst- point to the 24CO2 chips
While calibrating  info is written and stored in there and guess what they have CRCC failures.
Of course you can check the power rail and the I2C to the chips but they are on a weird place for debugging.

After i replaced them with new empty onces 250 nv storage too small more bytes requested than available was gone in the errorlog

what i don t understand is you swapped the ACQ boards out or what did you do...??  do you have the same boards? do both unit the same serial series..?

you swapped the dallas chips on the cpu boards already i assume.  1486 and 1650.

you dont have a console port ? to debug the scope.

this is all  the info i have for you.
did you see my post:  https://www.eevblog.com/forum/testgear/tds754d-fail-processor/

[flyte]

Hi all,

Forum member @ragge pointed me to this thread. I originally created those floppy backup scripts.

Here is the original post: https://www.eevblog.com/forum/testgear/tektronix-tds500600700-nvram-floppy-dump-tool/

I will attempt to end this misery 

I didn't have the time to read everything in detail, but one thing which a lot users still seem to ignore is that the dump tools also include a checksum verifier tool TDSNvrCV_2_0.zip written in Java which checks both the NVRAM and EEPROM dump files. It's there for reason! So you shouldn't guess whether the dump taken is garbage or not. So please use it. It computes the (way too) simple checksums Tektronix has put on the contents inside and verifies them. It attempts to make an educated guess regarding a (base) firmware level which the given dump is likely originating from or compatible with. There is one important thing to mention however: the tool may report valid checksums if the checksum result is 0x00 and the whole dump accidentally matches or has zeroes in it. I did not exclude this possibility from the tool, as 0x00 could really be a valid checksum result. So in case of checksum VALID and 0x00 checksum result, you have to look into the file and judge whether it could be a valid case or rather not because it's plenty of zeroes.

Comparing binary data may give an indication but especially for NVRAMs it's rather a waste of time as their contents continuously change during normal operation.

It's entirely possible some outputs of the floppy dump can occasionally be corrupt, especially using the dumper with the message box. Also, you will get corrupted dumps if you connect a GPIB or console interface to the scope. As pointed out in the original thread, you have to take the floppy dump with the scope minimally disturbed: no connections, no triggering, don't press buttons. It seems having to do with how the VxWorks OS executes the script vs. other tasks and interfering interrupts. I don't know more either, we have to accept it I guess. Older firmware seems more prone to these failures as well. The tdsMinimalXXX scripts seem to be the safest method in case of doubt, probably because there is virtually no interaction with the GUI.

The floppy script uses the function call _gtlX24c02Bcopy which directly accesses the EEPROM in hardware. The GPIB tool, so it seems from the code, is invoking library code which then takes values from the RAM location where the same data resides. At startup, the scope copies data from the EEPROM into RAM and then uses it. However, before the EEPROM is read, this section gets initialized with firmware default values. In case afterwards things somehow fail or get skipped at boot time, you will grab those default values using the GPIB tool.

In a similar fashion, if things fail and the function call _gtlX24c02Bcopy somehow aborts or doesn't exist in the given firmware, the RAM buffer the script temporarily allocates will contain garbage and will as such be written to disk.

That's for the general case.

Now, I will only take into consideration your first posting of dumps ACQEEPRM-Floppy.BIN and EEPROM-GPIB.bin. After that dumps seemed to be all over the place and I'm not sure you are looking at the root cause or rather side-effects of further attempts to diagnose the problem. The dump ACQEEPRM-Floppy.BIN has zeroes in it, the EEPROM-GPIB.bin dump is valid. However, inspecting that last dump and keeping in mind the above, it seems you grabbed the default values via GPIB. Beware, these aren't working defaults for the scope, it's more like a preformat the firmware can live with, ready to be set at factory production time. Your scope can't operate with them.

In summary, you have some hardware problem related to the EEPROMs which you should fix. I can't see anything wrong with both tools and there is an explanation for your observations. Could be a loose contact, could be a bad EEPROM, could be worse like a bad ASIC. Maybe attempt to read the EEPROM contents using a programmer, maybe you can still get something out of it. It's very important to try this wrt. the calibration data, most of which is set a factory time and never changes. If it's lost, it's lost. Your only chance is then to load another EEPROM dump into it and hope the scope calibration values more or less match.

So far I haven't discovered a way to write EEPROM data back into the EEPROM. I'm not sure it even exists in the firmware, but I suspect it does. It's unlikely I will invest time into this, as failed acquisition EEPROMs are not common and, once you have a good dump, they are easily restored by desoldering them and programming them externally. Another thing which is puzzling, is the fact there are two EEPROMs and one is always empty. Firmware can't even access the second one, so it seems. So this is either a backup EEPROM set by some unknown function, either at design time they estimated more space was needed and eventually never used it.
EDIT: sorry, I just realized the EEPROMs have always been 24c02's and not 24c04's, as I somehow thought they were in some models It's even in the function call's name, I must have been really distracted the day I remembered that wrong!    So data is simply split across them of course!

-- flyte

[flyte]

Ok, it happened quicker than expected, TDS scripts have been improved and can now also write the acquisition EEPROMs. Thanks to @ragge for his superb guess regarding the (ab)use of the function call !

Original post: https://www.eevblog.com/forum/testgear/tektronix-tds500600700-nvram-floppy-dump-tool/

Note to the topic starter: if you care about the calibration values of your scope, make sure you attempt to read them from the EEPROMs before trying to write them, whichever the effort required, including desoldering them and reading externally. You may loose values critical to your particular scope which perhaps even cannot be set back by the Tektronix service centers, as it may have been entered at the factory only. Loading a dump from a different device without having the original backup will make the scope "work" for sure, but it might never attain its former performance again. Note that it's possible you are facing other hardware problems which render EEPROM access impossible for some reason, and the EEPROMs/data could actually still be good.

[charlyd]

@Flyte:  i replaced them with new -never filled- empty onces and after that calibrated the scope works perfect.
Why did i replace them? Because my cpu error was generated somewhere in the past, which brought me to replacing them.  Maybe calibrate is just enough but the error is related to...
it took me a long time to fix the cpu error, nobody ever had this, not in yahoo groups, not on tek forum, but i made progression after playing arround with the content which jwalling gave me.
Of course assuming it is the same cpu fail ++  error i gave Tantratron the tip to read out with console port to get more info that he has now.
 

[flyte]

@Flyte:  i replaced them with new -never filled- empty onces and after that calibrated the scope works perfect.
Why did i replace them? Because my cpu error was generated somewhere in the past, which brought me to replacing them.  Maybe calibrate is just enough but the error is related to...
it took me a long time to fix the cpu error, nobody ever had this, not in yahoo groups, not on tek forum, but i made progression after playing arround with the content which jwalling gave me.
Of course assuming it is the same cpu fail ++  error i gave Tantratron the tip to read out with console port to get more info that he has now.

Okay, but you calibrated it afterwards, that's a really big difference compared to empty EEPROMs, lost values, plain defaults or a dump from a different unit. Most people won't have the capability to calibrate scopes, and even if you have, a calibration is only as good as your calibration equipment itself, which is likely not the same a calibration lab has, unless you're into the business of operating one of course. In addition, the price of a full calibration may rapidly exceed the value of a budget second hand scope.

It indeed may be the case the differences are small and perhaps also beyond your measuring capabilities or you're just lucky your scope is close to the default/average values, but I can confirm there are values contained in the EEPROM which are not altered by the calibration software. So they must have been set at factory production time and are for the life of the scope. I suspect it's more about parameters in relation to e.g. the specifics of maybe the ASIC batches, etc. Another fact is then one scope is not equal to another one. Calibration of a TDS794D or a TDS694C requires different internal specifics and precision than say a plain TDS520B or the likes. Your observation is certainly reassuring to read, but I wouldn't bet on it there is no difference at all with factory calibrated models which still have their original EEPROM cal data, and even less so it would be valid for any type of TDS5/6/7xx scope.

When the EEPROMs fail or do not contain calibration data, or corrupt data, the logged error is indeed CPU FAIL. It's just an error classification, CPU FAIL can mean lots of other things as well. The error log will tell you more about the specifics. In any case, if you see "ExtCal" errors, this means it has to do with the cal data in the EEPROMs, but only if it's the later series like -B/-C/-D types. If you have -A series without EEPROMs, "ExtCal" errors would point to an NVRAM failure as everything is stored in there.

[Tantratron]

Hello and sorry for not answering before some of you who are really helping me. A particular thanks to @madao @ragge @flyte and @charlyd where on a side note it EXCELLENT news that @flyte with @ragge have cracked some more stuff so we can Floppy-write the two EEPROMs on the acquisition board.

I've been thinking, analyzing, pondering on different suggestions wether from this thread or private PM with @madao @ragge and @charlyd along with new tests done last week and early this morning. As a reminder, I'm using the floppy-dump method of @flyte and the GPIB-USB getCalData method of @sven where for clarity, the complete set of applications are found inside the github repository of @ragge found here https://github.com/ragges/tektools

Unless i'm wrong, any heathy TDS540C should provide the same EEPROM content once dumped via Floppy method of @Flyte or dumped by GPIB-GetCalData of @sven. This was confirmed with my healthy TDS540C-2G prior the swapping of the Acquisition boards, let's say Acq-1G is the failed board and the Acq-2G is the healthy board.

Last january 23rd, I did swap each acquisition boards since I've two TDS540C's, one referred as TDS540C-2G is fully healthy and another referred as TDS540C-1G is partially failed in need of repair. From what I understand of detailed explanations from @flyte earlier in this thread, if for some reasons the CPU cannot read or confirm the checksum of EEPROM, it will use as default file inside the firmware for Calibration data. Once I've installed the Acq-1G inside the TDS540C-2G there is of course the Fail ++Processor, I cannot read the EEPROM from Floppy and the GetCalData-GPIB does dump a EEPROM view which is precisely the content inside the firmware (please see attached screenshot where I've found the local storage of default EEPROM inside the firmware)... so far so good. I also attach a view of both GetCalData dumps where the 2G is from the valid EEPROM, valid TDS540C before swapping.

However I feel we cannot conclude the EPPROM-1G to be the only failed component in the TDS540C-1G for different reasons after swapping and performing different tests.

Reason 1: we could have a situation where either the tektronix Hybrid ADG308 (U1050) in charge of the I2C communication between system and both EEPROM to be partially failed or a soldering issue.

Reason 2: besides the Acq-1G board issue, I start to believe that maybe the processor-1G board is also partially failed. For some mysterious reason once swapping the Acq-2G inside the TDS540C-1G, good news is complete PASS at start, the SPC pass all tests. BUT there is a BUT... the Floppy dump of EEPROM works BUT the GPIB-GetCalData only dumps a file full of ZEROS for the EEPROM.

Normally there should be no reason why the TDS540C-1G if healthy processor board to not be able to properly dump the EEPROM content of healthy Acq-2G via two methods: Floppy-dump and GPIB-dump. Furthermore what is quite strange or driving me crazy, I've dumped and checked the firmware zoning of the TDS540C-1G, it is the same as the TDS540C-2G. hence for some reason, the TDS540C-1G Processor board locally getCalData dumps a EEPROM content full of ZERO which would pass the checksum CRC check declaring the scope to system-pass.

Hoping it is clear because not easy to explain by mail but my feeling: it is confirmed the EEPROM-1G are bad but rather an issue of bus communication between the processor-board and the acquisition board.

I'll wait for your comments, suggestions before re-swapping the Acq-boards to recover my initial condition, namely a healthy TDS540C-2G and a failed TDS540C-1G.

On a side note, I'm trying to find an RS232-USB interface with drivers to connect the TDS540C to my Macintosh, idea being to run Terminal mode to check detailed test information as suggested by @charlyd. This way more info will manifest compared to the Error Log file being kind of synthetic. If some of you know good RS232-USB cable Option 13 with drivers compatible with MacOS, I'm interested to buy such. My understanding from other threads being that the RS232 should be mapped electrically to J40 (special connector needed) or J37 or J3 all found on processor board.

Thanks, Albert

[shakalnokturn]

While I respect your perseverence to try to get to the bottom of this with software only, not being that good at it myself, there are some cases where you can actually gain time by using a soldering iron and a probe.

A system such as the one you are trying to debug is both software and hardware, both are interlinked, it seems a shame to not at least do some basic hardware checks when you have the necessary tools at hand.

Keep it up, I'm actually learning a lot from this topic.

[flyte]

While I respect your perseverence to try to get to the bottom of this with software only, not being that good at it myself, there are some cases where you can actually gain time by using a soldering iron and a probe.

A system such as the one you are trying to debug is both software and hardware, both are interlinked, it seems a shame to not at least do some basic hardware checks when you have the necessary tools at hand.

Keep it up, I'm actually learning a lot from this topic.

Couldn't agree more. Don't get me wrong, but there is no longer a point here in drawing all sorts of conclusions from logical and functional swaps, as the parts you're swapping with could themselves be part of the defect. Your measurement tools are also biased for the same reason, as you measure via the same parts: the GPIB tools read via a shadow space, maybe intialized, maybe not, maybe subject to problems, the floppy read method is a direct hardware read but it still relies on a functional bus, the attached ASIC and correct CPU/RAM/OS operation, so it is biased too. You are attempting to diagnose by means of intersection and exclusion, but there are too many variables at play here.

If you were attempting to remotely diagnose a spacecraft, I would fully agree you must lay out all possible observations and only draw conclusions from them. But this isn't electronics in outer space, we're just considering two common 8-pin EEPROMs in front of you, the actual contents of which is still unknown at this point and perhaps masked by some other defects. Grab any soldering iron or heat gun and your preferred homebrew I2C reader, get them out, and you're done and you know for sure.

However, whether or not the CPU boards are good should be fairly easy to discover, as you have a fully good and working acquisition board you say. So both CPU boards should give same operational results and the same EEPROM reading results when loaded with the same firmware and same NVRAM contents, when attached to this good acquisition board, provided it is really good and fully functional of course and you didn't miss anything there. You have all tools to do this, if you didn't already do so. Before writing anything to the NVRAM, make backups and label them and label your boards.

EDIT: To draw valid conclusions, you should consider swapping only the CPU boards in the complete good unit, as even the connection cables and interconnection PCB may be the cause of a defect in the bad unit!

Once you confirm the above, there's only one conclusion left, and that is there is nothing else at play but a bad acquisition board. It could then be as simple as bad EEPROM contents which needs a rewrite, or more complex like a bus or ASIC problem.

The point however which you seem to be sort of missing here, is that there is a chance you could somehow ruin valuable calibration data inside those EEPROMs, by attempting to execute a write on them using potentially defective hardware. Perhaps the EEPROMs are bad, perhaps not and they contain valid cal data, perhaps they have been erased. You simply can't know using the diagnostic methods you are currently considering.