Series defect on agilent 167xx boards?

[Hamster]

Any of these would work - 16753A, 16754A, 16755A, 16756A

[John_ITIC]

I successfully transplanted a comparator from my burnt 16754A and I now have one more (almost) good board. The below failures still persist.

Per the attached image, I have three 16754As (four with the burnt board) that all fail the final 'comparatorCalTest' and 'dappAddrDataTest' tests.
'comparatorCalTest' is the same test that is made in the GUI 'calibration' tab and it fails 'RC compensation' and 'tap delay' in various combinations for the different boards.

The service manual self-test description says:

Comparator Calibrations Test.
The purpose of this test is to verify that each of the comparator one-time calibrations can successfully be performed. This verifies that all of the calibration circuitry and components are within the tolerance limits required for proper calibration. This test is executed only if all probes are detached.

Timing Zoom Memory Addr/Data Test.
This test verifies connectivity of components within the analysis chip. It verifies that the address, data, and clock lines of the timing zoom circuitry is correct.

I have one good 16754A board that passes these tests too. This is what I wanted to use DogP's extender boards for. I now have a way to do comparative measurements. But, I'm not sure where to start to look. I have a hunch that these remaining errors are not related to trace corrosion.

Thanks,
/John.

[MarkL]

I have several cards with the RC failure and have not been able to figure it out.  It's been a while since I looked at it, but as I recall, swapping comparators was not the answer.

Since the test says that the pods must be disconnected, perhaps start looking on the pod side of the comparator.  On the pod side, underneath is an OP184 and a mystery SOT23-5 for each comparator.  The SOT23-5 is marked AAAG, and by way of how it's connected, one guess is that it might be a MAX4516 analog SPST switch.  They're all ganged together, perhaps turned on when in test or calibration mode.

Anyway, some comparator to comparator comparisons on pod side components might turn up something interesting.  I don't think I got that far.  It can be frustrating trying to fix these cards and I had to put them aside to get other things done.

[John_ITIC]

I have done some more research on the "RC Comp" and "Tap Delay" failures.

The 16754A service manual is a bit fuzzy on the details:

"Comparators.
The comparators are differential input/differential output devices that interpret incoming data and clock signals as either high or low. A threshold voltage provided by an internal digital-to-analog-converter (DAC) is coupled to the negative side of the differential signal through a precision resistor. Alternatively, this voltage can be provided to the data channels by a user supplied threshold line in the probe cables. There are separate internal DAC driven thresholds for the data and clock in each pod.

In order to achieve performance, an extensive calibration is performed on each comparator when the board is manufactured and the results of this calibration are stored as Calibration Constants in non-volatile memory on the logic analyzer board. These constants are loaded into the comparators at power on."

The 16760A service manual is phrasing this differently:

"Comparators.
The comparators are differential input/differential output devices
that interpret the incoming data and clock signals as either high or low. Threshold
voltage, programmed by the user through the user interface, is set by a digital-to analog
converter (DAC) coupled to the negative side of the differential signal
through a precision resistor. There are separate DAC-driven threshold voltages
for the data and for the clock. In addition, the comparator contains a diode in
which the junction temperature is monitored to ensure the module is being
properly cooled.

Much of the performance optimization for the module is accomplished by the
comparators, including channel delay setting (EyeFinder), programming of input
resistance, and frequency compensation adjustment.


Module operation such as
state clock modes and configuration are also done by the comparators. A digitalto-
analog convertor (DAC) provides the module threshold voltage for single ended
operation. The voltage at the DAC outputs are buffered to prove sufficient
line drive. An analog switch is used to channel either the module threshold
voltage from the DAC or the threshold voltage input from the system under test
to the comparators."

So, it looks like comparators themselves handle the adjustments to the "Input R", "OS Null", RC Comp" and "Tap Delay". Could it be that the comparators are "aging" and that it makes the SW unable to bring them back into spec via the available soft adjustments?

I played around with the 'vp' debug GUI (started via './vp -debug 255' from the shell) and I was able to "fudge" the various settings until the "RC Comp" and "Tap Delay" calibration passed. See attached p579.

I am, however, not certain that such a "fudged" calibration will actually work correctly. Perhaps this just makes the test pass but the H/W may still be out of spec. I know that the 16754A service manual talks about timing zoom performance validation via external pulse generator. I will study this topic some more.

I also will make an attempt to replace the FPGA on my burnt board (the one that was connected via 80- lead flat cables). I found what I believe is a direct replacement on Ebay. It is one speed grade faster but that should not matter as it is usually okay to go from a slower device to a faster, but not the other way around.

https://www.ebay.com/itm/145127678615

If the FPGA swap works, then I will have to find a replacement comparator. I may take it from my other 16754A that has "bouncy waveforms" after FFFF to 0000 transitions as I'm at this point am not too hopeful of fixing that issue...

I will post a couple of IR images in the next post as the web site does not allow more images to be attached...

Thanks,
/John.

[John_ITIC]

I also attached a couple of images from my thermal imaging camera, where the FPGA can be seen is very hot after having been shorted out. I could not find any other parts on the board that were unusually hot when comparing with a good board. If the FPGA replacement works then I will have to put back the comparator that I borrowed for another board.

Step by step...

/John.

[John_ITIC]

My burnt 16754A came back from the dead after I replaced the FPGA. So, that is the solution if anyone makes the same mistake I did (connect via 80 pin flat cables).

I have noticed that the board-board interconnect connectors are also corroded on the bottom side. See attached images. After scraping off the green corrosion, I can see that the gold flashing is gone and a brown discolored surface is underneath. I suspect this will not make good contact so, in order to do a proper refurbishment, it would be nice to replace these connectors.

Does anyone happen to know the part number of these connector?

The flex connector cable that goes between the boards has HP/Agilent part number '16754-61601' (picture attached). The Part # 16754-60002 is the set of two.

Thanks,
/John.

[MarkL]

My burnt 16754A came back from the dead after I replaced the FPGA. So, that is the solution if anyone makes the same mistake I did (connect via 80 pin flat cables).

I have noticed that the board-board interconnect connectors are also corroded on the bottom side. See attached images. After scraping off the green corrosion, I can see that the gold flashing is gone and a brown discolored surface is underneath. I suspect this will not make good contact so, in order to do a proper refurbishment, it would be nice to replace these connectors.

Does anyone happen to know the part number of these connector?

The flex connector cable that goes between the boards has HP/Agilent part number '16754-61601' (picture attached). The Part # 16754-60002 is the set of two.

Thanks,
/John.

I believe it is 3M Pak 8 Plug Connectors and Pak 8 Socket Connectors (previously Robinson Nugent, acquired by 3M).  Board side here:

  https://www.3m.com/3M/en_US/p/d/b30000132/

The part number looks like it should be P08-100-SLxx-A-G, where the xx depends on packaging and vacuum pickup options (see datasheet).  It's not in the datasheet as a contact quantity option, but it does show up as an obsolete part.

I've used a fine SS brush to get most of the corrosion off, and sometimes had to pick off the remaining pieces with a needle.  But that unfortunately took a fair amount gold plating with it as you also experienced.  However, the connectors still seem to work ok.  Longevity may be an issue.

They can also be stolen from the top side of dead boards where they are usually corrosion-free.  The heating cycle needs to be carefully controlled as they are easy to deform (see datasheet for process rating).  I would use Chip Quik or other low melting point alloy to get them off.

[Hamster]

tarnix may help remove the corrosion as well.

[John_ITIC]

MarkL was kind enough to send me a few spare comparators for my "burnt" 16754A board. I soldered one in but that board still fails the adcTest (pv shows no response from the ADC [U54, bottom side, TLC2543C]). I put the scope on it and there is no activity from the host while running the adcTest. I also swapped U54 with another board and the errors stays with my "burnt" board. As a test, I also swapped the U144 (LVT16245A) as I figure this line driver might be involved in the host communication - still no improvement.

So, there is a good possibility something else got damaged on this board when I used those 80 lead flat cables. Perhaps some internal trace got burnt off? Hard to tell.

So, at this point, I have seven working 16754A's, one working 16756A and two bad 16754As. I will upgrade the 16754A's to 16756A's via removal of resistor B3 (image attached).

Of the seven working 16754A boards, most are failing the RC comp and tap delay tests (timing zoom). I will have to continue that research at a later time as I'm sort of stuck at that issue.

At least I can be sure that no further deterioration will happen due to rail corrosion so I can push further investigation into the future. And I have more boards than I need, really.

I now have multiple TDS5xx, TDS6xx and TDS7xx scopes I need to attend to - the usual recapping and saving of battery backed SRAM + upgrade to new SRAM modules. I'd better get to that before too late.

Thanks,
/John.

[FrodeM]

I just though this might be worthwhile to mention..

Got a 16534A today, with crusty rails. Fortunately, no matter how bad it looked, the card passed the GUI selftests and after removing the gunk it seems like all traces are indeed intact (although small discoloration on two and ). However, one thing I noticed is that there was a decent amount of corrosion on the underside bracket for the heatsink mounts.

Reading though various threads, there's some suggestions that the corrosion is caused by moisture collecting. Given my observation, and the nature of the corrosion, I would rather think that the cause here is fumes released by the rubbery tape as it decomposes, in combination with moistures in the air. That would also explain why the corrosion in some worst-case examples is able to penetrate vias and even cause corrosion across a significant area of the top side.

On that note, it would at some point be great to figure out what precisely is the cause of this corrosion. Eventually if it can be properly neutralized as a part of the repair process (given some claim it may reappear after some time, if just washed with IPA. I know battery-leak corrosion on old motherboards is often handled with vinegar, but the chemistry for that is pretty well known. I don't want to try anything like that here unless I know it works and is safe for the boards.

[dorkshoei]

I just though this might be worthwhile to mention..

Got a 16534A today, with crusty rails. Fortunately, no matter how bad it looked, the card passed the GUI selftests

You need to see if it will pass calibration using the T cable described in the manual.       I have one that passes self-test that totally fails the calibration procedure with a fatal error.

[MarkL]

I have a bunch of 16534A cards, and almost all of them had corrosion on the mounting bracket for the ADC.  I removed the bracket, removed the corrosion with a wire brush, and sealed the bracket with clear insulating varnish (Sprayon EL600).  And of course removed the plastic runners and their evil adhesive from the board.

It would be great to discover the actual chemical reaction that's happening so it can be neutralized properly.

And 100% agree with dorkshoei.  I've had self-test pass but calibration fail on a number of cards.  A couple of the boards had out of spec rail voltages from the local voltage regulators because of bad output voltage setting resistors.  Replacing the resistors fixed the rail voltages and allowed them to pass calibration.

[FrodeM]

I spoke too soon, after removing the rails it no longer pass selftest.

Today I went over with flux, a soldering iron and a microscope. Three of the thin lines that just looked discolorated at a spot ended up disintegrating, so I had to fix that. I also reflowed most components with visible corrosion on the solder pads. Two vias had corrosion into them, but one were connected to one planar layer and seems to make connection. For the other I found fresh metal before the in-board junction.

I will test it tomorrow, since I need to fix the power-supply of the analyzer again. Despite the grid voltage should be 230V according to the power company, during nighttime it can at times be as high as 240V. Either it's that that blows the regulators, or the PSU has an inherent design defect that triggers if you turn the analyzer off and then on again only a few minutes later.

[FrodeM]

I have a bunch of 16534A cards, and almost all of them had corrosion on the mounting bracket for the ADC.  I removed the bracket, removed the corrosion with a wire brush, and sealed the bracket with clear insulating varnish (Sprayon EL600).  And of course removed the plastic runners and their evil adhesive from the board.

It would be great to discover the actual chemical reaction that's happening so it can be neutralized properly.

Ok, put in the spare PSU. Good news is that the card passes self-test again after the repairs. Crossing my fingers for the calibration.

*Edit*
I reset to default and ran the calibration again after the screenshot, and it seems to pass fine, yay!

[darkroom]

How are you guys removing the evil rails/glue from the bottom of the boards?  One site I read said IPA, but I am curious if heat will work?  Thanks!

[MarkL]

How are you guys removing the evil rails/glue from the bottom of the boards?  One site I read said IPA, but I am curious if heat will work?  Thanks!

Yes on the heat.  My method, from earlier in the thread:

  https://www.eevblog.com/forum/repair/series-defect-on-agilent-167xx-boards/msg3536942/#msg3536942

Keith has a detailed write-up which is pretty much the same:

  https://www.techtravels.org/2019/12/corrosion-near-underneath-hp-logic-analyzer-module-plastic-runners/

[aeg]

I use heat and dental floss to separate the rails from the foam tape, then heat and plastic cut from an old credit card to separate the foam tape from the PCB, then alcohol and a Q-tip to remove the remaining residue. For mushy foam, I found that scraping/pushing from the side was more effective than trying to peel it away in the long direction. Your mileage may vary.

[FrodeM]

How are you guys removing the evil rails/glue from the bottom of the boards?  One site I read said IPA, but I am curious if heat will work?  Thanks!

I usually use the edge of the removed plastic-rail runner itself to just scrape whatever residue is left off with a little bit of elbow-grease. That works fairly well, and the plastic is soft enough to not damage the silk-screen and solder-mask layers - even if you put a fair bit of force behind it.

As mentioned in the other techniques, the edge will get a little dull quickly. However that's not an issue for me since I don't tend to re-attach the runners.

[darkroom]

Thank you for the replies!

I removed the rails on my 16717A boards with heat with ease.  They literally peeled off nicely.  Unfortunately (or fortunately I guess), no corrosion underneath...so that's not the issue with those boards.

I removed the rails on my 16754A board (up-resistor'd to a 16756A).  The rails came off fine, but the "tape" was hard and crusty and took a LOT of work to get off.  GooGone seemed to be the best to soften it up, then scrape it off with one of the 16717a wider rails beveled ends worked fine.  Unfortunately, no corrosion under those rails either.

Both boards do have a few vias corroded, so I'll buzz those out.

The documentation for the 16754A says that in Timing mode it supports all channels, so I expected all four pods could be configured for timing mode.  But only A & B can be configured for Timing mode.  C & D has that option grey'd out.  Anyone able to get all 64/68 channels working on a 1675xA board in Timing mode?  Or is the documentation just wrong?

The document I am referring to is Agilent-16754A-Specifications-53790.pdf.  It says:

"4 GHz “timing zoom” (simultaneous timing analysis on all channels) with 64 K memory depth"

And in the service manual:

"                                                        Full-Channel Mode Half-Channel Mode
Maximum Conventional Timing Rate               600 MHz             1.2 GHz
Sample Rate, Timing Zoom                             4 GHz                 4 GHz
Channel Count per Card                                   68                      34"

I have 4 16754A boards on their way, and I will check them before modifying them.

[darkroom]

I figured it out!  In Pod Assignments, it showed two Analyzers.  Analyzer 1 had pods A & B, and Analyzer 2 had pods C & D.  There was no obvious way, to me at least, to get C & D to Analyzer A,  nor did I guess I needed to.  So, for some strange reason, I decided to try dragging C & D into Analyzer 1....and it worked!  I now have all 64 channels available for timing!  The only limit I see now is 32 channels per label.  But I don't think I will run into that issue very often.

[MarkL]

There's also a bunch of inter-dependencies between storage depths, number of active pods, and capture rates to be aware of.  You can read about it in "Channel Count/Memory Depth Tradeoffs" and "Choosing the Sampling Mode" in the Help.

From the Format tab: Help --> On This Window

Dragging around pod groups for assignment is also covered.

[darkroom]

There appear to be two versions of the 16754A.  One has no "pulls" on the pod ends, and has rework on it (blue wires, and a big cap).  The other has the pulls on the pod ends, and no rework.