HDG2002B AWG Firmware Reverse Engineering

[kazam]

I see. Well, it looks like a viable option once the open firmware gets underway but that seems to be a bit longer then. Rusty case? Hm.

Good luck with the effort! I will follow your progress.

[markone]

Rusty case? Hm.

Yep, internal shielding chassis edges are rusty, because they cut the metal after the galvanization treatment.

[alex.forencich]

Alright, since I have not posted anything here in a while, here are some updates.

I have been doing quite a bit of work on the FPGA architecture to create a very powerful synthesizer platform.  Just yesterday I tested a very simple DDS that played back a 1024 point sinewave from an FPGA block RAM at 250 MSa/sec with the phase accumulator set up for a 1 MHz output frequency.  I only checked the direct DAC output as I do not have control over the front-end relays yet, so the output had quite a bit of 250 MHz noise on it from the DAC.  Otherwise, the frequency and shape were accurate.  This noise should be filtered out in the front end signal chain. 

The plan I have right now is to instantiate 6 DDS engines inside the FPGA.  Two will be basic DDS engines that can read from DRAM but do not support modulation.  Two will be the same, but will support full rate (250 MSa/sec) AM and FM.  The last two will be quadrature sine/cos DDS units with compressed tables (18 bit truncated phase = 256k entries, too big for block RAM without compression).  The quadrature sine/cos DDS units will support full rate FM and I/Q AM.  Then there will be some sort of routing network (crosspoint or collection of MUXes) to interconnect everything. 

The DDS units will operate in 3 modes.  One is with an internal memory addressed by the phase accumulator.  This will support fixed-length waveforms (1024, 2048, 4096, or 8192 points, block RAM resources permitting) and the frequency will be determined by the phase accumulator rollover rate and can be used to upsample or downsample the waveform.  The second two modes will output one sample per phase accumulator rollover, so the rollover rate determines the sample rate, not the repetition frequency.  This can be used to upsample only to play back at a lower rate (no point skipping).  The difference between these two modes is that one will use the internal buffer, and the other will stream from the DRAM, thus bypassing DRAM for short waveforms. 

This architecture will allow for quite a bit of fun stuff.  For example, you can trivially AM and FM modulate a sinewave or an arbitrary waveform with an arbitrary waveform.  You can have one channel modulate the other channel, or add the two together.  Or you can have two completely independent QAM modulators running with completely arbitrary inputs.  I'm also going to look at getting a high bandwidth random number generator installed (Mersenne twister, if I can get it to run fast enough) and a uniform-to-normal converter so it is possible to output very wideband noise, or add noise to the output or to a modulating signal. 

[tridentsx]

I don't know if you seen that the source code for the HDG2002b is available at hanteks webpage. There is a link not that obvious saying that it runs linux "click here for more info". When doing that it downloads the source in a zip file.

The source is Including the backlight and keyboard drivers fpga loader etc. Right now my HDG2002b is bricked will try and make a custom
image if and when I resurrect it.

[alex.forencich]

Yeah, that was posted here previously.  I downloaded it a while back.  Aside from the keypad drivers, there isn't really much interesting in there.  The AWG app itself is not in there. 

The reverse-engineering is more or less complete at this point, and the goal is to build an open source firmware for the unit with a much better synthesizer and more features, not to mention more stable. 

[alex.forencich]

I just finished implementing the configuration path into the switched 250 MHz clock domain.  To do this, I wrote Wishbone wrappers for the MCBs and I converted the SoC interface module to interface with wishbone instead of to the MCBs directly.  Next, I added a wishbone mux and async register to cross into the switched clock domain.  I have a RAM as a stand-in for the control registers right now, and this is reliably accessible over SPI.  The next step is to connect this to the DDS units so they can be controlled over the SPI interface. 

[alex.forencich]

Quadrature DDS with compressed lookup table is now working in Python.  Each instance should consume 2 block RAMs and 2 DSP slices and will provide highly accurate quadrature sine and cosine outputs.  Two more DSP slices will allow for independent full rate amplitude modulation of the quadrature outputs.  Phase resolution is 18 bits, output resolution is 16 bits.  The 'equivalent' lookup table requires 256K entries, which will not fit on the FPGA.  Instead, three lookup tables are used, two are 512x16 stored in one 512x32 block RAM and the other is 256x8.  That's 18kb instead of 4Mb, a savings of 227x.  The two wide tables store coarse sine and cosine while the last table stores fine sine and then they are combined with sin(A+B) = sin(A) + cos(A)*sin(B) and cos(A+B) = cos(A) - sin(A)*sin(B) where A and B come from dividing the input phase into 3 segments, 1 + 9 + 8 bits.  The highest MSB at the input determines the sign of the output while the rest of the bits index the lookup tables.  When testing with an output frequency of 62.500625 MHz (1.00001 * Fs/4), the output SFDR is around 97.6 dBc.  Which is about 20dB better than the DAC can do at around 70 to 80 dBc.  The equivalent brute force lookup table has an output SFDR of around 104.7 dBc at the same test frequency. 

[tridentsx]

Thanks for all the hard work thats been done seems that a lot of progress was made. I hope this does continue.
Wish I had the skillset to contribute.

[smgvbest]

Thanks for all the hard work thats been done seems that a lot of progress was made. I hope this does continue.
Wish I had the skillset to contribute.

Same here.  Wish I did.  if there is something a beginner could help with I'd be happy to give it a try FWIW

[alex.forencich]

Mainly we need people to work on the software running on the SoC.  The FPGA code is coming together little by little, and that will end up being a very powerful arbitrary waveform generator platform.  However, there needs to be a nice way to control it, both from the front panel and from the USB and Ethernet interfaces.  We need to write some sort of program or collection of programs to do this.  I'm not quite sure how to go about doing that myself, unfortunately, as I don't have much experience with embedded linux. 

[lukier]

I had a conference paper deadline recently, so I couldn't do much work on HDG2002B. I also have my HT4032L firmware project on a backburner. In the coming weeks I'm planning to get EM2416 devboard and start building embedded Linux with Qt-embedded libraries for the UI.

I thought it would be good to use ZeroMQ/NanoMSG as the internal interface between components, this way we could split the core app (service) responsible for AWG control and FPGA communications from the UI app and Ethernet/USBTMC interfacing apps.

The nice thing about these libraries is that they are very performant, portable and flexible and have bindings to many languages (including Python, so it may be possible to reuse Alex's Python code for VXI/USBTMC to do the  communications stuff instead of reimplementing it in C++). Also, for initial development & testing, it would be helpful because we could develop the UI on the PC first and it would communicate to the core app running on the HDG.

I also thought that the core app should have switchable backend interface to the FPGA, so for early testing, instead of Samsung SoC, I would solder FTDI dongle (SPI mode + some GPIO I believe) to the to-be-optoisolated lines that control the FPGA and therefore could develop and test the core app on the PC as well. Doing everything on embedded SoC from the very beginning is a pain in the ....

BTW few weeks ago I was playing with HDG a bit, I dumped the entire flash with OpenOCD and so on. I was also testing the network interface and while it seems to work fine with my old 10/100Mbit Linksys router switch in my lab it didn't work with D-Link 1Gb switch. In dmesg it said connection established, leds blinking, and then after a second it would disconnect and everything repeats. I wonder if maybe JTAG pins are shared somehow and interfere or is it some Hantek kernel peculiarity  or DM9000 chip issue. Did anyone experienced similar problems?

BTW2 I didn't check that, maybe someone remembers. Does Hantek's U-boot supports NFS? This is extremely handy when building/testing embedded Linux root filesystem and kernel.

[alex.forencich]

Wellll....we're probably going to have to write the VXI-11/USBTMC stuff from scratch because the Python versions only implement the client side, not the server side.  The VXI-11 side probably won't be all that bad as it's just a TCP server, but the USBTMC stuff will involve figuring out how to interface with the USB stack on the SoC to set up a USB device correctly, which may be a bit involved.  And that will almost certainly have to be developed on the SoC. 

[idpromnut]

BTW few weeks ago I was playing with HDG a bit, I dumped the entire flash with OpenOCD and so on. I was also testing the network interface and while it seems to work fine with my old 10/100Mbit Linksys router switch in my lab it didn't work with D-Link 1Gb switch. In dmesg it said connection established, leds blinking, and then after a second it would disconnect and everything repeats. I wonder if maybe JTAG pins are shared somehow and interfere or is it some Hantek kernel peculiarity  or DM9000 chip issue. Did anyone experienced similar problems?

I have my HDG plugged into a 1Gb switch (Procurve) and it is working well. I Have not tested the NFS boot but yes, it supposedly supports it. I have a SC2416 development board (it looks a LOT like Hantek used the one I have to do their development for the HDG & oscilloscopes) but I have not yet got a working bootloader on it yet. I am rather new to the embedded side of things, being primarily a server software developer (plus some client-side application stuff). My intent was to help with the UI side for this project, but I have been rather busy as of late.

[smgvbest]

Mainly we need people to work on the software running on the SoC.  The FPGA code is coming together little by little, and that will end up being a very powerful arbitrary waveform generator platform.  However, there needs to be a nice way to control it, both from the front panel and from the USB and Ethernet interfaces.  We need to write some sort of program or collection of programs to do this.  I'm not quite sure how to go about doing that myself, unfortunately, as I don't have much experience with embedded linux. 

Since you mentioned it I found this SCPI Library
http://www.the-control-freak.com/SCPI/SCPI.htm
I haven't fully looked at it yet but should be able to port to Linux which I could try working on if there would be interest in that

[alex.forencich]

Mainly we need people to work on the software running on the SoC.  The FPGA code is coming together little by little, and that will end up being a very powerful arbitrary waveform generator platform.  However, there needs to be a nice way to control it, both from the front panel and from the USB and Ethernet interfaces.  We need to write some sort of program or collection of programs to do this.  I'm not quite sure how to go about doing that myself, unfortunately, as I don't have much experience with embedded linux. 

Since you mentioned it I found this SCPI Library
http://www.the-control-freak.com/SCPI/SCPI.htm
I haven't fully looked at it yet but should be able to port to Linux which I could try working on if there would be interest in that

Here is another SCPI parser worth looking at: https://github.com/j123b567/scpi-parser .

The SCPI parser is a part of the system.  The tricky part will be building the VXI-11 and USBTMC interfaces that will allow external applications to send SCPI commands to the parser.  Actually, the really tricky part will probably be defining and implementing all of the commands. 

[smgvbest]

Here is another SCPI parser worth looking at: https://github.com/j123b567/scpi-parser .

The SCPI parser is a part of the system.  The tricky part will be building the VXI-11 and USBTMC interfaces that will allow external applications to send SCPI commands to the parser.  Actually, the really tricky part will probably be defining and implementing all of the commands.

I guess one question is do we want to start implementing the command currently used in the HDG2000's or start from scratch
I would say IMHO start by implementing the what we already have command wise then add more if we want.

If that's the case the deciding on a parser (I do like the one you found as it has built in callbacks and written for Linux already)
We could define all the callback to implement the current commands which would get us started

[alex.forencich]

Here is another SCPI parser worth looking at: https://github.com/j123b567/scpi-parser .

The SCPI parser is a part of the system.  The tricky part will be building the VXI-11 and USBTMC interfaces that will allow external applications to send SCPI commands to the parser.  Actually, the really tricky part will probably be defining and implementing all of the commands.

I guess one question is do we want to start implementing the command currently used in the HDG2000's or start from scratch
I would say IMHO start by implementing the what we already have command wise then add more if we want.

If that's the case the deciding on a parser (I do like the one you found as it has built in callbacks and written for Linux already)
We could define all the callback to implement the current commands which would get us started

I say duplicate the Agilent commands for their TrueForm generators.  The command set for the HDG2000 sucks. 

[smgvbest]

Here is another SCPI parser worth looking at: https://github.com/j123b567/scpi-parser .

The SCPI parser is a part of the system.  The tricky part will be building the VXI-11 and USBTMC interfaces that will allow external applications to send SCPI commands to the parser.  Actually, the really tricky part will probably be defining and implementing all of the commands.

I like that. A decision.  you're obviously not in managment

I guess one question is do we want to start implementing the command currently used in the HDG2000's or start from scratch
I would say IMHO start by implementing the what we already have command wise then add more if we want.

If that's the case the deciding on a parser (I do like the one you found as it has built in callbacks and written for Linux already)
We could define all the callback to implement the current commands which would get us started

I say duplicate the Agilent commands for their TrueForm generators.  The command set for the HDG2000 sucks.

[alex.forencich]

Here is another SCPI parser worth looking at: https://github.com/j123b567/scpi-parser .

The SCPI parser is a part of the system.  The tricky part will be building the VXI-11 and USBTMC interfaces that will allow external applications to send SCPI commands to the parser.  Actually, the really tricky part will probably be defining and implementing all of the commands.

I like that. A decision.  you're obviously not in managment

I guess one question is do we want to start implementing the command currently used in the HDG2000's or start from scratch
I would say IMHO start by implementing the what we already have command wise then add more if we want.

If that's the case the deciding on a parser (I do like the one you found as it has built in callbacks and written for Linux already)
We could define all the callback to implement the current commands which would get us started

I say duplicate the Agilent commands for their TrueForm generators.  The command set for the HDG2000 sucks.

Your reply was misplaced.  Anyway, since we're rebuilding the core of the generator (the FPGA) I don't think we're really limited to what the original FPGA design could (or couldn't) do.  For instance, originally the FPGA DDS completely sucked (I recall seeing some posts about really bad phase noise) and the arb could only generate output samples with an extremely limited selection of sample rates.  I believe this was due to a crappy architecture.  This will not be an issue with the new architecture, so we might as well start with a command set that does not have these limitations. 

[lukier]

I say duplicate the Agilent commands for their TrueForm generators.  The command set for the HDG2000 sucks.

I too was thinking about starting with HDG functionality replicated, but you're right, Agilent command set is probably more useful. The UI, however, probably will have to follow Hantek in some shape or form because of the front panel layout.

USBTMC could be done with linux kernel USB gadget subsystem API in user space (with libusbg - https://github.com/libusbg/libusbg). Kernel has some gadget drivers built-in (ether, cdc acm, mass storage, even midi) but I don't recall TM class. Fortunately, USBTMC is probably the simplest of all the USB classes.

I think we should decide on SCPI commands because these will largely reflect the main interface API calls.

With the API defined we could implement the interface for clients (e.g. wrapped Apache Thrift over NanoMSG) and then more or less all modules (in bold) can be developed separately:
- API Interface server part -> Core App -> FPGA (over SoC, over FTDI)
- USBTMC (libusbg)/VXI11 (socket) -> SCPI parser -> API Interface client part
- Qt-embedded UI -> API Interface client part

[smgvbest]

Keysight has stuff including a legacy UBMTMC driver along with LXI samples FWIW
http://www.keysight.com/main/editorial.jspx?cc=US&lc=eng&ckey=1189290&nid=-34952.0.00&id=1189290&cmpid=20586

Wish they'd stop changing names

[alex.forencich]

Keysight has stuff including a legacy UBMTMC driver along with LXI samples FWIW
http://www.keysight.com/main/editorial.jspx?cc=US&lc=eng&ckey=1189290&nid=-34952.0.00&id=1189290&cmpid=20586

I think that's also a host side driver, so I'm not  sure how much use that would be.  I believe that driver might now be included in the linux kernel. 

Wish they'd stop changing names

Sigh.......

[smgvbest]

I think that's also a host side driver, so I'm not  sure how much use that would be.  I believe that driver might now be included in the linux kernel. 

Looking at the code it is a device driver

Hopefully the kernel used will be a fixed version and this is a moot point

[alex.forencich]

I think that's also a host side driver, so I'm not  sure how much use that would be.  I believe that driver might now be included in the linux kernel. 

Looking at the code it is a device driver

Hopefully the kernel used will be a fixed version and this is a moot point

Device driver as in a kernel driver for controlling a device that implements USBTMC, not a driver for the 'device' end of the link.  "This kernel module allows you to control your USB instruments through a character device driver."  Hence it is not very useful for what we are trying to accomplish, as we need what this driver connects to at the other end of the USB cable. 

[tridentsx]

I used my google fu and the only projetcs I came accross that has implemented the device side of USB-TMC are

http://labs.ti.bfh.ch/gecko/wiki/systems/gecko3com/start

and

https://github.com/karlp/discotmc

Both of them in a firmware without an OS, don't know if that's usefull or not, maybe need to do this from scratch.