HDG2002B AWG Firmware Reverse Engineering

[alex.forencich]

Well, any existing design might be useful, if only for reference. 

[tridentsx]

I have been looking for a reasonable priced PCI USB perephial card online and today I found one on ebay
http://www.ebay.com/itm/LOT-OF-2-PLX-2280BC-F-USB-DUET-PCI-ADAPTERS-/271691946991?pt=LH_DefaultDomain_0&hash=item3f421a1bef

If I have time I will try to get libusbg https://github.com/libusbg/libusbg and scpi-parser https://github.com/j123b567/scpi-parser to work together to implement the perephial side of USBTMC

[alex.forencich]

I have been looking for a reasonable priced PCI USB perephial card online and today I found one on ebay
http://www.ebay.com/itm/LOT-OF-2-PLX-2280BC-F-USB-DUET-PCI-ADAPTERS-/271691946991?pt=LH_DefaultDomain_0&hash=item3f421a1bef

If I have time I will try to get libusbg https://github.com/libusbg/libusbg and scpi-parser https://github.com/j123b567/scpi-parser to work together to implement the perephial side of USBTMC

Looks like that could be a good route.  I did not realize that they made PCI cards for the USB device side.  I suppose I shouldn't be too surprised; I think the high end Agilent scopes use something similar to provide their USBTMC interfaces as well. 

I'm getting optimistic about this project again.  I am under a deadline for a 10G Ethernet project right now, but once that's out of the way I should be able to do some more work on the FPGA for this thing. 

[tridentsx]

I received the cards today, and I found out that my Linux box doesn't have a PCI slot.
Going to a surplus computer place tomorrow to see if I can find some cheap old computer.

Is there any specifications that shows what commands a Function generator should support ?
Would be fun to somehow test the implementation with a tool that believes its talking to a real
instrument.

[alex.forencich]

I received the cards today, and I found out that my Linux box doesn't have a PCI slot.
Going to a surplus computer place tomorrow to see if I can find some cheap old computer.

Well, that's a minor set back. 

Is there any specifications that shows what commands a Function generator should support ?
Would be fun to somehow test the implementation with a tool that believes its talking to a real
instrument.

I would suggest looking at the programming manual for the Agilent trueform generators.  That's more or less what I think we should be shooting for in terms of capabilities. 

[smgvbest]

Very Nice update

[tridentsx]

It seems TMC is not supported as a gadget in Linux Kernel!
First step is to investigate what it would take to create a driver for TMC and for 488

http://lxr.free-electrons.com/source/drivers/usb/gadget/function/

I don't have much experience from Linux kernel development.

[alex.forencich]

Yeah, I figured it wouldn't be supported out of the box.  Probably the hard part would be figuring out what the user-facing API needs to be.  It looks like there is some good reference code in there, might as well start with the serial interface and see if you can get that operational as an exercise. 

[lukier]

I was playing with my HDG2002B a bit in the last week.

It seems we might be stuck with Hantek old crappy u-boot and kernel for now. S3C2416 is an old platform that is not actively maintained by the OS community.

From what I've seen in current mainline u-boot s3c24xx is supported in general, but booting doesn't seem to work. s3c24xx has a special boot process where it loads and executes first 4kB from flash for the early stage bootloader and this bootloader has to load the rest. In u-boot case this is called "SPL", but this part doesn't cover s3c24xx.

Embedsky wrote these routines (loading rest of the u-boot from flash) by hand in their old u-boot (2009, the one Hantek uses).

With linux kernel situation is similar, but slightly better.

Nowadays the way to configure linux kernel for a particular board is to use Device Trees. Before each board designer added a bunch of source and header files for each and every possible arm board and Linus got mad merging this mess.

So now there are little scripts called Flattened Device Trees that are compiled into binary form and kernel parses them on boot and loads and configures peripherals appropriately.
Again, s3c24xx is poorly maintained and while many s3c peripheral drivers migrated to new device tree configuration (interrupt controller, pin controller, pwm, uarts, spi, i2c, hs-mmc, dm9000) there are few crucial things that need the old approach (framebuffer, nand, dma engine).

Other than that linux-4.x seems to boot fine over tftp/NFS on the HDG2002B. For now I'm trying to limit the number of kernel warnings/errors and get all peripherals properly detected (figuring out GPIO pins and EINT lines). Later I'll test them in practice one by one.

Current boot log:
http://pastebin.com/Tf3cKspW

[alex.forencich]

It looks like there might be a way to do a user-space USB gadget driver.  This is probably what hantek did.  Might be more portable between kernel versions, since it seems like we may be stuck with the current kernel that's on there rather than the latest version.  See http://www.linux-usb.org/gadget/ near the bottom of the page.

It's great to see some activity on the software side of things!

[tridentsx]

Yes I have been looking at that possibility too, however I really wanted a composite device where I can run serial console on same USB connection. I didn't get my Ethernet to work thats why I am focused on USB. It didn't seem possible to create a composite device using GadgetFS but I found a presentation that shows how a composite device can be created with GadgetFS and FunctionFS


http://events.linuxfoundation.org/sites/events/files/slides/USB%20Gadget%20Configfs%20API_0.pdf

[fremen67]

In the coming weeks I'm planning to get EM2416 devboard and start building embedded Linux with Qt-embedded libraries for the UI.

I got one of them. It is more like a TQ2440 rather than a TQ2416 from a hardware point of view : DM9000EP instead of DM9000A, other interrupt numbers... you will have to modify the Hantek uboot and kernel so that it works on it (mainly for the DM9000EP which has INT7 instead of INT4 and needs nWait to be activated). I have a working modified hantek uboot/kernel for the EM2416 I can give you.

BTW2 I didn't check that, maybe someone remembers. Does Hantek's U-boot supports NFS? This is extremely handy when building/testing embedded Linux root filesystem and kernel.

The Hantek u-boot version (2009) is supposed to support NFS but it does not work. There is a problem in the time out handling. Not a big deal as you can always load the kernel via tftp.

Edit: I also modifed the hantek uboot version so that:
- tftp works
- you can update u-boot or kernel via SD
- you can exit the default Hantek menu and type user command at the prompt.

Maybe I could post the sources on Alex server.

[fremen67]

With the API defined we could implement the interface for clients (e.g. wrapped Apache Thrift over NanoMSG) and then more or less all modules (in bold) can be developed separately:
- API Interface server part -> Core App -> FPGA (over SoC, over FTDI)
- USBTMC (libusbg)/VXI11 (socket) -> SCPI parser -> API Interface client part
- Qt-embedded UI -> API Interface client part

That makes sense.
What about DBus for inter process communication? Seems to be well supported in Qt...

[fremen67]

I was playing with my HDG2002B a bit in the last week.

It seems we might be stuck with Hantek old crappy u-boot and kernel for now. S3C2416 is an old platform that is not actively maintained by the OS community.

From what I've seen in current mainline u-boot s3c24xx is supported in general, but booting doesn't seem to work. s3c24xx has a special boot process where it loads and executes first 4kB from flash for the early stage bootloader and this bootloader has to load the rest. In u-boot case this is called "SPL", but this part doesn't cover s3c24xx.

Embedsky wrote these routines (loading rest of the u-boot from flash) by hand in their old u-boot (2009, the one Hantek uses).

With linux kernel situation is similar, but slightly better.

Nowadays the way to configure linux kernel for a particular board is to use Device Trees. Before each board designer added a bunch of source and header files for each and every possible arm board and Linus got mad merging this mess.

So now there are little scripts called Flattened Device Trees that are compiled into binary form and kernel parses them on boot and loads and configures peripherals appropriately.
Again, s3c24xx is poorly maintained and while many s3c peripheral drivers migrated to new device tree configuration (interrupt controller, pin controller, pwm, uarts, spi, i2c, hs-mmc, dm9000) there are few crucial things that need the old approach (framebuffer, nand, dma engine).

Other than that linux-4.x seems to boot fine over tftp/NFS on the HDG2002B. For now I'm trying to limit the number of kernel warnings/errors and get all peripherals properly detected (figuring out GPIO pins and EINT lines). Later I'll test them in practice one by one.

Current boot log:
http://pastebin.com/Tf3cKspW

As the HDG (and other DSO/MSO from Hantek) are based on the TQ2416, I think this would be easier to stick with the TQ2416 Tools, at least for u-boot and the kernel, and stick to linux 3.2.35, unless there are good reasons to change it.
You will find on the TQ2416 DVD:
- the 4.3.3 toolchain (compiles Hantek uboot and kernel without any problem)
- Qt 4.5 + QT Creator 1.3.0
I set up a Fedora 10 VMware image with those tools and was able to compile and run some QT examples on the HDG and on my EM2416 (with touch screen support)

If we want to use the latest Qt version for the S3c2416, the 4.8.6 seems to one to choose. But then we would need to change the toolchain as the 4.3.3 can't compile it.
FriendlyARM has a s3c2416 dev board (Tiny2416) which is provided with the 4.4.3 toolchain and Qt 4.8.5.
I tried with Qt 4.8.6 compiled with the 4.4.3 toolchain and this also works on the HDG.

[fremen67]

It looks like there might be a way to do a user-space USB gadget driver.  This is probably what hantek did.

Yes I think they did. Some time ago when digging in the Hantek GPL kernel sources, I saw something on the USB TMC server part. I will have another look but I think there is a module for that written by Hantek in those sources.

[smgvbest]

It looks like there might be a way to do a user-space USB gadget driver.  This is probably what hantek did.

Yes I think they did. Some time ago when digging in the Hantek GPL kernel sources, I saw something on the USB TMC server part. I will have another look but I think there is a module for that written by Hantek in those sources.

If it is in there maybe they gave it a open source license and we can use it?   if not maybe we can learn from it and write a new one.  assume the source is there.

[fremen67]

If it is in there maybe they gave it a open source license and we can use it?   if not maybe we can learn from it and write a new one.  assume the source is there.

Attached the f-tmc.c file I saw in the GPL sources: AFG_hdg2000b\src\linux-3.2.35\drivers\usb\gadget\f-tmc.c

Plus the USB declaration in mach-hantek2416.c where you find the PID/VID of the USB TMC port, those you can see in Agilent connection expert:

Code: [Select]

...
#ifdef CONFIG_HK_USBD_ANDROID
//android gadget
#include <linux/usb/android_composite.h>
static char *usb_functions_ums[] = {
"mass_storage",
};
static char *usb_functions_serial[] = {
"serial",
};
static char *usb_functions_tmc[] = {
"tmc",
};

#define PRODUCT_ID (0x505b)
static struct android_usb_product usb_products[] =
{
{
.product_id = PRODUCT_ID,
.num_functions = ARRAY_SIZE(usb_functions_tmc),
.functions = usb_functions_tmc,
},
{
.product_id = PRODUCT_ID,
.num_functions = ARRAY_SIZE(usb_functions_serial),
.functions = usb_functions_serial,
},
{
.product_id = 0x0ff9,
.num_functions = ARRAY_SIZE(usb_functions_ums),
.functions = usb_functions_ums,
},
};

static char *usb_functions_all[] =
{
"tmc",
"serial",
"mass_storage",
};

static struct android_usb_platform_data android_usb_pdata =
{
.vendor_id = 0x049f,
.product_id = PRODUCT_ID,
.version = 0x0100,
.product_name = "hdg",
.manufacturer_name = "hantek",
.serial_number = "HTG10000522222",
.num_products = ARRAY_SIZE(usb_products),
.products = usb_products,
.num_functions = ARRAY_SIZE(usb_functions_all),
.functions = usb_functions_all,
.update_pid_and_serial_num = NULL,
//.usb_id_pin_gpio = ELITE_GPIO_USB_ID1,
.fserial_init_string = "tty:serial",
.nluns = 3,
};
static struct platform_device android_usb_device = {
.name = "android_usb",
.id = -1,
.dev = {
.platform_data = &android_usb_pdata,
},
};
#endif
...


[lukier]

That makes sense.
What about DBus for inter process communication? Seems to be well supported in Qt...

I never used d-bus, need to check how it works. Not sure about the performance. On the other hand it has RPC mechanism already done (instead of using something like Thrift on top of ZeroMQ transport layer).

As the HDG (and other DSO/MSO from Hantek) are based on the TQ2416, I think this would be easier to stick with the TQ2416 Tools, at least for u-boot and the kernel, and stick to linux 3.2.35, unless there are good reasons to change it.
You will find on the TQ2416 DVD:
- the 4.3.3 toolchain (compiles Hantek uboot and kernel without any problem)
- Qt 4.5 + QT Creator 1.3.0
I set up a Fedora 10 VMware image with those tools and was able to compile and run some QT examples on the HDG and on my EM2416 (with touch screen support)

If we want to use the latest Qt version for the S3c2416, the 4.8.6 seems to one to choose. But then we would need to change the toolchain as the 4.3.3 can't compile it.
FriendlyARM has a s3c2416 dev board (Tiny2416) which is provided with the 4.4.3 toolchain and Qt 4.8.5.
I tried with Qt 4.8.6 compiled with the 4.4.3 toolchain and this also works on the HDG.

I don't know, for me this would be last resort to use Hantek's archaic custom versions. These things are years away from the stuff I have currently installed on my development machine. Sticking with old versions would make it more difficult to develop and maintain open source firmware in the long run.

Anyhow, I would probably settle on Hantek's flash partition scheme and u-boot (not a big issue, it cannot boot with device tree I believe but device trees can be appended to kernel binary - for old bootloaders).

Currently I use just HDG, didn't buy EM2416 in the end and I can compile Hantek's u-boot and kernel. Could you share your modified u-boot?

Also, are HDG2002B flash dumps available somewhere? I might have messed up my flash contents. I have some dd if=/dev/ubi* backups as well as backup of the entire flash via OpenOCD (took hours) but I wonder if someone uploaded flash image somewhere so I could have second source.

[tridentsx]

Which minimum version of u-boot is required to support device tree ? I found some patches from end of 2012 that implements s3c2416 in U-Boot.

http://patchwork.ozlabs.org/patch/185880/

[lukier]

Which minimum version of u-boot is required to support device tree ? I found some patches from end of 2012 that implements s3c2416 in U-Boot.

http://patchwork.ozlabs.org/patch/185880/

Looks very interesting, NAND boot done in a proper modernish u-boot way (SPL). It may be even possible to apply this patch to the current mainline. I might have a go at it over the weekend.

[tridentsx]

I had a quick look at the u-boot patches and looked at u-boot versions all the way back to late 2010 and the patches doesn't match the source trees. There are many dependent includes files used by the patches that I can't find in any of the u-boot source releases.  Its almost like the original designer worked against another set of patches containing the architecture stuff and the bootstrap code for s3c24xx. I sent an email with some questions to the original author to see if he could shed some light.



Ok I got my answer I had the wrong patch,
The below patch is the complete final version with all the files

http://patchwork.ozlabs.org/patch/185884/

[fremen67]

I don't know, for me this would be last resort to use Hantek's archaic custom versions. These things are years away from the stuff I have currently installed on my development machine. Sticking with old versions would make it more difficult to develop and maintain open source firmware in the long run.

Anyhow, I would probably settle on Hantek's flash partition scheme and u-boot (not a big issue, it cannot boot with device tree I believe but device trees can be appended to kernel binary - for old bootloaders).

You are right but even if u-boot is quite old, as long as it has enough functionalities for restoring nand and booting test kernels, this should be OK.
Regarding the kernel, the GPL one provided by Hantek has already all the drivers included for the HDG and is just ready to go. We could of course change it later on when needed but as is, it will allow us to start developping the front end and the other modules which is IMHO what we should focus on.

I setup a Fedora 20 Vmware image with QT Creator 3.2.2, Qt 4.8.6 and a 4.4.3 arm toolchain. Everything seems to work fine up to now. I hope I will have some time this week to start something regarding a possible UI. I will also investigate with Qt Quick.

Currently I use just HDG, didn't buy EM2416 in the end and I can compile Hantek's u-boot and kernel. Could you share your modified u-boot?

Also, are HDG2002B flash dumps available somewhere? I might have messed up my flash contents. I have some dd if=/dev/ubi* backups as well as backup of the entire flash via OpenOCD (took hours) but I wonder if someone uploaded flash image somewhere so I could have second source.

You will find the files needed to rebuild a 1.00.3 firmware in my OneDrive in the "04-Firmware 1.00.3 Nand Recover" directory. The u-boot there is the modified one.

[fremen67]

@Alex:
I have a PCB 1002 version, I think you worked on a PCB 1004 version for your FPGA development and now Hantek is shipping at least PCB 1004.1
All those versions have specific firmwares which are not compatible together and I suppose that your development will only work on PCB 1004.
What is the best method to update the data you posted on the FPGA links to other components for PCB 1002 and PCB 1004.1?
I tried with TopJTAG and was able to find some differences (connections to the front end relays f.e.) but most of the connections are hard to trace (at least for me  ). As you already did this, could you explain you methodology?
I think this will help also others having 1004.1 to take part to this.
Do you think it could be a problem to maintain different hardware configurations? I suppose that Hantek had good reason to change their design.

[alex.forencich]

Depends on what the changes are.  The SDRAM should not be different as those pins cannot be reassigned.  If things like shift register bits are different, then that can be handled in software on the SoC - I'm just going to pass that through directly, the FPGA configuration will not depend on certain bits being in certain places.  It's not going to be smart enough to configure itself, the software on the SoC will take care of that.  Any changes in things like output amplifier gain steps will be handled in software on the SoC, not on the FPGA.  Pretty much everything else off of the FPGA could be modified in some way, though I'm not sure if they did or not.  I can think up some good solutions for this once we figure out exactly what's different about the boards - it's possible all you need is a different UCF file for the FPGA. 

For probing the FPGA pins, I used the ftjrev utility that's in my github repo.  I think I did have to pop off a couple of ferrite beads to check a couple of connections from the ADC, but other than that you should be able to probe everything.  Everything except the SDRAM, anyway. 

[fremen67]

@Alex: find attached the UCF file for the PCB 1002 FPGA, plus  ftjrev.c modified for my OpenJtag cable (www.100ask.net) in case someone need it (updated with PID/VID and nTRST used for iprobe). I also attached an excel sheet with the différences I saw between PCB1002 & 1004.

The sync DAC doesn't exist on my board so I removed it from the UCF file.
I also added in comment some test points linked to the FPGA.
As I don't know the FPGA side, I am afraid that's all I can do on that part 

Do you think you could create a firmware file like the existing htg1000.bin that we could download to the FPGA using the same way? That would be the easiest solution for testings and later on for firmware updates.