HDG2002B AWG Firmware Reverse Engineering

[alex.forencich]

I'm trying to figure out an SPI protocol for controlling the FPGA, and I am looking at the SPI protocol for an ST microelectronics 128 Mbit flash chip for reference.  The fast read command on the ST part consists of an instruction byte, followed by 3 address bytes, and a dummy byte, at which point the data starts streaming out.  Here's the issue: with a 50 MHz SPI bus frequency, that's 8 * 20 = 160 ns to issue a read request to DDR2 and get the first word back.  Worst case (precharge/activate required) read latency for a single port MCB is 32 clocks @ 312.5 MHz = 102.4 ns.  However, it is possible that another read operation will be in progress on another port.  If there is currently a 32 word read operation taking place that also requires precharge/activate, then that could add 32 + 16 + 25.6 ns (precharge + CAS + transfer) = 73.6 ns.  So overall worst case might be 102.4 + 73.6 = 176 ns.  Unfortunately, 176 > 160, so there is a chance that the read will be corrupted if the data is not ready in time. 

There are a couple of possible solutions:

1. run the SPI interface slower.  40 MHz would be 200 ns per byte, leaving 24 ns (6 250 MHz clock cycles) extra.  Running at 50 MHz is preferable because it should be possible to upload a 128 Mpt waveform in ~43 seconds.  At 40 MHz that would take ~54 seconds. 
2. send more than one dummy byte.  2 dummy bytes would be 320 ns, which would give a larger 'buffer'.
3. don't use a fixed number of dummy bytes, instead send a start of read data indication.  This might be the best option as latency issue is completely decoupled, just discard leading zero bytes + 1.  However, reads may have to read a few extra bytes to make sure the entire read completes before CS is lowered. 
4. use a completely different protocol that avoids this problem entirely by decoupling the read request from the read data transfer. 

Also note that this is only really a problem for reading, writing is not affected since the incoming data words can simply be stuffed in a FIFO until the memory is ready (actually, this is how the MCB works: there is a command FIFO, a read FIFO, and a write FIFO - to write, stuff the data in the write FIFO and then stuff the address in the command FIFO; to read stuff the address in the command FIFO and wait for data in the read FIFO). 

Has anyone measured the default SPI clock rate to the FPGA?  I know the SoC is capable of 50 MHz, but did Hantek actually try to use it at that speed?

[Cyber7]

Hi Alex,

Clock source selection to generate HS_SPI clock-out can be: PCLK, USBCLK, EPLL clk.

Assuming PCLK, and High_speed_en set in the config reg, this can be 66.666Mhz as per the SOC boot up logs. USBCLK is a paltry 12Mhz, while I'm uncertain about EPLL without consulting the guide.

The SPI captures I made at the isolation tap point, which initialized the front end, had a 60ns sck cycle (16.66Mhz ) when enabled. This could hardly be the case if the entire SDRAM is read when TTsource performs a 'read' operation, as it does not take very long. Frankly I don't recall offhand if I had the entire buffer populated; I was more interested in getting arb waveforms loaded into SDRAM. I'll look over the SCPI commands logged to see if this is the case, and do a few caps during r/w ops for arb waveforms.

[alex.forencich]

Hi Alex,

Clock source selection to generate HS_SPI clock-out can be: PCLK, USBCLK, EPLL clk.

Assuming PCLK, and High_speed_en set in the config reg, this can be 66.666Mhz as per the SOC boot up logs. USBCLK is a paltry 12Mhz, while I'm uncertain about EPLL without consulting the guide.

The SPI captures I made at the isolation tap point, which initialized the front end, had a 60ns sck cycle (16.66Mhz ) when enabled. This could hardly be the case if the entire SDRAM is read when TTsource performs a 'read' operation, as it does not take very long. Frankly I don't recall offhand if I had the entire buffer populated; I was more interested in getting arb waveforms loaded into SDRAM. I'll look over the SCPI commands logged to see if this is the case, and do a few caps during r/w ops for arb waveforms.

66.66 MHz?  Well, I think a 15ns cycle time can be synced to the 4ns system clock period without too much trouble.  That would take the transfer time of all 128 M points to 32 seconds.  At the default 16.66 MHz clock, the transfer time is around 130 seconds.  I suppose it's really not that huge of a difference, but one of the strong points of this thing is the gigantic sample memory, so it would be nice to be able to access that at a reasonable speed.  It's more the write speed than the read speed that's important, though, as transferring a waveform in to the unit is much more useful than transferring it out. 

[Cyber7]

TTSource has no data read option. The read config option actually has no SOC<->FPGA SPI traffic. I can see traffic when I change any setting, or bulk transfers when writing.

I did see a reference in the Hantek code for the DSO5000 that seems to limit the SPI clk to 48Mhz. Then again, the max SPI speed as per the device structs are 1Mhz for spidev & <500khz for spi for fpga cfg:

static struct spi_board_info s3c2416_spi_board_info[] __initdata = {
   [0] = {
      .modalias   = "spidev",
      .max_speed_hz   = 1000000,
      .bus_num   = 0,
      .chip_select   = 0,
      .irq = IRQ_SPI0,
      .controller_data = &csinfo
   },
   [1] = {
      .modalias   = "AFG3050_fpga_cfg",
      .max_speed_hz   = 499000,
      .bus_num   = 0,
      .chip_select   = 1,
      .irq = IRQ_SPI0,
      .controller_data = &csinfo
   },
};

However, this is not in anyway written in stone as the DSO5000 fpga config utility overrides this to 2.5Mhz:

   spi->mode = SPI_MODE_0;
   spi->bits_per_word = 8;
   spi->max_speed_hz = 2500000;
   fpga_cfg = AFG3050_fpga_cfg_create(spi);

The specs are vague on the upper limit. I've seen a reference to 50MBits/s, but nothing to substantiate that.

Mode 0 (format A) is the original Microwire spec. 8 bits per frame. Each bit is 60ns wide.

For MOSI, they appear to use a page consisting of a 2 byte header + 16byte payload + 2 zero bytes. Each page being 5ms apart.

So there appears to be a delay or timeout that balances CPU resources vs throughput; TRX must be fulfilled while waiting for the RXD FIFO to drain.

See images below.

Here are some limitations on the Linux SPI device API. Nothing onerous, save the max page transfer size is not defined. I think its 256 bytes/page. Thats ~120us for the page at the current Hantek defined data rate.

    - At this time there is no async I/O support; everything is purely
      synchronous.

    - There's currently no way to report the actual bit rate used to
      shift data to/from a given device.

    - From userspace, you can't currently change the chip select polarity;
      that could corrupt transfers to other devices sharing the SPI bus.
      Each SPI device is deselected when it's not in active use, allowing
      other drivers to talk to other devices.

    - There's a limit on the number of bytes each I/O request can transfer
      to the SPI device.  It defaults to one page, but that can be changed
      using a module parameter.

    - Because SPI has no low-level transfer acknowledgement, you usually
      won't see any I/O errors when talking to a non-existent device.

[alex.forencich]

The Spartan 6 DC and switching DS (DS162) lists the max CCLK frequency at 80 MHz.  So the upper limit for configuring the FPGA in slave serial mode will either be the range of the SoC baud rate generator or board signal integrity.  The limit for a custom SPI implementation in LUTs will likely be synchronization.  I would suggest taking a look at the config user guide (ug380) if you haven't already, since we will eventually need to load the FPGA config via slave serial. 

Basically, there are two main ways to implement an SPI slave on the FPGA.  The first is to use SCK as a clock and feed this directly to the CLK inputs of the flops in the receive shift register.  Then higher-level signals will need to be synchronized to the local clock domain.  This is not necessarily the best idea.  The other method is to register all of the SPI signals into a local fast clock domain (in this case, 250 MHz / 4 ns) and then do rising edge detection with an additional flop.  Since 66.7 MHz has a period of 15 ns and the receive clock domain has a period of 4 ns, I think this might be doable.  7.5ns on, 7.5ns off will end up being 1 to 2 clock periods in the local clock domain.  To test the integrity, we should transfer large blocks into SRAM or DRAM and then read them back out and make sure no bits got flipped.  It is possible that we'll have to slow down the clock a bit.  50 MHz has a period of 20 ns and a half period of 10 ns, so each half period should end up being 2 to 3 clocks in the local clock domain.  With this method, no additional synchronization is required since it's already in the local clock domain.  In our case, there will have to be some method for crossing over to the switched 250 MHz clock domain, though this will be relatively low bandwidth (primarily configuration and status registers). 

[alex.forencich]

Clock source selection to generate HS_SPI clock-out can be: PCLK, USBCLK, EPLL clk.

Right, I saw that in the manual.  The question is, how fast are these different clock sources?

Assuming PCLK, and High_speed_en set in the config reg, this can be 66.666Mhz as per the SOC boot up logs. USBCLK is a paltry 12Mhz, while I'm uncertain about EPLL without consulting the guide.

Is PCLK 66.667 MHz or is the SPI module capable of running at 66.667 MHz when driven by PCLK?  If PCLK is 66.667 MHz, then I suppose we could just run the SPI interface at 33.333 MHz.  That would mean a byte would transfer in 320 ns, which would probably alleviate the latency issue out of DDR2. 

[Cyber7]

PCLK is the source for the SPI clock. As per bootup msgs and source structs:


S3C24XX Clocks, Copyright 2004 Simtec Electronics
CPU: MPLL on 800.000 MHz, cpu 400.000 MHz, mem 133.333 MHz, pclk 66.666 MHz
CPU: EPLL on 96.000 MHz, usb-bus 48.000 MHz

///

struct s3c_cpufreq_info s3c2440_cpufreq_info = {
   .max      = {
      .fclk   = 400000000,
      .hclk   = 133333333,
      .pclk   =  66666666,
   },

///

The desired baud rate is software selectable, however, without the source for the user mode app, we can't be certain what value is selected.

baud_rate = PCLK / (2 * ( prescaler + 1) )

ie:

if PCLK = 50000000, and desired baud_rate = 2500000 ( 2.5Mbps is the hantek default for spi fgpa config) ==> the prescaler = 9

The prescaler works out to a nice even number. This is not the case for PCLK = 66666666 as it yields 12.333, however the setting in the driver is the *maximum*, so it could well be the the user mode app is selecting any smaller value. Lets examine the observed baud rate of 16.666667Mhz:

if PCLK = 66666666 , and desired baud_rate = 16666666 ( 16.67Mbps ) ==> the prescaler = 1

However if we desire a baudrate = 33333333, then the prescaler becomes 0

Assuming 0 is a valid value, then this is the max baud rate with the given PCLK of 66.67Mhz.

If we utilized the EPLL = 96Mhz source clock instead, then the max baud rate, with a 0 value prescaler,  becomes 48Mbps.
I believe this clock is termed S3C2416_SPI_SRCCLK_48M within the source:


CPU S3C2416/S3C2450 (id 0x32450003)

#define S3C2416_SPI_SRCCLK_PCLK      0
#define S3C2416_SPI_SRCCLK_SPIBUS   1
#define S3C2416_SPI_SRCCLK_48M      2
static char *spi_src_clks[] = {
   [S3C2416_SPI_SRCCLK_PCLK] = "pclk",
   [S3C2416_SPI_SRCCLK_SPIBUS] = "spi-bus",
   [S3C2416_SPI_SRCCLK_48M] = "spi_48m",
};

...
s3c64xx_spi_set_info(0,0,2);
...

void __init s3c64xx_spi_set_info(int cntrlr, int src_clk_nr, int num_cs)
{
   struct s3c64xx_spi_info *pd;

   /* Reject invalid configuration */
   if (!num_cs || src_clk_nr < 0
         || src_clk_nr > S3C2416_SPI_SRCCLK_48M) {
      printk(KERN_ERR "%s: Invalid SPI configuration\n", __func__);
      return;
   }

   pd = &s3c64xx_spi0_pdata;
   pd->num_cs = num_cs;
   pd->src_clk_nr = src_clk_nr;
   pd->src_clk_name = spi_src_clks[src_clk_nr];
}

[Cyber7]

@Idpromnut:

Been walking like a penguin and have the DSO5000 u-boot and kernel cross-compiled with arm-none-linux-gnueabi-gcc (Sourcery CodeBench Lite 2014.05-29) 4.8.3 20140320. I copied permissions from original uboot/linux distros as the hantek source was zipped. Had to alter a header reference and add a video driver, but it otherwise compiled without a hitch. They did make a few tweaks regarding the s3C2416 and the LCD display.

I'll try the existing uboot with the new kernel on an ext4 SDcard, and let you know how it does.

mach-hantek2416.c has proved very interesting as it has many GPIO pin & device definitions, including SPI defines + initialization! Hopefully they were kept the same on the AWG.

[idpromnut]

@Cyber7:

I was looking at that source file last night. My approach was going to be to take a diff between Hantek's 3.2.35 source tree and the current tree that I'm working with, 3.2.63 and create a patch using that. But yes, I think we would be better off reusing their modifications to the kernel to get the various bits and pieces of hardware working.


EDIT: I just looked at the result of my "patched" linux-3.2.35 kernel with a patch generated from the DSO5000 kernel tree and so far, it looks like the build succeeded. I will try booting my HDG2002 tonight with it and see what goes "boom".   

EDIT, supplemental: I got that kernel to boot, but it segfaulted somewhere while booting the rootfs, but that's ok!  Just means the binaries all need to be compiled together (with the correct deps, etc). As an aside, I got some text on the screen when it segfaulted, so the framebuffer is working!

[idpromnut]

I'll create a patch and dump it in git a bit later tonight. The patch is created against a stock 3.2.35 Linux kernel, so it should make it easy to compile a run-able kernel.

[Cyber7]

@Idpromnut: Can you tell me how you prepared your rootfs and which fs you're using? I tried using nandsim to mount the rootfs nanddumps to no avail, ubinized or no, with/without OOB, with/without  bad blks. So, I then mounted the rootfs / as /mnt/backup and tarballed the whole thing. Then, I untared the entire contents to an ext3 partition (#2) on an sd card. But I get this....

Code: [Select]


Kernel command line: noinitrd console=ttySAC0 init=/linuxrc rootfstype=ext3 root=/dev/mmcblk0p2 rw rootwait lcd=X480Y272
...
VFS: Cannot open root device "mmcblk0p2" or unknown-block(179,2)
Please append a correct "root=" boot option; here are the available partitions:
1f00            1024 mtdblock0  (driver?)
1f01            2048 mtdblock1  (driver?)
1f02            4096 mtdblock2  (driver?)
1f03          123904 mtdblock3  (driver?)
b300         7761920 mmcblk0  driver: mmcblk
  b301            8192 mmcblk0p1 00000000-0000-0000-0000-000000000000
  b302          131072 mmcblk0p2 00000000-0000-0000-0000-000000000000
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,2)
 
Edit: Looks like the default hantek kernel build options baked out many file systems. I'll enable them and try again.

Code: [Select]

#
# File systems
#
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
# CONFIG_EXT4_FS is not set

[idpromnut]

@Cyber7:

Ah, I was just focusing on getting a kernel that could boot the original hantek rootfs in the nand flash. I had not tried rolling my own rootfs yet. However hantek have indeed striped back many "common" kernel options so we will need to modify the config as we go most likely.

[Cyber7]

@Idpromnut:

Did you get any of the boot params to actually persist & save to the NAND? Looks to me like all settings are uboot src header hardcode defaults. If I make changes, reboot and nanddump mtd1 it yields all 'FF' and all settings revert. 

Guess it's time to risk flashing a new uboot.bin from the Hantek src via uboot opt 1, and keep the backup & jtag handy...

EDIT: Adding kernel support for EXT3/EXT4 solved the boot issue. I also removed the NAND support, since I'm not using it at the moment. The fs statup does fault while starting. I do note some failures (namely driver failure messages), like the inability to 'download' the FPGA bit file, serial and keybord errors, DBUS, etc...

Code: [Select]

FPGA CONFIGURE DATA DOWN finish.0x0, 0
FPGA CONFIGURE timeout.
fpga firmware /lib/firmware/htg1000.bit write failed: -1
ln: /dso/app/etc/etc: File exists
ln: /dso/app/home/home: File exists
ln: /dso/app/root/root: File exists
ln: /dso/app/mnt/udisk/udisk: File exists
ln: /dso/app/mnt/sd/sd: File exists
anolis_set_app_init_fun:1172 fun != NULL failed.
anolis_set_app_exit_fun:1179 fun != NULL failed.
open usbd_serial device error !!!!
open kbd error
dbus is valid and addr is unix:path=/tmp/dbus-zegXRYLz8E,guid=d1b4438527287abd42fe94675459d030

[tinhead]

i've spoke with Hantek, they released the GPL sources now for the HDG's

http://www.hantek.com/download/HDG.zip

There are as well all sources of the custom drivers and some control/test apps,
not only for HDG AWGs but as well other S3C2416-SoC based Hantek gears.

[idpromnut]

Did you get any of the boot params to actually persist & save to the NAND? Looks to me like all settings are uboot src header hardcode defaults. If I make changes, reboot and nanddump mtd1 it yields all 'FF' and all settings revert.

No, I haven't tried modifying anything yet; I think I'd rather play around with the dev board first (I don't feel confident enough that I can "un-brick" my AWG if I do something un-good to it just yet). And yes, the boot params are stored in the u-boot partition, but I haven't tried to modify them yet. I might give that a shot tonight.

I also wanted to investigate the upgrade mechanism that Hantek use; I think that will be the best way to hook into the HDG2002 and do backups, restores, and upgrade with custom firmware.  Assuming that they actually implemented a firmware upgrade.   

[idpromnut]

i've spoke with Hantek, they released the GPL sources now for the HDG's

http://www.hantek.com/download/HDG.zip

There are as well all sources of the custom drivers and some control/test apps,
not only for HDG AWGs but as well other S3C2416-SoC based Hantek gears.

Wow, Hantek, if you're listening, thank you very much for the support!

EDIT: Double wow… I need to try and rebuild it, but from a quick check over the sources, this will help a lot when trying to figure out the custom hardware that Hantek has put into the HDG2002 (for the keyboard, etc). I will check and see, but if we can build a bootable image from this, perhaps we can start from this and focus on just the UI and FPGA parts of the firmware. I will check and see what license Hantek released the sources to their custom test and drivers in this package.

[tinhead]

I will check and see what license Hantek released the sources to their custom test and drivers in this package.

as long you use it with Hantek hardware it's anyway free to use , but seriously, it's under GPL v2.
As you said, these sources are really cool, not only for HDGs but as well all the handhelds/DSOs.,
not only for keyboard but as well things like DMM in handheld. I was hoped to get such sources from
Hantek / Tekway for a long time.

[alex.forencich]

Here is an interesting feature idea, courtesy of thewyliestcoyote: streaming data from a PC.  Now, this won't work for the full sample rate, but I think it would be possible to do this at 1 or 2 MSa/sec and then use it as, say, a modulation signal.  It would even be possible to stream I/Q data for digital modulation.  The carrier would be generated with high rate DDS modules inside the FPGA, then the streaming data could be used to modulate the high rate carrier (AM, FM, PM, FSK, PSK, BPSK, QPSK, QAM, etc.).  Space permitting, it might be possible to build an arbitrary I/Q modulator inside that would take in digital data and generate I/Q samples for modulation.  Arbitrary digital modulation at 30-40 Mbit/sec from data streamed from a PC could be very interesting. 

[idpromnut]

Here is an interesting feature idea, courtesy of thewyliestcoyote: streaming data from a PC.  Now, this won't work for the full sample rate, but I think it would be possible to do this at 1 or 2 MSa/sec and then use it as, say, a modulation signal.  It would even be possible to stream I/Q data for digital modulation.  The carrier would be generated with high rate DDS modules inside the FPGA, then the streaming data could be used to modulate the high rate carrier (AM, FM, PM, FSK, PSK, BPSK, QPSK, QAM, etc.).  Space permitting, it might be possible to build an arbitrary I/Q modulator inside that would take in digital data and generate I/Q samples for modulation.  Arbitrary digital modulation at 30-40 Mbit/sec from data streamed from a PC could be very interesting.

I like it!  Again, I think we need to aggregate all the features (accepted and/or proposed), perhaps in one of the first two posts in this thread. If everyone is ok with this, I'll go ahead and sift through this thread and gather all the proposed features together + add the obvious ones in the existing firmware.

[Cyber7]

There are as well all sources of the custom drivers and some control/test apps,
not only for HDG AWGs but as well other S3C2416-SoC based Hantek gears.

AWESOME! Its like an early Christmas present. Eagerly awaiting the 500MB download.

Space permitting, it might be possible to build an arbitrary I/Q modulator inside that would take in digital data and generate I/Q samples for modulation.

Nice idea. Regarding logic space: we can strip out unneeded logic add the specific function/mode, reload the FPGA with from the SOC with the different bit logic for the specialized sub function, and switch back bit sets when done.

To add to the feature set: a closed-loop automated way to calibrate the front end by VISA/VXI control of a scope and the AWG. I don't think we want to spend two hours taking and entering sample sets!
 

[alex.forencich]

Space permitting, it might be possible to build an arbitrary I/Q modulator inside that would take in digital data and generate I/Q samples for modulation.

Nice idea. Regarding logic space: we can simply strip out unneeded logic add the specific function/mode, reload the FPGA with from the SOC with the different bit logic for the specialized sub function, and switch back bit sets when done.

Well, we shall see.  The main thing I am worried about is the availability of DSP48 slices.  Most of these will be taken up by the polyphase filter for the fractional up/down converter.  Everything else will have to fight over what's left over after that.  I presume this will not take up very much space, but we shall see.  One other issue is that running complex logic at 250 MHz is likely to require a lot of pipelining, which will consume a lot of registers.  It may end up being necessary to swap bitfiles for certain configurations. 

To add to the feature set: a closed-loop automated way to calibrate the front end by VISA/VXI control of a scope and the AWG. I don't think we want to spend two hours taking and entering sample sets!

Well, that's a no-brainer.  I was hoping to be able to do some sort of self cal with the onboard ADC, but I don't know if the signal routing is set up for that.  I did some investigating, but didn't turn up anything useful. 

If we run Python IVI on the AWG, then it could connect to a scope via USB or Ethernet and control it remotely to automate the measurement procedure.  Not sure if that would be worth the trouble; a standalone calibration script would work just as well and would be easier to maintain. 

[idpromnut]

RE: the onboard ADC... there is a driver for that in the sourceball that Hantek made available.. I wonder if the ADCs are accessed directly from the SoC and not through the FPGA?

[alex.forencich]

RE: the onboard ADC... there is a driver for that in the sourceball that Hantek made available.. I wonder if the ADCs are accessed directly from the SoC and not through the FPGA?

There is no direct connection between the ADC and the SoC, so it can only be accessed through the FPGA. 

[idpromnut]

RE: the onboard ADC... there is a driver for that in the sourceball that Hantek made available.. I wonder if the ADCs are accessed directly from the SoC and not through the FPGA?

There is no direct connection between the ADC and the SoC, so it can only be accessed through the FPGA.

What about the SoC onboard ADCs? Are they connected out to the two channel outputs?

[alex.forencich]

RE: the onboard ADC... there is a driver for that in the sourceball that Hantek made available.. I wonder if the ADCs are accessed directly from the SoC and not through the FPGA?

There is no direct connection between the ADC and the SoC, so it can only be accessed through the FPGA.

What about the SoC onboard ADCs? Are they connected out to the two channel outputs?

Nope.  The board has footprints for digital isolators, and the only signals that cross those are power and SPI to configure and control the FPGA.  Nothing else.