Siglent .ads firmware file format

[janekivi]

Yes, I got it from the link there
https://www.eevblog.com/forum/testgear/siglent-ads-firmware-file-format/msg998481/#msg998481


And I changed here root password too SDG2000_eevblog_P22R5.ads

[hafrse]

Yes, I got it from the link there
https://www.eevblog.com/forum/testgear/siglent-ads-firmware-file-format/msg998481/#msg998481


And I changed here root password too SDG2000_eevblog_P22R5.ads

works fine, thanks for your valuable contribution !!!!!!!!!!!!!!!!!!!!!!

[janekivi]

One ADS format is teared to the bits... you know... bits. OK.
But what about the other ADS, like for scopes.

Let's take one and start looking. May be we find something.
How all this look to me. I take SDS2000X and SDS1000X files here and compare them.
There is header and parts have 72 byte header. Hard to tell about header and first
file line but I start from start. First 4 bytes is 00123456? Next 8 bytes is parts
table. One byte represents one part and if it's present then there is 01 otherwise
there is 00. So i think 8 possible parts in firmware. Obviously there must be some
addresses. Next 8 x 4 bytes is them. If there wasn't that part - there isn't there
address too. Let's look closer sds2k_V100R02B01D01P38R07_fvA1606060606M160516.ADS.
Then we see what is next and where is file header ends.

Code: [Select]

00000000   56 34 12 00  01 01 01 01  01 00 01 00  D1 EC 0D 00  V4..........Ñì..
00000010   FF B7 16 00  FF B7 16 00  3B BB 16 00  CC 4C 13 00  ÿ·..ÿ·..;»..ÌL..
00000020   00 00 00 00  FF FF 07 00  00 00 00 00  32 2E 31 2E  ....ÿÿ......2.1.
00000030   31 2E 39 00  89 03 04 00  39 00 00 32  30 31 36 30  1.9.....9..20160
00000040   36 30 36 00  03 04 00 39  00 00 32 30  31 36 30 36  606....9..201606
00000050   30 36 00 03  04 00 39 00  00 32 30 31  36 30 35 31  06....9..2016051
00000060   36 00 03 04  00 39 00 00  31 2E 32 2E  31 2E 33 38  6....9..1.2.1.38
00000070   2E 37 00 00  39 00 00 00  00 00 00 00  00 00 00 00  .7..9...........
00000080   00 00 00 00  00 00 33 2E  31 2E 31 2E  31 33 00 37  ......3.1.1.13.7
00000090   00 00 39 00  00 00 00 00  00 00 00 00  00 00 00 00  ..9.............
000000A0   00 00 00 00  01 00 00 00  00 00 00 00  00 00 00 00  ................
000000B0   00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00  ................
000000C0   00 00 00 00  FF BD DC C9  32 2E 31 2E  31 2E 38 00  ....ÿ½ÜÉ2.1.1.8.
........   ...
Third part in here is part version. In some cases it's in date format. But i'm not
sure what's between. For each part there is 15 bytes. May be version nr is 8 bytes
and between is part type. Somewhere there is bin and when version was 9 bytes, for
"bin" wasn't room and there is "in". Obviously version part is missing too when 00
is written to part and length bytes. So 8 parts x 15 bytes is 120 and next data is
01 for SDS2000 and 03 for SDS1000. May be rest is padding there and real stuff may
begin from address C4 making header 196 bytes long. May be...
First 4 bytes is Siglent CRC for first part which is starting from C8 and have his
length in first place - D1 EC 0D 00 - 0D EC D1. Strange thing there is - CRC bytes
are in different order in this ADS format.
From CRC to the next part is 0D EC D1 and here is next part data length B7 B7 16 -
16 B7 B7. Data start with FF there and header must be 72 bytes because from header
we see the whole part length is FF B7 16.
Like this all is going to continue. First part is a bit different.

I have here different timezone and that's all for today.
Let's continue this bedtime story at next time.

[janekivi]

All is not that simple as it looks. But that's how all is at the beginning.

This SDS firmware is like memory dump with predefined regions for parts-files.
Addresses in the header is kind of place markers. Data can be shorter there
and in some cases I can't find any length or crc or other part-file guidance at
this beginning address. Of course I don't know anything about those parts.
First of them is something like string collection.
All of them are bit confusing right now...
I got suitable parts from Siglent_ADS project junkyard and start beating together
new utility for this.
_____________________
v0.1.1 with file open check

[janekivi]

When we open SDS2000X file sds2k_V100R02B01D01P38R07_fvA1606060606M160516.ADS
and cut it to the parts, today part 5 is interesting for us. It can be executable and contain
packed part too like SDG1000 file. If we look it in hex editor and scroll thru the data we can
see structure change at address 00 01 09 A7. It is like zlib header again - 78 DA. Zlib with the
best compression. Then Last 6 bytes is some sort of padding and Adler32 must be 88 CE 76 FC.
But who knows...

[janekivi]

I made several utilities for unpacking zlib but they all are working only with SDG1000
file. For this is good utility there too  http://aluigi.altervista.org/mytoolz.htm#offzip
which can seek packed parts from file and extract those.
Part 5 of SDS1000/2000 firmware is like second part from SDG1000 where is unpacker
and from address 00 00 CF E4 located zlib packed part. SDS file has the same section
at the beginning with the same error messages strings. And from address 00 01 09 A7
staring zlib part... But whole file have like 5 byte counter after every 24 bytes. From the
beginning we can see FA 18 00 00 00 and it continues 18 00 18... 18 00 30... 18 00 48.
But not like this to the end. When I remove it with mask I lost tracking and can only
decompress 18502 bytes of that packed part.
I have seen this kind of pattern elsewhere too... but what system is this and what is
the first byte before 18.

[janekivi]

Some updates.
-----------------
OK, that was kind of right. Part 5 has a 4 byte footer.
But whole file have like 4 byte counter after every 24 bytes. From the beginning we can see
6 bytes where last two are 65536 byte counters - 02 00 00 04 00 00 which 8-byte checksum
is next - FA. After that is starting data counter 18 00 00 00 and it continues 18 00 18 00...
18 00 30 00... 18 00 48 00. But not like this to the end, after every 65536 bytes is that
6 byte counter part.
After every data part is 8-bit checksum from previous data and counter, usually from 28 bytes.

So, counters are zeroed out and file is beginning:
02 00 00 04 00 00 --- counted 00 bytes
18 00 00 00  --- 18 - count 24 bytes

next is
18 00 18 00
 |     |---------24 bytes is counted
 |---------------count 24 bytes

next is
18 00 30 00
 |     |---------48 bytes is counted
 |---------------count 24 bytes
...
and so on until counter fills up
...
18 FF D8 00
10 FF F0 00
 |     |---------65520 bytes is counted
 |---------------count 16 bytes

now counter is over FF FF and comes
02 00 00 04 00 01
                |----first 65536 bytes is counted

and counter starts again
18 00 00 00
 |     |---------0 bytes is counted
 |---------------count 24 bytes
...
and so on when last 2 are
18 F8 58 00
 |     |---------63576 bytes is counted
 |---------------count 24 bytes

07 F8 70 00
 |     |---------63600 bytes is counted
 |---------------count 7 bytes

and last bytes of file are
00 00 00 01 FF
 |
 |---------------count 0 bytes - so it is the end of file

This is 4 byte footer which 8-bit checksum is FF. Maybe 01 is marking "end of file 1".
Now if we remove all counter parts and checksums, we get clean file and after decompressing
it the Adler32 is matching with 95 88 CE 76 at the end in original Part_5.hex in last data packet
before checksum.

Now if I remove all of the counter parts I get clean file and after decompressing it
I get Adler32 from the end of data  - 95 88 CE 76
I was hoping to see there something interesting, like logo in SDG1000 file but ... boring...

-----------------------------------------------------------------------------------------------------------------------------------
decompressed second part from part 5 of sds2k_V100R02B01D01P38R07_fvA1606060606M160516.ADS

[janekivi]

New SDS ads viewer utility v0.1.2 can get that app saved straight from opened firmware file or from
one of extracted files (part 5) with dedicated tools menu. Deflating can be done for example with offzip.
Only help at this time comes only from ToolTipTexts when staying on menu items...
-------------------------------------------------------------
Windows NET 3.5 C# application (WinXP, Win7...)

[dav]

What about the SDG and SSA .ADS viewer utility?

[janekivi]

I might release SDS viewer but there is too many buttons.
So, is somebody english review my EEenglish help file - yes, I made help file,
first time in history, then we see...

[dav]

Just to let you know, Avira found this trojan in "SDS ads viewer utility v0.1.2"

Crypt.Xpack.vyifq

[janekivi]

Crypt.Xpack itself isn't trojan, this is only cryptic packer which was in programmers tool.
But from this day on nobody can't use it any more because somebody was packing some
trojan with this...

Other components from it I wanted to use were flagged already
https://yck1509.github.io/ConfuserEx/

[tridentsx]

Has anyone looked at the ads files for the power supplies http://www.siglentamerica.com/USA_website_2014/Firmware&Software/firmware/SPD3303X-Firmware-V100R001B01D02P03.rar ?

[tautech]

Has anyone looked at the ads files for the power supplies http://www.siglentamerica.com/USA_website_2014/Firmware&Software/firmware/SPD3303X-Firmware-V100R001B01D02P03.rar ?

That could be an interesting exercise with the possibility of improving a SPD3303X-E to an X model saving $150 and gaining 10x the V and A resolution. 

[janekivi]

Of course I looked in all files. But "looked" was the right word. There was no other experiments.
Inside there is one app file. I see in that directory I did look it with IDA too. But how or where I dug out this image I don't remember...

edit
-----
 * Actually they have different firmware, you may need to compare them (both are in the zip):
 * Boot image was just cut out from
    SPD3303X-E-V100R001B01D02P03.hex from 00 02 4A C8 to 00 02 7F 5B and saved as JPG
    SPD3303X-V100R001B01D02P03.hex 00 02 4A AC - 00 02 7F 3F

[janekivi]

I can make the same trick here like when I was uploading LeCroy in to SDG1025
but who can try this and there may be the same check routine and instrument
will say "Not supported firmware, please reflash correct. Otherwise I will wait 15 min".
This is stupid, it will wait that time anyway before you can access flash menu...

So... do not be the first who is using this firmware file on SPD3303X-E
but this first hack may be needed to be tried out by someone.

SPD3303X-V100R001B01D02P03_with _E_header.zip
OK, I give this only individually after request. I can't test it.

[janekivi]

New SDS ads viewer utility v0.1.2 can get that app saved straight from opened firmware file or from
one of extracted files (part 5) with dedicated tools menu. Deflating can be done for example with offzip.
Only help at this time comes only from ToolTipTexts when staying on menu items...
-------------------------------------------------------------
Windows NET 3.5 C# application (WinXP, Win7...)

Version 0.1.3 has some minor fixes and help file which is full of all kind of strange text.
What you gonna do, I can't write in normal english...

-------------------------------------------------------------
In help file is updated SDS file format description, which is not so relevant

-------------------------------------------------------------
Maybe part version end in header is space and next crap is leftover from something old,
so version 0.1.4 displays that table differently now.

[tv84]

Hi janekivi ,

I've been doing some research on Siglent .ADS files, trying to parse every single file that they have.

I've managed to understand most of the files packing, as you have done, including parsing the Blackfin BF54x and BF53x blocks in order to load them more easily in IDA. I've not analysed uBoot, ELF files. I let that to the Linux guys.

I have been doing it in C# (console). Once I have it more bullet-proof i'm thinking in releasing a compiled version so that people can unpack the files.

(I'm not sure if you have found it yet, but the 1st byte in your "5-byte type of blocks counter" is the last byte of the previous block. So, the blocks start (usually) with the 0x18 + 3 bytes + n (0x18) data bytes + last byte (which is the checksum of the whole block, including header). The checksum is the usual  "- Sum of all bytes".

This 5th block in the SDS files is the only understandable to me. Do you know what are the other blocks for?

[janekivi]

I don't understand almost anything in ads files, only taking it apart and looking into everything.
Linux is not my thing and in IDA I don't recognize much. Also I was looking into
ro_uImage, rw_uImage - partly packed
datafs.img, firmdata0.img - UBI images
and with GIMP or IrfanView I scan all files for graphics.

In SDS files it is 8-bit Checksum indeed. I may recompile my SDS viewer. But I was cutting out
right part anyway because unpacked checksum matches. Unpacker part in the beginning of
the 5th part is the same as in SDG1000 file. Part 1 is most likely help. Parts 2, 3, 4 almost the
same as FPGA file LcdFpga.bin from SSA3000X. I mean beginning and especially look at the end.
Who knows... maybe version numbers in the beginning can help if somebody can see them in
scope menu. Like 3.1.1.13, 2.1.1.8, 2.1.1.9
I have only SDG2000X and SDG1000 actually.
(Of course Flir E4 and Rigol DS1054Z too for other threads )

[tv84]

Yes, the beginning of the 5th part is similar to the SDG1000 but the Blackfin processor is a BF54x and, as such, has a totally diferent way of decoding (relative to the BF53x of the SGD1000).

OK, so now you've turned to the files inside the ZIPs. I'll be trying those also to see if anything comes up. I've also implemented many graphic detectors in my programs GIF / JPEG / MPEG / etc that work even if there are no (usual) magic numbers.

I also have a Rigol scope so might have a look at that.

I've also developed a deflate scanner that can detect zlib/gzip streams without proper headers and crc/adler, and sometimes becomes useful for dealing with this type of compression in stripped fw. I think there's notinhg around that is able to do this.

The goal is: as usual, just for fun.

[janekivi]

.......
I also have a Rigol scope so might have a look at that.
.......
The goal is: as usual, just for fun.

This is here... but we were far away from fun, we want to change PLUSES  to PULSES
https://www.eevblog.com/forum/testgear/rigol-dsxxxx-gel-firmware-file-format/

[tv84]

ro_uImage and rw_uImage both have a GZIP stream starting at 0x4966 with no valid header but with valid CRC32+Size. That's easy for my scanner.

I'll have a look at the RIGOL efforts... Will be difficult to help you guys.

[janekivi]

http://www.garykessler.net/library/file_sigs.html
1F 8B 08 is GZIP header at 495C, there was no problems but normal UBI-reader is hard to find.
Couple of times I was successful with ubidump but now I use something with Ubuntu.

[tv84]

Sorry for that. You're right, the magic bytes were there.

I'm so accustomed to finding deflate compression with no headers and no footers that many times I forget the easy way which is: the file is totally in there... 

[janekivi]

Yeah, I didn't try this before, but they use the same CRC calculation methods everywhere.
Somewhere output is 32 bit and in SDS it is 8 bit. And they can be used as MSB or LSB first.

Code: [Select]

import functools

# If you have file
input = 'File'
data = bytearray(open(input, 'rb').read())

# Or data can be declared directly
# data = bytes([0x02,0x00,0x00,0x04,0x00,0x00]);

csum = functools.reduce(lambda x,y: x+y, data, 0)
csum = ~csum + 1
csum = csum & 0xffffffff # the only difference is here
print (format(csum, 'X'),"- 32 bit checksum")
csum = csum & 0xff # the only difference is here
print ("     ",format(csum, 'X'),"-  8 bit checksum")