Ethernet transfer speed for Rigol DS2072A and DS4014

[NorthGuy]

I am interested in deep memory of Rigol scopes. My idea is to do a single acquisition of full memory range, download the data over the Ethernet (LXI) to my computer and then analyze the data. DS2072A has 56Mbytes, and DS4014 has 280Mbytes. It looks fantastic, but then I thought that downloading through the Ethernet may take really long time, so it might not be as useful as I imagine.

I don't know if the Ethernet transfer rates have ever been measured for these scopes. If anyone has any measurements done on the speed of transfers, would you please share the data.

[RoGeorge]

I have had some LAN speed issues that you might want to know, but it was because of the Windows OS, not the scope itself. There were no LAN speed issues with Linux.
https://hackaday.io/project/5807-driverless-rigol-ds1054z-screen-capture-over-lan/log/18807-microsoft-bullied-tcp-rfcs-the-result-a-painfully-slow-lan-on-windows-and-a-wasted-weekend

For DS1000Z series, it will take tens of seconds with Linux and many minutes with Windows in order to download 24 million data points. Also, when you download the raw data, the scope must be in STOP mode and you need to download multiple data batches to extract the whole waveform. If you want to download in real time, it won't work.

I didn't test the models you specified.

[NorthGuy]

I have had some LAN speed issues that you might want to know, but it was because of the Windows OS, not the scope itself.

TCP window is not a parameter. This is the size of the remaining buffer on the receiving side - to make sure the sender doesn't send too much data. It depends on the total size of the receiving buffer as well as the sizes and speed of your reads. If you read in awkward sizes or if you don't read often enough the window will shrink. So, to get the best speed you need to read fast and in big chunks.

It is also possible to set the size of the receiving buffer with setsockopt() SO_RCVBUF to make the buffer substantially bigger (although I've never tried doing this). The buffer could be made big enough to fit all the data at once.

10-s of seconds (e.g. 30 sec) for 24Mbytes is less than 1Mbyte/sec - rather slow for 100Mbit Ethernet connection. Is this number for binary data or for ASCII?

[RoGeorge]

TCP Receive buffer and TCP Receive Window (AKA RWIN), is not the same thing. RWIN is the amount of data that can be received without ACK, and it is gradually increasing in a normal TCP connection. Microsoft decided to implement it's own "RWIN Auto-Tuning" algorithm for RWIN, disregarding how the rest of the world is doing the same thing. I am not the only one who got into trouble because of this, just google "RWIN Microsoft", e.g. the first Google result https://www.duckware.com/blog/how-windows-is-killing-internet-download-speeds/index.html

My tests were made using a Telnet connection from Python 2.7, first with the native Win10 OS, and second with a Debian inside a VMware running on the same Win10 OS. A Rigol DS1054Z oscilloscope was the server. Linux transfer time was way more faster then Windows. Same results when opening a TCP socket from Python 2.7. Same results when using NetCat without Python.

To answer your question, we were talking about 24 Mbytes of binary data sent over a 100 Mbps TCP connection.
Please try it yourself and let me know your results.

[2N3055]

Those TCP parameters should have no influence on speed on a local network with latencies of less than 1ms..
I have transfer speeds in range of 100Mb/sec on a 1Gbit LAN between mixed hosts..

These Receive buffer and window problems are old news, on Vista and early Win7, when running on fast WAN with lots of bandwidth and relatively high latencies.. I has been patched since...

I'm not saying you're wrong, just it is a bit unusual..

I presume you are doing lots of small block transfers? Maybe buffering, maybe something in Python implementation on windows.. like shutil.copy slow on windows but working fine on linux...

I will look into it these days to see if I can find out something..

[thm_w]

Did you see the software from marmad?
https://www.eevblog.com/forum/projects/software-tips-and-tricks-for-rigol-ds200040006000-ultravision-dsos/

My software will connect with the scope using either USB or TCPIP VISA resources. If you're planning to save many recorded frames, it's best to use USB (USB: ~2.5s = ~6s transfer time on TCPIP). The VISA resource locator is not coded with a sniffer for finding TCPIP connections - so it's default behavior is as follows:

[Dwaine]

Running my DS1054z by LAN Connection under Fedora 25 with DSRemote.  The screen updates every second.  I was actually quite surprised.  No slow slow downs at all.

[RoGeorge]

I presume you are doing lots of small block transfers? Maybe buffering, maybe something in Python implementation on windows..

The DS1054Z oscilloscope can not transfer all 24 million data points at once. The programming Guide of the scope recommend maximum 250 000 data points at once, so 24 millions need 96 distinct TCP transfers. Before each of the 96 transfer, 2 SCPI commands for setting setting the start and the stop data point are sent:

Code: [Select]

:WAV:STAR 1
:WAV:STOP 250000
:WAV:DATA?
- here we receive the first 250 000 data points (plus a few bytes of header)
:WAV:STAR 250001
:WAV:STOP 500000
:WAV:DATA?
- here we receive the next 250 000 data points (plus a few bytes of header) and so on, until all 24 000 000 bytes are transfered.
The same speed problems was seen with NetCat, AKA 'nc', a very old tool (before Python) from Unix, also ported for Linux and Windows. I didn't checked the sources for NetCat, but I guess it's written in C, not Python.

RWIN algorithm guilty was my conclusion after looking at the TCP packets with WireShark. Maybe it's not from RWIN algorithm, maybe is something else.

The problem is still present today, in the latest Win10 updated this month, so if you have the opportunity to investigate this issue with a Rigol scope, please do it.

Running my DS1054z by LAN Connection under Fedora 25 with DSRemote.  The screen updates every second.  I was actually quite surprised.  No slow slow downs at all.

To update a screen trace, only 1200 data points are necessary. The OP want to transfer the RAW data for all 24000000 data points.

[2N3055]

Thanks for details, as soon as I get time I'll look into it... Have a good one.

[NorthGuy]

@RoGeorge: The "Windows" problem is probably unrelated to TCP window. The article you referred to is talking about speeds which are at least order of magnitude faster than the speed you get with Linux.

I cannot try any of this because I don't have any of these scopes. I started this thread hoping to get speed information so I could better choose which one (if any) to buy.

If you wish, I can write a simple test program in C which reads the data in big chunks with minimum overhead, and then you can run it with yours scope.

@thm_w: This is an interesting thread. Took me a while to sift through The 6s number is for a screenshot and suggests that the transfer speed is 0.2 MBytes/sec, which is horrible and even much worse than the data speed quoted by RoGeorge. But they use Rigol software and VISA (don't know what it is), so it is likely that there's a lot of overbloat and speed lost. Also, the speed of sending screenshots may be drastically different from the data speed.

The data speed is uncertain. Looks like Marmad uses ASCII transfers for data. In his post #243 he quotes:

:WAV:FORM ASC

instead of

:WAV:FORM BYTE

This not only increases the number of transfer bytes several times, but also burdens the scope because everything needs to be scaled and convert it to the text.

The only real time data that I could find is for 140-byte transfers for the screen data (in post #455 and then forward). They get 8ms per 140 data points on DS2072A, but this includes sending a command and waiting for the answer, and it most likely ASCII. So, it is not much you can infer from this.

BTW: They've got much worse and uneven speeds for DS1054Z.

[RoGeorge]

If you wish, I can write a simple test program in C which reads the data in big chunks with minimum overhead, and then you can run it with yours scope.

Sure, I will gladly test your code with my scope and publish here the transfer time for both Linux and Windows.

[Karel]

With the DS1000Z series it takes 55 seconds to download 24Mpts via usbtmc. Via ethernet it is more or less the same.
The bottleneck is the scope, not the connection.
Data needs to be downloaded in chunks of max 250,000 bytes.
Between those chunks there's a long delay in the response of the scope's firmware.
Why? Only Rigol knows...

Edit: I did a new test with the latest firmware (the one Rigol removed) and now the download time is 33 seconds for 24Mpts.

[NorthGuy]

Sure, I will gladly test your code with my scope and publish here the transfer time for both Linux and Windows.

Here it is. The program is called ds1064z (the Windows one has exe on the end, the Linux one will need execution permission set). Both need to be run from the terminal/command line. They'll print the amount of data received. They do this:

:STOP
:WAV:SOUR CHANnel1
:WAV:MODE RAW
:WAV:FORM BYTE
:WAV:STAR 1
:WAV:STOP 250000
:WAV:DATA?
...
:WAV:STAR 250001
:WAV:STOP 500000
:WAV:DATA?
...

etc. so the data must be acquaried on CH1 with memory depth of 24Mpts.

Of course, I don't know if they work correctly or if they're capable of getting data from the real scope (for example I didn't know whether to use \r or \r\n as the line ending (used \n).  I tested it with ds1054zs.exe (included in the zip), which pretends to be Rigol, but is really dumb - it only outputs 250000 of random data when it sees ":WAV:DATA?" and does nothing else. If you want to use it, you'll need to open port 5555 in your firewall. The transfer takes 2s either on Linux or Windows. So, there doesn't seem to be any problem with the connection speed. Rigol, certainly won't be that fast.

[RoGeorge]

Thank you. May I have the sources and compile them myself, please? I don't want to run a binary from a forum.

If the sources are not for publishing, it's OK, but you need to wait until Friday, when I will have time to prepare a clean machine isolated from the rest of the LAN.

[NorthGuy]

I don't want to run a binary from a forum.

That's a good idea. I think the same way.

If the sources are not for publishing, it's OK, but you need to wait until Friday, when I will have time to prepare a clean machine isolated from the rest of the LAN.

I wrote it in Pascal because I had a cross-compiler. And it uses few of my private libraries which I cannot give out. So, you won't be able to compile it anyway. I should've used C

I guess we'll have to wait until Friday. I predict it'll be the same speed as Karel reported 2 posts back.

[RoGeorge]

Found a clean Debian machine (don't have a clean win10 machine yet) and this is the result:

Code: [Select]

rogeorge@debian80:~/Downloads/tbt$ ./ds1054z
Usage: ds1054z ip.address.of.the.scope
rogeorge@debian80:~/Downloads/tbt$ ./ds1054z 192.168.1.3
Connected
Received 7272 bytes in 0.634 sec.
rogeorge@debian80:~/Downloads/tbt$
While running this, the screen of the scope displays the text message "Invalid Input!".

Looking inside the binary with IDA, it seems like the program is sending all the SCPI commands at once, with \n between them. This can not work. The client must wait for the instrument reply and, if necessary, to interrogate the instrument for its status. There is a state machine described in the SCPI protocol.

Please read the documentation for the SCPI protocol. SCPI is now part of the IVI Foundation. IVI specifications are not free, but SCPI specs are free to download.

[NorthGuy]

This can not work. The client must wait for the instrument reply and, if necessary, to interrogate the instrument for its status. There is a state machine described in the SCPI protocol.

I see. I've read the DSxxx programming scpecifications and it looked to me as request-response protocol over TCP - server fetches commands from the stream one by one and executes them consecutively (such as in FTP, SMTP, POP3 and many other Internet protocols).

Please read the documentation for the SCPI protocol. SCPI is now part of the IVI Foundation. IVI specifications are not free, but SCPI specs are free to download.

Ok. I downloaded the SCPI-99 document from here: http://www.ivifoundation.org/specifications/default.aspx

It does say that the commands can be concatenated with ";", such as ":STOP;:WAV:SOUR CHANnel1;MODE RAW". Is this what I'm supposed to use instead of "\n"? How do I know which commands can be chained with ";" and which cannot?

It doesn't say anything else about delimiters, but it has to be some way to delimit commands - the Rigol examples don't seem to contain any ";" at the end and same goes for the examples in SCPI-99. I assumed "\n", but it doesn't seem to work.

It doesn't say anything about state machines and ready states and responses to the commands. E.g. Am I supposed to receive something in the response to the ":STOP" command?

Could you point me to a document which describes this basic stuff?

[Karel]

Could you point me to a document which describes this basic stuff?

Have a look at the Rigol Programming Guide for the MSO2000A & DS2000A (Februari 2016) page 2-366.
It's all explained in detail. I don't know if Rigol is following the exact SCPI/TMC guidelines however...

What I noticed during my experiments some time ago is that, when you send two commands to quickly one after another,
strange things can occur. For me, the only way to get things working with Rigol stuff is by keeping a minimum delay between
consegutive commands. And for sure, don't send multiple commands in one packet...
But maybe now with the newer firmware versions they behave better, I haven't tried that.

[NorthGuy]

Have a look at the Rigol Programming Guide for the MSO2000A & DS2000A (Februari 2016) page 2-366.
It's all explained in detail.

It explains what to send, but it doesn't say how. May be this is obvious to someone familiar with SCPI, but I'm not. So, the questions I have are these:

1. How do I terminate commands. Do I send ":STOP", ":STOP\n", ":STOP\r\n"?

2. Can I concatenate commands. Like ":STOP;:WAV:MODE:RAW"?

3. What response should I expect. E.g. I say ":STOP\n". Does it say something back?

4. How do I figure if it's ready for the next command or not?

Looks like it is not simply "apply the specs and it complies", but needs lots of tweaking and obeying scope's idiosyncrasies. Easy to figure out when you have a scope. Not that easy otherwise.

@RoGeorge: Looks like it won't be that easy to write the test program without having the scope. If I buy one, I'll make the test program for you. But I can assure you that slow transfers are not Windows fault - the time necessary to transfer 24MBytes over the LAN is 2s for either Linux or Windows (tested W7 and W8.1) - everything above this is caused by the scope and/or the system you're using to access it.

[Karel]

Have a look at the Rigol Programming Guide for the MSO2000A & DS2000A (Februari 2016) page 2-366.
It's all explained in detail.

It explains what to send, but it doesn't say how. May be this is obvious to someone familiar with SCPI, but I'm not. So, the questions I have are these:

1. How do I terminate commands. Do I send ":STOP", ":STOP\n", ":STOP\r\n"?

:WAV:STOP\n

2. Can I concatenate commands. Like ":STOP;:WAV:MODE:RAW"?

According to the SCPI spec: yes.

In the case of Rigol: yes/no/maybe  (pick one)

3. What response should I expect. E.g. I say ":STOP\n". Does it say something back?

A command is just a command. There's no response. Only queries have a response. Queries end with a questionmark.

4. How do I figure if it's ready for the next command or not?

According to the SCPI spec you can use the *OPC? command to query the device if it has completed the operation.
In the case of Rigol... better to put a delay of 20 milli-Sec. between two consegutive commands...

Looks like it is not simply "apply the specs and it complies", but needs lots of tweaking and obeying scope's idiosyncrasies. Easy to figure out when you have a scope. Not that easy otherwise.

I completely agree.

[NorthGuy]

33 sec for 24 Mb is 0.73 MB/s. If I get similar rates then:

77 sec for 56 MB in DS2072A, which is worse than I expected, but not that bad.

Almost 7 min for 280 MB in DS4014 doesn't sound good.

Of course, these may be faster or they may be slower (because they are older). But looks like nobody knows

[RoGeorge]

":STOP;:WAV:SOUR CHANnel1;MODE RAW". Is this what I'm supposed to use instead of "\n"? How do I know which commands can be chained with ";" and which cannot?

- I would use ":STOP;:WAV:SOUR CHANnel1;:MODE RAW\n". ':' before a commands starts interpreting the command from the root of the commands tree. Otherwise, the command starts from the last branch. '\n' must be the terminator.
- Not sure how many and which commands can be sent on the same line. I saw this technique usually for sending multiple settings before executing a command, but for other instruments, not for Rigol scope, e.g. send the start frequency, stop frequency, amplitude, etc, all on the same line, then start a frequency sweep in frequency generator instrument on the next line. For Rigol scope, I just tried ":stop;:wav:form byte;:wav:form?\n" and it returned "BYTE\n"

(SCPI-99) doesn't say anything about state machines and ready states and responses to the commands. E.g. Am I supposed to receive something in the response to the ":STOP" command?

- If there is no state machine in SCPI-99, then I might have read about it somewhere else. I did a lot of googling when I start learning about SCPI.
- For the ':STOP\n' command no. Usually, for (correct) commands you don't receive an SCPI response. Only for queries. The difference between commands and queries is described in the SCPI standard.

Could you point me to a document which describes this basic stuff?

- I am not sure if such a thing exists. At least I couldn't find it, but after a few days of googling and reading various documents you will have a good grasp about SCPI.

From memory, what seemed most important to me at that time:
- I remember some manuals from Agilent instruments had good explanations and charts with status registers, enabling registers, etc. e.g.: http://cp.literature.agilent.com/litweb/pdf/ads2001/pdf/vsaprog.pdf
- Sometimes, when you send an invalid command (mostly because of a wrong parameter) you might receive something like "Command error". Parameter validation is not guaranteed, so if you send something wrong, it might be just ignored.
- There are a lot of SCPI registers, some mandatory, some others are instrument dependent. You need to test the status of your instrument before sending something.
- The most important is to test if the previous operation completed (*OPC?) and if there are errors. Even the simplest commands like :STOP might take a very long time, or might fail.
- Some queries will have an answer with the length of the expected reply, i.e. the :WAV:DATA? will send a header that contains the length of the binary data and will end in \n

====================================================

Long story short, it is not trivial to write a full program for SCPI, but this is not the point for the moment. We were testing how fast can your buffering method download the data on both Linux and Windows.

I will do all the scope settings manually, then run your test program. Try to write just the part for setting the start, stop, receiving/saving data and timing the whole loop. Don't bother treating errors for the moment, just make sure you received all the data you requested. If you feel you don't need this test any more, it's OK from my side.

[RoGeorge]

I placed the previous post before refreshing the page left open this morning.

... I can assure you that slow transfers are not Windows fault ...

OK. 

[NorthGuy]

I will do all the scope settings manually, then run your test program. Try to write just the part for setting the start, stop, receiving/saving data and timing the whole loop. Don't bother treating errors for the moment, just make sure you received all the data you requested. If you feel you don't need this test any more, it's OK from my side.

Based at what you and Karel say, it doesn't require much of special care. And based on your test, it did do smething for 0.6s and then it sent some data. So, looks like it actually worked. I think it simply got upset when it found the next "DATA?" command while still executing the previous one. So, we can try downloading a single segment and see how long it takes. Then I can modify the program to do it for all 96 segments. Sounds simple enough.

The improved version is attached. I removed the ":STOP" command from the beginning (because it may require some lengthy processing on the Rigol side). The program now downloads only one 250000 byte segment, and then it prints first 128 bytes of what it received and quits. Let's see if any gardeners show up

[RoGeorge]

@NorthGuy, you were correct, if there was a bug then it was fixed, so there is no garden any more 
I can not reproduce the transfer problem any more.

PS. Looking at the TCP packets with WireShark, the Windows and Linux versions of the last attachment are slightly different, meaning the 2 versions are not sending the same SCPI commands. These are the outputs:

Debian 8.0

Code: [Select]

rogeorge@debian80:~/Downloads/tbt/v2$ ./ds1054z 192.168.1.3
Connected
Received 1250073 bytes in 0.972 sec.
rogeorge@debian80:~/Downloads/tbt/v2$

Windows 10

Code: [Select]

C:\tbt from EEVblog forum\v2>ds1054z.exe 192.168.1.3
Connected
Received 250012 bytes in 0.438 sec.
23 39 30 30  30 32 35 30  30 30 30 83  83 83 83 83   |#9000250000.....|
83 83 83 83  83 83 83 83  83 83 83 83  83 83 83 83   |................|
83 83 83 83  83 83 83 83  83 83 83 83  83 83 83 83   |................|
83 84 83 84  83 84 83 84  83 84 83 83  83 83 83 83   |................|
83 83 83 83  83 83 83 83  83 83 83 83  83 83 83 84   |................|
83 84 83 84  83 84 83 83  83 83 83 83  83 83 83 84   |................|
83 84 83 84  83 83 83 84  83 84 83 83  83 84 83 83   |................|
83 83 83 83  84 84 84 84  83 84 83 84  83 84 84 84   |................|

C:\tbt from EEVblog forum\v2>
The Linux version in the first and the second attachment are the same:

Code: [Select]

C:\>fc /b "C:\tbt from EEVblog forum\v2\ds1054z" "C
:\tbt from EEVblog forum\ds1054z"
Comparing files C:\TBT FROM EEVBLOG FORUM\V2\ds1054z and C:\TBT FROM EEVBLOG FORUM\DS1054Z
FC: no differences encountered


C:\>
Anyway, the time calculated from the WireShark timestamp of the TCP packets for the first 250000 bytes only is basically the same, and I couldn't see the RWIN problem any more:

Debian80 transfer time = 0.066444
Win10 transfer time = 0.067693

The HaD info was updated with a 'Later edit' too: https://hackaday.io/project/5807-driverless-rigol-ds1054z-screen-capture-over-lan/log/18807-solved-a-painfully-slow-lan-on-windows-and-a-wasted-weekend

Thank you for the test programs.