Pocket-Sized 6 GHz 1 TS/s ET Scope

[SJL-Instruments]

To avoid confusion, we've switched to the more explicit notation shown in the attached image. Hopefully this is more clear.

[joeqsmith]

It's been running since that last update and I have not detected a single glitch.  I let it run overnight again using my software.  This time collecting a wider range.  I really have only been focusing on this one problem but I ran several tests with it yesterday and nothing else showed up. 

For the manual, you may want to remove the "Single-Point Upsets".    Also consider adding some additional details to the manual for the R(Acquire CDF) command.  When more than one channel is selected, the manual states:

The binary data consists of d chunks of 3Nch bytes each, where Nch is the number of channels requested via the bitmask. Each chunk containsNch three-byte entries, each representing a single CDF value F(V ;Δt) for each requested channel.   

It wasn't clear to me how this data was formatted.   I convert the entire chunk array (which includes data for all selected channels) to form a voltage/CDF array.  I then decimate the data  V1_Ch1,CDF1_Ch1;  V1_Ch2,CDF1_Ch2;  ... V1_Chn,CDF1_Chn;   V2_Ch1,CDF2_Ch1;  V2_Ch2,CDF2_Ch2;  ... V2_Chn,CDF2_Chn;   ...
where Ch is the channel number, Vt and CDFt  are the voltage and CDF for a specific sample.   I thought it may have sent all of the data for the first channel, then the second rather than interlace them. 

[SJL-Instruments]

For the manual, you may want to remove the "Single-Point Upsets".   

In the current revision (H5) of the manual, we changed this section to link to the firmware upgrade instructions. We'll remove the section once we've confirmed everyone is on the patched version.

It wasn't clear to me how this data was formatted.   I convert the entire chunk array (which includes data for all selected channels) to form a voltage/CDF array.  I then decimate the data  V1_Ch1,CDF1_Ch1;  V1_Ch2,CDF1_Ch2;  ... V1_Chn,CDF1_Chn;   V2_Ch1,CDF2_Ch1;  V2_Ch2,CDF2_Ch2;  ... V2_Chn,CDF2_Chn;   ...
where Ch is the channel number, Vt and CDFt  are the voltage and CDF for a specific sample.   I thought it may have sent all of the data for the first channel, then the second rather than interlace them.

We've added the attached example to the next manual revision - let us know if it's clear.

[joeqsmith]

When no trigger is present, the scope will trigger off its internal clock (>100 Mtrig/s) until a valid
trigger source returns.

With nothing attached to the scope's trigger your software will continue to sweep.  Connecting a DC signal to the scope's inputs, you are certainly reading the correct data.  When I try this with my software, the scope never appears to send data unless there is a valid trigger.  Is there another command you are using to force the scope to send the data?

*** Using your software, if there is no trigger when I start, it begins to sweep.  Normal.   If I apply a single trigger event, the software stops sweeping and waits for further triggers.  It never returns to what ever state it was in that allowed it to collect without triggers.   I don't see anything in the software menus that allows me to control this.   Assuming this is how you have designed it to behave,  could you maybe provide further details to the manual explaining it beyond that one sentence?  *** 


Also, while experimenting with my own software and yours, I got the scope into some mode where when I attempt to run your software, I get the attached  error.   Your software would come up to the main menu but that dialog would remain.    I could exit your software and restart and would get this same error.  Eventually, I pulled the USB connection to the scope (power cycling it) which cleared the problem.   I am not a fan of having to power cycle any device to clear an error.  I don't want to be driving down the highway and have to pull over to yank the battery cable to reset the engine's computer.

[SJL-Instruments]

With nothing attached to the scope's trigger your software will continue to sweep.  Connecting a DC signal to the scope's inputs, you are certainly reading the correct data.  When I try this with my software, the scope never appears to send data unless there is a valid trigger.  Is there another command you are using to force the scope to send the data?

There is a command that selects the internal clock source as the trigger (X1 sets internal source, X0 sets to normal trigger mode). For each acquisition, the software first tries in normal mode, and if no trigger is detected, issues the X1 command and retakes the CDF.

See attached image - we'll document this command in the next manual revision.

It never returns to what ever state it was in that allowed it to collect without triggers.

The software should automatically return to the internal trigger ("wait") mode if no triggers are detected (e.g. the signal source is removed). What happens when you try this - does the software freeze?

Also, while experimenting with my own software and yours, I got the scope into some mode where when I attempt to run your software, I get the attached  error.   Your software would come up to the main menu but that dialog would remain.    I could exit your software and restart and would get this same error.  Eventually, I pulled the USB connection to the scope (power cycling it) which cleared the problem.   I am not a fan of having to power cycle any device to clear an error.  I don't want to be driving down the highway and have to pull over to yank the battery cable to reset the engine's computer.

We can reproduce this error if some data remains in the serial buffer prior to starting our software.
We'll send you a preview build shortly that clears the buffers on startup - you can see if this fixes the problem.

[joeqsmith]

There is a command that selects the internal clock source as the trigger (X1 sets internal source, X0 sets to normal trigger mode). For each acquisition, the software first tries in normal mode, and if no trigger is detected, issues the X1 command and retakes the CDF.

See attached image - we'll document this command in the next manual revision.

Please take the time to document ALL of the commands.  Even if you have commands you feel are not something customers would/should use, they may still prove helpful.   

*** Sending X1 or X2, it responds with OK Xn.  Sending X1 followed with R, I do not get any data.  There must be more to it. 

The software should automatically return to the internal trigger ("wait") mode if no triggers are detected (e.g. the signal source is removed). What happens when you try this - does the software freeze?

The software displayed Wait but it stopped sweeping.  I could apply a trigger signal and it remained in Wait.  I can't tell you for sure that it froze.  I have yet to perform a three finger solute to shake your software loose.  I exited the software and restarted and it worked as normal.  I can tell you that I am having a difficult time replicating it.     

[joeqsmith]

Using my software to collect all four channels with min/max active.   I run the RF generator into a splitter then into each channel.  Using 0dBm, 1GHz.   

1) zoomed out showing 46 sweeps, 5ps resolution for 1ns.  I make no attempt to align the channels and we can see that one channel is a bit off compared with the others like I had mentioned.

2)  Looking at the minimum values of the peak area.  Note how channel 1 is smooth where the others have some sort of pattern to them. 

3) Looking at the maximum, we can see they are all smooth. 

If I reset the collection and watch it, I can see when it will create a single mini-glitch.  They are much smaller in amplitude than the original glitches.   This appears very reproducible.   Nothing special with the cables, splitter or RF gen.  Have you seen these smaller spikes?

[SJL-Instruments]

Please take the time to document ALL of the commands.  Even if you have commands you feel are not something customers would/should use, they may still prove helpful.   

Got it - there are two remaining serial commands we'll document in the next revision: set CDF epsilon (section 2.2.1), and direct CDF measurement at V,t.

*** Sending X1 or X2, it responds with OK Xn.  Sending X1 followed with R, I do not get any data.  There must be more to it.

We were able to get CDF data with the following procedure:
1. Plug in the scope.
2. Issue command X1.
3. Issue command R1 10000 10000 10000 10000 50000 50000 50000 50000.
Does this sequence fail to return data on your unit?

The software displayed Wait but it stopped sweeping.  I could apply a trigger signal and it remained in Wait.  I can't tell you for sure that it froze.  I have yet to perform a three finger solute to shake your software loose.  I exited the software and restarted and it worked as normal.  I can tell you that I am having a difficult time replicating it.     

The software is separated into a frontend and backend process. We can reproduce this behavior if we manually kill the backend process. We'll see if there's any code path that would crash the backend. (And also add code to restart it automatically, as a failsafe.)

In principle, this frontend/backend separation should also prevent the UI from freezing up, no matter what the scope does (i.e. the "three-finger salute" should never be needed).

[SJL-Instruments]

1) zoomed out showing 46 sweeps, 5ps resolution for 1ns.  I make no attempt to align the channels and we can see that one channel is a bit off compared with the others like I had mentioned.

This is consistent with the specified max THD of the scope at 1 GHz - it's limited by Channel 1 at early times (11-12 ns). All other channels perform better than spec.

2)  Looking at the minimum values of the peak area.  Note how channel 1 is smooth where the others have some sort of pattern to them. 

3) Looking at the maximum, we can see they are all smooth.

If I reset the collection and watch it, I can see when it will create a single mini-glitch.  They are much smaller in amplitude than the original glitches.   This appears very reproducible.   Nothing special with the cables, splitter or RF gen.  Have you seen these smaller spikes?

These effects are from the input hysteresis interacting in a subtle way with the delay generator.
This is what limits the specified noise floor of the scope. The actual RMS noise in the smooth regions is about 5x better than specs (200 uV RMS).
Getting rid of it completely would likely require a different architecture and push the price up significantly.

[joeqsmith]

As it continues to run, it's interesting that channel 1 never exhibits these downward spikes where the other three do.  It never happens in the valley, only the peak.  I would not have noticed it as I was only ever looking at channel 1 with my software until today.   I'll take your word for it that it is the expected behavior from your design perspective.  As a user, I would be concerned it was my signal generator causing this, then spending time hunting down a problem that isn't there.   

[joeqsmith]

*** Sending X1 or X2, it responds with OK Xn.  Sending X1 followed with R, I do not get any data.  There must be more to it.

We were able to get CDF data with the following procedure:
1. Plug in the scope.
2. Issue command X1.
3. Issue command R1 10000 10000 10000 10000 50000 50000 50000 50000.
Does this sequence fail to return data on your unit?

I was using 0 65535 rather than 10000 and 50000.  I also had the first 4 channels enabled.  I tried it with your settings and get the following:

NO TRIG ZERO SJLI

So no difference.  I assume X1 is sent with no other payload and terminated normally.  The fact it returns an OK it seems correct.  If X1 is selected and I supply a trigger, it does trigger normally.  It behaves the same for X0 and X1.   I'm sure I am missing something simple as your software appears to work correctly.

[SJL-Instruments]

So no difference.  I assume X1 is sent with no other payload and terminated normally.  The fact it returns an OK it seems correct.  If X1 is selected and I supply a trigger, it does trigger normally.  It behaves the same for X0 and X1.   I'm sure I am missing something simple as your software appears to work correctly.

That's odd. Are you sending any commands in between? Sending a D command will reset to normal trigger mode (X0) - see our reply #154.
Our software issues exactly this sequence (X1 then R).

[SJL-Instruments]

As it continues to run, it's interesting that channel 1 never exhibits these downward spikes where the other three do.  It never happens in the valley, only the peak.  I would not have noticed it as I was only ever looking at channel 1 with my software until today.   I'll take your word for it that it is the expected behavior from your design perspective.  As a user, I would be concerned it was my signal generator causing this, then spending time hunting down a problem that isn't there.   

The data you posted in #156 is within specified behavior, but a bit on the worse side (~5 mV min-max spread after 50 sweeps). The data in #159 is not within specifications (~15 mV min-max spread).

We are running a test unit in the same configuration as you (~240mVpp @ 1 GHz on all channels), and so far (300 sweeps) our min-max spread on all channels does not exceed 1.1 mV. This is using the interpolation method - doing an error function fit pushes the noise even lower.

Can you send the raw CDF data from some of the worst samples (~15 mV off)? What holdoff and trigger level are you using?

[joeqsmith]

So no difference.  I assume X1 is sent with no other payload and terminated normally.  The fact it returns an OK it seems correct.  If X1 is selected and I supply a trigger, it does trigger normally.  It behaves the same for X0 and X1.   I'm sure I am missing something simple as your software appears to work correctly.

That's odd. Are you sending any commands in between? Sending a D command will reset to normal trigger mode (X0) - see our reply #154.
Our software issues exactly this sequence (X1 then R).

Yes, I always send the D prior to sending R.  Good to know it resets the trigger mode.  May want to consider adding that note to the manual. 

I am trying it now and it appears to be working correctly. 

[joeqsmith]

We are running a test unit in the same configuration as you (~240mVpp @ 1 GHz on all channels), and so far (300 sweeps) our min-max spread on all channels does not exceed 1.1 mV. This is using the interpolation method - doing an error function fit pushes the noise even lower.

I'm using 30 samples, linear fit interpolation . 

Can you send the raw CDF data from some of the worst samples (~15 mV off)?

If I use your software to stream all the data to disk and let it run for a half hour, could you track it down from that?   

[SJL-Instruments]

If I use your software to stream all the data to disk and let it run for a half hour, could you track it down from that?   

Currently the streaming does not support raw CDF data. We'll patch it shortly and send you a version that does.

[joeqsmith]

Installed and running.  I have it recording all four channels.   If you have a way to sift through all of that data, I can send you the worse case ones, or I can compress the whole directory and send it.  I'll let it run a half hour. 

[SJL-Instruments]

Installed and running.  I have it recording all four channels.   If you have a way to sift through all of that data, I can send you the worse case ones, or I can compress the whole directory and send it.  I'll let it run a half hour. 

Thanks. Compressing the whole directory is fine - it provides more statistics anyway.

[joeqsmith]

No problem.  You should have it.

[SJL-Instruments]

We received it - thanks.
Analyzing the saved voltages, the largest min-max spread on any channel, at any time, was 6.7 mV. (This is the max over 119 sweeps * 736 pts/sweep * 4 channels = 366kpts.)
(All the data is plotted in the below images.) Channels 1 and 2 plotted (4-attachment limit).

The residuals (deviation from the average trace) are all within nominal specs. The noise is larger when the signal has a larger slope - this is caused by timing jitter translating into voltage jitter.
The RMS noise was well below 1 mV on all channels.

Overall, we didn't see anything out of the ordinary - no spikes. You can reproduce this analysis if you'd like (voltages are saved as a standard CSV).

I'm using 30 samples, linear fit.

Could you describe precisely how you implemented the linear fit? Is it as described in Section 4.3.2?

[joeqsmith]

It's VERY possible the problem is in my software, which is why I wanted to use yours to collect the data.  Best to divide these problems in half.

I am not fitting the data.  Rather I simply use linear interpolation.  Corrected previous statement, there is no fitter  Using this, with a 1GHz full scale signal applied, I measure -36.6dB for THD.  The published measured value was -37.65 +/-12dB.  Being the lazy person I am, I called it a day.   

Let me try adding a gaussian fit like was first mentioned and see if it improves things.   

***
A new day. 

From the manual:

you can recover the signal voltage V0 by interpolating between the two neighboring voltages where F(V ;Δt) first transitions from below 1/2 to above 1/2.

...
2. If two or more voltages in the CDF are equal, sort the corresponding CDF values (for that voltage value) from lowest to highest. If the interpolation is done in an incorrect order, a large apparent spike may result (example 2, attached).

These considerations do not apply if you directly fit an error function to the CDF data.

We just discovered effect (2) recently, so we may send you an updated firmware file tomorrow that implements this sorting for you.

...

Do you always use a Gaussian fit to get the centroid?

A bit complicated, but details if you're interested: to a first approximation, you can use the interpolation procedure we described in the last post. This usually gets you close, but the noise performance is poor since you're only using two samples.
Fitting a Gaussian error function is theoretically the best thing to do (in the sense of Fisher information), but in practice is overly sensitive to outliers on the wings. What we've found works well in practice is to fit a line to the inverse Gaussian CDF applied to the data, but truncated within a certain region (say 10-90%) of the CDF, with the truncation also seeded by the interpolation method. (This is also faster on the CPU.) You need to weight samples with some care to maintain equal sensitivity to all values (which minimizes the total propagated noise, and improves outlier robustness).

This gives a significantly lower noise floor than the interpolation method alone, but is still robust to spurious errors.

Attached plot shows a typical CDF values (WHITE) that appear out of order.  The GigaWave has all the recent patches.   If I don't sort them, and weigh CDF values that are outside of the 5-95% at 0 (no effect), all others inside the window weighted at 1, then run a linear fit (RED).   Then interpolate the fitted data to find the centroid index.  Use that index and linear interpolation to determine the voltage.  The two numbers are close.    Also shown using 10/90% window, same weighting, with sorting CDF.

First, do I always need to sort the CDF values?  Is there a problem that I am seeing what appears to be out of order data? 

With this calculation being crucial, it seems that for anyone wanting to reproduce the result obtained using your software,  your conversion needs to be well documented. 

[joeqsmith]

Plot comparing the linear and gaussian fit with sorted CDF and 10/90% window, 0 weight for anything outside.  Again, linear interpolation used to locate mid point. 

I have not tried these in my software to see which is more robust.  Rather I would like to replicate what your software does. 

At some point, when I start working on the review (remember the review?),  I am guessing there will be tests I want to conduct that your software does not support.  For example, maybe I decide to show the THD.  Your software does not have this feature built-in.   To automate such measurements, my software will need to replicate your results.   

[SJL-Instruments]

First, do I always need to sort the CDF values?  Is there a problem that I am seeing what appears to be out of order data?

Do not sort the CDF data. The out-of-order points are expected, and result from noise in the signal. (More precisely, the integrated portion of the low-frequency content up to the sampling timescale, Section 2.2.2.)
If you set Nmin=Nmax=N0 (fixed triggers per sample), the noise in the CDF will go to zero as N0 -> infinity.

Plot comparing the linear and gaussian fit with sorted CDF and 10/90% window, 0 weight for anything outside.  Again, linear interpolation used to locate mid point.

I have not tried these in my software to see which is more robust.  Rather I would like to replicate what your software does. 

You should not fit a line or a Gaussian to the CDF data. You should fit a Gaussian error function, of the form:

f(x) = 1/2 (1 + erf(A(x-x0)))

where A and x0 are free parameters. The fitted value of x0 is the extracted voltage.


We've added a section laying this out clearly in the next manual revision (see attached image).
There have been enough updates that we've released revision H6 of the manual, which contains this explanation.

[joeqsmith]

Thanks.  This helps.   

Could you do me a favor and write a simple program that maybe reads one of the streamed CSV files from your software, performs the calculations (identical to how your software works) and shows the result?  I wouldn't care if you released the source or not although it may be helpful to have it.  Basically, I want to prove that we obtain the same results. 

Reminds me of a few years ago, I ran into a similar problem where a coworker was using MatLab's filtfilt function and I was attempting to replicate their algorithms.

https://www.eevblog.com/forum/chat/matlab-zero-phase-digital-filtering-question/msg4429354/#msg4429354

I got lucky in that case as someone had found a document that detailed how it worked.  I did something similar where I fed data through both algorithms to prove out my math. 

***
Another option you may want to consider is where you provide interface libraries for the different OSs you support.  This would give you total control over the algorithms and takes that burden away from customers.   Personally, I see it as a wash, as long as you provide enough detail to implement your exact algorithms. 

Also, personally I like having the PC make all of these calculations rather than the embedded system.  Why not leverage the latest PC technology.  I have to believe it also makes your job much easier by minimizing the complexity of the firmware.  I think about that NanoVNA and just how unstable that firmware was.   

[SJL-Instruments]

Could you do me a favor and write a simple program that maybe reads one of the streamed CSV files from your software, performs the calculations (identical to how your software works) and shows the result?  I wouldn't care if you released the source or not although it may be helpful to have it.  Basically, I want to prove that we obtain the same results. 

Here's an open-source CoLab notebook that implements both the linear interpolation and curve fit methods:
https://colab.research.google.com/drive/1wAReesZiEhuauzbm_8LAfhkgA2IYOyij?usp=sharing

There's a sample file in the notebook if you just want to check your code.
If you want to use your own data, you will need to select "Raw CDF Data" when exporting/streaming to get a .npz file. The CSV files only contain the extracted voltage information (no CDFs).

You might want to check your linear interpolation code as well - we didn't see any spikes even when re-analyzing all our past data using linear interpolation. (The noise was a bit higher, though.)

Another option you may want to consider is where you provide interface libraries for the different OSs you support.  This would give you total control over the algorithms and takes that burden away from customers.   Personally, I see it as a wash, as long as you provide enough detail to implement your exact algorithms. 

Our Python API is cross-platform and implements the curve fitting algorithm. We understand not everyone wants to use Python, though - the description in manual revision H6 is hopefully clear enough to implement yourself.

Also, personally I like having the PC make all of these calculations rather than the embedded system.  Why not leverage the latest PC technology.  I have to believe it also makes your job much easier by minimizing the complexity of the firmware.  I think about that NanoVNA and just how unstable that firmware was.   

That's our philosophy as well. Software is much easier to update, lowers the end-user cost of the hardware, and gives the user more flexibility if they need it (as you've been doing in LabView).