NanoVNA Custom Software

[joeqsmith]

If I really push the data rates, basically asking for the next data set right after is finished sending a data set, without giving it some delay, the performance suffers.

Problem even have a name: multitasking race condition. PC software have to request data but time window while it is available, is very small. As we see, PC manages to miss each other scan data set. Needed change is quite "simple" - leave existing "polling" approach alone and introduce new command for continuous scan and "push" for nano. In short nano while said mode is initiated: 1) performs scan 2) optionally displays it on the built-in LCD 3) sends scan data to PC unconditionally and only when USB transfer fully done, continue with next scan.

But hey, it's $50.

Yes, you can say so when you are talking about tiny screen and fonts, awkward and fragile input "thingie", lack of proper case but such excuse is not OK in case of poor software *design* that clearly can be improved.

Sadly you seem to have ignored:

I don't think this is the problem with why it's so slow.   Without the Nano connected to the PC and just watching it, it will scan about once a second.   This is using the last image you uploaded.     On the PC I am seeing 325467 Bytes in 14.04 seconds or 185kbs.  This is plenty fast to keep up with how slow the Nano scans.

I have no race condition.  My software is synchronized to the Nano by the completion of the first request.   

As we see, PC manages to miss each other scan data set.

As "you" see, not "we".   What I see with the software I am writing, using Radiolistern's image is that my PC easily keeps pace with how slow the Nano sweeps.   

In case you would like to see what I am seeing, I made a short video showing my software plotting data.  The Nano is hung next to the screen so you can watch its' LED compared with the scan indicator of my software.  If you have firmware for the Nano that will sweep faster than this, post details about it and I will see if my PC can somehow manage to keep pace with it. 

https://youtu.be/N77my69u_b8

[radiolistener]

Why PC does get only each 2nd scan then?

Because NanoVNA Saver sends two data requests. First for S11 and second for S21. Each request requires one sweep cycle. So, two request needs two sweep cycles.

That's behavior is specific for NanoVNA Saver. You can request just S11 and it will take single sweep time. But NanoVNA Saver sends two data requests and this is why it takes two sweeps.

I am afraid that you need to update your knowledge about "race condition". Meaning is quite broad BTW.

There is no race condition for data command, because operations are synchronized with mutex.

Why don't you comment my suggestion of PC communications redesign?

I don't see reason to redesign something. Yes, transfer speed can be improved, but I'm too lazy to do something with no real needs. It already works good enough

[5q5r]

That's behavior is specific for NanoVNA Saver.

I don't know that there would be another way to do it if reading both S11 and S21?

[joeqsmith]

Why PC does get only each 2nd scan then?

Because NanoVNA Saver sends two data requests. First for S11 and second for S21. Each request requires one sweep cycle. So, two request needs two sweep cycles.

That's behavior is specific for NanoVNA Saver. You can request just S11 and it will take single sweep time. But NanoVNA Saver sends two data requests and this is why it takes two sweeps.

When I use both data0 & data1, I plot while alternating between the two data sets.  So my screen always updates once per request.   I guess I could have shown that as well.  It would have been just as exciting to watch. 

 

That's totally wrong with my current software.  I plot once after both data sets have been collected.

 

Wrong again....  It's actually the way I originally described.     Time for some wake up juice. 

[joeqsmith]

As a totally off subject, I happened to listen to the last Amp hour podcast.  Anytime they mention RF in the subject with both Dave and Chris, I'll have a listen.    Oddly, they are talking about VNAs.  Chris talks about getting the new Siglent one and they discuss the merits of the old 8753ES vs the new Siglent.  (I almost pulled the trigger on a 6Ghz ES to replace my old 8754A).    Anyway, to my surprise, even with it being as popular as it is, there wasn't a single mention of the NanoVNA.   

If I were buying my first ever VNA and was just getting started with RF, I would hands down spend the $50 on the Nano to at least learn some of the basics.   I wouldn't be surprised to find out that it wasn't the most sold VNA ever! 

[5q5r]

I don't know that I would say it's off-topic.

My guess is that there are two main reasons: The NanoVNA isn't being marketed, but only spreading via word of mouth, mainly among enthusiasts in the amateur radio community. And there is a preference, approaching a fetish at times, for expensive test gear from the big name brands among some parts of the electronics community

The NanoVNA is my first VNA, and I have no desire to buy anything more expensive than it, given the (limited) uses I have for it.

[xrunner]

If I were buying my first ever VNA and was just getting started with RF, I would hands down spend the $50 on the Nano to at least learn some of the basics.   I wouldn't be at all surprised to find out that it wasn't the highest sold VNA ever!

It's selling like hotcakes in my town as far as the local hams. A few of us got them and started talking about the NanoVNA. Most people didn't have a clue what a "vector network analyzer" did. But after an explanation and some in-person demos it's caught on Big Time. Companies like MFJ who sell antenna analyzers for hams here -

MFJ Analyzers

are in trouble because they will not be able to compete with something like the Nano for < $50.

[ogden]

As "you" see, not "we".

It's "you", not me/we I talk about NanoVNA Saver and it was clearly said BTW.

Because NanoVNA Saver sends two data requests. First for S11 and second for S21. Each request requires one sweep cycle. So, two request needs two sweep cycles.

Well, I did not know that. It makes sense then indeed. So how does NanoVNA Saver and other PC software can get Both - S11 and S21 using single request? - If there is no command then perhaps you can make one? 

[joeqsmith]

It's selling like hotcakes in my town as far as the local hams. A few of us got them and started talking about the NanoVNA. Most people didn't have a clue what a "vector network analyzer" did. But after an explanation and some in-person demos it's caught on Big Time. Companies like MFJ who sell antenna analyzers for hams here -

MFJ Analyzers

are in trouble because they will not be able to compete with something like the Nano for < $50.

That doesn't surprise me at all.   If anything, I would have expected him to plug his own site.  I find it hard to believe Dave's not aware of this threads' popularity. 

[ogden]

Companies like MFJ who sell antenna analyzers for hams here -
MFJ Analyzers
are in trouble because they will not be able to compete with something like the Nano for < $50.

Right, they should be worried Is it even legal to use square wave signal for antenna tuning? - Harmonics violate out-of band radiation regulations for sure. It's kinda "spread spectrum" unless CW mode used, but anyway. Many of you are HAMs, right? Perhaps you can comment? - Just curious.

[xrunner]

Right, they should be worried Is it even legal to use square wave signal for antenna tuning? - Harmonics violate out-of band radiation regulations for sure. It's kinda "spread spectrum" unless CW mode used, but anyway. Many of you are HAMs, right? Perhaps you can comment? - Just curious.

What does it output - something like -10 dBm, which is 100 uW? Maybe even less.

I just connected the NanoVNA to a 30m dipole outside, which is just under another fan dipole with the same band included (I have a lot of antennas ). I set the Nano to sweep from 10.1 to 10.150 MHz. I listened to the sweeps on an Icom IC-7300 and you can indeed hear the sweeping (and see it on the waterfall). I then connected the radio to my other main fan dipole which is about 30 feet away - the amplitude is already falling off considerably in that range.

However - I tuned through other ham bands (and other ranges in the shortwave bands) and could not hear any sweeping at all outside of the set sweep range. So, even though there may be sidebands out there, they are too low to detect with a shortwave radio

[radiolistener]

Well, I did not know that. It makes sense then indeed. So how does NanoVNA Saver and other PC software can get Both - S11 and S21 using single request? - If there is no command then perhaps you can make one? 

yes, technically there is no command which allows to read S11 and S21 simultaneously. And if you want both, you will need to send two requests and it will take two sweeps. It is possible to add such ability, but current PC software don't know about new features, so they will continue to use old command and it will still take two sweeps

This makes a vicious circle. Firmware don't have such command because it is not used in software. And software is unable to do that because there is no such command in firmware

Sending both S11 and S21 is much slower. And there is no needs for S11 and S21 simultaneously. So, I don't see the real needs for command which can send information about both channel simultaneously. If you needs S11 and S21 you can send two data commands. And if you needs just S11 or just S21, you can send single data command. It works pretty good.

[radiolistener]

Is it even legal to use square wave signal for antenna tuning? - Harmonics violate out-of band radiation regulations for sure. It's kinda "spread spectrum" unless CW mode used, but anyway. Many of you are HAMs, right? Perhaps you can comment? - Just curious.

NanoVNA has low power output. The maximum peak power from NanoVNA is about 10 dBm and it is even much smaller on high frequency. This is about 0.01 W power. The average NanoVNA power is about -10 dBm, it is about 0.0001 W. In most countries it is allowed to use such power with no license.

HAMs in most cases using resonant antennas, which actually works as effective bandpass filter due to it's high Q-factor. So, it radiates at ham bands and has very low efficiency outside HAM bands. If you use wideband antenna it will have low efficiency due to low Q needed for wide bandwidth.

As you can see, weak signal from NanoVNA can be radiated at HAM bands and it's power as low, so it cannot affect anything.

[Kalvin]

Possibly - though to confirm, you should probably do multiple sweeps with high averaging. If the dips (or peaks) stay in place, it's a "true" error, if they move around, maybe the averaging isn't good enough to catch it, or you have indeed found a bug that I just haven't recognized ;-)

Everything is fine, there is no bugs. Phew. Three 20x averages of 20dB reflection, they all look basically the same. Conclusion: to get meaningful averaging results, same level of averaging for calibration data is necessary. Now averaging just reveals calibration noise.

Yes, I was going to ask whether the calibration is using the averaging also. If not, then the averaging during actual measurement doesn't really make sense as the calibration data is noisy anyway.

Edit: Probably the calibration should be done using at least twice the averaging compared to the averaging used during the actual measurement.

[ogden]

And there is no needs for S11 and S21 simultaneously.

Nonsense. You again invent strange (to say it politely) arguments out of nowhere, just for argument sake. Not mentioning big VNA's, even Nanovna itself measure S11 and S21 in single scan pass and display both, yet you just decided for everybody that PC application do not need such feature, at least not as fast as nanovna *hardware* allows.    FYI it is very convenient to simultaneously see "both ends" of the filter or diplexer or whatever >=2 terminal thing you measure, especially if you tune it. Then it may be even necessity.

NanoVNA has low power output. The maximum peak power from NanoVNA is about 10 dBm and it is even much smaller on high frequency. This is about 0.01 W power. The average NanoVNA power is about -10 dBm, it is about 0.0001 W. In most countries it is allowed to use such power with no license.

Yes. When transmission is within amateur band. I was talking about out of band spurious emissions that are products of square wave. Let's check FCC 47 Radio Amateur Service regulations, §97.307.(e):

on a frequency between 30–225 MHz must be at least 60 dB below the mean power of the fundamental. For a transmitter having a mean power of 25 W or less, the mean power of any spurious emission supplied to the antenna transmission line must not exceed 25 μW and must be at least 40 dB below the mean power of the fundamental emission, but need not be reduced below the power of 10 μW.

10uW is -20dBm, 25uW is -16dBm. What is level of 3rd harmonic when fundamental of square wave is at -10dBm?

[5q5r]

When NanoVNA-Saver reads a segment, it reads frequencies, S11, S21. For those running a newer edy555 firmware (0.2.0+) supporting the "scan" function, the command sequence is as follows:

scan 50000 900000000 101
frequencies
data 0
data 1

(Example there for 50 kHz - 900 MHz)

The timing here is as follows, examples from my own NanoVNA:
0.412s: Sending scan command
0.551s: Asked for frequencies
1.320s: Finished receiving frequencies  <-- This is where the application primarily waits for the device
1.422s: Asked for S11
1.510s: Done reading S11
1.613s: Asked for S21
1.708s: Done reading S21

Total time spent 1.296 seconds plus the processing done in NanoVNA-Saver. Looks like the time between the app saving data is 1.303, so the application uses 7 milliseconds for its internal processing between segments.

I have tried moving the frequencies to be read later, but the pattern is clear that the first data requested after the "scan" command is slowed down by about 6-800ms.

After each command, the application waits 50ms before it starts emptying the buffer. I doubt that this moves the timings significantly, as emptying the buffers takes more than 50ms in all cases, and the PC speed is not the limit.

I don't know how this looks for different firmwares - I only have the one NanoVNA, and I'm not going to flash all kinds of different firmware on it, as I need it for app development However: For devices that do *not* support the scan command, the application sleeps for a full second after setting the sweep span, to allow the NanoVNA to sweep the frequencies requested. Prior to doing this (when the timing was 300ms), I would sometimes see values from the previous frequency span repeated. It was faster, but less reliable - and I made the decision that reliable is more important than fast. The code is there for anyone who wants to change it for their own priorities

[joeqsmith]

When NanoVNA-Saver reads a segment, it reads frequencies, S11, S21. For those running a newer edy555 firmware (0.2.0+) supporting the "scan" function, the command sequence is as follows:

scan 50000 900000000 101
frequencies
data 0
data 1

(Example there for 50 kHz - 900 MHz)
.....

I don't know how this looks for different firmwares - I only have the one NanoVNA, and I'm not going to flash all kinds of different firmware on it, as I need it for app development However: For devices that do *not* support the scan command, the application sleeps for a full second after setting the sweep span, to allow the NanoVNA to sweep the frequencies requested.
...

When I first noticed the Scan command someone was posting how it worked and I was interested in looking at it but it was disabled in the Hugen code I downloaded.   What was interesting to me is that it sent up the frequency, S11 and S21 data in one call.  Plus it would handle more than the 101 points.   I had hoped it would help speed things up.      While I archived the details how it worked, when I tried to use it with the last image from Radiolistener, it was nothing like what had been presented.  It seems like there were different people creating the same commands.   

Once again, the lack of documentation for the firmware makes it less appealing at least for me at least to try it out.   In the case of the Scan command that's implemented in this particular load,  I suspect it is similar to what you have shown.    The original documentation I have shows the command not applying the calibration.   You don't mention this detail, and I would be guessing and using trial and error.   Certainly one way to develop code.

Their Scanraw may be of more interest to me as it supports some sort of averaging which I don't currently support because of how slow the Nano is.   Currently I will run a smoothing filter across each sweep.  I can easily do this and keep up with that ultra fast 1Hz rate.      I showed a plot and a link to an article about it here:

https://www.eevblog.com/forum/rf-microwave/nanovna-custom-software/msg2703604/#msg2703604


Comment I had archived about the Scan command: 

I added one command to the eddy firmware to enable on demand scans of
arbitrary length (yes, you can scan with one million steps or much more if you want)

Usage:
First pause the continuous scanning with "pause" and the use the "scan" command
to scan [from frequency in Hz] [increment frequency in Hz] [number of steps]
The frequency increment step is for now an integer
The scan command outputs

start
frequency s11_real s11_imag s21_real s21_imag
done

during the scan the calibration is NOT used so the output are uncalibrated numbers
allowing alternative calibration strategies

Example:
ch> pause
ch> scan 5000000 20 5
ch> start
5000000 0.001503840 0.000420701 -0.306770563 0.018568072
5000020 0.000695601 0.000503197 -0.306792527 0.018579231
5000040 0.000532656 0.000520238 -0.306793421 0.018573865
5000060 0.000495833 0.000512704 -0.306819111 0.018593480
5000080 0.000520689 0.000523833 -0.306812644 0.018576323

[ogden]

Good. Seems like least effort solution to PC software "individual data request for each port which results in 2x slower scanning" problem is - just make new data request command ("data01" ?) that returns data for both ports. As we see format is already there, at least on Nano side. Obviously scan shall include NanoVNA calibration corrections.

5000000 0.001503840 0.000420701 -0.306770563 0.018568072
5000020 0.000695601 0.000503197 -0.306792527 0.018579231
5000040 0.000532656 0.000520238 -0.306793421 0.018573865

[OwO]

I'm currently working on porting the firmware to the V2 and I can tell you it's a cesspool of bad design, very typical of embedded code. Implicit function calls everywhere (calling a function without including a header file that contains the declaration, and usually there is no declaration at all), everything communicates through globals, no clear dependency relationship between files, hardcoded callbacks everywhere, compiler warnings seem to be all ignored (because I found a handful of bugs just by looking at the warnings), functions just call each other with no regard to which module it is in (e.g. the si5351 driver accesses global variables declared in main.c), forward declarations in the *caller*'s file, and of course I found one that had the wrong signature. The UI code directly manipulates the ADC FFS.

So far I've decided to simply refractor and integrate the UI code into my own firmware that implements all the VNA functionality from scratch. Of course, the command interface is completely replaced so compatibility is broken and you can get far higher sweep rates. Here is the firmware being worked on: https://github.com/nanovna/NanoVNA-V2-firmware-new

[radiolistener]

For those running a newer edy555 firmware (0.2.0+) supporting the "scan" function, the command sequence is as follows:

scan 50000 900000000 101
frequencies
data 0
data 1

you can do it in the following way:

scan 50000 900000000 101
frequencies
data 0
data 1
data 0
data 1
data 0
data 1
data 0
data 1
data 0
data 1
...

I don't know how this looks for different firmwares - I only have the one NanoVNA, and I'm not going to flash all kinds of different firmware on it, as I need it for app development

that's your mistake and the source of bugs. There is no risk for firmware update, except that you may loss your configuration settings and calibration and will needs to calibrate it from scratch after firmware update. But it takes not so long time.

Probably you're using very old firmware, it has race condition issues, high noise floor, high spikes, frequency step rounding error, frequency range rounding error, data transfer error, floating point precision loss and a bunch of other bugs. I highly recommend to use more new firmware.

However: For devices that do *not* support the scan command, the application sleeps for a full second after setting the sweep span, to allow the NanoVNA to sweep the frequencies requested. Prior to doing this (when the timing was 300ms), I would sometimes see values from the previous frequency span repeated.

such behavior was possible in the old firmware which has race conditions issues and very unstable.

Now I understand the reason why edy555 added scan command. The difference between sweep and scan commands is just that scan command is synchronized with sweep and waits for scan end before executing next command. The sweep command is not synchronized and don't wait's. That's the difference.

That's why you have long delay on frequencies command executed immediately after scan command. You will needs to wait while scan command execution will be completed (it needs to wait for sweep end).

Now I see that using sweep command very often is not thread safe and may leads to data transfer errors. I will think about it and how to fix it.

By the way, NanoVNA-Q has scan command from edy555 version. So you can use it if you needs scan command. Technically all fixes from edy555 branch are included in the NanoVNA-Q firmware, include these which still not released and will be released in the future release of firmware. In addition it also has some fixes from hugen79 branch. Also NanoVNA-Q has a lot of bug fixes.

[ogden]

it's a cesspool of bad design, very typical of embedded code

Right. It is usually in case of hobby projects that are started by hobbyists who do not foresee that their creation will ever become widely popular. We shall just live with that.

[radiolistener]

I'm currently working on porting the firmware to the V2 and I can tell you it's a cesspool of bad design, very typical of embedded code. Implicit function calls everywhere (calling a function without including a header file that contains the declaration, and usually there is no declaration at all), everything communicates through globals, no clear dependency relationship between files, hardcoded callbacks everywhere, compiler warnings seem to be all ignored (because I found a handful of bugs just by looking at the warnings), functions just call each other with no regard to which module it is in (e.g. the si5351 driver accesses global variables declared in main.c), forward declarations in the *caller*'s file, and of course I found one that had the wrong signature. The UI code directly manipulates the ADC FFS.

уeah, it has poor design. There is also variable which actually #define for struct member access. It very confusing when you debugging the code

The most of warnings, except these about goto statement and floating point conversion, they are fixed in NanoVNA-Q branch.

I also thought about refactoring, but it will take a lot of time. So, I just fixed critical issues in the code and that's it

[radiolistener]

Comment I had archived about the Scan command: 

this scan command was invented by @erikkaashoek, but it is was rejected by @edy555. He implemented scan command as a thread safe version of sweep command.

So, now scan command is just a thread safe replacement for sweep command. And it cannot sweep more than 101 point.

This is why scanraw command was introduced in the NanoVNA-Q. It has the same principle as @erikkaashoek, but a little optimized and improved.

scanraw don't returns frequency, because frequency can be easily calculated by adding step frequency on each step. And it allows to perform average measurement.

But scanraw don't apply calibration to the measurement results, it just returns raw uncalibrated measurements.

If you're planning to add the same command with calibration apply, I suggest to use name scancal for such command.

[5q5r]

you can do it in the following way:

That would just give me the same data over and over. The NanoVNA stops sweeping after sending the scan command.

Even if it did not, my most common use case is to sweep one segment, then the next, then the next, etc.

that's your mistake and the source of bugs.

Good, we got that squared away then.

Probably you're using very old firmware

I think my post makes it quite clear that I have coded the particular sequence for firmwares newer than 0.2.0.  I'm running 0.2.3.

However: For devices that do *not* support the scan command, the application sleeps for a full second after setting the sweep span, to allow the NanoVNA to sweep the frequencies requested. Prior to doing this (when the timing was 300ms), I would sometimes see values from the previous frequency span repeated.

such behavior was possible in the old firmware which has race conditions issues and very unstable.

Yes, that's where the delay is used.

By the way, NanoVNA-Q has scan command from edy555 version. So you can use it if you needs scan command. Technically all fixes from edy555 branch are included in the NanoVNA-Q firmware, include these which still not released and will be released in the future release of firmware. In addition it also has some fixes from hugen79 branch. Also NanoVNA-Q has a lot of bug fixes.

How would I programmatically recognize a NanoVNA-Q firmware? Which versions have the scan command? Is it from a specific version number and up? Is it fully compatible with the original firmware, or would I have to make special handling for it?

[joeqsmith]

......

By the way, NanoVNA-Q has scan command from edy555 version. So you can use it if you needs scan command. Technically all fixes from edy555 branch are included in the NanoVNA-Q firmware, include these which still not released and will be released in the future release of firmware. In addition it also has some fixes from hugen79 branch. Also NanoVNA-Q has a lot of bug fixes.

How would I programmatically recognize a NanoVNA-Q firmware? Which versions have the scan command? Is it from a specific version number and up? Is it fully compatible with the original firmware, or would I have to make special handling for it?

I use the Help command and check the list of supported commands before trying to test them.    I would imagine the Info command could be used to narrow it down further. 

While my software will work with the six or so images I have tried (at least to the degree that the firmware works), because I don't use the Scan command,  I can't comment on if that will hinder you or not.