TM4313 GPSDO: strange behavior after a night of poor reception

[shapirus]

I'm new to the GPSDO topic.

I bought a GPSDO during an Aliexpress sale out of curiosity, and it later turned out to be (per reviews) one of decent enough models, known as TM4313.

When the antenna is placed outside (it's enough for it to be just outside), it seems to work as expected: it starts receiving the GNSS signal(s) rather quickly, then switches to the "tracking" state in a few minutes, and then in some more minutes (~20-ish) it switches to what seems to be "fine" tracking, as described in more details at https://tomverbeure.github.io/2023/07/09/TM4313-GPSDO-Teardown.html

Once it is in this tracking state, it agrees to within a fraction of Hz with my TinySA Ultra's calibration output, and once both are warmed up, the frequency difference remains stable at 0.3-0.4 Hz or ~30-40 ppb. Since they both agree so well, and looking at the status messages received over the serial (to USB) interface, I assume that it is this state of the GPSDO that is correct.

Now, I decided to let it run overnight and moved the antenna inside the room, leaving it stuck to the window with some scotch tape (later in the morning I found that the tape failed and the antenna was lying on the window sill). I don't know how many satellites the GPSDO can see in either of those locations, but my smartphone barely sees one or two when placed right on the window glass inside, and a lot of them once just outside.

After a night spent with the antenna being inside, the messages from the controller (I changed the jumpers to receive the microcontroller's messages instead of the GPS module's messages over the serial-to-USB interface) looked way off, like this:

Code: [Select]

Ph =      4760, D = 1039687, GL = 0, LOS = 3, ST =     78213, AF =  37, RT = 78519
Ph =      4900, D = 1058441, GL = 0, LOS = 3, ST =     78214, AF =  37, RT = 78520
Ph =      4901, D = 1077805, GL = 0, LOS = 3, ST =     78215, AF =  37, RT = 78521
Ph =      4759, D = 1097213, GL = 0, LOS = 3, ST =     78216, AF =  37, RT = 78522
Ph =      4604, D = 1116081, GL = 0, LOS = 3, ST =     78217, AF =  37, RT = 78523
...
Ph =       281, D = 1162668, GL = 0, LOS = 3, ST =     78255, AF =  37, RT = 78561
Ph =       144, D = 1153634, GL = 0, LOS = 3, ST =     78256, AF =  37, RT = 78562
Ph =         6, D = 1144387, GL = 0, LOS = 3, ST =     78257, AF =  37, RT = 78563
Ph =      -136, D = 1134960, GL = 0, LOS = 3, ST =     78258, AF =  37, RT = 78564
Ph =      -280, D = 1125348, GL = 0, LOS = 3, ST =     78259, AF =  37, RT = 78565
...
Ph =     -5755, D = 1051665, GL = 0, LOS = 3, ST =     78293, AF =  37, RT = 78599                       
Ph =     -5918, D = 1027427, GL = 0, LOS = 3, ST =     78294, AF =  37, RT = 78600                       
Ph =     -5943, D = 1002474, GL = 0, LOS = 3, ST =     78295, AF =  37, RT = 78601                       
Ph =     -5826, D =  977371, GL = 0, LOS = 3, ST =     78296, AF =  37, RT = 78602                       
Ph =     -5700, D =  952698, GL = 0, LOS = 3, ST =     78297, AF =  37, RT = 78603                       
...
Ph =      -254, D =  937942, GL = 0, LOS = 3, ST =     78377, AF =  37, RT = 78683
Ph =      -139, D =  945679, GL = 0, LOS = 3, ST =     78378, AF =  37, RT = 78684
Ph =       -23, D =  953570, GL = 0, LOS = 3, ST =     78379, AF =  37, RT = 78685
Ph =        95, D =  961609, GL = 0, LOS = 3, ST =     78380, AF =  37, RT = 78686
Ph =       216, D =  969799, GL = 0, LOS = 3, ST =     78381, AF =  37, RT = 78687
...


The "Ph" (phase difference?) parameter oscillated up and down, never converging to zero, the "D" parameter was always way above normal (see below), and the AF parameter remained at the value indicating what seems to be "coarse" tracking. Weirdly enough, it never thought it lost tracking, which is indicated by the ST parameter, which means seconds since tracking started or stopped (and RT means seconds since power-on).

Most importantly, the frequency it produced in this state was way off, at least tens of Hz difference against the TinySA. I also suspect that the OCXO used in my unit is not the best, as the frequency quickly wanders off once GPS tracking is lost, but that's a different story.

Things did not change when I moved the GPS antenna outside. It still could not catch up after ~30 min or so.

Then I disconnected and reconnected the antenna.


It promptly detected the loss of signal (GL=1):

Code: [Select]

Ph =     -3064, D =  738791, GL = 0, LOS = 3, ST =     79728, AF =  37, RT = 80034
Ph =     -3005, D =  742375, GL = 0, LOS = 3, ST =     79729, AF =  37, RT = 80035
Ph =     -3005, D =  745982, GL = 1, LOS = 3, ST =     79730, AF =  37, RT = 80036
Ph =     -3005, D =  745982, GL = 1, LOS = 4, ST =         0, AF =  37, sK = 2.06052, RT = 80037
Ph =     -3005, D =  942403, GL = 1, LOS = 4, ST =         1, AF =  37, RT = 80038
Ph =     -3005, D =  942401, GL = 1, LOS = 4, ST =         2, AF =  37, RT = 80039


Then it, as it would normally do on power-on, started to receive the GPS signal (GL=1 transitioning to GL=0):

Code: [Select]

Ph =     -1343, D =  942376, GL = 1, LOS = 4, ST =        14, AF =  37, RT = 80051
Ph =     -1228, D =  942374, GL = 1, LOS = 4, ST =        15, AF =  37, RT = 80052
Ph =     -1110, D =  942372, GL = 0, LOS = 4, ST =        16, AF =  37, RT = 80053
Ph =      -996, D =  425986, GL = 0, LOS = 2, ST =         0, AF =  37, RT = 80054
Ph =      -951, D =  405986, GL = 0, LOS = 2, ST =         0, AF =  37, RT = 80055


-- at this point the 10 MHz frequency output becomes very close to the TinySA.


Then it starts tracking (LOS=2 -> LOS=3, ST resets, Ph fluctuates close to zero):

Code: [Select]

Ph =     -3774, D =  565986, GL = 0, LOS = 2, ST =        40, AF =  37, RT = 80182
Ph =     -3770, D =  565986, GL = 0, LOS = 2, ST =        41, AF =  37, cK = 0.003312E-9/LSB, RT = 80183
Ph =     -3764, D =  550892, GL = 0, LOS = 2, ST =        42, AF =  37, RT = 80184
Ph =     -3761, D =  550892, GL = 0, LOS = 2, ST =        43, AF =  37, RT = 80185
Ph =     -3760, D =  550892, GL = 0, LOS = 2, ST =        44, AF =  37, RT = 80186
Ph =     -3761, D =  550892, GL = 0, LOS = 2, ST =        45, AF =  37, RT = 80187
Ph =     -3760, D =  550892, GL = 0, LOS = 2, ST =        46, AF =  37, RT = 80188
Ph =         0, D =  550892, GL = 0, LOS = 3, ST =         0, AF =  37, RT = 80189
Ph =        -1, D =  550892, GL = 0, LOS = 3, ST =         1, AF =  37, RT = 80190
Ph =         0, D =  550892, GL = 0, LOS = 3, ST =         2, AF =  37, RT = 80191


-- at this point the 10 MHz frequency has the same difference measured earlier against the TinySA and the difference remains stable.

And then, finally, it switches to what seems to be "fine" tracking state (AF=37 -> AF=73, and the steps of the changes in D become very small):

Code: [Select]

Ph =         1, D =  550704, GL = 0, LOS = 3, ST =       997, AF =  37, RT = 81186
Ph =         0, D =  550704, GL = 0, LOS = 3, ST =       998, AF =  37, RT = 81187
Ph =        -1, D =  550704, GL = 0, LOS = 3, ST =       999, AF =  73, RT = 81188
Ph =         0, D =  550704, GL = 0, LOS = 3, ST =      1000, AF =  73, RT = 81189

No observable output frequency change at this point.


Now, just by looking at the output, I think it's safe to assume that this (the latest messages) is its normal state, and what I saw after a night of potentially poor signal reception and/or low number of satellites, was not. It seems that it got locked up in some bad state and needed an antenna reset to come out of it back to normal. Would it be able to resolve that bad state on its own with the antenna placed outside? I don't know, I didn't have the patience to wait.

What was it that I observed? Was it expected? There's not much information on the internet about this device, but I believe there are a few people here on this forum that own it, so maybe they have some thoughts, or maybe some general GPSDO considerations may be applicable.

[tverbeure]

There's very little literature on the TM4313 other than my blog post, and other than you and me, I don't think anyone has gone father in their analysis of the TM4313 log messages.

I'm surprised by the frequency difference of 0.3-0.4Hz that you're observing. That's more than I'm seeing. This weekend, as part of a new blog post (https://tomverbeure.github.io/2024/04/06/Guide-Tech-GT300-Frequency-Reference-Teardown.html), I measured the clock of free-running frequency standard against the TM4313 and got a precision of around 1mHz (see video at the bottom of the page). This is with the TM4313 being in lock for more than a day.

Earlier, I also measured the TM4313 against the 10MHz output of a modded SyncServer S200 that was in GPS lock. They are within 20mHz of each other, but the SyncServer S200 has a larger variation than the TM4313 when measured against the GT300 reference.

A retired HP veteran who was involved in the design of a bunch of frequency counters told me that GPSDOs are extremely tedious to work on because of the long time constants. It's one of the reasons why I decided to buy the 4313 instead of making one myself. You're kind of running into the same issue: "...I didn't have the patience to wait." 

It'd be interesting to see if you can reproduce it and check if it recovered eventually after getting a decent GPS signal again.

Tom

[shapirus]

There's very little literature on the TM4313 other than my blog post, and other than you and me, I don't think anyone has gone father in their analysis of the TM4313 log messages.

Of course I'm not even surprised to find you on this forum. Where else could I meet the author of a blog post about a very specialized piece of equipment? :)

I'm surprised by the frequency difference of 0.3-0.4Hz that you're observing. That's more than I'm seeing. This weekend, as part of a new blog post (https://tomverbeure.github.io/2024/04/06/Guide-Tech-GT300-Frequency-Reference-Teardown.html), I measured the clock of free-running frequency standard against the TM4313 and got a precision of around 1mHz (see video at the bottom of the page).

Well, the TinySA Ultra has never been supposed to serve as a frequency standard. It's a cheap device whose primary function is SA. I'd say it's surprising that the difference is so small, as it's a budget-constrained device and it has "only" a TCXO for the clock generator which isn't synced to any external reference. So it's quite stable and is well trimmed on the factory. I wonder if it's possible to readjust it btw, using the GPSDO as a reference. Given how stable it is, it might make sense to trim that 0.3 Hz difference further to make it match the GPSDO perfectly. That doesn't matter much for the SA part with its minimum RBW of 200 Hz, but it would be nice for the calibration output to provide an even better frequency reference (of course if we assume that the GPSDO is more accurate).

I have no other frequency reference to compare these two against.

A retired HP veteran who was involved in the design of a bunch of frequency counters told me that GPSDOs are extremely tedious to work on because of the long time constants. It's one of the reasons why I decided to buy the 4313 instead of making one myself. You're kind of running into the same issue: "...I didn't have the patience to wait."  :D

You know how it goes. Start with a simple ready made device, then develop a feeling that you need something better, decide that it might be possible to do it yourself...

It'd be interesting to see if you can reproduce it and check if it recovered eventually after getting a decent GPS signal again.

Yes I will check it. I won't try to cause it deliberately this night, though, because I want to see if it changes its state after tracking for more than 24 hours (the datasheet says, or I read it elsewhere, that the track LED has to start flashing less frequently then) and whether the frequency difference against the TinySA Ultra at that point will change.
GPS reception seems to become worse during the night, though, so it may happen on its own again, so I'll see how it goes this time.


Now the antenna is inside, stuck on the window glass again, and it's changing between good:

Code: [Select]

Ph =         0, D =  550850, GL = 0, LOS = 3, ST =     33590, AF =  73, RT = 113779
Ph =         0, D =  550848, GL = 0, LOS = 3, ST =     33591, AF =  73, RT = 113780
Ph =         1, D =  550848, GL = 0, LOS = 3, ST =     33592, AF =  73, RT = 113781
Ph =        -1, D =  550844, GL = 0, LOS = 3, ST =     33593, AF =  73, RT = 113782
Ph =         0, D =  550844, GL = 0, LOS = 3, ST =     33594, AF =  73, RT = 113783
Ph =        -1, D =  550844, GL = 0, LOS = 3, ST =     33595, AF =  73, RT = 113784
Ph =        -1, D =  550846, GL = 0, LOS = 3, ST =     33596, AF =  73, RT = 113785
Ph =        -1, D =  550848, GL = 0, LOS = 3, ST =     33597, AF =  73, RT = 113786
Ph =         0, D =  550850, GL = 0, LOS = 3, ST =     33598, AF =  73, RT = 113787


...and not so good, but not bad either:

Code: [Select]

Ph =         4, D =  550643, GL = 0, LOS = 3, ST =     33681, AF =  37, RT = 113870
Ph =         4, D =  550634, GL = 0, LOS = 3, ST =     33682, AF =  37, RT = 113871
Ph =         4, D =  550626, GL = 0, LOS = 3, ST =     33683, AF =  37, RT = 113872
Ph =         4, D =  550617, GL = 0, LOS = 3, ST =     33684, AF =  37, RT = 113873
Ph =         4, D =  550609, GL = 0, LOS = 3, ST =     33685, AF =  37, RT = 113874
Ph =         3, D =  550600, GL = 0, LOS = 3, ST =     33686, AF =  37, RT = 113875
Ph =         3, D =  550596, GL = 0, LOS = 3, ST =     33687, AF =  37, RT = 113876
Ph =         3, D =  550591, GL = 0, LOS = 3, ST =     33688, AF =  37, RT = 113877


but then goes back to good again:

Code: [Select]

Ph =         0, D =  550718, GL = 0, LOS = 3, ST =     33774, AF =  37, RT = 113963
Ph =         0, D =  550726, GL = 0, LOS = 3, ST =     33775, AF =  37, RT = 113964
Ph =         0, D =  550734, GL = 0, LOS = 3, ST =     33776, AF =  37, RT = 113965
Ph =         0, D =  550741, GL = 0, LOS = 3, ST =     33777, AF =  37, RT = 113966
Ph =         0, D =  550749, GL = 0, LOS = 3, ST =     33778, AF =  73, RT = 113967
Ph =         0, D =  550749, GL = 0, LOS = 3, ST =     33779, AF =  73, RT = 113968
Ph =         0, D =  550753, GL = 0, LOS = 3, ST =     33780, AF =  73, RT = 113969
Ph =         0, D =  550757, GL = 0, LOS = 3, ST =     33781, AF =  73, RT = 113970


Btw note how the AF values in the "coarse" state are different in your unit and mine: 19 vs 37.

The OCXO is also different, mine definitely doesn't have the logo yours has, but unfortunately I didn't take a good picture when I opened it. Will do when I power it off.

[nctnico]

Earlier, I also measured the TM4313 against the 10MHz output of a modded SyncServer S200 that was in GPS lock. They are within 20mHz of each other, but the SyncServer S200 has a larger variation than the TM4313 when measured against the GT300 reference.

For proper GPSDO testing you'll need 3 units (2 known-good) and compare against eachother using time interval counters. Or use a Cesium clock to be absolutely sure. Either way you'll need to measure over a 48 hour period so you have the GPS satellites fly in about the same formation over 2 times.

@shapirus: are you certain there is no GPS jamming going on in your region?

[tverbeure]

> For proper GPSDO testing you'll need 3 units (2 known-good) and compare against each other using time interval counters. Or use a Cesium clock to be absolutely sure. Either way you'll need to measure over a 48 hour period so you have the GPS satellites fly in about the same formation over 2 times.

I have a Symmetricom OT-21. It has a GPSDO with a Rubidium clock, but it's used for telecom timing and doesn't have a 10MHz or 1PPS output. So I first need to create a small interposer PCB that brings out those 2 signals from the Rb module, and buffers them up. (It's also annoying that they need to be powered with -48VDC.)

Anyway, the question still remains then: if you have 3 GPSDOs, how to do know that they're good? And also, what kind of measurements and post-processing does one typically make?

[shapirus]

Either way you'll need to measure over a 48 hour period so you have the GPS satellites fly in about the same formation over 2 times.

I thought GPS satellites were geostationary. I've just learned that they aren't. Good :).

@shapirus: are you certain there is no GPS jamming going on in your region?

I'm almost certain there is ;)
That's what I'm attributing the late evening and night reception degradation to.

During the day, however, even on the inner side of the windows, reception is solid. I want to try setting up an antenna on the outside permanently, but for that I'd need to buy a waterproof one (speaking of which, is there anything suitable on Aliexpress that's been verified by someone already?) and also buy a piece of flat RF cable that won't be damaged by the closing window to connect it.

[nctnico]

> For proper GPSDO testing you'll need 3 units (2 known-good) and compare against each other using time interval counters. Or use a Cesium clock to be absolutely sure. Either way you'll need to measure over a 48 hour period so you have the GPS satellites fly in about the same formation over 2 times.

I have a Symmetricom OT-21. It has a GPSDO with a Rubidium clock, but it's used for telecom timing and doesn't have a 10MHz or 1PPS output. So I first need to create a small interposer PCB that brings out those 2 signals from the Rb module, and buffers them up. (It's also annoying that they need to be powered with -48VDC.)

Anyway, the question still remains then: if you have 3 GPSDOs, how to do know that they're good? And also, what kind of measurements and post-processing does one typically make?

As I wrote before: two need to be known-good units. These don't need to be expensive but verified by others that they work. These should show similar performance;  if you derive the 1PPS from the 10MHz, the 1PPS shouldn't drift more than a few tens of ns over a 48 hour period.

[shapirus]

All right, time to lose signal again :)

Code: [Select]

Ph =         4, D =  550757, GL = 0, LOS = 3, ST =     40152, AF =  37, RT = 120341
Ph =         3, D =  550741, GL = 0, LOS = 3, ST =     40153, AF =  37, RT = 120342
Ph =         3, D =  550724, GL = 0, LOS = 3, ST =     40154, AF =  37, RT = 120343
Ph =         4, D =  550708, GL = 0, LOS = 3, ST =     40155, AF =  37, RT = 120344
Ph =         1, D =  550704, GL = 0, LOS = 3, ST =     40156, AF =  37, RT = 120345
Ph =         1, D =  550712, GL = 0, LOS = 3, ST =     40157, AF =  37, RT = 120346
Ph =         1, D =  550720, GL = 0, LOS = 3, ST =     40158, AF =  37, RT = 120347
Ph =         0, D =  550728, GL = 0, LOS = 3, ST =     40159, AF =  37, RT = 120348
Ph = -10000003, D =  550745, GL = 0, LOS = 3, ST =     40160, AF =  37, RT = 120349
Ph = -10000003, D =  550761, GL = 0, LOS = 3, ST =     40161, AF =  37, RT = 120350
Ph = -10000003, D =  550757, GL = 0, LOS = 3, ST =     40162, AF =  37, RT = 120351
Ph = -10000003, D =  550753, GL = 1, LOS = 3, ST =     40163, AF =  37, RT = 120352
Ph = -10000007, D =  550753, GL = 1, LOS = 4, ST =         0, AF =  37, sK = 0.00000, RT = 120353
Ph = -10000007, D =  550653, GL = 1, LOS = 4, ST =         1, AF =  37, RT = 120354
Ph = -10000007, D =  550653, GL = 1, LOS = 4, ST =         2, AF =  37, RT = 120355
Ph = -10000006, D =  550653, GL = 1, LOS = 4, ST =         3, AF =  37, RT = 120356
Ph = -10000002, D =  550653, GL = 0, LOS = 4, ST =         4, AF =  37, RT = 120357
Ph = -10000002, D =  425986, GL = 0, LOS = 2, ST =         0, AF =  37, RT = 120358

Meh. I won't be able to get a stable 24h+ of tracking without an outdoor antenna like this.
It's now flapping between the tracking and no-tracking states, so clearly the signal reception quality has degraded.
Well, at least I'm gonna leave it like this till morning and see if the lockup issue repeats.

I don't like the algorithm of what it does on signal loss. The output frequency changes quite abruptly when that happens. I don't have the numbers, but I see it on the scope: I'd estimate it at tens of Hz difference at least. Once it starts tracking again, it gets to sub-Hz difference against the TinySA cal output almost instantly. Apparently it simply stops applying the tuning voltage to the OCXO when the GNSS signal is lost, whereas I think the proper thing to do would be to retain the last known good voltage instead to at least let it run at a good enough frequency until the GNSS signal can be received again. I think that's a software flaw.

[tverbeure]

What kind of clock reference is TinySA using?

I've unplugged the GPS antenna from my TM4313, but I don't remember seeing the clock frequency jump. I'll check it out tonight.

[shapirus]

What kind of clock reference is TinySA using?

All I could find was "30MHz TCXO". It does take a good bit of time to warm up before finally stabilizing, but after that it doesn't seem to deviate from the GPSDO much, and so far it seem to be consistent across power cycles.

I've unplugged the GPS antenna from my TM4313, but I don't remember seeing the clock frequency jump. I'll check it out tonight.

Another scenario to test is keeping the antenna connected, but blocking the signal (e.g. with a metal box or something).

[tverbeure]

All I could find was "30MHz TCXO". It does take a good bit of time to warm up before finally stabilizing, but after that it doesn't seem to deviate from the GPSDO much, and so far it seem to be consistent across power cycles.

Ok. When you're talking about the GPSDO being off by 0.4Hz during lock, it will be the TCXO that's off instead.

[shapirus]

Ok. When you're talking about the GPSDO being off by 0.4Hz during lock, it will be the TCXO that's off instead.

I believe I never said that one or the other was off (except for when the GPSDO loses signal and stops tracking). I always referred to it as a difference between the TinySA and GPSDO. Of course I'm assuming that it's the latter that is more accurate of the two, as long as it's tracking and the logs look sane.

Now, here's something very funny:

Code: [Select]

Ph =        -7, D =  551672, GL = 0, LOS = 3, ST =       480, AF =  37, RT = 124297
Ph =         5, D =  551709, GL = 0, LOS = 3, ST =       481, AF =  37, RT = 124298
Ph =         6, D =  551730, GL = 0, LOS = 3, ST =       482, AF =  37, RT = 124299
Ph =         5, D =  551750, GL = 0, LOS = 3, ST =       483, AF =  37, RT = 124300
Ph =  10000005, D =  551734, GL = 0, LOS = 3, ST =       484, AF =  37, RT = 124301
Ph =  10000006, D =  551692, GL = 0, LOS = 3, ST =       485, AF =  37, RT = 124302
Ph =  10000002, D =  551692, GL = 0, LOS = 3, ST =       486, AF =  37, RT = 124303
Ph =   9999998, D =  551700, GL = 0, LOS = 3, ST =       487, AF =  37, RT = 124304
Ph =   9999997, D =  551700, GL = 0, LOS = 3, ST =       488, AF =  37, RT = 124305
Ph =   9999990, D =  551700, GL = 0, LOS = 3, ST =       489, AF =  37, RT = 124306

I'm not sure about the frequency, as I've already turned off the TinySA, but hey, this change in the Ph parameter. And neither GL nor LOS changed here. I wonder if it's really GNSS jamming that puts it into this state.

Several minutes later, the D parameter also becomes screwed up:

Code: [Select]

Ph =   9989475, D = 2117078271, GL = 0, LOS = 3, ST =      1860, AF =  37, RT = 125677
Ph =   9989440, D = 2119228159, GL = 0, LOS = 3, ST =      1861, AF =  37, RT = 125678
Ph =   9989295, D = 2128710399, GL = 0, LOS = 3, ST =      1862, AF =  37, RT = 125679
Ph =   9989153, D = 2138178175, GL = 0, LOS = 3, ST =      1863, AF =  37, RT = 125680
Ph =   9989015, D = -2147336449, GL = 0, LOS = 3, ST =      1864, AF =  37, RT = 125681
Ph =   9988879, D = -2137898241, GL = 0, LOS = 3, ST =      1865, AF =  37, RT = 125682
Ph =   9988746, D = -2128475393, GL = 0, LOS = 3, ST =      1866, AF =  37, RT = 125683
Ph =   9988619, D = -2119067393, GL = 0, LOS = 3, ST =      1867, AF =  37, RT = 125684
Ph =   9988482, D = -2109673985, GL = 0, LOS = 3, ST =      1868, AF =  37, RT = 125685
Ph =   9988335, D = -2100295425, GL = 0, LOS = 3, ST =      1869, AF =  37, RT = 125686

...and the "track" LED is still flashing!

It'll be interesting to see if it can recover from this, but so far it seems that the firmware is not able handle some conditions. It looks like the GPS signal is in reality lost, but the firmware fails to recognize it.

Would also be interesting to see the messages produced by the GPS module when it's in this state, but, alas, we can't hot-switch it. Or can we? I don't feel like trying it :)

[tverbeure]

Having to deal with GPS jamming may be something the designers never thought about, especially for something as cheap as this. I mean how do you even start to cover all the ways jamming can work.

[shapirus]

Having to deal with GPS jamming may be something the designers never thought about, especially for something as cheap as this. I mean how do you even start to cover all the ways jamming can work.

Well, if I had to write software handling it, I'd do it in the most straightforward way: no data? treat as lost signal; bogus data, an abrupt change of coordinates and/or timestamps received from the GPS module? discard data and treat as no signal.
That's not even about handling jamming situations, but just a regular input conditioning procedure that is a very common thing to do.

Besides, whatever is causing it, it has to be able to recover once good data starts coming back. If it starts. And this "if" makes me think that it may be not the firmware that locks up, but the GPS module itself, which requires a reset of the antenna to recover. I think it's quite likely. I'll switch the UART path to the GPS module next evening and see what it will report.

[tverbeure]

And this "if" makes me think that it may be not the firmware that locks up, but the GPS module itself, which requires a reset of the antenna to recover. I think it's quite likely. I'll switch the UART path to the GPS module next evening and see what it will report.

That's a good point. I don't remember if the unfiltered 1PPS signal that comes from the module is easily available, but it could be useful putting that up on a scope.

[tverbeure]

I don't like the algorithm of what it does on signal loss. The output frequency changes quite abruptly when that happens. I don't have the numbers, but I see it on the scope: I'd estimate it at tens of Hz difference at least. Once it starts tracking again, it gets to sub-Hz difference against the TinySA cal output almost instantly. Apparently it simply stops applying the tuning voltage to the OCXO when the GNSS signal is lost, whereas I think the proper thing to do would be to retain the last known good voltage instead to at least let it run at a good enough frequency until the GNSS signal can be received again. I think that's a software flaw.

I recorded the frequency every second on my frequency counter, unplugged the antenna and then plugged it back in:


StanfordResearchSystems,SR620,00182,1.48
 0 9.99999999761E6
 1 9.99999999769E6
 2 9.99999999785E6
 3 9.99999999766E6
 4 9.99999999766E6
 5 9.99999999815E6
 6 9.99999999796E6
 7 9.99999999774E6
 8 9.99999999796E6
 9 9.99999999764E6
10 9.99999999802E6
11 9.99999999788E6
12 9.99999999745E6
13 9.99999999755E6
14 9.99999999669E6
15 9.99999999747E6
16 9.99999999774E6
17 9.99999999758E6
18 9.99999999709E6
19 9.99999999802E6
20 9.99999999807E6
21 9.99999999780E6
22 9.99999999793E6
23 9.99999999802E6
24 9.99999999829E6
25 9.99999999907E6
26 9.99999999880E6
27 9.99999999918E6   <<<< Antenna unplugged
28 9.99999999883E6
29 9.99999999932E6
30 9.99999999926E6
31 9.99999999910E6
32 9.99999999924E6
33 9.99999999956E6
34 9.99999999940E6
35 9.99999999986E6
36 1.00000000001E7
37 1.00000000006E7
38 9.99999999934E6
39 9.99999999850E6
40 9.99999999856E6
41 9.99999999829E6
42 9.99999999858E6
43 9.99999999821E6
44 9.99999999861E6
45 9.99999999856E6
...
138 9.99999999812E6
139 9.99999999802E6
140 9.99999999804E6
141 9.99999999802E6
142 9.99999999791E6
143 9.99999999845E6
144 9.99999999848E6
145 9.99999999829E6
146 9.99999999791E6
147 9.99999999788E6    <<< Antenna plugged in, immediate GPS relock
148 9.99999999815E6
149 9.99999999815E6
150 9.99999999793E6
151 9.99999999821E6
152 9.99999999875E6
153 9.99999999837E6
154 9.99999999856E6
155 9.99999999837E6
156 9.99999999823E6
157 9.99999999831E6
158 1.00000000010E7
159 1.00000000000E7
160 9.99999999921E6
161 9.99999999818E6
162 9.99999999717E6
163 9.99999999658E6
164 9.99999999620E6
165 9.99999999566E6
166 9.99999999536E6
167 9.99999999509E6
168 9.99999999419E6
169 9.99999999351E6
170 9.99999999397E6
171 9.99999999359E6
172 9.99999999340E6
173 9.99999999240E6
174 9.99999999232E6
175 9.99999999188E6
176 9.99999999202E6
177 9.99999999167E6
178 9.99999999183E6
179 9.99999999164E6
180 9.99999999245E6


I'd have to repeat this test a couple of times and create a plot, but the effect is minimal. Instead of have measurements that different by ~1 or 2 mHz, there's maybe a little jump but it's definitely the a couple of Hz that you're seeing. The interesting part is if there's a jump, it settle back down after that, so the change could just be due to the vibrations caused by unscrewing the antenna connector.

Edit: I redid the exercise with the counter set to a 5s update rate. Unplugging has zero effect, plugging the cable back in is more disruptive (~10mHz) with the output frequency swinging around a bit, which is what you'd expect from a control loop.

[shapirus]

I'll repeat the same test as yours with the antenna disconnection a bit later. Well, somewhat the same -- I don't have a freq counter, so will have to use the oscilloscope to watch two signals, GPSDO and TinySA, along each other. Assuming that the disconnection of the antenna from the GPSDO is not supposed to disturb the output of the TinySA, then if a significant difference occurs in what's seen on the scope (particularly in the XY mode), then it can only mean that it's the GPSDO that loses its frequency stabilization.

As far as the 1 PPS output, there is a respective jack, and I will try to trace where it's connected to (not sure if it'll be possible with that black solder mask). Is it supposed to output nothing when there's no GPS reception?

Right now the GPSDO is locked up again (has been since the evening -- see the ST value) and isn't recovering:

Code: [Select]

Ph =  49838453, D = -296976385, GL = 0, LOS = 3, ST =     40169, AF =  37, RT = 163986
Ph =  49838537, D = -288784385, GL = 0, LOS = 3, ST =     40170, AF =  37, RT = 163987
Ph =  49838566, D = -280559617, GL = 0, LOS = 3, ST =     40171, AF =  37, RT = 163988
Ph =  49838540, D = -272400385, GL = 0, LOS = 3, ST =     40172, AF =  37, RT = 163989
Ph =  49838443, D = -264208385, GL = 0, LOS = 3, ST =     40173, AF =  37, RT = 163990
Ph =  49838325, D = -256049153, GL = 0, LOS = 3, ST =     40174, AF =  37, RT = 163991
Ph =  49838364, D = -247857153, GL = 0, LOS = 3, ST =     40175, AF =  37, RT = 163992
Ph =  49838395, D = -239697921, GL = 0, LOS = 3, ST =     40176, AF =  37, RT = 163993


I have checked the 1 PPS output: it is outputting a pulsed signal, 1 Hz (I can't check how accurate), unipolar 0..4.75V square (into 1 MOhm), 100 ms pulse width.

Does it mean that the GPS module is definitely not locked up?

[nctnico]

I'd say it is a software issue. But it is quite possible that the type of jamming used near you, is designed to confuse GPS receivers to a point where they don't come up after the jamming ends. Makes it harder to pinpoint the transmitter. It would be very interesting if you could test other GPSDOs using different GPS receiver modules and look what is coming out of the module itself in the form of NMEA data.

[shapirus]

Right now the GPSDO is locked up again (has been since the evening -- see the ST value) and isn't recovering:

Here's how it looks visually:




Looks more like a software issue to me. Purple trace is a stable enough reference (TinySA Ultra), blue trace is the TM4313.

I was going to record how it behaves when I disconnect and reconnect the antenna, but alas, have just had a power outage, and the GPSDO got powered off. Next time I'll power it from the UPS.

Anyway, I will at least be able to record its reaction to a disconnection of the antenna when it's in a "good" tracking state to compare the results with those posted by tverbeure above.

[shapirus]

Anyway, I will at least be able to record its reaction to a disconnection of the antenna when it's in a "good" tracking state to compare the results with those posted by tverbeure above.

...and we have a match here. When the antenna is disconnected, there is no visually observable change in the output frequency, at least within a few minutes. When the antenna is connected back, it goes into "fine" tracking mode again almost instantly. This means that the OCXO tuning voltage level is maintained in the last known good state in this scenario.

The next thing to test is to bring the antenna, without disconnecting it, further into inside of the room to simulate a signal level degradation.

[shapirus]

The next thing to test is to bring the antenna, without disconnecting it, further into inside of the room to simulate a signal level degradation.

This test is passed too.


1) antenna moved deep indoors:

Code: [Select]

Ph =       -16, D =  552276, GL = 0, LOS = 3, ST =       552, AF =  37, RT = 2005
Ph =       -15, D =  552269, GL = 0, LOS = 3, ST =       553, AF =  37, RT = 2006
Ph =       -15, D =  552263, GL = 0, LOS = 3, ST =       554, AF =  37, RT = 2007
Ph =       -15, D =  552256, GL = 0, LOS = 3, ST =       555, AF =  37, RT = 2008
Ph =       -14, D =  552254, GL = 0, LOS = 3, ST =       556, AF =  37, RT = 2009
Ph =       -14, D =  552247, GL = 1, LOS = 3, ST =       557, AF =  37, RT = 2010
Ph =       -14, D =  552247, GL = 1, LOS = 4, ST =         0, AF =  37, sK = 0.00000, RT = 2011
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =         1, AF =  37, RT = 2012
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =         2, AF =  37, RT = 2013
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =         3, AF =  37, RT = 2014
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =         4, AF =  37, RT = 2015
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =         5, AF =  37, RT = 2016
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =         6, AF =  37, RT = 2017
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =         7, AF =  37, RT = 2018


2) antenna brought back to the window:

Code: [Select]

Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =       156, AF =  37, RT = 2167
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =       157, AF =  37, RT = 2168
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =       158, AF =  37, RT = 2169
Ph =       -14, D =  551097, GL = 1, LOS = 4, ST =       159, AF =  37, RT = 2170
Ph =         0, D =  551097, GL = 1, LOS = 4, ST =       160, AF =  37, RT = 2171
Ph =         0, D =  551097, GL = 1, LOS = 4, ST =       161, AF =  37, RT = 2172
Ph =         0, D =  551097, GL = 1, LOS = 4, ST =       162, AF =  37, RT = 2173
Ph =         0, D =  551097, GL = 1, LOS = 4, ST =       163, AF =  37, RT = 2174
Ph =        -1, D =  551097, GL = 1, LOS = 4, ST =       164, AF =  37, RT = 2175
Ph =         1, D =  551097, GL = 1, LOS = 4, ST =       165, AF =  37, RT = 2176
Ph =         3, D =  551097, GL = 1, LOS = 4, ST =       166, AF =  37, RT = 2177
Ph =         4, D =  551097, GL = 1, LOS = 4, ST =       167, AF =  37, RT = 2178
Ph =         5, D =  551097, GL = 0, LOS = 4, ST =       168, AF =  37, RT = 2179
Ph =         0, D =  551097, GL = 0, LOS = 3, ST =         0, AF =  37, RT = 2180
Ph =         2, D =  551097, GL = 0, LOS = 3, ST =         1, AF =  37, RT = 2181
Ph =         3, D =  551097, GL = 0, LOS = 3, ST =         2, AF =  37, RT = 2182
Ph =         5, D =  551097, GL = 0, LOS = 3, ST =         3, AF =  37, RT = 2183
Ph =         6, D =  551097, GL = 0, LOS = 3, ST =         4, AF =  37, RT = 2184
Ph =         7, D =  550771, GL = 0, LOS = 3, ST =         5, AF =  37, RT = 2185
Ph =        10, D =  550656, GL = 0, LOS = 3, ST =         6, AF =  37, RT = 2186


The output frequency remained stable during the entire length of this test.

So, I guess, no night-time frequency reference for me, for now :). At least without an outdoors antenna.

Yes it would be interesting to test how other GPSDO behave in this scenario, but the level of this curiosity is not high enough to justify the expense required to get another unit.

[shapirus]

I took it apart to change the jumpers to connect to the GPS module's serial port and took some pictures while I was there.

OCXO:




Date and HW revision:




The 1 PPS output is connected like so (i.e., not to the GPS module):

[Zenith]

There's very little literature on the TM4313 other than my blog post, and other than you and me, I don't think anyone has gone father in their analysis of the TM4313 log messages.

I'm glad this thread has appeared. I've had a TM4313 for about three years. It came with no instructions and until now I couldn't find anything about it on the WWW, although I hadn't looked for some time. It sits there with the antenna on a window sill, not outside, and agrees with a Racal 1992 with the better OCXO (04E) option, to within a few ppb over the course of a week. I'd always wanted to explore its data reporting features. It's lost lock a couple of times when the antenna has been covered up with something or other.

I assume that for practical purposes, calibrating synthesised sig gens and adjusting frequency counters with ancient OCXOs, (so no better than 1 part in 10^8) that the TM4313 is perfect.

One thing that has surprised me is how good the AliExpress ex-equipment CTI OCXOs are.

[tverbeure]

The 1 PPS output is connected like so (i.e., not to the GPS module):

Yeah, that’s expected. The 1PPS coming out of the GPS module is way too jittery.

[shapirus]

Update.

I cannot reproduce this lock-up when the jumpers are set to receive messages from the GPS module!

Yesterday in the evening it was running with the jumpers set for the MCU's UART output. The GPSDO locked up as described in first post. I disconnected and reconnected the antenna, it started tracking, but then went into that unstable state again. Then I changed the jumpers and let it run again. Guess what, no issues. It's tracking just fine. Yes it did lose tracking a couple of times at some point, but then resumed it properly.

Right now, and it was a second (non-consecutive though) night with no lock-up issues, it's generating a good output frequency with no "seeking" oscillation.

I wonder what do the jumpers have to do with it and how could it affect things. At this point it does not look like a coincidence.