Using a microcontroller and FreeRTOS for a data logger

[SteveThackery]

PLEASE MOVE OR DELETE THIS IF IT'S IN THE WRONG FORUM

Colleagues!

A few years ago, for my degree dissertation, I developed a data logger for monitoring various parameters of a motorcycle. Because of time constraints I decided to use a handful of Arduino Pro Minis as I knew them well. Each would monitor one or two inputs, and five times a second they would send their data to a "master" Arduino which was responsible for storing the data on an SD card for later analysis. The reason for dividing the workload across several Arduinos was due to their limited processing power. The peripheral Arduinos communicated with the master Arduino over I2C.

Examples of the data gathered are: lambda sensor voltage; engine temperature; throttle position sensor (TPS); fuel injection pulse duration; manifold absolute pressure (MAP); road speed; engine rpm; brakes (applied or not); acceleration (from an accelerometer chip).

These inputs were "scanned" five times per second, and this fact might be important for my question. So, take the fuel injection pulse duration, for example. It was not necessary or intended to log every pulse; rather, five times per second the duration of the most recent pulse was measured and reported back to the master Arduino. Another important point is that the fuel injection pulse duration is short (a few milliseconds) and needs measuring to tenths of a millisecond. Another Arduino monitored the TPS and reported its position when prompted by the master, and so on with the rest of the Arduinos.

I always knew that using half a dozen Arduino Pro Minis all talking on an I2C bus was a horrendous bodge (although it worked well enough) and I always planned to implement the whole thing on a single, much more powerful, microcontroller. Each Arduino would be replaced by a thread (or task?) running in an RTOS.

To my question... Firstly, from the above overview, is it the kind of application that could be met by a powerful microcontroller running FreeRTOS (for example)? If so, how frequently can I task switch the RTOS, bearing in mind that at least one of the parameters needs measuring to sub-millisecond resolution? I've read that an ESP32 Arduino running FreeRTOS task switches every millisecond, which would seem far too coarse to measure an input to one tenth of a millisecond.

I realise I am probably missing some key concepts. I also realise I've skimmed over a lot of detail. But really I'm just hoping to get a handle on the feasibility of the project, and what the main issues would be regarding problems and solutions. Any thoughts or observations would be very much appreciated.

[SteveThackery]

Just for a giggle, here is a picture of the Arduino-based prototype. Top right is a time-and-date clock module, and there is also an Arduino which I didn't mention, which did idle stabilisation.

[SteveThackery]

Here is an extract showing how I compared the prototype architecture with the intended final architecture.  I've just noticed a mistake in the Arduino numbering. These were illustrative only. In reality there were more Arduinos and proposed tasks.

[pqass]

My first thoughts would be to apply analog signal conditioning to convert pulse-width or frequency sources to voltages.  Then scale up/down the voltages via opamp/divider and read via single MCU ADC MUX. None of the sources move particularly fast (temp, pressure, RPM, etc.).  You don't really need a real-time OS here.

I'd setup a timer event (100Hz?) to collect one round of instantaneous voltages from all inputs, process a running average of each to suppress outliers, and append each to their queue/array of last-n averages. Occasional writes to the SDCARD are done in large-ish chunks to maximize life (suspending data collection while writing).  If you're also managing multiplexed (say LED) displays, another timer event (60Hz * total digits) will be needed to update the current digit of the variable displayed.  The single main thread can manage a command processor via serial.

[brucehoult]

The reason for dividing the workload across several Arduinos was due to their limited processing power.

Nooo. A tenth of a ms (100 µs, 1600 instructions) is a very long time even on an AVR. Write the software properly and one would be just fine.

Not so long ago most 3D printers were controlled by AVR Arduinos, controlling the timing of individual stepper motor step pulses to X, Y, Z, and extruder stepper motors, running a PID control algorithm for the extruder heater, calculating acceleration profiles for the X and Y steppers, measuring the temperature, AND receiving and interpreting the G-code stream of commands, and often displaying status on an LCD.

A Mega2560 was used in order to have enough pins, and enough code size, but it was no faster than the CPU in an Uno or Pro Mini.

A better reason to use multiple MPUs would be to put them close to what they were measuring to reduce signal integrity issues, while having the logging communications over a channel with error detection and retry.

You might also want/need a dedicated one to write the SD card, depending on how long writing a sector takes and whether it can be interrupted. TBH you'd be better off using a hardwired SPI flash chip, and then have a mode to dump its contents to an CD card (or out the UART or whatever) after the ride.

But, yes, technology has moved on.

For the last five years I'd say the go-to device for your needs would be the $25 or so Teensy 4.0 (or 4.1) board, with a 32 bit 600 MHz dual-issue Cortex-M7 (close to 900 MIPS and they're 32 bit MIPS), 1 MB RAM, a lot of GPIO (including analogue) pins. And it runs Arduino sketches with a fantastic OOTB experience -- I had mine running blinky in about 3 minutes flat.

As an example of the performance, on my personal primes benchmark [1] a Teensy 4.0 took 43.5 seconds at its default 600 MHz, while a 20 MHz ATMega2560 took 13449.5 seconds, a factor of just over 300x.

Ok, so that's 2019 tech.

My go-to now is the $3 Milk-V Duo. It's got not 1 MB RAM but 64 MB, has a 700 MHz "microcontroller" core *plus* a 1 GHz (actually the default software runs it 850 MHz, but w/e) core running full Linux (typically Buildroot for the user-land, but you could put Debain/Ubuntu Server etc). It's got 26 GPIOs (only 2 ADC capable, sadly). And you can run Arduino sketches on the microcontroller core at the same time Linux is running on its core.

https://milkv.io/docs/duo/overview



I don't know the exact times off-hand, but either the Teensy 4 or the Duo should be able to switch threads in an RTOS (or Linux for the Duo) in well under a µs let alone a ms.

Heck, I'd think an AVR could switch threads in under 10 µs.


[1] https://hoult.org/primes.txt

[NorthGuy]

If an arduino can collect data from other arduinos, why cannot it collect the same data directly from sensors?

[SiliconWizard]

More so than speed, I'd say the insane idea of using something like a RTOS on a AVR chip is the very limited amount of RAM, rather than context switching time. But to each their own. If one absolutely wants to do it, doable it is.

Yes, there are tons of much more powerful and even cheaper alternatives these days, even for those that want to keep using the Arduino framework.

[ledtester]

Examples of the data gathered are: lambda sensor voltage; engine temperature; throttle position sensor (TPS); fuel injection pulse duration; manifold absolute pressure (MAP); road speed; engine rpm; brakes (applied or not); acceleration (from an accelerometer chip).

These inputs were "scanned" five times per second, and this fact might be important for my question. So, take the fuel injection pulse duration, for example. It was not necessary or intended to log every pulse; rather, five times per second the duration of the most recent pulse was measured and reported back to the master Arduino. Another important point is that the fuel injection pulse duration is short (a few milliseconds) and needs measuring to tenths of a millisecond. Another Arduino monitored the TPS and reported its position when prompted by the master, and so on with the rest of the Arduinos.

I can see where interleaving the sensor access and SDcard writes could be kinda tricky on an Arduino. I think, though, you could still do all of the data collection with one Arduino and then use serial communication to pass it to another for writing to the SDcard. Have a look at the OpenLog project for an implementation of such a logger:

https://github.com/sparkfun/OpenLog

For the fuel injection pulse length measurement from what you described this is how I would go about doing it:

- Set up a pin change interrupt service routine for the GPIO pin
- Inside the service routine, if the GPIO pin is currently HIGH, call millis() to get the current time and store it in the global variable pulse_start_time.
- If the GPIO pin is LOW, compute millis()-pulse_start_time to get the duration of the pulse in milliseconds and store that in another global variable pulse_length. This variable will always reflect the length of the last pulse so your main loop can decide how often it wants to report it.

There's a bunch of implementation details that you need to get right, but this approach avoids needing to poll the GPIO pin in a loop.

Here's a pretty good tutorial on Arduino interrupt routines and covers pin change interrupts:

https://dronebotworkshop.com/interrupts/

[mianos]

Yes, the ESP runs a fork of FreeRTOS but that's not going to give you the ability to read all those sensors in real time.

This is more in the domain of interrupts. There are modules for almost all the peripherals you mention that have read (past tense) trigger pins or direct triggers after some conditioning.

If I was reading injector pulse width, I would just condition the signal, connect it to an interrupt and read them all. Then put this into a proper real time queue and chuck out the ones that I don't use.

If there are too many pins you can use an io multiplexor and just read it to find out what pin changed in the interrupt and send that to the queue with the real time timestamp.

[pcprogrammer]

You don't need a RTOS for doing this.

Use a micro with a couple of timers and fast adc's to do it all interrupt based and just run a main loop to do some processing on the data and write it to the SD card. Might even be doable on an AVR, but an ARM Cortex running on 40MHz should certainly be able to pull it off.

The Raspberry pico (RP2040) could be a cheap option and allows doing some "task" in "hardware".

An ESP32 running on 200MHz might even be overkill, but is also cheap enough.

The Milk-V Duo brucehoult mentioned would definitely be overkill, but it is cheap and with Linux running on it makes writing to an SD card a simple job.

[pcprogrammer]

My go-to now is the $3 Milk-V Duo. It's got not 1 MB RAM but 64 MB, has a 700 MHz "microcontroller" core *plus* a 1 GHz (actually the default software runs it 850 MHz, but w/e) core running full Linux (typically Buildroot for the user-land, but you could put Debain/Ubuntu Server etc). It's got 26 GPIOs (only 2 ADC capable, sadly). And you can run Arduino sketches on the microcontroller core at the same time Linux is running on its core.

A bit of a side track, but I wonder why the Milk-V Duo is rather expensive on Aliexpress or Amazon?

https://nl.aliexpress.com/item/1005005699176591.html
https://nl.aliexpress.com/item/1005006064411603.html
On Amazon

Did not look at the shipping cost to France for the cheapest one I found, but the price difference in its base is big.

https://arace.tech/products/milk-v-duo?variant=42555014512820

[SiliconWizard]

The lowest price for the Duo on Aliexpress is around $7 currently. It has always been higher than on Arace, which makes sense as Aliexpress is mainly resellers. They need to take their share too.

Anything above $7-$8 (for the Duo, not the Duo-256M which obviously is more expensive) is to be avoided. Takes a bit of time to go through all offers to get those cheaper prices.

Not talking about Amazon, which is obviously mostly people importing stuff from China and reselling there with a x3 to x4 markup.

[brucehoult]

The lowest price for the Duo on Aliexpress is around $7 currently. It has always been higher than on Arace, which makes sense as Aliexpress is mainly resellers. They need to take their share too.

Arace works for me, though even for just one Duo they want NZ$21 (US$12.50) shipping. But you can order 25 Duos for the same $21 shipping (then it goes to $26), so you're averaging US$3.84 shipped.

Some of the Aliexpress probably have cheaper shipping.

Anything above $7-$8 (for the Duo, not the Duo-256M which obviously is more expensive) is to be avoided. Takes a bit of time to go through all offers to get those cheaper prices.

This one is showing me $2.67 for 1, with free shipping...  They've sold quite a lot of them so probably ok.

https://www.aliexpress.us/item/3256805512861839.html

Not talking about Amazon, which is obviously mostly people importing stuff from China and reselling there with a x3 to x4 markup.

Exactly.

Ok, if you really need one tomorrow ... (and you're in the US)

[brucehoult]

More so than speed, I'd say the insane idea of using something like a RTOS on a AVR chip is the very limited amount of RAM, rather than context switching time.

I dunno.  If each thread is just a main function that doesn't really call anything else, but just waits on something and processes it and loops, then it might not be using any more stack/RAM than the 32 bytes of registers you have, or even less. It's not really any different to using an interrupt handler, except easier to write. If you're only using 30 bytes or even 100 bytes in each thread then you can fit quite a lot of them into even 2k bytes on an ATMega328.  Not to mention that the ATMega2560 (Arduino Mega) has 8k of RAM.

[peter-h]

I've done a product which runs on a 32F417 and FreeRTOS runs at the usual Systick of 1kHz. Then a separate time interrupt runs various timers also at 1kHz (complicated reasons why those two are separate). But FR could run at 10kHz perfectly well, on this 168MHz CPU.

At some £6 I would not bother with a slower CPU for anything unless it is high volume and has to be really cheap, or very low power, etc.

Integrating FR is not hard. It is basically a standalone chunk of code, you make a call to initialise it, other calls to hang tasks onto it, and then a call to start it. It has to be hooked up to a 1kHz (etc) tick. It is super-reliable.

One drawback of an RTOS is that it does need more RAM than doing things in a loop or state machine type of structure, because each task needs enough RAM for its own variables and stack, and since you need some fair margin for the stack, this margin gets duplicated by the # of tasks you have.

[brucehoult]

I just tried the following test (used by OpenBenchmarking / Phoronix) on the $3 64 MB Duo:

https://github.com/detailyang/ipc_benchmark

Code: [Select]

[root@milkv-duo]~/ipc_benchmark# ./run_tests.sh
posixq                                                 
128     256         512         1024        2048       
        10MB/s      18MB/s      32MB/s      59MB/s      89MB/s
        81051msg/s  73967msg/s  65330msg/s  60416msg/s  45607msg/s

pipe                                                 
128   256         512         1024        2048       
      8MB/s       14MB/s      28MB/s      47MB/s      81MB/s
      65461msg/s  57734msg/s  57439msg/s  48405msg/s  41419msg/s

fifo                                                 
128   256         512         1024        2048       
      8MB/s       16MB/s      28MB/s      48MB/s      82MB/s
      65198msg/s  65094msg/s  58310msg/s  49627msg/s  42043msg/s

socketpair                                                 
128         256         512         1024        2048       
            7MB/s       14MB/s      26MB/s      49MB/s      85MB/s
            56833msg/s  57458msg/s  52297msg/s  50592msg/s  43704msg/s

uds                                                 
128  256         512         1024        2048       
     7MB/s       14MB/s      25MB/s      50MB/s      83MB/s
     60991msg/s  55764msg/s  51126msg/s  51191msg/s  42537msg/s

tcp                                               
128  256        512        1024       2048       
     1MB/s      1MB/s      2MB/s      29MB/s      41MB/s
     4749msg/s  4604msg/s  4750msg/s  30117msg/s  21126msg/s

shm                                                 
128  256         512         1024        2048       
     6MB/s       13MB/s      26MB/s      46MB/s      89MB/s
     52358msg/s  51260msg/s  52298msg/s  47284msg/s  45452msg/s

The first five methods for IPC are all 12-17 MICRO seconds per 128 byte message.

This is between Linux processes, not just threads in one process -- each test program fork()s a child and then communicates with it.

The only changes I made to the github code were to use riscv64-linux-gnu-gcc as the compiler and to add "-static" to CFLAGS, scp'd the binaries and scripts (well, ok .. the whole directory actually) to the Duo and ran ./run_tests.sh

All toooo easy.

The VisionFive 2 runs the first four methods at 0.3 to 1.0 µs per message.

[SiliconWizard]

This one is showing me $2.67 for 1, with free shipping...  They've sold quite a lot of them so probably ok.

https://www.aliexpress.us/item/3256805512861839.html

Yeah, that must be one of these first-time-buy prices, because for me the Duo on this page shows 11.75€ per 1 and 9.60€ for >= 3.

[brucehoult]

This one is showing me $2.67 for 1, with free shipping...  They've sold quite a lot of them so probably ok.

https://www.aliexpress.us/item/3256805512861839.html

Yeah, that must be one of these first-time-buy prices, because for me the Duo on this page shows 11.75€ per 1 and 9.60€ for >= 3.

Yeah. I made sure I was logged in to my Ali account, but I haven't used *that* store before. And I went right to the "Charge my card NOW" screen and the price didn't change.

It does say "Max. 1 pcs/shopper".

You might not be seeing the cheap price, but probably a lot of people here would.

[SteveThackery]

If an arduino can collect data from other arduinos, why cannot it collect the same data directly from sensors?

That sounds like a very good question. The answer - which may well be ridiculous nonsense - is as follows...

The data from each of the peripheral Arduinos is very small, typically 16 bits, if I recall correctly. So sending it over I2C is very quick. On the other hand, gathering that data might take several milliseconds (timing the fuel injection pulse duration) or tens of milliseconds (timing the road speed pulse interval). That's why I wanted to offload those tasks to peripheral Arduinos.

Having read the other comments, it seems increasingly clear that this was a misconception, because measuring the interval between two pulses doesn't tie up the processor for all that time. Having said that, measuring the fuel injection pulses to 100us seems like it might be hard while doing everything else as well.

[pcprogrammer]

This one is showing me $2.67 for 1, with free shipping...  They've sold quite a lot of them so probably ok.

https://www.aliexpress.us/item/3256805512861839.html

Yeah, that must be one of these first-time-buy prices, because for me the Duo on this page shows 11.75€ per 1 and 9.60€ for >= 3.

I see the same here. So they add to the price when you shop in Europe. Build in shipping cost probably.

Arace works for me, though even for just one Duo they want NZ$21 (US$12.50) shipping. But you can order 25 Duos for the same $21 shipping (then it goes to $26), so you're averaging US$3.84 shipped.

It most likely won't work out like that for me due to import duties since VAT is not included. I read their shipping policy and it is also a bit of a risk with them shifting liability for the shipping to you, so when it gets lost there is no money back guarantee.

That is a nice thing of Aliexpress, they give some assurance, but you clearly pay for it too.

[brucehoult]

Having read the other comments, it seems increasingly clear that this was a misconception, because measuring the interval between two pulses doesn't tie up the processor for all that time.

Exactly.

Having said that, measuring the fuel injection pulses to 100us seems like it might be hard while doing everything else as well.

Not really.

Just use attachInterrupt(digitalPinToInterrupt(pin), my_handler, CHANGE) to specify a small function to call whenever the value of an input pin changes. Then in that function call micros() to get the current time and add it into some global variable or buffer that will be read and processed by the main program.

Interrupt handlers should be short and return as quickly as possible.

https://www.arduino.cc/reference/en/language/functions/external-interrupts/attachinterrupt/

https://www.arduino.cc/reference/en/language/functions/time/micros/

[brucehoult]

Arace works for me, though even for just one Duo they want NZ$21 (US$12.50) shipping. But you can order 25 Duos for the same $21 shipping (then it goes to $26), so you're averaging US$3.84 shipped.

It most likely won't work out like that for me due to import duties since VAT is not included. I read their shipping policy and it is also a bit of a risk with them shifting liability for the shipping to you, so when it gets lost there is no money back guarantee.

It will work out like that for me. I've made several purchases at Arace.

That is a nice thing of Aliexpress, they give some assurance, but you clearly pay for it too.

Aliexpress are big enough (sell $80k+ of things to NZ a year) that they are in the system where they automatically add NZ GST to prices and forward it to the NZ government. Such shipments automatically fly through the border.

Arace are small and under the radar so they don't add GST.  Shipments to NZ valued less than NZ$1000 (US$600) to private individuals don't attract any taxes or duties or fees at the border, so they are released as soon as they determine the value.

I'm sorry if you live in a less enlightened country or regional protectionist trade bloc.  And the UK recently left one but then instituted their own stupid import charges! ID10Ts. What was even the point?

[SiliconWizard]

Yeah, there used to be a minimum value over here too for import taxes for private individuals, but the limit kept decreasing over the years. And now I think it's due from the first euro in the EU zone.

[pcprogrammer]

If an arduino can collect data from other arduinos, why cannot it collect the same data directly from sensors?

That sounds like a very good question. The answer - which may well be ridiculous nonsense - is as follows...

The data from each of the peripheral Arduinos is very small, typically 16 bits, if I recall correctly. So sending it over I2C is very quick. On the other hand, gathering that data might take several milliseconds (timing the fuel injection pulse duration) or tens of milliseconds (timing the road speed pulse interval). That's why I wanted to offload those tasks to peripheral Arduinos.

Having read the other comments, it seems increasingly clear that this was a misconception, because measuring the interval between two pulses doesn't tie up the processor for all that time. Having said that, measuring the fuel injection pulses to 100us seems like it might be hard while doing everything else as well.

Examine all the different tasks that you have and think of other ways to implement them without some sort of polling. Measuring a pulse time can even be done completely in hardware with the more advanced timers, and only interrupt the processor after the end of the pulse. It might even be possible with the STM32 timers to only interrupt every so many pulses measured to reduce the load a bit further.

It does require studying the reference manual and understanding all the possibilities of the hardware. Can be a good learning project.

[SteveThackery]

Firstly, I just want to say how thankful I am for all the brilliant replies. Collectively you have shown me that my basic assumption - that this is a job for an RTOS - is completely flawed. That's a great learning point for me and I really appreciate it.

On the broader question, having shown that my data logger should be implemented differently, what sort of application would be suitable for an RTOS?