Arduino Datalogger Timing

[Thane of Cawdor]

Hey everyone,

I'm thinking of running a datalogger (simple DHT11 temp/humidity system) for about a year or so using an Arduino (using the Simple Timer library to set intervals of about 30mins) would this drift considerably over the long term and is there a better way to do timing without a hardware RTC?

Thanks

[sleemanj]

Yes it will drift.  Use an RTC.

[Nusa]

It's possible to empirically calibrate individual Arduino clocks to be almost as accurate as RTC modules. However, temperature affects clock cyrstals, and outdoor temperatures will affect them significantly. The crystals on cheap RTC clock modules are not immune to temperature based drift.

One solution is to find an external timebase. For an outdoor application, adding a GPS antenna would let you read the extremely accurate GPS time broadcast signal.

And there's always the lazy man's approach, which also works. Simply be aware of the total time drift at collection time, and apply scaled adjustments to the timestamps when you process the data.

As another idea, you could add a photoresistor to monitor light levels as well. That would serve as an additional data points giving you approximate sunrise/sunset times, which would have to correlate with actual sunrise/sunset times for your location.

[mariush]

Yeah, gps would be best but also the most expensive (both price in hardware and in battery life).
Another silly idea would be to add a light sensor to measure the light intensity and you know the sunrise and sunset every day (unless it's cloudy or rainy or whatever) where you place your device so you could sort of correct the drift at the end of the year.
Or even sillier, plase a digital clock with alarm function inside the box and set the alarm every day at 12:00 ... your device listens for the alarm and corrects itself if the time drifts.  But now you're better off just going with a real time clock chip.

[bitseeker]

More information is needed. How accurate does it have to be? Is this going to be indoor or outdoor (how much temperature change)? Is power a limitation, if so how much? Are other resources available (network, external devices, RTC, GPS, RF, IR, etc.)? Does the system have to be sealed from the rest of the world? What will be its geographical location?

Answers to these questions will narrow down which of many possible solutions could work for your application.

[tooki]

Hey everyone,

I'm thinking of running a datalogger (simple DHT11 temp/humidity system) for about a year or so using an Arduino (using the Simple Timer library to set intervals of about 30mins) would this drift considerably over the long term and is there a better way to do timing without a hardware RTC?

Thanks

Given that an RTC module costs something like $2, why wouldn't you want to use one??

[Ian.M]

Arduino onboard ceramic resonator - at constant temperature, about +/- 1 minute per week, after careful calibration.  (uncalibrated, varying temperature, can be worse than a minute a day)

RTC module - typically +/- 5 minutes per year, and even the crappiest are unlikely to be worse than +/-15 minutes per year.

If you need better than that you need an external time reference, and there are many ways of skinning that cat including GPS, radio time signals or if you have network connectivity, NTP.

[obiwanjacobi]

Can't you use an atomic clock radio receiver (60kHz I believe)? Less than 10,=

Just wake on an internal timer (with drift) and record the correct (atomic) time and data in the logger...
You only power on the atomic clock receiver when actually taking a measurement of course, saving battery life.

[2c]

[NivagSwerdna]

Have a look at https://edwardmallon.wordpress.com/2015/10/24/diy-arduino-logger-build-instructions-part-1/

DS3221 is what I use... so simple and very cheap.  You can use the DS3221 to wake up the AVR from sleep so quiescent power consumption is low

[Ian.M]

Yes. A DS3221 module would be a good choice - it can hold better than 2 minutes per year.

[brucehoult]

Hey everyone,

I'm thinking of running a datalogger (simple DHT11 temp/humidity system) for about a year or so using an Arduino (using the Simple Timer library to set intervals of about 30mins) would this drift considerably over the long term and is there a better way to do timing without a hardware RTC?

Thanks

I used an Arduino Uno without RTC to control my home heating system, with a schedule of daily times to change the set temperature (actually, smoothly changing over a period of hours, not step changes).

If I recall correctly, the raw millis() time ran fast a minute or so a day, but it was a consistent amount on that particular Arduino. I did a little hack that subtracted 1 from the Arduino library's internal ms variable every 1300 ms (your number may vary) and with that it kept time to something like 1 minute per month, and that was good enough.

The whole point of the thing was to be a thermostat, and my temperature changes weren't big -- completely turning the system off overnight would make the house go from 21 C to maybe 18 C, and I didn't let it get *that* cold -- so I didn't have to worry about the clock running at different speeds over a wide range of temperatures like you'd find outdoors.

[brucehoult]

Hey everyone,

I'm thinking of running a datalogger (simple DHT11 temp/humidity system) for about a year or so using an Arduino (using the Simple Timer library to set intervals of about 30mins) would this drift considerably over the long term and is there a better way to do timing without a hardware RTC?

Thanks

I used an Arduino Uno without RTC to control my home heating system, with a schedule of daily times to change the set temperature (actually, smoothly changing over a period of hours, not step changes).

If I recall correctly, the raw millis() time ran fast a minute or so a day, but it was a consistent amount on that particular Arduino. I did a little hack that subtracted 1 from the Arduino library's internal ms variable every 1300 ms (your number may vary) and with that it kept time to something like 1 minute per month, and that was good enough.

The whole point of the thing was to be a thermostat, and my temperature changes weren't big -- completely turning the system off overnight would make the house go from 21 C to maybe 18 C, and I didn't let it get *that* cold -- so I didn't have to worry about the clock running at different speeds over a wide range of temperatures like you'd find outdoors.

Oh, it was also important to not let anything turn off interrupts for as long as 1 ms, otherwise you could miss millis() updates. I was using a standard library to do 433 MHz carrier wave remote control stuff and it was turning interrupts off for many ms while sending a burst. Completely pointless as it didn't need anything like us accuracy in the timing. I ripped out the turning off interrupts stuff and all was good with the world again.

[Thane of Cawdor]

More information is needed. How accurate does it have to be? Is this going to be indoor or outdoor (how much temperature change)? Is power a limitation, if so how much? Are other resources available (network, external devices, RTC, GPS, RF, IR, etc.)? Does the system have to be sealed from the rest of the world? What will be its geographical location?

Answers to these questions will narrow down which of many possible solutions could work for your application.

Thanks for the responses! It doesn't need to be too accurate something like +- 15mins should be fine. It's measuring temperature/humidity outdoors and power is not a limitation. I am also using the ESP8266 connected to a Wi-Fi network so is there a way to get time data and then recalibrate an onboard RTC for timing each sensor reading (so that if it does not connect correctly to Wi-Fi, my readings would still occur at the specified period)?

[Avacee]

Getting the time from a TimeServer is one of the included examples with the Arduino Ethernet Library.
https://www.arduino.cc/en/Tutorial/UdpNTPClient

You can then set the time in the RTC module ....
You can also check the RTC drift and correct .. ie after 12 hours if the drift is 12 seconds then correct 1 second per hour assuming linear drift.

But if you've got access to a Time Server and you don't need a high level of accuracy you can ditch the RTC and just correct the Arduino's drift.
So many options!

[bitseeker]

I second Avacee's recommendation since you have plenty of power available and Wi-Fi. Of course, if you already have an RTC in your parts supply, it's convenient to have it take care of the ongoing time.

[Nusa]

If you've got an 8266 involved, the Arduino is pretty redundant. The 8266 has the power and gpio pins to handle this task.

In fact, this exact project has already been documented using the 8266: https://learn.adafruit.com/esp8266-temperature-slash-humidity-webserver

[NivagSwerdna]

I'm finding this thread a bit challenging!

If you have WiFi then forget time... just send the data!  If you want to be really pedantic then keep track of the number of millis in the past when you took the sample and send it with the data to help with calculating times for delayed/retried samples.  No accurate local clock required.

[Thane of Cawdor]

 

If you have WiFi then forget time... just send the data!  If you want to be really pedantic then keep track of the number of millis in the past when you took the sample and send it with the data to help with calculating times for delayed/retried samples.  No accurate local clock required.

So I'm just going to go with a single ESP8266 configured to run with the Arduino IDE but I just wanted the intervals between each measurement to be stable over time. Wouldn't I need to have a local RTC (which is calibrated each time connection is re-established with Wi-Fi) always running in the event that the Wi-Fi connection drops and start each sensor reading by the monitoring the time from RTC rather than a millis or delay counter?

Could you elaborate on keeping track of the number of millis for making sensor reading intervals more consistent?
 

[ebclr]

RTC was invented just for it, and is extremely cheap

Arduino compatible modules based on ESP32 have the 2nd crystal just for timer, and can be a better option for standard Arduino for data logging

Also, GPS modules can give you timestamps with extremely high accuracy

[Avacee]

I just wanted the intervals between each measurement to be stable over time.

The $64,000 is "how stable?"
Do you need millisecond accuracy for a reading you take every 30 minutes?
How much drift per day / per week / per year is acceptable to you?  ... you said +/- 15 mins but didn't give a scale.

5 min per week (worst case) - Internal Timer is fine
1 min per week (best case) - Internal Time is fine
5 mins over the year = RTC is fine .. seriously does it matter if the recording is taken at 12:05 instead of 12:00 after 1 year?

Anything more accurate needs a calibration check - NTP TimeServer (you're on wifi, right?)/ Communication to your PC (that's going to be pretty accurate) / GPS (very accurate but cost more and needs power)/ More Accurate (and expensive) RTC / A.N.Other

So, how accurate do you want?