Author Topic: next step learning after Arduino ?  (Read 4378 times)

0 Members and 2 Guests are viewing this topic.

Offline glenenglishTopic starter

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: au
  • RF engineer. AI6UM / VK1XX . Aviation pilot. MTBr
Re: next step learning after Arduino ?
« Reply #50 on: July 14, 2024, 08:20:30 pm »
well, I think single steppng, register inspection, etc   is enormously useful.
It is not. The reason is that you'll also need to know what each bit and/or instruction means. There is no use in looking at a bundle of data and franticly going through documentation which describes each bit and/or instruction. Errors are easely made. A much better way is to simply print the data as text where the meaning of bits is translated into a human readable form. People don't program in assembly nowadays and scripted languages are also being used increasingly.

Not if you use a modern debugger that permits single stepping of source code, or single stepping of the disassembly, wih modification of registers and CC while the CPU is halted......  there's no need to remember the bits and opcodes anymore, unless you want to, like Bruce, which is fine.
I'm not writing about opcodes but assembler instructions and what bits mean in peripheral registers. Another problem is that you can only step through C / C++ code which isn't optimised OR has been optimised to allow debugging (gcc flags -O0 or -Og). Stepping through optimised code is hit & miss (with mostly miss) as there is no longer a 1 to 1 relation between the machine code and source code.

Helpful  debuggers have all the peripheral registers graphically illustrated, even Microchip AVR studio does that. No need to remember anything, except of course you will have the datasheet for the device open on another monitor as you trying and figure out, for example what the counter-timer bits must be for some bespoke config , some of these timer peripherals are tricky. 
Yes, the program needs to be able to run with -Og or -O0 otherwise single stepping is a nightmare.
Something though which is more difficult with single stepping, is when you have a microprocessor with a deep / reordering pipeline, This is fairly uncommon with small micros.
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7058
  • Country: va
Re: next step learning after Arduino ?
« Reply #51 on: July 14, 2024, 08:31:07 pm »
Quote
as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

Source debugging on the hardware shows that the branches in the code you expected it to take are actually taken. That variables have the value you expect them to have, that this function is called, etc. For that the source-level on-device debugging is fine, in not actually perfect for the task. The only real question is "what are you looking for"?

(If this - the source-level ignore machine code thing - wasn't the case then you couldn't run unit tests.)
 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 20170
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: next step learning after Arduino ?
« Reply #52 on: July 14, 2024, 08:35:08 pm »
well, I think single steppng, register inspection, etc   is enormously useful.
It is not. The reason is that you'll also need to know what each bit and/or instruction means. There is no use in looking at a bundle of data and franticly going through documentation which describes each bit and/or instruction. Errors are easely made. A much better way is to simply print the data as text where the meaning of bits is translated into a human readable form. People don't program in assembly nowadays and scripted languages are also being used increasingly.

Not if you use a modern debugger that permits single stepping of source code, or single stepping of the disassembly, wih modification of registers and CC while the CPU is halted......  there's no need to remember the bits and opcodes anymore, unless you want to, like Bruce, which is fine.
I'm not writing about opcodes but assembler instructions and what bits mean in peripheral registers. Another problem is that you can only step through C / C++ code which isn't optimised OR has been optimised to allow debugging (gcc flags -O0 or -Og). Stepping through optimised code is hit & miss (with mostly miss) as there is no longer a 1 to 1 relation between the machine code and source code.

One of the problems with C/C++ is determining whether you have given the compiler the correct instructions (i.e. source code plus arguments/flags) to generate the code you want. Just looking at disassembled optimised machine code is too difficult to understand within a reasonable timeframe.

Stepping through at the machine code level can speed up that understanding. Then all you have to do is work out why the compiler created that spaghetti, and how you have to change your instructions to the compiler. That is left as an exercise for the student  >:D

OTOH, I know of a toolchain that starts with the optimised assembler, and works back to define how many cycles (i.e. min/max) it takes to get from here to there. The route isn't always obvious!

None of that helps with the interaction between multiple threads/processes. Back in 68020 days, you could capture the bus activity, and (painfully!) work out what was happening; no longer. Yes, I once had to do that to find the fault inside a third-party RTOS; ignoring irrelevant/unexecuted prefetched instructions was tedious in the extreme.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 20170
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: next step learning after Arduino ?
« Reply #53 on: July 14, 2024, 08:41:43 pm »
Quote
as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

One difficulty is to understand why the processor isn't doing what you thought you told it to do. That's a real problem with C/C++ code, where nctnico's statement is correct. A second difficulty is to understand why.

A good toolset will minimise problems in the first place, and help to spot and diagnose problems. A bad toolset will start by blaming the developer for not understanding the tools.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7058
  • Country: va
Re: next step learning after Arduino ?
« Reply #54 on: July 14, 2024, 09:02:23 pm »
Quote
as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

One difficulty is to understand why the processor isn't doing what you thought you told it to do.

Hence the 'yet' in my comment. In order of probability, the code may not be working because 1) you wrote it wrong, 2) you wrote it correctly to do the wrong thing, 3) there's a compiler issue, 4) there's a processor issue. It is exceedingly rare that you get past 2 IME, but it sometimes happens and you find it out by the source level stepping going wrong.
[/quote]
 
The following users thanked this post: nctnico

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 20170
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: next step learning after Arduino ?
« Reply #55 on: July 14, 2024, 09:35:30 pm »
Quote
as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

One difficulty is to understand why the processor isn't doing what you thought you told it to do.

Hence the 'yet' in my comment. In order of probability, the code may not be working because 1) you wrote it wrong, 2) you wrote it correctly to do the wrong thing, 3) there's a compiler issue, 4) there's a processor issue. It is exceedingly rare that you get past 2 IME, but it sometimes happens and you find it out by the source level stepping going wrong.

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

Examples:
  • C++ templates. The committee refused to believe the complexity until Erwin Unruh rubbed their noses in the problem by giving them a simple, correct, valid, conforming program that can never finish being compiled. Why not? Because the compilation process itself generates the (infinite) sequence of prime numbers. If the committee can't understand what they have created, ordinary engineers employed to "drain the swamp" have no chance
  • Hans Boehm had to forcibly "remind" people that you couldn't - by design - create a threading library in C. As a consequence the committee finally realised C needed a defined memory model. (Finally, since everybody else had known it for several decades!)

Then there are the compiler/linker flags, which are compiler dependent and whose actions change over time. Hence the requirement that "this code is compiled and linked using tool X version a.b.c" - which became obsolete just after the code was first delivered a decade or more ago.
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 
The following users thanked this post: pardo-bsso

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7058
  • Country: va
Re: next step learning after Arduino ?
« Reply #56 on: July 14, 2024, 09:50:41 pm »
Quote
You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

As I noted previously, if single-stepping is useless without intimate knowledge of tiniest part of your toolset and the intricacies of the chip then unit tests would be useless. You need to concentrate on the particular abstraction level that you''re debugging at, and at source level I reckon we can hazard a good guess as to what that is.
 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 20170
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: next step learning after Arduino ?
« Reply #57 on: July 14, 2024, 10:07:07 pm »
Quote
You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

As I noted previously, if single-stepping is useless without intimate knowledge of tiniest part of your toolset and the intricacies of the chip then unit tests would be useless. You need to concentrate on the particular abstraction level that you''re debugging at, and at source level I reckon we can hazard a good guess as to what that is.

There's some value to that statement in that context.

But the point is that the tools have become so complex that they are part of the problem. A change in an apparently unrelated part of the source code can change what you have already "debugged".
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline Postal2

  • Frequent Contributor
  • **
  • Posts: 317
  • Country: ru
Re: next step learning after Arduino ?
« Reply #58 on: July 14, 2024, 10:36:53 pm »
..... The hardest decision would likely be what to do with it - some target to work towards is useful.
I can give you an idea. Make an universal touchscreen chip converter (i2c). This is a big problem when you have a screen with a touch where the touch can't connect anywhere.

I have more ideas, but they are little complex.
« Last Edit: July 14, 2024, 10:40:53 pm by Postal2 »
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 27485
  • Country: nl
    • NCT Developments
Re: next step learning after Arduino ?
« Reply #59 on: July 14, 2024, 10:40:22 pm »
Quote
You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.
And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 20170
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: next step learning after Arduino ?
« Reply #60 on: July 14, 2024, 10:55:50 pm »
Quote
You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.
And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

The defined behaviour has a tendency to change over time, as the compilers become ever more "capable".

Undefined behaviour also changes over time and exposes more problems, as the compilers become ever more "capable".
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 27485
  • Country: nl
    • NCT Developments
Re: next step learning after Arduino ?
« Reply #61 on: July 15, 2024, 12:15:50 pm »
Quote
You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.
And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

The defined behaviour has a tendency to change over time, as the compilers become ever more "capable".

Undefined behaviour also changes over time and exposes more problems, as the compilers become ever more "capable".
I don't think that is 100% true. For example: there is some ambiguity where it comes to how the << and >> work in C. The way compiler makers have implemented these is a way which works as expected. Nobody with a sane mind is going to change this behaviour because it would break a huge amount of code.

Most problems with incompatibilities I've encountered are with header files, pre-defined values and things which used to threw a warning becoming an error.
« Last Edit: July 15, 2024, 02:00:20 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline RoGeorge

  • Super Contributor
  • ***
  • Posts: 6522
  • Country: ro
Re: next step learning after Arduino ?
« Reply #62 on: July 15, 2024, 12:30:19 pm »
Try stepping away from the Arduino IDE, and use C and Makefile from the command line, or use some other IDE, like Eclipse.

For the hardcore, learn about gdb, and try to make use of "debugWIRE" with an "Arduino UNO" or with an "Arduino nano" board.  Many AVR microcontrollers can use debugWire as a hardware debugger, programmer, step-by -step execution, memory and peripheral inspection, breakpoints, etc.

Did that setup once for an ATtiny13 microcontroller, and a generic CH340G USB-to-serial adapter as a programmer/hardware debugger:  https://www.eevblog.com/forum/microcontrollers/avr-(arduino)-linux-debug/ (which reminds me I still didn't post the clean instructions of how to set all that)

« Last Edit: July 15, 2024, 12:42:18 pm by RoGeorge »
 
The following users thanked this post: glenenglish

Offline tggzzz

  • Super Contributor
  • ***
  • Posts: 20170
  • Country: gb
  • Numbers, not adjectives
    • Having fun doing more, with less
Re: next step learning after Arduino ?
« Reply #63 on: July 15, 2024, 12:52:52 pm »
Quote
You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.
And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

The defined behaviour has a tendency to change over time, as the compilers become ever more "capable".

Undefined behaviour also changes over time and exposes more problems, as the compilers become ever more "capable".
I don't think that is 100% true. For example: there is some abiquity where it comes to how the << and >> work in C. The way compiler makers have implemented these is a way which works as expected. Nobody with a sane mind is going to change this behaviour because it would break a huge amount of code.

Most problems with incompatibilities I've encountered are with header files, pre-defined values and things which used to threw a warning becoming an error.

Breaking working code/applications is unpleasant wherever it occurs. It is a traditional problem with C and C++. Other languages cause zero or far fewer problems in that way.

Previous/historic code presumed the language definition (and language ambiguity) in force at that time.  Not unreasonable.

Compilers have, over the years, become more aggressive at interpreting the language specification to the compiler's advantage, in the name of gaining minor performance enhancements. That breaks some historic code which used to work, sometimes to the point of omitting to generate the object code! [1]

The traditional response is to blame the developer, implicitly for not taking account of what might happen in the future. Personally I think that is disreputable; the tools are the problem.

[1] That has been a problem with some well-known benchmarks; later versions of the compiler have run the benchmarks "infinitely" fast. Some compilers long before VolksWagen and Toyota - detected they are compiling the benchmark, and emitted unjustifiably optimised code .
There are lies, damned lies, statistics - and ADC/DAC specs.
Glider pilot's aphorism: "there is no substitute for span". Retort: "There is a substitute: skill+imagination. But you can buy span".
Having fun doing more, with less
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4251
  • Country: us
Re: next step learning after Arduino ?
« Reply #64 on: July 15, 2024, 06:53:16 pm »
Quote
executing with the debugger running still messes with things


It is important to learn about the Heisenberg Uncertainty Principle as it applies to software debugging.  (well, WE called some problems "Heisenbugs"!)
That is, you need to understand in what what ways your debugging process (whatever it is) affects the behavior of your code.


Also, different types of debugging imply that different tools are useful.
Single-stepping through source code mat be great for debugging your C program flow, especially if you've made some stupid mistake that you're not seeing in the source (I mean, mistakes that you SHOULD be able to see in the source, but...)
But of course it's pretty useless for debugging timing-related bugs in your bitbang code.
And if you think you're chasing a compiler bug, you'll need other tools.

 
The following users thanked this post: Psi

Offline glenenglishTopic starter

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: au
  • RF engineer. AI6UM / VK1XX . Aviation pilot. MTBr
Re: next step learning after Arduino ?
« Reply #65 on: July 15, 2024, 09:03:12 pm »
Good suggestion RoGeorge.

could- stay with simple platform - and learn a bit of the background - make, gdb

I think learning the background stuff is important, because once you are in the land of pre-canned GUI environments, like Eclipse, VS code, if something doesnt work in the build sequence, if you dont understand the basics and what is going on in the background, you'll find it time consuming to solve.

 

Offline Johnny B Good

  • Frequent Contributor
  • **
  • Posts: 827
  • Country: gb
Re: next step learning after Arduino ?
« Reply #66 on: July 21, 2024, 04:29:05 am »
 I've been drawn to this topic thread on account of a gremlin (most definitely not a bug!) that I've been struggling to resolve over the past few weeks while I was tuning the temperature control algorithm of a rubidium oscillator's base plate to cope with sub 10 degree ambient temperatures.

 I had finally figured out how to get it to cope with ambient temperatures to within a degree of its theoretical 33 deg maximum (a three degree delta between the base plate's 36.05 deg set point and maximum ambient) and now wanted to verify that it would still cope with 4 deg as it had seemingly done during the winter months test runs in my outside workshop.

 Unfortunately this is the wrong time of the year to be using the outside workshop as a low temperature test chamber so I had the bright idea to use my fridge as an 'all seasons' low temperature test chamber. The initial test runs exceeded my wildest dreams of milli - Kelvin stability right down to 10 degrees. Unfortunately, my dreams were shattered just as the temperature had dropped to 8 degrees when it suddenly took a deep dive into endless cycles of under/over shooting, only managing to return to a stable set point once the temperature had risen back to 10 degrees.

 At first I assumed I hadn't scaled the corrective responses at low fan intake temperatures enough to avoid this instability when dealing with a thermal time constant in the region of some 35 to 40 seconds or so. I had been under the impression that the Atmel M328p used on Nano and its slightly larger cousin used on the UNO could do no wrong and therefore it was down to me for not properly considering the increased instability with fan cooling as the sole means of temperature control at these low temperatures.

 Either that, or I had simply reached the limits of just what was possible and was now "Asking for The Moon", despite my observing overshoot excursions with the fan just barely ticking over, rather than being brought to a total standstill as in my initial versions of my program, suggesting otherwise. Indeed it was my curiosity over just how far I could raise this minimum tick-over drive level to before it would prevent overshoot and start reducing the temperature ever so slightly under such low ambient temperature conditions.

 To this end, I simply over-rode the fan control with a set of static fanDrive test values, trying to home in on the 'magic number' and it was when I had reached a value of 255 that I spotted a rapid temperature drop rather than the expected small variation in temperature. Sure enough, trying out adjacent values (254 and 256) did give the expected resulting slight change of temperature drift rate and going back to 255 to retest this peculiar behavior did confirm that with pins D9 and D10 configured to output 10 bit PWM, that one specific 'gremlin value' would mimic the effect of generating a value of 1023.

 Later testing with my bread boarded "Flight Spare" followed by writing a small test program to exercise the UNO's 10 bit PWM performance confirmed this to be a shortcoming in these micro controllers (or possibly some cockup in the Arduino's compiler or a built in library function).

 With this 'gremlin' only appearing at a rather suspicious 100% for the 8 bit case, I did begin to wonder whether this gremlin repeated itself over the next two multiples of 256 (511 and 767). In this case I resorted to the age old remedy of "Shoot first, ask questions later" approach, using the simple work around of testing for fanDrive for values of 255 and substituting this gremlin value with 254 on every such occurrence.

 As later testing proved, 255 was the only gremlin value over the full 10 bit range so proved an almost perfect solution until I closed a backdoor I'd forgotten I'd left open in the display update code for the gremlin to have its wicked way. More extreme low temperature testing had hinted at a shadow of its presence elsewhere in my code. It would have been a more obvious presence but for the fact that I'd added a test to override whatever fanDrive value was going to be used with a fanDrive level of the 215 tickover value whenever the base plate temperature undershot the set point by 25mK or more.

 The whole point of the above description was to highlight the fact that not all bugs can be analysed using debugging tools (or even logic analysers for that matter). More accurately put, debugging tools cant handle gremlins. In such cases, as a few in this thread have already pointed out, careful analysis of the code and the behavior of the hardware under its control may be the only way to resolve such gremlins.

 If anyone is interested in getting to the bottom of this particular gremlin, I've attached the gremlin test code I'd run on the UNO (should work just the same on a nano). I'd be interested in knowing whether it's just clones that suffer this 10 bit PWM gremlin or whether it also afflicts genuine Arduino boards as well.

visit <https://nerdytechy.com/how-to-change-the-pwm-frequency-of-arduino/#Pins_D9_and_D10_Timer_1_10_bit> for the original reference I'd used to set the timer 1 /D9 /D10 PWM to 10 bit mode.
« Last Edit: July 21, 2024, 04:42:29 am by Johnny B Good »
John
 

Offline glenenglishTopic starter

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: au
  • RF engineer. AI6UM / VK1XX . Aviation pilot. MTBr
Re: next step learning after Arduino ?
« Reply #67 on: July 21, 2024, 04:54:15 am »
Hi JBG
this should really go in its own thread...... suggest move it there, since this will generate alot of responses ....
-----------------------------------------------------------

I've never encountered badness in ye-olde AVR (1999-2008 stuff)  and given the value, it's likely a cock up in the avr library,.
I''ve only ever used peripherals directly, never used arduino for anything more than a doorbell.

Might be a good time to manipulate the port directly. The AVR timers are quite simple. 

For what you're doing, I'd make everything signed so you can never have a rollover F.U.
and watch out for non-monotonic sensors !...

I assume you are using the PWM library https://github.com/atmelino/Arduino/blob/master/libraries/PWM/utility/ATimerDefs.cpp

watch out there  is void pwmWrite(uint8_t pin, uint8_t val) and also there is void pwmWriteHR(uint8_t pin, uint16_t val)

in  pwmWriteHR(), somehow verify that       
   if(td.Is16Bit)
is being executed correctly.

, anyway, suggest another thread.  cheers

 
The following users thanked this post: Johnny B Good

Offline PlainName

  • Super Contributor
  • ***
  • Posts: 7058
  • Country: va
Re: next step learning after Arduino ?
« Reply #68 on: July 21, 2024, 07:29:09 am »
...

I am not an Arduino guru, but a red flag to me in that code is that you're using a float to store 1033 values that you can then average. A float can represent huge numbers, but it only uses 32 bits to do so, and taking away the sign and scaling info you're left with 23 bits of actual number. So the resolution of a float is very much poorer than a single 32-bit int.

I would think about placing a bet that your use of a float like this is introducing the 255 magic value (and, indeed, '255' being a magic value fires off multiple bells).

I think, if you want to average that way then you should use a double long, which is a 64-bit variable. It will also be quite a bit faster to use.

But... assuming that is the answer to your problem, 1033 values is a lot (and oddly precise), and I would think about alternative ways to achieve the end of smoothing that out. Perhaps, through dropping the float as store, an appropriate number of values to average would be fewer.

Edit: fixed fingers not catching up with brain error (typed double, meant long). Also it seems the Arduino int is 16-bit whereas I assumed 32-bit. Doesn't make any practical difference though.
« Last Edit: July 21, 2024, 09:39:33 am by PlainName »
 
The following users thanked this post: Johnny B Good

Offline glenenglishTopic starter

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: au
  • RF engineer. AI6UM / VK1XX . Aviation pilot. MTBr
Re: next step learning after Arduino ?
« Reply #69 on: July 21, 2024, 09:18:22 am »
I'd be averaging with an integer boxcar filter. no precision issues.  maybe use int32 for the PID filter. (decide on a decimal point location perhaps 8) . Also if you do=do float to int, be sure to add 0.5f for all positive float values so that you round to nearest instead of round down 
« Last Edit: July 21, 2024, 09:20:56 am by glenenglish »
 

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 4313
  • Country: nz
Re: next step learning after Arduino ?
« Reply #70 on: July 21, 2024, 10:05:43 am »
If I wanted to smooth values over about 1000 (equal) time periods then I'd do like this...

Code: [Select]
// init
int32_t smoothed = first_sample << 10;

// then each time period
smoothed = smoothed - ((smoothed + 512) >> 10) + new_sample;

Obviously you need 10 more bits for "smoothed" than for the samples, so if the samples are in 0..255 or 0..1023 you'll need "smoothed" to be 24 bits, which you can do if you're hand-coding it, or just use the 32 bit integers the compiler provides.


You could use "smoothed" directly as the control variable for your PID, or maybe shift it right a few places to bring it into range of a 16 bit integer.

About a dozen years ago I used an Uno to control my home heating using a 2400W oil column heater. I used a thermistor to read the temperature, did an ADC 100 times a second, and smoothed over 30 seconds.  This gave me a nice 0.01 C resolution on the temperature that moved very smoothly, without noise, as the room changed temperature. Obviously the accuracy wasn't to 0.01 C, but it gave a very nice indication of the direction and speed of the room changing temperature.

I made a decision whether the heater should be on or off every 30 seconds, using a similar smoothing process to integrate an approximation of how much heat was currently stored in the heater, adding to the stored heat every 30 seconds and assuming the heater decayed to room temperature with a ~15 minute time constant (i.e. assume 63% (?) of the remaining stored energy is transferred to the room every 15 minutes)

It worked really really well, with the (self measured) temperature variation in normal use about 0.02 C. A transient such as starting or stopping cooking generally gave about a maximum 0.2 C change in temperature, brought smoothly back to the set value within 20 minutes. I could also make reasonable changes to the set temperature (±0.5 or ±1 C, say) and the temperature would smoothly converge on the new temperature in 20-30 minutes, with just a little overshoot (like 0.2 C)
« Last Edit: July 21, 2024, 10:19:47 am by brucehoult »
 

Offline glenenglishTopic starter

  • Frequent Contributor
  • **
  • Posts: 422
  • Country: au
  • RF engineer. AI6UM / VK1XX . Aviation pilot. MTBr
Re: next step learning after Arduino ?
« Reply #71 on: July 21, 2024, 10:20:05 am »
agreed with Bruce there if you want to smooth 1000 samples. Boxcar is accurate and super cheap, but needs memory...  other option would be carefully precise single pole IIR filter. you might get limit cycles if you look hard enough.
just be careful not to try and use any more precision than your control system provides for, specifically relating to non-monotonicity of sensors and actuators. They can cause loops and cycles.

Bruce what did it do if you repeatidly opened a door every exactly 30 seconds ?
« Last Edit: July 21, 2024, 10:21:52 am by glenenglish »
 

Offline ozcar

  • Frequent Contributor
  • **
  • Posts: 329
  • Country: au
Re: next step learning after Arduino ?
« Reply #72 on: July 21, 2024, 11:23:42 am »

...

I would think about placing a bet that your use of a float like this is introducing the 255 magic value (and, indeed, '255' being a magic value fires off multiple bells).


The documented maximum value for analogWrite is 255.

Some random has mislead Johnny into believing that a simple trick will allow 10 bit resolution, with values up to 1023, by messing with the timer registers to invoke 10-bit rather than 8-bit PWM (obviously not possible for 8-bit timers).

From what I can see, this happens to work for all values from 0 to 1023, except 255.

Source:

https://github.com/arduino/ArduinoCore-avr/blob/master/cores/arduino/wiring_analog.c

Code: [Select]
void analogWrite(uint8_t pin, int val)
{
        ...
if (val == 0)
{
digitalWrite(pin, LOW);
}
else if (val == 255)
{
digitalWrite(pin, HIGH);
}
else
{
              ...
              ... set the appropriate timer OCR register from val -
              ... for a 16bit timer it happens to write to the OCR as a 16-bit field, rather than only to
              ... the low 8-bit half - eg
              OCR1A = val; // set pwm duty
              ... rather than
              OCR1AL = val;
              ...
        }
}
 
The following users thanked this post: PlainName

Online brucehoult

  • Super Contributor
  • ***
  • Posts: 4313
  • Country: nz
Re: next step learning after Arduino ?
« Reply #73 on: July 21, 2024, 11:26:26 am »
Bruce what did it do if you repeatidly opened a door every exactly 30 seconds ?

I didn't have enough doors to do that.
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3225
  • Country: ca
Re: next step learning after Arduino ?
« Reply #74 on: July 21, 2024, 02:55:44 pm »
More accurately put, debugging tools cant handle gremlins.

Sure they can. Debugging tools are just tools you can use to view things, like a microscope. The trick is not to know how to use a microscope. The trick is to know what to look at. The same with the debugger. You could have stepped through the code to see how the code sets the value of the register. Or, you could have used the debugger  to inspect the register value after you have set it. This would immediately reveal the nature of the problem.

Moreover, you could have looked at the same things without a debugger. For example, you could have looked at the source code to see how it sets the register. Or you could have inspected the disassembly. Or you could re-write the code and set the value of the register directly (instead of using Arduino function) and then observe if this fixes the problem.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf