next step learning after Arduino ?

[glenenglish]

well, I think single steppng, register inspection, etc   is enormously useful.

It is not. The reason is that you'll also need to know what each bit and/or instruction means. There is no use in looking at a bundle of data and franticly going through documentation which describes each bit and/or instruction. Errors are easely made. A much better way is to simply print the data as text where the meaning of bits is translated into a human readable form. People don't program in assembly nowadays and scripted languages are also being used increasingly.

Not if you use a modern debugger that permits single stepping of source code, or single stepping of the disassembly, wih modification of registers and CC while the CPU is halted......  there's no need to remember the bits and opcodes anymore, unless you want to, like Bruce, which is fine.

I'm not writing about opcodes but assembler instructions and what bits mean in peripheral registers. Another problem is that you can only step through C / C++ code which isn't optimised OR has been optimised to allow debugging (gcc flags -O0 or -Og). Stepping through optimised code is hit & miss (with mostly miss) as there is no longer a 1 to 1 relation between the machine code and source code.

Helpful  debuggers have all the peripheral registers graphically illustrated, even Microchip AVR studio does that. No need to remember anything, except of course you will have the datasheet for the device open on another monitor as you trying and figure out, for example what the counter-timer bits must be for some bespoke config , some of these timer peripherals are tricky. 
Yes, the program needs to be able to run with -Og or -O0 otherwise single stepping is a nightmare.
Something though which is more difficult with single stepping, is when you have a microprocessor with a deep / reordering pipeline, This is fairly uncommon with small micros.

[PlainName]

as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

Source debugging on the hardware shows that the branches in the code you expected it to take are actually taken. That variables have the value you expect them to have, that this function is called, etc. For that the source-level on-device debugging is fine, in not actually perfect for the task. The only real question is "what are you looking for"?

(If this - the source-level ignore machine code thing - wasn't the case then you couldn't run unit tests.)

[tggzzz]

well, I think single steppng, register inspection, etc   is enormously useful.

It is not. The reason is that you'll also need to know what each bit and/or instruction means. There is no use in looking at a bundle of data and franticly going through documentation which describes each bit and/or instruction. Errors are easely made. A much better way is to simply print the data as text where the meaning of bits is translated into a human readable form. People don't program in assembly nowadays and scripted languages are also being used increasingly.

Not if you use a modern debugger that permits single stepping of source code, or single stepping of the disassembly, wih modification of registers and CC while the CPU is halted......  there's no need to remember the bits and opcodes anymore, unless you want to, like Bruce, which is fine.

I'm not writing about opcodes but assembler instructions and what bits mean in peripheral registers. Another problem is that you can only step through C / C++ code which isn't optimised OR has been optimised to allow debugging (gcc flags -O0 or -Og). Stepping through optimised code is hit & miss (with mostly miss) as there is no longer a 1 to 1 relation between the machine code and source code.

One of the problems with C/C++ is determining whether you have given the compiler the correct instructions (i.e. source code plus arguments/flags) to generate the code you want. Just looking at disassembled optimised machine code is too difficult to understand within a reasonable timeframe.

Stepping through at the machine code level can speed up that understanding. Then all you have to do is work out why the compiler created that spaghetti, and how you have to change your instructions to the compiler. That is left as an exercise for the student 

OTOH, I know of a toolchain that starts with the optimised assembler, and works back to define how many cycles (i.e. min/max) it takes to get from here to there. The route isn't always obvious!

None of that helps with the interaction between multiple threads/processes. Back in 68020 days, you could capture the bus activity, and (painfully!) work out what was happening; no longer. Yes, I once had to do that to find the fault inside a third-party RTOS; ignoring irrelevant/unexecuted prefetched instructions was tedious in the extreme.

[tggzzz]

as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

One difficulty is to understand why the processor isn't doing what you thought you told it to do. That's a real problem with C/C++ code, where nctnico's statement is correct. A second difficulty is to understand why.

A good toolset will minimise problems in the first place, and help to spot and diagnose problems. A bad toolset will start by blaming the developer for not understanding the tools.

[PlainName]

as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

One difficulty is to understand why the processor isn't doing what you thought you told it to do.

Hence the 'yet' in my comment. In order of probability, the code may not be working because 1) you wrote it wrong, 2) you wrote it correctly to do the wrong thing, 3) there's a compiler issue, 4) there's a processor issue. It is exceedingly rare that you get past 2 IME, but it sometimes happens and you find it out by the source level stepping going wrong.
[/quote]

[tggzzz]

as there is no longer a 1 to 1 relation between the machine code and source code

I don't think there needs to be.  When doing source-level debugging you are not (yet!) debugging machine code or register details. You are debugging the source. It either does what's expected or it doesn't, and if it doesn't then you've either written something wrong or - and this is where the machine code comes in - the compiler has boobied. That quite rare and shouldn't be your first or second suspicion of what the problem is. You are definitely not debugging the compiler of microprocessor.

One difficulty is to understand why the processor isn't doing what you thought you told it to do.

Hence the 'yet' in my comment. In order of probability, the code may not be working because 1) you wrote it wrong, 2) you wrote it correctly to do the wrong thing, 3) there's a compiler issue, 4) there's a processor issue. It is exceedingly rare that you get past 2 IME, but it sometimes happens and you find it out by the source level stepping going wrong.

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

Examples:

• C++ templates. The committee refused to believe the complexity until Erwin Unruh rubbed their noses in the problem by giving them a simple, correct, valid, conforming program that can never finish being compiled. Why not? Because the compilation process itself generates the (infinite) sequence of prime numbers. If the committee can't understand what they have created, ordinary engineers employed to "drain the swamp" have no chance
• Hans Boehm had to forcibly "remind" people that you couldn't - by design - create a threading library in C. As a consequence the committee finally realised C needed a defined memory model. (Finally, since everybody else had known it for several decades!)

Then there are the compiler/linker flags, which are compiler dependent and whose actions change over time. Hence the requirement that "this code is compiled and linked using tool X version a.b.c" - which became obsolete just after the code was first delivered a decade or more ago.

[PlainName]

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

As I noted previously, if single-stepping is useless without intimate knowledge of tiniest part of your toolset and the intricacies of the chip then unit tests would be useless. You need to concentrate on the particular abstraction level that you''re debugging at, and at source level I reckon we can hazard a good guess as to what that is.

[tggzzz]

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

As I noted previously, if single-stepping is useless without intimate knowledge of tiniest part of your toolset and the intricacies of the chip then unit tests would be useless. You need to concentrate on the particular abstraction level that you''re debugging at, and at source level I reckon we can hazard a good guess as to what that is.

There's some value to that statement in that context.

But the point is that the tools have become so complex that they are part of the problem. A change in an apparently unrelated part of the source code can change what you have already "debugged".

[Postal2]

..... The hardest decision would likely be what to do with it - some target to work towards is useful.

I can give you an idea. Make an universal touchscreen chip converter (i2c). This is a big problem when you have a screen with a touch where the touch can't connect anywhere.

I have more ideas, but they are little complex.

[nctnico]

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

[tggzzz]

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

The defined behaviour has a tendency to change over time, as the compilers become ever more "capable".

Undefined behaviour also changes over time and exposes more problems, as the compilers become ever more "capable".

[nctnico]

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

The defined behaviour has a tendency to change over time, as the compilers become ever more "capable".

Undefined behaviour also changes over time and exposes more problems, as the compilers become ever more "capable".

I don't think that is 100% true. For example: there is some ambiguity where it comes to how the << and >> work in C. The way compiler makers have implemented these is a way which works as expected. Nobody with a sane mind is going to change this behaviour because it would break a huge amount of code.

Most problems with incompatibilities I've encountered are with header files, pre-defined values and things which used to threw a warning becoming an error.

[RoGeorge]

Try stepping away from the Arduino IDE, and use C and Makefile from the command line, or use some other IDE, like Eclipse.

For the hardcore, learn about gdb, and try to make use of "debugWIRE" with an "Arduino UNO" or with an "Arduino nano" board.  Many AVR microcontrollers can use debugWire as a hardware debugger, programmer, step-by -step execution, memory and peripheral inspection, breakpoints, etc.

Did that setup once for an ATtiny13 microcontroller, and a generic CH340G USB-to-serial adapter as a programmer/hardware debugger:  https://www.eevblog.com/forum/microcontrollers/avr-(arduino)-linux-debug/ (which reminds me I still didn't post the clean instructions of how to set all that)

[tggzzz]

You are missing others: the toolset is so complex even the designers of the toolset don't understand the tool they are creating.

No, I don't think I am. There is no tool that covers every possible situation or circumstance. Horses for courses, and source-level debugging is not a tool for fixing toolset complexity issues. All you're concerned with is: does the code do what I think it should, and single stepping the source around the probably area is a good first step. That will most probably show you the code error (whatever the language is), but if not it may well point to a deeper problem. That's when you start furkling with toolchains and stuff.

And despite weak spots in language specification, toolchain creators will have to implement defined behaviour. Only problem may be different choices which affects portability between compilers but this is already a problem due to using different pragmas, compiler specific defines and attributes. So as usual a weak spot in a specification is solved by what is industry standard behaviour.

The defined behaviour has a tendency to change over time, as the compilers become ever more "capable".

Undefined behaviour also changes over time and exposes more problems, as the compilers become ever more "capable".

I don't think that is 100% true. For example: there is some abiquity where it comes to how the << and >> work in C. The way compiler makers have implemented these is a way which works as expected. Nobody with a sane mind is going to change this behaviour because it would break a huge amount of code.

Most problems with incompatibilities I've encountered are with header files, pre-defined values and things which used to threw a warning becoming an error.

Breaking working code/applications is unpleasant wherever it occurs. It is a traditional problem with C and C++. Other languages cause zero or far fewer problems in that way.

Previous/historic code presumed the language definition (and language ambiguity) in force at that time.  Not unreasonable.

Compilers have, over the years, become more aggressive at interpreting the language specification to the compiler's advantage, in the name of gaining minor performance enhancements. That breaks some historic code which used to work, sometimes to the point of omitting to generate the object code! [1]

The traditional response is to blame the developer, implicitly for not taking account of what might happen in the future. Personally I think that is disreputable; the tools are the problem.

[1] That has been a problem with some well-known benchmarks; later versions of the compiler have run the benchmarks "infinitely" fast. Some compilers long before VolksWagen and Toyota - detected they are compiling the benchmark, and emitted unjustifiably optimised code .

[westfw]

executing with the debugger running still messes with things

It is important to learn about the Heisenberg Uncertainty Principle as it applies to software debugging.  (well, WE called some problems "Heisenbugs"!)
That is, you need to understand in what what ways your debugging process (whatever it is) affects the behavior of your code.


Also, different types of debugging imply that different tools are useful.
Single-stepping through source code mat be great for debugging your C program flow, especially if you've made some stupid mistake that you're not seeing in the source (I mean, mistakes that you SHOULD be able to see in the source, but...)
But of course it's pretty useless for debugging timing-related bugs in your bitbang code.
And if you think you're chasing a compiler bug, you'll need other tools.

[glenenglish]

Good suggestion RoGeorge.

could- stay with simple platform - and learn a bit of the background - make, gdb

I think learning the background stuff is important, because once you are in the land of pre-canned GUI environments, like Eclipse, VS code, if something doesnt work in the build sequence, if you dont understand the basics and what is going on in the background, you'll find it time consuming to solve.