UART weird stuff, and UART DMA

[wek]

ST's peripherals generally appear having been designed by folks - probably young engineers - with little practical experience.

Would you say that about their USB or ETH peripherals too? They are damned complicated.

Those are not designed by ST but purchased from Synopsys - it's not a secret, it's written in the RM too.

And they come from an entirely different world, and in that world, users simply don't deal with peripherals directly, so their complexity does not matter.

JW

[SiliconWizard]

Yep, note that this is ultra common. MCU vendors buy tons of IPs and rarely design their own peripherals. They do only for those odd ones that would give them an edge competition-wise, and that's certainly not going to be gained with stuff as basic as UARTs and SPI, or even Ethernet.

[peter-h]

OK, let me get this right.

You config DMA to copy from a UART to memory address x..., max y bytes.

When the transfer ends (say NDTR=0 or LISR=1), you disable DMA, then enable it again, and it writes the same UART's data again at address x..., max y bytes.

Reading the RM, it seems to confirm the above i.e. the address registers are not incremented by the transfer

[NorthGuy]

OK, let me get this right.

You config DMA to copy from a UART to memory address x..., max y bytes.

When the transfer ends (say NDTR=0 or LISR=1), you disable DMA, then enable it again, and it writes the same UART's data again at address x..., max y bytes.

Reading the RM, it seems to confirm the above i.e. the address registers are not incremented by the transfer

Yes. When you stop in the middle (say at the address X), if you set the DMA address to X before resuming, then DMA continues writing the buffer where it left off. However, when it restarts after the end of the transfer, it'll start from X again. Obviously, you do not want that - you want to adjust the address back to the beginning of the buffer. Therefore, you cannot use circular mode. So, you would have to re-start manually and adjust the address before re-starting.

Or you can see if you can re-program priorities. But to figure out if this fixes wek's phenomenon, you must be able to create a situation where the phenomenon occurs. This is not easy to do, if at all possible.

[peter-h]

Can I get this right.

If the transfer is stopped before NDTR=0, the DMA reloads the address registers from the values you previously wrote in.

If the transfer comes to its natural end (NDTR=0), the DMA doesn't reload the address registers and the transfer will run to/from the address after the last one written/read?

[westfw]

ST's peripherals generally appear having been designed by folks - probably young engineers - with little practical experience.

My impression is that most of the simple ST peripherals are essentially "reference design" VHDL (or whatever) "8250 compatible UART" and similar.  Maybe that's OK; lots of consistency.  But not really matching the features (or quality?) of many more original implementations.

[wek]

Or you can see if you can re-program priorities. But to figure out if this fixes wek's phenomenon, you must be able to create a situation where the phenomenon occurs. This is not easy to do, if at all possible.

It's surprisingly simple.

Attached program for the classic Disco'F4 ('F407), a jumper loops back UART Tx to UART Rx. Clocks are at default, no need to go fancy here. DMA set up in the usual way for UART Rx, to store bytewise from data register to memory buffer rx_buf and increment memory-side pointer register, noncircular, FIFO off.

I transmit a byte (from an incremented variable) manually to UART Tx. Then in a loop I watch NDTR, and when it changes, I then read out the content of rx_buf at the index where FIFO is supposed to store it, and store it into the same index in rx_buf2. Then I blink a LED using a trivial loopdelay (the blinking is confusingly through timer used in fact as simple pin-switcher, this is leftover from some other test), and repeat 16x (I did not bother to do any serious ending, so NDTR ends at 0 and the whole program gets stuck at waiting for it to change).

The result is

Code: [Select]

(gdb) p /x rx_buf
$2 = {0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb, 0xc, 0xd, 0xe,
  0xf, 0x10}
(gdb) p /x rx_buf2
$3 = {0x1, 0x2, 0x0, 0x4, 0x5, 0x0, 0x7, 0x8, 0x0, 0xa, 0xb, 0x0, 0xd, 0xe,
  0x0, 0x10}zeros in rx_buf2 are the premature reads from rx_buf (which being global variable was zeroed by the startup code).

The original blinky test had this thing that if you press blue button, timing changes, blinking slows down. So I press down the blue button, reset, et voila:

Code: [Select]

(gdb) p /x rx_buf2
$5 = {0x1, 0x2, 0x3, 0x0, 0x0, 0x6, 0x7, 0x8, 0x0, 0x0, 0xb, 0xc, 0xd, 0x0,
  0x0, 0x10}

There are no other DMA streams involved, so I don't see how priority would change anything here.

The key portion is this:

Code: [Select]

  while(1) {
    n++;
 80002c4: 6823      ldr r3, [r4, #0]
 80002c6: 3301      adds r3, #1
 80002c8: 6023      str r3, [r4, #0]
    USART2->DR = n;
 80002ca: 6823      ldr r3, [r4, #0]
 80002cc: f8cc 3004 str.w r3, [ip, #4]
    while(old_ndtr == DMA1_Stream5->NDTR);
 80002d0: 684b      ldr r3, [r1, #4]
 80002d2: 4293      cmp r3, r2
 80002d4: d0fc      beq.n 80002d0 <main+0xfc>
    rx_buf2[buf_idx] = rx_buf[buf_idx];
 80002d6: f818 2000 ldrb.w r2, [r8, r0]
 80002da: 4b16      ldr r3, [pc, #88] ; (8000334 <main+0x160>)
 80002dc: b2d2      uxtb r2, r2
 80002de: f80a 2000 strb.w r2, [sl, r0]
    buf_idx++;
 80002e2: 3001      adds r0, #1
    old_ndtr = DMA1_Stream5->NDTR;
 80002e4: f8d9 2004 ldr.w r2, [r9, #4]


JW

[wek]

Can I get this right.

If the transfer is stopped before NDTR=0, the DMA reloads the address registers from the values you previously wrote in.

If the transfer comes to its natural end (NDTR=0), the DMA doesn't reload the address registers and the transfer will run to/from the address after the last one written/read?

No,  NDTR and the pointer registers simply reload in both cases.

See attached code, a regurgitation of the previous one. It sets NDTR to 8 and first runs down until NDTR == 5, then aborts/restarts DMA without touching NDTR and the pointer registers, runs down until "natural completion" (i.e. xSR.TCx == 1), and then again restarts without touching NDTR. The counter which gets transferred through USART keeps counting up all the time.

"Samples" of rx_buf are taken manually, at random spacing, but I believe the gist of the thing is clear:

Code: [Select]

(gdb) p /x rx_buf
$84 = {0x1, 0x2, 0x0 <repeats 14 times>}
(gdb)
$85 = {0x1, 0x2, 0x3, 0x0 <repeats 13 times>}
(gdb)
$86 = {0x1, 0x2, 0x3, 0x0 <repeats 13 times>}
(gdb)
$87 = {0x4, 0x2, 0x3, 0x0 <repeats 13 times>}
(gdb)
$88 = {0x4, 0x5, 0x3, 0x0 <repeats 13 times>}
(gdb)
$89 = {0x4, 0x5, 0x3, 0x0 <repeats 13 times>}
(gdb)
$90 = {0x4, 0x5, 0x6, 0x0 <repeats 13 times>}
(gdb)
$91 = {0x4, 0x5, 0x6, 0x7, 0x0 <repeats 12 times>}
(gdb)
$92 = {0x4, 0x5, 0x6, 0x7, 0x8, 0x0 <repeats 11 times>}
(gdb)
$93 = {0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
  0x0, 0x0}
(gdb)
$94 = {0xc, 0x5, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
  0x0, 0x0}
(gdb)
$95 = {0xc, 0xd, 0x6, 0x7, 0x8, 0x9, 0xa, 0xb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
  0x0, 0x0}
(gdb)
$96 = {0xc, 0xd, 0xe, 0x7, 0x8, 0x9, 0xa, 0xb, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0,
  0x0, 0x0}
(gdb)
$97 = {0xc, 0xd, 0xe, 0xf, 0x10, 0x11, 0x12, 0x13, 0x0, 0x0, 0x0, 0x0, 0x0,
  0x0, 0x0, 0x0}
(gdb)
JW

[paulca]

It's not cluttering.  It's simplifying. The "more complete" way is to use braces.  Therefore having only one way in your style simplifies things.  No?

The time you  spend thinking about the style is the time when you don't think about other things. These other things are more important - data structures, algorithms, code flow. You should be thinking about these. It is entirely possible to write perfectly styled code which is full of bugs, or bloated beyond comprehension,  or written very badly with lots of hard-to-test special cases. The style is the least of concerns.

I agree 100% until you have more than you and the compiler as audience.  But if you put that line across in a job interview for software engineer, you will not get the job.  You just won't be fit to work in a team.

Coding/hacking/bashing out some MCU code is not the same as software engineering.  The biggest point is...  BIG code, BIG data, BIG platforms, many people, many teams.  Way more than half the effort is in communications, not in making it work.

Like a plumber could come into your house and throw some pipes in willy nilly and it will work perfectly.  It's not a professional job.  The professional will do it neat, tidy, to any standards that apply AND he will make it easy to access, to fix, to understand by others later etc. etc.  Partly about pride too.  Leaving a job knowing, to the best of your ability, the next guy along to work on it will think, "Nice professional job"

I'm sure if you stated the above line in a job interview for an electrical/electronics engineer you would also be shown the door.  I see convention and style picked at CONSTANTLY on here in terms of PCBs and Schematics.  I'm sure the suggestion that none of that is important and all that matter is, does the circuit work... will not go down well.  Again, if it's just you, nobody cares except you.  If you are one of a team of 6 of a program of 4 teams.... people will care.

At interview.  If you can't do the basics of making it work, you are worthless.  If you believe that all that is important is making it work, you are a potential liability or at least missing part of the pie.

[peter-h]

Interesting discussion!

It means that some code in a typical DMA application can be loaded just once:

Code: [Select]

DMA1_Stream5->NDTR = num_values; // buffer size
DMA1_Stream5->PAR = (uint32_t) (&DAC1->DHR12RD); // destination (DAC) address, 32 bit DAC mode
DMA1_Stream5->M0AR = (uint32_t) buffer0; // base of DMA table 0
DMA1_Stream5->M1AR = (uint32_t) buffer1; // base of DMA table 1

This can be quite significant since the arm32 peripherals are very slow - not only because they are running at a fraction of the CPU clock but also because there are multiple clocks involved in syncing register reads/writes.

And probably lots of other registers too.

Old 1980s DMA controllers had this shadow register setup, but they were dedicated chips.

[Siwastaja]

Yes, the whole idea is that you can configure the DMA once. In circular mode, you just enable it once, too; in non-circular mode, you enable it for every transfer.

The requirement of clearing the interrupt flags before enabling incurs one extra register write besides the EN bit, but it's not too bad; two writes in total.

I would say, if you need to rely on exact data count during on-going transfer, you are just operating beyond what the DMA was designed for. Reconsider the design. If you absolutely must do this, could you design it so that you allow one extra transfer when evaluating the NDTR? AFAIK, the DMA does not buffer more than one request to memory, so the problem as discussed here is limited to off-by-one. I would rather just delay the processing by one data item, instead of trying to work around the (uncertain) delay.

[wek]

Code: [Select]

This can be quite significant since the arm32 peripherals are very slow - not only because they are running at a fraction of the CPU clock but also because there are multiple clocks involved in syncing register reads/writes.This is most probably not case of DMA registers, as they sit directly on an AHB bus, and are not behind an AHB/APB bridge which is the primary source of such slowdown.

JW

[peter-h]

I am glad to hear that

But are the DMAs running at say 168MHz?

[NorthGuy]

I would say, if you need to rely on exact data count during on-going transfer, you are just operating beyond what the DMA was designed for.

How else would you use the circular mode?

[Siwastaja]

I would say, if you need to rely on exact data count during on-going transfer, you are just operating beyond what the DMA was designed for.

How else would you use the circular mode?

Circular mode can be used to automatically update memory, if atomic units are at most 4 bytes and aligned, so that DMA updates each variable in one piece. If there is interdependency between variables in a packet (so they need to be from the same "sample"), this does not work, but this often is not the case. I have used this pattern multiple times due to extreme simplicity, for example, to transfer current/voltage setpoints and measured values between MCUs. Then NDTR is completely meaningless.

Another way to use circular mode without looking at exact value of NDTR is to implement double buffering by dividing the memory in two and use half transfer and transfer complete interrupts. Which of the two interrupt flags is active tells which of the buffers is now accessible by CPU, while the DMA is accessing the other.

[peter-h]

What do you load into NDTR if you want continuous circular loading?

Doesn't a 32 bit CPU store also write the 32 bits atomically (possibly if assuming 4-alignment)?

[Siwastaja]

What do you load into NDTR if you want continuous circular loading?

Size of the datagram (memory region you are accessing) (in word size used by DMA, so you would usually divide by 4).

Doesn't a 32 bit CPU store also write the 32 bits atomically (possibly if assuming 4-alignment)?

Yes, if your variables are correctly aligned, both DMA and CPU will access them atomically. For example,

volatile struct __attribute__((align(4)))
{
   uint32_t time;
   uint16_t volts;
   int16_t amps;
   uint8_t status;
   uint8_t padding[3]; // to be explicit
}

All of them are updated by DMA atomically, and can be accessed by CPU without a problem, but status could be from a different frame than volts, for example. If this does not matter to you, then things are pretty simple: just configure continuous DMA and enjoy the automagic memory sharing with no CPU intervention nor any parsers.

[peter-h]

Yes, if your variables are correctly aligned, both DMA and CPU will access them atomically

If they are not 4-aligned, and taking the 32F4 which supports unaligned access, would a 32 bit store really not be atomic from

- an interrupt
- a DMA

I can see the latter may not be atomic (DMA has been like that since for ever; even in the Z80 DMA days, the DMA could cycle steal within instructions, IIRC) but the former??

[NorthGuy]

Yes, if your variables are correctly aligned, both DMA and CPU will access them atomically. For example,

volatile struct __attribute__((align(4)))
{
   uint32_t time;
   uint16_t volts;
   int16_t amps;
   uint8_t status;
   uint8_t padding[3]; // to be explicit
}

But what if you start when the peer is already transmitting amps, but you will record it as the first member - time, or if you encounter any glitch on the UART line - everything will be shifted off permanently.

Another way to use circular mode without looking at exact value of NDTR is to implement double buffering by dividing the memory in two and use half transfer and transfer complete interrupts. Which of the two interrupt flags is active tells which of the buffers is now accessible by CPU, while the DMA is accessing the other.

Sure, but this works more like a ping-pong buffer - you will only get the results when the entire buffer (or half of it) is filled in. For example, when you receive a single character, you'll never know untill the rest is received, possibly days after it was sent.

To replace a traditional circular buffer you need to access NDTR.

The fact that you can access NDTR already means that the original designers of the module thought that such access might have been useful. Simply put, the use of AXI streams broke the functionality and they didn't figure out that it needs to be fixed.

[Siwastaja]

But what if you start when the peer is already transmitting amps, but you will record it as the first member - time, or if you encounter any glitch on the UART line - everything will be shifted off permanently.

Oh! Sorry; because the UART dropped off the discussion long time ago, my mind shifted on SPI, which has hardware packet delimitation signal.

[paulca]

To replace a traditional circular buffer you need to access NDTR.

Did you have a crack with the multi-buffering DMA?  Maybe even a scrap book project with MX and HAL just to test it.

I have no used it in anger, but when it came to double buffers being nearly what I want, just not quite flexible enough it did look like the better option.

I still honestly don't quite track what you are doing, I know there have been several threads around UART and your struggling, but I've lost the thread of the project and it's purpose.  Normally when I find myself wandering around in a project in random directions trying to whack a mole down, it's time to stop, sigh, take a walk and try and think about it in a completely different way.  If I end up in the same place, fine.  However sometimes I end up in a much better place.  Example being the DSP buffering system I posted about, which by the end of the thread got completely deleted and a completely different approach taken.

I also realise you have gone over it dozens of times and are sick of doing so.

[NorthGuy]

To replace a traditional circular buffer you need to access NDTR.

I know there have been several threads around UART and your struggling ...

You confuse me for someone else.

[paulca]

To replace a traditional circular buffer you need to access NDTR.

I know there have been several threads around UART and your struggling ...

You confuse me for someone else.

Oops.  Sorry I meant that for peter-h, not you.

[peter-h]

My UART issue was fixed by wek's suggestion that interrupt driven TX does not need a byte placed into the UART (and then enabling TX int). That is for many past UARTs. For the 32F4 ones you just enable TX int and off it goes.

Spent weeks on that. I inherited that code. Funnily enough it lost only 1 byte every few million.

Had I used DMA to service the UARTs I would have never had this issue.

[Siwastaja]

Inherited code is notoriously painful to deal with, so I don't blame peter-h for that. It is often easier to write from scratch.

For microcontroller projects, code that is easiest to reuse is pure algorithmic stuff.