Is ST Cube IDE a piece of buggy crap?

[dietert1]

One font, does that mean you have a graphics library that fits into 16K? Which one?
For battery supply, i would rather use something like a Nucleo-L432. It has 256K Flash and is available at 11 €. Recently i tested a L431 and without PLL it runs at 1.5 mA on the internal 16 MHz clock and with 1000 Ticks per second and sending UART messages. And the two DAC outputs were on.

Regards, Dieter

[DavidAlfa]

Use optimization for size when debugging, then manually unoptimize the functions by adding the directive:

Code: [Select]

__attribute__((optimize("O0"))
void my_Function(){
   ...
}

Only that function won't be optimized, so repeat with every function you want to debug.

What display are you using?

If monochrome, U8g2 can get pretty small, uses some sort of font compression/packing, also you can create your fonts with its tool bdfconv, adding only the required characters, cutting down a lot of space.

[paulca]

I was using a little SSD1306 screen.  So it's monochrome 64x128.

https://github.com/afiskon/stm32-ssd1306

Seems to fit.  As I said, you'll need to strip it down.  Drop all the tests, enable a small font. 

So I2C Bus + RTC + SSD1306 libs,  HAL autogenerated with CubeIDE, gives a Release flash size of about 14Kb.

That doesn't leave much.  Even enabling another timer for screen redraw is too expensive (with autogenerated HAL)

The MCU RTC being out of whack is kinda obvious.  Not least the datasheets recommend not using it for anything you want to be accurate.  But it also there is no 32.xxx xtal.  So it's driving the clock at 40k which is why it's about 7% fast.

[DavidAlfa]

Yeah I always lifted my eyebrows when seeing the memory used by HAL_RCC_OscConfig.
Anyways those 1.6KB are too much, clearly you're compiling with optimization disabled.

But the issue is clear:
- HAL code includes every possible scenario
- Most HAL functions use pointers to non-constant data (Thus treated as volatile)
- So the compiler only optimizes the code, but can't stripe unused code because it expects any possible scenario.

For example, your clock initialization probably only uses HSE or HSI oscillator and never changes, but all HSE, HSI, LSE, LSI "ifs" will still be there after compiling.
Actually there's no such "bloat". The library is capable of changing the clock source at any time, on every condition, that's all.
If they removed those capabilities, people would complain anyways, "I can't change to low power oscillator! CUBEMX crap blah blah"

Once you delete these sections (Modify for your current setup), HAL_RCC_OscConfig memory usage reduces to almost 1/3:

Code: (HAL_RCC_OscConfig) [Select]

/*----------------------------- HSI Configuration --------------------------*/
/*------------------------------ LSI Configuration -------------------------*/
/*------------------------------ LSE Configuration -------------------------*/

Giving the following results:
No optimizations: 1.25KB -> 588 bytes
Optimized for size: 824 bytes -> 360 bytes

And that probably applies to every other HAL function using pointers.
So if you know you requirements, you can strip a lot of code.

Interestingly, compiling the same tests on emBitz produces no difference, not a single byte! So emBitz is actually removing those unused parts.
Searching the differences, found that emBitz has Linker Time Optimization (-flto flag) enabled by default.
It not only reduced the overall code size by half, it also somehow causes absolutely no difference in code size when removing those code sections.
When disabling LTO, yeah, those code sections made a difference and the final size was very close to CubeIDE's.

Last time I tried LTO on CubeIDE the stm32 just booted right away into HardFault.
I didn't spend more time researching, but might worth reading, specially on that starving 16KB mcu:
https://www.disk91.com/2020/technology/programming/code-optimization-with-stm32-cube-ide/
Here it explains what probably was my issue:

When using LTO optimization, you need to make some change in the startup file.
It seems that’s a bug with weak function in the GCC version used by ST Cube IDE, so it may be fixed later.
The current problem is: when weak interrupt functions are declared in the Core/Startup/startup_stm32xxxxx.S file the irq handler in the Core/Src/stm32lOxx_it.c file are ignored and removed.
The consequence is the device is not booting.

In the comments:

James Wilson on 11 July 2020 at 23:06 said:
After some experimentation, I found that argument order matters with the linker.
The generated Makefile orders the assembler objects after the C sources (OBJECTS=main.o startup_xxx.o).
By putting the objects with weak symbols first (OBJECTS=startup_xxx.o main.o), the linker correctly processes the weak symbols with LTO, and modification to startup_xxx.s is unnecessary.

Paulon 12 July 2020 at 11:11 said:
I’ve seen that also but I did not found how to change this in a convenient way in a CubeMx / Eclipse generated project.
If you have a tips it is welcome to share it.

About the display lcd library:
The problem with those simple libraries is the memory usage by the font, if you only need the letters 'a' and 'z', the whole range a-z needs to be there, and you're stuck with the default fonts unless you try with different programs and formats until finally find the correct one if you get lucky, spending a lot of precious time in the way.
Won't say it again: U8g2 greatly enhances the font memory usage and comes with its own font generator!

UGUI is also a nice library, mainy for touch screens and windows-like interfaces, although it supports the basics too.
But had the same issue, every font had chars 0-255, eating a lot of space, so I modified it with a matching font generator program which allowed any char, any range.
Although mainly intended for color screens, works fine with monochrome screen too, and the library initialization is extremely easy.
You can check it here, some examples here

[paulca]

I sort of agree, however thinking it through I ran into a weird realisation from history.

Look back to the 8bit personal computer days.  240x320 display res, black and white or 4 bit color.  Very similar to the displays we are writing for.  8bit CPUs with mangled address buses and barely 32K of addressable RAM.  Or 1k-16K RAM with the likes of the ZX81.  Very similar levels of hardware to minimal embedded stuff today.

Well, look at what they did.  They pushed everything they could into that font bitmap lookup table.  GUI elements, graphical niceties, everything.  The question is why?  What is different?  What were they using that we aren't that would explain why?

I'm thinking that most of those platforms had some form of hardware supported display memory and they had at least 16Kb of fast addressable ROM, and plenty of (wasted) power making heat.

How do we replicate that?  Modern low power EPROM for bulky data that's easily hardware indexed and maybe even support it with a separate MCU handling just the text->pixel translation with DMA  from a "VGA style character based video memory".

I mean if you look at old 1980s "GUIs" and compare them with modern Unix "Slang" or similar console dialog UIs, like the kernel "make menuconfig" interface.... they look similar because they are done the same way.  ASCII art and bitmap lookup gfx fonts.

EDIT:  Thinking more.  In the days prior to 8 bit personal computers, computers were entirely customised to "handing off to a teletype terminal" by ASCII, for display and input.  So that's 'where' it comes from.  Do we just reproduce that in hardware with an MCU processing as a "VGA text buffer" style addressable display?

[peter-h]

I may have found a workaround for Cube grabbing focus during a build:

Run it on a second monitor; different to the one you are working on.

The library is capable of changing the clock source at any time, on every condition, that's all.

That is very much the ST way of doing stuff. For each Init function there is a DeInit. First time in 40+ years of embedded I have ever seen stuff like that. Who ever wants to de-init something? Fortunately GCC seems to strip out unused code...

But it can bite you badly; for example in the USB code they have a USB Init and a USB DeInit. If you are diligent you will spot a malloc() in Init and you will think the free() in DeInit will never get called so this is not a problem (malloc is safe if the memory is never freed - a good way to do product options for example). Well... windoze's USB goes to sleep every so often, say once an hour, and then DeInit gets called, so after some days you get a fragmented heap and the system crashes. The fix was to use a static buffer (it is very small; why use the heap??) and then DeInit does nothing with that buffer (obviously).

[DavidAlfa]

Don't forget today's MCUs are orders of magnitude more versatile and capable than in 1980, so the functions get more complex naturally.

For example, you might have a power failure detection, using some big caps to give enough time to save data safely.
You might want to switch to the internal very low power oscillator and turn off everything except the very essentials.
So the system clock function must support that.

If you work at very low level and you absolutely know what you need, then you can strip out every bit.
But that requires a lot of time, just initializing the all clock/pll bits manually takes a whole afternoon with the datasheet.
And you for sure, **you will** induce a lot of bugs, some easy, some sneaky and hard to find and solve.
The final result will be very much the same unless you run out of space or speed, and often you'll overcome the issue by spending time on optimizing *only** that part.
In any case, spend few cents (Talking before 2020 lol) and you get a better MCU with more speed and storage.
That's the modern way of programming. You're no longer paying $$$ for extra 64KB ram or +10MHz. (Correction: Yeah, going that route again!)

Anyways. It seems the LTO bug in CubeIDE was fixed?
I tried with the STM32 soldering firmware, booted right away, didn't modify the startup file.
Only had some "undefined function blah" errors, fixed by adding "__attribute__((used))" to them.

Some tests:

Optimization        Normal           LTO
  O2                    87.19KB       84.71KB
  Ofast                93.14KB       103.78KB
  Os                    79.74KB       73.02KB


For the fragmented heap issue, try what I did for the soldering firmware, run this at the boot, at first line of main, so nothing else allocated memory yet.

Code: [Select]

// Allocate max possible ram, then release it. This fill the heap pool and avoids internal fragmentation due (ST's?) poor malloc implementation.
#define START_ALLOC 256*1024 // This is an aproximate initial value. Will gradually reduce the value until eventually malloc returns a valid pointer.
void malloc_fragmentation_fix(void){
  uint32_t *ptr = NULL;
  uint32_t try= START_ALLOC;
  while(!ptr && try){
    ptr = malloc(try);
    try-=16;
  }
  free(ptr);
}

Explaning the issue I had:
[uninitialized pool]->allocate  1, 2, 7KB
[1KB][2KB][7KB]
free those blocks
[1KB free][2KB free][7KB free]
allocate 10KB
[1KB free][2KB free][7KB free][10KB]
free 10KB
[1KB free][2KB free][7KB free][10KB free]
allocate 12KB
[1KB free][2KB free][7KB free][10KB free][12KB]
free 12KB
[1KB free][2KB free][7KB free][10KB free][12KB free]
...

After running that code at boot, malloc sees a giant pool of ex. 100KB, and has no problem stacking and releasing blocks, the said behavior no longer happens.

[100KB free]->allocate  1, 2, 7KB
[1KB][2KB][7KB][90KB free]
free those blocks
[100KB free]->allocate  10KB
[10KB][90KB free]
free 10KB
[100KB free]->allocate  12KB
[12KB][88KB free]

Of course if you only release the 2KB block, you'll be left with that part fragmented, but still will be used by any other smaller allocation.

[paulca]

Keeping the heap defragmented in malloc and derivatives in non-trival.  It will cost you more memory that it solves in many cases I imagine.

To do so requires you create a linked list of blocks and correctly manage the bi-directional linkage and indexes.  Usually by blocks we would mean paging.  (The actual linux implementation even includes the option to move pages of memory around, the advantage of paging and virtual addressing and consequative memory becomes an illusion anyway. except maybe in DMA buffers)

I'm going to guess that whomever's implementation of malloc it is, it's going to be optimised a lot and for very small SRAM sizes it probably excludes paging and defragmentation all together.  I'd agree with that, if memory fragmentation is enough to worry about, you probably don't have enough RAM to have a generic malloc implementation that would help.

It's one of my constant battles in C/embedded is desperately trying to avoid ANY and ALL dynamic allocation.  It's things like random runtime strings, damn char*s everywhere, they get tedious to store and I'm not calling malloc for them!  I hate the Arduino JSON library for example because it's got dynamic shit all over it.  "I just want to parse a JSON string!  I don't want your demo on smart arsed C++ memory allocation and I don't need you to implement the ENTIRE damn JSON spec and use Kb of RAM to do it!" /rant.

[paulca]

I'm actually tempted to use the little STMF030F4 as an ARM assembler tutorial vehicle.  The STM32 debugger is pretty awesome in the way you can open and mess with registers live.  The only other place you get that really (outside of arm hardware debug) is with a chip emulator/simulator.

I doubt there is anything will be learning there that will require more than 16k flash or 4k ram.

I may well start by "bare metalling" it by bringing together MY personal setup and stripping back their "setup*.s" file and removing their libs.

I might keep all of the their constants/defines though, they are nice and handy.

[peter-h]

For example, you might have a power failure detection, using some big caps to give enough time to save data safely.
You might want to switch to the internal very low power oscillator and turn off everything except the very essentials.
So the system clock function must support that.

True and I have done that here
https://www.eevblog.com/forum/microcontrollers/ideas-for-storing-frequently-updated-data-in-a-flash-chip/msg3913820/#msg3913820
https://www.eevblog.com/forum/microcontrollers/how-fast-does-st-32f417-enter-standby-mode/msg4063231/#msg4063231

but I decided it is much better to explicitly stop the stuff which draws a lot of power, than hope the ST DeInit function does it. In fact the biggest single draw was by the LAN8742 ETH PHY chip and that could be stopped only by sending it a special command via the "64 bit USART" which the 32F4 uses to communicate with it. I guess switching to a 32k oscillator is another thing to do but then the CPU runs much slower.

Opinions vary of course but a lot of the time one can get great "low power" result by running at normal speed and shut down most of the time. It is usually very easy to do (even if you put a PMOS FET in VCC ) and you can get as "low power" as you like.

[paulca]

I'm sure not that many care, but the issues I was having with network drives got worse and worse until I could no longer accept things were working as they should be.

I spent 3 hours this evening, most of my free time tracking it down to a Windows bug applied somewhere in the last year.  Not a lot of info on it, I can see it being reported and queried on the MS support forums, but the answer is basically, "Oh, that, don't want to know, that will cost you money to investigate.  Sorry.  Bye"

The bug is, the Windows SMB client overloads the server by duplicating every single file access block request with a spammed GUEST access request on top of it.

Literally I have megabytes of error log files now with this in them millions of times:

[2022/10/18 20:03:15.804498,  1] ../../source3/smbd/service.c:353(create_connection_session_info)
  create_connection_session_info: guest user (from session setup) not permitted to access this share (paul)
[2022/10/18 20:03:15.804532,  1] ../../source3/smbd/service.c:543(make_connection_snum)
  create_connection_session_info failed: NT_STATUS_ACCESS_DENIED

Suspiciously and conformationally ... adding

guest user = ok

to my share security config and .... the network drives work fine again.

As a lowly Windows 10 Home user, with about 6 valid MS licenses, I don't think it's worth trying to raise it as a bug.  It's windy tomorrow I'll go piss into the wind instead.

Guest access isn't that bad.  I already kicked the spyware devices onto the guest network to stop work laptops hacking my file shares and politely giving me access to my porn folder on the work laptop!  (True story!)

I even recall adding guest access way, way back to test authentication issues and left it enabled.  I had only just disabled it a week ago, when all these issues started to occur.

[DavidAlfa]

Try NFS, works much better, and W10/11 already includes support for it.

[paulca]

Try NFS, works much better, and W10/11 already includes support for it.

Interesting.  NFS.  Does Windows do the UID mapping or does it require forced mappings on the server side?

It's just that traditionally NFS is a raw block network filesystem and security and access limitation is performed by the client.  Which is why it terrifies network admins.

EDIT: I checked.  Windows mounts the exports with -o anon.  It then sends UID:-2 and GID:-2. 

However, as a single user, I can hack the registry entry and have it send my actual Unix ID instead.  I think. 

I'll see if it works.

[DavidAlfa]

I did the same to overcome permission issues.
I had a small embedded board as home NAS (Seagate Dockstar), speeds doubled from.25-25 to 45-50MB/s.
The limit was always on the CPU, used a very old armv5 architecture, doubling the RAM and overclocking to 1.5GHz did barely anything.
Sadly It passed away recently, after almost 10 years non stop!

[riscy00]

I enjoyed reading this, it should be put together and published under title "Is ST Cube a Piece of Buggy Crap", I would buy it and read it lol. 

[riscy00]

As a starter with STM32 5 months ago, has anyone tried maximum optimization on HAL or LL library, last time I did that (as starter) and it seems to churns out all sorts of error/warning from the complier/linker, so I had to limits optimization to Os.
I have never seen this before compared with Code Composer Studio with C28x and TM4C project which I set to 02, encouraged by TI compiler text.
 

[peter-h]

I think optimisations beyond -O1 are likely to break some ST HAL code, because a 168MHz CPU can run code fast enough to break various things like min CS=1 time on various SPI chips.

I build everything with -Og. Possibly switch to -O0 (no optimisation) if, when stepping through code, I see "optimised out" in the debugger and I really want to see that value. Beyond -Og there is negligible improvement in code size.

[DavidAlfa]

Nah, only if your coding is shame and you don't know how to prevent critical delays from being optimized

[voltsandjolts]

As a starter with STM32 5 months ago, has anyone tried maximum optimization on HAL or LL library

The best optimisation is not to use HAL or LL

[SiliconWizard]

As a starter with STM32 5 months ago, has anyone tried maximum optimization on HAL or LL library, last time I did that (as starter) and it seems to churns out all sorts of error/warning from the complier/linker, so I had to limits optimization to Os.

Yes, I have used up to -O3 without any issue. (Never used auto-generated code though, but the HAL and LL directly in my own code.)

Now let us know what those errors or warnings were. If linking errors for some opt. level, I'd suspect it was a matter of not having enough code memory available on said MCU, so obviously, the higher the opt. level, usually the larger the code size. If you are limited in code size, don't use -O3 or even -O2. I do when I have ample code memory available, but otherwise -Os appears to be a nice compromise.

[peter-h]

you don't know how to prevent critical delays from being optimized

My own code is just fine; thank you.

I was talking about stuff within ST supplied code.

[DavidAlfa]

I was messing with you 
No idea, never had any issue though, would be strange, are you sure they're not using some directive or volatile tag to prevent optimizations?

[paulca]

I remember we had issues when they asked us to compile out stock exchange gateway with -o3.

Compile went mental about all the obfuscated types.  Lot of decoupling via void* and char* pointers.  GCC went, "I'm not optimizing that!".  Pointer frame alignment issues.  Unable to optimize on word/address access if a struct has been cast to a char*.

In the end we disabled frame alignment optimisations and spent the next week trying to find the memory leaks which made it seg fault on shutdown.

[peter-h]

There is no point is playing with optimisation levels IF the product is working correctly.

All you are doing is inviting trouble.

You have to do a potentially huge amount of "regression testing". On a product of any complexity, this is impossible.

[Kjelt]

You have to do a potentially huge amount of "regression testing". On a product of any complexity, this is impossible.

Not impossible if you automate it and run at nights/weekends.
This is even necessary for each new fw release imo, because each code change could break functionality. Sunny day scenarios and 30% rainy day scenarios testing is standard practice in the last three companies I worked for. Tests are expanded for the new added functionality but also for any bug found in the field (which triggers an internal process to find the cause of the bug and how to prevent this in the future). In combination with (as far as possible) continuous integration, continuous delivery it is essential that each release is tested.