The Imperium programming language - IPL

[Sherlock Holmes]

I've been pondering the idea of attempting to create a new (compiled) programming language specifically designed for hardware programming, microcontrollers. The impetus for this is primarily a frustration and disappointment with the popular languages that are out there like C++, Rust and C.

This is not as crazy as it might at first appear, I'm a very experienced software developer and very experienced with C as well as other languages on mainframes, minicomputers and Windows. I have also implemented a full working compiler before for Windows NT including code generation, optimization and COFF object file creation, I'm also trained in electronics and telecommunications but never worked professionally in that field, entering the software world shortly after I left college.

Recently I resumed playing around with electronics as a hobbyist and have been working on STM23F4 boards to get my hands dirty so to speak, and although I can get stuff done in C, it is nagging at me, just how much better a language could be, should be, for this kind of work.

So I could start by asking what people's views are here, what do you - as an MCU developer - find good or bad about the prominent languages, what was a language you used in the past that had features you'd like to see today when working with MCUs?

This is informal, although I do have ideas of my own and have a draft list of goals for such a thing, I am interested in what others have to say about this subject, I'm also drawn to leveraging LLVM but know very little about it, it's generic intermediate IR is clearly helpful here as that reduces the back end effort substantially.

[ataradov]

C is fine for embedded. So far all attempts to make something better ended up in bloat and feature creep.

A slightly cleaned up version of C would be nice, but language inventors never stop at that because it is "boring".

If you absolutely can't make C work for you, it may better to figure out what you think is wrong and how others are dealing with that.

[coppice]

C is, and has always been, a hardware level oriented language. That's why it has always excelled in applications that need to be near the metal, like MCUs and operating systems. What would your new language bring to the table.

[mcovington]

I agree - C is hardware-oriented already, which is why it lends itself to efficient use of CPUs.

What was PL/M like, and should it come back?  I've never used it.

Arduino language is actually rather nice, I think; it's C with minor extensions.  Maybe that's the direction to go in.  Full C++ definitely isn't.

[Sherlock Holmes]

C is, and has always been, a hardware level oriented language. That's why it has always excelled in applications that need to be near the metal, like MCUs and operating systems. What would your new language bring to the table.

There are several things that a new language would bring, here's a summary of the more salient:

• No reserved words, thus enabling new keywords to be added over time.
• Support 'bit' as a native data type.
• Support 'strings' as a native type, BCD/decimal as well.
• Support for namespaces.
• Computed gotos.
• Flexible alignment, packing and padding directives.
• Nested functions.
• Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.
• Support for an async/await model.

These are the kinds of things that I've seen other people raise or complain about sometimes, things that a freshly designed language could readily accomodate.

[coppice]

I agree - C is hardware-oriented already, which is why it lends itself to efficient use of CPUs.

What was PL/M like, and should it come back?  I've never used it.

Arduino language is actually rather nice, I think; it's C with minor extensions.  Maybe that's the direction to go in.  Full C++ definitely isn't.

PL/M was a stripped down PL/1 Intel produced in the late 70s to program their small processors. PL/1 was a monster language, designed to do everything IBM could think of. PL/M extracted out a subset that was functionally comparable C, before C had become popular. Motorola made something similar called MPL. Other people went down a similar road. Its not entirely clear why everyone in the late 70s chose PL/1 as their starting point, but C killed that path in the early 80s.

[ataradov]

Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.

This is plain stupid on modern MCUs and if you think that NOPs are a valid timing technique, then I'm not sure I want to use that language.

Everything else is a nice incremental improvement. If it was there, it could be useful, but I would not say C suffers because of that.

Support for an async/await model.

There is a lot of this in the recent versions of C++. I'm not sure how useful it actually is as a language feature.

Making a language is a lot of effort, and I don't think stuff above justifies it. You can do it for fun, but in that case don't make polls, start doing it and if you have something useful implemented, we can discuss.

Right now this has a vibe of "making your own high performance oscilloscope" threads from 10 years ago. Those went for pages and pages with people daydreaming about it and not actually doing anything.

[coppice]

Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.

That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.

[Sherlock Holmes]

I agree - C is hardware-oriented already, which is why it lends itself to efficient use of CPUs.

What was PL/M like, and should it come back?  I've never used it.

Arduino language is actually rather nice, I think; it's C with minor extensions.  Maybe that's the direction to go in.  Full C++ definitely isn't.

PL/M was a stripped down PL/1 Intel produced in the late 70s to program their small processors. PL/1 was a monster language, designed to do everything IBM could think of. PL/M extracted out a subset that was functionally comparable C, before C had become popular. Motorola made something similar called MPL. Other people went down a similar road. Its not entirely clear why everyone in the late 70s chose PL/1 as their starting point, but C killed that path in the early 80s.

Well PL/I was influential but was heavily sidelined by the growth in popularity of C in universities. Full PL/I is indeed a large language, but it was revised into a Subset G standard (an ANSI standard) that's a smaller language, smaller than many modern languages like ADA (recently adopted by Nvidia for their firmware language of choice).

PL/I was also one of the first languages used to write an operating system, some years before C arose. Contrary to popular belief too, C was created primarily for ease of implementing a compiler, programming hardware was never a goal of the designers any more than several other languages at the time.

[jpanhalt]

I can only write Microchip Assembly and thought that was pretty hardware oriented.

In accord with ataradov's comment, using NOP's to pad a routine to an exact time may be OK, so long as only a few are used.  For longer delays there are other ways, not the least of which is to use a timer, e.g., TMR2, to set a beat.  With such a beat, one can do other things during any appreciable delay, yet keep that time-critical routine exact -- or almost so. 

[Sherlock Holmes]

Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.

That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

These are legitimate needs arising from their problem domain, enabling these to be met by supporting it in a language is surely fully appropriate for a language designed for hardware programming.

[Sherlock Holmes]

I can only write Microchip Assembly and thought that was pretty hardware oriented.

In accord with ataradov's comment, using NOP's to pad a routine to an exact time may be OK, so long as only a few are used.  For longer delays there are other ways, not the least of which is to use a timer, e.g., TMR2, to set a beat.  With such a beat, one can do other things during any appreciable delay, yet keep that time-critical routine exact -- or almost so.

Yes, I was chatting to a very experienced 8 bit PIC developer recently about this, he tries to use C for some parts of the code only to find he has no control over these hardware characteristics any more. The same thing is true of computed goto, C has no support for this yet it is trivial in assembly programming and much faster than using arrays of function pointers.

[coppice]

PL/I was also one of the first languages used to write an operating system, some years before C arose. Contrary to popular belief too, C was created primarily for ease of implementing a compiler, programming hardware was never a goal of the designers any more than several other languages at the time.

PL/1 was quite late to the "writing operating systems in a high level language" game. Which is why so few were written in it. Several languages, often described as real time oriented rather than operating system oriented, were developed for building operating systems before PL/1 existed.

C was intended to be close to the metal. That's really all a systems oriented language is.

[ataradov]

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C.

You are very wrong here. And I think this alone is enough to stop discussion of the language design.

You understand that even on simplest MCUs NOP timing is affected by flash wait states?  This is not taking into account micro architectural optimizations, out of order execution, multiple issue pipelines, etc

[Sherlock Holmes]

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C.

You are very wrong here. And I think this alone is enough to stop discussion of the language design.

You understand that even on simplest MCUs NOP timing is affected by flash wait states?  This is not taking into account micro architectural optimizations, out of order execution, multiple issue pipelines, etc

That's rather odd, since you wrote this in January:

Here is my standard way of generating small blocking delays:

Code: [Select]

__attribute__((noinline, section(".ramfunc")))
void delay_cycles(uint32_t cycles)
{
  cycles /= 4;

  asm volatile (
    "1: sub %[cycles], %[cycles], #1 \n"
    "   nop \n"
    "   bne 1b \n"
    : [cycles] "+l"(cycles)
  );
}

Code is located in SRAM so that execution time is not subject to flash wait states.

This is better than a plain C loop, since it does not depend on the compiler and optimization settings.

Anyway, if you have nothing to contribute then of course, you can stop discussing this subject.

[coppice]

Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.

That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

These are legitimate needs arising from their problem domain, enabling these to be met by supporting it in a language is surely fully appropriate for a language designed for hardware programming.

It is very much true. Something like a PIC or an 8051 can run cycle exact code, and developers have gotten very used to that. Look at more modern small machines and you'll see them gradually drifting away from cycle exact operation. Even chips without an explicit cache to play with the timing often now have longish memory lines, which are demuxed in a non-deterministic way, to mitigate the slow operation of flash. Often people used to cycle exact operation struggle to adapt to this, and produce some very quirky results.

One area where there is some merit in cycle exact code is security. If you implement something like an encryption algorithm, its much less easy to attack it if all the paths through it are of equal length. You can't make them run in equal time every time, but if you can make their averages comparable you have made the code a lot more robust against attacks.

[ataradov]

Code is located in SRAM so that execution time is not subject to flash wait states.

Yes, and I had to account for the flash execution by placing the code in SRAM. And I knew exactly the platform this code would be running on, so I knew what optimizations would be performed and I knew that they would be predictable.

This code would not work at all on Cortex-A5 core, for example.

[Sherlock Holmes]

Code is located in SRAM so that execution time is not subject to flash wait states.

Yes, and I had to account for the flash execution by placing the code in SRAM. And I knew exactly the platform this code would be running on, so I knew what optimizations would be performed and I knew that they would be predictable.

This code would not work at all on Cortex-A5 core, for example.

Of course, one should always account for all factors like this in any design, I never suggested otherwise nor did I suggest it would be appropriate for all target platforms. The facts are the facts, today one must do as you did, embed assembler, all I'm suggesting here is that you'd have another option, one that C itself doesn't afford you today, the list I gave is just a starting point for discussion.

[ataradov]

What is the point of the language feature that won't be even theoretically possible to implement on a lot of modern hardware?

You don't need to embed the assembler, it is just a quick hack. The correct way of making delays is to use the timer. Again, not a compiler feature.

[Sherlock Holmes]

Precision timing features like emit multiple NOP operations or ensure identical execution time for (say) case clauses in a switch.

That one is certainly not going to happen. Very few machines now permit cycle exact execution of anything. Even really small devices have at least some elementary kind of cache, that messes with the instruction timing.

That's simply not true, we can emit multiple platform specific NOP's today by embedding assembler in C. There are assembler MCU developers out there that struggle to use C because of this kind of thing. They have carefully crafted code where the want some execution path to take exactly the same number of clock cycles as some other, so they embed multiple NOPs sometimes, their designs require that.

These are legitimate needs arising from their problem domain, enabling these to be met by supporting it in a language is surely fully appropriate for a language designed for hardware programming.

It is very much true. Something like a PIC or an 8051 can run cycle exact code, and developers have gotten very used to that. Look at more modern small machines and you'll see them gradually drifting away from cycle exact operation. Even chips without an explicit cache to play with the timing often now have longish memory lines, which are demuxed in a non-deterministic way, to mitigate the slow operation of flash. Often people used to cycle exact operation struggle to adapt to this, and produce some very quirky results.

One area where there is some merit in cycle exact code is security. If you implement something like an encryption algorithm, its much less easy to attack it if all the paths through it are of equal length. You can't make them run in equal time every time, but if you can make their averages comparable you have made the code a lot more robust against attacks.

Yes, this subject of equal length paths came up. There was an example:

Code: [Select]


if (A | ^B & C | ^D)
{
    // do stuff
}
else
{
   // do other stuff
}


Something like that anyway. The problem is that the C compiler will use short circuit evaluation when evaluating the boolean expression, but his requirement was that the method (which contained more less just this code) must take exactly the same number of clock cycles for all possible values of A, B, C and D - that is simply impossible in C, the language itself does not offer such a feature.

I speculate that it might be feasible to somehow achieve this by either providing a way to stop short circuit evaluations or guide the compiler in some way, perhaps

Code: [Select]

exact void some_function()
{

if (A | ^B & C | ^D)
{
    // do stuff
}
else
{
   // do other stuff
}

}

This is rather contrived, but you get the idea, now the effort involved might be high, but I've not really analyzed it, it might well be possible to deliver something like that - bearing in mind the realities that ataradov pointed out.

[Sherlock Holmes]

What is the point of the language feature that won't be even theoretically possible to implement on a lot of modern hardware?

You don't need to embed the assembler, it is just a quick hack. The correct way of making delays is to use the timer. Again, not a compiler feature.

The point would be that there are use cases, situations, problems and platforms where it is an extremely helpful solution for the developer. C offers flexible memory allocation and freeing as you know, yet that, right there, is an example of a language feature that is used with great caution, if at all, on many MCU designs.

The correct way to do something is also rather subjective ataradov, as an engineer yourself, you'll appreciate that all designs are compromises.

[ataradov]

The beauty of C is that I can switch platforms with relative ease. If your language would only be helpful on some limited (and ever shrinking) subset of platforms, then it would be too dangerous to rely on it.

I just don't feel like this solves a big enough problem to justify a whole new language.

Just to be clear, I'm not stopping you, go ahead. Making compilers is fun, I've made a few myself. Just don't expect to come up with something really useful that would displace existing tools. This is very hard to do. And in this respect, dreaming of features is a waste of time. Start making a C-like compiler first and see how fast you lose interest. If you don't lose interest, then you will have a base where you can add features and then there will be something to discuss.

You are not the first one to want to do something like this, but it always ends up in just discussions. Don't ask us what you want to do, if you have a vision, implement it and then we can say if it makes sense or not.

[SiliconWizard]

Also, have a look at existing languages before designing your own. Literally hundreds out there already. You're not the only one having tried.

For something close to C, you could have a look at Zig. See what it brings.

Before writing a compiler, try writing "real code" in your new language, just to see if it makes sense. Not just trivial code chunks.

Also, if you really want to try, I'd suggest "transpiling" to C first. Don't bother with the low-level code generation. At least not until it makes sense to do so. Additional benefit, apart from the saved time and effort, is that it will make interfacing with actual C code trivial.

[Sherlock Holmes]

Also, have a look at existing languages before designing your own. Literally hundreds out there already. You're not the only one having tried.

For something close to C, you could have a look at Zig. See what it brings.

Before writing a compiler, try writing "real code" in your new language, just to see if it makes sense. Not just trivial code chunks.

Also, if you really want to try, I'd suggest "transpiling" to C first. Don't bother with the low-level code generation. At least not until it makes sense to do so. Additional benefit, apart from the saved time and effort, is that it will make interfacing with actual C code trivial.

I agree with you. I've not looked at Zig actually, so thanks, I will indeed take a look. I did look at Hare recently another new language. I'm certainly not starting on a compiler, not yet anyway, that's a big undertaking even if using LLVM to manage the back end.

Transpiling is already on my list of things to think about, being able to generate standard looking C that performed the same operation as the new language is a great way to evaluate things.

The grammar I have started to outline resembles C only slightly, there are grammatical traits in C that are problematic not least of which is its odd way mixing procedures and functions into some hybrid concept with the useless "void" keyword and the ability to invoke a function yet overlook its return value and so on.

The use of things like ++X and J++ and so on, effectively expressions and assignments combined, reduces one's ability to reason about the code sometimes.

Then we have "forward declarations" and reliance on reserved words and so on, so there are a few things in C that are useful like braces for block delimiters and simplified structure definitions, but not a lot else.

[Sherlock Holmes]

This is the kind of thing I like:

https://ziglang.org/documentation/master/#Vectors