The Imperium programming language - IPL

[Sherlock Holmes]

@guys, ignore this troll, seriously, and he will leave the forum.
Otherwise we have to babysit this grown dude.
I am seriously going to ignore the whole topic, I have seriously enough.

At last, no more juvenile posts and outbursts!

If the subject doesn't interest you or you feel you can't positively contribute then you SHOULD leave the thread and everyone else here, alone, what harm is being done? what are you afraid of? why be a pest?

The way you whine here you'd think I was sending you personal message or spam or making harassing phone calls, the fact is the thread is just a discussion nobody asked you for anything, you are clearly the the troll here FFS.

There's always one, someone who wants to bicker and insult and offend, it's sad, if you don't want to discuss the subject then don't what's hard to understand here?

[EEVblog]

Both of you, please quit the bickering. Either contribute to the thread or ignore it and/or ignore each other.

[Sherlock Holmes]

I've drafted a document that compares language features of C, PL/I and the hypothetical IPL, where the features are conducive to writing an operating system in that language, for those interested jut visit the page in GitHub.

This arose from discussions elsewhere. The impetus was my view that a language good for MCU work must also be suitable for developing an operating system, any language claiming to be "good" at hardware programming must also be good for OS development.

To my knowledge few language have a track record of delivering real operating systems, PL/I and C are the most well known I think.

[tggzzz]

I've drafted a document that compares language features of C, PL/I and the hypothetical IPL, where the features are conducive to writing an operating system in that language, for those interested jut visit the page in GitHub.

This arose from discussions elsewhere. The impetus was my view that a language good for MCU work must also be suitable for developing an operating system, any language claiming to be "good" at hardware programming must also be good for OS development.

To my knowledge few language have a track record of delivering real operating systems, PL/I and C are the most well known I think.

Algol is another such language. It continues to be commercially successful in delivering operating systems.

xC is another for MCUs.

Smalltalk and Squeak/Pharo are another, and have been at least as influential as C, albeit indirectly. I'll bet that in the long term Smalltalk will be more influential than C.

You would benefit from having a wider experience and less of a tunnel vision.

[tggzzz]

BTW, I hate the quotation ("When you have eliminated all which is impossible, then whatever remains, however improbable, must be the truth.") in your .sig with a vengence. It implies that the person really can imagine all possibilities, thus precluding genuine step advances.

If I was to say "I can't imagine how spooky action at a distance can work, therefore it can be discounted", then I would be correctly vilified, and/or be a laughing stock, and/or ignored.

[cfbsoftware]

To my knowledge few language have a track record of delivering real operating systems

Here are some such languages I can think of:

Pascal:
  UCSD Pascal and CP/M-68K

Modula-2: 
  Acorn Archimedes and ETH Medos-2 

Oberon:
  Project Oberon, ETHOS, AOS/A2

Also read the article in The Register: The wild world of non-C operating systems

https://www.theregister.com/2022/03/29/non_c_operating_systems/

[Sherlock Holmes]

I've drafted a document that compares language features of C, PL/I and the hypothetical IPL, where the features are conducive to writing an operating system in that language, for those interested jut visit the page in GitHub.

This arose from discussions elsewhere. The impetus was my view that a language good for MCU work must also be suitable for developing an operating system, any language claiming to be "good" at hardware programming must also be good for OS development.

To my knowledge few language have a track record of delivering real operating systems, PL/I and C are the most well known I think.

<Waffle, Waffle, Waffle>

You would benefit from having a wider experience and less of a tunnel vision.

So would you.

[Sherlock Holmes]

To my knowledge few language have a track record of delivering real operating systems

Here are some such languages I can think of:

Pascal:
  UCSD Pascal and CP/M-68K

Modula-2: 
  Acorn Archimedes and ETH Medos-2 

Oberon:
  Project Oberon, ETHOS, AOS/A2

Also read the article in The Register: The wild world of non-C operating systems

https://www.theregister.com/2022/03/29/non_c_operating_systems/

Thanks, I did however say "few languages" there are just a few that's the point I was making, its like six or seven languages it seems.

[Sherlock Holmes]

My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language, I'm using that as a kind of criteria, what would one expect from a language if they wanted to write an OS? those same capabilities would be huge when using the language to code for MCUs.

[Sherlock Holmes]

Operator symbols

Given that I want to support arithmetic/logical shift left/right along with rotate left and right, the question emerges as to what priorities these operators should be given as well as the actual symbol to use for them.

For rotation these sprang to mind:

Rotate left: <@<
Rotate right: >@>

The circularity of the @ seems a reasonable sign for rotation.

Anyway, suggestions welcome.

[gf]

My core thesis here though is that if a language is ill suited to writing an OS...

In a completely abstract, machine-independent way?
With or without a certain amount of additional assembly code?

[SiliconWizard]

My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language,

Well, I think almost everyone says that, even these days, although it's usually defined in the positive form rather than the negative one that you used here.
It's what is commonly called a "system programming language". A reasonable list is there:
https://en.wikipedia.org/wiki/System_programming_language

It's not exhaustive though by any means. Just regarding Wirth languages, they don't even list Oberon, while it's been designed (and used) to develop a full OS, and it's being used as a language for programming MCUs (which is the whole business of one of the forum users here.)

[Sherlock Holmes]

My core thesis here though is that if a language is ill suited to writing an OS...

In a completely abstract, machine-independent way?

With or without a certain amount of additional assembly code?

To the most reasonable extent possible yes. The changes we'd need to make to the OS codebase if we moved from one target to another, should be as small as we can reasonably make it. Ideally we'd "just" recompile the bulk of the source for the new target and we're good. I don't seek to eliminate assembler, at least that's not a goal, having a small core of assembly language bootstrap code etc is not a big deal, seems reasonable.

If you had to write a serious OS like Windows, Linux, Android etc, what essential language features would you need, if there were umpteen potentially suitable language available, how would you choose (ignoring things like skills availability, popularity etc).

I guess "portability" is an obvious one I need to add to my list, although the term itself needs some kind of definition. But reducing unpredictability is very high on the list, semantic behavior that is left to the implementor or is somehow undefined should be kept to an absolute minimum, stuff like the way C has an undefined order of evaluation when passing args and so on.

[Sherlock Holmes]

My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language,

Well, I think almost everyone says that, even these days, although it's usually defined in the positive form rather than the negative one that you used here.
It's what is commonly called a "system programming language". A reasonable list is there:
https://en.wikipedia.org/wiki/System_programming_language

It's not exhaustive though by any means. Just regarding Wirth languages, they don't even list Oberon, while it's been designed (and used) to develop a full OS, and it's being used as a language for programming MCUs (which is the whole business of one of the forum users here.)

Right, so what makes a language a "systems programming language"? The Wikipedia article lists a bunch but those are systems programming language by virtue of the fact they've been used to do "systems programming", I'm curious about what specifically must a language have to "be" such a language.

Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?

[westfw]

My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language

I dunno.  Any language can write an OS, given appropriate "support functions" written in some other language.

Are you really counting early CP/M and MSDOS 1 as "operating systems"?  That's a pretty low bar.

BTW, CP/M 2.2 (which was what was heavily used) appears to have been re-written in assembly language. http://www.cpm.z80.de/source.html

What about "self-hosting"?  Is it important for your language to be able to compile itself?  This seems to be an implicit requirement for modern languages, but I'm not really sure it should be.

Finally, I'm not sure what the hangup is about avoiding assembly language is, in the first place.  By the time you talk about some of the features that have come up ("equal cycle length code paths"), I think you've already thrown portability - a major selling point of HLLs - out the door...

[Sherlock Holmes]

For an OS too I'm including device driver code, code like we see on MCUs, that must interface to very specific aspects of buses and device protocols and so on.

[Sherlock Holmes]

My core thesis here though is that if a language is ill suited to writing an OS then it's ill suited for use as an MCU language

I dunno.  Any language can write an OS, given appropriate "support functions" written in some other language.

Are you really counting early CP/M and MSDOS 1 as "operating systems"?  That's a pretty low bar.

BTW, CP/M 2.2 (which was what was heavily used) appears to have been re-written in assembly language. http://www.cpm.z80.de/source.html

What about "self-hosting"?  Is it important for your language to be able to compile itself?  This seems to be an implicit requirement for modern languages, but I'm not really sure it should be.

Finally, I'm not sure what the hangup is about avoiding assembly language is, in the first place.  By the time you talk about some of the features that have come up ("equal cycle length code paths"), I think you've already thrown portability - a major selling point of HLLs - out the door...

Interesting points. Sure CP/M was (at the time anyway) an OS, Kildall crafted most of it in PL/I too, they were simple by todays standards but they managed to write a lot of code not in assembler. As for "self hosting" or "boot strapping" that's a great way to harden a language, compiler but not that important. I'd write a new language in C# for example, a means to an end. I think bootstrapping was important in the 60s because there was no other language around to write the compiler, so it was assembler first then some version 0.0001 of the language used to rewrite that assembler and so on, the Multics guys did that in fact initially.

Portability? well yes, but we would want to minimize, isolate as much as we can of that non-portability.

[Sherlock Holmes]

If we look at "modern" C there are a long list of things that have been added, some in recent standards and others as vendor extensions, so the way I am thinking of this is to devise a new language with all of these historic changes and extensions as hindsight, stuff one can incorporate from the outset rather than forty years later.

The grammar I've selected (basically derived from PL/I) is not selected for systems programming reasons but primarily for lacking reserved words, letting new keywords appear over time with ZERO impact on existing source code. If that requirement is not met then we get silliness - eventually - like this post describes.

That's pretty fundamental, all of a sudden we can't use lowercase names for classes, structs? This is a modern version of rules like all labels must begin column 2 and so on!

[westfw]

Re: binary file inclusion...

Code: [Select]

LIBS += -Wl,--format=binary -Wl,--start-group $(RESOURCES) -Wl,--end-group -Wl,--format=default

The gnu binutils "objcopy" utility also gives you the option of creating a linkable (.elf format) file from arbitrary binary data, and produces the same convenient symbols (I imagine it shares binary input conversion code with the linker):

Code: [Select]

> avr-objcopy -O elf32-avr -I binary binfile foo.elf

Code: [Select]

> avr-objdump -x foo.elf
Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .data         0000000b  00000000  00000000  00000034  2**0
                  CONTENTS, ALLOC, LOAD, DATA
SYMBOL TABLE:
00000000 l    d  .data  00000000 .data
00000000 g       .data  00000000 _binary_binfile_start
0000000b g       .data  00000000 _binary_binfile_end
0000000b g       *ABS*  00000000 _binary_binfile_size

I don't know whether I consider that more, or less, convenient than the linker commands; it gives you the opportunity to make an ordinary makefile rule for "compiling" .png files (or whatever) to .o files, which can then also be linked normally.

I about died when I discovered that a team at work was using the whole "convert to a C source file containing a big char array" method on a several-hundred megabyte linux .iso image :-(

[MIS42N]

Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?

I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

[tggzzz]

Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?

I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

Well, pointers were there in languages that influenced the design of C.

To take a less well known example, consider Algol68. INT x states x is an immutable integer, REF INT x states x is a reference to an integer which is variable, REF  REF INT state x is a pointer to an integer. In Algol 68 the types [10]INT, [10]REF INT, REF[10]INT, REF[10]REF INT are all distinct types and all useful. They closely correspond to the types declared by these C declarations:
typedef const int i[10]; typedef const int *ir[10]; typedef int const *ri[10]; typedef int const *rir[10]; http://www.cap-lore.com/Languages/aref.html

I wonder if the OP realises that Algol68 has many of the features he deems desirable, e.g. "if" can be a variable, and it has symbol set (with alternatives) that work on the various different character sets available on computers at the time. And those are just the start. https://opensource.com/article/20/6/algol68 Truly, Algol is an improvement on most of its successors.

I suspect the OP is unaware of https://rosettacode.org/wiki/Rosetta_Code which gives the same program in many different languages. Examples: there are 68 implementations of Dijkstra's algorithms, and bitwise operations implemented in 144 languages.

That really ought to give him something to "compare and contrast" before committing to something for his language.

[MIS42N]

Well, pointers were there in languages that influenced the design of C.

I was not intending to imply C was the first language to use pointers. What I was trying to say is the confluence of pointers, all the logical operators, and a standard library that has useful system oriented functions gives C the right ingredients to be a systems programming language. Which was what OP asked.

[tggzzz]

Well, pointers were there in languages that influenced the design of C.

I was not intending to imply C was the first language to use pointers. What I was trying to say is the confluence of pointers, all the logical operators, and a standard library that has useful system oriented functions gives C the right ingredients to be a systems programming language. Which was what OP asked.

True, but they weren't unique to C.

What was unique to C was that it came as part of a zero cost operating system available on the minicomputers that were becoming widely available.

Get UNIX for cheapness and experimentation, get a free high (for the time) level language. All important for universities, research labs, and medium sized companies. What was in C was more or less irrelevant; all languages had problems and everybody found ways of working around them

Unfortunately current computers are nothing like PDP11s

[Sherlock Holmes]

Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?

I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

Well, pointers were there in languages that influenced the design of C.

To take a less well known example, consider Algol68. INT x states x is an immutable integer, REF INT x states x is a reference to an integer which is variable, REF  REF INT state x is a pointer to an integer. In Algol 68 the types [10]INT, [10]REF INT, REF[10]INT, REF[10]REF INT are all distinct types and all useful. They closely correspond to the types declared by these C declarations:
typedef const int i[10]; typedef const int *ir[10]; typedef int const *ri[10]; typedef int const *rir[10]; http://www.cap-lore.com/Languages/aref.html

I wonder if the OP realises that Algol68 has many of the features he deems desirable, e.g. "if" can be a variable, and it has symbol set (with alternatives) that work on the various different character sets available on computers at the time. And those are just the start. https://opensource.com/article/20/6/algol68 Truly, Algol is an improvement on most of its successors.

I suspect the OP is unaware of https://rosettacode.org/wiki/Rosetta_Code which gives the same program in many different languages. Examples: there are 68 implementations of Dijkstra's algorithms, and bitwise operations implemented in 144 languages.

That really ought to give him something to "compare and contrast" before committing to something for his language.

Algol68 was the language that most influenced PL/I which I've spoken about at length here, there was also a build of it that accepted Russian keywords, I wonder if you were aware of that.

I have to ask, politely, would you please stop insinuating that I am "unqualified" in some way or other, to discuss this subject? Repeatedly making disparaging remarks and insulting comments is really not what I expect in any forum like this.

If you cannot disagree with someone without implying they are inferior, then there's really no point in discussing anything with anyone.

I went to the trouble in my very first post to mention that I was experienced with compiler design and development, I anticipated that some might be skeptical, that a language is a large undertaking and not to be treated glibly, I even pointed to the source code for the entire system. I know all that and therefore summarized what I had done before, yet you speak to me as if I'm a dimwitt, well I have the experience buddy, I know of what I speak, I know how large a project this can be, I know what's involved, I'm likely far more able to do this than you.

I've attached an example of the assembler code listing generated by that compiler for the 32 bit Intel CPU, tell me, how does that not carry any weight with you? The compiler did not simply generate assembler, it generated COFF object and dll files, getting assembler was a compile option, it generated that listing as it generated the code.

So I've actually done this before, thirty years ago buddy, the whole shebang, when there was no internet, no tools like Antlr, and I did most of it with Borland C on DOS with a VGA screen, have you ever built anything comparable from the ground up? what makes you such an expert that you feel you can tell me how to do things?

[Sherlock Holmes]

Leaving ubiquity and popularity and availability aside, what is it about C that give it any kind of status as a systems programming language?

I think it was the concept of a pointer *, which often resolved to a memory address, and that address could be just about anything. Fun things like **thing were simple to write (but maybe not so simple to understand). Also the inclusion of all bit operators and/or/xor/shifts. The standard library includes system type functions like malloc(). A lot of C maps well to machine instructions. And the lack of any nanny code like array bound checking made it fast (but dangerous).

It persists because it is still dangerous. Many languages try to avoid the danger and impose rules that are 95% helpful and 5% frustrating.

In my opinion.

So this potential new language will have pointers but also offsets. That is we'll be able to manipulate the address of a datum and also the offset of a datum relative to some base. This is a powerful feature lacking in C.

Changing the subject for a moment I once designed a fast in memory database technology that ran on Windows. This leveraged memory mapped files, so a "database" was in fact a snapshot of a portion of process address space, say 16GB for args sake.

That memory consisted of a tree of data structures, quite complex with index trees and all kinds of data with pointers to other data all situated within that 16GB contiguous block.

But any process could map that file into any part of its address space (this was shared memory, multiple processes could access it concurrently), the OS decides where to map such data into the virtual address space as and when a process maps it and that actual address within each process can vary (one can force it to map at some specific address but that's hardly a good idea).

Therefore in the data the links could not be stored as pointers and referenced and dereferenced as pointers, the tree nodes had to store an offset (from the start address of the 16GB block).

Then at runtime (using C macros) we referenced and deference using the offsets, the macro did the arithmetic to compute the actual process relative pointer).

If C had offered me the ability to use language supported offsets directly, all of that plumbing would vanish, and the cost of calculating pointers/offsets would have all but vanished, being done wholly in the addressing code generated by the compiler.

PL/I always offered that and the example I give, although not directly applicable to an MCU, is a real world example of how such a feature is valuable.

So we'll be able to write code that can load a linked list into memory (perhaps from disk or somewhere) and have that linked list 100% relocatable, the actual address that it gets put is immaterial because no absolute pointers are stored only relative offsets, the notation used in the code "->" is the same, whether we're referring to a pointer or an offset, the language does the work.

We'd be able to save/load self referential data structures that can be loaded at any address and still work fine, that's quite a neat feature.

That's the kind of systems programming capability I'm talking about here.