Python becomes the most popular language

[PlainName]

Personally, I think that even if there's tons of 'bad' code written in Python, it's a much better situation than if that same code were all in 'good' assembly (ignoring that that's totally impractical, in real terms), because it's very unlikely that Python code is going to do something dangerous, most likely it will just be inefficient, which is not really a big deal.

Well, I agree it's better but for a different reason - the state of the art progresses much faster if you have more shoulders to stand on, even if they aren't the strongest. Often, just knowing something can be done allows you to do it better, whereas previously it may never have occurred to try.

[PKTKS]

Personally, I think that even if there's tons of 'bad' code written in Python, it's a much better situation than if that same code were all in 'good' assembly (ignoring that that's totally impractical, in real terms), because it's very unlikely that Python code is going to do something dangerous, most likely it will just be inefficient, which is not really a big deal.

Well, I agree it's better but for a different reason - the state of the art progresses much faster if you have more shoulders to stand on, even if they aren't the strongest. Often, just knowing something can be done allows you to do it better, whereas previously it may never have occurred to try.

it just happens that  the real de facto good code in what matters... most...
(Scientific apps, Numeric algos, fully tested and decades reviewed libs.. )

they are already written,  fully stressed tested, and optimized and ported...

It just happens they are written in C and FORTRAN mostly. Several reasons

All reasons boils down to a very steady RATIONALE.

Nobody needs to re-write them.  Unless obviously these newbies...
It would take a decade to acquaint these several decades tools..

it seems cheap easy to jump start to Pythonish... look ma I can code Python

Paul

[madires]

A typical real world scenario is that a bad security bug is found in WordPress plugin X and someone adds that to his scanning bot to automatically seize control of WordPress instances running that specific vulnerable plugin. In a short time the bad guy controls 30k WordPress instances.

Here we go: Massive WordPress JavaScript Injection Campaign Redirects to Ads (https://blog.sucuri.net/2022/05/massive-wordpress-javascript-injection-campaign-redirects-to-ads.html)

[AndyBeez]

I will debug Python, C++, C#, Java, Bash, Rust and even Javascript, in it's many self inflicted framework mutations. But it will be a very cold day in hell if I find myself touching LUA.

[bd139]

Personally, I think that even if there's tons of 'bad' code written in Python, it's a much better situation than if that same code were all in 'good' assembly (ignoring that that's totally impractical, in real terms), because it's very unlikely that Python code is going to do something dangerous, most likely it will just be inefficient, which is not really a big deal.

Well, I agree it's better but for a different reason - the state of the art progresses much faster if you have more shoulders to stand on, even if they aren't the strongest. Often, just knowing something can be done allows you to do it better, whereas previously it may never have occurred to try.

it just happens that  the real de facto good code in what matters... most...
(Scientific apps, Numeric algos, fully tested and decades reviewed libs.. )

they are already written,  fully stressed tested, and optimized and ported...

It just happens they are written in C and FORTRAN mostly. Several reasons

All reasons boils down to a very steady RATIONALE.

Nobody needs to re-write them.  Unless obviously these newbies...
It would take a decade to acquaint these several decades tools..

it seems cheap easy to jump start to Pythonish... look ma I can code Python

Paul

I’m regularly finding bus sized holes in off the shelf C.

In fact I’m responsible for a couple of mid level CVEs 

Stuff does need to be rewritten.

[Karel]

Personally, I think that even if there's tons of 'bad' code written in Python, it's a much better situation than if that same code were all in 'good' assembly (ignoring that that's totally impractical, in real terms), because it's very unlikely that Python code is going to do something dangerous, most likely it will just be inefficient, which is not really a big deal.

Well, I agree it's better but for a different reason - the state of the art progresses much faster if you have more shoulders to stand on, even if they aren't the strongest. Often, just knowing something can be done allows you to do it better, whereas previously it may never have occurred to try.

it just happens that  the real de facto good code in what matters... most...
(Scientific apps, Numeric algos, fully tested and decades reviewed libs.. )

they are already written,  fully stressed tested, and optimized and ported...

It just happens they are written in C and FORTRAN mostly. Several reasons

All reasons boils down to a very steady RATIONALE.

Nobody needs to re-write them.  Unless obviously these newbies...
It would take a decade to acquaint these several decades tools..

it seems cheap easy to jump start to Pythonish... look ma I can code Python

Paul

I’m regularly finding bus sized holes in off the shelf C.

In fact I’m responsible for a couple of mid level CVEs 

Stuff does need to be rewritten.

Rewritten means new bugs and new security issues.
It's my impression that programmers are not getting smarter, on the contrary...
It's astonishing to see that, today, programmers don't check input and assume many things.
Also because their bosses don't give them the time to carefully think and test.

[PlainName]

In fact I’m responsible for a couple of mid level CVEs

I'm sure you didn't mean it the way it sounds! 

[nfmax]

Knowing @bd139 by repute, I wouldn’t be surprised if he did!

[bd139]

That was intentional and can be interpreted both ways simultaneously

[Vtile]

As a general thought regarding programming languages, and not specific to Python (although it definitely is concerned), I have a problem with languages that are tightly coupled with their implementation. This is wrong on many levels.

Yes, this is annoying, especially when they feel free to make seriously incompatible changes between versions so that old programs don't work on new versions.

This.

Even if there is a use_version_x param or whatever, nobody seems to use it. And then to build a interpreter that -could- deal with it at worst with a noisy warning but instead fatal errors out. 

It's as though they resurrected all the worst bits of winword *.doc obsolete versioning, DLL hell and syntax stupidity like it's some kind of competition or homage to MS. If the next version of Python shows up with a floating paper clip, I'm gunna do backflips.

To be fair to Python, this business of producing a new version of the programming language itself every few weeks/months/each-year is not a Python exclusive "feature". Most of the new kid on the block programming languages do this, and to my mind it is an insane way to go about producing tools. Would anyone produce mechanical tools that had a new version every few months which required you to use, say, a version 4.3 screwdriver to insert/remove a version 4.3 screw? No one would try to do this with actual physical tools to a bunch of burly hammer wielding men for fear of serious bodily damage.

Moreover, the practice leads to a sense that everything is ephemeral, everything is in beta, nothing is built to last, nothing has passed the test of time, so why should you bother to produce anything stable, long lived and itself capable of passing the test of time with these tools?

Where it comes to Python, I have to agree with this. Over the years I have tried to install some Python based software but ran into many incompatibilities between versions and OS. Initially I had the impression that Python was easely portable and maintainable but the opposite is true. It is even worse compared to the DLL hell that plagued the older Windows versions. IMHO it is better to write an application in C++ using Qt or WxWidgets and link it statically to have a fighting chance of being able to install & run an application next year compared to using Python.

If you want to distribute a Python application, you have to hand-out pre-configured VM images.

Bottom line: Python is great to tinker a script together but it absolutely sucks for writing applications (unless you want to keep sinking money into software that is essentially finished).

It was so nice to find a program backthen written with delphi, that just did run without any dll frameworks nor heavy installers .. most of the time they still run as back then.

[Nominal Animal]

IMHO it is better to write an application in C++ using Qt or WxWidgets and link it statically to have a fighting chance of being able to install & run an application next year compared to using Python.

It was so nice to find a program backthen written with delphi, that just did run without any dll frameworks nor heavy installers .. most of the time they still run as back then.

I wonder if it would be useful to show how to create a dynamically linked application that can use either system libraries, or specific versions of the dynamic libraries provided with the software itself.  (I can only show an example on Linux, and maybe BSDs and Macs, since they too use ELF binaries.)

Let me sketch the basic idea, since it is rather simple.

Let's say your binary is at /path/bin/appname.  It is actually a shell script that sets up the environment variables for the dynamic linker, and then executes (replaces itself with) the actual application binary, /path/bin/appname.bin.  This is already very common in Unix/Linux/POSIX/BSD environment: for example, /usr/bin/firefox is a shell script that does exec /usr/lib/firefox/firefox.

You have a full set of dynamic libraries provided with the app installed in say /path/lib/appname/app.libs/, and a full set of symlinks to either system libraries or provided versions in say /path/lib/appname/lib/.  The dynamic linker is instructed to only use that directory, /path/lib/appname.lib/, for the dynamic libraries.

The shell script also verifies that none of the dynamic libraries is dangling.  (They shouldn't be, as they should be symlinked to the backwards-compatible version of the library, typically the major version.)  If it finds any, it just reports that the operating system environment has changed; one way is to launch a helper app that can report (and fix) the dangling symlinks (by pointing them to the provided versions of the dynamic libraries – changing the symlinks is usually a privileged operation on these OSes).  That helper app would run exclusively on top of the app-provided libraries, of course.

In essence, each application has a set of symlinks to the dynamic libraries it uses, and a launcher script that tells the dynamic library to use that directory for the dynamic libraries.

At install time, you do need logic that looks at the standard dynamic libraries already installed on the OS, and either heuristically determines (and lets the user override) for each library file whether the OS-provided one or the app-provided one is used; typically just based on the library minor version number.

In Linux, one can use ldd to list the dynamic libraries needed by an application (although you may need to run it recursively to find all the dependencies, since it is perfectly normal for a dynamic library to have other dynamic libraries as dependencies), so you don't actually need to do anything in the C/C++ source code.  It is just a matter of determining the full set of dynamic libraries needed.
With Python it is slightly more involved, because the interpreter itself has one set of dependencies, and the modules and scripts another set that are only accessed if actually used.

This kind of app should be self-contained if necessary, but also able to use (bug-fixed) versions of libraries when desired without recompiling.

[bd139]

So like windows?

[Siwastaja]

The problem with all the bloat hidden in libraries is, it is like brushing all the crap under the carpet. It is still there.

Then you need to choose between truly dynamic or practically static linking. Truly dynamic, you end up with all the pain of managing dependencies and getting the right versions, and for some weird reason I don't fully understand, all automated attempts to manage the dependencies sometimes fail. Another option is to provide the right library versions with the app, either statically linked, or just zipped together so they are "dynamically" loaded but practically static, per app. But this reveals the true amount of bloat, and suddenly a trivial app is 500 megabytes of crap to be downloaded. But at least it works. Hopefully.

[bd139]

I think apple have a reasonable solution to this. The apps arrive in bundles which you just drop in the Applications directory. Each bundle is a directory which contains all the resources required to run it. The ABI and API is stable on the OS and the responsibilities of each actor are well defined. The bundles contain one “universal” binary which contains native compiled output for ARM and x86-64.

The current mess we have, which isn’t that bad on Linux really, is mostly API churn on poorly defined crap like the desktop environments and toolkits. The core is very stable.

Sort of the same with windows which has hundreds of different versions of each DLL where they forgot to do the thinking bit before defining a persistent API.

I will say I like Go though. Everything static. This has been the least painful thing for me to deploy anywhere in the last 30 odd years of doing it. Binary might be 30 megs but the heap is tiny!

[bd139]

That's fine but most proprietary stuff needs to work on more than one platform so Linuxisms need to go away.

[Cerebus]

I think apple have a reasonable solution to this. The apps arrive in bundles which you just drop in the Applications directory. Each bundle is a directory which contains all the resources required to run it. The ABI and API is stable on the OS and the responsibilities of each actor are well defined. The bundles contain one “universal” binary which contains native compiled output for ARM and x86-64.

Two points:

Stable-ish API, in that you specify a minimum system API version that you're happy to use when you build a native macOS binary application.

The application bundle doesn't have to be a native macOS binary application. As long as the 'binary' is in the right place in the application bundle directory it can be a script and with macOS also being Unix you can use the classic hash bang mechanism to use a script interpreter of choice, allowing you to package anything as a macOS bundle with a little ingenuity. I have seen Python applications, complete with bundled Python interpreter and libraries distributed as macOS application bundles. CrossOver (a commercial repackaging of the wine Windows emulator) uses macOS application bundles to repackage Windows applications into macOS application bundles making running an emulated Windows application look just like you're starting a native application (the resemblance often falls apart immediately after this point of course).

[Nominal Animal]

So like windows?

None of the applications I used ever did that.  They included the dynamic libraries they needed, and if you wanted to change one, you had to replace the file in the application directory.

So, no, nothing like Windows.

That's fine but most proprietary stuff needs to work on more than one platform so Linuxisms need to go away.

No, not everything needs to run on Windows.  Works absolutely fine everywhere else.

Really, some days I think it is Windows that needs to go away.  (Because it always needs special coddling, always being the odd one out.)

[bd139]

I really thing Windows should stay. It should however be tidied up.

But linux, BSD and macOS work slightly differently as well.

[Nominal Animal]

But linux, BSD and macOS work slightly differently as well.

If you write service/daemon code, they are surprisingly similar.  POSIX.1-2008 interfaces in particular are quite portable across these.  Usually, the kind of code I write works on all three as-is, except when I design a daemon around some Linux-specific feature.  When someone wants to port them to Windows, there is usually some kind of wonkyness that would be easiest to fix by modifying the perfectly working code and use a slightly inferior choice instead. But if there is an incompatibility in BSD or MacOS, it is usually a trivial fix (and I don't mean peppering the code with #if defined(__APPLE__) && defined(__MACH__) ... #else ... #endif stuff).
Just my personal experience, though.

But GUI toolkits, enumerating devices, notifications and other non-GUI events, those indeed are a pain even between Linux, BSD, and macOS.

[bd139]

The problem is that eventually you end up with autoconf which no one deserves

I prefer languages and toolchains that are platform agnostic. So Python, Go, .Net, Java generally have much more longevity than native APIs.

Although I'm currently working on something in Swift and SwiftUI because the Qt version of the software doesn't work properly on Macs. It's just completely different and a very comfortable place to be.

[Nominal Animal]

The problem is that eventually you end up with autoconf which no one deserves

Eww, no, I meant actual source-level compatibility.  No autoconf, no build time tests.  Just portable code, with usually no OS-specific stuff.

(And when there is any, I like to separate that stuff into os-specific files, with the common file pulling the correct one based on preprocessor macros.  So no configuration either at build time.  But easy to change to use OS_name macros set at build time based on uname -o if not preset via a shell or environment variable.  Just a plain Makefile to compile suffices.)

I prefer languages and toolchains that are platform agnostic. So Python, Go, .Net, Java generally have much more longevity than native APIs.

Well, POSIX.1-2008 is as old as Python 3 is, without any API changes up to current POSIX.1-2017, and very compatible with its earlier versions (including Single UNIX Specification) back to 1997 or earlier.  (The SUS roots link POSIX back to BSD, although since then, POSIX has added quite a lot of useful stuff.  In practice, current BSDs, macOS, and Linux C libraries provide pretty much all of POSIX.1-2017.)

(Anyone writing code for these OSes and using opendir()/readdir()/closedir() instead of glob()/scandir()/nftw() or fts_{open,read,children,set,close}() is not doing their job properly, for example.)

And if you write in C instead of C++, you'll use Gtk, with Gtk 3 released in Feb 2011, although I'd consider 3.8 (Mar 2013) the first stable version.  It does have some annoying deprecations in later 3.x versions, and although 3.22 (Sep 2016) was supposed to be the latest version (with only bug fixes, no new features added as minor revisions; latest is 3.22.30)), a 3.24 was released in Sep 2018 (with latest revision, 3.24.34, released in May 2022 thus far).
While the widgets did change from Gtk 2 to 3, it wasn't a painful change for me, although some other developers I know did find it very annoying to deal with.

Thus, based on my personal experience with these, I do disagree.

[PKTKS]

The problem is that eventually you end up with autoconf which no one deserves

While the widgets did change from Gtk 2 to 3, it wasn't a painful change for me, although some other developers I know did find it very annoying to deal with.

It is an enormous  and over bloated  overhead just for cosmetic changes introduced by Gtk3
So there is no really much advantage on it..  sounds exactly like MS wonder world..
upgrade just for upgrade

And... considering AUTOTOOLS..   just look the recent AVRDUDE shift from a messy bad implemented autotools made by folks wo a clue on it.. using a very competent  done CMAKE by some clever folk

AVRDUDE now uses a VERY VERY VERY well done cmake template..

nobody is forced to use a veteran master tool like autoconf..

... however... competent tools like OCTAVE running on multiple *NIX and mainframes are likely to use it
Paul

[SiliconWizard]

I don't know any magical solution to this library dependency crap. The only mitigation that I can think of is to limit dependencies to the strict minimum, which is unfortunately the exact opposite of what is being done.

As to bundling all required shared libraries with every application, that's a mitigation for getting a working piece of software, but that's even more storage bloat. Shared libraries were supposed to save storage (and in some cases, RAM too) space. If they don't, then they are basically getting absurd.

Python doesn't solve anything much here: you have to distribute the whole sheband (the version of Python you validated your software against, the right libraries, etc) in a bundle as well. Otherwise users are very likely to end-up with a non-working piece of shit outside of your own R&D department. Talk about bloat.

Sure the non-binary approach alleviates this to an extent - just let the user recompile the application for their given environment, with some version requirements for the libraries being used. Of course this doesn't scale - you'll be stuck with a very narrow pool of users using this approach.

In fact, this is one of the main reasons Linux has not taken off on the desktop, much more so than the desktop environments themselves. The difficulty for third parties to distribute software in a binary form, compared to commercial OSs.

[PKTKS]

(..)
As to bundling all required shared libraries with every application, that's a mitigation for getting a working piece of software, but that's even more storage bloat. Shared libraries were supposed to save storage (and in some cases, RAM too) space. If they don't, then they are basically getting absurd.
(.)

This **IS** what most (if not all)  apps do for the MS so called compatibilty

It is either a complete mess of DLLs hell ..
or a self contained duplicated bundle of already compiled libs.. (kinda LD_LIBRARY_PATH in reverse)

The DLL mess in MS is know to be an insane and over bloated solution...
although it works it can barely be called like that.

Paul

[Picuino]

Tiobe Index.

"August Headline: Python going through the roof

Python seems to be unstoppable. The scripting language gained another 2% this month. It is now at an all time high of 15.42% market share. It is hard to find a field of programming in which Python is not used extensively nowadays. The only exception is (safety-critical) embedded systems because of Python being dynamically typed and too slow. That is why the performant languages C and C++ are gaining popularity as well at the moment. If we look at the rest of the TIOBE index, not that much happened last month. Swift and PHP swapped places again at position 10, Rust is getting close to the top 20, Kotlin is back in the top 30, and the new Google language Carbon enters the TIOBE index at position 192. -- Paul Jansen CEO TIOBE Software"