Python becomes the most popular language

[Picuino]

Yes, but eCommerce platforms still use interpreted languages like PHP.
In most cases, a single server is enough to provide service quickly.

This forum is a user intensive use case and works on PHP, a "slow" interpreted language, which is not so slow after all.
3094 Guests and 303 Users at this time!

[Cerebus]

So all high speed trains are slow because a hypersonic rocket is 20 times faster than a high speed train?

That reasoning is a fallacy. High-speed trains travel 20 times slower than a rocket, but that doesn't make them slow, they're still very fast.

Therefore I still argue that Python is not slow. C code can be 20 times faster, but that just means that C is very fast, not that Python is slow.

So your thesis is that a computer that runs like someone has poured treacle in it is not slow, it is actually fast, and we should refine our notions of fast on any particular piece of hardware as the speed that Python runs at and anything capable of running at something near the speed of the hardware as really, really fast. OK, good luck with that.

[tszaboo]

So all high speed trains are slow because a hypersonic rocket is 20 times faster than a high speed train?

That reasoning is a fallacy. High-speed trains travel 20 times slower than a rocket, but that doesn't make them slow, they're still very fast.

Therefore I still argue that Python is not slow. C code can be 20 times faster, but that just means that C is very fast, not that Python is slow.

So your thesis is that a computer that runs like someone has poured treacle in it is not slow, it is actually fast, and we should refine our notions of fast on any particular piece of hardware as the speed that Python runs at and anything capable of running at something near the speed of the hardware as really, really fast. OK, good luck with that.

How much time was to compile the C code?

[Cerebus]

So all high speed trains are slow because a hypersonic rocket is 20 times faster than a high speed train?

That reasoning is a fallacy. High-speed trains travel 20 times slower than a rocket, but that doesn't make them slow, they're still very fast.

Therefore I still argue that Python is not slow. C code can be 20 times faster, but that just means that C is very fast, not that Python is slow.

So your thesis is that a computer that runs like someone has poured treacle in it is not slow, it is actually fast, and we should refine our notions of fast on any particular piece of hardware as the speed that Python runs at and anything capable of running at something near the speed of the hardware as really, really fast. OK, good luck with that.

How much time was to compile the C code?

cerebus@shu:~/Desktop$ time gcc -O0 -o Loop Loop.c

real   0m0.086s
user   0m0.043s
sys   0m0.034s

[SiliconWizard]

Code: [Select]

#include <stdio.h>

int main (int argc, char const *argv[])
{
    int x;
   
    printf ("starting\n");
    for (int i = 0; i < 1000000; i++)
        x = x + 1;
    printf ("stopping\n");
   
    return 0;
}


Code: [Select]

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

print ("starting");
x = 1
for i in range(1000000):
    x = x + 1
print ("stopping");

We run both twice to ensure that they are on an even footing, not needing to service page faults for runtime binaries and the like (this will be to Python's advantage, it has a large runtime to load before it can do anything).
(...)
C - 4ms, Python - 113ms. Python manages a mere 3.5% of the performance of C for the most trivial task possible.

That C is a lot faster than Python comes as no surprise (except maybe for the most extreme fanboys)? But what may bite them even more is that... even Lua is much faster than Python:

Test.lua

Code: [Select]

function Test()
print("starting")
x = 1
for i = 1, 1000000 do
x = x + 1
end
print("stopping")
end

Test()
(x is a global variable here for Lua - and that's less efficient than using local variables...)

time lua Test.lua
starting
stopping

real   0m0,021s
user   0m0,020s
sys   0m0,001s

If I declare x as a local variable:

starting
stopping

real   0m0,009s
user   0m0,008s
sys   0m0,000s

[Cerebus]

time lua Test.lua
starting
stopping

real   0m0,021s
user   0m0,020s
sys   0m0,001s

If I declare x as a local variable:

starting
stopping

real   0m0,009s
user   0m0,008s
sys   0m0,000s

Ah, but you could have used one of them specially fast computers rather than the steam powered 2018 i7-8850H driven MacBook that I was using. 

I like it because it's a "Coffee Lake". That's got to be a proper programmer's processor, right?

[SiliconWizard]

The example I gave was run on a headless Linux box which is not my main workstation, and fine but not a beast by any means.
For a fair comparison though, I ran the Python code on the same machine:

starting
stopping

real   0m0,141s
user   0m0,109s
sys   0m0,012s

[Picuino]

Youtube works with Python. Not lua, nor PHP, much less in C.
Goggle believes that Python is fast enough to support that language.

[Siwastaja]

Youtube works with Python.

Just  .

This is exactly what Python fanboyism is all about. If something, such as Youtube, contains a line of Python, the whole thing "runs on Python".

Do you really think your Youtube experience is enabled by Python interpreter going through gazillion of Python code lines to implement the whole of Youtube?

Exactly the same as "Python is the AI language", when you configure a few input vectors on a python script and call a complete NN implementation (which is actually made in C, C++, assembly, OpenCL, CUDA, VHDL, Verilog, whatever combination thereof).

Python's actual role is to instantiate large modules, libraries. This avoids performance limitations. And also avoids working with the language in low level, when the language really isn't tailored for low level stuff. Try to learn this, and you'll sound less .

I know Python fanboys hate Python being called just a scripting language, but there's a grain of truth here. No one would implement a PCB CAD packege in Matlab/Octave code. Python is actually very close to that. It's an "intermediate" language which glues together pre-engineered modules. Highly useful, but not a full application development language. Yet a very good replacement for many simple use cases which would have been BASH scripts or written in BASIC.

Cerebus' simple demonstration is exactly to show you can't write responsive software low-level in Python. You are not even supposed to, no one in their right mind would do that. It's surprising some people apparently need that demonstrated and still don't get the point. You can read pretty much the same from every half-decent Python tutorial: let the libraries do the work. No, 100ms is not acceptable if it is, for example, a response time to a keyboard keypress while writing. And sluggish text editing fields on web apps is a real thing today.

But if you want to really know how Youtube works, there is quite a bit of learning involved. It's a huge infrastructure to make the cat video appear on your screen, and at least 90% of it is written in C, by very knowledgeable people, over many decades. The whole Internet works almost solely based on Unix/POSIX/C, like it or not.


This is, BTW, also why "Python becomes the most popular language". Also, if you count people who build cars, and people who drive cars, you will see driving cars is way more popular than building them. Yet, if you claim that you driving a car is the same thing as you building one, this is where you go wrong. Your experience is completely enabled by the work of others. Which is all good, enjoy it.

[Nominal Animal]

I'm kinda repeating what Siwastaja wrote above, but I think it is worth repeating.

Youtube works with Python. Not lua, nor PHP, much less in C.
Goggle believes that Python is fast enough to support that language.

You cannot infer qualitative statements from popularity, other than popularity.

I do bet Youtube/Alphabet uses Python for the same reason I do: it is good enough for the business and glue logic that the majority of their developers produce.
Anything critical, a very small but very good group of developers will implement in C optimized for their hardware architectures.

It also works well as a focusing mechanism.  They employ a lot of mathematically-oriented data scientists.  Let them develop and test their algorithms in some very high-level language.  Then, when it works, have them document the algorithm (something like an annotated peer-reviewed article; I have no idea what they actually use – I'm just describing what I would do).  Then, let them work on the next algorithm.  Have that small dedicated team turn the proof-of-concept Python idea and algorithm description into a C library, with a native Python interface.  You save three ways: cheaper mathematicians (by not being limited to the ones that can do both math and program), cheaper key developers (by not being limited to the ones that can also do math), easier management (by pulling the plug if the algorithm developer cannot even demonstrate it in Python in a reasonable timeframe; thus limiting sunken costs per idea).

Really, Python is simply very, very easy to interface to native libraries written in C; easier than any other scripting language I know.  It also has a gentle learning curve compared to e.g. Perl, similar to say PHP (but without the confusing dichotomies PHP has, like whether it is an OO or non-OO scripting language).  This makes it a very good candidate for this kind of use cases.

(To repeat, I use Python to implement user interfaces using e.g. Qt, with all the heavy computation and work done in native compiled C code.  This lets even end users tweak the user interface, without requiring any kind of development environment, while allowing me or the vendor keep the proprietary stuff in the binary-only libraries.
On web servers, the Python code does relatively little processing; most of it is business logic, and connecting data sources to templating libraries and such.  For example, the code I use for securing/fixing URLs before they're mapped to physical or virtual filesystem paths, uses regular expressions – which obviously use a very efficient native regular expression engine, and is not implemented in high-level Python, and is therefore quite efficient.  I do take care of documenting well what the regular expressions need to achieve and why.)

Edit: The reason I use plain pure C and not Python in the PulseAudio Volume Meter (in a nearby thread here), is that there just isn't enough glue logic there to make it useful to have a Python-based UI.  The actual interface to draw the VU bars (or however you want to visualize it) in C is a simple Gtk-Cairo one, which is very easy to tweak (if correctly modularized, i.e. the cranky bits like maxima decay done elsewhere), from just an example by even a beginner in C.  There just isn't any use or need for Python there.  It would just unnecessarily waste CPU power, without anything useful in return.

[Picuino]

I do not deny that, it seems to me that you have misunderstood me. I have developed applications in other languages ​​(Javascript, Java, C, ladder, VisualBasic, assembler, etc)
And each one is valid for its field of application.
When using Python, heavy use is made of libraries for low-level work, but program logic is more effectively developed in high level Python.
Pareto's law says that 20% of the code will take 80% of the execution time and that code is better off written in C, but high level program logic code is much better off written in Python.
Many of you sound like C fanboys and don't seem to understand why Python is so popular and widely used. It's not just a fad, it's widely used for good performance reasons. Execution speed is no longer the only important variable. At least not with current hardware. The speed and ease of software development is more important.

Edit: I agree with Nominal Animal

[Siwastaja]

Except that 20% of the code won't be in C and 80% in Python, in a given project. It just doesn't work that way. The "plumbing" takes a lot of work, while successful high-level code does seemingly a lot with small number of LoC, that's the whole point. If it's taking 80% of the code, then you are doing something seriously wrong.

You might be surprised how many C code lines is actually involved when you write a 100-LoC Python piece. It's easily in millions. Not 80-20.

It's just that most of that plumbing is hidden from you.

A very simple project might be Python alone, or a project which seemingly does a lot, but the work is done by libraries.

Any complex piece of software (PCB EDA or Youtube), and the percentage of Python falls towards zero, and it should be this way. Use Python where it shines. Number of LoC is small exactly because it is quick and easy to write as you say!

[jfiresto]

... Any complex piece of software (PCB EDA or Youtube), and the percentage of Python falls towards zero, and it should be this way. Use Python where it shines. Number of LoC is small exactly because it is quick and easy to write as you say!

I am just trying to unpack that last thought. Wouldn't it make more sense to write more LOC in the language that is quick and easy to write, and less in one the one that is more tedious? The opposite tastes of technical masochism. There must be other reason(s) that support your argument, for example, exploiting existing, well tested code that predates the popularity of Python.

[Siwastaja]

... Any complex piece of software (PCB EDA or Youtube), and the percentage of Python falls towards zero, and it should be this way. Use Python where it shines. Number of LoC is small exactly because it is quick and easy to write as you say!

I am just trying to unpack that last thought. Wouldn't it make more sense to write more LOC in the language that is quick and easy to write, and less in one the one that is more tedious? The opposite tastes of technical masochism. There must be other reason(s) that support your argument, for example, exploiting existing, well tested code that predates the popularity of Python.

No, it's the opposite way:

Complex low-level interactions can only be written in low-level languages, and they end up complex, because the actual thing they are dealing with, is complex. Someone's got to flip those bits!

Think about all the details that go in modern video compression, for example. They need to deal with every pixel, and gigabytes of datastreams. And then do the same to draw these pixels on screen.

Thanks to all that complexity hidden away, now the high-level implementation can use a very high-level language, and it ends up doing a lot using just a few lines of code visible, in that high-level language: videoPlayer.load("videofile")

It's not masochism, it's the reality. Trying to do the opposite, write complex low-level implementations in Python, is not only actual masochism, it also just does not work, not only because the language lacks the low level constructs, but also because of two-three orders of magnitude of performance penalty.

The key to understand this is that languages are only tools, they do not define the problems. The actual problems are what they are, they are layered (i.e., someone has to manufacture the electric drill, but you don't need to do that in order to use the drill to manufacture a coffee table), sometimes complex, sometimes easy, and easy languages are picked to solve easy problems!

[brucehoult]

... Any complex piece of software (PCB EDA or Youtube), and the percentage of Python falls towards zero, and it should be this way. Use Python where it shines. Number of LoC is small exactly because it is quick and easy to write as you say!

I am just trying to unpack that last thought. Wouldn't it make more sense to write more LOC in the language that is quick and easy to write, and less in one the one that is more tedious? The opposite tastes of technical masochism. There must be other reason(s) that support your argument, for example, exploiting existing, well tested code that predates the popularity of Python.

There is no technical reason for a language that is quick and easy to write code in to have to run slowly.

Examples of languages that are both productive to write in, and also run very nearly as fast as C, include Common Lisp, Dylan, and Julia. Oh, and these days, Javascript.

[madires]

Yes, but eCommerce platforms still use interpreted languages like PHP.
In most cases, a single server is enough to provide service quickly.

I'd call that wishful thinking. I see so many small and large online shops with discouraging response times - it sucks. Web 2.0 is driven by scripting languages, add tracking and tons of bloat, and it moves like molasses. Even with FTTH you only get WorldWideWait. It seems that users have forgotten how fast websites can be, even with dynamic content.

This forum is a user intensive use case and works on PHP, a "slow" interpreted language, which is not so slow after all.
3094 Guests and 303 Users at this time!

I bet most of the 3k+ users would love it when this forum would have a 20 times lower response time. The whole issue is about scaling. A single user wouldn't complain about 200ms vs. 10ms. But with many users at the same time the 200ms become 2s or more. So you have to find a compromise between how many users a server can serve with an acceptable response time and the cost of running that server (while hopefully making some money). Ten times more users would be a significant benefit. What is less expensive for you? Running 20 servers with fiddleware or running two servers (one for resiliency) with optimized software? Or maybe you want to run 10 servers with optimized software and outperform your competitors by an order of magnitude?

[Picuino]

Sure, most of the 3k+ users would love it when this forum would have a 20 times lower response time.
But, after all, this forum is based on SMF, written in PHP. And there is no news that David plans to switch to another faster platform.

Edit: And note that for me there are no speed problems in the forum.

[Vtile]

I did quickly slab together that 'for loop' thing, this time with GUI in freepascal with LCL, got something like 16ms for 5M iterations inside IDE and and all the debug data with this really poor web-book laptop.



Anyhow, I do like Python, but not for speed.

[Siwastaja]

As N. Animal repeatedly says, popularity is not proof of anything else than popularity.

Pretty much everybody agrees that PHP is pure horror and nothing else. Yet is is and stays popular. And it seems to "work". Many horrific contraptions "work", it's not surprising.

You need to look far into history of things and put it all in perspective, including many non-technical reasons, to see why PHP, for example, is popular.

I still remember the time the PHP came, the selling argument was the ease of getting started. You just wrote some simplistic code, variables from the URL came automatically into variables in your code, and just upload the php file on the server and access it! It got the job done.

But just getting the job done is the wrong metric. It needs to be done right, too. Like you should not coat still wet concrete with vinyl flooring as it will cause massive chemical emission issues after a few years.

PHP is a prime example what happens when developers use "ease of getting started" as the most important metric. It makes me feel sad how people still use this argument when it comes to STM32Cube for example, or whatever. Don't do this to yourself. It hurts in the long run. Ease of getting started is important in small hobby projects, but its importance gradually falls off as the level of seriousness goes up.

PHP maybe felt good at first, but it has since caused unimaginable amount of human suffering and misery, and it still does that. But this is all hidden from people who just use software written in PHP. Those who need to maintain it suffer PTSD. And those who suffer from stolen personal data as a result of gaping security holes, are actual victims, too.

Luckily, Python causes much less pain than PHP.

[Marco]

Meh, most of the misery "caused" by PHP was actually caused by SQL. But plaintext API's with escape sequences are Unix so beyond criticism.

[Nominal Animal]

Meh, most of the misery "caused" by PHP was actually caused by SQL. But plaintext API's with escape sequences are Unix so beyond criticism.

?

No, most of the misery caused by PHP was the tool (the PHP interpreter) trying to do the developers' work for them, and guess when escape sequences were needed and when not, and do it automagically (magic quotes).

I also do not understand the claim about plaintext Application Programming Interfaces.  Plain text is traditionally used in two ways in Unix: for configuration data (as opposed to binary registries or databases), and as the lowest common denominator for interprocess communication.

While tools like the command-line sqlite3 interpreter do use escape sequences, none of the API DB libraries have had such limitations.  Even if you were to use the command-line interpreter to access a database, a sane approach would be to do any escaping required at that interface layer.  Exposing transport details to upper layers is definitely not Unix.

In other words, there is NOTHING Unix about PHP using a pure text interface to databases.  It was just the simplest way the PHP developers could implement it.  They knew that passing variables separately would have been better and safer; it just wasn't what they wanted.  And that makes the misery caused PHP's own damn fault.

[Siwastaja]

And of course if we are pedantic, note the difference between the PHP language, its built-in libraries, and the whole PHP framework including server configurations.

The language specification itself is crappy, too, many historical layers of poor design, different styles of naming conventions etc. But that's manageable. It's not that C's standard library is logical or safe, either. (Python does libraries better, that's why it can be used at truly high level.)

When you combine the crappy base PHP language, varieties of server configurations which make the same piece of code perform differently, the handy "register_globals" thing and magic quotes enabled (these are history, now, of course), and shoddy SQL practices, this all forms the well-known disaster.

Many people, me included, just call all that "PHP" short-hand. Actual use of PHP the language is more of a red flag, than directly responsible for all the horror we attach to the name.

[Siwastaja]

Magic quotes is actually a very relevant example of the "don't reinvent the wheel!!" discussion in the Microcontrollers subforum.

The whole point is, security is hard, security is complex, so you don't want to roll your own implementation, right?

An existing implementation must be good, right?

"help [prevent] code written by beginners from being dangerous" was the specified rationale. Sounds right? Anybody who does not use Magic quotes and rolls their own instead must be an idiot, right?

But the history shows what really happened.

Never blindly trust work by others, especially if the track record is poor.

[Picuino]

It seems that criticizing is easier than explaining the reasons why things work.
It is also more exciting and more satisfying to criticize something than to try to explain reality.
Now PHP has had its moment of criticism.
I'm not going to be the one to support the use of PHP, because I don't think it's a good language.
But, after all, PHP is still used in more than 80% of dynamic websites. That is a fact and it can be criticized or it can be explained by the benefits of interpreted languages, which are real, despite their slower speed compared with compiled ones.


Edit: PHP popularity has dropped below 80% of websites.
https://en.wikipedia.org/wiki/PHP#Popularity_and_usage_statistics
https://w3techs.com/technologies/details/pl-php

[Nominal Animal]

When you combine the crappy base PHP language, varieties of server configurations which make the same piece of code perform differently, the handy "register_globals" thing and magic quotes enabled (these are history, now, of course), and shoddy SQL practices, this all forms the well-known disaster.

Many people, me included, just call all that "PHP" short-hand. Actual use of PHP the language is more of a red flag, than directly responsible for all the horror we attach to the name.

Yep, agreed.  PHP is one of the languages that if one wants to write a security-sensitive tool to do something, you need to make sure the language/environment (interpreter, libraries, and configuration combined) is not and does not do anything stupid: development in PHP is basically fighting against PHP, instead of working with PHP, using the shorthand Siwastaja described.

Contrast that to Python.  Not any language features, but the entire environment, and things like the Python PEP index.  Just by reading the PEPs that are relevant to whatever one is working on in Python, you can delve deep into the developers mind as to how they intended that sort of expressions to be done in Python.  This is useful.  Take a minute to read through PEP 572 -- Assignment Expressions, and you immediately learn what kinds of patterns cause unnecessary slowdown in Python, and how to write better Python code.  Or, if you are writing a Python interface, and wondering how to add type hints, read PEP 484 -- Type Hints, and you'll not only understand how, but also why.

One reason Dylan and Julia are not that widely used among the Linux community –– that is, 'used' in the sense as in 'implementing tools or applications' –– is that they are not packaged for Linux distributions, unlike e.g. Rust.  You might say that the binary installers (and the sources for the 'open source zealots') are sufficient, but they aren't as, uh, effortless, as native packages.  This effortless-ness is a key part in the popularity of the language.  (Note: I am not saying this is a good or a bad thing.  I am only trying to describe how things happen.)  In my own experience, across say three decades, it looks like this effortlessness (or whatever you want to call the combined ease of use with minimal risk and high availability of reliable information) is much bigger factor in the popularity than technical quality is.

Circling back to the topic of this thread, you could summarize my own opinion that 1) Python is sufficiently effortless (in the abovementioned sense) to use in many environments to make it a valid easy choice, with 2) lacking any show-stopper flaws like in PHP (at least in the past had), which makes it both useful and popular.  (This is in my use cases and in my opinion based on using Python in actual real-world utilities and applications; others opinions, and my own, will vary as times and needs change.)

It is not Python is particularly good; it is more that it is not so bad that one needs, or has a reason to find, something/anything better.  It is sufficient for the task.

If someone argues some other scripting language over Python, any counterarguments I might have would necessarily revolve around one or both of the above two points.  The moment something that fulfills both points at least as well, I'm very likely to switch myself.

All that said, it is interesting to see how different other peoples' metrics/measuring sticks are when comparing programming languages.
(That said, I definitely have completely different ones wrt. compiled languages and scripting languages, because I use them for very different use cases.)