Author Topic: what would you use to replace Bash for scripting?  (Read 15942 times)

0 Members and 1 Guest are viewing this topic.

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4538
  • Country: nz
Re: what would you use to replace Bash for scripting?
« Reply #150 on: June 25, 2023, 10:43:47 pm »
MacOS switched back to zsh as the default shell in a later version. I think it was Catalina (10.15, October 2019).

This of course is only for new accounts. Old accounts keep their existing settings which, for me, have been bash ever since Rhapsody, which IIRC had tcsh as the default shell.

Any ideas why they have kept changing?

Nope. New people?
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15439
  • Country: fr
Re: what would you use to replace Bash for scripting?
« Reply #151 on: June 25, 2023, 10:48:33 pm »
Yes, maybe just the preference of a given manager at a given time. I was just wondering whether there could be definite technical reasons for changing.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4538
  • Country: nz
Re: what would you use to replace Bash for scripting?
« Reply #152 on: June 25, 2023, 11:24:09 pm »
Yes, maybe just the preference of a given manager at a given time. I was just wondering whether there could be definite technical reasons for changing.

From Rhapsody to now is 25 years ...

Most sources say OS X always used bash and zsh is the new thing. I couldn't immediately find a list of default shell by version. But this O'Reilly book confirms that tcsh was the default shell as recently as Jag-wire (10.2) -- and my memory is that was the same for all previous versions, back to NeXTStep.

https://docstore.mik.ua/orelly/unix3/mac/ch01_05.htm
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4283
  • Country: gb
Re: what would you use to replace Bash for scripting?
« Reply #153 on: June 26, 2023, 09:53:23 am »
Which is fine, really, except I'm not interested in interacting with such people at all.  I want to keep doing the grass-roots level that hopefully will help/lead to a true step forward, ending the programming language stagnation I've seen cripple software development for the last quarter century.  It has been papered over by the huge advances in computing capacity, but that just makes it wasteful.  With the current laughably bad reliability and security record, we need, and deserve, something better.  It may not make business sense to try to do that, but that is the kind of change I'd like to be part of in this world.

I honestly don't think you understand that scale.  It's all very well saying it's wasteful, but honestly, again, I don't think you understand it.

I've read your walls of text, mostly.  It's all out of date in terms of software development.  It's all narrow minded "I've been doing this my whole life" small scale crap.

Sure it's needed at times, but honestly, I'm not tricking you, we do everything in our power to get away from that bullshift hardcoded cryptic nonesense.

If you want a main driver for this, beyond anything else.... go and source yourself 100 developers for software project.

Now come and tell me what language and platform are you going to develop it in?
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Offline DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4247
  • Country: gb
Re: what would you use to replace Bash for scripting?
« Reply #154 on: June 26, 2023, 11:23:04 am »
I think the OP was looking for a scripting language a bit more elaborate than just a "shell"

Yup, I confirm  :D
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline helius

  • Super Contributor
  • ***
  • Posts: 3680
  • Country: us
Re: what would you use to replace Bash for scripting?
« Reply #155 on: June 26, 2023, 05:17:38 pm »
If you want a main driver for this, beyond anything else.... go and source yourself 100 developers for software project.
It appears that the meaning of "The Mythical Man-Month", after 50 years, has yet to be learned.
 

Offline Picuino

  • Super Contributor
  • ***
  • Posts: 1033
  • Country: es
    • Picuino web
Re: what would you use to replace Bash for scripting?
« Reply #156 on: June 26, 2023, 05:54:21 pm »
I think the OP was looking for a scripting language a bit more elaborate than just a "shell"

Yup, I confirm  :D
Python is the tool you are looking for.
 

Online SiliconWizard

  • Super Contributor
  • ***
  • Posts: 15439
  • Country: fr
Re: what would you use to replace Bash for scripting?
« Reply #157 on: June 26, 2023, 08:59:05 pm »
The end of this thread is getting particularly funny.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4538
  • Country: nz
Re: what would you use to replace Bash for scripting?
« Reply #158 on: June 27, 2023, 12:16:14 am »
If you want a main driver for this, beyond anything else.... go and source yourself 100 developers for software project.
It appears that the meaning of "The Mythical Man-Month", after 50 years, has yet to be learned.

I barely restrained myself yesterday from commenting that I have never seen a project that needed 100 developers that wasn't a complete fuck up before it even started -- and the only place they happen is typically some "Big Four" consulting company hoodwinking a government department into spending a billion dollars on an IT project that everyone with a clue knows could perfectly well be done for 10 million dollars with a team of 10.

No one in the commercial world uses teams of 100 people working on the same codebase.

 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: what would you use to replace Bash for scripting?
« Reply #159 on: June 27, 2023, 12:51:22 am »
No one in the commercial world uses teams of 100 people working on the same codebase.
That's an interesting claim. I have a counter-claim that many large tech companies do exactly that: https://en.wikipedia.org/wiki/Monorepo#:~:text=shared%20codebase,%20very%20large%20monorepos
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4538
  • Country: nz
Re: what would you use to replace Bash for scripting?
« Reply #160 on: June 27, 2023, 01:45:15 am »
No one in the commercial world uses teams of 100 people working on the same codebase.
That's an interesting claim. I have a counter-claim that many large tech companies do exactly that: https://en.wikipedia.org/wiki/Monorepo#:~:text=shared%20codebase,%20very%20large%20monorepos

"the code for a number of projects is stored in the same repository"

i.e. different codebases.
 

Offline helius

  • Super Contributor
  • ***
  • Posts: 3680
  • Country: us
Re: what would you use to replace Bash for scripting?
« Reply #161 on: June 27, 2023, 01:51:55 am »
The FAANG SV companies are anticompetitive monopolies, not "the commercial world".
Besides, the structure of a version control system is not pertinent to the scaling issues of software product management.
 

Offline DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4247
  • Country: gb
Re: what would you use to replace Bash for scripting?
« Reply #162 on: June 27, 2023, 09:24:23 am »
I think the OP was looking for a scripting language a bit more elaborate than just a "shell"

Yup, I confirm  :D
Python is the tool you are looking for.


I'm currently moving from bash to lua, I wrote about it a few pages ago, and only recently added that I am also writing some "helpers" in C, compiled to help bash scripting, and used as a library for other C helpers/tools, or imported into Lua scripts.

hybrid solution for now

"emerge" is written in Python, I am re-writing(1) it in C because Python has too many critical dependencies,  eats too many CPU cycles, and has too much ram on my routers (it's a serious problem when the ram is <256Mbyte and the CPU clock < 400Mhz)


(1) only essential functionalities
- dependency analysis(2)
- calculating the correct order of the packets to emerge
- calculating the correct order of packages from un-emerge
- panic() in case of cyclicality, so I always solve it by hand
- interpretation of Portage recipe classes
- interpretation of the ebuils, from EABI=1 to EABI=9
- so I support all things that are "deprecated and removed" (1..4) to Portage/emerge 2023

(2) I am re-using this algorithm to calculate the order of starting and stopping /etc/init.d services
one algorithm, several use-cases, and re-using code is a very good point!
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4283
  • Country: gb
Re: what would you use to replace Bash for scripting?
« Reply #163 on: June 27, 2023, 09:48:07 am »
If you want a main driver for this, beyond anything else.... go and source yourself 100 developers for software project.
It appears that the meaning of "The Mythical Man-Month", after 50 years, has yet to be learned.

I barely restrained myself yesterday from commenting that I have never seen a project that needed 100 developers that wasn't a complete fuck up before it even started -- and the only place they happen is typically some "Big Four" consulting company hoodwinking a government department into spending a billion dollars on an IT project that everyone with a clue knows could perfectly well be done for 10 million dollars with a team of 10.

No one in the commercial world uses teams of 100 people working on the same codebase.

Nobody said the same "codebase".  If by "codebase" you mean the configuration layer of the project, aka git.

Last project I left had 1039 people on it.  Yes, 1039.  In about 14 different countries, including, but not least, Russia and Ukraine... at the same time.

Not 1039 developers.  Probably about 40% tech.  Something in the order of around 200 "code bases" in about 5 git repos.

"codebase" in this sense could be anything from an Oracle distributed configuration and control framework, a kafka serialization frameworks, table compaction jobs, maintenance jobs, ingestion jobs, transformation and enrichment jobs, batch jobs, streaming jobs, micro-services..... WebUIs, data warehouses, data scientist clusters, GPU in memory database systems.  And all the security, command/control/reporting, "infrastructure as a service" required to back it all into a somewhat coherent and uniform fashion.

We migrated around about 150 'data feeds' for an international company from one platform, entirely to another.  Current feed and historical from around 100 different external and internal 'vendors'/'sources'.

"We" .. my part was about 5 different data feeds, including some in the order of 300million records and day and a datasize in memory of around 250Gb.  Others were near-realtime streams of data from external providers arriving in 30 second micro-batches containing 1000-2000 messages.  Others where just opening a CSV file and setting it down again in a different format, like parquet.

Writing code and piping things between processes on a CLI....  LOL

EDIT... and yes, I left because it became a "cluster fuck". 

It's easy to say, "that's because it was too big".

However, saying that all software can be written by a single person or a tiny team is also easy to say and catagorically false.

Software Engineering today is 90% about that problem and 10% writing code or SQL.
« Last Edit: June 27, 2023, 09:52:35 am by paulca »
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4283
  • Country: gb
Re: what would you use to replace Bash for scripting?
« Reply #164 on: June 27, 2023, 10:17:18 am »
On that scaling and this comment:

It appears that the meaning of "The Mythical Man-Month", after 50 years, has yet to be learned.

It doesn't work like that, but if your point is that it doesn't scale linearly, then thank you captain obvious.  This is also part of every single software engineering degree you will take. 

In software and I imagine most or all engineering communication is key.

Electronics and hardware people know this.  If it's just you making the circuit and product end to end, you can skip a VAST amount of diagrams, documentation and specifications.  You can skip a vast number of meetings too.  You can develop such a product (usually a prototype) amazingly fast.

However, when you can no longer be the single soul person doing the work end to end, the amount of documentation and the details it must go into go up non-linear with the more subdivisions of the work.  The number of meetings and workshops and debates and changings of minds and requirements go up and up and up.

In software engineering when a task/system/requirement becomes too big for a single person, you can move to a two man team (Pair programming).  However, social and civil issues arise from a team of 2.  It's a marriage and will suffer similar issues to one.  Often burning out in arguments after 6 weeks or so.  So 3 is usually better as no vote can hang without abstention.

3 developers rarely produce 3 times the work of 1.  The finger in the air rule in 60% per additional.  So 2 devs do as much work as 1.2 devs.  3 about 1.8.

The issue is the communication channels between team members and the inter-coupling of their work requiring communications multiply geometrically.  When there are 2 devs, there is only one pathway.  When there are 3 devs there are 3.  When there are 4 devs there are 6.

Honestly when you get to about 4 devs, you need some management or team leadership and they do not count directly towards teh "man month" count although they do tend to stop "pulling in many directions" problems.

The "preferred" "modern" model is to divide groups of devs up into teams of 5-10 with a "scrum master" serving one or two of these teams and a project manager over-seeing 5 related teams.  Software is broken down into "Epics" by management, then into smaller one to three day tasks which are then ordered by priority and dependencies, but ultimately left for developers to pick up and assign to themselves.

This is the development team manages and owns the development.

No.  It doesn't work with pure business types.  They simply cannot understand that we can tell them "what" they will have but not when, OR, we can tell them when they will have it, but not what. 

When you do get through to a team of business types and they trust you, they are usually happy for it.  Far, far too often business analysts and customer managers will UNDER estimate just what can be delivered with modern software tools and techniques.  They under sell the likely output and enforce stupid requirements resulting in inherent functionality being lost. 

Instead by operating very rapidly on a 2-3 week cycle and then demonstrating functioning software each cycle and allowing customer feedback each cycle allows a lot of the otherwise, "Would you just open you eyes and LOOK instead of talking crap!" to be skipped and wasted effort avoided.

Spending 2-3 weeks on an idea and having the customer say, "Nope, absolutely not!".  Is far better than doing it on a waterfall for 3 years like civil-service projects where, under their certified methodologys, if a requirement turns out to be wrong after 2.5 years the are legally required to go back to square ONE.  This is why many, many government IT projects cost so fucking much it's got nothing to do with software engineering, it's high brow legal quality and control methodologies that require every single line of code be documented.  Which is fine if you are writing a 777 autopilot control software, but not a fucking "Apply for your vehicle tax form", only form!

There is no man month.  Most tasks are assigned and estimated on a individual basis.  Either based on days or just on "complexity".  I've been in teams where the estimates where facial expression emojis!  Having management constantly under-estimate those aggregates is only because management are ... well management.  aka, usually skill-less.

Like everything, it depends where you are and who you are working for and with.  Projects come and go, stuff changes.  New skills are required, you are expected to learn.  Tech lead wants to switch from SpringBoot to a random app container framework you have never heard of.... best get googling.
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Offline DiTBhoTopic starter

  • Super Contributor
  • ***
  • Posts: 4247
  • Country: gb
Re: what would you use to replace Bash for scripting?
« Reply #165 on: June 27, 2023, 10:23:12 am »
my super-heroes here are Dominic Giampaolo and Cyril Meurillon, who in 1996 designed, implemented, tuned, documented, and tested, and fully debugged, one of the best innovative filesystems ever written!

All alone, only those two guys at Be Inc  :o :o :o
The opposite of courage is not cowardice, it is conformity. Even a dead fish can go with the flow
 

Offline Picuino

  • Super Contributor
  • ***
  • Posts: 1033
  • Country: es
    • Picuino web
Re: what would you use to replace Bash for scripting?
« Reply #166 on: June 27, 2023, 02:32:24 pm »
I think the OP was looking for a scripting language a bit more elaborate than just a "shell"

Yup, I confirm  :D
Python is the tool you are looking for.


I'm currently moving from bash to lua, I wrote about it a few pages ago, and only recently added that I am also writing some "helpers" in C, compiled to help bash scripting, and used as a library for other C helpers/tools, or imported into Lua scripts.

hybrid solution for now

"emerge" is written in Python, I am re-writing(1) it in C because Python has too many critical dependencies,  eats too many CPU cycles, and has too much ram on my routers (it's a serious problem when the ram is <256Mbyte and the CPU clock < 400Mhz)


(1) only essential functionalities
- dependency analysis(2)
- calculating the correct order of the packets to emerge
- calculating the correct order of packages from un-emerge
- panic() in case of cyclicality, so I always solve it by hand
- interpretation of Portage recipe classes
- interpretation of the ebuils, from EABI=1 to EABI=9
- so I support all things that are "deprecated and removed" (1..4) to Portage/emerge 2023

(2) I am re-using this algorithm to calculate the order of starting and stopping /etc/init.d services
one algorithm, several use-cases, and re-using code is a very good point!
Sorry, I missed reading that you were working in an embedded environment.
In that case Python is oversized and Lua has a lot of advantage.
 
The following users thanked this post: DiTBho

Offline Mattjd

  • Regular Contributor
  • *
  • Posts: 230
  • Country: us
Re: what would you use to replace Bash for scripting?
« Reply #167 on: July 09, 2023, 01:31:09 am »
I started looking for a way to program the same application in Python and I have found that it is necessary to replicate the operation of standard unix tools like "file", "find", "wc", "grep", etc.

Uh? These are all individual programs. You can call them from Bash, but otherwise from any other scripting language that allows executing external programs, which both Lua and Python can.

Bash doesn't do anything else than spawning processes for any command you ask it to execute and that's something you can do in many languages.
The only thing that may be more or less difficult in other languages is directly piping one executable to another - that would be something to check.
For Python, this is where to start: https://docs.python.org/3/library/subprocess.html

Or just goto pypi, a wrapper for subprocess for calling linux commands directly in python

https://pypi.org/project/sh/
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf