"Please stop writing shell scripts"

https://pythonspeed.com/articles/shell-scripts/

"Everything works fine.

And then, one day, your shell script does something completely wrong.

That’s when you realize your mistake: bash, and shell scripting
languages in general, are mostly broken by default. Unless you are very
careful from day one, any shell script above a certain complexity level
is almost guaranteed to be buggy… and retrofitting the correctness
features is quite difficult."

On Mon, 28 Mar 2022 13:25:55 -0400, Andrei Z. <no-email@invalid.invalid> wrote:
> "Please stop writing shell scripts"
>
> https://pythonspeed.com/articles/shell-scripts/
>
> "Everything works fine.
>
> And then, one day, your shell script does something completely wrong.
>
> That’s when you realize your mistake: bash, and shell scripting
> languages in general, are mostly broken by default. Unless you are very
> careful from day one, any shell script above a certain complexity level
> is almost guaranteed to be buggy… and retrofitting the correctness
> features is quite difficult."

In that case, no one should ever write computer programs, since they are too complex,
depend on other programs that may change, or hardware that users may change, or data
the programmer didn't anticipate, and suddenly go completely wrong.

Ignore the article. While it's true that a lot of computer software, including shell
scripts, are created by people who don't know enough to do it properly, that doesn't
mean software should not be created. It just means people need to learn to do things
properly. Unfortunately, most people either learn by trial and error, or not at
all, or from other people who also don't know how to do things properly. People need
to learn to only use software and information from trusted sources, or that they know
how to verify.

Regards, Dave Hodgins

On 3/28/22 11:25 AM, Andrei Z. wrote:
> Everything works fine.
>
> And then, one day, your shell script does something completely wrong.
>
> That’s when you realize your mistake: bash, and shell scripting
> languages in general, are mostly broken by default. Unless you are very
> careful from day one, any shell script above a certain complexity level
> is almost guaranteed to be buggy… and retrofitting the correctness
> features is quite difficult.

You can generalize that to any form of programming, not just shell scripts.

--
Grant. . . .
unix || die

On 2022-03-28 14:06, David W. Hodgins wrote:
> On Mon, 28 Mar 2022 13:25:55 -0400, Andrei Z. <no-email@invalid.invalid>
> wrote:
>> "Please stop writing shell scripts"
>>
>> https://pythonspeed.com/articles/shell-scripts/
>
> In that case, no one should ever write computer programs, since they
> are too complex, depend on other programs that may change, or
> hardware that users may change, or data the programmer didn't
> anticipate, and suddenly go completely wrong.
>
> Ignore the article. While it's true that a lot of computer software,
> including shell scripts, are created by people who don't know enough
> to do it properly, that doesn't mean software should not be created.
> It just means people need to learn to do things properly.
I agree about ignoring the article and learning to do things properly.

Most of the examples in that article were of the form:

some_command
echo "Success"

I.e., they never once checked the exit status of the command and always
unconditionally reported success. They then used their broken examples
to claim shell scripting is bad. The article was either written by
somebody with little to no shell experience or is deliberately being
misleading. Either way, it's not worth reading.

"David W. Hodgins" <dwhodgins@nomail.afraid.org> writes:

> On Mon, 28 Mar 2022 13:25:55 -0400, Andrei Z. <no-email@invalid.invalid> wrote:
>> "Please stop writing shell scripts"
>>
>> https://pythonspeed.com/articles/shell-scripts/
>>
>> "Everything works fine.
>>
>> And then, one day, your shell script does something completely wrong.
>>
>> That’s when you realize your mistake: bash, and shell scripting
>> languages in general, are mostly broken by default. Unless you are very
>> careful from day one, any shell script above a certain complexity level
>> is almost guaranteed to be buggy… and retrofitting the correctness
>> features is quite difficult."
>
> In that case, no one should ever write computer programs, since they are too complex,
> depend on other programs that may change, or hardware that users may change, or data
> the programmer didn't anticipate, and suddenly go completely wrong.

This is about choice of languages, not whether programs should be
written at all.

Back when I was still working there was more than one project in trouble
that I rescued by converting a bunch of shell/sed/awk/grep stuff into
Perl.

Now I'd use Python.

I prefer to use one language that can do everything rather than creating
a nest of multiple languages.

Every once in a while I'll create a .sh file but if there's the
slightest change the script will grow, I prefer to use a more robust
language.

--
Dan Espen

On 3/28/22 1:17 PM, Dan Espen wrote:
> Back when I was still working there was more than one project in
> trouble that I rescued by converting a bunch of shell/sed/awk/grep
> stuff into Perl.

In my experience, sed and awk are vary rarely used by themselves for
automation. They are almost always used in conjunction with something else.

I'd up-vote Perl. Many people I know would down-vote Perl.

> Now I'd use Python.

I'd definitely down-vote Python.

> I prefer to use one language that can do everything rather than
> creating a nest of multiple languages.

There is something to be said for using a /single/ language that can do
everything that's needed. But that's a question of number of languages,
not which language. Contemporary Bash and Zsh can now do many of the
things that I've seen others use sed / awk / etc. to do.

> Every once in a while I'll create a .sh file but if there's the
> slightest change the script will grow, I prefer to use a more robust
> language.

I feel like there is an unspoken subtext that I believe really should be
spoken.

There are many things that shell scripts are quite appropriate for.
There are many things that shell scripts are inappropriate for. The
biggest difference to me is the size and complexity. I find that things
that are smaller (and not likely to grow) and / or cross multiple other
things are better for shell scripts. Especially thing that would need
to shell() to run commands outside of their native language; e.g.
external executables. I think that shell is probably a poor language
for something that will be more vertically integrated. I think that
shell can be a good language when horizontally acting across multiple
things.

Aside: Remember, most init systems execute small shell scripts.
(Independent of the technology of the init system.) Said scripts are
small and written specifically to do a small number of things (start,
stop, restart, status, etc.) with a singular entity.

--
Grant. . . .
unix || die

On Mon, 28 Mar 2022 15:17:08 -0400, Dan Espen <dan1espen@gmail.com> wrote:
> I prefer to use one language that can do everything rather than creating
> a nest of multiple languages.

Every programmer has their own preferences. Having different preferences is about
choice, not what is right or wrong.

> Every once in a while I'll create a .sh file but if there's the
> slightest change the script will grow, I prefer to use a more robust
> language.

Using the right tool for the job is important. A hammer may be used to put in a
wood screw, but that won't last and is not the right tool.

Shell scripts have their uses, as do other programming languages. I've seen lots
of badly written python scripts, as well as bash scripts. Any language takes
education to learn to use properly. Putting out a statement "Stop writing shell
scripts", is just click bait, not a valid or remotely useful instruction.

Regards, Dave Hodgins

Grant Taylor <gtaylor@tnetconsulting.net> writes:
> On 3/28/22 11:25 AM, Andrei Z. wrote:
>> Everything works fine.
>> And then, one day, your shell script does something completely
>> wrong.
>> That’s when you realize your mistake: bash, and shell scripting
>> languages in general, are mostly broken by default. Unless you are
>> very careful from day one, any shell script above a certain
>> complexity level is almost guaranteed to be buggy… and retrofitting
>> the correctness features is quite difficult.
>
> You can generalize that to any form of programming, not just shell scripts.

That would be missing the point, which is that shell is especially bad
as a programming language. I agree, for the reasons given in the
article.

--
https://www.greenend.org.uk/rjk/

Grant Taylor schrieb am 28.03.22:

> On 3/28/22 11:25 AM, Andrei Z. wrote:
>> Everything works fine.
>>
>> And then, one day, your shell script does something completely wrong.
>>
>> That’s when you realize your mistake: bash, and shell scripting
>> languages in general, are mostly broken by default. Unless you are very
>> careful from day one, any shell script above a certain complexity level
>> is almost guaranteed to be buggy… and retrofitting the correctness
>> features is quite difficult.
>
> You can generalize that to any form of programming, not just shell scripts.

With one difference: The syntax of bash shell scripts is more of a
practical joke that for some strange reasons made it to reality.

MfG
Gerald

--
Wenn ich Schokolade sehe, höre ich zwei Stimmen in mir.
Die eine sagt: "Iss sie!"
Die andere sagt: "Hast du nicht gehört? Du sollst sie essen!"

On Mon, 28 Mar 2022, Andrei Z. wrote:
> "Please stop writing shell scripts"
>
> https://pythonspeed.com/articles/shell-scripts/
>
> "Everything works fine.
>
> And then, one day, your shell script does something completely wrong.

The discussion overlooked one thing: The adminstration of unix

Beeing an sysadmin of several unix flavours, I can administrate
of of them with posix shell scripts.

That said, I can't program in any other language. But shell scripts are
sufficent.

On Mon, 28 Mar 2022 20:25:55 +0300, Andrei Z. wrote:

> "Please stop writing shell scripts"
>
> https://pythonspeed.com/articles/shell-scripts/
>
> "Everything works fine.
>
> And then, one day, your shell script does something completely wrong.
>
> That’s when you realize your mistake: bash, and shell scripting
> languages in general, are mostly broken by default. Unless you are very
> careful from day one, any shell script above a certain complexity level
> is almost guaranteed to be buggy… and retrofitting the correctness
> features is quite difficult."

But, if you use python instead, everything will be peachy - you
will even get minty fresh breath round the clock.

Is that it?

On 3/28/22 2:21 PM, Gerald Gruner wrote:
> With one difference: The syntax of bash shell scripts is more of a
> practical joke that for some strange reasons made it to reality.

Have you seen m4?

--
Grant. . . .
unix || die

On 3/28/22 2:47 PM, Jens Stuckelberger wrote:
> But, if you use python instead, everything will be peachy

I question the veracity of that statement.

Not the least of which is systems that don't have Python installed for
one reason or another.

$DEITY help you if white space is lost through copy and pasting snafu.

I'm not aware of any single Unix (like OS) that doesn't have a shell
that will run scripts.

--
Grant. . . .
unix || die

In comp.os.linux.misc, Grant Taylor <gtaylor@tnetconsulting.net> wrote:
> On 3/28/22 2:47 PM, Jens Stuckelberger wrote:
>> But, if you use python instead, everything> will be peachy
> I question the veracity of that statement.

I read Jens' statement as sarcasm.

> Not the least of which is systems that don't have Python installed for
> one reason or another.

Not unknown but rare. Just like systems with /bin/sh but no bash exist
but are rare.

> $DEITY help you if white space is lost through copy and pasting snafu.

Yeah, I'm not a fan of python (and yaml) whitepace rules.

Elijah
------
has been working in some bare minimal alpine environs recently

Eli the Bearded <*@eli.users.panix.com> writes:

> In comp.os.linux.misc, Grant Taylor <gtaylor@tnetconsulting.net> wrote:
>> On 3/28/22 2:47 PM, Jens Stuckelberger wrote:
>>> But, if you use python instead, everything> will be peachy
>> I question the veracity of that statement.
>
> I read Jens' statement as sarcasm.
>
>> Not the least of which is systems that don't have Python installed for
>> one reason or another.
>
> Not unknown but rare. Just like systems with /bin/sh but no bash exist
> but are rare.

I'm just reading about these credit card size computers like the
rasberry pi. These tiny little things run linux and
are programmed in python.

I think the odds of finding bash might be slightly higher than finding
python but not significantly different.

--
Dan Espen

On 3/28/22 9:18 PM, Dan Espen wrote:
> I'm just reading about these credit card size computers like the
> rasberry pi. These tiny little things run linux and are programmed
> in python.

Python is one of many different languages that are available on
distributions that run on Raspberry Pis.

If it's a language that Debian provides support for, you can probably
just install it.

If it's a language that FreeBSD ports provides support for, you can
probably just ""install (compile) it.

If you have language source code, you can likely compile it on any Unix
(like) OS that runs on the Pis.

With the various architecture emulations that are available, you aren't
restricted to just what will run natively on the Raspberry Pi. Want to
emulate a PDP, go for it. Want to run Windows programs, I'm quite
confident that you can (all be it slowly). IBM mainframe OS &
associated programming languages, yep, that's an old hat trick for the
Raspberry Pi.

--
Grant. . . .
unix || die

"Andrei Z." <no-email@invalid.invalid> writes:

> "Please stop writing shell scripts"
>
> https://pythonspeed.com/articles/shell-scripts/
>
> "Everything works fine.

While I don't agree with the recommendation I find the article has value
in that it points out four gotchas in bash scripts and how to avoid
them, namely by starting a bash script with
set -euo pipefail. Interestingly that works as is for zsh as well.

The article even links to the "unofficial bash strict mode" which offers
some more detail as to the why and also links to Shellcheck which is
static checker for bash and sh code.

Anssi Saari <as@sci.fi> writes:
> "Andrei Z." <no-email@invalid.invalid> writes:
>> "Please stop writing shell scripts"
>>
>> https://pythonspeed.com/articles/shell-scripts/
>>
>> "Everything works fine.
>
> While I don't agree with the recommendation I find the article has value
> in that it points out four gotchas in bash scripts and how to avoid
> them, namely by starting a bash script with
> set -euo pipefail. Interestingly that works as is for zsh as well.
>
> The article even links to the "unofficial bash strict mode" which offers
> some more detail as to the why and also links to Shellcheck which is
> static checker for bash and sh code.

Not enough. See http://mywiki.wooledge.org/BashFAQ/105 for further
issues with error handling in shell.

--
https://www.greenend.org.uk/rjk/

Grant Taylor <gtaylor@tnetconsulting.net> writes:

> On 3/28/22 9:18 PM, Dan Espen wrote:
>> I'm just reading about these credit card size computers like the
>> rasberry pi. These tiny little things run linux and are programmed
>> in python.
>
> Python is one of many different languages that are available on
> distributions that run on Raspberry Pis.
>
> If it's a language that Debian provides support for, you can probably
> just install it.

The stuff I've read so far shows python "import"ing a bunch of
libraries specific to the raspberry, like usb access and access to the
sensors that surround the circuit board. You don't just need the
language to work, you need access to device features. Python seems to
be the preferred language.

--
Dan Espen

On 29/03/2022 10:14, Anssi Saari wrote:
> "Andrei Z." <no-email@invalid.invalid> writes:
>
>> "Please stop writing shell scripts"
>>
>> https://pythonspeed.com/articles/shell-scripts/
>>
>> "Everything works fine.
>
> While I don't agree with the recommendation I find the article has value
> in that it points out four gotchas in bash scripts and how to avoid
> them, namely by starting a bash script with
> set -euo pipefail. Interestingly that works as is for zsh as well.
>
> The article even links to the "unofficial bash strict mode" which offers
> some more detail as to the why and also links to Shellcheck which is
> static checker for bash and sh code.

The problem with shell is that it looks like it started as a batch
language, and then got extended where it shouldn't orta been.

Its fine for batch stuff but even simple conditionals start making it
look ugly

--
Religion is regarded by the common people as true, by the wise as
foolish, and by the rulers as useful.

(Seneca the Younger, 65 AD)

The Natural Philosopher <tnp@invalid.invalid> writes:

> On 29/03/2022 10:14, Anssi Saari wrote:
>> "Andrei Z." <no-email@invalid.invalid> writes:
>>
>>> "Please stop writing shell scripts"
>>>
>>> https://pythonspeed.com/articles/shell-scripts/
>>>
>>> "Everything works fine.
>> While I don't agree with the recommendation I find the article has
>> value
>> in that it points out four gotchas in bash scripts and how to avoid
>> them, namely by starting a bash script with
>> set -euo pipefail. Interestingly that works as is for zsh as well.
>> The article even links to the "unofficial bash strict mode" which
>> offers
>> some more detail as to the why and also links to Shellcheck which is
>> static checker for bash and sh code.
>
> The problem with shell is that it looks like it started as a batch
> language, and then got extended where it shouldn't orta been.
>
> Its fine for batch stuff but even simple conditionals start making it
> look ugly

The half-arsed approach to error handling goes back to the start.
Acceptable for interactive use but it’s never been particularly
practical to write anything reliable in it. Like other 1970s tech it
holds on despite better options being available because reskilling and
rewriting are expensive.

--
https://www.greenend.org.uk/rjk/

Richard Kettlewell wrote:
> The Natural Philosopher <tnp@invalid.invalid> writes:
>
>> On 29/03/2022 10:14, Anssi Saari wrote:
>>> "Andrei Z." <no-email@invalid.invalid> writes:
>>>
>>>> "Please stop writing shell scripts"
>>>>
>>>> https://pythonspeed.com/articles/shell-scripts/
>>>>
>>>> "Everything works fine.
>>> While I don't agree with the recommendation I find the article has
>>> value
>>> in that it points out four gotchas in bash scripts and how to avoid
>>> them, namely by starting a bash script with
>>> set -euo pipefail. Interestingly that works as is for zsh as well.
>>> The article even links to the "unofficial bash strict mode" which
>>> offers
>>> some more detail as to the why and also links to Shellcheck which is
>>> static checker for bash and sh code.
>>
>> The problem with shell is that it looks like it started as a batch
>> language, and then got extended where it shouldn't orta been.
>>
>> Its fine for batch stuff but even simple conditionals start making it
>> look ugly
>
> The half-arsed approach to error handling goes back to the start.
> Acceptable for interactive use but it’s never been particularly
> practical to write anything reliable in it. Like other 1970s tech it
> holds on despite better options being available because reskilling and
> rewriting are expensive.
>
Now there are Best Practices for Bash Scripts, e.g.

https://betterprogramming.pub/best-practices-for-bash-scripts-17229889774d

Jens Stuckelberger <Jens_Stuckelberger@nowhere.net> wrote:
>On Mon, 28 Mar 2022 20:25:55 +0300, Andrei Z. wrote:
>
>> "Please stop writing shell scripts"
>>
>> https://pythonspeed.com/articles/shell-scripts/
>>
>> "Everything works fine.
>>
>> And then, one day, your shell script does something completely wrong.
>>
>> That’s when you realize your mistake: bash, and shell scripting
>> languages in general, are mostly broken by default. Unless you are very
>> careful from day one, any shell script above a certain complexity level
>> is almost guaranteed to be buggy… and retrofitting the correctness
>> features is quite difficult."
>
> But, if you use python instead, everything will be peachy - you
>will even get minty fresh breath round the clock.

There are many automation tasks that are written as a two-liner in
shell and a twenty liner in python or any other "real" programming
language. Using command-line binaries in python and perl is often done
using an extra shell layer, which gives you another layer of quoting
hell if your work is getting any serious.

There is a niche for shell scripts, and we all would be better off if
we'd ditch shell scripts of mid level complexity in favor of "real"
programs, but sometimes (always) things evolve.

Who has never decided to rewrite a shell script in perl and got caught
by things like filename globbing or directory iteration that are
built-in in shell but need to be explicitly programmed in any other
language?

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Mannheim, Germany | Beginning of Wisdom " |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

On 2022-03-28, John-Paul Stewart <jpstewart@personalprojects.net> wrote:
>
> On 2022-03-28 14:06, David W. Hodgins wrote:
>> On Mon, 28 Mar 2022 13:25:55 -0400, Andrei Z. <no-email@invalid.invalid>
>> wrote:
>>> "Please stop writing shell scripts"
>>>
>>> https://pythonspeed.com/articles/shell-scripts/
>>
>> In that case, no one should ever write computer programs, since they
>> are too complex, depend on other programs that may change, or
>> hardware that users may change, or data the programmer didn't
>> anticipate, and suddenly go completely wrong.
>>
>> Ignore the article. While it's true that a lot of computer software,
>> including shell scripts, are created by people who don't know enough
>> to do it properly, that doesn't mean software should not be created.
>> It just means people need to learn to do things properly.
> I agree about ignoring the article and learning to do things properly.
>
> Most of the examples in that article were of the form:
>
> some_command
> echo "Success"
>
> I.e., they never once checked the exit status of the command and always
> unconditionally reported success. They then used their broken examples
> to claim shell scripting is bad.

Not checking the returned status from a function/command call is VERY
VERY common no matter what language! People who don't check the returned
status are just bad programmers.

It appears the guy who wrote the article was just saying bad programmers
shouldn't write programs!

On 29/03/2022 15:59, Jim Jackson wrote:
> Not checking the returned status from a function/command call is VERY
> VERY common no matter what language! People who don't check the returned
> status are just bad programmers.
People who do check the return status of void functions are very bad
programmers

--
No Apple devices were knowingly used in the preparation of this post.