On 29.03.2024 21:59, Lawrence D'Oliveiro wrote:
> On Fri, 29 Mar 2024 10:37:22 +0100, Janis Papanagnou wrote:
>
>> Program text is initially text.[*] During parsing (either in
>> an interpreted or in a compiled language) you split the text
>> in tokens.
>
> And then, how do you interpret the tokens?

In an interpreter the tokens are interpreted, in a compiler
they are subject to the parsing. (But you know that I'm sure.)

What I was saying is that there's initially literal program text
that is transformed to tokens in the lexical analysis, and then
further processed.

It was a reply on your original statement which was:
>>> In a shell language, everything you type is assumed to be a
>>> literal string, unless you use special substitution sequences.

> In a shell language, you have
> the assumption that “everything is literal text until indicated
> otherwise”;

Who is that "you"? (Not me, for sure.) And where did you get that
from?

> in a programming language, you have the assumption that
> “everything is a program construct until indicated otherwise”.

So what is 'for i in a ; do ... ; done' then in your world?

This is one of many basic shell constructs that I use in shell
programming (not "shell scripting") regularly.

Janis

On 29.03.2024 19:51, David W. Hodgins wrote:
> The distinctions between script and programming languages made sense when
> they were first introduced. [...]

I abandoned the term "shell script" as soon as I got aware that
writing a reliable piece of software with "scripts" follows the
same engineering techniques and requires the same quality means
as non-"scripts". (Occasionally I still use the term informally
though I don't see any formal, technical, or practical necessity
for any such distinction between Command and Programming Language
as far as _programming_ is concerned.)

Janis

On 30.03.2024 00:14, Janis Papanagnou wrote:
> On 29.03.2024 21:59, Lawrence D'Oliveiro wrote:
>> On Fri, 29 Mar 2024 10:37:22 +0100, Janis Papanagnou wrote:
>>
>>> Program text is initially text.[*] During parsing (either in
>>> an interpreted or in a compiled language) you split the text
>>> in tokens.
>>
>> And then, how do you interpret the tokens?
>
> In an interpreter the tokens are interpreted, in a compiler
> they are subject to the parsing. [...]

Just noticed that this may be misleading. Of course the shell
does also a syntax analysis step and report syntax errors. So
don't get me wrong here.

Janis

On Sat, 30 Mar 2024 00:14:42 +0100, Janis Papanagnou wrote:

> What I was saying is that there's initially literal program text
> that is transformed to tokens in the lexical analysis, and then
> further processed.

In a shell language, that is “further processed” as literal text, except
for any instances of substitution markers.

>> in a programming language, you have the assumption that
>> “everything is a program construct until indicated otherwise”.
>
> So what is 'for i in a ; do ... ; done' then in your world?

“for” is just the name of a command, like any other. In POSIX, this one
happens to be built into the shell; it might not in other shells.

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> On 29.03.2024 21:59, Lawrence D'Oliveiro wrote:
>> On Fri, 29 Mar 2024 10:37:22 +0100, Janis Papanagnou wrote:
>>
>>> Program text is initially text.[*] During parsing (either in
>>> an interpreted or in a compiled language) you split the text
>>> in tokens.
>>
>> And then, how do you interpret the tokens?
>
> In an interpreter the tokens are interpreted, in a compiler
> they are subject to the parsing. (But you know that I'm sure.)

Are you suggesting that interpreters don't parse their input? No doubt
there are interpreters that handle tokens one at a time, but shells
typically don't work that way.

There are a lot of different ways to distinguish between languages like
sh/ksh/bash that are normally interpreted and languages like C that are
normally compiled to machine code:

- Is the language primarily designed for interactive command-line use?
- Is the language normally compiled to machine code?
- Is the language compiled by some kind of byte code that is then
interpreted? Is that byte code *sometimes* translated to machine
code?
- Is foo treated by default as a string or as an identifier?
(Treating it as a string is convenient for interactive use.)
- Does a syntax error on line 10 prevent lines 1-9 from being executed?

I don't think *any* of these distinctions are fundamental.

For example, a Perl or Python program/script is parsed and "compiled"
into some in-memory form before execution begins. A syntax error
prevents the entire script from being executed. In Bourne-line shells,
a syntax error won't be detected until execution reaches that line.
(Things like Perl's BEGIN and eval can complicate things.)

Shells also have to parse their input, though it's done a bit
differently. Typically the entire script isn't parsed as a unit.

[...]

> So what is 'for i in a ; do ... ; done' then in your world?
>
> This is one of many basic shell constructs that I use in shell
> programming (not "shell scripting") regularly.

I use it both in shell scripting/programming and interactively.

And the entire construct needs to be processed before the shell can
begin to execute it. Misspelling "done" prevents the whole thing from
running.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> On 30.03.2024 00:14, Janis Papanagnou wrote:
>> On 29.03.2024 21:59, Lawrence D'Oliveiro wrote:
>>> On Fri, 29 Mar 2024 10:37:22 +0100, Janis Papanagnou wrote:
>>>
>>>> Program text is initially text.[*] During parsing (either in
>>>> an interpreted or in a compiled language) you split the text
>>>> in tokens.
>>>
>>> And then, how do you interpret the tokens?
>>
>> In an interpreter the tokens are interpreted, in a compiler
>> they are subject to the parsing. [...]
>
> Just noticed that this may be misleading. Of course the shell
> does also a syntax analysis step and report syntax errors. So
> don't get me wrong here.

I did indeed get you wrong here (see my other followup).

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On 30.03.2024 01:03, Keith Thompson wrote:
>>
>> Just noticed that this may be misleading. Of course the shell
>> does also a syntax analysis step and report syntax errors. So
>> don't get me wrong here.
>
> I did indeed get you wrong here (see my other followup).

Sorry for the inconvenience!

Janis

Lawrence D'Oliveiro <ldo@nz.invalid> writes:
> On Sat, 30 Mar 2024 00:14:42 +0100, Janis Papanagnou wrote:
>> What I was saying is that there's initially literal program text
>> that is transformed to tokens in the lexical analysis, and then
>> further processed.
>
> In a shell language, that is “further processed” as literal text, except
> for any instances of substitution markers.
>
>>> in a programming language, you have the assumption that
>>> “everything is a program construct until indicated otherwise”.
>>
>> So what is 'for i in a ; do ... ; done' then in your world?
>
> “for” is just the name of a command, like any other. In POSIX, this one
> happens to be built into the shell; it might not in other shells.

"for" is not just a command. It's a keyword, part of the shell language
syntax.

By contrast, "echo" is a command that may be built into the shell or
not, and behaves similarly either way. There's no way to implement
"for" as an external command.

(Very early UNIX shells had "goto" as an external command. The
implementation was convoluted.)

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

ram@zedat.fu-berlin.de (Stefan Ram) writes:

> Johanne Fairchild <jfairchild@tudado.org> wrote or quoted:
>>A scripting language is a programming language made for a hypothetical
>>machine, not too different from a programming language made for a real
>>machine, one made of hardware.
>
> C is clearly a programming language, yet its specification
> says, "The semantic descriptions in this document describe
> the behavior of an abstract machine". And you cannot buy
> this abstract C machine as a piece of hardware anywhere!

Of course. :) But we both know what that means. It's abstract because
there are so many real machines for which this abstract one is an
abstraction of. And the real ones are the target of the language.

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

> On Fri, 29 Mar 2024 08:09:46 -0300, Johanne Fairchild wrote:
>
>> Consider looking at a shell language like a domain-specific programming
>> language. A shell is a programming language made specifically for
>> running programs. ... (Of course, the idea evolves and you want to
>> glue programs, do variable substitution et cetera.)
>
> That’s the thing. The design for a “language made specifically for running
> programs” always seems to start with the assumption that what the user
> types is to be passed as literal text, unless special markers are present
> to indicate that they want to “glue programs, do variable substitution et
> cetera”. Notice your use of the term “variable substitution”, which is
> characteristic of a shell language: in a programming language, you don’t
> call it “substitution”, you call it “evaluation”.

That's right. Substitution is evaluation; a specific form of.

>> You seem to find trouble with using a programming language in a REPL.
>
> I find REPLs annoying and inconvenient. If I want to do “scratchpad”
> programming, I do it in a Jupyter notebook.

That's something to think about. Your perception is wildly different
from a lot of people who have thought and think very deeply about the
whole craft.

Consider your Lisp writing, which violates the culture of Lisp writing.
Of course you should keep your independence, but maybe there are good
reasons why the culture is as it is---not all culture is fashion.

On 30.03.2024 00:51, Lawrence D'Oliveiro wrote:
> On Sat, 30 Mar 2024 00:14:42 +0100, Janis Papanagnou wrote:
>
>> So what is 'for i in a ; do ... ; done' then in your world?
>
> “for” is just the name of a command, like any other. In POSIX, this one
> happens to be built into the shell; it might not in other shells.

'for' is a "reserved word" and part of a "compound command". It's
part of the _shell syntax_! - You get syntax errors like

$ for ; do : ; done
ksh: syntax error: `;' unexpected

And, specifically, "i in a" are *not* arguments of a 'for' command
but also part of the control construct syntax.

But, okay, I see where you're coming from.

Besides the naming, keep in mind that there's a semantical differences
between a "command", a "built-in command", and "shell construct". An
example where you can observe operational differences is:
'/bin/test' and '/bin/['
'test' and '['
'[['
where (for example) the latter does not need quoting.

$ x="Hello world" ; [[ $x == "Hello world" ]] ; echo $?
0
$ x="Hello world" ; [ $x == "Hello world" ] ; echo $?
ksh: [: world: unknown operator
2
$ x="Hello world" ; [ "$x" == "Hello world" ] ; echo $?
0

The distinction is important since there's operational differences
associated with these forms. The same holds for other control constructs
("built-in compound commands"), like 'case', or even simple assignments.

Janis

On 30.03.2024 01:01, Keith Thompson wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>> So what is 'for i in a ; do ... ; done' then in your world?
>>
>> This is one of many basic shell constructs that I use in shell
>> programming (not "shell scripting") regularly.
>
> I use it both in shell scripting/programming and interactively.

So do I. (To me there's no significant difference. Only that larger
projects I do not start to develop in interactive mode, of course.)

> And the entire construct needs to be processed before the shell can
> begin to execute it. Misspelling "done" prevents the whole thing from
> running.

Indeed.

From the other posters statements I got the impression that he may
think that control constructs is what makes the difference (between
scripting and programming, or, command interpreters and programming
languages; still not sure what he thinks). In a later post I read
it as if the naming of e.g. 'for' as a "command" leads to his view.
Anyway.

I can see a point where people use for interactive use other shells
than for programming; like tcsh (interactively) and bash (programming),
because of the powerful features tcsh supports. Since the increase of
interactive features supported by the shells that are typically used
for programming I prefer to have the same shell with same syntax and
features for both, and to be able to pass code from one application
context to the other.

Janis

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
[...]
> I can see a point where people use for interactive use other shells
> than for programming; like tcsh (interactively) and bash (programming),
> because of the powerful features tcsh supports. Since the increase of
> interactive features supported by the shells that are typically used
> for programming I prefer to have the same shell with same syntax and
> features for both, and to be able to pass code from one application
> context to the other.

Indeed. I spent several years using tcsh interactively (because it had
a few interactive features that I found convenient) and bash for writing
scripts (previously I had actually written *gasp* csh and tcsh scripts).
Eventually I found that using a single shell for both was easier, and
that bash's interactive features are about as good as tcsh's. (And I
can write complex nested commands on one line, something I probably
wouldn't have attempted in [t]csh even if it were possible.) I haven't
looked back.

zsh has some nice features, but I haven't learned it well enough to
consider switching.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On Fri, 29 Mar 2024 21:24:01 -0300, Johanne Fairchild wrote:

> Consider your Lisp writing, which violates the culture of Lisp writing.
> Of course you should keep your independence, but maybe there are good
> reasons why the culture is as it is---not all culture is fashion.

Let me know what they are. The only reasons I have seen offered so far are
basically just “that’s the way it’s always been done”.

On Fri, 29 Mar 2024 17:08:35 -0700, Keith Thompson wrote:

> "for" is not just a command. It's a keyword, part of the shell language
> syntax.

Remember, that’s only true of POSIX shells.

Lawrence D'Oliveiro <ldo@nz.invalid> writes:
> On Fri, 29 Mar 2024 17:08:35 -0700, Keith Thompson wrote:
>
>> "for" is not just a command. It's a keyword, part of the shell language
>> syntax.
>
> Remember, that’s only true of POSIX shells.

It may well be true of some non-POSIX shells as well (not that it
matters).

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On Sat, 30 Mar 2024 01:30:53 +0100, Janis Papanagnou wrote:

> Besides the naming, keep in mind that there's a semantical differences
> between a "command", a "built-in command", and "shell construct". An
> example where you can observe operational differences is:
> '/bin/test' and '/bin/['
> 'test' and '['
> '[['
> where (for example) the latter does not need quoting.

I know that there are places in POSIX shells where the usual rules for
interpretation of special-substitution markers are altered or suspended.
But those situations are always delimited in some special way (distinctive
syntactic constructs or reserved words). Throughout the entire rest of the
language, the principles I have described still apply.

Lawrence D'Oliveiro <ldo@nz.invalid> writes:
>On Fri, 29 Mar 2024 17:08:35 -0700, Keith Thompson wrote:
>
>> "for" is not just a command. It's a keyword, part of the shell language
>> syntax.
>
>Remember, that’s only true of POSIX shells

Not necessarily. Bash is not a POSIX shell. It can be,
but without special configuration it is not POSIX.

ksh is as close as one might get to a true POSIX shell,
but even ksh has extensions.

But nitpicking aside, the context of the discussion that
you responded to was about clearly POSIX-like shells.

On Fri, 29 Mar 2024 15:38:48 -0400, Johanne Fairchild <jfairchild@tudado.org> wrote:
> "David W. Hodgins" <dwhodgins@nomail.afraid.org> writes:
>
> [...]
>
>> The terminology will continue to be used, but the distinction does not
>> matter, except from a speed of processing difference.
>
> Just to share that I, personally, don't use the distinction. For
> instance, I say that
>
> "the answer is %.2f\n"
>
> is a program that builds a string given its usual context. I say that
>
> awk '1; { print "" }'
>
> is a program to double-space a file. I haven't said ``script'' in
> years.

I still refer to text files, intended to be run through an interpreter such
as bash, as scripts, while things like c text files that must be compiled to
an object file and then linked to be executable as programs.

Regards, Dave Hodgins

Johanne Fairchild <jfairchild@tudado.org> wrote or quoted:
>ram@zedat.fu-berlin.de (Stefan Ram) writes:
>>Johanne Fairchild <jfairchild@tudado.org> wrote or quoted:
>>>A scripting language is a programming language made for a hypothetical
>>>machine, not too different from a programming language made for a real
>>>machine, one made of hardware.
>>C is clearly a programming language, yet its specification
>>says, "The semantic descriptions in this document describe
>>the behavior of an abstract machine". And you cannot buy
>> this abstract C machine as a piece of hardware anywhere!
>Of course. :) But we both know what that means. It's abstract because
>there are so many real machines for which this abstract one is an
>abstraction of. And the real ones are the target of the language.

If you want to see it this way ...

But look at Pascal, Java, or Python. They are usually compiled
into an intermediate code (called "p-code" in the case of
Pascal) which is then interpreted (the interpreter is called
"JVM" in the case of Java). Yet, we think of Pascal and Java
as programming languages and of Python as a scripting language.

But this is actually an implementation detail: Java also can
be compiled into machine code.

In any case, we can write a small batch file "execute" which
can be called for source code in any programming language and
will execute it:

execute Main.pas
execute Main.java
execute Main.py
execute main.c
execute main.cpp
execute main.bas
execute main.bat
....

They all will print "Hello World". Whether the execution happens
via translation to machine code or via interpretation by another
program or by a mixture of both is just an implementation
detail of "execute", that usually will not matter much for the
programmer. And often for the same language, one is free to
either compile it to machine language or interpret it via another
program.

Some language, like LISP or Python, have "eval": These languages
still can be compiled, but they require an interpreter at run-
time. Java often is executed by interpretation first, but when
the "Hotspot" interpreter sees that some code is executed often,
it will then decide /at run-time/ to compile it into actual
machine code! (And there are Python implementations that run
on the JVM.)

How can such implementation details matter for the question whether
a language is called a programming language or a scripting language,
when the programmer often does not even need to know about them?

Yes, there also are C interpreters IIRC, but they are rare.

On Fri, 29 Mar 2024 17:25:18 -0000 (UTC)
Kaz Kylheku <643-408-1753@kylheku.com> wrote:
>On 2024-03-29, Muttley@dastardlyhq.com <Muttley@dastardlyhq.com> wrote:
>>>Scripting is an activity, a use case, not a language.
>>
>> So if I write a program to for example process some files in a directory by
>> your argument its a script whether I write it in shell, python, C++ or
>> assembler.
>
>I also wrote: "Scripting refers to executing commands which are so high
>level that they are entire applications or functional blocks within an
>application."

So if I write:

int main()
{ system("ls | wc -l");
return 0;
}

Thats a script? No? What if I use popen() or execve() then? Where do you
draw the line?

>You're just being deliberately obtuse, not to mention snippy with the
>scissors.

I'm not being obtuse. There is no hard dividing line between scripts and
programs - as I said, its shades of grey.

On Fri, 29 Mar 2024 17:58:41 -0000 (UTC)
Kaz Kylheku <643-408-1753@kylheku.com> wrote:
>On 2024-03-29, Muttley@dastardlyhq.com <Muttley@dastardlyhq.com> wrote:
>> Were the mucky bits actually written in Lisp or was Lisp simply calling some
>> routines written in assembler?
>
>Sorry, could you demarcate where exactly the goalposts are? Which mucky
>bits?

Oh I dunno, the parts that walk a kernel memory structure for example.

>In kernels written in C, there are mucky bits in assembler, like
>entry and exit into an trap/interrupt handler. You usually can't save

Sure, but far less that there would be in a higher level language.

On 30.03.2024 08:47, Stefan Ram wrote:
>
> But look at Pascal, Java, or Python. They are usually compiled
> into an intermediate code (called "p-code" in the case of
> Pascal) which is then interpreted (the interpreter is called
> "JVM" in the case of Java). Yet, we think of Pascal and Java
> as programming languages and of Python as a scripting language.

I never used an interpreted Pascal, nor one that produced p-code.
(As far as memory serves it was the UCSD Pascal dialect that
decided to use intermediate code to be interpreted.) My Pascal
programs have always been compiled.

And there were BASIC compilers on mainframes before the BASIC
interpreters on PCs became popular and widespread.

It's _the same_ language (modulo dialects), and languages are
usually defined by their grammar and semantics and not whether
it is interpreted/compiled or how it is run (VM or else).

I think it's obvious that interpretation vs. compilation or any
intermediate interpreted p-code is not an appropriate criterion
to declare something as a "scripting" language. You can't tell
whether a language, Pascal, BASIC, or any other language, is a
"scripting" language by that criterion. It's also generally a
fuzzy term; literature speaks vaguely about "typical" criteria
but cannot pin them down. - And that term isn't even helpful in
any way! - So why use it at all or religiously dispute about it.

Janis

>
> But this is actually an implementation detail: Java also can
> be compiled into machine code.
>
> [...]

On 30.03.2024 11:16, Muttley@dastardlyhq.com wrote:
>
> I'm not being obtuse. There is no hard dividing line between scripts and
> programs - as I said, its shades of grey.

The terms "script" and "scripting languages" are very fuzzy and
just trying to be descriptive, but are effectively meaningless,
the terms don't serve any practical purpose (in my book).

Janis

On 2024-03-29 02:14, Lawrence D'Oliveiro wrote:
> At one time, we distinguished between “scripting” languages and
> “programming” languages. To begin with, the “scripting” languages were
> somehow more limited in functionality than full-fledged “programming”
> languages. Or they were slower, because they were interpreted.
[...]

The key difference is that a program in a scripting language need not to
be complete or known in order to be executed.

The limitations and ugliness of scripting languages is determined by
this requirement, but also easiness of use.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de