See the following example bash script:

```
#!/usr/bin/env bash

sudo apt install -y pkg_1 pkg_2 ... pkg_N \
# There are some comments here.
pkg_N+1 ...
```
The above script will fail due to the there are comments on the next line of the line continuation symbol (\). Is there a workaround/trick which can let me write bash script this way?

Regards,
HZ

In article <8f390ffa-4f1e-42e0-9d31-16ce30d8fda9n@googlegroups.com>,
hongy...@gmail.com <hongyi.zhao@gmail.com> wrote:
>See the following example bash script:
>
>```
>#!/usr/bin/env bash
>
>sudo apt install -y pkg_1 pkg_2 ... pkg_N \
> # There are some comments here.
> pkg_N+1 ...
>```
>The above script will fail due to the there are comments on the next
>line of the line continuation symbol (\). Is there a workaround/trick
>which can let me write bash script this way?

Yes. Don't do that.

--
"Women should not be enlightened or educated in any way. They should be
segregated because they are the cause of unholy erections in holy men.

-- Saint Augustine (354-430) --

"hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:

> See the following example bash script:
>
> ```
> #!/usr/bin/env bash
>
> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
> # There are some comments here.
> pkg_N+1 ...
> ```
> The above script will fail due to the there are comments on the next
> line of the line continuation symbol (\). Is there a workaround/trick
> which can let me write bash script this way?

Backslash the end of line on the comment.

--
Dan Espen

"hongy...@gmail.com" <hongyi.zhao@gmail.com>:
>See the following example bash script:
>

>#!/usr/bin/env bash
>
>sudo apt install -y pkg_1 pkg_2 ... pkg_N \
> # There are some comments here.
> pkg_N+1 ...

>The above script will fail due to the there are comments on the
>next line of the line continuation symbol (\). Is there a
>workaround/trick which can let me write bash script this way?

The backslash at the end of the line (also known as line
continuation) will tell the shell to remove it and the line feed
following it prior to further parsing the command line.  That is,
the shell parser will see

sudo apt ... pkg_N # There are some comments here.
pkg_N+1 ...

You could do

(
set -- sudo apt install -y pkg_1 pkg_2 ... pkg_N &&
# There are some comments here.
set -- "$@" pkg_N+1 ... &&
exec "$@"
)

though I don't know if it's worth while doing that.  I would place
the comment before the "sudo" invocation.

Note the difference:  The "&&" operator is not disturbed by a
(sequence of) following newline characters, including comment lines.

Dan Espen <dan1espen@gmail.com> writes:
> "hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:
>
>> See the following example bash script:
>>
>> ```
>> #!/usr/bin/env bash
>>
>> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
>> # There are some comments here.
>> pkg_N+1 ...
>> ```
>> The above script will fail due to the there are comments on the next
>> line of the line continuation symbol (\). Is there a workaround/trick
>> which can let me write bash script this way?
>
> Backslash the end of line on the comment.

I wasn't able to get that to work. Were you?

$ cat zero.bash
#!/bin/bash

echo foo \
bar
$ ./zero.bash
foo bar
$ cat one.bash
#!/bin/bash

echo foo \
# comment
bar
$ ./one.bash
foo
../one.bash: line 5: bar: command not found
$ cat two.bash
#!/bin/bash

echo foo \
# comment \
bar
$ ./two.bash
foo
../two.bash: line 5: bar: command not found
$

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Philips
void Void(void) { Void(); } /* The recursive call of the void */

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:

> Dan Espen <dan1espen@gmail.com> writes:
>> "hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:
>>
>>> See the following example bash script:
>>>
>>> ```
>>> #!/usr/bin/env bash
>>>
>>> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
>>> # There are some comments here.
>>> pkg_N+1 ...
>>> ```
>>> The above script will fail due to the there are comments on the next
>>> line of the line continuation symbol (\). Is there a workaround/trick
>>> which can let me write bash script this way?
>>
>> Backslash the end of line on the comment.
>
> I wasn't able to get that to work. Were you?

Nope, my mistake.

--
Dan Espen

Dan Espen <dan1espen@gmail.com>:
> "hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:
>
>> See the following example bash script:
>>
>> ```
>> #!/usr/bin/env bash
>>
>> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
>> # There are some comments here.
>> pkg_N+1 ...
>> ```
>> The above script will fail due to the there are comments on the
>> next line of the line continuation symbol (\). Is there a
>> workaround/trick which can let me write bash script this way?
>
> Backslash the end of line on the comment.
>

I don't think so.

POSIX
(<https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_03>)
says:

“If the current character is a '#', it and all subsequent
characters up to, but excluding, the next <newline> shall
be discarded as a comment. The <newline> that ends the
line is not considered part of the comment.”

=> The backslash character at the end of the line on the comment
will be treated and discarded as part of the comment.  No line
continuation to the next line will occur.  That won't join
"pkg_N+1" to the argument list ending with "pkg_N".

Helmut Waitzmann <nn.throttle@xoxy.net> writes:

> Dan Espen <dan1espen@gmail.com>:
>> "hongy...@gmail.com" <hongyi.zhao@gmail.com> writes:
>>
>>> See the following example bash script:
>>> ```
>>> #!/usr/bin/env bash
>>>
>>> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
>>> # There are some comments here.
>>> pkg_N+1 ...
>>> ```
>>> The above script will fail due to the there are comments on the
>>> next line of the line continuation symbol (\). Is there a
>>> workaround/trick which can let me write bash script this way?
>>
>> Backslash the end of line on the comment.
>
> I don't think so.
>
> POSIX
> (<https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_03>)
> says:
>
> “If the current character is a '#', it and all subsequent
> characters up to, but excluding, the next <newline> shall
> be discarded as a comment. The <newline> that ends the
> line is not considered part of the comment.”
>
> => The backslash character at the end of the line on the comment will
> be treated and discarded as part of the comment.  No line
> continuation to the next line will occur.  That won't join
> "pkg_N+1" to the argument list ending with "pkg_N".

Agree.

If I had to have those comments, I suppose this would work:

a='pkg_1'
# Here's that comment...
b='pkg_2'
sudo ... "$a" "$b"

--
Dan Espen

On Sunday, December 26, 2021 at 11:26:44 AM UTC+8, Dan Espen wrote:
> Helmut Waitzmann <nn.th...@xoxy.net> writes:
>
> > Dan Espen <dan1...@gmail.com>:
> >> "hongy...@gmail.com" <hongy...@gmail.com> writes:
> >>
> >>> See the following example bash script:
> >>> ```
> >>> #!/usr/bin/env bash
> >>>
> >>> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
> >>> # There are some comments here.
> >>> pkg_N+1 ...
> >>> ```
> >>> The above script will fail due to the there are comments on the
> >>> next line of the line continuation symbol (\). Is there a
> >>> workaround/trick which can let me write bash script this way?
> >>
> >> Backslash the end of line on the comment.
> >
> > I don't think so.
> >
> > POSIX
> > (<https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_03>)
> > says:
> >
> > “If the current character is a '#', it and all subsequent
> > characters up to, but excluding, the next <newline> shall
> > be discarded as a comment. The <newline> that ends the
> > line is not considered part of the comment.”
> >
> > => The backslash character at the end of the line on the comment will
> > be treated and discarded as part of the comment. No line
> > continuation to the next line will occur. That won't join
> > "pkg_N+1" to the argument list ending with "pkg_N".
> Agree.
>
> If I had to have those comments, I suppose this would work:
>
> a='pkg_1'
> # Here's that comment...
> b='pkg_2'
> sudo ... "$a" "$b"

Thank you and all others for the explanation given here. I figured out the following solution based on your above suggestion:

------------------= begin =--------------
#!/usr/bin/env bash

# Here's that comment for packages related to a.
a="pkga_1 pkga_2 ... pkga_n \
pkga_n+1"
# Here's that comment for packages related to b.
b="pkgb_1 pkgb_2 ... pkgb_n \
pkgb_n+1"

sudo apt install -y $a $b
------------------= end =--------------

HZ

On Sunday, December 26, 2021 at 12:53:39 PM UTC+8, hongy...@gmail.com wrote:
> On Sunday, December 26, 2021 at 11:26:44 AM UTC+8, Dan Espen wrote:
> > Helmut Waitzmann <nn.th...@xoxy.net> writes:
> >
> > > Dan Espen <dan1...@gmail.com>:
> > >> "hongy...@gmail.com" <hongy...@gmail.com> writes:
> > >>
> > >>> See the following example bash script:
> > >>> ```
> > >>> #!/usr/bin/env bash
> > >>>
> > >>> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
> > >>> # There are some comments here.
> > >>> pkg_N+1 ...
> > >>> ```
> > >>> The above script will fail due to the there are comments on the
> > >>> next line of the line continuation symbol (\). Is there a
> > >>> workaround/trick which can let me write bash script this way?
> > >>
> > >> Backslash the end of line on the comment.
> > >
> > > I don't think so.
> > >
> > > POSIX
> > > (<https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_03>)
> > > says:
> > >
> > > “If the current character is a '#', it and all subsequent
> > > characters up to, but excluding, the next <newline> shall
> > > be discarded as a comment. The <newline> that ends the
> > > line is not considered part of the comment.”
> > >
> > > => The backslash character at the end of the line on the comment will
> > > be treated and discarded as part of the comment. No line
> > > continuation to the next line will occur. That won't join
> > > "pkg_N+1" to the argument list ending with "pkg_N".
> > Agree.
> >
> > If I had to have those comments, I suppose this would work:
> >
> > a='pkg_1'
> > # Here's that comment...
> > b='pkg_2'
> > sudo ... "$a" "$b"
> Thank you and all others for the explanation given here. I figured out the following solution based on your above suggestion:
>
> ------------------= begin =--------------
> #!/usr/bin/env bash
>
> # Here's that comment for packages related to a.
> a="pkga_1 pkga_2 ... pkga_n \
> pkga_n+1"
> # Here's that comment for packages related to b.
> b="pkgb_1 pkgb_2 ... pkgb_n \
> pkgb_n+1"
>
> sudo apt install -y $a $b

The following form is also valid:

sudo apt install -y $a \
$b

> ------------------= end =--------------
>
> HZ

"hongy...@gmail.com" <hongyi.zhao@gmail.com>:
>Thank you and all others for the explanation given here. I figured
>out the following solution based on your above suggestion:
>
>------------------= begin =--------------
>#!/usr/bin/env bash
>
># Here's that comment for packages related to a.
>a="pkga_1 pkga_2 ... pkga_n \
> pkga_n+1"
># Here's that comment for packages related to b.
>b="pkgb_1 pkgb_2 ... pkgb_n \
> pkgb_n+1"
>
>sudo apt install -y $a $b
>------------------= end =--------------

It only works if neither of the package names contains white space.

To be not dependent on that condition I used in
<83h7awirdl.fsf@helmutwaitzmann.news.arcor.de> the array‐like
positional parameter list "@" of the shell to set up the command to
be invoked.

Imagine, what would happen if the names of the packages contained
white space like

"pkg a 1", "pkg a 2", … "pkg a n+1",
"pkg b 1", "pkg b 2", … "pkg b n+1".

Then your example would invoke the command

sudo apt install -y pkg a 1 pkg a 2 … pkg a n+1 … \
pkg b 1 pkg b 2 … pkg b n+1

(i. e. the package names would have been split at their internal
blank characters) rather than the intended command

sudo apt install -y 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
'pkg b 1' 'pkg b 2' … 'pkg b n+1'

because of the reference to the unquoted variables $a and $b.

If one the other hand I do

(
set -- sudo apt install -y &&
# Here's that comment for packages related to a.
set -- "$@" 'pkg a 1' 'pkg a 2' … 'pkg a n+1' &&
# Here's that comment for packages related to b.
set -- "$@" 'pkg b 1' 'pkg b 2' … 'pkg b n+1' &&
exec "$@"
)

then the command invoked will effectively be

sudo apt install -y 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
'pkg b 1' 'pkg b 2' … 'pkg b n+1'

as intended.

On Sunday, December 26, 2021 at 2:59:36 PM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongyi.zhao@gmail.com>:
> >Thank you and all others for the explanation given here. I figured
> >out the following solution based on your above suggestion:
> >
> >------------------= begin =--------------
> >#!/usr/bin/env bash
> >
> ># Here's that comment for packages related to a.
> >a="pkga_1 pkga_2 ... pkga_n \
> > pkga_n+1"
> ># Here's that comment for packages related to b.
> >b="pkgb_1 pkgb_2 ... pkgb_n \
> > pkgb_n+1"
> >
> >sudo apt install -y $a $b
> >------------------= end =--------------
> It only works if neither of the package names contains white space.

For Debian derivative systems, the package name specification is described here [1]:

--------
This manual page documents the dpkg-name program which provides an easy way to rename Debian packages into their full package names. A full package name consists of package_version_architecture.package-type as specified in the control file of the package. The version part of the filename consists of the upstream version information optionally followed by a hyphen and the revision information. The package-type part comes from that field if present or fallbacks to deb.
--------

As you can see, the package names don't contain white space. As for other operating systems, I'm not sure, but I believe it's not a good practice to use whitespace in the naming of software packages on a system.

[1] https://manpages.debian.org/bullseye/dpkg-dev/dpkg-name.1.en.html

> To be not dependent on that condition I used in
> <83h7awirdl.fsf@helmutwaitzmann.news.arcor.de> the array‐like
> positional parameter list "@" of the shell to set up the command to
> be invoked.
>
> Imagine, what would happen if the names of the packages contained
> white space like
>
> "pkg a 1", "pkg a 2", … "pkg a n+1",
> "pkg b 1", "pkg b 2", … "pkg b n+1".
>
>
> Then your example would invoke the command
>
>
> sudo apt install -y pkg a 1 pkg a 2 … pkg a n+1 … \
> pkg b 1 pkg b 2 … pkg b n+1
>
> (i. e. the package names would have been split at their internal
> blank characters) rather than the intended command
>
> sudo apt install -y 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
> 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
>
> because of the reference to the unquoted variables $a and $b.
>
>
> If one the other hand I do
>
>
> (
> set -- sudo apt install -y &&
> # Here's that comment for packages related to a.
> set -- "$@" 'pkg a 1' 'pkg a 2' … 'pkg a n+1' &&
> # Here's that comment for packages related to b.
> set -- "$@" 'pkg b 1' 'pkg b 2' … 'pkg b n+1' &&
> exec "$@"
> )
>
> then the command invoked will effectively be
>
>
> sudo apt install -y 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
> 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
>
> as intended.

Thank you for pointing this out.

HZ

On 2021-12-25, hongy...@gmail.com wrote:
> See the following example bash script:
>
> ```
> #!/usr/bin/env bash
>
> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
> # There are some comments here.
> pkg_N+1 ...
> ```
> The above script will fail due to the there are comments on the next line of the line continuation symbol (\). Is there a workaround/trick which can let me write bash script this way?
>

I kinda like this style:

pkgs=(
# This is package foo
pkg_1
# This is package bar
pkg_2
pkg_3
# This is package baz
pkg_4
pkg_5 # Added on 19-Jan-22
)

apt install ${pkgs[@]}

Tavis.

--
_o) $ lynx lock.cmpxchg8b.com
/\\ _o) _o) $ finger taviso@sdf.org
_\_V _( ) _( ) @taviso

On 26.12.2021 18:19, Tavis Ormandy wrote:
> On 2021-12-25, hongy...@gmail.com wrote:
>> See the following example bash script:
>>
>> ```
>> #!/usr/bin/env bash
>>
>> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
>> # There are some comments here.
>> pkg_N+1 ...
>> ```
>> The above script will fail due to the there are comments on the next line of the line continuation symbol (\). Is there a workaround/trick which can let me write bash script this way?
>>
>
> I kinda like this style:
>
> pkgs=(
> # This is package foo
> pkg_1
> # This is package bar
> pkg_2
> pkg_3
> # This is package baz
> pkg_4
> pkg_5 # Added on 19-Jan-22
> )
>
> apt install ${pkgs[@]}

As already suggested upthread; quote that expression

apt install "${pkgs[@]}"

Janis

> Tavis.
>

Short answer, no.

(A) A backslash at the end of a line does not, to the shell, designate
a "new line". The next line is a continuation of the current line,
as though the newline character(s) simply wasn't there.

(B) A comment character is absolute -- anything on a line after the
comment character is ignored by the shell.

--hymie!

In our last episode, the evil Dr. Lacto had captured our hero,
hongy...@gmail.com <hongyi.zhao@gmail.com>, who said:
> See the following example bash script:
>
> ```
> #!/usr/bin/env bash
>
> sudo apt install -y pkg_1 pkg_2 ... pkg_N \
> # There are some comments here.
> pkg_N+1 ...
> ```
> The above script will fail due to the there are comments on the next line of the line continuation symbol (\). Is there a workaround/trick which can let me write bash script this way?
>
> Regards,
> HZ

"hongy...@gmail.com" <hongyi.zhao@gmail.com>:
> On Sunday, December 26, 2021 at 2:59:36 PM UTC+8, Helmut Waitzmann
> wrote:
>> It only works if neither of the package names contains white
>> space.
>
> For Debian derivative systems, the package name specification is
> described here [1]:
>
> --------
> This manual page documents the dpkg-name program which provides an
> easy way to rename Debian packages into their full package names. A
> full package name consists of
> package_version_architecture.package-type as specified in the
> control file of the package. The version part of the filename
> consists of the upstream version information optionally followed by
> a hyphen and the revision information. The package-type part comes
> from that field if present or fallbacks to deb.
> --------
>
> [1]
> https://manpages.debian.org/bullseye/dpkg-dev/dpkg-name.1.en.html
>
> As you can see, the package names don't contain white space.
>

As far as I understand, the cited text does not say anything about
the package, version, architecture, and package-type components of a
full package name other than that they are specified in the control
file (of unknown contents) of the package, and that the version part
is a concatenation of the upstream version information (of unknown
contents) followed by a hyphen and the revision information (of
unknown contents).  And finally these parts are concatenated by
underscores.

If any of these parts contains some white space, then that white
space will be in the full package name, as well.

> As for other operating systems, I'm not sure, but I believe it's
> not a good practice to use whitespace in the naming of software
> packages on a system.

I agree, but I believe, it's good practice to carefully design
software (here: the shell script) to be as robust as possible.

The best practice with regard to shell programming:  Don't ever
misuse a (non‐array) shell variable v as an array variable by merely
gluing together the array elements with white space and then using
the value of the variable by unquoted references ($v) to have it be
split at white space.

With Bash and other modern shells you have got even true array
variables.  So you could use them in this case.  (But as you might
know, I prefer (if at all possible) to use solutions which work with
any shell conforming to the POSIX standard.)

What's the cause of the log4shell exploit and many other exploits? 
It's always that disastrous "Strange variable values? – That won't
happen!" ignorance.

On Tuesday, December 28, 2021 at 8:55:13 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
> > On Sunday, December 26, 2021 at 2:59:36 PM UTC+8, Helmut Waitzmann
> > wrote:
> >> It only works if neither of the package names contains white
> >> space.
> >
> > For Debian derivative systems, the package name specification is
> > described here [1]:
> >
> > --------
> > This manual page documents the dpkg-name program which provides an
> > easy way to rename Debian packages into their full package names. A
> > full package name consists of
> > package_version_architecture.package-type as specified in the
> > control file of the package. The version part of the filename
> > consists of the upstream version information optionally followed by
> > a hyphen and the revision information. The package-type part comes
> > from that field if present or fallbacks to deb.
> > --------
> >
> > [1]
> > https://manpages.debian.org/bullseye/dpkg-dev/dpkg-name.1.en.html
> >
> > As you can see, the package names don't contain white space.
> >
> As far as I understand, the cited text does not say anything about
> the package, version, architecture, and package-type components of a
> full package name other than that they are specified in the control
> file (of unknown contents) of the package, and that the version part
> is a concatenation of the upstream version information (of unknown
> contents) followed by a hyphen and the revision information (of
> unknown contents). And finally these parts are concatenated by
> underscores.
>
> If any of these parts contains some white space, then that white
> space will be in the full package name, as well.

Sorry. I should have shown you, but I haven't found it till now, the following description given in "Debian RFC822 control data format" [1]:

I should have shown you, but I didn't find the following description given in "Debian RFC822 Control Data Formats" until now:

$ man deb822 | egrep -A3 -i 'whitespace must not'
Whitespace must not appear inside names (of packages, architectures,
files or anything else) or version numbers, or between the characters of
multi-character version relationships.

[1] https://man7.org/linux/man-pages/man5/deb822.5.html

> > As for other operating systems, I'm not sure, but I believe it's
> > not a good practice to use whitespace in the naming of software
> > packages on a system.
> I agree, but I believe, it's good practice to carefully design

Whether you should use the following wording:

.... it's a good practice ...

> software (here: the shell script) to be as robust as possible.
>
> The best practice with regard to shell programming: Don't ever
> misuse a (non‐array) shell variable v as an array variable by merely
> gluing together the array elements with white space and then using
> the value of the variable by unquoted references ($v) to have it be
> split at white space.
>
> With Bash and other modern shells you have got even true array
> variables. So you could use them in this case. (But as you might
> know, I prefer (if at all possible) to use solutions which work with
> any shell conforming to the POSIX standard.)

But this may lead to the loss of some convenience. There is no best of both worlds in the world.
> What's the cause of the log4shell exploit and many other exploits?
> It's always that disastrous "Strange variable values? – That won't
> happen!" ignorance.

"hongy...@gmail.com" <hongyi.zhao@gmail.com>:
> On Tuesday, December 28, 2021 at 8:55:13 AM UTC+8, Helmut Waitzmann wrote:
>> "hongy...@gmail.com" <hongy...@gmail.com>:
>> As far as I understand, the cited text does not say anything
>> about the package, version, architecture, and package-type
>> components of a full package name other than that they are
>> specified in the control file (of unknown contents) of the
>> package, and that the version part is a concatenation of the
>> upstream version information (of unknown contents) followed by a
>> hyphen and the revision information (of unknown contents). And
>> finally these parts are concatenated by underscores.
>>
>> If any of these parts contains some white space, then that white
>> space will be in the full package name, as well.
>
> Sorry. I should have shown you, but I haven't found it till now,
> the following description given in "Debian RFC822 control data
> format" [1]:
>
> I should have shown you, but I didn't find the following
> description given in "Debian RFC822 Control Data Formats" until
> now:
>
> $ man deb822 | egrep -A3 -i 'whitespace must not'
> Whitespace must not appear inside names (of packages,
> architectures, files or anything else) or version numbers,
> or between the characters of multi-character version
> relationships.
>
> [1] https://man7.org/linux/man-pages/man5/deb822.5.html
>

Thank you.  So we are lucky, and in this case it's not necessary to
do what is described below.

>> The best practice with regard to shell programming: Don't ever
>> misuse a (non‐array) shell variable v as an array variable by
>> merely gluing together the array elements with white space and
>> then using the value of the variable by unquoted references ($v)
>> to have it be split at white space.
>>
>> With Bash and other modern shells you have got even true array
>> variables. So you could use them in this case. (But as you might
>> know, I prefer (if at all possible) to use solutions which work
>> with any shell conforming to the POSIX standard.)
>
> But this may lead to the loss of some convenience. There is no best
> of both worlds in the world.

Compared with the disatrous consequences of an exploit I'd clearly
prefer array variables with Bash and other modern shells.

And even if I want to stay compatible with the POSIX standard, I
consider an exploit far more inconvenient than the increased
complexity in programming for the shell.

Simulating an array variable using the POSIX standard could be done
like this:

Define a shell function "quote_words" (see below), which, when
given zero or more invocation arguments, computes (a part of) a
shell command line (of a simple command) resembling a command line
to invoke those arguments and writes it to standard output, where
it can be put in a simple shell variable by command substitution,
like this:

v="$(quote_words printf '%s\n' 'Hello, world!')"

It can then invoked using the "eval" shell builtin command:

eval "$v"

Try the following command to see, what that is.  The commands

v="$(quote_words printf '%s\n' 'Hello, world!')" &&
printf '%s\n' "$v" &&
eval "$v"

will output

'printf' '%s\n' 'Hello, world!'
Hello, world!

Now, this content of the variable v, which is a (part of a) shell
command line of a simple command, can even be glued together with
more parts of a command line by putting a blank between each of the
command line parts to be glued together and stored in a simple
shell variable.  Finally, this shell variable can be fed to the
"eval" shell builtin command to let it be invoked as a command
line.

In the example of the OP this would look like these shell commands:

v="$(quote_words sudo apt install -y)" &&

# Here's that comment for packages related to a.
#
v="$v $(quote_words 'pkg a 1' 'pkg a 2' … 'pkg a n+1')" &&

# Here's that comment for packages related to b.
#
v="$v $(quote_words 'pkg b 1' 'pkg b 2' … 'pkg b n+1')" &&

# Finally, let the "eval" command parse and invoke it:
#
eval "$v" &&

That quote_words function might look like this:

### The function quote_words starts here.
quote_words()
(
# Outputs a (part of) a command line resembling a 'simple
# command' to be used by a POSIX-compliant shell.
#
# Usage:
#
# quote_words words to be quoted...
#
# exit code:
# 0, if the command line has been successfully constructed,
# !=0, if the function failed to construct the command line.
#
# Examples:
#
# Put a word list resembling a command invocation into a command
# line:
#
# if command_line="$(
# quote_words program with parameters
# )"
# then
# # "eval" it:
# eval " $command_line"
#
# # or pass it to a new shell:
# sh -c -- "$command_line" sh
#
# # or pass it to "su":
# su -- - a_user -c -- "$command_line" -su
#
# # or use it remotely:
# ssh user@remote.host.example " $command_line"
#
# else
# # failed to compute the command line.
# fi
#
#
# Store the shell positional parameters in a variable and
# restore them later:
#
# if args="$(quote_words "$@")"
# then
# # the positional parameters may be changed for any purpose
# # ...
#
# # restore the positional parameters:
#
# eval "set -- $args"
# else
# # failed to save the positional parameters.
# fi

# "wordsep" contains a separator used for the list of the quoted
# words. The first need not to be preceded by a separater:

wordsep for word
do
# "$wordsep" separates each word (except the first one) from
# its precedessor:
printf '%s' "$wordsep"
if test -z "$word"
then
# The word is empty. Then the result is "''":
printf '%s' "''"
else
# The word is not empty.
while
{
prefix="${word%%\'*}"
word="${word#"$prefix"}"
# "$prefix" is the longest prefix of "$word", that does
# not contain any apostrophes; it is removed from "$word".
# Conclusion: "$word" is either empty or starts with an
# apostrophe.
if test -n "$prefix"
then
# "$prefix" consists of one or more characters. Put
# them in between of two apostrophes:
printf '%s' \'"${prefix}"\'
fi
test -n "$word" &&
{
# "$word" is not empty. Conclusion: "$word" starts with
# one or more apostrophes.
apostr="${word%%[!\']*}"
# "$apostr" ist the longest beginning part of "$word",
# that contains apostrophes only.
if test -n "${apostr#"'"}"
then
# There are at least 2 apostrophes: Put it in between
# of two double quotes:
printf '"%s"' "${apostr}"
else
# There is 1 apostrophe: Precede it by a backslash:
printf '\\%s' "${apostr}"
fi
# Remove the beginning apostrophes from "$word":
word="${word#"$apostr"}"
# If nothing is left, we are done:
${word:+:} false
}
}
do
:
done
fi
# All following words (except the first one) have to be
# separated from their precedessor with a blank:
wordsep=' '
done
printf '\n'
)
### The function quote_words ends here.

Of course I won't want to write this function from scratch in every
shell script where it is to be used.  Therefore I put it in a file,
say "~/shell-lib/quote_words.sh".  Then, if I like to use it in a
shell script, I just do

On Wednesday, December 29, 2021 at 6:42:40 AM UTC+8, Helmut Waitzmann wrote:
> "hongy...@gmail.com" <hongy...@gmail.com>:
> > On Tuesday, December 28, 2021 at 8:55:13 AM UTC+8, Helmut Waitzmann wrote:
> >> "hongy...@gmail.com" <hongy...@gmail.com>:
> >> As far as I understand, the cited text does not say anything
> >> about the package, version, architecture, and package-type
> >> components of a full package name other than that they are
> >> specified in the control file (of unknown contents) of the
> >> package, and that the version part is a concatenation of the
> >> upstream version information (of unknown contents) followed by a
> >> hyphen and the revision information (of unknown contents). And
> >> finally these parts are concatenated by underscores.
> >>
> >> If any of these parts contains some white space, then that white
> >> space will be in the full package name, as well.
> >
> > Sorry. I should have shown you, but I haven't found it till now,
> > the following description given in "Debian RFC822 control data
> > format" [1]:
> >
> > I should have shown you, but I didn't find the following
> > description given in "Debian RFC822 Control Data Formats" until
> > now:
> >
> > $ man deb822 | egrep -A3 -i 'whitespace must not'
> > Whitespace must not appear inside names (of packages,
> > architectures, files or anything else) or version numbers,
> > or between the characters of multi-character version
> > relationships.
> >
> > [1] https://man7.org/linux/man-pages/man5/deb822.5.html
> >
> Thank you. So we are lucky, and in this case it's not necessary to
> do what is described below.
> >> The best practice with regard to shell programming: Don't ever
> >> misuse a (non‐array) shell variable v as an array variable by
> >> merely gluing together the array elements with white space and
> >> then using the value of the variable by unquoted references ($v)
> >> to have it be split at white space.
> >>
> >> With Bash and other modern shells you have got even true array
> >> variables. So you could use them in this case. (But as you might
> >> know, I prefer (if at all possible) to use solutions which work
> >> with any shell conforming to the POSIX standard.)
> >
> > But this may lead to the loss of some convenience. There is no best
> > of both worlds in the world.
> Compared with the disatrous consequences of an exploit I'd clearly
> prefer array variables with Bash and other modern shells.
>
>
> And even if I want to stay compatible with the POSIX standard, I
> consider an exploit far more inconvenient than the increased
> complexity in programming for the shell.
>
> Simulating an array variable using the POSIX standard could be done
> like this:
>
> Define a shell function "quote_words" (see below), which, when
> given zero or more invocation arguments, computes (a part of) a
> shell command line (of a simple command) resembling a command line
> to invoke those arguments and writes it to standard output, where
> it can be put in a simple shell variable by command substitution,
> like this:
>
> v="$(quote_words printf '%s\n' 'Hello, world!')"
>
> It can then invoked using the "eval" shell builtin command:
>
>
> eval "$v"
>
>
> Try the following command to see, what that is. The commands
>
>
> v="$(quote_words printf '%s\n' 'Hello, world!')" &&
> printf '%s\n' "$v" &&
> eval "$v"
>
>
> will output
>
>
> 'printf' '%s\n' 'Hello, world!'
> Hello, world!
>
>
> Now, this content of the variable v, which is a (part of a) shell
> command line of a simple command, can even be glued together with
> more parts of a command line by putting a blank between each of the
> command line parts to be glued together and stored in a simple
> shell variable. Finally, this shell variable can be fed to the
> "eval" shell builtin command to let it be invoked as a command
> line.
>
>
> In the example of the OP this would look like these shell commands:
>
>
> v="$(quote_words sudo apt install -y)" &&
> # Here's that comment for packages related to a.
> #
> v="$v $(quote_words 'pkg a 1' 'pkg a 2' … 'pkg a n+1')" &&
> # Here's that comment for packages related to b.
> #
> v="$v $(quote_words 'pkg b 1' 'pkg b 2' … 'pkg b n+1')" &&
>
> # Finally, let the "eval" command parse and invoke it:
> #
> eval "$v" &&
>
>
> That quote_words function might look like this:
>
>
> ### The function quote_words starts here.
> quote_words()
> (
> # Outputs a (part of) a command line resembling a 'simple
> # command' to be used by a POSIX-compliant shell.
> #
> # Usage:
> #
> # quote_words words to be quoted...
> #
> # exit code:
> # 0, if the command line has been successfully constructed,
> # !=0, if the function failed to construct the command line.
> #
> # Examples:
> #
> # Put a word list resembling a command invocation into a command
> # line:
> #
> # if command_line="$(
> # quote_words program with parameters
> # )"
> # then
> # # "eval" it:
> # eval " $command_line"
> #
> # # or pass it to a new shell:
> # sh -c -- "$command_line" sh
> #
> # # or pass it to "su":
> # su -- - a_user -c -- "$command_line" -su
> #
> # # or use it remotely:
> # ssh us...@remote.host.example " $command_line"
> #
> # else
> # # failed to compute the command line.
> # fi
> #
> #
> # Store the shell positional parameters in a variable and
> # restore them later:
> #
> # if args="$(quote_words "$@")"
> # then
> # # the positional parameters may be changed for any purpose
> # # ...
> #
> # # restore the positional parameters:
> #
> # eval "set -- $args"
> # else
> # # failed to save the positional parameters.
> # fi
>
> # "wordsep" contains a separator used for the list of the quoted
> # words. The first need not to be preceded by a separater:
>
> wordsep=
> for word
> do
> # "$wordsep" separates each word (except the first one) from
> # its precedessor:
> printf '%s' "$wordsep"
> if test -z "$word"
> then
> # The word is empty. Then the result is "''":
> printf '%s' "''"
> else
> # The word is not empty.
> while
> {
> prefix="${word%%\'*}"
> word="${word#"$prefix"}"
> # "$prefix" is the longest prefix of "$word", that does
> # not contain any apostrophes; it is removed from "$word".
> # Conclusion: "$word" is either empty or starts with an
> # apostrophe.
> if test -n "$prefix"
> then
> # "$prefix" consists of one or more characters. Put
> # them in between of two apostrophes:
> printf '%s' \'"${prefix}"\'
> fi
> test -n "$word" &&
> {
> # "$word" is not empty. Conclusion: "$word" starts with
> # one or more apostrophes.
> apostr="${word%%[!\']*}"
> # "$apostr" ist the longest beginning part of "$word",
> # that contains apostrophes only.
> if test -n "${apostr#"'"}"
> then
> # There are at least 2 apostrophes: Put it in between
> # of two double quotes:
> printf '"%s"' "${apostr}"
> else
> # There is 1 apostrophe: Precede it by a backslash:
> printf '\\%s' "${apostr}"
> fi
> # Remove the beginning apostrophes from "$word":
> word="${word#"$apostr"}"
> # If nothing is left, we are done:
> ${word:+:} false
> }
> }
> do
> :
> done
> fi
> # All following words (except the first one) have to be
> # separated from their precedessor with a blank:
> wordsep=' '
> done
> printf '\n'
> )
> ### The function quote_words ends here.
>
>
> Of course I won't want to write this function from scratch in every
> shell script where it is to be used. Therefore I put it in a file,
> say "~/shell-lib/quote_words.sh". Then, if I like to use it in a
> shell script, I just do
>
> . ~/shell-lib/quote_words.sh
>
> in that shell script to have that function defined. It's far more
> convenient than to be hit by any exploit.
>
>
> Look again at the example above, annotated by some comments on the
> value of the variable v:
>
> v="$(quote_words sudo apt install -y)" &&
> #
> # Now the variable v has got the contents
> #
> # 'sudo' 'apt' 'install' '-y'
> #
> # Note, how the 4 parameters have been quoted. (That will be
> # needed by the "eval" command, below.)
> # Here's that comment for packages related to a.
> #
> # Note, how the package names are glued with a blank character
> # to the end of the existing value of the variable v:
> # v="$v $(quote_words …)"
> #
> v="$v $(quote_words 'pkg a 1' 'pkg a 2' … 'pkg a n+1')" &&
> #
> # Now the variable v has got the contents:
> #
> # 'sudo' 'apt' 'install' '-y' 'pkg a 1' 'pkg a 2' … 'pkg a n+1'
> # Here's that comment for packages related to b.
> #
> v="$v $(quote_words 'pkg b 1' 'pkg b 2' … 'pkg b n+1')" &&
> #
> # Now the variable v has got the contents (shown wrapped for
> # better readability):
> #
> # 'sudo' 'apt' 'install' '-y' \
> # 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
> # 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
> #
> # Note, how the parameters have been quoted. (Thus they will
> # be protected from being broken apart by the "eval" command.)
>
> eval "$v" &&
> #
> # This command line has got the contents (shown wrapped
> # because of its length):
> #
> # ''\''sudo'\'' '\''apt'\'' '\''install'\'' '\''-y'\'' \
> # '\''pkg a 1'\'' '\''pkg a 2'\'' … '\''pkg a n+1'\'' \
> # '\''pkg b 1'\'' '\''pkg b 2'\'' … '\''pkg b n+1'\'
> #
> # Yes, that's terrible for humans but perfect for the shell.
> #
> # The shell will parse that command line, thus invoking the
> # "eval" builtin command with the following single literally
> # parameter (shown wrapped because of its length):
> #
> # 'sudo' 'apt' 'install' '-y' \
> # 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
> # 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
> #
> # The "eval" builtin command will parse its parameter like
> # a shell command line -- that's the purpose of the "eval"
> # command -- and finally invoke the equivalent of the following
> # command line:
> #
> # 'sudo' 'apt' 'install' '-y' \
> # 'pkg a 1' 'pkg a 2' … 'pkg a n+1' \
> # 'pkg b 1' 'pkg b 2' … 'pkg b n+1'
>
> This is the difference between using the "quote_words" function and
> merely gluing parameters together with white space: "quote_words"
> will proper quote each of the arguments before gluing them
> together. That allows and mandates the resulting string to be
> parsed by the "eval" builtin command. And of course that string is
> not to be split at white space prior to be passed to the "eval"
> command, therefore it shall be quoted. In the example above, that
> is
>
> eval "$v"
>
> Because the argument to the "eval" command has been quoted, it will
> not be broken apart at white space as would be the case with an
> unquoted reference to the variable.

On 29.12.2021 13:14, hongy...@gmail.com wrote:
> On Wednesday, December 29, 2021 at 6:42:40 AM UTC+8, Helmut Waitzmann
> wrote:
>> [ 325 lines of quoted text and code deleted ]

(It's neither necessary [in Usenet] nor welcome to quote all
the text if you don't refer to it substantially, especially if
the quote is 100+ times as large as your own "contribution".)
(It's also Netiquette [in Usenet] to break long lines at word
boundaries after around 72-76 characters of text, although in
Google epoch that seems to have partly passed out of mind.)

> I read through your long reply and didn't quite understand it.

I only skimmed through Helmut's text and think he was addressing
the question and the method how to handle array type information
with spaces and whatnot in the data values through flat variable
structures by adding quoting. Doing that in the right way is not
trivial, thus the lengthy code.

Using arrays (if your shell supports them) in the first place is
preferable if only to avoid complex (potentially error-prone) code.

> But I think for a problem with a small probability of occurrence,
> is it necessary to do such complex processing and consideration?

I think your thinking is flawed here. (Or maybe it's just how you
formulated it?) - It's not about "probabilities". If you write
code that's not robust it may fail at any time, just depending
on the data.

And failing code includes potential loss of data or security
issues, depending on the software context. It may be irrelevant
for you on your private machine at home. Though I am shuddering
if one is responsible for the systems of a company and for other
folks that is relying on your code.

Janis

>
> HZ
>