On 2023-10-02, Ed Morton <mortonspam@gmail.com> wrote:
> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>> Ed Morton <mortonspam@gmail.com> writes:
>>> So I don't see any of that as justifying adding a bunch of options to
>>> grep to do something other than it's primary purpose of g/re/p and
>>> making it different from all other text processing tools in that
>>> regard.
>>
>> Why do you think it needs justifying? It’s their code, they can add
>> whatever they like to it.
>>
>
> The GNU people added functionality to grep in a way that is
> contradictory to the Unix philosophy and makes it inconsistent with all
> other text processing tools.

- GNU stands for GNU is Not Unix.

- The GNU coding standards document says:

The GNU Project regards standards published by other organizations
as suggestions, not orders. We consider those standards, but we do
not “obey” them. In developing a GNU program, you should implement
an outside standard’s specifications when that makes the GNU system
better overall in an objective sense. When it doesn’t, you shouldn’t.

- Unix programs have recursion built in: from memory I can think of
ls, rm, cp, mv, chgrp, chmod and chown.

- BSD Unixes have a -R option on grep; some, like FreeBSD, have an -r
option as well as a rgrep utility.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 03.10.2023 09:26, Kaz Kylheku wrote:
>
> - Unix programs have recursion built in: from memory I can think of
> ls, rm, cp, mv, chgrp, chmod and chown.

When I saw in the 'tw' examples command calls like 'tw chmod go-w' I
wondered whether it would have been a good design idea to use such a
paradigm generally; tw ls; tw rm, etc., instead of implementing some
-r (or -R or other) tree walks in each program separately. This ship
may have sailed already but the idea seems sensible. (And 'tw' seems
more powerful than 'find', yet with a less "archaic" syntax.) Options
to 'tw' would control the tree walk and the commands would have their
own options; a good separation of control, no multiple duplications.
Of course we have such a beast already as standard with find -exec +,
but the 'tw' interface seems smarter. I wonder why 'tw' didn't get in
wider use.

Janis

On 02.10.2023 13:59, Ed Morton wrote:
> [...]
>
> If reducing how much typing we need to do for that was their only goal,
> they could have introduced a separate tool named something like "ftf"
> that does the equivalent of "Find -Type F" [...]

Have a look at 'tw' (man page: http://volatile.gridbug.de/tw.out) which
is not restricted to '-type f' but can be parameterized with all options
necessary to control the search.

tw -tw-options... cmd -cmd-options... cmd-args...

All file subsets are determined by the 'tw' options and the commands use
just their own options. (The commands need not implement the tree walk
and the options to control it.) And each command used with 'tw' just
cares about its own command specific functional options.

Janis

On 2023-10-03, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> On 03.10.2023 09:26, Kaz Kylheku wrote:
>>
>> - Unix programs have recursion built in: from memory I can think of
>> ls, rm, cp, mv, chgrp, chmod and chown.
>
> When I saw in the 'tw' examples command calls like 'tw chmod go-w' I
> wondered whether it would have been a good design idea to use such a
> paradigm generally; tw ls; tw rm, etc., instead of implementing some
> -r (or -R or other) tree walks in each program separately. This ship
> may have sailed already but the idea seems sensible. (And 'tw' seems
> more powerful than 'find', yet with a less "archaic" syntax.)

In the case of chmod, you have to be careful about order. E.g. if you're
*removing* read/execute permissions from everything, you have to go
bottom up. If adding, then top down.

> to 'tw' would control the tree walk and the commands would have their
> own options; a good separation of control, no multiple duplications.
> Of course we have such a beast already as standard with find -exec +,
> but the 'tw' interface seems smarter. I wonder why 'tw' didn't get in
> wider use.

Performance? Launch a new chmod process for every visited object.

That could be addressed by allowing executables to be loadable as shared
libraries that expose the main function.

Then tw could find chmod in PATH, try to dlopen it, and if that works
and a "main" symbol is found, repeatedly call that function rather
than fork + exec.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

On 03.10.2023 17:20, Kaz Kylheku wrote:
> On 2023-10-03, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> On 03.10.2023 09:26, Kaz Kylheku wrote:
>>>
>>> - Unix programs have recursion built in: from memory I can think of
>>> ls, rm, cp, mv, chgrp, chmod and chown.
>>
>> When I saw in the 'tw' examples command calls like 'tw chmod go-w' I
>> wondered whether it would have been a good design idea to use such a
>> paradigm generally; tw ls; tw rm, etc., instead of implementing some
>> -r (or -R or other) tree walks in each program separately. This ship
>> may have sailed already but the idea seems sensible. (And 'tw' seems
>> more powerful than 'find', yet with a less "archaic" syntax.)
>
> In the case of chmod, you have to be careful about order. E.g. if you're
> *removing* read/execute permissions from everything, you have to go
> bottom up. If adding, then top down.

My expectation is that a tree walk function would be controllable
(by options) in which order the tree nodes should be visited.[*]

>
>> to 'tw' would control the tree walk and the commands would have their
>> own options; a good separation of control, no multiple duplications.
>> Of course we have such a beast already as standard with find -exec +,
>> but the 'tw' interface seems smarter. I wonder why 'tw' didn't get in
>> wider use.
>
> Performance? Launch a new chmod process for every visited object.

I don't see how that would be an unavoidable implementation. I mean
find -exec + will also collect more than one item. There could even
be a yet more flexible and sophisticated implementation than find.
(Not sure about what 'tw' does.) There are only a couple of tree walk
strategies (you could choose from by options).

(It may be worth studying 'tw' in more detail and making some tests.)

>
> That could be addressed by allowing executables to be loadable as shared
> libraries that expose the main function.
>
> Then tw could find chmod in PATH, try to dlopen it, and if that works
> and a "main" symbol is found, repeatedly call that function rather
> than fork + exec.

Janis

[*] Disclaimer: I haven't spent any time studying details of 'tw'.

Kaz Kylheku <864-117-4973@kylheku.com> wrote:
> - Unix programs have recursion built in: from memory I can think of
> ls, rm, cp, mv, chgrp, chmod and chown.

It's also interesting to compare tar and cpio. Tar also has
recursion built in, and filtering functionality duplicated with
find. Cpio instead takes a list of files to archive from stdin,
which also avoids the maximum argument number and maximum command
length limits that complicate using command-line arguments.

GNU cpio actually supports creating tar archives, but it's notable
that people (including me) don't seem to use it for that, even
though its behaviour is more UNIXy than tar. POSIX apparantly
recommends pax, which supports doing things both ways, but everyone
ignores that too.

--
__ __
#_ < |\| |< _#

On 10/3/2023 10:19 AM, Janis Papanagnou wrote:
> On 02.10.2023 13:59, Ed Morton wrote:
>> [...]
>>
>> If reducing how much typing we need to do for that was their only goal,
>> they could have introduced a separate tool named something like "ftf"
>> that does the equivalent of "Find -Type F" [...]
>
> Have a look at 'tw' (man page: http://volatile.gridbug.de/tw.out) which
> is not restricted to '-type f' but can be parameterized with all options
> necessary to control the search.
>
> tw -tw-options... cmd -cmd-options... cmd-args...
>
> All file subsets are determined by the 'tw' options and the commands use
> just their own options. (The commands need not implement the tree walk
> and the options to control it.) And each command used with 'tw' just
> cares about its own command specific functional options.
>
> Janis
>

Yup, that looks like the kind of command we should be using instead of
grep having been given arguments to find files. Thanks for that link and
discussion.

Ed.

On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
> Ed Morton <mortonspam@gmail.com> writes:
>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>> Ed Morton <mortonspam@gmail.com> writes:
>>>> So I don't see any of that as justifying adding a bunch of options to
>>>> grep to do something other than it's primary purpose of g/re/p and
>>>> making it different from all other text processing tools in that
>>>> regard.
>>> Why do you think it needs justifying? It’s their code, they can add
>>> whatever they like to it.
>>
>> The GNU people added functionality to grep in a way that is
>> contradictory to the Unix philosophy and makes it inconsistent with
>> all other text processing tools.
>
> Why does that matter?

For the same reasons that cohesion and consistency matter in any system.
Google "software" with those 2 terms to find more information on how
those concepts apply to software.

If Unix were being invented today and the design decision being
presented in an attempt to answer the question "how do we call a text
processing tool on every file found under a directory?" was:

IF (tool == grep) && (provider == GNU) THEN
tool -r '...'
ELSE
find . -type f tool '...' {} +
ENDIF

instead of just:

find . -type f tool '...' {} +

the person suggesting it would be ridiculed and sent back to the drawing
board.

>
>> So far the only suggestion I've heard for why they did that is so
>> people could do:
>>
>> grep -r 'regexp'
>>
>> instead of:
>>
>> find . -type f -exec grep -H 'regexp' {} +
>>
>> which saves us about 20 simple, common characters over find+grep but
>> does nothing for find+awk, find+sed, etc.
>
> Sounds good to me. Less typing = better. Given the uptake of the GNU
> tools I suspect my opinion is widely shared.
>

The GNU providers have made many worthwhile contributions to many tools,
people are not adopting GNU tools because they added "-r" to grep.

Ed.

Computer Nerd Kev <not@telling.you.invalid> wrote:
> Kaz Kylheku <864-117-4973@kylheku.com> wrote:
>> - Unix programs have recursion built in: from memory I can think of
>> ls, rm, cp, mv, chgrp, chmod and chown.
>
> It's also interesting to compare tar and cpio. Tar also has
> recursion built in, and filtering functionality duplicated with
> find. Cpio instead takes a list of files to archive from stdin,
> which also avoids the maximum argument number and maximum command
> length limits that complicate using command-line arguments.
>
> GNU cpio actually supports creating tar archives, but it's notable
> that people (including me) don't seem to use it for that, even
> though its behaviour is more UNIXy than tar. POSIX apparantly
> recommends pax, which supports doing things both ways, but everyone
> ignores that too.

tar added later in its history the option to work as a filter

find . | tar --create --files-from=- --file=- > archive.tar

Had the feature not been added cpio or pax would have higher popularity.

That is at least the case of GNU/tar. I just checked the OpenBSD/tar
manpage and the feature to get files from stdin does not appear.

On 2023-10-04, Javier <invalid@invalid.invalid> wrote:

> find . | tar --create --files-from=- --file=- > archive.tar
>
> That is at least the case of GNU/tar. I just checked the OpenBSD/tar
> manpage and the feature to get files from stdin does not appear.

Actually, it does: -I <file>

Where <file> can be "-" for stdin.

--
Christian "naddy" Weisgerber naddy@mips.inka.de

Ed Morton <mortonspam@gmail.com> writes:
> On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
>> Ed Morton <mortonspam@gmail.com> writes:
>>> The GNU people added functionality to grep in a way that is
>>> contradictory to the Unix philosophy and makes it inconsistent with
>>> all other text processing tools.
>> Why does that matter?
>
> For the same reasons that cohesion and consistency matter in any
> system. Google "software" with those 2 terms to find more information
> on how those concepts apply to software.

Prioritizing consistency over usability, then.

> If Unix were being invented today and the design decision being
> presented in an attempt to answer the question "how do we call a text
> processing tool on every file found under a directory?" was:
>
> IF (tool == grep) && (provider == GNU) THEN
> tool -r '...'
> ELSE
> find . -type f tool '...' {} +
> ENDIF
>
> instead of just:
>
> find . -type f tool '...' {} +
>
> the person suggesting it would be ridiculed and sent back to the
> drawing board.

Are you somehow under the impression that the existence of “grep -r”
prevents the second answer from working? If not then why on earth would
you imagine anyone would give the first answer?

> The GNU providers have made many worthwhile contributions to many
> tools, people are not adopting GNU tools because they added "-r" to
> grep.

Well it’s demonstrably not an obstacle to many people.

--
https://www.greenend.org.uk/rjk/

gazelle@shell.xmission.com (Kenny McCormack) writes:
> Richard Kettlewell <invalid@invalid.invalid> wrote:
>>Ed Morton <mortonspam@gmail.com> writes:
>>> So I don't see any of that as justifying adding a bunch of options to
>>> grep to do something other than it's primary purpose of g/re/p and
>>> making it different from all other text processing tools in that
>>> regard.
>>
>>Why do you think it needs justifying? Its their code, they can add
>>whatever they like to it.
>
> 1) "Justifying" was, perhaps, too strong of a word. Obviously, there's no
> need to do so, in a strict, legalistic sense. Though (total aside coming
> up), it reminds me of when lawyers get uppity when they see computer
> messages like "illegal instruction" - when they know that, in their terms
> and frame of reference, there's nothing illegal about it.

It’s the OP’s choice of word...

> 3) I personally find the "find" command archaic and hard to use (meaning:
> If it were being designed today, it wouldn't be like it is). From a
> usability standpoint, it is much better to be able to just include "-r" on
> the grep command line, than to have to write out a long, ugly "find"
> invocation.

Quite.

--
https://www.greenend.org.uk/rjk/

In article <wwvv8bl2cf8.fsf@LkoBDZeT.terraraq.uk>,
Richard Kettlewell <invalid@invalid.invalid> wrote:
....
>>>Why do you think it needs justifying? Its their code, they can add
>>>whatever they like to it.
>>
>> 1) "Justifying" was, perhaps, too strong of a word. Obviously, there's no
>> need to do so, in a strict, legalistic sense. Though (total aside coming
>> up), it reminds me of when lawyers get uppity when they see computer
>> messages like "illegal instruction" - when they know that, in their terms
>> and frame of reference, there's nothing illegal about it.
>
>It's the OPs choice of word...

Indeed. I was basically a) rebuking OP for using too strong of a word, and
b) re-assuring you that your reaction to his wording was reasonable.

>> 3) I personally find the "find" command archaic and hard to use (meaning:
>> If it were being designed today, it wouldn't be like it is). From a
>> usability standpoint, it is much better to be able to just include "-r" on
>> the grep command line, than to have to write out a long, ugly "find"
>> invocation.
>
>Quite.

Agreed.

Note also that the real bad thing about the "find ... -exec grep ..." type
solution (that OP seems to favor) is that it inefficiently spawns a new
process for each file. This has, of course been noted many times in this
thread, but it cannot be over-stressed. Note that you can use "xargs" to
mitigate this problem (as I routinely do) or use the (non-standard) "+"
option in (GNU) find. I've never used the later option, just because I got
used to using xargs and never saw the need for a different method.

--
Elect a clown, expect a circus.

In article <wwv1qe93r39.fsf@LkoBDZeT.terraraq.uk>,
Richard Kettlewell <invalid@invalid.invalid> wrote:
....
>> IF (tool == grep) && (provider == GNU) THEN
>> tool -r '...'
>> ELSE
>> find . -type f tool '...' {} +
>> ENDIF
>>
>> instead of just:
>>
>> find . -type f tool '...' {} +
>>
>> the person suggesting it would be ridiculed and sent back to the
>> drawing board.
>
>Are you somehow under the impression that the existence of grep -r
>prevents the second answer from working? If not then why on earth would
>you imagine anyone would give the first answer?

I think OP is somewhat cryptically arguing that:

a) People can't use the new fancy GNU stuff in portable scripts. except
if they...
b) In order to use them in a portable script, they'd have to code as
shown above - that is, only use the new fancy GNU stuff if they are
running in an environment known to have that stuff. But ...
c) Nobody is going to do that, because it is just silly, so you end up
just coding it the old-fashioned way, and thus ...
d) The new stuff never gets used, so why should anyone bother designing
and implementing it?

I don't agree with OP, but I think that's the position he is trying to
argue.

--
The randomly chosen signature file that would have appeared here is more than 4
lines long. As such, it violates one or more Usenet RFCs. In order to remain
in compliance with said RFCs, the actual sig can be found at the following URL:
http://user.xmission.com/~gazelle/Sigs/Windows

Ed Morton <mortonspam@gmail.com> writes:

> On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
>> Ed Morton <mortonspam@gmail.com> writes:
>>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>> So I don't see any of that as justifying adding a bunch of options to
>>>>> grep to do something other than it's primary purpose of g/re/p and
>>>>> making it different from all other text processing tools in that
>>>>> regard.
>>>> Why do you think it needs justifying? It’s their code, they can add
>>>> whatever they like to it.
>>>
>>> The GNU people added functionality to grep in a way that is
>>> contradictory to the Unix philosophy and makes it inconsistent with
>>> all other text processing tools.
>> Why does that matter?
>
> For the same reasons that cohesion and consistency matter in any
> system. Google "software" with those 2 terms to find more information on
> how those concepts apply to software.

Those are not the only concepts that matter.

> If Unix were being invented today and the design decision being presented
> in an attempt to answer the question "how do we call a text processing tool
> on every file found under a directory?" was:
>
> IF (tool == grep) && (provider == GNU) THEN
> tool -r '...'
> ELSE
> find . -type f tool '...' {} +
> ENDIF
>
> instead of just:
>
> find . -type f tool '...' {} +

find . -type f -exec tool '...' '{}' +

> the person suggesting it would be ridiculed and sent back to the drawing
> board.

It's not "just" find ... find is a mess. If the file system, the shell
and all the rest were such that adding some command or prefix like

rec grep pattern .

made grep behave pretty much like grep -r then no one would bother to
add options like -r to grep.

But find is the best we have, and it's a mess. It uses mandatory syntax
that needs to be quoted (the {}) and does not always preserve the desred
behaviour. For example

grep -rq needle haystack

can succeed (exit 0) where

find haystack -type f -exec grep -q needle '{}' +

can fail (exit non-zero). This is because find can run multiple
commands and does not always combine the exit statuses correctly. I
would be happy to be wrong about this, but I'm sure I tested it some
time ago.

Of course what's really needed is proper functional composition of
tools. What I mean is that where a list of arguments is needed a lazy
list resulting from a program (something like a generator if you prefer
the term) could be given. We can't use bash's **/* nor $(find . -type
f) for various reasons, not least because the argument list must be
fully evaluated before being passed to exec'd program. A lot would have
to change before (to invent syntax on the fly)

grep needle $[files-in haystack]

started to produce output immediately and could not run out of
resources.

Unix tries with pipes and macro expansion, but the input stream is not
always available (or logical to use) and the shell's expansion and
quoting rules are so intricate that I doubt there is anyone here who has
not been caught out by them even after years of Unix use.

--
Ben.

On 05.10.2023 20:32, Kenny McCormack wrote:
>
> Note also that the real bad thing about the "find ... -exec grep ..." type
> solution (that OP seems to favor) is that it inefficiently spawns a new
> process for each file.

Then just use: find ... -exec grep ... {} +

> This has, of course been noted many times in this
> thread, but it cannot be over-stressed. Note that you can use "xargs" to
> mitigate this problem (as I routinely do) or use the (non-standard) "+"
> option in (GNU) find. [...]

It is non-standard? - Here's a quote from the standard...

-exec utility_name [argument ...] ;
-exec utility_name [argument ...] {} +
[...]
If the primary expression is punctuated by a <plus-sign>, the primary
shall always evaluate as true, and the pathnames for which the primary
is evaluated shall be aggregated into sets. The utility utility_name
shall be invoked once for each set of aggregated pathnames. [...]

Did I misread the POSIX standard? [*]

Janis

[*] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html

On 10/5/2023 2:37 PM, Ben Bacarisse wrote:
> Ed Morton <mortonspam@gmail.com> writes:
>
>> On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
>>> Ed Morton <mortonspam@gmail.com> writes:
>>>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>>> So I don't see any of that as justifying adding a bunch of options to
>>>>>> grep to do something other than it's primary purpose of g/re/p and
>>>>>> making it different from all other text processing tools in that
>>>>>> regard.
>>>>> Why do you think it needs justifying? It’s their code, they can add
>>>>> whatever they like to it.
>>>>
>>>> The GNU people added functionality to grep in a way that is
>>>> contradictory to the Unix philosophy and makes it inconsistent with
>>>> all other text processing tools.
>>> Why does that matter?
>>
>> For the same reasons that cohesion and consistency matter in any
>> system. Google "software" with those 2 terms to find more information on
>> how those concepts apply to software.
>
> Those are not the only concepts that matter.

Of course, they're just the answer to the question Richard asked.

>
>> If Unix were being invented today and the design decision being presented
>> in an attempt to answer the question "how do we call a text processing tool
>> on every file found under a directory?" was:
>>
>> IF (tool == grep) && (provider == GNU) THEN
>> tool -r '...'
>> ELSE
>> find . -type f tool '...' {} +
>> ENDIF
>>
>> instead of just:
>>
>> find . -type f tool '...' {} +
>
> find . -type f -exec tool '...' '{}' +

Right, thanks. I typed my response once and my newsreader discarded it
so I had to type it again and rushed it.

>> the person suggesting it would be ridiculed and sent back to the drawing
>> board.
>
> It's not "just" find ... find is a mess. If the file system, the shell
> and all the rest were such that adding some command or prefix like
>
> rec grep pattern .
>
> made grep behave pretty much like grep -r then no one would bother to
> add options like -r to grep.

Agreed, that is the right solution if people are unhappy with `find` and
apparently there is an existing tool to do just that, see Janis's posts
in this thread.

>
> But find is the best we have, and it's a mess. It uses mandatory syntax
> that needs to be quoted (the {}) and does not always preserve the desred
> behaviour. For example
>
> grep -rq needle haystack
>
> can succeed (exit 0) where
>
> find haystack -type f -exec grep -q needle '{}' +
>
> can fail (exit non-zero). This is because find can run multiple
> commands and does not always combine the exit statuses correctly. I
> would be happy to be wrong about this, but I'm sure I tested it some
> time ago.

Interesting - that's not something I've ever come across but thanks for
bringing it up as a possible concern.

Ed.

>
> Of course what's really needed is proper functional composition of
> tools. What I mean is that where a list of arguments is needed a lazy
> list resulting from a program (something like a generator if you prefer
> the term) could be given. We can't use bash's **/* nor $(find . -type
> f) for various reasons, not least because the argument list must be
> fully evaluated before being passed to exec'd program. A lot would have
> to change before (to invent syntax on the fly)
>
> grep needle $[files-in haystack]
>
> started to produce output immediately and could not run out of
> resources.
>
> Unix tries with pipes and macro expansion, but the input stream is not
> always available (or logical to use) and the shell's expansion and
> quoting rules are so intricate that I doubt there is anyone here who has
> not been caught out by them even after years of Unix use.
>

On 10/5/2023 12:40 PM, Richard Kettlewell wrote:
> Ed Morton <mortonspam@gmail.com> writes:
>> On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
>>> Ed Morton <mortonspam@gmail.com> writes:
>>>> The GNU people added functionality to grep in a way that is
>>>> contradictory to the Unix philosophy and makes it inconsistent with
>>>> all other text processing tools.
>>> Why does that matter?
>>
>> For the same reasons that cohesion and consistency matter in any
>> system. Google "software" with those 2 terms to find more information
>> on how those concepts apply to software.
>
> Prioritizing consistency over usability, then.

No. Having one doesn't mean you can't have the other. Assuming that some
people find it unusable to type "find . -type f exec grep" for calling
grep when they already do "find . -type f" for finding files and "find .
-type f -exec sed" for calling sed, then had the GNU people solved that
usability problem by providing a tool to let us do:

tool finding-args grep greping-args
tool finding-args sed seding-args
tool finding-args awk seding-args

instead of:

grep finding-args greping-args
find finding-args -exec sed seding-args
find finding-args -exec awk awking-args

we would have usability and consistency.

My point was that if what you're considering doing will introduce
inconsistencies then you should think long and hard about whether or not
it's the right solution to any given problem and there will almost
always be a better possible alternative, as there is in this case.

>
>> If Unix were being invented today and the design decision being
>> presented in an attempt to answer the question "how do we call a text
>> processing tool on every file found under a directory?" was:
>>
>> IF (tool == grep) && (provider == GNU) THEN
>> tool -r '...'
>> ELSE
>> find . -type f tool '...' {} +
>> ENDIF
>>
>> instead of just:
>>
>> find . -type f tool '...' {} +
>>
>> the person suggesting it would be ridiculed and sent back to the
>> drawing board.
>
> Are you somehow under the impression that the existence of “grep -r”
> prevents the second answer from working? If not then why on earth would
> you imagine anyone would give the first answer?

The imaginary proposed existence of "GNU grep -r" when first designing
Unix is what I described in that first answer and no, you cannot give
the second answer when there are exceptions to the rule. I don't know
how to restate what I'm saying in a way you might better understand, sorry.

>> The GNU providers have made many worthwhile contributions to many
>> tools, people are not adopting GNU tools because they added "-r" to
>> grep.
>
> Well it’s demonstrably not an obstacle to many people.
>

Of course not.

Ed.

Ed Morton <mortonspam@gmail.com> writes:

> On 10/5/2023 2:37 PM, Ben Bacarisse wrote:
>> Ed Morton <mortonspam@gmail.com> writes:
>>
>>> On 10/3/2023 2:00 AM, Richard Kettlewell wrote:
>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>> On 10/2/2023 2:09 AM, Richard Kettlewell wrote:
>>>>>> Ed Morton <mortonspam@gmail.com> writes:
>>>>>>> So I don't see any of that as justifying adding a bunch of options to
>>>>>>> grep to do something other than it's primary purpose of g/re/p and
>>>>>>> making it different from all other text processing tools in that
>>>>>>> regard.
>>>>>> Why do you think it needs justifying? It’s their code, they can add
>>>>>> whatever they like to it.
>>>>>
>>>>> The GNU people added functionality to grep in a way that is
>>>>> contradictory to the Unix philosophy and makes it inconsistent with
>>>>> all other text processing tools.
>>>> Why does that matter?
>>>
>>> For the same reasons that cohesion and consistency matter in any
>>> system. Google "software" with those 2 terms to find more information on
>>> how those concepts apply to software.
>> Those are not the only concepts that matter.
>
> Of course, they're just the answer to the question Richard asked.

They are /your/ answer to his question -- these are the concepts that
make it matter to you. Other concepts (like ease of use) don't
necessarily give the same answer which is why I mention them. /My/
answer would be "It doesn't matter much -- ease of use is too important
in this case".

(I hope you are not being hyper literal here. My "answer" is not an
answer in the literal sense. If I took "Why does that matter?"
absolutely literally, I'd have to give your answer as well with maybe
just "but only a teeny, tiny bit" added. But ripostes like "Why does
that matter?" are almost always rhetorical, and not literal requests for
the reasons, no matter how insignificant the respondent might consider
them to be.)

>>> If Unix were being invented today and the design decision being presented
>>> in an attempt to answer the question "how do we call a text processing tool
>>> on every file found under a directory?" was:
>>>
>>> IF (tool == grep) && (provider == GNU) THEN
>>> tool -r '...'
>>> ELSE
>>> find . -type f tool '...' {} +
>>> ENDIF
>>>
>>> instead of just:
>>>
>>> find . -type f tool '...' {} +
>> find . -type f -exec tool '...' '{}' +
>
> Right, thanks. I typed my response once and my newsreader discarded it so I
> had to type it again and rushed it.
>
>>> the person suggesting it would be ridiculed and sent back to the drawing
>>> board.
>> It's not "just" find ... find is a mess. If the file system, the shell
>> and all the rest were such that adding some command or prefix like
>> rec grep pattern .
>> made grep behave pretty much like grep -r then no one would bother to
>> add options like -r to grep.
>
> Agreed, that is the right solution if people are unhappy with `find` and
> apparently there is an existing tool to do just that, see Janis's posts in
> this thread.

tw may be better than find, but that's a low bar. My "rec" was intended
to be hypothetical. I should maybe have said that if it were possible
to have a rec prefix that magically just "did the right thing" then -r
would not be needed.

But your remark raises an interesting question. Are you happy with
find? I use it quite a lot, but I am unhappy with it pretty much every
time!

>> But find is the best we have, and it's a mess. It uses mandatory syntax
>> that needs to be quoted (the {}) and does not always preserve the desred
>> behaviour. For example
>> grep -rq needle haystack
>> can succeed (exit 0) where
>> find haystack -type f -exec grep -q needle '{}' +
>> can fail (exit non-zero). This is because find can run multiple
>> commands and does not always combine the exit statuses correctly. I
>> would be happy to be wrong about this, but I'm sure I tested it some
>> time ago.
>
> Interesting - that's not something I've ever come across but thanks for
> bringing it up as a possible concern.

It's not a possible concern, it's an actual concern! If you want to
know if "pattern" occurs in any file from . down, you /can't/ use

find . -type f -exec grep -q pattern '{}' +

because you will get the wrong answer in some cases.

But consider the general case... Imagine a collection of tools to test
if a file has this or that property. The tools can all have multiple
arguments but what do they do with conflicting answers? For some
properties it might be useful for the tool to report success (0) if
/all/ the arguments have that property and for other properties it might
be desirable to report success if /any/ of the arguments have it. If
find splits an argument list into two and the tool reports 0 on one half
and 1 on the second, what should find report? If the tool is an "all"
tool it should report 1, and if the tool is an "any" tool it should
report 0.

--
Ben.

Ed Morton <mortonspam@gmail.com> writes:
> On 10/5/2023 12:40 PM, Richard Kettlewell wrote:
>> Prioritizing consistency over usability, then.
>
> No. Having one doesn't mean you can't have the other. Assuming that
> some people find it unusable to type "find . -type f exec grep" for
> calling grep when they already do "find . -type f" for finding files
> and "find . -type f -exec sed" for calling sed, then had the GNU
> people solved that usability problem by providing a tool to let us do:
>
> tool finding-args grep greping-args
> tool finding-args sed seding-args
> tool finding-args awk seding-args
>
> instead of:
>
> grep finding-args greping-args
> find finding-args -exec sed seding-args
> find finding-args -exec awk awking-args
>
> we would have usability and consistency.

To me, banning ‘grep -r’ would not introduce enough additional
consistency to justify the reduction in convenience. Not even close.

>>> If Unix were being invented today and the design decision being
>>> presented in an attempt to answer the question "how do we call a text
>>> processing tool on every file found under a directory?" was:
>>>
>>> IF (tool == grep) && (provider == GNU) THEN
>>> tool -r '...'
>>> ELSE
>>> find . -type f tool '...' {} +
>>> ENDIF
>>>
>>> instead of just:
>>>
>>> find . -type f tool '...' {} +
>>>
>>> the person suggesting it would be ridiculed and sent back to the
>>> drawing board.
>> Are you somehow under the impression that the existence of “grep -r”
>> prevents the second answer from working? If not then why on earth would
>> you imagine anyone would give the first answer?
>
> The imaginary proposed existence of "GNU grep -r" when first designing
> Unix is what I described in that first answer and no, you cannot give
> the second answer when there are exceptions to the rule. I don't know
> how to restate what I'm saying in a way you might better understand,
> sorry.

What prevents you from giving the second answer? We know from the
real-life situation that exists today that it’ll work if you use it.

--
https://www.greenend.org.uk/rjk/

On 06.10.2023 04:25, Ben Bacarisse wrote:
>
> But your remark raises an interesting question. Are you happy with
> find? I use it quite a lot, but I am unhappy with it pretty much every
> time!

At times you get used to the 'find' standards. But GNU 'find' can
even be dangerous(!) with 'find's generally crude syntax and logic;
I faintly recall a thread where we discussed the -delete option...

Janis

In our last episode, the evil Dr. Lacto had captured our hero,
Kenny McCormack <gazelle@shell.xmission.com>, who said:
> I think OP is somewhat cryptically arguing that:
>
> a) People can't use the new fancy GNU stuff in portable scripts. except
> if they...
> b) In order to use them in a portable script, they'd have to code as
> shown above - that is, only use the new fancy GNU stuff if they are
> running in an environment known to have that stuff. But ...
> c) Nobody is going to do that, because it is just silly, so you end up
> just coding it the old-fashioned way, and thus ...
> d) The new stuff never gets used, so why should anyone bother designing
> and implementing it?

This is starting to remind me of the csh vs bash argument, where csh is
viewed as "better than bash for interactive use, but worse than bash for
shell scripting." Similarly, "assuming the added features of GNU grep"
would be useful for interactive use (where I know GNU grep is available)
while shell scripting would expect the standard grep.

--hymie! http://nasalinux.net/~hymie hymie@nasalinux.net
===============================================================================
Another quality sig from hymie! Collect the whole set - trade with your friends
===============================================================================

Ben Bacarisse wrote:

> But find is the best we have, and it's a mess. It uses mandatory syntax
> that needs to be quoted (the {})

The {} does not need to be quoted. That's because { is a reserved
word in the shell, not an operator like (.

> and does not always preserve the desred
> behaviour. For example
>
> grep -rq needle haystack
>
> can succeed (exit 0) where
>
> find haystack -type f -exec grep -q needle '{}' +
>
> can fail (exit non-zero). This is because find can run multiple
> commands and does not always combine the exit statuses correctly.

It combines them in the way specified in POSIX, which says "If any
invocation returns a non-zero value as exit status, the find utility
shall return a non-zero exit status." If I recall correctly, the
POSIX spec for this was based on SVR4. Presumably the SVR4 developers
did it that way because that's what you want for the usual case where
non-zero means an error occurred. I.e. if find exits non-zero you
know that either find itself or at least one of the utility
invocations encountered an error.

However, it does mean find's exit status is misleading if the invoked
utility can exit with non-zero in some non-error situations (such as
grep's exit of 1 indicating no match). It shouldn't be too hard to
come up with a tweak that gives the right info. Maybe something like
(untested):

test -n "$(find haystack -type f -exec \
sh -c 'grep -q needle "$@" && echo found' sh {} +)"

Obviously it's only worth going to that much trouble in a script.
For casual interactive use, I'd be inclined to use:

find haystack -type f -exec grep -l needle '{}' + | head -n 1

and just observe whether it writes a pathname or not. (Or even just
omit the "head -n 1" and hit Ctrl-C if it starts writing pathnames.)

--
Geoff Clare <netnews@gclare.org.uk>

Geoff Clare <geoff@clare.See-My-Signature.invalid> writes:

> Ben Bacarisse wrote:
>
>> But find is the best we have, and it's a mess. It uses mandatory syntax
>> that needs to be quoted (the {})
>
> The {} does not need to be quoted. That's because { is a reserved
> word in the shell, not an operator like (.

Is this true of all shells? I was going by the find man page. Maybe
it needs an update?

>> and does not always preserve the desred
>> behaviour. For example
>>
>> grep -rq needle haystack
>>
>> can succeed (exit 0) where
>>
>> find haystack -type f -exec grep -q needle '{}' +
>>
>> can fail (exit non-zero). This is because find can run multiple
>> commands and does not always combine the exit statuses correctly.
>
> It combines them in the way specified in POSIX, which says "If any
> invocation returns a non-zero value as exit status, the find utility
> shall return a non-zero exit status." If I recall correctly, the
> POSIX spec for this was based on SVR4. Presumably the SVR4 developers
> did it that way because that's what you want for the usual case where
> non-zero means an error occurred. I.e. if find exits non-zero you
> know that either find itself or at least one of the utility
> invocations encountered an error.
>
> However, it does mean find's exit status is misleading if the invoked
> utility can exit with non-zero in some non-error situations (such as
> grep's exit of 1 indicating no match). It shouldn't be too hard to
> come up with a tweak that gives the right info. Maybe something like
> (untested):
>
> test -n "$(find haystack -type f -exec \
> sh -c 'grep -q needle "$@" && echo found' sh {} +)"
>
> Obviously it's only worth going to that much trouble in a script.
> For casual interactive use, I'd be inclined to use:
>
> find haystack -type f -exec grep -l needle '{}' + | head -n 1

For casual interactive use I'd be inclined to use grep -r :-)

An alternative would be to give the user a way to tell find how to
combine the exit statuses: -minstatus and -maxstatus? A couple more
options can't hurt.

> and just observe whether it writes a pathname or not. (Or even just
> omit the "head -n 1" and hit Ctrl-C if it starts writing pathnames.)

The bigger picture here is that find is not a simple way to make
commands handle nested file structures. It mostly works, but many cases
need care. find imposes a cognitive load that undoes the supposed gain
from having everything consistent. For day-to-day use, it can be easier
to remember that -r exists (if you are lucky enough to have it) than it
is to do the mental check "will find get this one right?".

--
Ben.

On 06/10/2023 08:12, Richard Kettlewell wrote:
> To me, banning ‘grep -r’ would not introduce enough additional
> consistency to justify the reduction in convenience. Not even close.

Well, that's the trouble with feeping creaturism. Once the
creature has fept, it's impossible to reconsider it; there are always
some people using it who mustn't be upset. The time to decide is
/before/ adding it. My point would be that the addition is not cost-
free to those who don't use it. Grep is a case in point:

$ man grep | wc
645 4249 34365

For comparison, the 7th Edition manual entry is 61 lines. Is Gnu grep
more than 10x as useful as V7 grep? How many users read and understand
the whole entry? Or try

$ man cc | wc
22363 125223 1063468

That's a whole /book/ size! [V7 entry is less than two pages.] Worse,
it's unreadable by normal people. The same applies to command after
command. It's not just the size of the manual entries that are a bar
to understanding, it's their number. There are 3447 commands in my
$PATH, the great majority of which I've never heard of, never used,
and didn't do anything [conscious] to install; they just got dragged
in by other packages and commands.

The result is that no-one, but no-one, actually understands the
whole, or even half, of Linux. For V7, I have the entire manual plus all
the supporting documentation, plus about half as much again in local and
imported add-ons, in two A4 document folders totalling about 20cm thick;
and the entire source code [inc the operating system and the same add-ons]
in a modest pile of print-out on a shelf in my garage. Lots of people
understood the /whole/ of Unix-on-the-PDP11 [and later the VAX, Sun, SGI
Indy, ...], inc the PDP11 Processor Manual [about the size of a modest
paperback novel]. Today, it's a black box which changes faster than you
can keep up and only a tiny corner of which is within human grasp.

Well, I suppose that's progress, of a sort. But I'm not sure it's
been in the right direction. End of rant.

--
Andy Walker, Nottingham.
Andy's music pages: www.cuboid.me.uk/andy/Music
Composer of the day: www.cuboid.me.uk/andy/Music/Composers/Forbes