On 2024-02-01, Scott Lurndal <scott@slp53.sl.home> wrote:
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>>On 01.02.2024 16:41, Malcolm McLean wrote:
>>>
>>> So could you list one or two reasons why you might prefer a program with
>>> five subroutines, and one or two reasons why you might prefer to write
>>> five programs which communicate via piped data?
>>
>>A quite appealing and naturally appearing task (from the past) to use
>>pipes was to model communication cascades. Something like (off the top
>>of my head)...
>>
>> data-source | sign | compress | crc | encrypt | channel-enc |
>> interleaver | channel-simulator | deinterleaver | channel-dec |
>> decrypt | crc-check | uncompress | check-sign | data-sink
>>
>>Component-pairs can be omitted, say you may leave out the un-/compress
>>function. And every component may be either special purpose or general.
>>A special purpose entity could be BCH-enc and RCPC-enc, or it can also
>>be (if better suited) a combined module, say 'crc -16' vs. 'crc -32'
>>with the function realized as option argument.
>
> There was also the widely used netpbm package for translating
> between different image formats.
>
> https://en.wikipedia.org/wiki/Netpbm
>
> $ giftopnm somepic.gif | ppmtobmp > somepic.bmp
> $ for i in *.png; do pngtopam $i | ppmtojpeg >`basename $i .png`.jpg; done

Also, in regard to some silly objections upthread about the danger of
binary data on standard ouptut, programs in Unix can easily do the
Following (and arguably should):

if (isatty(STDOUT_FILENO)) {
fprintf(stderr, "Cowardly refusing to dump binary data to a terminal.\n");
exit(EXIT_FAILURE);
}

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 01/02/2024 19:50, Kaz Kylheku wrote:
> On 2024-02-01, Scott Lurndal <scott@slp53.sl.home> wrote:
>> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
>>> On 01.02.2024 16:41, Malcolm McLean wrote:
>>>>
>>>> So could you list one or two reasons why you might prefer a program with
>>>> five subroutines, and one or two reasons why you might prefer to write
>>>> five programs which communicate via piped data?
>>>
>>> A quite appealing and naturally appearing task (from the past) to use
>>> pipes was to model communication cascades. Something like (off the top
>>> of my head)...
>>>
>>> data-source | sign | compress | crc | encrypt | channel-enc |
>>> interleaver | channel-simulator | deinterleaver | channel-dec |
>>> decrypt | crc-check | uncompress | check-sign | data-sink
>>>
>>> Component-pairs can be omitted, say you may leave out the un-/compress
>>> function. And every component may be either special purpose or general.
>>> A special purpose entity could be BCH-enc and RCPC-enc, or it can also
>>> be (if better suited) a combined module, say 'crc -16' vs. 'crc -32'
>>> with the function realized as option argument.
>>
>> There was also the widely used netpbm package for translating
>> between different image formats.
>>
>> https://en.wikipedia.org/wiki/Netpbm
>>
>> $ giftopnm somepic.gif | ppmtobmp > somepic.bmp
>> $ for i in *.png; do pngtopam $i | ppmtojpeg >`basename $i .png`.jpg; done
>
> Also, in regard to some silly objections upthread about the danger of
> binary data on standard ouptut, programs in Unix can easily do the
> Following (and arguably should):
>
> if (isatty(STDOUT_FILENO)) {
> fprintf(stderr, "Cowardly refusing to dump binary data to a terminal.\n");
> exit(EXIT_FAILURE);
> }
>

Yes, so common that the shell has test -t

bart <bc@freeuk.com> writes:
> On 01/02/2024 16:30, Scott Lurndal wrote:
[...]
>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null^Jthen^Jecho "it is a jpeg"^Jfi
>> it is a jpeg
>
> That doesn't work for me:

Not if you type the "^J"s as '^' and 'J'. They were intended to
represent newlines. I would use semicolons instead:

$ if file /tmp/garage.jpg | grep JPEG > /dev/null ; then echo "it is a jpeg" ; fi
it is a jpeg

(I might also use "grep -q" rather than redirecting to /dev/null.)

[...]

> I think anyway that you need to grep for JFIF not JPEG, but that is a
> really poor way to check for a JPEG file. Any text or binary file can
> have a JFIF byte sequence.

That's not an issue. "file" doesn't just look for "JFIF" to determine
that a file is a jpg.

Try running the "file" command on a jpg file. Its output includes both
"JPEG" and "JFIF". Try running it on a text file containing the string
"JFIF".

The output of "file" is primarily meant to be human-readable, so
processing it automatically can be tricky, but it's usually doable --
and it also has a "--mime" option if you want more specificity.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
[...]
> Some things are interesting in themselves and worth talking about at
> lenght. Like how Haskell builds up functions of functions. Other
> things really aren't. And how to set up a Unix pipeline is one of
> those that really aren't (unless actually faced with a such a system
> and with a practical need to do it).

That's your opinion. You seem to be expecting others to find it correct
and intuitively obvious if they'd only *think* about it. You're wrong.
You won't change your mind.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

bart <bc@freeuk.com> writes:
[...]
> I've just realised why it is that your filter programs don't show
> prompts or any kinds of messages: because those are sent to stdout,
> and therefore will screw up any data that is being sent there as the
> primary output.

Yes! (But your use of "your" is odd. Nobody here owns those programs.)

> THAT'S why sending what ought to be packaged data to stdout is Wrong.

No.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
>bart <bc@freeuk.com> writes:
>> On 01/02/2024 16:30, Scott Lurndal wrote:
>[...]
>>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null^Jthen^Jecho "it is a jpeg"^Jfi
>>> it is a jpeg
>>
>> That doesn't work for me:
>
>Not if you type the "^J"s as '^' and 'J'. They were intended to
>represent newlines. I would use semicolons instead:

Yes, that's an artifact of ksh history entries for multiline commands.

I should have edited it before posting.

On 01/02/2024 20:09, Keith Thompson wrote:
> bart <bc@freeuk.com> writes:
>> On 01/02/2024 16:30, Scott Lurndal wrote:
> [...]
>>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null^Jthen^Jecho "it is a jpeg"^Jfi
>>> it is a jpeg
>>
>> That doesn't work for me:
>
> Not if you type the "^J"s as '^' and 'J'. They were intended to
> represent newlines. I would use semicolons instead:
>
> $ if file /tmp/garage.jpg | grep JPEG > /dev/null ; then echo "it is a jpeg" ; fi
> it is a jpeg
>
> (I might also use "grep -q" rather than redirecting to /dev/null.)
>
> [...]
>
>> I think anyway that you need to grep for JFIF not JPEG, but that is a
>> really poor way to check for a JPEG file. Any text or binary file can
>> have a JFIF byte sequence.
>
> That's not an issue. "file" doesn't just look for "JFIF" to determine
> that a file is a jpg.

I see, so 'file' is a special command that does all the work. grep
checks whether the description contains JPEG. Although it won't work for
any of my private formats.

On 2/1/2024 1:25 PM, bart wrote:
> On 01/02/2024 20:09, Keith Thompson wrote:
>> bart <bc@freeuk.com> writes:
>>> On 01/02/2024 16:30, Scott Lurndal wrote:
>> [...]
>>>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null^Jthen^Jecho "it is
>>>> a jpeg"^Jfi
>>>> it is a jpeg
>>>
>>> That doesn't work for me:
>>
>> Not if you type the "^J"s as '^' and 'J'.  They were intended to
>> represent newlines.  I would use semicolons instead:
>>
>>      $ if file /tmp/garage.jpg | grep JPEG > /dev/null ; then echo "it
>> is a jpeg" ; fi
>>      it is a jpeg
>>
>> (I might also use "grep -q" rather than redirecting to /dev/null.)
>>
>> [...]
>>
>>> I think anyway that you need to grep for JFIF not JPEG, but that is a
>>> really poor way to check for a JPEG file. Any text or binary file can
>>> have a JFIF byte sequence.
>>
>> That's not an issue.  "file" doesn't just look for "JFIF" to determine
>> that a file is a jpg.
>
> I see, so 'file' is a special command that does all the work. grep
> checks whether the description contains JPEG. Although it won't work for
> any of my private formats.
>
>

Why would it work with your private formats? ;^)

On 01/02/2024 18:35, Janis Papanagnou wrote:
> On 01.02.2024 11:30, David Brown wrote:
>> On 31/01/2024 14:35, Janis Papanagnou wrote:
>>>
>>> First; the EU publishes in all languages of the member states,
>>> for example. (There's no single lingua franca.)
>>
>> Weirdly, while Norway is not in the EU but Sweden and Denmark are, they
>> publish (for some things at least) in Norwegian but not in Swedish or
>> Danish. [...]
>
> Hmm.. - in my ears this sounds strange. I've looked it up and found...
>
> "The EU has 24 official languages:
>
> Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish,
> French, German, Greek, Hungarian, Irish, Italian, Latvian, Lithuanian,
> Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish and
> Swedish."

I can't say I have looked this up myself, or particularly care what
languages are used there. Maybe it only applied to some documents, or
used to apply but no longer does. Maybe some things don't stick rigidly
to the official languages, or maybe different guidelines are used for
internal documents.

On 01/02/2024 17:24, Janis Papanagnou wrote:
> On 01.02.2024 11:34, David Brown wrote:
>> On 31/01/2024 19:35, Keith Thompson wrote:
>>> David Brown <david.brown@hesbynett.no> writes:
>>>>
>>>> I regularly see it as more symmetrical and clearer to push data left
>>>> to right. So I might write "cat infile | grep foo | sort > outfile".
>>>> Of course I could use "<" redirection, but somehow it seems more
>>>> natural to me to have this flow. I'll use "<" for simpler cases.
>>>>
>>>> But perhaps this is just my habit, and makes little sense to other
>>>> people.
>
> I completely understand that.
>
>>>
>>> You can also use:
>>>
>>> < infile grep foo | sort > outfile
>>>
>>> Redirections don't have to be written after a command.
>
> Indeed. And if we also respect that 'grep' accepts arguments,
> then it's even more compact and yet probably better legible... :-)
>
> grep foo infile | sort > outfile
>
>
>> I did not know you could write it that way - thanks for another
>> off-topic, but useful, tip.
>
> Yes. We certainly should instead have written
>
> grep foo iso646.h | sort > outfile
>
>

I'm happy using different arrangements at different times. Sometimes I
think one way is clearer, or easier to type, sometimes I think another
way is better. As a vague rule, I will usually use "grep foo infile" if
it is stand alone, or at most piped into "less". If I have a larger
chain, it seems more natural to me to move the data left to right in a pipe.

I'm sure I cost my computer a few microseconds of extra effort, but I
don't worry too much about that!

On 01/02/2024 18:06, bart wrote:
> On 01/02/2024 14:50, David Brown wrote:
>> On 01/02/2024 02:29, bart wrote:
>>> On 01/02/2024 00:47, Scott Lurndal wrote:
>>>> bart <bc@freeuk.com> writes:
>>>>> On 31/01/2024 23:36, Ben Bacarisse wrote:
>>>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>
>>>>>> According to you, these tools are poorly designed.  I don't think so.
>>>>>> How would you design them?  Endless input and output file names to be
>>>>>> juggled and tidied up afterwards?
>>>>>
>>>>> I think they're poorly designed too.
>>>>
>>>> Of course you do.   They're not bart programs.
>>>>
>>>>>
>>>>>  From the POV of interactive console programs, they /are/ poor.
>>>>
>>>> You don't provide any reason why - do elucidate!
>>>
>>> They only do one thing, like you can't first do A, then B. They don't
>>> give any prompts. They often apparently do nothing (so you can't tell
>>> if they're busy, waiting for input, or hanging). There is no dialog.
>>
>> That's the whole point!
>>
>> If you want to do A, then B, then you do "A | B", or "A; B", or "A &&
>> B" or "A || B".  And if you want to do A, then B twice, then C, then A
>> again, you write "A | B | B | C | A".  Other operator choices let you
>> say "do this then that", or "do this, and if successful do that", etc.
>>
>> Your monolithic AB program fails when you want to do C, or want to do
>> A and B in a way the AB author didn't envisage.
>>
>> You have a Transformer - a toy that can be either a car or a robot.
>> I've got a box of Lego.  Sometimes I need instructions and a bit of
>> time, but I can have a car, a robot, a plane, an alien, a house, and
>> anything else I might want.
>
>
> You can only do one thing, as you can only have one unbroken byte
> sequence as output sent to stdout.

I can do one sequence of many things.

Your suggested "clipboard" idea was no different.

But it's not difficult to have intermediary files if you want to do more
complicated things.

>
> You can't send output A to stdout, then B to stdout, and certainly can't
> interleave messages to the console on stdout, as that would then be all
> mixed up with the possibly binary data, and if redirected, you won't see
> it.

$ cat A
one
two
three

$ cat B

cat
dog
cow

$ (cat A; cat B) | wc -l
6

That's the output of two commands, "cat A" and "cat B", each going to
their stdout, and they are concatenated into a single pipe going to the
"wc -l" command to count the lines.

And if I wanted to redirect them to a file "x" and also view them, I'd
write :

$ (cat A; cat B) | tee x
one
two
three
cat
dog
cow

$ wc -l x
6 x

I'm not sure we are getting anywhere with you trying to invent more and
more complex situations in an attempt to find something that can't be
done from a Linux bash shell.

>
> I can see the idea of having one permanently open channel, but call it
> stdbinout or stdpipeout. But you still won't be able to generate a
> sequence of distinct data blocks along that one channel because it is
> continuous.
>
> This why 'as' only ever produces one object file, even for multiple
> input source files.

"as" produces one object file, because that's what the program does. If
you want two object files, run it twice. In Unix systems, starting
programs and running lots of programs at a time is cheap. It's not a
system that requires monolithic programs in order to work efficiently.

>
> And explains why 'as' treats multiple .s input files as though they were
> all part of the same single source file: you can take one .s file, chop
> it up into multiple .s files, and submit them all to 'as' (keeping the
> right order).

It does that because that's what makes sense. If you want to assembly
multiple .s files into individual .o files, then you do that. If you
want to assemble them into a single .o file, then you do that. Your
choice. Having it generate multiple .o files for multiple .s inputs
would restrict that choice.

>
> It's a feature! It's also the whackiest assembler I've encountered, this
> century anyway. That fact that it's implemented as a crude filter with
> one input stream and one output streams helps explain it.
>
> Although it works differently from most such filters, because if its
> output is not piped, and not redirected, it is sent to a file (always
> called a.out). It's not quite crazy enough to send binary object file
> data to the termimal; I wonder why not?

You really are scraping the bottom of the barrel to try to justify your
irrational hatreds, aren't you? You put a lot of effort into
desperately trying to dislike programs that don't work exactly the way
your programs work. It's a very strange hobby you have.

On 01/02/2024 16:41, Malcolm McLean wrote:
> On 01/02/2024 14:55, David Brown wrote:
>> On 01/02/2024 02:53, Malcolm McLean wrote:
>>> On 31/01/2024 23:36, Ben Bacarisse wrote:
>>>>
>>>> An example where it's really useful not to care: I have a suite of
>>>> tools
>>>> for doing toy cryptanalysis.  Some apply various transformations and/or
>>>> filters to byte streams and others collect and output (on stderr)
>>>> various statistics.  Plugging them together in various pipelines is
>>>> very
>>>> handy when investigating an encrypted text.  The output is almost
>>>> always
>>>> "binary" in the sense that there would be not point in looking at on a
>>>> terminal.
>>>>
>>>> According to you, these tools are poorly designed.  I don't think so.
>>>> How would you design them?  Endless input and output file names to be
>>>> juggled and tidied up afterwards?
>>>>
>>> I'd write a monolithic program.
>>
>> It's very strange to me to see people that consider themselves
>> programmers talk about having multiple small functions to do specific
>> tasks and combining them into bigger functions to solve bigger
>> problems, yet are reduced to quivering jellies at the thought of
>> multiple small programs to do specific tasks that can be combined to
>> solve bigger tasks.
>>
>> Do you think the C standard library would be improved by a single
>> function "flubadub" that takes 20 parameters and can calculate
>> logarithms, print formatted text, allocate memory and write it all to
>> a file?
>>
> By breaking down the problem into several parts e.g. "collect
> statistical data, analyse statistics, form hypothesis, attempt
> decryption, check decrypt for plausible plaintext" we can usually attack
> it better. And you're right, there's not a fundamental difference
> between writing one program with five subroutines, or five programs
> which pass data to each other via pipelines.

That's not what I said. Try re-reading. I can't be bothered arguing
against yet another straw man.

> But that doesn't mean that decision must not be made, or that you can't
> give reasons for and against each option.
>
> So could you list one or two reasons why you might prefer a program with
> five subroutines, and one or two reasons why you might prefer to write
> five programs which communicate via piped data?

bart <bc@freeuk.com> writes:
>On 01/02/2024 20:09, Keith Thompson wrote:
>> bart <bc@freeuk.com> writes:
>>> On 01/02/2024 16:30, Scott Lurndal wrote:
>> [...]
>>>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null^Jthen^Jecho "it is a jpeg"^Jfi
>>>> it is a jpeg
>>>
>>> That doesn't work for me:
>>
>> Not if you type the "^J"s as '^' and 'J'. They were intended to
>> represent newlines. I would use semicolons instead:
>>
>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null ; then echo "it is a jpeg" ; fi
>> it is a jpeg
>>
>> (I might also use "grep -q" rather than redirecting to /dev/null.)
>>
>> [...]
>>
>>> I think anyway that you need to grep for JFIF not JPEG, but that is a
>>> really poor way to check for a JPEG file. Any text or binary file can
>>> have a JFIF byte sequence.
>>
>> That's not an issue. "file" doesn't just look for "JFIF" to determine
>> that a file is a jpg.
>
>I see, so 'file' is a special command that does all the work. grep
>checks whether the description contains JPEG. Although it won't work for
>any of my private formats.

Like anything in unix, the 'file(1)' command is flexible.

There is a file /etc/magic, that an installation can use
to describe custom file formats.

"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>On 2/1/2024 1:25 PM, bart wrote:
>> On 01/02/2024 20:09, Keith Thompson wrote:
>>> bart <bc@freeuk.com> writes:
>>>> On 01/02/2024 16:30, Scott Lurndal wrote:
>>> [...]
>>>>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null^Jthen^Jecho "it is
>>>>> a jpeg"^Jfi
>>>>> it is a jpeg
>>>>
>>>> That doesn't work for me:
>>>
>>> Not if you type the "^J"s as '^' and 'J'.  They were intended to
>>> represent newlines.  I would use semicolons instead:
>>>
>>>      $ if file /tmp/garage.jpg | grep JPEG > /dev/null ; then echo "it
>>> is a jpeg" ; fi
>>>      it is a jpeg
>>>
>>> (I might also use "grep -q" rather than redirecting to /dev/null.)
>>>
>>> [...]
>>>
>>>> I think anyway that you need to grep for JFIF not JPEG, but that is a
>>>> really poor way to check for a JPEG file. Any text or binary file can
>>>> have a JFIF byte sequence.
>>>
>>> That's not an issue.  "file" doesn't just look for "JFIF" to determine
>>> that a file is a jpg.
>>
>> I see, so 'file' is a special command that does all the work. grep
>> checks whether the description contains JPEG. Although it won't work for
>> any of my private formats.
>>
>>
>
>Why would it work with your private formats? ;^)

It will. He needs to describe the classification criteria
in /etc/magic.

On 01/02/2024 22:06, David Brown wrote:
> On 01/02/2024 16:41, Malcolm McLean wrote:
>> On 01/02/2024 14:55, David Brown wrote:
>>> On 01/02/2024 02:53, Malcolm McLean wrote:
>>>> On 31/01/2024 23:36, Ben Bacarisse wrote:
>>>>>
>>>>> An example where it's really useful not to care: I have a suite of
>>>>> tools
>>>>> for doing toy cryptanalysis.  Some apply various transformations
>>>>> and/or
>>>>> filters to byte streams and others collect and output (on stderr)
>>>>> various statistics.  Plugging them together in various pipelines is
>>>>> very
>>>>> handy when investigating an encrypted text.  The output is almost
>>>>> always
>>>>> "binary" in the sense that there would be not point in looking at on a
>>>>> terminal.
>>>>>
>>>>> According to you, these tools are poorly designed.  I don't think so.
>>>>> How would you design them?  Endless input and output file names to be
>>>>> juggled and tidied up afterwards?
>>>>>
>>>> I'd write a monolithic program.
>>>
>>> It's very strange to me to see people that consider themselves
>>> programmers talk about having multiple small functions to do specific
>>> tasks and combining them into bigger functions to solve bigger
>>> problems, yet are reduced to quivering jellies at the thought of
>>> multiple small programs to do specific tasks that can be combined to
>>> solve bigger tasks.
>>>
>>> Do you think the C standard library would be improved by a single
>>> function "flubadub" that takes 20 parameters and can calculate
>>> logarithms, print formatted text, allocate memory and write it all to
>>> a file?
>>>
>> By breaking down the problem into several parts e.g. "collect
>> statistical data, analyse statistics, form hypothesis, attempt
>> decryption, check decrypt for plausible plaintext" we can usually
>> attack it better. And you're right, there's not a fundamental
>> difference between writing one program with five subroutines, or five
>> programs which pass data to each other via pipelines.
>
> That's not what I said.  Try re-reading.  I can't be bothered arguing
> against yet another straw man.
>
>> But that doesn't mean that decision must not be made, or that you
>> can't give reasons for and against each option.
>>
>> So could you list one or two reasons why you might prefer a program
>> with five subroutines, and one or two reasons why you might prefer to
>> write five programs which communicate via piped data?
>
I'm sure you're capable of going through the exercise and then you might
gain a bit of insight on how to design such software systems. And, no,
arguing that you'd go for a monolithic program doesn't necessarily mean
that you are a "quivering jelly" at the thought of writing several
simpler ones. And in fact to start you off I actually mentioned a few
advantages of the pipeline approach.

There are advantages and drawbacks to both. But I can't force you to
think about what those might be if you won't, and from experience just
telling you provokes your natural contentiousness and isn't very
effective.

It's reasonable to write a function flubadub which woirks as you say.
It's not going to be a good candidate for the standard library as it
would be too specific to a particular task. Unless it has several modes,
making it effectively several functions with a common call address.
That's not really similar to a monolithic program, even if it does have
several "modes", because the modes would be related. You might have one
mode for attempting decryption and another mode for achiving intercepts,
and because they operate on the same archive, keeping the source in the
same physical program prevents the archiver and the archive reader
getting into incompatible versions. (There's an advantage of monolithic
programs for the list). However you wouldn't write a monolithic program
to decryot and to play space invaders. At least not normally.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

On 2/1/2024 2:25 PM, Scott Lurndal wrote:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>> On 2/1/2024 1:25 PM, bart wrote:
>>> On 01/02/2024 20:09, Keith Thompson wrote:
>>>> bart <bc@freeuk.com> writes:
>>>>> On 01/02/2024 16:30, Scott Lurndal wrote:
>>>> [...]
>>>>>> $ if file /tmp/garage.jpg | grep JPEG > /dev/null^Jthen^Jecho "it is
>>>>>> a jpeg"^Jfi
>>>>>> it is a jpeg
>>>>>
>>>>> That doesn't work for me:
>>>>
>>>> Not if you type the "^J"s as '^' and 'J'.  They were intended to
>>>> represent newlines.  I would use semicolons instead:
>>>>
>>>>      $ if file /tmp/garage.jpg | grep JPEG > /dev/null ; then echo "it
>>>> is a jpeg" ; fi
>>>>      it is a jpeg
>>>>
>>>> (I might also use "grep -q" rather than redirecting to /dev/null.)
>>>>
>>>> [...]
>>>>
>>>>> I think anyway that you need to grep for JFIF not JPEG, but that is a
>>>>> really poor way to check for a JPEG file. Any text or binary file can
>>>>> have a JFIF byte sequence.
>>>>
>>>> That's not an issue.  "file" doesn't just look for "JFIF" to determine
>>>> that a file is a jpg.
>>>
>>> I see, so 'file' is a special command that does all the work. grep
>>> checks whether the description contains JPEG. Although it won't work for
>>> any of my private formats.
>>>
>>>
>>
>> Why would it work with your private formats? ;^)
>
> It will. He needs to describe the classification criteria
> in /etc/magic.

Oh, I totally forgot about that!

https://www.ibm.com/docs/en/zos/2.1.0?topic=formats-magic-format-etcmagic-file

On 01/02/2024 22:02, David Brown wrote:
> On 01/02/2024 18:06, bart wrote:

>> You can't send output A to stdout, then B to stdout, and certainly
>> can't interleave messages to the console on stdout, as that would then
>> be all mixed up with the possibly binary data, and if redirected, you
>> won't see it.
>
> $ cat A
> one
> two
> three
>
> $ cat B
>
> cat
> dog
> cow
>
> $ (cat A; cat B) | wc -l
> 6
>
> That's the output of two commands, "cat A" and "cat B", each going to
> their stdout, and they are concatenated into a single pipe going to the
> "wc -l" command to count the lines.

I see you don't get it. This is the equivalent of a program which is
supposed to do this:

print A to TTY
print B to LPT
print C to TTY
print D to LPT

but instead is written as this:

print A to TTY
print B to TTY
print C to TTY
print D to TTY

and you are expected to redirect all TTY output to LPT.

At least, on LPT, B and D can each start with a separate title page; on
stdout directed to a file, it will be all mixed up.

> I'm not sure we are getting anywhere with you trying to invent more and
> more complex situations in an attempt to find something that can't be
> done from a Linux bash shell.

They're remarkably simple situations!

>> And explains why 'as' treats multiple .s input files as though they
>> were all part of the same single source file: you can take one .s
>> file, chop it up into multiple .s files, and submit them all to 'as'
>> (keeping the right order).
>
> It does that because that's what makes sense.

Sorry, but it is rubbish.

> Having it generate multiple .o files for multiple .s inputs
> would restrict that choice.

And yet that is exactly what gcc does; see below.

> You really are scraping the bottom of the barrel to try to justify your
> irrational hatreds, aren't you?  You put a lot of effort into
> desperately trying to dislike programs that don't work exactly the way
> your programs work.

Because their UIs are rubbish. They are inconsistent. They are
restricted. And yet they are deified for some inexplicable reason. Over
the past few decades nobody has written a better assembler?

At least there are external ones you can use instead, but gcc will not
generate .s files in the right syntax (I guess that's another tool you
will pull of that bottomless bag of such tools).

>  It's a very strange hobby you have.

I write language tools. Ones which are always derided in this newsgroup.
They've included quite a few assemblers.

And yet they all have sensible command line interfaces that do what you
expect.

Which is more that can be said for gcc and especially 'as'.

If you do this on gcc:

gcc one.s two.s

it will create one.o and two.s. Do this on as:

as one.s two.s

it will not only create one file a.out, but will concatenate both,
giving unexpected results:

c:\c>gcc -S cipher.c hmac.c sha2.c

c:\c>as cipher.s hmac.s sha2.s
hmac.s: Assembler messages:
hmac.s:328: Error: symbol `.L18' is already defined
hmac.s:352: Error: symbol `.L17' is already defined
...

WTF? Compare with this:

c:\c>mcc -s cipher hmac sha2
Compiling 3 files to .asm

c:\c>aa cipher hmac sha2
Assembling cipher.asm to cipher.exe

It works impeccably. Even better than 'as', even if that had worked as
expected, because there you still have the task of linking the outputs.

scott@slp53.sl.home (Scott Lurndal) writes:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>>On 2/1/2024 1:25 PM, bart wrote:
[...]
>>> I see, so 'file' is a special command that does all the work. grep
>>> checks whether the description contains JPEG. Although it won't work for
>>> any of my private formats.
>>
>>Why would it work with your private formats? ;^)
>
> It will. He needs to describe the classification criteria
> in /etc/magic.

Modifying system files like that is a bad idea. Fortunately, the "file"
command supports mechanisms for specifying an alternate "magic" file.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On Wed, 31 Jan 2024 15:10:05 +0200, Michael S wrote:

> their earlier mainframe line (PDP-6/10/20).

Nitpick: there was PDP-6 and PDP-10, but never “PDP-20”. There were
systems sold as “DECsystem-10” and “DECsystem-20”, but they were all based
on PDP-10 hardware.

The difference was I think primarily down to the OS: the “DECsystem-10”
installations shipped with TOPS-10, while the “DECsystem-20” machines
shipped with TOPS-20, which was based on the groundbreaking TENEX from
BBN.

TOPS-20 was not in any sense a “successor” to TOPS-10. I suppose it should
have been, but it was too big a shift from the old way of doing things, so
some customers preferred to stick with the clunkier OS. So both the -10
and -20 lines were being sold concurrently.

(End nitpick.)

(Begin extra trivia part.)

The PDP-6 was DEC’s first foray into “large-scale” systems (it had a 36-
bit word length, greater than any earlier DEC machine). They only made a
few, at great expense, and lost money on them. So the line was
discontinued, with the proclamation that they were never getting into 36-
bit machines again.

Then two years later, the architecture was revived as the PDP-10 ...

On Wed, 31 Jan 2024 18:06:13 +0100, Janis Papanagnou wrote:

> Yet I don't understand the relation to Linus Torvalds that was the
> source of mentioning VMS. - I mean; only that he dislikes it is not much
> of a news.

It was the reason he gave for disliking it: you could not easily determine
the length in bytes of a file.

On 01/02/2024 20:18, Keith Thompson wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> [...]
>> Some things are interesting in themselves and worth talking about at
>> lenght. Like how Haskell builds up functions of functions. Other
>> things really aren't. And how to set up a Unix pipeline is one of
>> those that really aren't (unless actually faced with a such a system
>> and with a practical need to do it).
>
> That's your opinion. You seem to be expecting others to find it correct
> and intuitively obvious if they'd only *think* about it. You're wrong.
> You won't change your mind.
>
No not really.
Elsethread people are talking about the precise syntax for redirecting
output in the UNIX shell. Now of course that's off topic for comp.lang.
If we were talking about the precise syntax for redirecting stdout to
another stream in a C program, it would be different. However I'm one to
talk and you coukd say that many of things I like talking abiut are no
more topical.

The difference is that the syntax for redirecting output in the UNIX
shell is ony of the slightest use if you happen to run that particular
type of system. Now I can't use Haskell for any pratical programaming.
It's a total non-starter. But that doesn't mean that we can't look at
the way Haskell composes functions of functions and use that to gain
insights into how to specify functions, as general programming task.
So you can say "But pipes are a concept not necessarily limited to
Unix." OK, fair enough. But what provoked this was "you don't understand
hiw Unix works". And whilst pipes are a concept, they are no way
comparable in depth and fundamental importance to the concept of
functions of functions.

The point is the two are not comparable. It's not a matter of my
subjective tastes or opinion.
--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

On Wed, 31 Jan 2024 14:45:49 +0100, David Brown wrote:

> On 31/01/2024 07:07, Lawrence D'Oliveiro wrote:
>
>> On Tue, 30 Jan 2024 17:25:31 +0100, David Brown wrote:
>>
>>> Mixing binary data with formatted text data is very unlikely to be
>>> useful.
>>
>> PDF does exactly that. To the point where the spec suggests putting
>> some random unprintable bytes up front, to distract format sniffers
>> from thinking they’re looking at a text file.
>
> PDF files start with the "magic" indicator "%PDF", which is enough for
> many programs to identify them correctly.

Sure, if you were looking for PDF files specifically.

But consider the more generic case of file-transfer tools that try to
automatically convert between line-endings for text files on different
platforms: if they mistook a PDF file for text, they could screw it up
royally.

On 01.02.2024 22:41, David Brown wrote:
> On 01/02/2024 18:35, Janis Papanagnou wrote:
>> On 01.02.2024 11:30, David Brown wrote:
>>> On 31/01/2024 14:35, Janis Papanagnou wrote:
>>>>
>>>> First; the EU publishes in all languages of the member states,
>>>> for example. (There's no single lingua franca.)
>>>
>>> Weirdly, while Norway is not in the EU but Sweden and Denmark are, they
>>> publish (for some things at least) in Norwegian but not in Swedish or
>>> Danish. [...]
>>
>> Hmm.. - in my ears this sounds strange. I've looked it up and found...
>>
>> "The EU has 24 official languages:
>>
>> Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish,
>> French, German, Greek, Hungarian, Irish, Italian, Latvian, Lithuanian,
>> Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish and
>> Swedish."
>
> I can't say I have looked this up myself, or particularly care what
> languages are used there. Maybe it only applied to some documents, or
> used to apply but no longer does. Maybe some things don't stick rigidly
> to the official languages, or maybe different guidelines are used for
> internal documents.

Extremely unlikely. - More likely that you were just misremembering
or the document you have in mind was not an official EU document.

(But if you can dig it up and provide that evidence I'm interested.)

Janis

On Thu, 1 Feb 2024 23:02:17 +0100, David Brown wrote:

> But it's not difficult to have intermediary files if you want to do more
> complicated things.

This is the point where someone says “I wish a shell script pipeline could
express a general flow graph”.

..

..

..

..

..

.... nobody?

On 02.02.2024 01:08, Lawrence D'Oliveiro wrote:
>
> But consider the more generic case of file-transfer tools that try to
> automatically convert between line-endings for text files on different
> platforms: if they mistook a PDF file for text, they could screw it up
> royally.

What tools are you specifically thinking of? - I recall in FTP you
explicitly set bin-mode or text-mode. I assume that protocols like
FTAM (CCITT) would also transfer files reliably. I would certainly
try to avoid tools that operate unreliably or can't be switched to
operate correctly with [8 bit] "binary" or [ASCII] text files.

Janis