On 1/31/24 01:12, Lawrence D'Oliveiro wrote:
> On Tue, 30 Jan 2024 19:39:24 +0000, Richard Harnden wrote:
>
>> Nobody uses printf to output binary data.
>
> Do terminal-control escape sequences count?

The standard says "Data read in from a text stream will necessarily
compare equal to the data that were earlier written out to that stream
only if: the data consist only of printing characters and the control
characters horizontal tab and new-line; no new-line character is
immediately preceded by space characters; and the last character is a
new-line character." (7.23.2p2).

I would say that any output which would invalidate that guarantee
qualifies as binary data, since you would need to output it in binary
mode in order to guarantee being able to read it back. That would
include the other control characters (vertical tab and form feed,
backspace, and carriage return).

The letters and digits are guaranteed to be printing characters. Note that:

"A letter is an uppercase letter or a lowercase letter as defined above;
in this document the term does not include other characters that are
letters in other alphabets."

The other printing characters are locale-specific, so you'll have to
test them with isprint(). In particular, many terminal control escape
sequences start with the ESC character, which doesn't qualify above.

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> On 30/01/2024 07:27, Tim Rentsch wrote:
>
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>
>>> On 29/01/2024 20:10, Tim Rentsch wrote:
>>>
>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>
>>>> [...]
>>>>
>>>>> I've never used standard output for binary data.
>>>>> [...] it strikes me as a poor design decision.
>>>>
>>>> How so?
>>>
>>> Because the output can't be inspected by humans, and because it might
>>> have unusual effects if passed though systems designed to handle
>>> human-readable text. For instance in some systems designed to receive
>>> ASCII text, there is no distinction between the nul byte and "waiting
>>> for next data byte". Obviously this will cause difficuties if the data
>>> is binary.
>>> Also many binary formats can't easily be extended, so you can pass one
>>> image and that's all. While it is possible to devise a text format
>>> which is similar, in practice text formats usually have enough
>>> redundancy to be easily extended.
>>>
>>> So it's harder to correct errors, more prone to errors, and harder to
>>> extend.
>>
>> Your reasoning is all gobbledygook. Your comments reflect only
>> limitations in your thinking, not any essential truth about using
>> standard out for binary data.
>
> I must admit that it's nothing I have ever done or considered doing.
>
> [...]

Simple example (disclaimer: not tested):

ssh foo 'cd blah ; tar -cf - . | gzip -c' | \
(mkdir foo.blah ; cd foo.blah ; gunzip -c | tar -xf -)

Of the five main programs in this command, four are using
standard out to send binary data:

tar -cf - .
gzip -c
ssh foo [...]
gunzip -c

The tar -xf - at the end reads binary data on standard in
but doesn't output any (or anything else for that matter).

It is FAR more cumbersome to accomplish what this command
is doing without sending binary data through standard out.
Anyone who doesn't understand this doesn't understand Unix.

On 31/01/2024 07:04, Lawrence D'Oliveiro wrote:
> On Tue, 30 Jan 2024 21:04:56 +0000, Malcolm McLean wrote:
>
>> Linus Torvald's native language is Finnish, for example.
>
> No, it would be Swedish. He’s an ethnic Swede, from Finland.

He is Finnish, but has Swedish as his mother tongue (like about 5% of
Finns). Speaking Swedish as your main language does not make you
ethically Swedish. As a university-educated Finn, brought up in
Helsinki, he will also speak Finnish quite fluently.

On 31/01/2024 07:18, Tim Rentsch wrote:
> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>
>> On 30/01/2024 07:27, Tim Rentsch wrote:
>>
>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>
>>>> On 29/01/2024 20:10, Tim Rentsch wrote:
>>>>
>>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>>
>>>>> [...]
>>>>>
>>>>>> I've never used standard output for binary data.
>>>>>> [...] it strikes me as a poor design decision.
>>>>>
>>>>> How so?
>>>>
>>>> Because the output can't be inspected by humans, and because it might
>>>> have unusual effects if passed though systems designed to handle
>>>> human-readable text. For instance in some systems designed to receive
>>>> ASCII text, there is no distinction between the nul byte and "waiting
>>>> for next data byte". Obviously this will cause difficuties if the data
>>>> is binary.
>>>> Also many binary formats can't easily be extended, so you can pass one
>>>> image and that's all. While it is possible to devise a text format
>>>> which is similar, in practice text formats usually have enough
>>>> redundancy to be easily extended.
>>>>
>>>> So it's harder to correct errors, more prone to errors, and harder to
>>>> extend.
>>>
>>> Your reasoning is all gobbledygook. Your comments reflect only
>>> limitations in your thinking, not any essential truth about using
>>> standard out for binary data.
>>
>> I must admit that it's nothing I have ever done or considered doing.
>>
>> [...]
>
> Simple example (disclaimer: not tested):
>
> ssh foo 'cd blah ; tar -cf - . | gzip -c' | \
> (mkdir foo.blah ; cd foo.blah ; gunzip -c | tar -xf -)
>
> Of the five main programs in this command, four are using
> standard out to send binary data:
>
> tar -cf - .
> gzip -c
> ssh foo [...]
> gunzip -c
>
> The tar -xf - at the end reads binary data on standard in
> but doesn't output any (or anything else for that matter).
>
> It is FAR more cumbersome to accomplish what this command
> is doing without sending binary data through standard out.
> Anyone who doesn't understand this doesn't understand Unix.
>
Yes. I don't do that sort of thing.
Whilst I have used Unix, it is as a platform for interactive programs
which work on graohics, or a general C compilation environment. I don;t
build pipeliens to do that sort of data processing. If I had to download
a tar file I'd either use a graphical tool or type serveal commands into
the shell, each launching single executable, interactively.

The reason is that I'd only run the command once, and it's so likely
that there will be either a syntax misunderstanding or a typing error
that I'd have to test to ensure that it was right. And by the time
you've done that any time saved by typing only one commandline is lost.
Of course if you are writing scripts then that doesn't apply. But now
it's effectively a programming language, and, from the example code, a
very poorly designed one which is cryptic and fussy and liable to be
hard to maintain. So it's better to use a language like Perl to achieve
the same thing, and I did have a few Perl scripts handy for repetitive
jobs of that nature in my Unix days.

You admit this with "not tested". Says it all. '"Understandig Unix" is
an intellectually useless achievement. You might have to do it if you
have to use the system and debug and trouble shoot. But it's nothing to
be proud about.

--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

On Tue, 30 Jan 2024 23:18:21 -0800
Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:

> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>
> > On 30/01/2024 07:27, Tim Rentsch wrote:
> >
> >> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> >>
> >>> On 29/01/2024 20:10, Tim Rentsch wrote:
> >>>
> >>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
> >>>>
> >>>> [...]
> >>>>
> >>>>> I've never used standard output for binary data.
> >>>>> [...] it strikes me as a poor design decision.
> >>>>
> >>>> How so?
> >>>
> >>> Because the output can't be inspected by humans, and because it
> >>> might have unusual effects if passed though systems designed to
> >>> handle human-readable text. For instance in some systems
> >>> designed to receive ASCII text, there is no distinction between
> >>> the nul byte and "waiting for next data byte". Obviously this
> >>> will cause difficuties if the data is binary.
> >>> Also many binary formats can't easily be extended, so you can
> >>> pass one image and that's all. While it is possible to devise a
> >>> text format which is similar, in practice text formats usually
> >>> have enough redundancy to be easily extended.
> >>>
> >>> So it's harder to correct errors, more prone to errors, and
> >>> harder to extend.
> >>
> >> Your reasoning is all gobbledygook. Your comments reflect only
> >> limitations in your thinking, not any essential truth about using
> >> standard out for binary data.
> >
> > I must admit that it's nothing I have ever done or considered doing.
> >
> > [...]
>
> Simple example (disclaimer: not tested):
>
> ssh foo 'cd blah ; tar -cf - . | gzip -c' | \
> (mkdir foo.blah ; cd foo.blah ; gunzip -c | tar -xf -)
>
> Of the five main programs in this command, four are using
> standard out to send binary data:
>
> tar -cf - .
> gzip -c
> ssh foo [...]
> gunzip -c
>
> The tar -xf - at the end reads binary data on standard in
> but doesn't output any (or anything else for that matter).
>
> It is FAR more cumbersome to accomplish what this command
> is doing without sending binary data through standard out.

If I am not mistaken, tar, gzip and gunzip do not write binary data to
standard output by default. They should be specifically told to do so.
For ssh I don't know. Anyway, ssh is not a "normal" program so it's
not surprising when textuality of ssh output is the same as textuality
of the command it carries.

> Anyone who doesn't understand this doesn't understand Unix.

Frankly, Unix redirection racket looks like something hacked together
rather than designed as result of the solid thinking process.
As long as there were only standard input and output it was sort of
logical. But when they figured out that it is insufficient, they had
chosen a quick hack instead of constructing a solution that wouldn't
offend engineering senses of any non-preconditioned observer.

On Wed, 31 Jan 2024 08:59:22 +0100
David Brown <david.brown@hesbynett.no> wrote:

> On 31/01/2024 07:04, Lawrence D'Oliveiro wrote:
> > On Tue, 30 Jan 2024 21:04:56 +0000, Malcolm McLean wrote:
> >
> >> Linus Torvald's native language is Finnish, for example.
> >
> > No, it would be Swedish. He’s an ethnic Swede, from Finland.
>
> He is Finnish, but has Swedish as his mother tongue (like about 5% of
> Finns). Speaking Swedish as your main language does not make you
> ethically Swedish.

Linus has Swedish as his mother tongue.
Linus has Swedish family name. Or at least Scandinavian, for me it
sounds more Danish than Swedish, but I am not an expert. It certainly
does not sound Finnish.
When Linus was younger, he used to like to tell stereotypical jokes
about Finns.

When it quacks like a duck...

> As a university-educated Finn, brought up in
> Helsinki, he will also speak Finnish quite fluently.
>

That's true.

On 30.01.2024 20:46, David Brown wrote:
> On 30/01/2024 19:22, bart wrote:
>> [...]
>
> If you have a program like that, then it probably makes sense to have a
> flag to say "output the data to stdout" and the default being writing to
> a file.

Did you here mean to say "output the data to the terminal"? (I noticed
that a lot of the posts here have a misconception about what 'stdout'
is; they seem to use it synonymously to "terminal" or "screen/display".
But you are not guaranteed that stdout will produce screen output; it
depends on the environment. Being more accurate with the distinction
might help prevent misconceptions if replying to these people.)

More to your point you wrote, I don't think this would be a good design
as you've written it. A default would imply the necessity of some fixed
name (or naming schema) - think of the disputable "a.out" default. What
would be the default name (or naming scheme) for a tool's data output?
(I could make a maybe even sensible choice for _one_ tool, like using
the base name of a file name argument for cc's output file entity in
the special case of compiling a "main module". Similar the schema '.o'
when used with option -c . But generally?)

Tools that output their data to the [default] stdout channel have many
options; they may redirect that output, or they may pipe it into other
programs, or they may also just suppress the data, unless they want to
see it on their terminal display. Any quite some tools surely (where
it's sensible) provide options to define a output file name in addition
to the default stdout.

And there's even some conventions for tools that require a file name
as (also output) argument, using '-' to let it go to standard output,
or /dev/tty to let it go to the terminal console. (It depends on the
OS, of course.) This all said from the Unix perspective. Users with
experiences only from other (often more primitive) OSes might have
some problems to recognize such design principles.

For sure we have tools that inherently operate with binary data and of
course use the same I/O-channels for binary that a text processing tool
uses for text interchange.

There's no compelling reason why there should be _additional_ channels
for "binary" data exchange; and I put "binary" in quotes for reasons we
already spoke about.

Every interface has a defined input and a defined output mechanism. Use
them as they are specified. If you're calling, say (to keep it simply),
'cat some_image.jpg' what do people expect, a textual description of
the image? (Which would certainly be a nice feature in our AI hype era.
But then we'd probably rather use 'describe_image some_image.jpg' or
any redirected variant thereof. And we would still want binary data
exchanged, to use it as (e.g.) 'AI_image_generator | describe_image'.)

Janis

On 30.01.2024 22:16, James Kuyper wrote:
> On 1/30/24 11:49, David Brown wrote:
> ...
>> If the program can reasonably be expected to generate binary output,
>> then it is the user's fault if they do this accidentally. Examples
>> shown in this thread include cat and zcat - that's what these programs do.
>
> ? There's no problem using cat to concatenate binary files. I've used
> 'split' to split binary files into smaller pieces, and then used 'cat'
> to recombine them, and it worked fine. I don't remember why, but I had
> to transfer the files from one place to another by a method that imposed
> an upper limit on the size of individual files.

Yes. Been there. Faint memories only. But some instances were;
* allowed attachment sizes for email exchange
* posting limits in Usenet (binary files)
* unreliable FTP(?) download processes[*]
* (and something else I can't wrap my mind around at the moment)

Janis

[*] Worth an anecdote...
Downloading standard documents from the CCITT (now ITU-T).
They decided to provide the documents in [non-standard] "MS Word"
'doc' format. Short standard texts in blown up doc documents could
not reliably be transmitted due to connection drops; splitting that
binary doc could have helped (as any more sensible standard format).
But since the original data was a bulky monolith, to no avail.

On 31.01.2024 07:02, Lawrence D'Oliveiro wrote:
> On Tue, 30 Jan 2024 11:50:35 -0800, Keith Thompson wrote:
>
>> VMS (now OpenVMS) was also a significant system at the time, and it had
>> some rather complex file formats ...
>
> [...]
>
> Apparently Linus Torvalds used VMS for a while, and hated it.

I don't understand the intention of this comment.
VMS and Torvalds are completely different eras.
And were is the relation?

(Or just meant as anecdotal trivia?)

Janis

On Wed, 31 Jan 2024 12:43:04 +0100
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

> On 31.01.2024 07:02, Lawrence D'Oliveiro wrote:
> > On Tue, 30 Jan 2024 11:50:35 -0800, Keith Thompson wrote:
> >
> >> VMS (now OpenVMS) was also a significant system at the time, and
> >> it had some rather complex file formats ...
> >
> > [...]
> >
> > Apparently Linus Torvalds used VMS for a while, and hated it.
>
> I don't understand the intention of this comment.
> VMS and Torvalds are completely different eras.
> And were is the relation?
>
> (Or just meant as anecdotal trivia?)
>
> Janis
>

Linus is older than you probably realize. He entered the University of
Helsinki in 1988. Back then VMS was only slightly behind its peak of
popularity. By value, likely still bigger than all Unixen combined.

On 31/01/2024 10:43, Michael S wrote:
> On Tue, 30 Jan 2024 23:18:21 -0800
> Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>
>>> On 30/01/2024 07:27, Tim Rentsch wrote:
>>>
>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>
>>>>> On 29/01/2024 20:10, Tim Rentsch wrote:
>>>>>
>>>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>>>
>>>>>> [...]
>>>>>>
>>>>>>> I've never used standard output for binary data.
>>>>>>> [...] it strikes me as a poor design decision.
>>>>>>
>>>>>> How so?
>>>>>
>>>>> Because the output can't be inspected by humans, and because it
>>>>> might have unusual effects if passed though systems designed to
>>>>> handle human-readable text. For instance in some systems
>>>>> designed to receive ASCII text, there is no distinction between
>>>>> the nul byte and "waiting for next data byte". Obviously this
>>>>> will cause difficuties if the data is binary.
>>>>> Also many binary formats can't easily be extended, so you can
>>>>> pass one image and that's all. While it is possible to devise a
>>>>> text format which is similar, in practice text formats usually
>>>>> have enough redundancy to be easily extended.
>>>>>
>>>>> So it's harder to correct errors, more prone to errors, and
>>>>> harder to extend.
>>>>
>>>> Your reasoning is all gobbledygook. Your comments reflect only
>>>> limitations in your thinking, not any essential truth about using
>>>> standard out for binary data.
>>>
>>> I must admit that it's nothing I have ever done or considered doing.
>>>
>>> [...]
>>
>> Simple example (disclaimer: not tested):
>>
>> ssh foo 'cd blah ; tar -cf - . | gzip -c' | \
>> (mkdir foo.blah ; cd foo.blah ; gunzip -c | tar -xf -)
>>
>> Of the five main programs in this command, four are using
>> standard out to send binary data:
>>
>> tar -cf - .
>> gzip -c
>> ssh foo [...]
>> gunzip -c
>>
>> The tar -xf - at the end reads binary data on standard in
>> but doesn't output any (or anything else for that matter).
>>
>> It is FAR more cumbersome to accomplish what this command
>> is doing without sending binary data through standard out.
>
> If I am not mistaken, tar, gzip and gunzip do not write binary data to
> standard output by default. They should be specifically told to do so.
> For ssh I don't know. Anyway, ssh is not a "normal" program so it's
> not surprising when textuality of ssh output is the same as textuality
> of the command it carries.
>
>> Anyone who doesn't understand this doesn't understand Unix.
>
> Frankly, Unix redirection racket looks like something hacked together
> rather than designed as result of the solid thinking process.
> As long as there were only standard input and output it was sort of
> logical. But when they figured out that it is insufficient, they had
> chosen a quick hack instead of constructing a solution that wouldn't
> offend engineering senses of any non-preconditioned observer.
>
It was designed for very memory constrained systems which handled text
on a line by line basis. So one line of a long file wuld be processed
and passed down the pipeline, and you wouldn't need temporary disk files
or large amounts of memory. I'm sure it worked quite well for that.

--
Check out Basic Algorithms and my other books:
https://www.lulu.com/spotlight/bgy1mm

On 31.01.2024 12:58, Michael S wrote:
> On Wed, 31 Jan 2024 12:43:04 +0100
> Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>
>> On 31.01.2024 07:02, Lawrence D'Oliveiro wrote:
>>> On Tue, 30 Jan 2024 11:50:35 -0800, Keith Thompson wrote:
>>>
>>>> VMS (now OpenVMS) was also a significant system at the time, and
>>>> it had some rather complex file formats ...
>>>
>>> [...]
>>>
>>> Apparently Linus Torvalds used VMS for a while, and hated it.
>>
>> I don't understand the intention of this comment.
>> VMS and Torvalds are completely different eras.
>> And were is the relation?
>>
>
> Linus is older than you probably realize.

Why do you think that I'd be thinking that?

I know that he's quite some years younger than I am. So what?

> He entered the University of
> Helsinki in 1988. Back then VMS was only slightly behind its peak of
> popularity.

What? - I'm not sure where you're coming from.

I associate DEC's VMS with the old DEC VAX-11 system, both
from around the mid of the 1970's. I programmed on a DEC's
VAX with VMS obviously before Linus Torvalds started his
studies. And that was at a time when the DEC VAX and VMS
were replaced at our sites by Unix systems.

> By value, likely still bigger than all Unixen combined.

Not sure what (to me strange sounding) ideas you have here.

Janis

On 31.01.2024 13:15, Malcolm McLean wrote:
> On 31/01/2024 10:43, Michael S wrote:
>>
>> Frankly, Unix redirection racket looks like something hacked together
>> rather than designed as result of the solid thinking process.
>> As long as there were only standard input and output it was sort of
>> logical. But when they figured out that it is insufficient, they had
>> chosen a quick hack instead of constructing a solution that wouldn't
>> offend engineering senses of any non-preconditioned observer.

(What a bullshit.)

>>
> It was designed for very memory constrained systems which handled text
> on a line by line basis. So one line of a long file wuld be processed
> and passed down the pipeline, and you wouldn't need temporary disk files
> or large amounts of memory. I'm sure it worked quite well for that.

You are right about the intention of pipes. (Though not only for
"memory constrained systems", but generally to avoid unnecessary and
costly disk I/O.)

But it's not line oriented. (That would be non-performant.) In Unix
systems there's a distinction between line-oriented and buffered I/O.
You can configure I/O with what you want if you program your tools.
Typically tools when in non-interactive mode use efficient buffered
mode. And there's also methods to force buffered I/O to become line
oriented (cf. 'pty') for specific cases where you want it.

Janis

On 30.01.2024 22:04, Malcolm McLean wrote:
> Linus Torvald's native language is Finnish, for example. But git was
> released in English. There might be Finnish language bindings for it
> now, but I'm pretty sure not in the original version. Similarly Bjarne
> Strousup is Swedish, but C++ uses keywords like "class" and "friend",
> not the Swedish terms.

Okay something had already been said about Linus T.'s ethnicity,
so let me also point out that Bjarne Stroustrup (sp!) is Danish.
(Just for the record.)

Janis

On 30/01/2024 22:04, Malcolm McLean wrote:
> On 30/01/2024 20:06, Keith Thompson wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>> On 29/01/2024 21:00, Keith Thompson wrote:
>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>> On 29/01/2024 16:18, David Brown wrote:
>>>>>> On 28/01/2024 20:49, Malcolm McLean wrote:
>>>>>>> On 28/01/2024 18:24, David Brown wrote:
>>>>>>> I'd expect that most general purpose programs written by Norwegians
>>>>>>> use an English interface, even if it isn't really expected that the
>>>>>>> program will find an audience beyond some users in Norway. Except
>>>>>>> of course for programs which in some way are about Norway.
>>>>>> Why?
>>>>>>
>>>>> Generally programmers are educated people and educated people use
>>>>> English for serious purposes. Not always of course and Norway might be
>>>>> an exception. But I'd expect that in a Norweigian university, for
>>>>> example, it would be forbidden to document a program in Norwegian or
>>>>> to use non-English words for identifiers. And probably the same in a
>>>>> large Norwegina company. I might be wrong about that and I have never
>>>>> visited Norway or worked for a Norweigian employer (and obviously I
>>>>> couldn't do so unless the policy I expect was followed).
>>>> You assert that "educated people use English for serious purposes".
>>>> I don't have the experience to refute that claim, but I suspect
>>>> it's arrogant nonsense.  I could be wrong, of course.
>>>
>>> Everything which is at all intellectually serious is these days
>>> written in English. It's the new Latin. It's the language all educated
>>> people use to communicate with each other when discussing scientific,
>>> philosophical, or scholarly matters. And also technical matters to a
>>> large extent.
>>
>> Even if that's true, you assertion was about user interfaces.
>>
>> Are you under the impression that mobile phones show their messages only
>> in English because all their users are scholars?
>>
> Things tend to trickle down.
> Teenagers from non-English speaking countries hum along to pop songs in
> English.

And you think that makes them fluent, or be as happy communicating in
English than in their own language(s) ?

It is certainly true that in many non-English speaking countries, people
get quite familiar with English, and they do so from a younger age than
in previous generations. It is also true that in many non-English
speaking countries, people /don't/ learn much English unless they are
involved in a profession that needs it.

A guiding factor here is the size of the population. In European
countries with maybe 20-30 million or more, everything important is
translated or dubbed. You can watch American movies in your own
language - they are dubbed. You can read translated books, or natively
written books, even for specialised non-fiction books. You don't need
to understand any English of significance to have a full professional,
cultural and social life.

But it is correct that English has become the main language for
international communication, and is therefore critical for anything that
involves cross-border communication, or where there are significant
numbers of foreign workers. That includes academic work. Different
parts of Europe previously used German or Russian for this, and that is
still found amongst the older generations, but it is changing to English.

I live in Norway, which is a country near the top of any list of English
fluency. It is small (in population), so relies on a lot of imported
culture and information. Films are subtitled, not dubbed (except for
films for small kids), and only popular books are translated. We have a
very high standard of education, including higher education, a great
deal of foreign workers in some fields (in my company, perhaps a third
of employees are not natives), and Norwegians travel extensively. There
can be few non-English speaking countries where English is significantly
more prevalent than Norway.

Yet be in no doubt that the solid majority of Norwegians prefer
Norwegian for most purposes.

Yes, pretty much any programmer in Norway will be confident in English.
So will most scholars. But the huge majority of people in Norway are
not programmers or scholars.

Most Norwegians will be reasonably good at understanding English -
written or spoken, as most are exposed to it regularly (though not
everyone watches American films or plays English-language computer
games). A large proportion can speak English well - most of the younger
generations, and those with higher education. But many people speak
little English in their normal life, after their education, and are out
of practice and under-confident.

And despite their impressive English skills, most /prefer/ Norwegian,
and are more comfortable in Norwegian. It does not matter how fluent
they are in English - if you put some Norwegians together, they will
talk in Norwegian, because that is the native language here.

You use Norwegian unless there is good reason to do otherwise.

So if I am writing a program aimed at Norwegians, the output will be in
Norwegian, unless the balance of development effort versus use effort
makes English the rational choice (since I write English faster and more
accurately). I would normally write any user documentation in
Norwegian. I would normally write internal or code documentation in
English - but the issue was about the user-visible language, not the
programmer's language. I would definitely use English for the code -
but again, that was not the issue.

If the program might reasonably be used by non-Norwegian speakers, then
the choice is between English only, or bilingual.

> And whilst programmers aren't usually scholars, if they are C
> programmers they will use a programming language with keywords based on
> English.

The actual programming is almost always in English. Some Norwegians
will use Norwegian terms for identifiers, usually ASCII'fied. I do that
sometimes when the code uses terms that I know in Norwegian but not in
English. Some Norwegians like to comment in Norwegian. Since a lot of
companies have at least some non-Norwegian programmers, and larger
software development projects are often somewhat international, a most
coding is in English.

But we were not talking about the language programmers use. We were
talking about the language output by programs - how /users/ interact
with the program.

> And you will likley get quite a bit of English coming through the mobile
> phone.

Not unless you are talking to someone in English. Do you think mobile
phone software in Norway is in English?

> Linus Torvald's native language is Finnish, for example.

It is Swedish.

> But git was
> released in English.

It was written long after Torvalds (that's how his name is spelt) had
moved to the USA. It was written for a highly international community,
all over the world - a community of programmers who were already working
on an English-language project.

It is thus utterly irrelevant as an example.

> There might be Finnish language bindings for it
> now, but I'm pretty sure not in the original version. Similarly Bjarne
> Strousup is Swedish, but C++ uses keywords like "class" and "friend",
> not the Swedish terms.

Bjarne Stroustrup (again, you got the spelling wrong) is Danish. And he
designed a programming language, for programmers, as an extension for an
existing English-language programming language.

Are you actively trying to make yourself seem more absurdly ignorant of
the world outside your doorstep? If you are going to use famous
programmers as examples, at least do them the courtesy of getting their
names and languages right. It would have been even better if the
examples had been even remotely relevant. (I do understand that you
can't think of any actually relevant examples - you are not familiar
with any programs written primarily for Norwegians. How you then think
you are qualified to pontificate about them is an unfathomable question.)

> Now I think that would rub off on Norwegian programmers. It would be
> surprising of it did not.
>> Are you aware of the existence of medical devices (my current $DAYJOB)
>> that can be configured to display messages in any of a number of
>> languages?
>>
> Some software is internationalised. But it takes quite a lot of
> resources to translate software. With medical device software the
> software is likely so expensive to develop anyway because of all the
> safety critical portions that the cost is tolerable.

On Wed, 31 Jan 2024 13:35:47 +0100
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:

> On 31.01.2024 12:58, Michael S wrote:
> > On Wed, 31 Jan 2024 12:43:04 +0100
> > Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> >
> >> On 31.01.2024 07:02, Lawrence D'Oliveiro wrote:
> >>> On Tue, 30 Jan 2024 11:50:35 -0800, Keith Thompson wrote:
> >>>
> >>>> VMS (now OpenVMS) was also a significant system at the time, and
> >>>> it had some rather complex file formats ...
> >>>
> >>> [...]
> >>>
> >>> Apparently Linus Torvalds used VMS for a while, and hated it.
> >>
> >> I don't understand the intention of this comment.
> >> VMS and Torvalds are completely different eras.
> >> And were is the relation?
> >>
> >
> > Linus is older than you probably realize.
>
> Why do you think that I'd be thinking that?
>
> I know that he's quite some years younger than I am. So what?
>
> > He entered the University of
> > Helsinki in 1988. Back then VMS was only slightly behind its peak of
> > popularity.
>
> What? - I'm not sure where you're coming from.
>
> I associate DEC's VMS with the old DEC VAX-11 system, both
> from around the mid of the 1970's.

Released in 1977.
Reached the peak of popularity in mid 1980s, when DEC decided to use
VAX not just as mini/super-mini, but also as competitor to mainframes,
effectively killing their earlier mainframe line (PDP-6/10/20).
In late 1980s and early 1990s was used as desktop/workstation OS as
well. Never was very popular in that role, but the reason for moderate
popularity was high price of software and relative weakness of hardware
(MicroVAX) rather than technical deficiencies of the Operation System.
Ported to Alpha in early 1990s.
Ported to Itanium in early 2000s.
Ported to x86-64 starting from mid-2010s. Since, unlike previous ports,
this one was done by small company, it took plenty of time. However by
now VMS on x86-64 is already in production stage.

> I programmed on a DEC's
> VAX with VMS obviously before Linus Torvalds started his
> studies. And that was at a time when the DEC VAX and VMS
> were replaced at our sites by Unix systems.
>

There were places like that.
There were far more places where VMS was replaced much later and not
necessarily by Unix.
There are places where VMS is still running.
Most likely, VMS will still be used in production after the last
"vendor's Unix", of which I'd bet on AIX, is replaced by ether Linux of
BSD.

> > By value, likely still bigger than all Unixen combined.
>
> Not sure what (to me strange sounding) ideas you have here.
>
> Janis
>

I can say the same.

On 31/01/2024 11:53, Michael S wrote:
> On Wed, 31 Jan 2024 08:59:22 +0100
> David Brown <david.brown@hesbynett.no> wrote:
>
>> On 31/01/2024 07:04, Lawrence D'Oliveiro wrote:
>>> On Tue, 30 Jan 2024 21:04:56 +0000, Malcolm McLean wrote:
>>>
>>>> Linus Torvald's native language is Finnish, for example.
>>>
>>> No, it would be Swedish. He’s an ethnic Swede, from Finland.
>>
>> He is Finnish, but has Swedish as his mother tongue (like about 5% of
>> Finns). Speaking Swedish as your main language does not make you
>> ethically Swedish.
>
> Linus has Swedish as his mother tongue.
> Linus has Swedish family name. Or at least Scandinavian, for me it
> sounds more Danish than Swedish, but I am not an expert. It certainly
> does not sound Finnish.

His name is of Swedish language origin, yes. I can't answer for
Torvalds family, but the solid majority of Swedish-speaking Finn
families have been in Finland for hundreds of years. I'm not sure what
the definitions of "ethnically Swedish" or "ethnically Finnish" might
be, but as a general rule Swedish-speaking Finns consider themselves to
be Swedish-speaking Finns - not Swedes living in Finland.

> When Linus was younger, he used to like to tell stereotypical jokes
> about Finns.

So do all Finns I have ever met. They are particularly fond of jokes
about how absurd the Finnish language can be - whether they are natively
Swedish speakers or Finnish speakers. They are all proud of being
Finnish, including traits that may be viewed as eccentric by other cultures.

>
> When it quacks like a duck...
>
>> As a university-educated Finn, brought up in
>> Helsinki, he will also speak Finnish quite fluently.
>>
>
> That's true.
>
>

On Wed, 31 Jan 2024 06:03:37 -0000 (UTC)
Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

> On Tue, 30 Jan 2024 15:21:01 +0000, Malcolm McLean wrote:
>
> > Most systems run Windows where the model of piping from standard
> > output to standard input of the next program is much less used than
> > in Unix, this is true. That sometimes generates a feeling of
> > superiority amongst those who use the less common, often more
> > expensive Unix systems. It's very silly, but that's how people
> > think.
>
> Also we can do select/poll on pipes on *nix systems, you can’t on
> Windows.

You can't do select/poll on Windows anonymous pipes, which are an odd
bird in Win32 API. To me Windows anonymous pipes look like poorly
implemented late addition to Win32 that, I would imagine, was done in
order to claim POSIX compatibility back when it mattered for USA
government contracts.
However you can do Windows' equivalent of poll (WaitForMultipleObjects)
on *named* pipes.

On 31.01.2024 14:05, David Brown wrote:
>
> But it is correct that English has become the main language for
> international communication, and is therefore critical for anything that
> involves cross-border communication, or where there are significant
> numbers of foreign workers. That includes academic work. Different
> parts of Europe previously used German or Russian for this,

Don't forget the importance of French! - The whole postal and
telecommunication sectors were (and probably still are) massively
influenced by France.

(You're always writing so much text, so I'll skip it and avoid
more comments.)

Just two (unrelated) notes concerning statements I've seen
somewhere in the thread (maybe here as well)...

First; the EU publishes in all languages of the member states,
for example. (There's no single lingua franca.)

And the second note; we have to distinguish the language of the
programming language's keywords, the comments in the source
code, and the language(s) used for user-interaction.

I don't know whether there's some native language that use
non-English keywords, but I'd suppose so, since in the past
I've seen some using preprocessors for a "native language"
source code. So while not typical, probably a demand at some
places. (Elsethread I mentioned the German TR440 commands,
but a [primitive] command language, as opposed to, say, the
Unix shell, I don't consider much as a language.)

The comments' languages varies, in my experience. Sometimes
there's coding standards (that demand the native language, or
that demand English), sometimes it's not defined. Myself I'm
reluctant to switch between languages and stay with English.
But there were also other cases with longer descriptions on
a conceptual basis; if you come from a native language's
perspective it can be better to stay with the language of the
specification instead of introducing sources of misunderstanding.

The user interface, finally, is of course as specified, and can
be anything, or even multi-lingual.

Janis

> [...]

On 31/01/2024 07:07, Lawrence D'Oliveiro wrote:
> On Tue, 30 Jan 2024 17:25:31 +0100, David Brown wrote:
>
>> Mixing binary data with formatted text data is very unlikely to be
>> useful.
>
> PDF does exactly that. To the point where the spec suggests putting some
> random unprintable bytes up front, to distract format sniffers from
> thinking they’re looking at a text file.

PDF files start with the "magic" indicator "%PDF", which is enough for
many programs to identify them correctly. And they are usually
compressed so that the content text is not directly readable or
identifiable as strings. If they are not compressed, then yes, there is
can be text mixed in with everything else. But I would not call that
"mixing binary data and formatted text" - I would just say that some of
the binary data happens to be strings. It's the same as elf files
containing copies of strings from the program, or identifiers for
external linking.

However, I learned a new trick when checking that I was not mistaken
about this - it turns out that "less file.pdf" gives a nice text-only
output from the pdf file (by passing it through "lesspipe"). There's
always something new to learn from inane conversations on Usenet :-)

On 30.01.2024 20:39, Richard Harnden wrote:
>
> Nobody uses printf to output binary data. fwrite(3) would be common, as
> would write(2).

Right. I'm using the OS'es write(2), but also printf with ANSI escapes,
e.g. sprintf (buf, "\033[%d;%dH", ...

>
> Maybe you could use printf("%c%c%c" ... but it'd be beyond tedious.

Since I recall to have used it in some thread I want to clarify that
it was just meant as an example countering an incorrect argument of
"not being able to output binary data on stdout", or some such.

Janis

On 31/01/2024 12:22, Janis Papanagnou wrote:
> On 30.01.2024 20:46, David Brown wrote:
>> On 30/01/2024 19:22, bart wrote:
>>> [...]
>>
>> If you have a program like that, then it probably makes sense to have a
>> flag to say "output the data to stdout" and the default being writing to
>> a file.
>
> Did you here mean to say "output the data to the terminal"?

No.

I said that if you have a program that sometimes gives binary output on
stdout, and sometimes gives text messages, and this leads people to have
a significant chance of accidentally dumping binary output to their
terminal, then it probably makes sense to require an explicit flag to
have the program generate binary output on stdout. Then you can use
pipes or other redirection if you want, but are less likely to make a
mess of your terminal by mistake.

> (I noticed
> that a lot of the posts here have a misconception about what 'stdout'
> is; they seem to use it synonymously to "terminal" or "screen/display".
> But you are not guaranteed that stdout will produce screen output; it
> depends on the environment. Being more accurate with the distinction
> might help prevent misconceptions if replying to these people.)
>

I am not sure I have noticed that mistake here, but it is something that
people do mix up - especially if they are used to systems where pipes
and shell redirection are uncommon.

However, here we have used "terminal" intentionally. It is usually not
a big problem if you accidentally pipe binary data into grep or redirect
it to a file. It is, however, annoying when it goes to the terminal and
generates a screenful of flashing garbage, 42 beeps, and leaves you with
a red on red colour scheme.

> More to your point you wrote, I don't think this would be a good design
> as you've written it. A default would imply the necessity of some fixed
> name (or naming schema) - think of the disputable "a.out" default.

Yes.

Or perhaps just an error message saying you need to specify the filename
for output or "-" for stdout. Those are just details.

On 30/01/2024 22:16, James Kuyper wrote:
> On 1/30/24 11:49, David Brown wrote:
> ...
>> If the program can reasonably be expected to generate binary output,
>> then it is the user's fault if they do this accidentally. Examples
>> shown in this thread include cat and zcat - that's what these programs do.
>
> ? There's no problem using cat to concatenate binary files. I've used
> 'split' to split binary files into smaller pieces, and then used 'cat'
> to recombine them, and it worked fine. I don't remember why, but I had
> to transfer the files from one place to another by a method that imposed
> an upper limit on the size of individual files.
>

I think there's a misunderstanding here - I gave "cat" is an example of
a program that /can/ be expected to produce binary output. (It can also
produce text output - you get what you put in.) So it is the user's
fault if the type "cat /bin/cat" and are surprised by a mess in their
terminal.

I would expect that the majority of uses of "cat" are with just one
file, but certainly it is useful when you want to combine files in
different ways.

On 31/01/2024 09:36, Malcolm McLean wrote:
> On 31/01/2024 07:18, Tim Rentsch wrote:
>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>
>>> On 30/01/2024 07:27, Tim Rentsch wrote:
>>>
>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>
>>>>> On 29/01/2024 20:10, Tim Rentsch wrote:
>>>>>
>>>>>> Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:
>>>>>>
>>>>>> [...]
>>>>>>
>>>>>>> I've never used standard output for binary data.
>>>>>>> [...] it strikes me as a poor design decision.
>>>>>>
>>>>>> How so?
>>>>>
>>>>> Because the output can't be inspected by humans, and because it might
>>>>> have unusual effects if passed though systems designed to handle
>>>>> human-readable text.  For instance in some systems designed to receive
>>>>> ASCII text, there is no distinction between the nul byte and "waiting
>>>>> for next data byte".  Obviously this will cause difficuties if the
>>>>> data
>>>>> is binary.
>>>>> Also many binary formats can't easily be extended, so you can pass one
>>>>> image and that's all.  While it is possible to devise a text format
>>>>> which is similar, in practice text formats usually have enough
>>>>> redundancy to be easily extended.
>>>>>
>>>>> So it's harder to correct errors, more prone to errors, and harder to
>>>>> extend.
>>>>
>>>> Your reasoning is all gobbledygook.  Your comments reflect only
>>>> limitations in your thinking, not any essential truth about using
>>>> standard out for binary data.
>>>
>>> I must admit that it's nothing I have ever done or considered doing.
>>>
>>> [...]
>>
>> Simple example (disclaimer: not tested):
>>
>>      ssh foo 'cd blah ; tar -cf - . | gzip -c' | \
>>         (mkdir foo.blah ; cd foo.blah ; gunzip -c | tar -xf -)
>>
>> Of the five main programs in this command, four are using
>> standard out to send binary data:
>>
>>      tar -cf - .
>>      gzip -c
>>      ssh foo [...]
>>      gunzip -c
>>
>> The tar -xf - at the end reads binary data on standard in
>> but doesn't output any (or anything else for that matter).
>>
>> It is FAR more cumbersome to accomplish what this command
>> is doing without sending binary data through standard out.
>> Anyone who doesn't understand this doesn't understand Unix.
> >
> Yes. I don't do that sort of thing.
> Whilst I have used Unix, it is as a platform for interactive programs
> which work on graohics, or a general C compilation environment. I don;t
> build pipeliens to do that sort of data processing. If I had to download
> a tar file I'd either use a graphical tool or type serveal commands into
> the shell, each launching single executable, interactively.
>
> The reason is that I'd only run the command once, and it's so likely
> that there will be either a syntax misunderstanding or a typing error
> that I'd have to test to ensure that it was right. And by the time
> you've done that any time saved by typing only one commandline is lost.
> Of course if you are writing scripts then that doesn't apply. But now
> it's effectively a programming language, and, from the example code, a
> very poorly designed one which is cryptic and fussy and liable to be
> hard to maintain. So it's better to use a language like Perl to achieve
> the same thing, and I did have a few Perl scripts handy for repetitive
> jobs of that nature in my Unix days.
>

That gave me a laugh! You think bash is cryptic, fussy and poorly
designed, and choose /Perl/ as the alternative :-)

> You admit this with "not tested". Says it all. '"Understandig Unix" is
> an intellectually useless achievement. You might have to do it if you
> have to use the system and debug and trouble shoot. But it's nothing to
> be proud about.
>

It is "useless" for people who don't use it. For people who /do/ use
it, is very useful.

I've used sequences like Tim's - it's a way to copy data remotely from a
different machine. I would likely write it slightly differently - I'd
probably do the mkdir and cd first, thus avoiding the need for a
subshell, and I'd use "ssh -C" or "tar -z" to do the compression rather
than "gzip".

There's no doubt that the learning curve is longer for doing this sort
of thing from the command line than using gui programs. There is also
no doubt that when you are used to it, command line utilities and a good
shell are very flexible and efficient.

Learn to use the tools that are conveniently available, and then pick
the right tool for the job - whether it is command line or gui.

On 31.01.2024 14:58, David Brown wrote:
> On 31/01/2024 12:22, Janis Papanagnou wrote:
>> On 30.01.2024 20:46, David Brown wrote:
>>> On 30/01/2024 19:22, bart wrote:
>>>> [...]
>>>
>>> If you have a program like that, then it probably makes sense to have a
>>> flag to say "output the data to stdout" and the default being writing to
>>> a file.
>>
>> Did you here mean to say "output the data to the terminal"?
>
> No.
>
> I said that if you have a program that sometimes gives binary output on
> stdout, and sometimes gives text messages,

Umm, the same program? - I mean, sure, technically a program can
produce binary output (with some options) and textual output (with
some other options); but that is explicitly controlled by the user.

Your statement reads as if it would arbitrarily output binary or
text; and in that case the "output type" distinction appears to be
artificially and should instead be called just "data".

It might be enlightening to consider for example: decrypt f.cpt
What would one expect as output ("binary"? "text"? or "data"?) and
what we gain with a (spurious) flag?

> and this leads people to have
> a significant chance of accidentally dumping binary output to their
> terminal, [...]

Okay, after all, "output the data to the terminal", as I suspected
(and that was what I had been asking to clarify).

Myself (as I'd suppose everyone else also) has dumped non-text
code onto the terminal, by a typo, or by forgetting to provide
an option. 'stty sane' or 'reset' and continue...

Janis