Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?

piątek, 4 sierpnia 2023 o 11:49:18 UTC+2 Malcolm McLean napisał(a):
> Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?

it should be text afait ..what opening in text mode do it translates 0x0d 0x0a into only 0x0d in ram afair - unicode has nothing to do it (its btw the flop of os that needs 0x0a 0x0d instead of 0xd only)

On 04/08/2023 10:49, Malcolm McLean wrote:
> Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?

I open all files in binary mode.

It means that if processing text files on Windows, you may encounter
either CRLF or LF-only line endings. But that's easy enough to deal
with: just ignore CR.

(Doubtless someone will be along soon to point out that some 1970s
mainframes had completely different arrangements, and that any such code
will not work on any such machines that are still running.)

piątek, 4 sierpnia 2023 o 12:30:20 UTC+2 Bart napisał(a):
> On 04/08/2023 10:49, Malcolm McLean wrote:
> > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?
> I open all files in binary mode.
>
> It means that if processing text files on Windows, you may encounter
> either CRLF or LF-only line endings. But that's easy enough to deal
> with: just ignore CR.
>
> (Doubtless someone will be along soon to point out that some 1970s
> mainframes had completely different arrangements, and that any such code
> will not work on any such machines that are still running.)

and if you will save yours strings like "\nsome\nhere" it will work as
a proper windows text file?
i use opening in text mode but if you will write your own library you covered such
things in there (LoadTextFile("some.asm"); LoadBinaryFile("some.bmp"); so no big deal

On Friday, August 4, 2023 at 1:30:20 PM UTC+3, Bart wrote:
> On 04/08/2023 10:49, Malcolm McLean wrote:
> > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?
> I open all files in binary mode.
>
> It means that if processing text files on Windows, you may encounter
> either CRLF or LF-only line endings. But that's easy enough to deal
> with: just ignore CR.
>
> (Doubtless someone will be along soon to point out that some 1970s
> mainframes had completely different arrangements, and that any such code
> will not work on any such machines that are still running.)

Far more recent and far less obscure example is Mac OS classic.
Of course, in practice it's even less likely to be used today by real users (as
opposed to computer history enthusiasts) than "some 1970s mainframes"

On 04/08/2023 11:47, Michael S wrote:
> On Friday, August 4, 2023 at 1:30:20 PM UTC+3, Bart wrote:
>> On 04/08/2023 10:49, Malcolm McLean wrote:
>>> Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?
>> I open all files in binary mode.
>>
>> It means that if processing text files on Windows, you may encounter
>> either CRLF or LF-only line endings. But that's easy enough to deal
>> with: just ignore CR.
>>
>> (Doubtless someone will be along soon to point out that some 1970s
>> mainframes had completely different arrangements, and that any such code
>> will not work on any such machines that are still running.)
>
> Far more recent and far less obscure example is Mac OS classic.
> Of course, in practice it's even less likely to be used today by real users (as
> opposed to computer history enthusiasts) than "some 1970s mainframes"

Do you mean CR-only line endings?

I don't think I've encountered a file using that since the 1990s. It can
probably be accommodated too (treat CR as though it was LF when not
followed by LF), but I don't bother.

A file with mixed line-endings though would not work, and would anyway
be ambiguous: CRLF could mean one newline, or two with an intervening
blank line.

On 04/08/2023 11:34, fir wrote:
> piątek, 4 sierpnia 2023 o 12:30:20 UTC+2 Bart napisał(a):
>> On 04/08/2023 10:49, Malcolm McLean wrote:
>>> Should UTF-8 files be opened in binary ("rb") or text ("r") mode
when calling fopen()? Are they "text" or a binary representation of text?
>> I open all files in binary mode.
>>
>> It means that if processing text files on Windows, you may encounter
>> either CRLF or LF-only line endings. But that's easy enough to deal
>> with: just ignore CR.
>>
>> (Doubtless someone will be along soon to point out that some 1970s
>> mainframes had completely different arrangements, and that any such code
>> will not work on any such machines that are still running.)
>
> and if you will save yours strings like "\nsome\nhere" it will work as
> a proper windows text file?

TBH I hardly know whether my strings (even from my language) use CRLF or
LF, or whether text files are opened for writing as binary or text.

Because whether the resulting files contain CRLF or only LF, doesn't
matter any more because my programs that read them are oblivious to that
detail.

However, my script language uses a 'createfile' function, a wrapper
around 'fopen', and the default file mode is "wb". My text editor
creates files using that, so most text files from it use LF-only.

But I didn't even known until I checked it just now.

Where 'fopen' is called directly for text output, sometimes it uses "w",
and sometimes "wb".

> i use opening in text mode but if you will write your own library you
covered such
> things in there (LoadTextFile("some.asm");
LoadBinaryFile("some.bmp"); so no big deal

If you open an image file in the PPM 'P6' format, it uses an ASCII
header, which has text separated by newlines, with the last newline
followed by binary data.

This needs to be opened in binary mode, but it means any CRLF line
endings are exposed as 13, 10 bytes, which may confuse a C program
expecting only '\n' characters.

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
> calling fopen()? Are they "text" or a binary representation of text?

On Unix-like systems it does not matter. On Windows I would say it
depends on whether you want the \n <=> \r\n translation to be done for
you (but it's been a while since I used Windows).

--
Ben.

piątek, 4 sierpnia 2023 o 14:23:33 UTC+2 Bart napisał(a):
> On 04/08/2023 11:34, fir wrote:
> > piątek, 4 sierpnia 2023 o 12:30:20 UTC+2 Bart napisał(a):
> >> On 04/08/2023 10:49, Malcolm McLean wrote:
> >>> Should UTF-8 files be opened in binary ("rb") or text ("r") mode
> when calling fopen()? Are they "text" or a binary representation of text?
> >> I open all files in binary mode.
> >>
> >> It means that if processing text files on Windows, you may encounter
> >> either CRLF or LF-only line endings. But that's easy enough to deal
> >> with: just ignore CR.
> >>
> >> (Doubtless someone will be along soon to point out that some 1970s
> >> mainframes had completely different arrangements, and that any such code
> >> will not work on any such machines that are still running.)
> >
> > and if you will save yours strings like "\nsome\nhere" it will work as
> > a proper windows text file?
> TBH I hardly know whether my strings (even from my language) use CRLF or
> LF, or whether text files are opened for writing as binary or text.
>
> Because whether the resulting files contain CRLF or only LF, doesn't
> matter any more because my programs that read them are oblivious to that
> detail.
>
> However, my script language uses a 'createfile' function, a wrapper
> around 'fopen', and the default file mode is "wb". My text editor
> creates files using that, so most text files from it use LF-only.
>
> But I didn't even known until I checked it just now.
>
> Where 'fopen' is called directly for text output, sometimes it uses "w",
> and sometimes "wb".
> > i use opening in text mode but if you will write your own library you
> covered such
> > things in there (LoadTextFile("some.asm");
> LoadBinaryFile("some.bmp"); so no big deal
> If you open an image file in the PPM 'P6' format, it uses an ASCII
> header, which has text separated by newlines, with the last newline
> followed by binary data.
>
> This needs to be opened in binary mode, but it means any CRLF line
> endings are exposed as 13, 10 bytes, which may confuse a C program
> expecting only '\n' characters.

it seemd i was also not clearly seen this (for some reason for example i thought "\n" in c is 13 - maybe becouse i was coding on commodore64 a lot as 13 year old, and on commodere its said it was 13 and i seem ro remember that)

i also checked in fact windows seem to have no trouble with unix LF only files
with some surprise i discovered i save unix asm files in furia..but that is what i in fact want..i also read input files in org-asm in binary to have eventually crlf but
it was also what i wanted (i optionally wanted do soem text statistics etc)

but it seem in normal cirucumstances its probably to open in text mode (to have only LF and save in binary - also to have only LF if c is LF based internally to)

piątek, 4 sierpnia 2023 o 12:15:29 UTC+2 fir napisał(a):
> piątek, 4 sierpnia 2023 o 11:49:18 UTC+2 Malcolm McLean napisał(a):
> > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?
> it should be text afait ..what opening in text mode do it translates 0x0d 0x0a into only 0x0d in ram afair - unicode has nothing to do it (its btw the flop of os that needs 0x0a 0x0d instead of 0xd only)

in utf-8 all the unicode signs are coded by 128+ values afaik so text mode binary mode do not interfere with unicode/ansi

piątek, 4 sierpnia 2023 o 15:25:27 UTC+2 fir napisał(a):
> piątek, 4 sierpnia 2023 o 14:23:33 UTC+2 Bart napisał(a):
> > On 04/08/2023 11:34, fir wrote:
> > > piątek, 4 sierpnia 2023 o 12:30:20 UTC+2 Bart napisał(a):
> > >> On 04/08/2023 10:49, Malcolm McLean wrote:
> > >>> Should UTF-8 files be opened in binary ("rb") or text ("r") mode
> > when calling fopen()? Are they "text" or a binary representation of text?
> > >> I open all files in binary mode.
> > >>
> > >> It means that if processing text files on Windows, you may encounter
> > >> either CRLF or LF-only line endings. But that's easy enough to deal
> > >> with: just ignore CR.
> > >>
> > >> (Doubtless someone will be along soon to point out that some 1970s
> > >> mainframes had completely different arrangements, and that any such code
> > >> will not work on any such machines that are still running.)
> > >
> > > and if you will save yours strings like "\nsome\nhere" it will work as
> > > a proper windows text file?
> > TBH I hardly know whether my strings (even from my language) use CRLF or
> > LF, or whether text files are opened for writing as binary or text.
> >
> > Because whether the resulting files contain CRLF or only LF, doesn't
> > matter any more because my programs that read them are oblivious to that
> > detail.
> >
> > However, my script language uses a 'createfile' function, a wrapper
> > around 'fopen', and the default file mode is "wb". My text editor
> > creates files using that, so most text files from it use LF-only.
> >
> > But I didn't even known until I checked it just now.
> >
> > Where 'fopen' is called directly for text output, sometimes it uses "w",
> > and sometimes "wb".
> > > i use opening in text mode but if you will write your own library you
> > covered such
> > > things in there (LoadTextFile("some.asm");
> > LoadBinaryFile("some.bmp"); so no big deal
> > If you open an image file in the PPM 'P6' format, it uses an ASCII
> > header, which has text separated by newlines, with the last newline
> > followed by binary data.
> >
> > This needs to be opened in binary mode, but it means any CRLF line
> > endings are exposed as 13, 10 bytes, which may confuse a C program
> > expecting only '\n' characters.
> it seemd i was also not clearly seen this (for some reason for example i thought "\n" in c is 13 - maybe becouse i was coding on commodore64 a lot as 13 year old, and on commodere its said it was 13 and i seem ro remember that)
>
> i also checked in fact windows seem to have no trouble with unix LF only files
> with some surprise i discovered i save unix asm files in furia..but that is what i in fact want..i also read input files in org-asm in binary to have eventually crlf but
> it was also what i wanted (i optionally wanted do soem text statistics etc)
>
> but it seem in normal cirucumstances its probably to open in text mode (to have only LF and save in binary - also to have only LF if c is LF based internally to)

this crlf is bessides annoyance so better use LF really if c support that and windows work on that (hovever im also sure most files on windows i sen used crlf not lf)

On Friday, 4 August 2023 at 14:14:48 UTC+1, Ben Bacarisse wrote:
> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>
> > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
> > calling fopen()? Are they "text" or a binary representation of text?
> On Unix-like systems it does not matter. On Windows I would say it
> depends on whether you want the \n <=> \r\n translation to be done for
> you (but it's been a while since I used Windows).
>
I tested the Baby X resource compiler on Windows. The project is hosted
on github, and it contains a short example UTF-8 file, as well as an example
UTF-16 file. The UTF-8 file has the extension ",txt". On my Apple Mac,
both the UTF-8 file and the UTF-16 file, when translated to UTF-16 and output,
output a single newline. On Windows, the UTF-8 file has a carriage return,
whlst the UTF-16 file does not. So what must have happened is that git has
added the carriage return when downloading to Windows.
I can fix the problem by opening the UTF-8 file with "r" rather than "rb", but
I'm not sure this is the right thing to do. The idea of UTF-8 is surely one
representation?

piątek, 4 sierpnia 2023 o 15:30:13 UTC+2 Malcolm McLean napisał(a):
> On Friday, 4 August 2023 at 14:14:48 UTC+1, Ben Bacarisse wrote:
> > Malcolm McLean <malcolm.ar...@gmail.com> writes:
> >
> > > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
> > > calling fopen()? Are they "text" or a binary representation of text?
> > On Unix-like systems it does not matter. On Windows I would say it
> > depends on whether you want the \n <=> \r\n translation to be done for
> > you (but it's been a while since I used Windows).
> >
> I tested the Baby X resource compiler on Windows. The project is hosted
> on github, and it contains a short example UTF-8 file, as well as an example
> UTF-16 file. The UTF-8 file has the extension ",txt". On my Apple Mac,
> both the UTF-8 file and the UTF-16 file, when translated to UTF-16 and output,
> output a single newline. On Windows, the UTF-8 file has a carriage return,
> whlst the UTF-16 file does not. So what must have happened is that git has
> added the carriage return when downloading to Windows.
> I can fix the problem by opening the UTF-8 file with "r" rather than "rb", but
> I'm not sure this is the right thing to do. The idea of UTF-8 is surely one
> representation?

ansi can have CR (mac/commodore 64) LF(unix/amiga) CRLF(dos/windows)
so i think utf is exactly the same...native for windows is crlf but as all newer mac,
windows, unix use LF (and "\n" in c are also LF) the best is imo to use LF

so open in text , save in binary (if you got such approach as i i only code on system i know i kode on, not generalized way) - more generalized woudl be probably open as text save as text

piątek, 4 sierpnia 2023 o 15:36:31 UTC+2 fir napisał(a):
> piątek, 4 sierpnia 2023 o 15:30:13 UTC+2 Malcolm McLean napisał(a):
> > On Friday, 4 August 2023 at 14:14:48 UTC+1, Ben Bacarisse wrote:
> > > Malcolm McLean <malcolm.ar...@gmail.com> writes:
> > >
> > > > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
> > > > calling fopen()? Are they "text" or a binary representation of text?
> > > On Unix-like systems it does not matter. On Windows I would say it
> > > depends on whether you want the \n <=> \r\n translation to be done for
> > > you (but it's been a while since I used Windows).
> > >
> > I tested the Baby X resource compiler on Windows. The project is hosted
> > on github, and it contains a short example UTF-8 file, as well as an example
> > UTF-16 file. The UTF-8 file has the extension ",txt". On my Apple Mac,
> > both the UTF-8 file and the UTF-16 file, when translated to UTF-16 and output,
> > output a single newline. On Windows, the UTF-8 file has a carriage return,
> > whlst the UTF-16 file does not. So what must have happened is that git has
> > added the carriage return when downloading to Windows.
> > I can fix the problem by opening the UTF-8 file with "r" rather than "rb", but
> > I'm not sure this is the right thing to do. The idea of UTF-8 is surely one
> > representation?
> ansi can have CR (mac/commodore 64) LF(unix/amiga) CRLF(dos/windows)
> so i think utf is exactly the same...native for windows is crlf but as all newer mac,
> windows, unix use LF (and "\n" in c are also LF) the best is imo to use LF
>
> so open in text , save in binary (if you got such approach as i i only code on system i know i kode on, not generalized way) - more generalized woudl be probably open as text save as text

i assumed that savin as text adds crlf - and i tested and it do its a waste of disk space as i tested it on my furia compilation to assembly and "wb" makes 42 578 where "wt" 44 678 - notable wase of space (shows btw that my assumption that line of code is 20 bytes is ok , though here the code is asm not c)

easy to cont way save in both "wb" "wt" and difference is number of lines, compare it to size to have average line length

piątek, 4 sierpnia 2023 o 15:36:31 UTC+2 fir napisał(a):
> piątek, 4 sierpnia 2023 o 15:30:13 UTC+2 Malcolm McLean napisał(a):
> > On Friday, 4 August 2023 at 14:14:48 UTC+1, Ben Bacarisse wrote:
> > > Malcolm McLean <malcolm.ar...@gmail.com> writes:
> > >
> > > > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
> > > > calling fopen()? Are they "text" or a binary representation of text?
> > > On Unix-like systems it does not matter. On Windows I would say it
> > > depends on whether you want the \n <=> \r\n translation to be done for
> > > you (but it's been a while since I used Windows).
> > >
> > I tested the Baby X resource compiler on Windows. The project is hosted
> > on github, and it contains a short example UTF-8 file, as well as an example
> > UTF-16 file. The UTF-8 file has the extension ",txt". On my Apple Mac,
> > both the UTF-8 file and the UTF-16 file, when translated to UTF-16 and output,
> > output a single newline. On Windows, the UTF-8 file has a carriage return,
> > whlst the UTF-16 file does not. So what must have happened is that git has
> > added the carriage return when downloading to Windows.
> > I can fix the problem by opening the UTF-8 file with "r" rather than "rb", but
> > I'm not sure this is the right thing to do. The idea of UTF-8 is surely one
> > representation?
> ansi can have CR (mac/commodore 64) LF(unix/amiga) CRLF(dos/windows)
> so i think utf is exactly the same...native for windows is crlf but as all newer mac,
> windows, unix use LF (and "\n" in c are also LF) the best is imo to use LF
>
> so open in text , save in binary (if you got such approach as i i only code on system i know i kode on, not generalized way) - more generalized woudl be probably open as text save as text

from the other point of wiev one could say in fact some system
could support that "printer scripts" where you could CR and type one text
onto another and not only this but olso rest of codes loke bell etc

in such view the incompatibility made those people who changed
that LF will make CR automatically (which is as i guess breaking
compatybiity with orygnal script)

Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

> On Friday, 4 August 2023 at 14:14:48 UTC+1, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>>
>> > Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
>> > calling fopen()? Are they "text" or a binary representation of text?
>> On Unix-like systems it does not matter. On Windows I would say it
>> depends on whether you want the \n <=> \r\n translation to be done for
>> you (but it's been a while since I used Windows).
>>
> I tested the Baby X resource compiler on Windows. The project is hosted
> on github, and it contains a short example UTF-8 file, as well as an example
> UTF-16 file. The UTF-8 file has the extension ",txt". On my Apple Mac,
> both the UTF-8 file and the UTF-16 file, when translated to UTF-16 and output,
> output a single newline. On Windows, the UTF-8 file has a carriage return,
> whlst the UTF-16 file does not. So what must have happened is that git has
> added the carriage return when downloading to Windows.
> I can fix the problem by opening the UTF-8 file with "r" rather than "rb", but
> I'm not sure this is the right thing to do. The idea of UTF-8 is surely one
> representation?

Ditto ASCII. But the problem is that

line\n

and

line\r\n

are different things so they need to be represented as different
streams. Unlike UTF-8, ASCII does define some meanings, but that never
stopped a proliferation of line endings.

Now, I may have missed something, and /Unicode/ might have decreed how a
line is terminated, but I really doubt they could have got away with it.

--
Ben.

On Fri, 4 Aug 2023 06:29:59 -0700 (PDT)
Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
> I tested the Baby X resource compiler on Windows. The project is hosted
> on github, and it contains a short example UTF-8 file, as well as an example
> UTF-16 file. The UTF-8 file has the extension ",txt". On my Apple Mac,
> both the UTF-8 file and the UTF-16 file, when translated to UTF-16 and output,
> output a single newline. On Windows, the UTF-8 file has a carriage return,
> whlst the UTF-16 file does not. So what must have happened is that git has
> added the carriage return when downloading to Windows.

git has a specific option on how to handle line endings and whether to do
automatic conversions based on operating system. For specifics sreng (my
own verb ; it means "use a search engine") for "git line endings".

> I can fix the problem by opening the UTF-8 file with "r" rather than "rb", but
> I'm not sure this is the right thing to do. The idea of UTF-8 is surely one
> representation?

UTF-8 is an algorithm/specification for maping a Unicode codepoint (actually
it works for any integer N with 0 <= N < 2**31) to a sequence of octets. It
says nothing about line endings in operating systems. Whether Unicode as a
whole says anything about line endings , I don't know.

On 2023-08-04, Malcolm McLean <malcolm.arthur.mclean@gmail.com> wrote:
> Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
> calling fopen()? Are they "text" or a binary representation of text?

They are text.

Practically speaking, if a UTF-8 file comes from a CR-LF system, and you
open it in binary mode, you will see the \r and \n characters.

The main reason we deal with text versus binary streams on mainstream
platforms is the line ending representation.

That is orthogonal to whether there are UTF-8 characters.

On the other hand, it's possible that the values 0x80 to 0xFF do not
correspond to characters of text on some system.

If we are concerned about the broadest possible portability,
then there is pretty ominous wording in the Standard,
which has these kinds of things to say about text streams:

Characters may have to be added, altered, or deleted on input and
output to conform to differing conventions for representing text in
the host environment.

Thus, there need not be a one-to-one correspondence between the
characters in a stream and those in the external representation.

Data read in from a text stream will necessarily compare equal to the
data that were earlier written out to that stream only if: the data
consist only of printing characters and the control characters
horizontal tab and new-line; no new-line character is immediately
preceded by space characters; and the last character is a new-line
character.

That's from C99, and the paragraph division is mine.

So text streams are allowed to trash UTF-8 according to the Standard.

In practical terms, though, I reiterate that it behooves you to treat
them as text, so that you get the line ending conversions on common
platforms.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 8/4/23 5:49 AM, Malcolm McLean wrote:
> Should UTF-8 files be opened in binary ("rb") or text ("r") mode when calling fopen()? Are they "text" or a binary representation of text?

Since the distinction is whether the data should be interpreted as using
the local line-ending method, I would use text. The fact that it is
UTF-8 encoded means it will almost certainly use some combination of new
line and/or carriage return for line ending. If you open as text, you
know that you will get the \n code for line endings.

Of course, if you don't want to affect the nature of line endings in the
file (or want to process them specially yourself) you want to open as
binary.

On 8/4/23 9:29 AM, Malcolm McLean wrote:
> On Friday, 4 August 2023 at 14:14:48 UTC+1, Ben Bacarisse wrote:
>> Malcolm McLean <malcolm.ar...@gmail.com> writes:
>>
>>> Should UTF-8 files be opened in binary ("rb") or text ("r") mode when
>>> calling fopen()? Are they "text" or a binary representation of text?
>> On Unix-like systems it does not matter. On Windows I would say it
>> depends on whether you want the \n <=> \r\n translation to be done for
>> you (but it's been a while since I used Windows).
>>
> I tested the Baby X resource compiler on Windows. The project is hosted
> on github, and it contains a short example UTF-8 file, as well as an example
> UTF-16 file. The UTF-8 file has the extension ",txt". On my Apple Mac,
> both the UTF-8 file and the UTF-16 file, when translated to UTF-16 and output,
> output a single newline. On Windows, the UTF-8 file has a carriage return,
> whlst the UTF-16 file does not. So what must have happened is that git has
> added the carriage return when downloading to Windows.
> I can fix the problem by opening the UTF-8 file with "r" rather than "rb", but
> I'm not sure this is the right thing to do. The idea of UTF-8 is surely one
> representation?

Yes, it is one representation, but doesn't define what a "line ending"
is. That is an orthogonal definition of the MEANING of the control
characters, which is beyond the definition of the basic character set.

On 04/08/2023 11:47, Michael S wrote:
> Far more recent and far less obscure example is Mac OS classic.
> Of course, in practice it's even less likely to be used today by real users (as
> opposed to computer history enthusiasts) than "some 1970s mainframes"

I've seen cr only, lf only, lf-cr and cr-lf. Don't ask me where...

(Not to mention the 1970s mainframes, where a file was a set of records
rather than a stream of bytes and needed to be treated as such. There
might even be embedded line breaks, form feeds, etc inside one record.
It's not possible to accurately maps such files onto the C text file)

Andy