Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Steal my cash, car and TV - but leave the computer! -- Soenke Lange <soenke@escher.north.de>


computers / news.software.readers / Re: I'm hacking on trn...want to join me?

SubjectAuthor
* I'm hacking on trn...want to join me?Richard
+* Re: I'm hacking on trn...want to join me?Chime Hart
|`- Re: I'm hacking on trn...want to join me?Richard
`* Re: I'm hacking on trn...want to join me?Michael Bäuerle
 +* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |`* Re: I'm hacking on trn...want to join me?Richard
 | `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |  `* Re: I'm hacking on trn...want to join me?Richard
 |   `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |    `* Re: I'm hacking on trn...want to join me?Richard
 |     +* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |`* Re: I'm hacking on trn...want to join me?Michael Bäuerle
 |     | +* Getting MIME headers right (was: I'm hacking on trn...want to join me?)Richard
 |     | |+* Re: Getting MIME headers rightAdam H. Kerman
 |     | ||`* Re: Getting MIME headers rightRichard
 |     | || `* Re: Getting MIME headers rightAdam H. Kerman
 |     | ||  `* Re: Getting MIME headers rightRichard
 |     | ||   `* Re: Getting MIME headers rightAdam H. Kerman
 |     | ||    +* Re: Getting MIME headers rightRichard
 |     | ||    |`- Re: Getting MIME headers rightAdam H. Kerman
 |     | ||    `* Re: Getting MIME headers rightFrank Slootweg
 |     | ||     `- Re: Getting MIME headers rightRichard
 |     | |`* Re: Getting MIME headers rightMichael Bäuerle
 |     | | `- Re: Getting MIME headers rightRichard
 |     | `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |  +- Re: I'm hacking on trn...want to join me?Richard
 |     |  `* Re: I'm hacking on trn...want to join me?Michael Bäuerle
 |     |   +* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |   |`* Re: I'm hacking on trn...want to join me?Richard
 |     |   | `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |   |  `- Re: I'm hacking on trn...want to join me?Richard
 |     |   `- Re: I'm hacking on trn...want to join me?Richard
 |     `* Re: I'm hacking on trn...want to join me?Eli the Bearded
 |      `* Re: I'm hacking on trn...want to join me?Richard
 |       `* Re: I'm hacking on trn...want to join me?Eli the Bearded
 |        `- Re: I'm hacking on trn...want to join me?Richard
 `* Re: I'm hacking on trn...want to join me?Eli the Bearded
  `- Re: I'm hacking on trn...want to join me?Richard

Pages:12
I'm hacking on trn...want to join me?

<u0ffth$1u1oi$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1243&group=news.software.readers#1243

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: I'm hacking on trn...want to join me?
Date: Mon, 3 Apr 2023 21:19:45 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u0ffth$1u1oi$1@news.xmission.com>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Mon, 3 Apr 2023 21:19:45 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2033426"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Mon, 3 Apr 2023 21:19 UTC

[Please do not mail me a copy of your followup]

As near as I can tell the current "maintainer" of trn isn't. The
official home page is on sourceforge and I don't see any activity
there for 10+ years. The trn-4.0-test77.tar.gz distribution -- the
most recent -- was created in 02-Sep-2010. The test76 release was
another ten years before that on 02-Apr-2001.

I may be the last user and it's somewhat of a pity/passion project :-).

I forked the code from sourceforge into github and I'm hacking on the
cmake branch. <https://github.com/LegalizeAdulthood/trn/tree/cmake>

So far:
- Added a CMake based build
- Use vcpkg to get curses dependency (pdcurses on Windows)
- Use CMake inspection of your environment and configure_file to
generate the config.h instead of chatty bash Configure script.
- Converted from C to C++
- Applied various automated clean-ups to the code (yay ReSharper for C++!)
- Gradually introducing more use of 'const'
- Gradually introducing use of std::string instead of C style string
- Unit tests with GTest
- Working on adding tests before making any major changes.
- A bunch of % interpolator tests added and some minor bugs fixed
- Using CMake to generate test local news spool data and articles
- Will use this to verify article/newsgroup related % interpolation
- This part is current work-in-progress
- Replace use of int with bool where appropriate
- Replace use of int with strong enum types where appropriate
- Legacy code considerations dropped; e.g. no special VMS code, no
non-POSIX standard unix code, K&R style code modernized, no optional
features to "save instruction and data space", etc.

General target direction:
- Replace low-level termios code with curses
- Become more "event driven" insted of mingled read/write stdio
- Decouple newsgroup/article logic from TUI to allow for GUI
- Replace synchronous NNTP/newsgroup processing with asynchronous
processing

Vision of the final result (no particular order):
- True STARTTLS/NNTPS support
- Curses windows used to show all the various things:
- newsgroup selection
- article selection
- thread selection
- KILL files
- etc.
- A GUI for reading news that's as convenient as the trn TUI.
- Let you manipulate the various windows like people do in vim/emacs
allowing you to choose what is on-screen and where
- Client should be asynchronously advancing through the unread articles
at the "speed of light" to apply scoring/threading/KILL file processing
as far ahead as possible while you read.
- As easy to build on Windows as it is on *nix.

I mostly develop on Windows, but try to keep the #ifdef'ed linux
branches at least building by occasionally building in WSL. A github
build workflow would help but not my immediate priority.

If you're interested in joining me for the ride, pull requests are
always welcome or drop an issue in the github repo.

I'm surprised that after all this time there is still no decent C++
library for NNTP. The fallout from this effort may be such a library
using asynchronous I/O processing to get as close as possible to "the
speed of light" for an NNTP client.

I did a presentation on doing basic NNTP with POCO last year:
Writing a Network Client with POCO
<https://www.youtube.com/watch?v=rRR9RTUEn4k>

I'm doing another presentation next week on basic NNTP with boost.asio:
<https://www.meetup.com/utah-cpp-programmers/events/zhljbtyfcgbqb/>

-- Richard

--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<4143c063-c260-2a66-fb34-70d5b8ab7d66@hubert-humphrey.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1244&group=news.software.readers#1244

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!peer03.iad!feed-me.highwinds-media.com!news.highwinds-media.com!Xl.tags.giganews.com!local-1.nntp.ord.giganews.com!news.giganews.com.POSTED!not-for-mail
NNTP-Posting-Date: Fri, 07 Apr 2023 23:38:57 +0000
Newsgroups: news.software.readers
Date: Fri, 7 Apr 2023 16:38:56 -0700
From: chi...@hubert-humphrey.com (Chime Hart)
Subject: Re: I'm hacking on trn...want to join me?
In-Reply-To: <u0ffth$1u1oi$1@news.xmission.com>
Message-ID: <4143c063-c260-2a66-fb34-70d5b8ab7d66@hubert-humphrey.com>
References: <u0ffth$1u1oi$1@news.xmission.com>
MIME-Version: 1.0
Content-Type: text/plain; format=flowed; charset=US-ASCII
Lines: 13
X-Usenet-Provider: http://www.giganews.com
X-Trace: sv3-3slnx44kX5qmNqZt30Ds66XMduTzVBMZLEGQjUlBiRNy8m7mY0Q5zstwPzB6h5nIY21bY3CYx8Cvt8m!CGXXnvTcOLwrHFDelgFVwP6vzA39sSLx5nW9Np/TwSUPI54OhGjtSTzt2Sh8GzHpq7alkJ+NHWYB!i0f1QU3D6ECftr8=
X-Complaints-To: abuse@giganews.com
X-DMCA-Notifications: http://www.giganews.com/info/dmca.html
X-Abuse-and-DMCA-Info: Please be sure to forward a copy of ALL headers
X-Abuse-and-DMCA-Info: Otherwise we will be unable to process your complaint properly
X-Postfilter: 1.3.40
X-Received-Bytes: 1946
 by: Chime Hart - Fri, 7 Apr 2023 23:38 UTC

Hi Richard: While I am only a user-and-not a programmer, I've probably been
enjoying trn since at least 1997. Other than hoping trn would some day have
yenc support, I have 1 quite specific happening which mysteriously comes up
while saving certain articles to a file.
In separate binary groups, eventually I will see an article number save to a
file name, but I am back at a prompt on the same line, no error messages,
nothing obvious when I examin an end of that file. I can seemingly grab that
same article in Alpine. I asked a Debian maintainer about this, but he hasn't
looked at this software in 10years. However, we tried compiling trn for a debug
option, so-far no luck. I surely am not running a GUI, but hopefully, we will
some day have improvements we can more consistantly enjoy. Thanks in advance
Chime

Re: I'm hacking on trn...want to join me?

<u0s6bu$24h80$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1245&group=news.software.readers#1245

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Sat, 8 Apr 2023 16:56:30 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u0s6bu$24h80$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <4143c063-c260-2a66-fb34-70d5b8ab7d66@hubert-humphrey.com>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Sat, 8 Apr 2023 16:56:30 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2245888"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Sat, 8 Apr 2023 16:56 UTC

[Please do not mail me a copy of your followup]

Chime Hart <chime@hubert-humphrey.com> spake the secret code
<4143c063-c260-2a66-fb34-70d5b8ab7d66@hubert-humphrey.com> thusly:

>[...] Other than hoping trn would some day have yenc support,

I'll open a github issue for yenc support.

>In separate binary groups, eventually I will see an article number save to a
>file name, but I am back at a prompt on the same line, no error messages,
>nothing obvious when I examin an end of that file. I can seemingly grab that
>same article in Alpine.

The next time you encounter such an article, if you can email it to me
I'll take a look.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<AABkMaYlOmoAAA2C.A3.flnews@WStation7.micha.freeshell.org>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1246&group=news.software.readers#1246

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!news.szaf.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: michael....@gmx.net (Michael Bäuerle)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Sat, 8 Apr 2023 19:36:37 +0200 (CEST)
Lines: 21
Message-ID: <AABkMaYlOmoAAA2C.A3.flnews@WStation7.micha.freeshell.org>
References: <u0ffth$1u1oi$1@news.xmission.com>
Mime-Version: 1.0
X-Trace: individual.net 7ELUB9X9s4rmeU+7VgDCiQLTZ1/JeyxUjcsfpqcghQlVg6U4rv
X-Orig-Path: not-for-mail
Cancel-Lock: sha1:1jVOO5CPGJkunHhqHAkVYRqcNfo= sha256:q+7AOYe5gGBl2PxpBa1B4yhJrWkIZ4sxwu04B0dao44= sha1:AVhfOsk2KiXNxbhNHnzVHmzMsMg=
Injection-Date: Sat, 8 Apr 2023 17:36:37 -0000
User-Agent: flnews/1.2.0pre21 (for GNU/Linux)
 by: Michael Bäuerle - Sat, 8 Apr 2023 17:36 UTC

Richard wrote:
>
> [...]
> Vision of the final result (no particular order):
> - True STARTTLS/NNTPS support
> - Curses windows used to show all the various things:
> - newsgroup selection
> - article selection
> - thread selection
> - KILL files
> - etc.
> - A GUI for reading news that's as convenient as the trn TUI.
> - Let you manipulate the various windows like people do in vim/emacs
> allowing you to choose what is on-screen and where
> - Client should be asynchronously advancing through the unread articles
> at the "speed of light" to apply scoring/threading/KILL file processing
> as far ahead as possible while you read.
> - As easy to build on Windows as it is on *nix.

I'm not using trn myself, but in the german hierarchy users sometimes
break things with trn. Better support for MIME would be nice.

Re: I'm hacking on trn...want to join me?

<u0suom$1eoqb$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1247&group=news.software.readers#1247

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Sat, 8 Apr 2023 23:52:54 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 8
Message-ID: <u0suom$1eoqb$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkMaYlOmoAAA2C.A3.flnews@WStation7.micha.freeshell.org>
Injection-Date: Sat, 8 Apr 2023 23:52:54 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="70c629345cb83ceaaca46cbc7f00c2a2";
logging-data="1532747"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19rwqrq+7dJF7eFjby7La1sn/pu/jJzJ1U="
Cancel-Lock: sha1:wX0KBEFIx9fXAkEFR66t0X6F+Yw=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Sat, 8 Apr 2023 23:52 UTC

Michael Bauerle <michael.baeuerle@gmx.net> wrote:

>I'm not using trn myself, but in the german hierarchy users sometimes
>break things with trn. Better support for MIME would be nice.

When I declare a character set, I just copy MIME headers into the
article. I can do it with a macro. The headers are right there for the
user to edit.

Re: I'm hacking on trn...want to join me?

<eli$2304091625@qaz.wtf>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1248&group=news.software.readers#1248

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.panix5.panix.com!qz!not-for-mail
From: *...@eli.users.panix.com (Eli the Bearded)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Sun, 9 Apr 2023 20:27:20 -0000 (UTC)
Organization: Some absurd concept
Message-ID: <eli$2304091625@qaz.wtf>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkMaYlOmoAAA2C.A3.flnews@WStation7.micha.freeshell.org>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Injection-Date: Sun, 9 Apr 2023 20:27:20 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="panix5.panix.com:166.84.1.5";
logging-data="19167"; mail-complaints-to="abuse@panix.com"
User-Agent: Vectrex rn 2.1 (beta)
X-Liz: It's actually happened, the entire Internet is a massive game of Redcode
X-Motto: "Erosion of rights never seems to reverse itself." -- kenny@panix
X-US-Congress: Moronic Fucks.
X-Attribution: EtB
XFrom: is a real address
Encrypted: double rot-13
 by: Eli the Bearded - Sun, 9 Apr 2023 20:27 UTC

In news.software.readers, Michael Bäuerle <michael.baeuerle@gmx.net> wrote:
> I'm not using trn myself, but in the german hierarchy users sometimes
> break things with trn. Better support for MIME would be nice.

The acli fork of trn vastly improves charset support.

https://github.com/acli/trn

It's what I use now.

Elijah
------
notes the accent mark in the name

Re: I'm hacking on trn...want to join me?

<u11osn$27c1f$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1249&group=news.software.readers#1249

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Mon, 10 Apr 2023 19:43:19 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u11osn$27c1f$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkMaYlOmoAAA2C.A3.flnews@WStation7.micha.freeshell.org> <u0suom$1eoqb$1@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Mon, 10 Apr 2023 19:43:19 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2338863"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Mon, 10 Apr 2023 19:43 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u0suom$1eoqb$1@dont-email.me> thusly:

>Michael Bauerle <michael.baeuerle@gmx.net> wrote:
>
>>I'm not using trn myself, but in the german hierarchy users sometimes
>>break things with trn. Better support for MIME would be nice.
>
>When I declare a character set, I just copy MIME headers into the
>article. I can do it with a macro. The headers are right there for the
>user to edit.

Presumably the problem is that the reply doesn't contain the necessary
MIME headers that are relevant for the quoed material or for the
content of the reply?
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<u11ouj$27c1f$2@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1250&group=news.software.readers#1250

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Mon, 10 Apr 2023 19:44:19 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u11ouj$27c1f$2@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkMaYlOmoAAA2C.A3.flnews@WStation7.micha.freeshell.org> <eli$2304091625@qaz.wtf>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Mon, 10 Apr 2023 19:44:19 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2338863"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Mon, 10 Apr 2023 19:44 UTC

[Please do not mail me a copy of your followup]

Eli the Bearded <*@eli.users.panix.com> spake the secret code
<eli$2304091625@qaz.wtf> thusly:

>In news.software.readers, Michael Bäuerle <michael.baeuerle@gmx.net> wrote:
>> I'm not using trn myself, but in the german hierarchy users sometimes
>> break things with trn. Better support for MIME would be nice.
>
>The acli fork of trn vastly improves charset support.

I looked at his fork and there are more edits there than I thought
based on the original patch ticket in sourceforge.

It will take some time for me to integrate those edits over to my
repository.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<u11ugv$2anur$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1251&group=news.software.readers#1251

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Mon, 10 Apr 2023 21:19:27 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 52
Message-ID: <u11ugv$2anur$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkMaYlOmoAAA2C.A3.flnews@WStation7.micha.freeshell.org> <u0suom$1eoqb$1@dont-email.me> <u11osn$27c1f$1@news.xmission.com>
Injection-Date: Mon, 10 Apr 2023 21:19:27 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="784a433c619854720ebd766221ce10a7";
logging-data="2449371"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX191uIIdJPcs4RiQgWktD2PJgHPFoUkYPgU="
Cancel-Lock: sha1:aaeWRRXh5D26m1Btnsm4S4bjO2c=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Mon, 10 Apr 2023 21:19 UTC

legalize+jeeves@mail.xmission.com (Richard) wrote:
>"Adam H. Kerman" <ahk@chinet.com> spake:
>>Michael Bauerle <michael.baeuerle@gmx.net> wrote:

>>>I'm not using trn myself, but in the german hierarchy users sometimes
>>>break things with trn. Better support for MIME would be nice.

>>When I declare a character set, I just copy MIME headers into the
>>article. I can do it with a macro. The headers are right there for the
>>user to edit.

>Presumably the problem is that the reply doesn't contain the necessary
>MIME headers that are relevant for the quoed material or for the
>content of the reply?

I don't see why that matters. You're expecting the newsreader to do
something it cannot do and no newsreader can.

I try to substitute for or eliminate non-ASCII characters, and then not
bother to declare a character set. All too often, I post in a thread
with a string of precursor followups. The root article copied and pasted
from the Web and may not have preserved the original character set in
pasting, or if it did, the MIME header doesn't match.

The author of the next followup uses a newsreader that cannot parse for
the character set in use and messes up all the non-ASCII characters
because the author declared a mismatched character set.

In my followup, I inherit the mess, which may be a mix of non-ASCII
8-bit characters and UTF-8 characters or just bad translations.
Sometimes I have to go back to the original Web article that was
plagarized to figure out what the hell the original character set was in
a portion of the quote, then figure out what character set was in use in
another portion of the quote.

Then I perform ASCII substitutions so that when I'm quoted, there's no
more mess.

In followup, no newsreader should ever declare that the character set in
use is the one used by the precursor article or the one that the user
uses by default. It has to parse and the user has to fix the inherited
mess at times anyway which will be unparseable.

I don't expect trn to do any of that. That's why I use macros or just my
own eyeballs. I really try to avoid using non-ASCII characters whenever
possible.

I have seen newsreaders that inherit encoded word on Subject, decode it,
then put unencoded non-ASCII characters on Subject because it's not
programmed to re-encoded before injecting. Aargh.

Use ASCII. It's universally readable.

Re: I'm hacking on trn...want to join me?

<u127t0$27k3v$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1252&group=news.software.readers#1252

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Mon, 10 Apr 2023 23:59:28 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u127t0$27k3v$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u0suom$1eoqb$1@dont-email.me> <u11osn$27c1f$1@news.xmission.com> <u11ugv$2anur$1@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Mon, 10 Apr 2023 23:59:28 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2347135"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Mon, 10 Apr 2023 23:59 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u11ugv$2anur$1@dont-email.me> thusly:

>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>"Adam H. Kerman" <ahk@chinet.com> spake:
>>>Michael Bauerle <michael.baeuerle@gmx.net> wrote:
>
>>>>I'm not using trn myself, but in the german hierarchy users sometimes
>>>>break things with trn. Better support for MIME would be nice.
>
>>>When I declare a character set, I just copy MIME headers into the
>>>article. I can do it with a macro. The headers are right there for the
>>>user to edit.
>
>>Presumably the problem is that the reply doesn't contain the necessary
>>MIME headers that are relevant for the quoed material or for the
>>content of the reply?
>
>I don't see why that matters. You're expecting the newsreader to do
>something it cannot do and no newsreader can.

I see now, thanks for clarifying.

So when you say "better support for MIME would be nice", what exactly
is missing?
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<u129dh$2c6tu$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1253&group=news.software.readers#1253

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Tue, 11 Apr 2023 00:25:21 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 32
Message-ID: <u129dh$2c6tu$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <u11osn$27c1f$1@news.xmission.com> <u11ugv$2anur$1@dont-email.me> <u127t0$27k3v$1@news.xmission.com>
Injection-Date: Tue, 11 Apr 2023 00:25:21 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="34e68e3891eed7817c8a8e0390cf771b";
logging-data="2497470"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18mUxQmnRrMX17vhsYQ190mYpRPnsml+0w="
Cancel-Lock: sha1:m6OlPSs6odPJr81n1dUqjZ/RXhE=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Tue, 11 Apr 2023 00:25 UTC

legalize+jeeves@mail.xmission.com (Richard) wrote:
>Adam H. Kerman <ahk@chinet.com> spake:
>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>Adam H. Kerman <ahk@chinet.com> spake:
>>>>Michael Bauerle <michael.baeuerle@gmx.net> wrote:

>>>>>I'm not using trn myself, but in the german hierarchy users sometimes
>>>>>break things with trn. Better support for MIME would be nice.

>>>>When I declare a character set, I just copy MIME headers into the
>>>>article. I can do it with a macro. The headers are right there for the
>>>>user to edit.

>>>Presumably the problem is that the reply doesn't contain the necessary
>>>MIME headers that are relevant for the quoed material or for the
>>>content of the reply?

>>I don't see why that matters. You're expecting the newsreader to do
>>something it cannot do and no newsreader can.

>I see now, thanks for clarifying.

>So when you say "better support for MIME would be nice", what exactly
>is missing?

I didn't say that; Michael did. I'm guessing that what's breaking
articles is a mix of UTF-8 and 8-bit used by different users, then not
declaring which one is in use in followup.

You'd need to add a parsing mechanism, then add the appropriate MIME
header to declare the character set. Still, the user has to clean up the
mess of broken quotes himself.

Re: I'm hacking on trn...want to join me?

<u141o0$28gtp$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1257&group=news.software.readers#1257

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Tue, 11 Apr 2023 16:26:40 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u141o0$28gtp$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u11ugv$2anur$1@dont-email.me> <u127t0$27k3v$1@news.xmission.com> <u129dh$2c6tu$1@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Tue, 11 Apr 2023 16:26:40 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2376633"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Tue, 11 Apr 2023 16:26 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u129dh$2c6tu$1@dont-email.me> thusly:

>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>So when you say "better support for MIME would be nice", what exactly
>>is missing?
>
>I didn't say that; Michael did.

Oops, sorry for misattributing.

>I'm guessing that what's breaking
>articles is a mix of UTF-8 and 8-bit used by different users, then not
>declaring which one is in use in followup.

Didn't we conclude on this thread that it's up to the user to set the
headers correctly?

Technically, trn doesn't do the posting; inews does the posting as a
separate program after having invoked your editor on the post.

I suppose inews could inspect the content of the post more deeply and
complain about use of 8-bit characters and/or detect UTF-8. It could
offer to adjust the headers as best it can and let you re-edit.

>You'd need to add a parsing mechanism, then add the appropriate MIME
>header to declare the character set. Still, the user has to clean up the
>mess of broken quotes himself.

You could improve this yourself by setting NEWSPOSTER to a different
program/script that does this detecting and munging of the headers
before invoking inews.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<u143j5$2m3va$4@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1258&group=news.software.readers#1258

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Tue, 11 Apr 2023 16:58:13 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 42
Message-ID: <u143j5$2m3va$4@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <u127t0$27k3v$1@news.xmission.com> <u129dh$2c6tu$1@dont-email.me> <u141o0$28gtp$1@news.xmission.com>
Injection-Date: Tue, 11 Apr 2023 16:58:13 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="34e68e3891eed7817c8a8e0390cf771b";
logging-data="2822122"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/Mf7+/yhXC4P3+owXMQAKLIFLWHsSGNsM="
Cancel-Lock: sha1:P23IWj2eFjNyL8KyHDh31u6UgwE=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Tue, 11 Apr 2023 16:58 UTC

legalize+jeeves@mail.xmission.com (Richard) wrote:
>"Adam H. Kerman" <ahk@chinet.com> spake:
>>legalize+jeeves@mail.xmission.com (Richard) wrote:

>>>So when you say "better support for MIME would be nice", what exactly
>>>is missing?

>>I didn't say that; Michael did.

>Oops, sorry for misattributing.

>>I'm guessing that what's breaking
>>articles is a mix of UTF-8 and 8-bit used by different users, then not
>>declaring which one is in use in followup.

>Didn't we conclude on this thread that it's up to the user to set the
>headers correctly?

Absolutely. trn expects the user to fix up his own headers!

>Technically, trn doesn't do the posting; inews does the posting as a
>separate program after having invoked your editor on the post.

>I suppose inews could inspect the content of the post more deeply and
>complain about use of 8-bit characters and/or detect UTF-8. It could
>offer to adjust the headers as best it can and let you re-edit.

>>You'd need to add a parsing mechanism, then add the appropriate MIME
>>header to declare the character set. Still, the user has to clean up the
>>mess of broken quotes himself.

>You could improve this yourself by setting NEWSPOSTER to a different
>program/script that does this detecting and munging of the headers
>before invoking inews.

You're right, of course. If an acceptable parsing mechanism already
exists, why re-invent the wheel?

Still, there's just no way to fix up multi-level quotes that used
different character sets mis-translated without the user going back to
the original text to figure out what it was supposed to be and then
translating it consistently with the rest of the text.

Getting MIME headers right (was: I'm hacking on trn...want to join me?)

<u16j67$29qek$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1260&group=news.software.readers#1260

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Getting MIME headers right (was: I'm hacking on trn...want to join me?)
Date: Wed, 12 Apr 2023 15:36:39 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u16j67$29qek$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u141o0$28gtp$1@news.xmission.com> <u143j5$2m3va$4@dont-email.me> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 15:36:39 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2419156"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 15:36 UTC

[Please do not mail me a copy of your followup]

Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake the secret code
<AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> thusly:

>I think such mistakes would not occur if trn would automatically convert
>the content to quote into the encoding used by the editor.
>The source encoding is declared in the MIME header of the article.

The source encoding *MAY* be declared in the MIME header of the article.
There's no guarantee that it actually is declared as such. We've
already stated that malformed messages can be injected into news due
to user error (e.g. they didn't configure their editor to add
appropriate MIME headers).

>The target encoding should be the one the editor is using (manually
>configured, if trn cannot automatically detect it).

How can trn know anything about your editor? It's entirely outside of
trn's control.

>The conversion itself can be done with iconv.

This is a unix-centric assumption. It *could* be done with iconv.

>If the user has configured e.g. US-ASCII to quote an article written in
>Unicode, replacement characters should be inserted if the encoding
>conversion is not possible (instead of simply copying the bytes that are
>then interpreted wrong by the editor and the recipients).

All of this seems outside the control of trn and mostly a consequence
of how you edit the article. The existing NEWSPOSTER environment variable
can be configured to a script that would do all that you propose.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<u16j68$32uql$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1261&group=news.software.readers#1261

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 15:36:40 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 95
Message-ID: <u16j68$32uql$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <u141o0$28gtp$1@news.xmission.com> <u143j5$2m3va$4@dont-email.me> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de>
Injection-Date: Wed, 12 Apr 2023 15:36:40 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="595da4453ea6f81ea9a07e161a9c1d7b";
logging-data="3242837"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18qKH0LcRCI8WTg2KkBRtgQvOYgVWqZS3s="
Cancel-Lock: sha1:3f816bdPsXxjlFQsW2cdbmw6aOE=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Wed, 12 Apr 2023 15:36 UTC

Michael Bauerle <michael.baeuerle@gmx.net> wrote:
>Adam H. Kerman wrote:

>>[...]
>>Still, there's just no way to fix up multi-level quotes that used
>>different character sets mis-translated without the user going back to
>>the original text to figure out what it was supposed to be and then
>>translating it consistently with the rest of the text.

>If it already happened, this is something that is hard to impossible
>to repair for some encodings. Most newsreaders don't even try and
>such a repair algorithm is not what I had in mind.

>What I wanted to propose is that trn does not create and send such
>mixture of encodings and correctly label the encoding used according
>to MIME (the problem is that trn users produce such broken articles,
>not that they cannnot read them).

How is it trn's fault? trn isn't translating anything! It receives
characters and passes them along to inews -h. It's up to the author to
recognize the encoding.

Take a typical situation I encounter: Root article has non-ASCII
characters and declares UTF-8. The author of the followup has his
default character set ISO-8859-1 and has no parsing mechanism. Because
it's an Apple client, it does weird character substitutions, different
from what a Windows client might do. Either way, it ignored the declared
character set of the precursor article.

If I do a further followup, I have to edit manually AND go back to the
root article to see if I can figure out what the character was intended
to be. If I can, I perform ASCII substitutions.

There's no way to write a parsing mechanism to repair inherited
mistranslated characters getting rid of the bad substitutions.

At least with trn, given that the author is able to edit headers and is
actually expected to add necessary headers, these things can be
repaired. The author using the Apple client has no ability to fix the
mess he created in the composer within the newsreader.

In followup, if the declared character set truly describes the character
set inherited from the precursor article, at least a decent parser
called by trn could deal with it, or even translate to another character
set if necessary and an outside process could add the matching MIME
header.

Since I'm a traditionalist, I'd like to have another chance to edit to
verify that the parser caught everything, and I still want to get rid of
characters that do not belong in plain text, like nonbreaking space.

>It looks like the users don't know what they are doing (not really the
>fault of trn in this sense) and their editor is not configured for the
>encoding of the content that is quoted. Maybe it is too inconvenient to
>change the encoding configuration.

The editor is an outside process. It's not built into trn. It's not the
job of the editor to parse, although there seems to be process that
decodes BASE64 or QP, neither of which belong in Usenet. That happens
before the editor sees it. There'd have to be another process before it
gets to the editor to test that the declared character set of the
precursor article matches what's actually there.

The editor is likely using a character set from an environment variable.

>I think such mistakes would not occur if trn would automatically convert
>the content to quote into the encoding used by the editor.
>The source encoding is declared in the MIME header of the article.

Snarf

When it actually matches!

>The target encoding should be the one the editor is using (manually
>configured, if trn cannot automatically detect it).

Not what trn does. An outside process would have to be called.

>The conversion itself can be done with iconv.
>This would preserve the users choice for the target encoding (would
>not enforce the usage of Unicode).

In my opinion, if there are non-ASCII characters but the lowest
denomination character set is 8-bit, that's the character set to use.

My opinion is shared with nearly no one.

>If the user has configured e.g. US-ASCII to quote an article written in
>Unicode, replacement characters should be inserted if the encoding
>conversion is not possible (instead of simply copying the bytes that are
>then interpreted wrong by the editor and the recipients).

Absolutely, especially ASCII punctuation characters for 8-bit or UTF-8
punctuation characters. Typesetting characters do not belong in plain
text communication.

Re: Getting MIME headers right

<u16jt6$32uql$3@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1263&group=news.software.readers#1263

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 15:48:54 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 6
Message-ID: <u16jt6$32uql$3@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <u143j5$2m3va$4@dont-email.me> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j67$29qek$1@news.xmission.com>
Injection-Date: Wed, 12 Apr 2023 15:48:54 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="595da4453ea6f81ea9a07e161a9c1d7b";
logging-data="3242837"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18dC+dp8+XBFnAeMf4fifu/qGT8YBeUlC0="
Cancel-Lock: sha1:KBbtJkULIy0ryGARvgdpSzNAw14=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Wed, 12 Apr 2023 15:48 UTC

legalize+jeeves@mail.xmission.com (Richard) wrote:
>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:

How did you get undecoded encoded word on the attribution line?

>. . .

Re: I'm hacking on trn...want to join me?

<u16nt8$29spu$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1265&group=news.software.readers#1265

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 16:57:12 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u16nt8$29spu$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u143j5$2m3va$4@dont-email.me> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j68$32uql$1@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 16:57:12 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2421566"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 16:57 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u16j68$32uql$1@dont-email.me> thusly:

>Absolutely, especially ASCII punctuation characters for 8-bit or UTF-8
>punctuation characters. Typesetting characters do not belong in plain
>text communication.

With MIME headers, usenet isn't necessarily restricted to text/plain
style communication, but it has been the historical norm.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: Getting MIME headers right

<u16o20$29spu$2@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1266&group=news.software.readers#1266

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 16:59:44 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u16o20$29spu$2@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j67$29qek$1@news.xmission.com> <u16jt6$32uql$3@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 16:59:44 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2421566"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 16:59 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u16jt6$32uql$3@dont-email.me> thusly:

>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:
>
>How did you get undecoded encoded word on the attribution line?

%[from] in my ATTRIBUTION setting.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: Getting MIME headers right

<AABkNtz+CKEAAAdh.A3.flnews@WStation5.stz-e.de>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1267&group=news.software.readers#1267

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!news.szaf.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: michael....@stz-e.de (Michael Bäuerle)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 18:31:58 +0200 (CEST)
Lines: 54
Message-ID: <AABkNtz+CKEAAAdh.A3.flnews@WStation5.stz-e.de>
References: <u0ffth$1u1oi$1@news.xmission.com> <u141o0$28gtp$1@news.xmission.com> <u143j5$2m3va$4@dont-email.me> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j67$29qek$1@news.xmission.com>
Reply-To: Michael Bäuerle <michael.baeuerle@gmx.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=fixed
Content-Transfer-Encoding: 8bit
X-Trace: individual.net k2vqfZrTRi8WTnKcjxKi0wpzQPdkoXXCP2/sNupDTxnea8uT+K
X-Orig-Path: not-for-mail
Cancel-Lock: sha1:2EQTZrw5QJ3xf67ZzRANMIsV+IQ= sha256:kbdzYgIi8uKvO75eArWOm1QFyFKJtPzyyHoT1np8H8g= sha1:GC//L3qrDEgt36E3vC7VAwcQKSM=
Injection-Date: Wed, 12 Apr 2023 16:31:58 -0000
User-Agent: flnews/1.2.0pre21 (for NetBSD)
 by: Michael Bäuerle - Wed, 12 Apr 2023 16:31 UTC

Richard wrote:
> Michael Bäuerle wrote:
> >
> > I think such mistakes would not occur if trn would automatically convert
> > the content to quote into the encoding used by the editor.
> > The source encoding is declared in the MIME header of the article.
>
> The source encoding *MAY* be declared in the MIME header of the article.
> There's no guarantee that it actually is declared as such. We've
> already stated that malformed messages can be injected into news due
> to user error (e.g. they didn't configure their editor to add
> appropriate MIME headers).

Please forget incoming malformed messages completely. If something is
already broken, it is completely acceptable that you need to repair it
manually. My proposal is not about repairing malformed articles.

What should be expected from a newsreader is correct behaviour with
RFC 5536 conformant messages.
Because RFC 5536 requires MIME, every article with 8-bit data in the
body, but without a MIME declaration in the header, is malformed by
definition.

> > The target encoding should be the one the editor is using (manually
> > configured, if trn cannot automatically detect it).
>
> How can trn know anything about your editor? It's entirely outside of
> trn's control.

To be honest I don't know how trn works, I only see the results.

If trn does not know what the editor expects, the user should be able
to configure the encoding.

> > The conversion itself can be done with iconv.
>
> This is a unix-centric assumption. It *could* be done with iconv.

Nobody will complain if something else is used, as long as it works.

> > If the user has configured e.g. US-ASCII to quote an article written in
> > Unicode, replacement characters should be inserted if the encoding
> > conversion is not possible (instead of simply copying the bytes that are
> > then interpreted wrong by the editor and the recipients).
>
> All of this seems outside the control of trn and mostly a consequence
> of how you edit the article. The existing NEWSPOSTER environment variable
> can be configured to a script that would do all that you propose.

Not every user is a programmer too.

If this must be implemented with a script, it should be shipped with
the newsreader and the documentation should explain how to produce
RFC 5536 conformant articles with it.

Re: Getting MIME headers right

<u16p9v$29tdt$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1268&group=news.software.readers#1268

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 17:21:03 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u16p9v$29tdt$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j67$29qek$1@news.xmission.com> <AABkNtz+CKEAAAdh.A3.flnews@WStation5.stz-e.de>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 17:21:03 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2422205"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 17:21 UTC

[Please do not mail me a copy of your followup]

Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake the secret code
<AABkNtz+CKEAAAdh.A3.flnews@WStation5.stz-e.de> thusly:

>Richard wrote:
>> Michael B�uerle wrote:
>> > The target encoding should be the one the editor is using (manually
>> > configured, if trn cannot automatically detect it).
>>
>> How can trn know anything about your editor? It's entirely outside of
>> trn's control.
>
>To be honest I don't know how trn works, I only see the results.

trn is a traditional command-line news reader and doesn't directly
post anything at all. When you request to post a follow-up or new
article, trn invokes your editor to prepare the message (including all
headers appearing "naked" in the editor) and then confirms that you
want to post the edited message. If confirmed, trn injects the
article into the newsfeed with a program called inews that does the
actual posting. The source for inews comes with trn and is
distributed with trn but the act of injecting a news article is
considered the responsibility of a program separate from trn and can
be configured to be some other program besides inews.

Similarly, composing an article -- including all it's headers -- is a
responsibility of an editor program and not trn.

>If trn does not know what the editor expects, the user should be able
>to configure the encoding.

They can, but often don't :).

>> All of this seems outside the control of trn and mostly a consequence
>> of how you edit the article. The existing NEWSPOSTER environment variable
>> can be configured to a script that would do all that you propose.
>
>Not every user is a programmer too.

True, but trn assumes a certain level of knowledge in order to use its
more advanced features, such as configuring things through environment
variables.

Because trn doesn't own "the whole process" like a canned GUI-based
news reader on a macOS or Windows system, it assumes the user knows
how to deal with such things on their own.

>If this must be implemented with a script, it should be shipped with
>the newsreader and the documentation should explain how to produce
>RFC 5536 conformant articles with it.

There really isn't any way to write such a script and make it useful
to everyone. Trn can supply examples (and does) but invoking it
unconditionally would be a mistake.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<AABkNuo349EAAAdh.A3.flnews@WStation5.stz-e.de>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1269&group=news.software.readers#1269

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!news.mixmin.net!newsreader4.netcologne.de!news.netcologne.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: michael....@stz-e.de (Michael Bäuerle)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 19:28:23 +0200 (CEST)
Lines: 133
Message-ID: <AABkNuo349EAAAdh.A3.flnews@WStation5.stz-e.de>
References: <u0ffth$1u1oi$1@news.xmission.com> <u141o0$28gtp$1@news.xmission.com> <u143j5$2m3va$4@dont-email.me> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j68$32uql$1@dont-email.me>
Reply-To: Michael Bäuerle <michael.baeuerle@gmx.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=fixed
Content-Transfer-Encoding: 8bit
X-Trace: individual.net fye154ye78MW07SJodhsbwOcW+6kr/+gL9Dg3y6InPIYR6xFIb
X-Orig-Path: not-for-mail
Cancel-Lock: sha1:zzUlfPzAdyh1MLqEUZagRCkcs6E= sha256:+zRyEoBSDVlfSYkFhJr193UZkISJy94LyGkr8sIEnc0= sha1:qUIYUc6IuLqsvEIMdPtHoI3cAUc=
Injection-Date: Wed, 12 Apr 2023 17:28:23 -0000
User-Agent: flnews/1.2.0pre21 (for NetBSD)
 by: Michael Bäuerle - Wed, 12 Apr 2023 17:28 UTC

Adam H. Kerman wrote:
> Michael Bäuerle wrote:
> > Adam H. Kerman wrote:
> > >
> > > [...]
> > > Still, there's just no way to fix up multi-level quotes that used
> > > different character sets mis-translated without the user going back to
> > > the original text to figure out what it was supposed to be and then
> > > translating it consistently with the rest of the text.
> >
> > If it already happened, this is something that is hard to impossible
> > to repair for some encodings. Most newsreaders don't even try and
> > such a repair algorithm is not what I had in mind.
> >
> > What I wanted to propose is that trn does not create and send such
> > mixture of encodings and correctly label the encoding used according
> > to MIME (the problem is that trn users produce such broken articles,
> > not that they cannnot read them).
>
> How is it trn's fault? trn isn't translating anything! It receives
> characters and passes them along to inews -h. It's up to the author to
> recognize the encoding.

Then read my proposal "make it easier for the users to avoid mistakes".

> Take a typical situation I encounter: Root article has non-ASCII
> characters and declares UTF-8. The author of the followup has his
> default character set ISO-8859-1 and has no parsing mechanism. Because
> it's an Apple client, it does weird character substitutions, different
> from what a Windows client might do. Either way, it ignored the declared
> character set of the precursor article.

Then we are back to the case of malformed incoming data.
This is not what I am talking about.

> [...]
> In followup, if the declared character set truly describes the character
> set inherited from the precursor article, at least a decent parser
> called by trn could deal with it, or even translate to another character
> set if necessary and an outside process could add the matching MIME
> header.

This is what I propose. It should be easy for the user to do this.
The code for it should be shipped with the newsreader, not every user
is a programmer too.

> Since I'm a traditionalist, I'd like to have another chance to edit to
> verify that the parser caught everything, and I still want to get rid of
> characters that do not belong in plain text, like nonbreaking space.

I think this would always be the case, because all the automatic
processing should be finished before the editor is launched.

> > It looks like the users don't know what they are doing (not really the
> > fault of trn in this sense) and their editor is not configured for the
> > encoding of the content that is quoted. Maybe it is too inconvenient to
> > change the encoding configuration.
>
> The editor is an outside process. It's not built into trn. It's not the
> job of the editor to parse, although there seems to be process that
> decodes BASE64 or QP, neither of which belong in Usenet.

No longer true since RFC 5536:
<https://www.rfc-editor.org/rfc/rfc5536#section-2.3>
|
| User agents MUST meet the definition of MIME conformance in [RFC2049]
| and MUST also support [RFC2231]. This level of MIME conformance
| provides support for internationalization and multimedia in message
| bodies [RFC2045], [RFC2046], and [RFC2231], and support for
| internationalization of header fields [RFC2047] and [RFC2231]. [...]

> That happens
> before the editor sees it. There'd have to be another process before it
> gets to the editor to test that the declared character set of the
> precursor article matches what's actually there.

This part should be added.

Decode transfer encoding first.
Then convert the encoding to match the editor.

Create a MIME declaration for the outgoing data, if not US-ASCII.

> The editor is likely using a character set from an environment variable.

Maybe not generic enough to be used by the newsreader too.

> > I think such mistakes would not occur if trn would automatically convert
> > the content to quote into the encoding used by the editor.
> > The source encoding is declared in the MIME header of the article.
>
> Snarf
>
> When it actually matches!

It does not match in the articles I talk about (the user has not
converted the encoding of the incoming data, but has written his reply
with a different encoding). This is the case that should be easier to
avoid.

> > The target encoding should be the one the editor is using (manually
> > configured, if trn cannot automatically detect it).
>
> Not what trn does. An outside process would have to be called.

Nontrivial outside processes (that are required to correctly process
RFC 5536 conformant articles) should be shipped with the newsreader.

> > The conversion itself can be done with iconv.
> > This would preserve the users choice for the target encoding (would
> > not enforce the usage of Unicode).
>
> In my opinion, if there are non-ASCII characters but the lowest
> denomination character set is 8-bit, that's the character set to use.
>
> My opinion is shared with nearly no one.

I share your opinion (as you can see in my articles that use ISO 8859-1,
if possible).

From RFC 2046:
<https://www.rfc-editor.org/rfc/rfc2046#section-4.1.2>
|
| [...]
| In general, composition software should always use the "lowest common
| denominator" character set possible. [...] More generally,
| if a widely-used character set is a subset of another character set,
| and a body contains only characters in the widely-used subset, it
| should be labelled as being in that subset. [...]

But this text is decades old. If a successor to this RFC would be
written today, it likely would say something like "Always use Unicode
with UTF-8 encoding".

Re: Getting MIME headers right

<u16rfl$emd$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1270&group=news.software.readers#1270

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 17:58:13 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 10
Message-ID: <u16rfl$emd$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16j67$29qek$1@news.xmission.com> <u16jt6$32uql$3@dont-email.me> <u16o20$29spu$2@news.xmission.com>
Injection-Date: Wed, 12 Apr 2023 17:58:13 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="595da4453ea6f81ea9a07e161a9c1d7b";
logging-data="15053"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+93fzM4mfTKEtfxIuGmAo5RP+aef5jlBA="
Cancel-Lock: sha1:Ttsj98GOfZIyGPZinAK/AeSiSp0=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Wed, 12 Apr 2023 17:58 UTC

legalize+jeeves@mail.xmission.com (Richard) wrote:
>"Adam H. Kerman" <ahk@chinet.com> spake:
>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:

>>How did you get undecoded encoded word on the attribution line?

>%[from] in my ATTRIBUTION setting.

Right, but you aren't decoding first before creating the attribution line.

Re: I'm hacking on trn...want to join me?

<u16sn5$o1d$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1271&group=news.software.readers#1271

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 18:19:17 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 111
Message-ID: <u16sn5$o1d$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j68$32uql$1@dont-email.me> <AABkNuo349EAAAdh.A3.flnews@WStation5.stz-e.de>
Injection-Date: Wed, 12 Apr 2023 18:19:17 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="595da4453ea6f81ea9a07e161a9c1d7b";
logging-data="24621"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19Csltsme62oXWAMjAMM3jsfJH9Z12jGzM="
Cancel-Lock: sha1:jOv398GyGpUAF/lp+BkdqzBf21o=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Wed, 12 Apr 2023 18:19 UTC

Michael Bauerle <michael.baeuerle@gmx.net> wrote:
>Adam H. Kerman wrote:
>>Michael Bauerle wrote:
>>>Adam H. Kerman wrote:

>>>>[...]

>>[...]
>>In followup, if the declared character set truly describes the character
>>set inherited from the precursor article, at least a decent parser
>>called by trn could deal with it, or even translate to another character
>>set if necessary and an outside process could add the matching MIME
>>header.

>This is what I propose. It should be easy for the user to do this.
>The code for it should be shipped with the newsreader, not every user
>is a programmer too.

This isn't programming. The best I can do is write macros. Richard was
talking about calling outside processes that already exist, not
requiring the user to write his own parser.

>>Since I'm a traditionalist, I'd like to have another chance to edit to
>>verify that the parser caught everything, and I still want to get rid of
>>characters that do not belong in plain text, like nonbreaking space.

>I think this would always be the case, because all the automatic
>processing should be finished before the editor is launched.

No, you need another parsing step after completing the editor to declare
the character set in the MIME header. For instance, if I think I've used
ASCII but there's still an invisible character like nonbreaking space
that's undesireable, I'd want to figure out why the proto article failed
parsing and get rid of the non-plain-text character.

>>>It looks like the users don't know what they are doing (not really the
>>>fault of trn in this sense) and their editor is not configured for the
>>>encoding of the content that is quoted. Maybe it is too inconvenient to
>>>change the encoding configuration.

>>The editor is an outside process. It's not built into trn. It's not the
>>job of the editor to parse, although there seems to be process that
>>decodes BASE64 or QP, neither of which belong in Usenet.

>No longer true since RFC 5536:
><https://www.rfc-editor.org/rfc/rfc5536#section-2.3>

>| User agents MUST meet the definition of MIME conformance in [RFC2049]
>| and MUST also support [RFC2231]. This level of MIME conformance
>| provides support for internationalization and multimedia in message
>| bodies [RFC2045], [RFC2046], and [RFC2231], and support for
>| internationalization of header fields [RFC2047] and [RFC2231]. [...]

The editor isn't the user agent for the purpose of complying with this
standard. trn is noncompliant and it's up to the user to add his own
MIME headers to declare the character set.

>>That happens
>>before the editor sees it. There'd have to be another process before it
>>gets to the editor to test that the declared character set of the
>>precursor article matches what's actually there.

>This part should be added.

>Decode transfer encoding first.
>Then convert the encoding to match the editor.

The editor? No. Use the character set in the terminal emulation,
otherwise you'll introduce yet another point of mismatch.

The trouble is that the terminal emulation needs to switch character
sets on the fly by somehow reading MIME headers and I have no idea how
any of that would happen. It's really really really outside trn.

When I follow up to certain articles, I deliberately create a character
set mismatch as it makes nonbreaking space visible and I can get rid of
that. No one but me would do that.

>Create a MIME declaration for the outgoing data, if not US-ASCII.

AFTER parsing the output from the composer

>>The editor is likely using a character set from an environment variable.

>Maybe not generic enough to be used by the newsreader too.

trn just outputs to display. It's up to the display -- in this case, a
terminal emulation or an XTERM-like environment -- to show the article.
Sometimes I have to change the character set the terminal emulation
displays to see the article as intended.

trn doesn't communicate the MIME header to the display. It's up to the
user to make sure there's a character set match between the displayed
article and the terminal emulation.

>>>I think such mistakes would not occur if trn would automatically convert
>>>the content to quote into the encoding used by the editor.
>>>The source encoding is declared in the MIME header of the article.

>>Snarf

>>When it actually matches!

>It does not match in the articles I talk about (the user has not
>converted the encoding of the incoming data, but has written his reply
>with a different encoding). This is the case that should be easier to
>avoid.

I agree with you here.

>>>. . .

Re: Getting MIME headers right

<u16uko$2a1b0$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1272&group=news.software.readers#1272

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 18:52:08 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u16uko$2a1b0$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16jt6$32uql$3@dont-email.me> <u16o20$29spu$2@news.xmission.com> <u16rfl$emd$1@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 18:52:08 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2426208"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 18:52 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u16rfl$emd$1@dont-email.me> thusly:

>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>"Adam H. Kerman" <ahk@chinet.com> spake:
>>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:
>
>>>How did you get undecoded encoded word on the attribution line?
>
>>%[from] in my ATTRIBUTION setting.
>
>Right, but you aren't decoding first before creating the attribution line.

The From: header value appears in the attribution the way it appears in
the message.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<u16upu$2a1b0$2@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1273&group=news.software.readers#1273

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 18:54:54 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u16upu$2a1b0$2@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de> <u16j68$32uql$1@dont-email.me> <AABkNuo349EAAAdh.A3.flnews@WStation5.stz-e.de>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 18:54:54 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2426208"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 18:54 UTC

[Please do not mail me a copy of your followup]

Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake the secret code
<AABkNuo349EAAAdh.A3.flnews@WStation5.stz-e.de> thusly:

>But this text is decades old. If a successor to this RFC would be
>written today, it likely would say something like "Always use Unicode
>with UTF-8 encoding".

The current NNTP RFC recommends using UTF-8 wherever possible (and
annotating it as such in messages).
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Pages:12
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor