Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

"I may be synthetic, but I'm not stupid" -- the artificial person, from _Aliens_


computers / news.software.readers / Re: I'm hacking on trn...want to join me?

SubjectAuthor
* I'm hacking on trn...want to join me?Richard
+* Re: I'm hacking on trn...want to join me?Chime Hart
|`- Re: I'm hacking on trn...want to join me?Richard
`* Re: I'm hacking on trn...want to join me?Michael Bäuerle
 +* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |`* Re: I'm hacking on trn...want to join me?Richard
 | `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |  `* Re: I'm hacking on trn...want to join me?Richard
 |   `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |    `* Re: I'm hacking on trn...want to join me?Richard
 |     +* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |`* Re: I'm hacking on trn...want to join me?Michael Bäuerle
 |     | +* Getting MIME headers right (was: I'm hacking on trn...want to join me?)Richard
 |     | |+* Re: Getting MIME headers rightAdam H. Kerman
 |     | ||`* Re: Getting MIME headers rightRichard
 |     | || `* Re: Getting MIME headers rightAdam H. Kerman
 |     | ||  `* Re: Getting MIME headers rightRichard
 |     | ||   `* Re: Getting MIME headers rightAdam H. Kerman
 |     | ||    +* Re: Getting MIME headers rightRichard
 |     | ||    |`- Re: Getting MIME headers rightAdam H. Kerman
 |     | ||    `* Re: Getting MIME headers rightFrank Slootweg
 |     | ||     `- Re: Getting MIME headers rightRichard
 |     | |`* Re: Getting MIME headers rightMichael Bäuerle
 |     | | `- Re: Getting MIME headers rightRichard
 |     | `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |  +- Re: I'm hacking on trn...want to join me?Richard
 |     |  `* Re: I'm hacking on trn...want to join me?Michael Bäuerle
 |     |   +* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |   |`* Re: I'm hacking on trn...want to join me?Richard
 |     |   | `* Re: I'm hacking on trn...want to join me?Adam H. Kerman
 |     |   |  `- Re: I'm hacking on trn...want to join me?Richard
 |     |   `- Re: I'm hacking on trn...want to join me?Richard
 |     `* Re: I'm hacking on trn...want to join me?Eli the Bearded
 |      `* Re: I'm hacking on trn...want to join me?Richard
 |       `* Re: I'm hacking on trn...want to join me?Eli the Bearded
 |        `- Re: I'm hacking on trn...want to join me?Richard
 `* Re: I'm hacking on trn...want to join me?Eli the Bearded
  `- Re: I'm hacking on trn...want to join me?Richard

Pages:12
Re: I'm hacking on trn...want to join me?

<u16v0a$2a1b0$3@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1274&group=news.software.readers#1274

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 18:58:18 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u16v0a$2a1b0$3@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16j68$32uql$1@dont-email.me> <AABkNuo349EAAAdh.A3.flnews@WStation5.stz-e.de> <u16sn5$o1d$1@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 18:58:18 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2426208"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 18:58 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u16sn5$o1d$1@dont-email.me> thusly:

>Michael Bauerle <michael.baeuerle@gmx.net> wrote:
>>Adam H. Kerman wrote:
>>>Michael Bauerle wrote:
>>>>Adam H. Kerman wrote:
>
>>>>>[...]
>
>>>[...]
>>>In followup, if the declared character set truly describes the character
>>>set inherited from the precursor article, at least a decent parser
>>>called by trn could deal with it, or even translate to another character
>>>set if necessary and an outside process could add the matching MIME
>>>header.
>
>>This is what I propose. It should be easy for the user to do this.
>>The code for it should be shipped with the newsreader, not every user
>>is a programmer too.
>
>This isn't programming. The best I can do is write macros. Richard was
>talking about calling outside processes that already exist, not
>requiring the user to write his own parser.

I think it's a reasonable request for trn's inews program to be
smarter about encodings and analyze the input file and add the
necessary headers to decorate the content to the best of it's ability.

I don't know if acli's fork of trn is already doing these things; I
think that fork has been focused on proper presentation of UTF-8
content in articles. It is my intention to merge those changes into
my fork.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1275&group=news.software.readers#1275

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!news.szaf.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: michael....@stz-e.de (Michael Bäuerle)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 15:32:53 +0200 (CEST)
Lines: 36
Message-ID: <AABkNrMFcqkAAAFo.A3.flnews@WStation5.stz-e.de>
References: <u0ffth$1u1oi$1@news.xmission.com> <u127t0$27k3v$1@news.xmission.com> <u129dh$2c6tu$1@dont-email.me> <u141o0$28gtp$1@news.xmission.com> <u143j5$2m3va$4@dont-email.me>
Reply-To: Michael Bäuerle <michael.baeuerle@gmx.net>
Mime-Version: 1.0
X-Trace: individual.net n1kom9H362E6Hhqtg1kTTgmvLhqWwRaCKkf7L+BwaezDyRV4jB
X-Orig-Path: not-for-mail
Cancel-Lock: sha1:5WusrklOSKTUJz/KdFPetNlgCiE= sha256:Q4JMJidb+XFKmSVf+Z9Do4dtWuqIFKJezNmamzAfH+o= sha1:71sIkN0u2Bybx+6U1/HrfHxDZ28=
Injection-Date: Wed, 12 Apr 2023 13:32:53 -0000
User-Agent: flnews/1.2.0pre21 (for NetBSD)
 by: Michael Bäuerle - Wed, 12 Apr 2023 13:32 UTC

Adam H. Kerman wrote:
>
> [...]
> Still, there's just no way to fix up multi-level quotes that used
> different character sets mis-translated without the user going back to
> the original text to figure out what it was supposed to be and then
> translating it consistently with the rest of the text.

If it already happened, this is something that is hard to impossible
to repair for some encodings. Most newsreaders don't even try and
such a repair algorithm is not what I had in mind.

What I wanted to propose is that trn does not create and send such
mixture of encodings and correctly label the encoding used according
to MIME (the problem is that trn users produce such broken articles,
not that they cannnot read them).

It looks like the users don't know what they are doing (not really the
fault of trn in this sense) and their editor is not configured for the
encoding of the content that is quoted. Maybe it is too inconvenient to
change the encoding configuration.

I think such mistakes would not occur if trn would automatically convert
the content to quote into the encoding used by the editor.
The source encoding is declared in the MIME header of the article.
The target encoding should be the one the editor is using (manually
configured, if trn cannot automatically detect it).
The conversion itself can be done with iconv.
This would preserve the users choice for the target encoding (would
not enforce the usage of Unicode).

If the user has configured e.g. US-ASCII to quote an article written in
Unicode, replacement characters should be inserted if the encoding
conversion is not possible (instead of simply copying the bytes that are
then interpreted wrong by the editor and the recipients).

Re: I'm hacking on trn...want to join me?

<u17a4d$2if3$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1276&group=news.software.readers#1276

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 22:08:14 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 41
Message-ID: <u17a4d$2if3$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <AABkNuo349EAAAdh.A3.flnews@WStation5.stz-e.de> <u16sn5$o1d$1@dont-email.me> <u16v0a$2a1b0$3@news.xmission.com>
Injection-Date: Wed, 12 Apr 2023 22:08:14 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e209180b4b1fb5c5735b4c45c4131c69";
logging-data="84451"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/AIF39Fa0KdPl11c5kAdhT4zqsLh7NGhE="
Cancel-Lock: sha1:4I0mBrlt5+ZnGzItf2Ee0+4oVgQ=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Wed, 12 Apr 2023 22:08 UTC

legalize+jeeves@mail.xmission.com (Richard) wrote:
>"Adam H. Kerman" <ahk@chinet.com> spake:
>>Michael Bauerle <michael.baeuerle@gmx.net> wrote:
>>>Adam H. Kerman wrote:
>>>>Michael Bauerle wrote:
>>>>>Adam H. Kerman wrote:

>>>>>>[...]

>>>>[...]
>>>>In followup, if the declared character set truly describes the character
>>>>set inherited from the precursor article, at least a decent parser
>>>>called by trn could deal with it, or even translate to another character
>>>>set if necessary and an outside process could add the matching MIME
>>>>header.

>>>This is what I propose. It should be easy for the user to do this.
>>>The code for it should be shipped with the newsreader, not every user
>>>is a programmer too.

>>This isn't programming. The best I can do is write macros. Richard was
>>talking about calling outside processes that already exist, not
>>requiring the user to write his own parser.

>I think it's a reasonable request for trn's inews program to be
>smarter about encodings and analyze the input file and add the
>necessary headers to decorate the content to the best of it's ability.

I thought we were using inews from INN. I remember decades ago inews had
significant delays and timeouts but it got rewritten by Russ for
background processing. I didn't recall we used a home-grown inews.

The inews sanity checks are minimal and do not parse the body of the
article to declare a character set.

>I don't know if acli's fork of trn is already doing these things; I
>think that fork has been focused on proper presentation of UTF-8
>content in articles. It is my intention to merge those changes into
>my fork.

Ok.

Re: Getting MIME headers right

<u17aip$2if3$2@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1277&group=news.software.readers#1277

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 22:15:53 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 23
Message-ID: <u17aip$2if3$2@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16o20$29spu$2@news.xmission.com> <u16rfl$emd$1@dont-email.me> <u16uko$2a1b0$1@news.xmission.com>
Injection-Date: Wed, 12 Apr 2023 22:15:53 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e209180b4b1fb5c5735b4c45c4131c69";
logging-data="84451"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18tDve9z9h/nMX6GSuJw8YKHjXuz+skieo="
Cancel-Lock: sha1:jedNdqkglP4eCCyKAvewERT2gwQ=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Wed, 12 Apr 2023 22:15 UTC

legalize+jeeves@mail.xmission.com (Richard) wrote:
>"Adam H. Kerman" <ahk@chinet.com> spake:
>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>"Adam H. Kerman" <ahk@chinet.com> spake:
>>>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>>>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:

>>>>How did you get undecoded encoded word on the attribution line?

>>>%[from] in my ATTRIBUTION setting.

>>Right, but you aren't decoding first before creating the attribution line.

>The From: header value appears in the attribution the way it appears in
>the message.

I checked my environment variables and the environment in trnrc. I just
don't see what's sending the encoded word on From to be decoded.

When I'm reading, if I press "v" I see all headers and the encoded text.
If I then press "F", quoted text from the precursor article that was
encoded has not been decoded but that doesn't apply to forming the
attribution line.

Re: Getting MIME headers right

<u17fmb$2a9nb$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1278&group=news.software.readers#1278

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Wed, 12 Apr 2023 23:43:07 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u17fmb$2a9nb$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16rfl$emd$1@dont-email.me> <u16uko$2a1b0$1@news.xmission.com> <u17aip$2if3$2@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 23:43:07 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2434795"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 23:43 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u17aip$2if3$2@dont-email.me> thusly:

>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>"Adam H. Kerman" <ahk@chinet.com> spake:
>>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>>"Adam H. Kerman" <ahk@chinet.com> spake:
>>>>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>>>>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:
>
>>>>>How did you get undecoded encoded word on the attribution line?
>
>>>>%[from] in my ATTRIBUTION setting.
>
>>>Right, but you aren't decoding first before creating the attribution line.
>
>>The From: header value appears in the attribution the way it appears in
>>the message.
>
>I checked my environment variables and the environment in trnrc. I just
>don't see what's sending the encoded word on From to be decoded.

No encoding or decoding is happening. It's just literally putting the
value of the header into the body of the article.

>When I'm reading, if I press "v" I see all headers and the encoded text.
>If I then press "F", quoted text from the precursor article that was
>encoded has not been decoded but that doesn't apply to forming the
>attribution line.

I'm running:

Trn version: 4.0-test77 (Sep 1, 2010).

Perhaps you are running the UTF-8 hacked derivative from acli.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<u17ft6$2a9nb$2@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1279&group=news.software.readers#1279

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 12 Apr 2023 23:46:46 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u17ft6$2a9nb$2@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16sn5$o1d$1@dont-email.me> <u16v0a$2a1b0$3@news.xmission.com> <u17a4d$2if3$1@dont-email.me>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 12 Apr 2023 23:46:46 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2434795"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 12 Apr 2023 23:46 UTC

[Please do not mail me a copy of your followup]

"Adam H. Kerman" <ahk@chinet.com> spake the secret code
<u17a4d$2if3$1@dont-email.me> thusly:

>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>I think it's a reasonable request for trn's inews program to be
>>smarter about encodings and analyze the input file and add the
>>necessary headers to decorate the content to the best of it's ability.
>
>I thought we were using inews from INN.

My local ISP is providing inews from INN, but there is an inews
executable built in the trn sources as well. I was proposing making
the one from trn smarter; I don't control INN.

Again, all of this can be intercepted by setting appropriate
environment variables to cause trn to send messages through an
arbitrary processor before handing them off to a program like inews
for injecting into the news feed.

Also, as has been mentioned on this thread, you can configure your
editor to be smarter about non-ASCII content and add the appropriate
headers.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: Getting MIME headers right

<u17j50$3nbr$1@dont-email.me>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1282&group=news.software.readers#1282

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: ahk...@chinet.com (Adam H. Kerman)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Thu, 13 Apr 2023 00:42:08 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 40
Message-ID: <u17j50$3nbr$1@dont-email.me>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16uko$2a1b0$1@news.xmission.com> <u17aip$2if3$2@dont-email.me> <u17fmb$2a9nb$1@news.xmission.com>
Injection-Date: Thu, 13 Apr 2023 00:42:08 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="e209180b4b1fb5c5735b4c45c4131c69";
logging-data="122235"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18SPFL8WP8fMMvoTFYi3stxK63b883YZPo="
Cancel-Lock: sha1:OeeunJHp5rtWp9DbvGMOabGxml4=
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
 by: Adam H. Kerman - Thu, 13 Apr 2023 00:42 UTC

Richard <> wrote:
>[Please do not mail me a copy of your followup]
>
>"Adam H. Kerman" <ahk@chinet.com> spake the secret code
><u17aip$2if3$2@dont-email.me> thusly:
>
>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>"Adam H. Kerman" <ahk@chinet.com> spake:
>>>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>>>"Adam H. Kerman" <ahk@chinet.com> spake:
>>>>>>legalize+jeeves@mail.xmission.com (Richard) wrote:
>>>>>>>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:
>>
>>>>>>How did you get undecoded encoded word on the attribution line?
>>
>>>>>%[from] in my ATTRIBUTION setting.
>>
>>>>Right, but you aren't decoding first before creating the attribution line.
>>
>>>The From: header value appears in the attribution the way it appears in
>>>the message.
>>
>>I checked my environment variables and the environment in trnrc. I just
>>don't see what's sending the encoded word on From to be decoded.
>
>No encoding or decoding is happening. It's just literally putting the
>value of the header into the body of the article.

>>When I'm reading, if I press "v" I see all headers and the encoded text.
>>If I then press "F", quoted text from the precursor article that was
>>encoded has not been decoded but that doesn't apply to forming the
>>attribution line.
>
>I'm running:
>
>Trn version: 4.0-test77 (Sep 1, 2010).
>
>Perhaps you are running the UTF-8 hacked derivative from acli.

No MIME headers in my articles unless I add them.

Re: Getting MIME headers right

<u19iic.m3o.1@ID-201911.user.individual.net>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1283&group=news.software.readers#1283

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!lilly.ping.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: thi...@ddress.is.invalid (Frank Slootweg)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: 13 Apr 2023 16:44:42 GMT
Organization: NOYB
Lines: 37
Message-ID: <u19iic.m3o.1@ID-201911.user.individual.net>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16o20$29spu$2@news.xmission.com> <u16rfl$emd$1@dont-email.me> <u16uko$2a1b0$1@news.xmission.com> <u17aip$2if3$2@dont-email.me>
X-Trace: individual.net SMufS/rFdFvxKJoCtgvg/gXFwnYh7ZHJwhjUPi2wM/EthGHHsb
X-Orig-Path: not-for-mail
Cancel-Lock: sha1:grfA48i4+XOGi6WcGnqK5efoG/Y=
User-Agent: tin/1.6.2-20030910 ("Pabbay") (UNIX) (CYGWIN_NT-10.0-WOW/2.8.0(0.309/5/3) (i686)) Hamster/2.0.2.2
 by: Frank Slootweg - Thu, 13 Apr 2023 16:44 UTC

Adam H. Kerman <ahk@chinet.com> wrote:
> legalize+jeeves@mail.xmission.com (Richard) wrote:
> >"Adam H. Kerman" <ahk@chinet.com> spake:
> >>legalize+jeeves@mail.xmission.com (Richard) wrote:
> >>>"Adam H. Kerman" <ahk@chinet.com> spake:
> >>>>legalize+jeeves@mail.xmission.com (Richard) wrote:
> >>>>>Michael =?ISO-8859-1?Q?B=E4uerle?= <michael.baeuerle@gmx.net> spake:
>
> >>>>How did you get undecoded encoded word on the attribution line?
>
> >>>%[from] in my ATTRIBUTION setting.
>
> >>Right, but you aren't decoding first before creating the attribution line.
>
> >The From: header value appears in the attribution the way it appears in
> >the message.
>
> I checked my environment variables and the environment in trnrc. I just
> don't see what's sending the encoded word on From to be decoded.
>
> When I'm reading, if I press "v" I see all headers and the encoded text.
> If I then press "F", quoted text from the precursor article that was
> encoded has not been decoded but that doesn't apply to forming the
> attribution line.

FWIW, tin works similarly.

I.e. it decodes the encoded From text, both to display the author of
the currently viewed article and to form the attribution line when
composing a followup.

And - as happened here - if some user agent does not decode the
encoded From text and uses that encoded From text in an attribution
line, then that encoded text is left encoded, because it's now part of
the body (not the headers) of the article and hence will not be decoded
unless MIME headers say to do so (but Richard's followup to Michael did
not contain any MIME headers).

Re: Getting MIME headers right

<u19e2r$2b92t$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1284&group=news.software.readers#1284

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: Getting MIME headers right
Date: Thu, 13 Apr 2023 17:27:55 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u19e2r$2b92t$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u16uko$2a1b0$1@news.xmission.com> <u17aip$2if3$2@dont-email.me> <u19iic.m3o.1@ID-201911.user.individual.net>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Thu, 13 Apr 2023 17:27:55 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2466909"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Thu, 13 Apr 2023 17:27 UTC

[Please do not mail me a copy of your followup]

Frank Slootweg <this@ddress.is.invalid> spake the secret code
<u19iic.m3o.1@ID-201911.user.individual.net> thusly:

> And - as happened here - if some user agent does not decode the
>encoded From text and uses that encoded From text in an attribution
>line, then that encoded text is left encoded, because it's now part of
>the body (not the headers) of the article and hence will not be decoded
>unless MIME headers say to do so (but Richard's followup to Michael did
>not contain any MIME headers).

Even if I had added MIME headers, it would still appear encoded in the
body, because in trn %[header] gives the value of the named header field
and doesn't do anything else to the value. The value was QP encoded,
so it showed up in the body as QP encoded.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<eli$2304171842@qaz.wtf>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1286&group=news.software.readers#1286

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.panix5.panix.com!qz!not-for-mail
From: *...@eli.users.panix.com (Eli the Bearded)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Mon, 17 Apr 2023 22:42:16 -0000 (UTC)
Organization: Some absurd concept
Message-ID: <eli$2304171842@qaz.wtf>
References: <u0ffth$1u1oi$1@news.xmission.com> <u127t0$27k3v$1@news.xmission.com> <u129dh$2c6tu$1@dont-email.me> <u141o0$28gtp$1@news.xmission.com>
Injection-Date: Mon, 17 Apr 2023 22:42:16 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="panix5.panix.com:166.84.1.5";
logging-data="27100"; mail-complaints-to="abuse@panix.com"
User-Agent: Vectrex rn 2.1 (beta)
X-Liz: It's actually happened, the entire Internet is a massive game of Redcode
X-Motto: "Erosion of rights never seems to reverse itself." -- kenny@panix
X-US-Congress: Moronic Fucks.
X-Attribution: EtB
XFrom: is a real address
Encrypted: double rot-13
 by: Eli the Bearded - Mon, 17 Apr 2023 22:42 UTC

In news.software.readers, Richard <> wrote:
> "Adam H. Kerman" <ahk@chinet.com> spake the secret code
>> I'm guessing that what's breaking articles is a mix of UTF-8 and
>> 8-bit used by different users, then not declaring which one is in use
>> in followup.

In standard trn version 4, UTF-8 gets hosed because trn tries to strip
control characters, and includes the 32 octets starting at 128 in that.
Disabling the control character squash with -j helps, or you can make a
~ one line patch to the code to not squash highbit "control" characters.

That's a display-for-the-user problem.

Separately, there's a problem in display, if you have a utf-8 terminal
(as is correct) and encounter a properly headered nonASCII, nonUTF8 post
using highbits. Then trn just feeds raw wrong-charset stuff to your
terminal. The acli trn fork fixes that.

Thirdly there's a what-gets-sent issue if you don't post with correct
MIME headers and people read with some tool expecting them. That's the
not-declarining issue.

> Didn't we conclude on this thread that it's up to the user to set the
> headers correctly?

Setting headers is tedious, tedium is best for computers.

> Technically, trn doesn't do the posting; inews does the posting as a
> separate program after having invoked your editor on the post.

Kinda right, but not always. trn can act as inews for you. But it's true
that trn always uses an external program as post editor, and that can
correct headers for the user. Before acli trn existed, I modified my
Pnews to lint check the highbit characters in my posts and warn me of
mismatched headers.

> You could improve this yourself by setting NEWSPOSTER to a different
> program/script that does this detecting and munging of the headers
> before invoking inews.

Depending on your install, NEWSPOSTER may just be looking for the first
Pnews on the PATH. When I was new to shell scripting, I found Pnews and
Rnmail to be excellent examples of how to do powerful stuff in
lowest-common-denominator sh.

Elijah
------
modified his mailx to use Rnmail, too, for header editing

Re: I'm hacking on trn...want to join me?

<u1mfvg$2htb8$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1288&group=news.software.readers#1288

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Tue, 18 Apr 2023 16:20:00 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u1mfvg$2htb8$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <u129dh$2c6tu$1@dont-email.me> <u141o0$28gtp$1@news.xmission.com> <eli$2304171842@qaz.wtf>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Tue, 18 Apr 2023 16:20:00 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2684264"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Tue, 18 Apr 2023 16:20 UTC

[Please do not mail me a copy of your followup]

Eli the Bearded <*@eli.users.panix.com> spake the secret code
<eli$2304171842@qaz.wtf> thusly:

>Thirdly there's a what-gets-sent issue if you don't post with correct
>MIME headers and people read with some tool expecting them. That's the
>not-declarining issue.

I'm open to ideas about how trn can handle this issue better; for
instance, it's certainly possible to scan the edited article before
posting and suggest header additions/changes based on content.

>In news.software.readers, Richard <> wrote:
>> Didn't we conclude on this thread that it's up to the user to set the
>> headers correctly?
>
>Setting headers is tedious, tedium is best for computers.

I agree, I just don't know how the computer is supposed to distinguish
between, say, Big5 encoding or UTF-8. Certainly it can assume UTF-8
and then reject that assumption if the non-ASCII bytes aren't valid
UTF-8 encoded code points.

>> Technically, trn doesn't do the posting; inews does the posting as a
>> separate program after having invoked your editor on the post.
>
>Kinda right, but not always. trn can act as inews for you.

Yeah, there's an inews executable in the trn source tree, but I
suspect that people don't include that in the trn package when they're
packaging things up for linux distros.

BTW, even though Wayne Davison isn't maintaining trn anymore, the trn
related sourceforge mailing lists still work just fine and I've been
posting updates to trn-workers mailing list:
<https://sourceforge.net/p/trn/mailman/>
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Re: I'm hacking on trn...want to join me?

<eli$2304181836@qaz.wtf>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1289&group=news.software.readers#1289

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.panix5.panix.com!qz!not-for-mail
From: *...@eli.users.panix.com (Eli the Bearded)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Tue, 18 Apr 2023 22:37:01 -0000 (UTC)
Organization: Some absurd concept
Message-ID: <eli$2304181836@qaz.wtf>
References: <u0ffth$1u1oi$1@news.xmission.com> <u141o0$28gtp$1@news.xmission.com> <eli$2304171842@qaz.wtf> <u1mfvg$2htb8$1@news.xmission.com>
Injection-Date: Tue, 18 Apr 2023 22:37:01 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="panix5.panix.com:166.84.1.5";
logging-data="14368"; mail-complaints-to="abuse@panix.com"
User-Agent: Vectrex rn 2.1 (beta)
X-Liz: It's actually happened, the entire Internet is a massive game of Redcode
X-Motto: "Erosion of rights never seems to reverse itself." -- kenny@panix
X-US-Congress: Moronic Fucks.
X-Attribution: EtB
XFrom: is a real address
Encrypted: double rot-13
 by: Eli the Bearded - Tue, 18 Apr 2023 22:37 UTC

In news.software.readers, Richard <> wrote:
> Eli the Bearded <*@eli.users.panix.com> spake
>> Thirdly there's a what-gets-sent issue if you don't post with correct
>> MIME headers and people read with some tool expecting them. That's the
>> not-declarining issue.
> I'm open to ideas about how trn can handle this issue better; for
> instance, it's certainly possible to scan the edited article before
> posting and suggest header additions/changes based on content.

I'd say scan for declared charset, see if it fits that, and object if
not or if highbit and no declaration.

For new articles, this can be tricky to be polite to user about it, but
a guess based on LANG or similar environment variable probly gets you in
the neighborhood.

For follow-ups, you can start with the declared charset in the original
post, and maybe provide a configuration for defaults by hierarchy.

> I agree, I just don't know how the computer is supposed to distinguish
> between, say, Big5 encoding or UTF-8. Certainly it can assume UTF-8
> and then reject that assumption if the non-ASCII bytes aren't valid
> UTF-8 encoded code points.

Letter (octet) frequency heuristics, if LANG and hierarchy guided
guesses are wrong.

> Yeah, there's an inews executable in the trn source tree, but I
> suspect that people don't include that in the trn package when they're
> packaging things up for linux distros.

I've been using my own "mini inews" since the 1990s. I hacked it from nn
sources to add some special configuration. The original readme dates to
1989. I don't know that I've _ever_ used a distro package of trn. For a
long time I was using the Panix locally compiled (for netbsd) version.
But now I use my own build on Panix.

> posting updates to trn-workers mailing list:
> <https://sourceforge.net/p/trn/mailman/>

I'm subscribed, but I've been a bit lax about reading it.

Elijah
------
panix.com offers Unix shell accounts with rich program selection

Re: I'm hacking on trn...want to join me?

<u1pm26$2jpkd$1@news.xmission.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1290&group=news.software.readers#1290

 copy link   Newsgroups: news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!xmission!nnrp.xmission!.POSTED.shell.xmission.com!not-for-mail
From: legalize...@mail.xmission.com (Richard)
Newsgroups: news.software.readers
Subject: Re: I'm hacking on trn...want to join me?
Date: Wed, 19 Apr 2023 21:22:14 -0000 (UTC)
Organization: multi-cellular, biological
Sender: legalize+jeeves@mail.xmission.com
Message-ID: <u1pm26$2jpkd$1@news.xmission.com>
References: <u0ffth$1u1oi$1@news.xmission.com> <eli$2304171842@qaz.wtf> <u1mfvg$2htb8$1@news.xmission.com> <eli$2304181836@qaz.wtf>
Reply-To: (Richard) legalize+jeeves@mail.xmission.com
Injection-Date: Wed, 19 Apr 2023 21:22:14 -0000 (UTC)
Injection-Info: news.xmission.com; posting-host="shell.xmission.com:2607:fa18:0:beef::4";
logging-data="2745997"; mail-complaints-to="abuse@xmission.com"
X-Reply-Etiquette: No copy by email, please
Mail-Copies-To: never
X-Newsreader: trn 4.0-test77 (Sep 1, 2010)
Originator: legalize@shell.xmission.com (Richard)
 by: Richard - Wed, 19 Apr 2023 21:22 UTC

[Please do not mail me a copy of your followup]

Eli the Bearded <*@eli.users.panix.com> spake the secret code
<eli$2304181836@qaz.wtf> thusly:

>In news.software.readers, Richard <> wrote:
>> posting updates to trn-workers mailing list:
>> <https://sourceforge.net/p/trn/mailman/>
>
>I'm subscribed, but I've been a bit lax about reading it.

I saw that you'd submitted some PR's to acli's fork which I've merged
into my fork.

You're not missing much on trn-workers, just me posting progress
updates :)
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

Pages:12
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor