I'm doing a bunch of EAI readiness tests for the Universal Acceptance
Steering Group. The tests are described here:

https://uasg.tech/wp-content/uploads/documents/UASG021B-en-digital.pdf

I ran all of the MSA and MTA tests on the 17.0 beta, and they basically all passed,
once Claus sent me a patch for a bug I ran across. The only ones that didn't pass
are what I would call quality of implementation issues.

The EAI RFCs say that domains in trace headers such as Received and Return-Path should
be represented as UTF-8 U-labels, rather than ASCII A-labels. The domain names in
the Received headers are A-labels. Postfix, Exim, Courier, and many other mail
systems do the same thing so I can't get too upset.

One of the tests checks whether ASCII mail is sent with an unneeded
SMTPUTF8 flag to a recipient mail server that supports EAI. It is, but
again, I can't get too upset about it. It does correctly check that it
doesn't send EAI mail to a non-EAI server.

One thing I would change is to get rid of the -U flag to sendmail. In
every other system I've tested, there is no flag at submission time.
Instead it looks at the submitted message to see whether it has UTF-8
in the envelope addresses or (in most systems) in the message headers.
This is also why the needless SMTPUTF8 flag above doesn't matter; most
systems look at the message rather than the flag to decide whether it
needs EAI handling.

In my tests I worked around it with a one line shim that always adds
-U to the sendmail command and my guess is that everyone will do that.
I have found that a lot of user mail software, such as the hoary Unix
Mail command, handles EAI mail pretty well because it passes through
eight bit strings without interpreting them. They have the call to
/usr/sbin/sendmail hardcoded with no way to tell it to add new flags.

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

Very interesting report. Thank you for sharing John.

On 5/27/21 8:29 PM, John Levine wrote:
> In my tests I worked around it with a one line shim that always adds
> -U to the sendmail command and my guess is that everyone will do that.
> I have found that a lot of user mail software, such as the hoary Unix
> Mail command, handles EAI mail pretty well because it passes through
> eight bit strings without interpreting them. They have the call to
> /usr/sbin/sendmail hardcoded with no way to tell it to add new flags.

It sounds to me like this might be a compile time option. Default
without -U enabled, thus require the -U option. Or, default with -U
enabled, thus not needing the -U option.

--
Grant. . . .
unix || die

John Levine wrote:

> The EAI RFCs say that domains in trace headers such as Received and Return-Path should
> be represented as UTF-8 U-labels, rather than ASCII A-labels. The domain names in
> the Received headers are A-labels. [[...]]

Can you please post examples of the problem?
Do you mean the "for <user@domain>" part?
If that is an U-label then it is shown as such in my tests.

--
Note: please read the netiquette before posting. I will almost never
reply to top-postings which include a full copy of the previous
article(s) at the end because it's annoying, shows that the poster
is too lazy to trim his article, and it's wasting the time of all readers.

It appears that Claus Aßmann <ml+sendmail(-no-copies-please)@esmtp.org> said:
>John Levine wrote:
>
>> The EAI RFCs say that domains in trace headers such as Received and Return-Path should
>> be represented as UTF-8 U-labels, rather than ASCII A-labels. The domain names in
>> the Received headers are A-labels. [[...]]
>
>Can you please post examples of the problem?

According to RFC all of those xn--blah A-labels are supposed to be U-labels. In this
case it's the same host, but I could construct a relay example where it's not.

Again, it's not a big deal, and Postfix and Exim do the same thing you do.

Received: from xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s (localhost [127.0.0.1])
by xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s (8.17.0.0/8.16.1) with UTF8SMTPS id 14PKr72o080569
(version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO)
for <bøb@家.电子邮件测试.中国>; Tue, 25 May 2021 20:53:08 GMT
(envelope-from 用户1@发送邮件.电子邮件测试.中国)
Received: (from eaitest@localhost)
by xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s (8.17.0.0/8.16.1/Submit) id 14PKr7Mh080568
for bøb@家.电子邮件测试.中国; Tue, 25 May 2021 20:53:07 GMT
(envelope-from 用户1@发送邮件.电子邮件测试.中国)

What's the setup and what are the commands to reproduce this?
Just curious - maybe I can add something to my tests.

--
Note: please read the netiquette before posting. I will almost never
reply to top-postings which include a full copy of the previous
article(s) at the end because it's annoying, shows that the poster
is too lazy to trim his article, and it's wasting the time of all readers.

According to Claus Aßmann <INVALID_NO_CC_REMOVE_IF_YOU_DO_NOT_POST_ml+sendmail(-no-copies-please)@esmtp.org>:
>What's the setup and what are the commands to reproduce this?
>Just curious - maybe I can add something to my tests.

The scripts are here although I haven't uploaded the latest tweaks.

https://github.com/jrlevine/eaitesttools

The host where sendmail is running is called xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s and smeai.services.net,
with the IDN name being the hostname. My script uses ssh to log in and send the message with sendmail -U to an external
EAI address where I pick up the message and look at the Received header that sendmail added.

Again, a lot of other MTAs also put A-labels in the FROM and BY parts
of the Received header so it's not a big deal to change.

Here's the whole thing as received. My MTA adds EAI-From and EAI-Rcpt in the SMTP session to help figure
out what's going on. My MTA is a heavily patched version of qmail. For EAI support, I change all of the
envelope domains to A-labels on the way in. On the way out I change the envelopes back to U-labels if there
are UTF-8 local parts in the envelope or UTF-8 in the message headers. That limits the number of places I
have to deal with U-lablels, and means I only have to handle one version of each address for local routing.

Return-Path: <用户1@发送邮件.电子邮件测试.中国>
Delivered-To: johnl-eaihome-bøb@xn--fct.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s
Received: (qmail 75693 invoked from network); 26 May 2021 02:44:22 -0000
Authentication-Results: iecc.com; spf=none spf.mailfrom=用户1@发送邮件.电子邮件测试.中国 spf.helo=xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s smtp.remote-ip="2606:4300:0:4::1003"
EAI-From: 用户1@发送邮件.电子邮件测试.中国
EAI-Rcpt: bøb@家.电子邮件测试.中国
Received: from xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s ([IPV6:2606:4300:0:4::1003])
by mail1.iecc.com ([IPV6:2001:470:1f07:1126:33:5370:616d:6d31])
with UTF8SMTPS via TCP6 (port 18822/25) id 677070195
tls TLS1_3_ECDHE_RSA_AES_256_GCM_AEAD; 26 May 2021 02:44:22 -0000
Received: from xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s (localhost [127.0.0.1])
by xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s (8.17.0.0/8.16.1) with UTF8SMTPS id 14Q2iLR7082122
(version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO)
for <bøb@家.电子邮件测试.中国>; Wed, 26 May 2021 02:44:21 GMT
(envelope-from 用户1@发送邮件.电子邮件测试.中国)
Received: (from eaitest@localhost)
by xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s (8.17.0.0/8.16.1/Submit) id 14Q2iLY5082121
for bøb@家.电子邮件测试.中国; Wed, 26 May 2021 02:44:21 GMT
(envelope-from 用户1@发送邮件.电子邮件测试.中国)
Message-Id: <202105260244.14Q2iLY5082121@xn--5nq21jyu9d1ta.xn--5nqx41au4nqohsp3axcg.xn--fiqs8s>
X-Authentication-Warning: localhost.my.domain: eaitest set sender to 用户1@发送邮件.电子邮件测试.中国 using -f
From: Test Sender <用户1@发送邮件.电子邮件测试.中国>
To: Test Recipient <bøb@家.电子邮件测试.中国>
Date: Wed, 26 May 2021 02:44:20 +0000
Subject: EAI simple test message (测试讯息) for header checks c2d58dca71070cbe0b58

This test message header addresses and header generation
--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly