I have sendmail running successfully on a Debian 10 derivative, 8.15.2,
and have installed sendmail to my Debian 12, 8.17.1, virtual server. I
copied the configuration files from my Debian 10 server to the Debian 12
server. I can send mail to the server using a Thunderbird client or
telnet from anywhere. What I can't do is connect to the submision port
and relay mail. The error I'm getting from Thunderbird is: "Sending of
the message failed.
An error occurred while sending mail. The mail server responded:
Authentication required. Please verify that your email address is
correct in your account settings and try again."

I can send mail to the Debian 12 server from anywhere. I can log into
the Debian 12 server and using telnet on port 25 send mail to my Debian
10 server elsewhere. That's probably because the access database says
localhost can relay.

I have an authentication issue somewhere. The certificates are self
signed as they are on the Debian 10 server. I have sasl2-bin installed
and running. Thunderbird is configured to use STARTTLS.

Any ideas greatly appreciated.

If I try to connect to the Debian 12 server with telnet from home I get
this:

knute@knute-XPS-8700:~$ telnet domain.com 25
Trying 176.126.240.209...
Connected to domain.com.
Escape character is '^]'.
ehlo knute
554 domain.com ESMTP not accepting messages
250-domain.com Hello [216.24.32.226], pleased to meet you
250 ENHANCEDSTATUSCODES

I get the same on port 587. If I use HELO instead of EHLO it looks like
it is going to go but when I enter the mail from: I get 550 5.0.0
Command rejected.

knute@knute-XPS-8700:~$ telnet domain.com 25
Trying 176.126.240.209...
Connected to domain.com.
Escape character is '^]'.
helo knute
554 domain.com ESMTP not accepting messages
250 domain.com Hello [216.24.32.226], pleased to meet you
mail from: knute@otherdomain.com
550 5.0.0 Command rejected

So the result is if the connection to the Debian 12 server is local on
port 25 or is an inbound connection it works. If the connection is to
relay mail it doesn't work.

My sendmail.mc file:

divert(-1)dnl

#-----------------------------------------------------------------------------
divert(0)dnl
# # Copyright (c) 1998-2005 Richard Nelson. All Rights Reserved.
# # This file is used to configure Sendmail for use with Debian systems.
# define(`_USE_ETC_MAIL_')dnl
include(`/usr/share/sendmail/cf/m4/cf.m4')dnl
VERSIONID(`$Id: sendmail.mc, v 8.17.1.9-2 2023-01-11 23:26:28 cowboy Exp $')
OSTYPE(`debian')dnl
DOMAIN(`debian-mta')dnl
dnl # Items controlled by /etc/mail/sendmail.conf - DO NOT TOUCH HERE
undefine(`confHOST_STATUS_DIRECTORY')dnl #DAEMON_HOSTSTATS=
dnl # Items controlled by /etc/mail/sendmail.conf - DO NOT TOUCH HERE
dnl #
dnl # General defines
define(`confLOG_LEVEL',25)dnl
dnl #
FEATURE(`no_default_msa')dnl
DAEMON_OPTIONS(`Family=inet, Name=MTA-v4, Port=smtp, M=A')dnl
DAEMON_OPTIONS(`Family=inet, Name=MSP-v4, Port=submission, M=Ea')dnl
dnl #
define(`confPRIVACY_FLAGS',dnl
`needmailhelo,needexpnhelo,needvrfyhelo,restrictqrun,restrictexpand,nobodyreturn,authwarnings')dnl
dnl #
dnl # Define connection throttling and window length
define(`confCONNECTION_RATE_THROTTLE', `15')dnl
define(`confCONNECTION_RATE_WINDOW_SIZE',`10m')dnl
dnl #
dnl # Features
dnl #
dnl # use /etc/mail/local-host-names
FEATURE(`use_cw_file')dnl
dnl #
dnl # The access db is the basis for most of sendmail's checking
FEATURE(`access_db', , `skip')dnl
dnl #
dnl # The greet_pause feature stops some automail bots - but check the
dnl # provided access db for details on excluding localhosts...
FEATURE(`greet_pause', `1000')dnl 1 seconds
dnl #
dnl # Delay_checks allows sender<->recipient checking
FEATURE(`delay_checks', `friend', `n')dnl
dnl #
dnl # If we get too many bad recipients, slow things down...
define(`confBAD_RCPT_THROTTLE',`3')dnl
dnl #
dnl # Stop connections that overflow our concurrent and time connection
rates
FEATURE(`conncontrol', `nodelay', `terminate')dnl
FEATURE(`ratecontrol', `nodelay', `terminate')dnl
dnl #
dnl # I added this
TRUST_AUTH_MECH(`PLAIN')dnl
define(`confAUTH_MECHANISM',`PLAIN')dnl
define(`confAUTH_OPTIONS',`A p y')dnl
include(`/etc/mail/sasl/sasl.m4')dnl
include(`/etc/mail/tls/starttls.m4')dnl
dnl #
include(`/etc/mail/m4/dialup.m4')dnl
include(`/etc/mail/m4/provider.m4')dnl
dnl #
dnl # Default Mailer setup
MAILER_DEFINITIONS
MAILER(`local')dnl
MAILER(`smtp')dnl

My /etc/mail/access file:

Connect:127 RELAY
GreetPause:127 0
ClientRate:127 0
ClientConn:127 0
Connect:IPv6:::1 RELAY
GreetPause:IPv6:::1 0
ClientRate:IPv6:::1 0
ClientConn:IPv6:::1 0
# Defaults
GreetPause: 5000
ClientRate: 10
ClientConn: 10
# Spam:postmaster@ FRIEND
Spam:abuse@ FRIEND
Spam:spam@ FRIEND
# # Blacklisted users
# Connect:169.254 REJECT
Connect:192.0.2 REJECT
Connect:224 REJECT
Connect:255 REJECT

My submit.mc file:

divert(-1)dnl
#-----------------------------------------------------------------------------
divert(0)dnl
# define(`_USE_ETC_MAIL_')dnl
include(`/usr/share/sendmail/cf/m4/cf.m4')dnl
VERSIONID(`$Id: submit.mc, v 8.17.1.9-2 2023-01-11 23:26:28 cowboy Exp $')
OSTYPE(`debian')dnl
DOMAIN(`debian-msp')dnl
dnl #---------------------------------------------------------------------
FEATURE(`msp', `[127.0.0.1]', `25')dnl
dnl
include(`/etc/mail/sasl/sasl.m4')dnl
include(`/etc/mail/tls/starttls.m4')dnl

--

Knute Johnson