On Fri, 3 Nov 2023 at 12:21, AVI GROSS via Python-list
<python-list@python.org> wrote:
> My guess is that a first test of an email address might be to see if a decent module of that kind fills out the object to your satisfaction. You can then perhaps test parts of the object, rather than everything at once, to see if it is obviously invalid. As an example, what does user@alpha.......com with what seems to be lots of meaningless periods, get parsed into?
>

What do you mean by "obviously invalid"? Have you read the RFC?

ChrisA

On 2023-11-03, Chris Angelico <rosuav@gmail.com> wrote:
> On Fri, 3 Nov 2023 at 12:21, AVI GROSS via Python-list
><python-list@python.org> wrote:
>> My guess is that a first test of an email address might be to see if
>> a decent module of that kind fills out the object to your
>> satisfaction. You can then perhaps test parts of the object, rather
>> than everything at once, to see if it is obviously invalid. As an
>> example, what does user@alpha.......com with what seems to be lots of
>> meaningless periods, get parsed into?
>
> What do you mean by "obviously invalid"? Have you read the RFC?

What do you mean by 'What do you mean by "obviously invalid"?'
Have you read the RFC?

On 11/3/2023 6:51 AM, Jon Ribbens via Python-list wrote:
> On 2023-11-03, Chris Angelico <rosuav@gmail.com> wrote:
>> On Fri, 3 Nov 2023 at 12:21, AVI GROSS via Python-list
>> <python-list@python.org> wrote:
>>> My guess is that a first test of an email address might be to see if
>>> a decent module of that kind fills out the object to your
>>> satisfaction. You can then perhaps test parts of the object, rather
>>> than everything at once, to see if it is obviously invalid. As an
>>> example, what does user@alpha.......com with what seems to be lots of
>>> meaningless periods, get parsed into?
>>
>> What do you mean by "obviously invalid"? Have you read the RFC?
>
> What do you mean by 'What do you mean by "obviously invalid"?'
> Have you read the RFC?

About reading the RFC, there's this ... but read the comments too ...

https://haacked.com/archive/2007/08/21/i-knew-how-to-validate-an-email-address-until-i.aspx/

>

> On 11/3/2023 6:51 AM, Jon Ribbens via Python-list wrote:
>

> > On 2023-11-03, Chris Angelico rosuav@gmail.com wrote:
> >

> > > On Fri, 3 Nov 2023 at 12:21, AVI GROSS via Python-list
> > > python-list@python.org wrote:
> > >

> > > > My guess is that a first test of an email address might be to see if
> > > > a decent module of that kind fills out the object to your
> > > > satisfaction. You can then perhaps test parts of the object, rather
> > > > than everything at once, to see if it is obviously invalid. As an
> > > > example, what does user@alpha.......com with what seems to be lots of
> > > > meaningless periods, get parsed into?
> > >

> > > What do you mean by "obviously invalid"? Have you read the RFC?
> >

> > What do you mean by 'What do you mean by "obviously invalid"?'
> > Have you read the RFC?
>

>

> About reading the RFC, there's this ... but read the comments too ...
>

> https://haacked.com/archive/2007/08/21/i-knew-how-to-validate-an-email-address-until-i.aspx/
>

>

Wow. I'm half tempted to make a weird email address to see how many websites get it wrong.

Thank you for the link.

Simon.

On 11/4/23 02:51, Simon Connah via Python-list wrote:
> Wow. I'm half tempted to make a weird email address to see how many websites get it wrong.
>
> Thank you for the link.

Nearly all websites seem to reject simple correct email addresses such
as myemail+sometext@example.domain. I like to use this kind of email
address when I can to help me filter out the inevitable spam that comes
from companies selling off my address even after claiming they won't.

So I suspect that nearly all websites are going to reject other kinds of
weird email addresses you can create that are actually correct.

On 2023-11-04, Michael Torrie via Python-list <python-list@python.org> wrote:
> On 11/4/23 02:51, Simon Connah via Python-list wrote:
>
>> Wow. I'm half tempted to make a weird email address to see how many
>> websites get it wrong.

In my experience, they don't have to be very weird at all.

>> Thank you for the link.
>
> Nearly all websites seem to reject simple correct email addresses
> such as myemail+sometext@example.domain. I like to use this kind of
> email address when I can to help me filter out the inevitable spam
> that comes from companies selling off my address even after claiming
> they won't.

I've always suspected that's intentional. They refuse those sorts of
e-mail addresses because they know that's what they are used for. If
they allowed "plus suffixed" e-mail addresses, then all the crap they
want to send to you would go into /dev/null where it belongs -- and we
can't have that!

> So I suspect that nearly all websites are going to reject other
> kinds of weird email addresses you can create that are actually
> correct.

Definitely. Syntactic e-mail address "validation" is one of the most
useless and widely broken things on the Interwebs. People who do
anything other than require an '@' (and optionally make you enter the
same @-containing string twice) are deluding themselves.

--
Grant

On 2023-11-05 00:39, Grant Edwards via Python-list wrote:
> Definitely. Syntactic e-mail address "validation" is one of the most
> useless and widely broken things on the Interwebs. People who do
> anything other than require an '@' (and optionally make you enter the
> same @-containing string twice) are deluding themselves.

And don't get me started on phone number validation. The most annoying
thing to me, though, is sites that reject names that have an apostrophe
in them. I hate being told that my name, that I have been using for
over seventy years, is invalid.

OK, now that I am started, what else? Oh yah. Look at your credit
card. The number has spaces in it. Why do I have to remove them. If
you don't like them then you are a computer, just remove them.

When do we stop working for computers and have the computers start
working for us?

--
D'Arcy J.M. Cain
Vybe Networks Inc.
http://www.VybeNetworks.com/
IM:darcy@Vex.Net VoIP: sip:darcy@VybeNetworks.com

On 2023-11-05, D'Arcy Cain <darcy@VybeNetworks.com> wrote:
> On 2023-11-05 00:39, Grant Edwards via Python-list wrote:
>> Definitely. Syntactic e-mail address "validation" is one of the most
>> useless and widely broken things on the Interwebs. People who do
>> anything other than require an '@' (and optionally make you enter the
>> same @-containing string twice) are deluding themselves.
>
> And don't get me started on phone number validation. The most annoying
> thing to me, though, is sites that reject names that have an apostrophe
> in them. I hate being told that my name, that I have been using for
> over seventy years, is invalid.

Sometimes I think that these sorts of stupid, wrong, validation are the
fault of idiot managers. When it's apostrophes though I'm suspicious
that it may be idiot programmers who don't know how to prevent SQL
injection attacks without just saying "ban all apostrophes everywhere".
Or perhaps it's idiot "security consultancies" who make it a tick-box
requirement.

> OK, now that I am started, what else? Oh yah. Look at your credit
> card. The number has spaces in it. Why do I have to remove them. If
> you don't like them then you are a computer, just remove them.

Yes, this is also very stupid and annoying. Does nobody who works for
the companies making these sorts of websites ever use their own, or
indeed anyone else's, website?

Another one that's become popular recently is the sort of annoying
website that insists on "email 2FA", i.e. you try to login and then
they send you an email with a 6-digit code in that you have to enter
to authenticate yourself. So you go to your mail client and double-click
on the number to select it, and stupid thing (A) happens: for no sane
reason, the computer selects the digits *and also an invisible space
after them*. Then you copy the digits into the web form, then invisible
space tags along for the ride, and stupid thing (B) happens: the web
server rejects the code because of the trailing space.

Honestly I don't understand why every web application platform doesn't
automatically strip all leading and trailing whitespace on user input
by default. It's surely incredibly rare that it's sensible to preserve
it. (I see Django eventually got around to this in version 1.9.)

Grant (and others),

I am asking about the overall programming process of dealing with email
addresses beyond checking the string for some validity.

You mentioned requiring you type in your email twice as one example. I
generally do a copy/paste to avoid typing or have my browser fill it in.
Rarely the code is set to force me to actually type it in. And I note sites
that force me to do too much typing of any kind or make me jump through
hoops like having to get an email or text with a secondary number to type in
or make me look at pictures and find the right ones and so on, encourage me
to not use them much. There is a price for taking away convenience even if
you see it as some form of security. Yes, there are tradeoffs.

It really may be important to know what you want from your email addresses.
If I sign YOU up for something like the Word of the day in a dozen languages
by supplying your valid email address, then checking if it looks valid is
less useful than sending an email to that address and asking the recipient
to opt-in and verify they legitimately want it. If you want to ensure that
your newsletter is still wanted, you may do something similar every year or
so to everyone, or perhaps just those that have not had activity. If a
mailbox starts rejecting messages, perhaps you send messages to their
secondary contact info or just remove them.

There are many such strategies and some may be way harder to implement than
a simple and perhaps simplistic syntax check.

I do wonder how much it sometimes matters when we see real-world scenarios
where people who died a decade ago remain on voter registration rolls. If my
mailing list has a few hundred bad emails on it, the costs of sending may be
small albeit dealing with rejection messages may clog my logs.

As for fake email addresses, there are many ways to play that game that are
unlikely to be caught. Will they realize there is nobody at
erewhon@gmail.com? If you want to know if someone is going to sell your
hello.there@gmail.com address could you supply hell.other.e@gmail.com and
then monitor mail that you will still receive as it seems google ignores
periods in your email name? And, since others generally see the above as
distinct, you can even use such a method to sign up for something multiple
times.

Complexity leaves room for loopholes.

Still, obviously there are good reasons to do what you can to do some
validation at many points along the way and especially when it may be
critical. Asking someone to type in a new password twice when they cannot
easily see what they are typing, is obviously useful as the consequence of
losing it is high. Are getting the email addresses right as important?

I know my wife registered a fairly common name of the jane.doe@gmail.com
variety that is now useless as it keeps receiving messages someone provided
or typed in wrong that were supposed to go to janedoe@ or doe.jane@ or
janedoe123@ or j.doe@ and so on. These include receipts, subscriptions to
newsletters and much more. Some are inadvertent but the reality is she
stopped using that email as it is now mostly full of SPAM as the others ...

-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On
Behalf Of Grant Edwards via Python-list
Sent: Sunday, November 5, 2023 12:39 AM
To: python-list@python.org
Subject: Re: Checking if email is valid

On 2023-11-04, Michael Torrie via Python-list <python-list@python.org>
wrote:
> On 11/4/23 02:51, Simon Connah via Python-list wrote:
>
>> Wow. I'm half tempted to make a weird email address to see how many
>> websites get it wrong.

In my experience, they don't have to be very weird at all.

>> Thank you for the link.
>
> Nearly all websites seem to reject simple correct email addresses
> such as myemail+sometext@example.domain. I like to use this kind of
> email address when I can to help me filter out the inevitable spam
> that comes from companies selling off my address even after claiming
> they won't.

I've always suspected that's intentional. They refuse those sorts of
e-mail addresses because they know that's what they are used for. If
they allowed "plus suffixed" e-mail addresses, then all the crap they
want to send to you would go into /dev/null where it belongs -- and we
can't have that!

> So I suspect that nearly all websites are going to reject other
> kinds of weird email addresses you can create that are actually
> correct.

Definitely. Syntactic e-mail address "validation" is one of the most
useless and widely broken things on the Interwebs. People who do
anything other than require an '@' (and optionally make you enter the
same @-containing string twice) are deluding themselves.

--
Grant
--
https://mail.python.org/mailman/listinfo/python-list

On 11/5/23 05:32, D'Arcy Cain via Python-list wrote:
> On 2023-11-05 00:39, Grant Edwards via Python-list wrote:
>> Definitely. Syntactic e-mail address "validation" is one of the most
>> useless and widely broken things on the Interwebs.  People who do
>> anything other than require an '@' (and optionally make you enter the
>> same @-containing string twice) are deluding themselves.
>
> And don't get me started on phone number validation.  The most annoying
> thing to me, though, is sites that reject names that have an apostrophe
> in them.  I hate being told that my name, that I have been using for
> over seventy years, is invalid.
>
> OK, now that I am started, what else?  Oh yah.  Look at your credit
> card.  The number has spaces in it.  Why do I have to remove them.  If
> you don't like them then you are a computer, just remove them.
>
> When do we stop working for computers and have the computers start
> working for us?
>
If this is being voted on, pretend you are in Georgia, vote often and
early. Best question of the century.

Cheers, Gene Heskett.
--
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
- Louis D. Brandeis

On 2023-11-05, D'Arcy Cain via Python-list <python-list@python.org> wrote:
> On 2023-11-05 00:39, Grant Edwards via Python-list wrote:
>> Definitely. Syntactic e-mail address "validation" is one of the most
>> useless and widely broken things on the Interwebs. People who do
>> anything other than require an '@' (and optionally make you enter the
>> same @-containing string twice) are deluding themselves.
>
> And don't get me started on phone number validation.

I can see how the truley dim-witted might forget that other countries
have phone numbers with differing lengths and formatting/punctuation,
but there are tons of sites where it takes multiple tries when
entering even a bog-standard USA 10-0digit phone nubmer because they
are completely flummuxed by an area code in parens or hyphens in the
usual places (or lack of hyhpens in the usual places). This stuff
isn't that hard, people...

> The most annoying thing to me, though, is sites that reject names
> that have an apostrophe in them. I hate being told that my name,
> that I have been using for over seventy years, is invalid.
>
> OK, now that I am started, what else? Oh yah. Look at your credit
> card. The number has spaces in it. Why do I have to remove them. If
> you don't like them then you are a computer, just remove them.

Indeed. There is a tiny but brightly burning kernel of hate in my
heart for web sites (and their developers) that refuse to accept
credit card numbers entered with spaces _as_they_are_shown_on_the_card_!

I've concluded that using PHP causes debilitating and irreversible
brain damage.

> When do we stop working for computers and have the computers start
> working for us?

--
Grant

On 2023-11-05, Grant Edwards <grant.b.edwards@gmail.com> wrote:
> On 2023-11-05, D'Arcy Cain via Python-list <python-list@python.org> wrote:
>> On 2023-11-05 00:39, Grant Edwards via Python-list wrote:
>>> Definitely. Syntactic e-mail address "validation" is one of the most
>>> useless and widely broken things on the Interwebs. People who do
>>> anything other than require an '@' (and optionally make you enter the
>>> same @-containing string twice) are deluding themselves.
>>
>> And don't get me started on phone number validation.
>
> I can see how the truley dim-witted might forget that other countries
> have phone numbers with differing lengths and formatting/punctuation,
> but there are tons of sites where it takes multiple tries when
> entering even a bog-standard USA 10-0digit phone nubmer because they
> are completely flummuxed by an area code in parens or hyphens in the
> usual places (or lack of hyhpens in the usual places). This stuff
> isn't that hard, people...

Indeed - you just do "pip install phonenumbers" :-)

On 11/5/23 10:34, Grant Edwards via Python-list wrote:

> Indeed. There is a tiny but brightly burning kernel of hate in my
> heart for web sites (and their developers) that refuse to accept
> credit card numbers entered with spaces _as_they_are_shown_on_the_card_!
>
> I've concluded that using PHP causes debilitating and irreversible
> brain damage.

I think it's the attitude that speed of deployment is more important
than any other factor, rather than just PHP :-) Plus a bunch of that
stuff is also coded in the front end (aka Javascript).

Phone numbers.
Credit card numbers.
Names (in my case - my wife has a hypenated surname which is almost as
deadly as non-alpha characters in a name which was already mentioned in
this diverging thread)

and addresses. living rurally we have two addresses: a post office
rural route box for USPS and a "street address" for anyone else. The
former looks like "{locationID} Box {number}". The single word "Box"
often triggers "we don't deliver to P.O. Boxes" - it's not a PO Box, and
it's the only address USPS will deliver to, so get over yourself. Or
triggers fraud detection alerts, because "billing address" != "shipping
address".

it's astonishing how bad so many websites are at what should be a
fundamental function: taking in user-supplied data in order to do
something valuable with it.

On 2023-11-05 06:48, Jon Ribbens via Python-list wrote:
> Sometimes I think that these sorts of stupid, wrong, validation are the
> fault of idiot managers. When it's apostrophes though I'm suspicious
> that it may be idiot programmers who don't know how to prevent SQL
> injection attacks without just saying "ban all apostrophes everywhere".
> Or perhaps it's idiot "security consultancies" who make it a tick-box
> requirement.

https://xkcd.com/327/

>> OK, now that I am started, what else? Oh yah. Look at your credit
>> card. The number has spaces in it. Why do I have to remove them. If
>> you don't like them then you are a computer, just remove them.
>
> Yes, this is also very stupid and annoying. Does nobody who works for
> the companies making these sorts of websites ever use their own, or
> indeed anyone else's, website?

Gotta wonder for sure. It could also be the case of programmers
depending on user input but the users insist on living with the bugs
and/or working around them. We made crash reporting dead simple to
report on and still users didn't bother. We would get the traceback and
have to guess what the user was doing.

> Honestly I don't understand why every web application platform doesn't
> automatically strip all leading and trailing whitespace on user input
> by default. It's surely incredibly rare that it's sensible to preserve
> it. (I see Django eventually got around to this in version 1.9.)

Yes, I have done that forever. Never had a complaint about it dropping
characters.

--
D'Arcy J.M. Cain
Vybe Networks Inc.
http://www.VybeNetworks.com/
IM:darcy@Vex.Net VoIP: sip:darcy@VybeNetworks.com

On Sun, 5 Nov 2023 19:22:49 -0600, D'Arcy Cain wrote:

> Gotta wonder for sure. It could also be the case of programmers
> depending on user input but the users insist on living with the bugs
> and/or working around them. We made crash reporting dead simple to
> report on and still users didn't bother. We would get the traceback and
> have to guess what the user was doing.

We've found even if you directly ask the user often the answer is 'I
dunno' or some mythology they have constructed to explain the problem. We
had one site that reported if they hit the Enter key hard the query would
work. It was a rather simple bug where the query would be randomly sent to
the wrong interface.

There is quite a bit of literature in psychology about intermittent
reinforcement and the behavioral strategies that are developed. Works for
rats, works for humans.

> I can see how the truley dim-witted might forget that other countries
> have phone numbers with differing lengths and formatting/punctuation,
> but there are tons of sites where it takes multiple tries when
> entering even a bog-standard USA 10-0digit phone nubmer because they
> are completely flummuxed by an area code in parens or hyphens in the
> usual places (or lack of hyhpens in the usual places). This stuff
> isn't that hard, people...

The thing I truly hate is when you have two telephone number fields. One for landline and one for mobile. I mean who in hell has a landline these days? And not accepting your mobile number in the landline number field is just when I give up. Or having a landline only field that does not accept mobile phones.

Simon.

On 2023-11-06, D'Arcy Cain <darcy@VybeNetworks.com> wrote:
> On 2023-11-05 06:48, Jon Ribbens via Python-list wrote:
>> Sometimes I think that these sorts of stupid, wrong, validation are the
>> fault of idiot managers. When it's apostrophes though I'm suspicious
>> that it may be idiot programmers who don't know how to prevent SQL
>> injection attacks without just saying "ban all apostrophes everywhere".
>> Or perhaps it's idiot "security consultancies" who make it a tick-box
>> requirement.
>
> https://xkcd.com/327/

Indeed. My point is that the correct way to solve this problem is not
to declare vast swathes of valid inputs verboten, but to *not execute
user input as code*. Controversial, I know.

>>> OK, now that I am started, what else? Oh yah. Look at your credit
>>> card. The number has spaces in it. Why do I have to remove them. If
>>> you don't like them then you are a computer, just remove them.
>>
>> Yes, this is also very stupid and annoying. Does nobody who works for
>> the companies making these sorts of websites ever use their own, or
>> indeed anyone else's, website?
>
> Gotta wonder for sure. It could also be the case of programmers
> depending on user input but the users insist on living with the bugs
> and/or working around them. We made crash reporting dead simple to
> report on and still users didn't bother. We would get the traceback and
> have to guess what the user was doing.

That was another thing that I used to find ridiculous, but seems to have
improved somewhat in recent years - website error pages that said "please
contact us to let us know about this error". I'm sorry, what? You want
me to contact you to tell you about what your own website is doing? How
does that make any sense? Websites should be self-reporting problems.

(Not least because, as you say, people are absolutely terrible at
reporting problems, with almost all bug reports reading effectively as
"I was doing something that I'm not going to tell you and I as expecting
something to happen which I'm not going to tell you, but instead
something else happened, which I'm also not going to tell you".)

On 11/6/23 01:57, Simon Connah via Python-list wrote:

> The thing I truly hate is when you have two telephone number fields. One for landline and one for mobile. I mean who in hell has a landline these days?

People who live in places with spotty, or no, mobile coverage. We do exist.

On 2023-11-06, Mats Wichmann <mats@wichmann.us> wrote:
> On 11/6/23 01:57, Simon Connah via Python-list wrote:
>> The thing I truly hate is when you have two telephone number fields.
>> One for landline and one for mobile. I mean who in hell has a
>> landline these days?
>
> People who live in places with spotty, or no, mobile coverage. We do
> exist.

Catering for people in minority situations is, of course, important.

Catering for people in the majority situation is probably important too.

On 2023-11-06 08:57, Simon Connah via Python-list wrote:
>
>> I can see how the truley dim-witted might forget that other countries
>> have phone numbers with differing lengths and formatting/punctuation,
>> but there are tons of sites where it takes multiple tries when
>> entering even a bog-standard USA 10-0digit phone nubmer because they
>> are completely flummuxed by an area code in parens or hyphens in the
>> usual places (or lack of hyhpens in the usual places). This stuff
>> isn't that hard, people...
>
> The thing I truly hate is when you have two telephone number fields. One for landline and one for mobile. I mean who in hell has a landline these days? And not accepting your mobile number in the landline number field is just when I give up. Or having a landline only field that does not accept mobile phones.
>
I have a landline. It's also how I access the internet.

On Tue, 7 Nov 2023 at 02:05, Jon Ribbens via Python-list
<python-list@python.org> wrote:
> That was another thing that I used to find ridiculous, but seems to have
> improved somewhat in recent years - website error pages that said "please
> contact us to let us know about this error". I'm sorry, what? You want
> me to contact you to tell you about what your own website is doing? How
> does that make any sense? Websites should be self-reporting problems.

Actually, I think those serve a very important purpose. The reports
are almost certainly being discarded unread; the value of such a
reporting system is to give the user the sensation that they've "done
something" to "help". It makes some people feel better about running
into a bug.

But you're right, they serve little purpose in solving web site problems.

ChrisA

On 11/6/23 08:23, Jon Ribbens via Python-list wrote:
> On 2023-11-06, Mats Wichmann <mats@wichmann.us> wrote:
>> On 11/6/23 01:57, Simon Connah via Python-list wrote:
>>> The thing I truly hate is when you have two telephone number fields.
>>> One for landline and one for mobile. I mean who in hell has a
>>> landline these days?
>>
>> People who live in places with spotty, or no, mobile coverage. We do
>> exist.
>
> Catering for people in minority situations is, of course, important.
>
> Catering for people in the majority situation is probably important too.

A good experience would do both, in a comfortable way for either.

Continuing with the example, if you have a single phone number field, or
let a mobile number be entered in a field marked for landline, you will
probably assume you can text to that number. I see this all the time on
signups that are attempting to provide some sort of 2FA - I enter the
landline number and the website tries to text a verification code to it
(rather have an authenticator app for 2FA anyway, but that's a different
argument)

Suggests maybe labeling should be something like:

* Number you want to be called on
* Number for texted security messages, if different

Never seen that, though :-)

On Tue, 7 Nov 2023 at 07:10, Mats Wichmann via Python-list
<python-list@python.org> wrote:
> Suggests maybe labeling should be something like:
>
> * Number you want to be called on
> * Number for texted security messages, if different
>
> Never seen that, though :-)
>

My responses would be:

* Don't.
* That's what cryptographic keys are for.

:)

ChrisA

On 7/11/23 7:45 am, Mats Wichmann wrote:
> Continuing with the example, if you have a single phone number field, or
> let a mobile number be entered in a field marked for landline, you will
> probably assume you can text to that number.

But if the site can detect that you've entered a mobile number into
the landline field or vice versa and reject it, then it can figure out
whether it can text to a given numner or not without you having
to tell it!

--
Greg

On 6/11/23 6:34 pm, rbowman wrote:
> We've found even if you directly ask the user often the answer is 'I
> dunno' or some mythology they have constructed to explain the problem.

This seems to apply to hardware issues as well. Louis Rossmann has
a philosophy of "Never believe what the customer tells you."

--
Greg