In python 2 one can do:-

for msg in maildir:
print msg # or whatever you want to do with the message

However in python 3 this produces "TypeError: string argument
expected, got 'bytes'".

How should one iterate over a maildir in python3?

(I've been here before but this problem is subtly different from the
one I had before and I can't find the answer)

--
Chris Green
·

On 25/06/21 7:06 am, Chris Green wrote:
> In python 2 one can do:-
>
> for msg in maildir:
> print msg # or whatever you want to do with the message
>
>
> However in python 3 this produces "TypeError: string argument
> expected, got 'bytes'".
>
> How should one iterate over a maildir in python3?

You're already iterating over it just fine. Your problem is
actually how to *print* a mail message.

The answer to this will depend on what your purpose is. If
you just want a rough-and-ready idea of what the message
contains, you could do this:

print(repr(msg))

However, that won't work very well if the message doesn't
consist of mostly text in an ASCII-compatible encoding.

If you want something better, Python comes with some standard
library code for dealing with mail messages. Check out the
'email' module.

--
Greg

Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
> On 25/06/21 7:06 am, Chris Green wrote:
> > In python 2 one can do:-
> >
> > for msg in maildir:
> > print msg # or whatever you want to do with the message
> >
> >
> > However in python 3 this produces "TypeError: string argument
> > expected, got 'bytes'".
> >
> > How should one iterate over a maildir in python3?
>
> You're already iterating over it just fine. Your problem is
> actually how to *print* a mail message.
>
> The answer to this will depend on what your purpose is. If
> you just want a rough-and-ready idea of what the message
> contains, you could do this:
>
> print(repr(msg))
>
> However, that won't work very well if the message doesn't
> consist of mostly text in an ASCII-compatible encoding.
>
> If you want something better, Python comes with some standard
> library code for dealing with mail messages. Check out the
> 'email' module.
>
The error comes from the line "for msg in maildir:", not the print.

Here's the full program where I'm encountering the error (yes, I
should have posted this first time around) :-

#!/usr/bin/python3

import mailbox
import sys
import email

# open the existing maildir and the target mbox file
maildir = mailbox.Maildir(sys.argv [-2], email.message_from_file)
mbox = mailbox.mbox(sys.argv[-1])

# lock the mbox
mbox.lock()

# iterate over messages in the maildir and add to the mbox
for msg in maildir:
mbox.add(msg)

# close and unlock
mbox.close()
maildir.close()

(... and it's not my coding style, just copied)

Here's the error trace:-

chris$ ./md.py houseSitting fred
Traceback (most recent call last):
File "/home/chris/tmp/./md.py", line 18, in <module>
for msg in maildir:
File "/usr/lib/python3.9/mailbox.py", line 110, in itervalues
value = self[key]
File "/usr/lib/python3.9/mailbox.py", line 77, in __getitem__
return self._factory(file)
File "/usr/lib/python3.9/email/__init__.py", line 54, in message_from_file
return Parser(*args, **kws).parse(fp)
File "/usr/lib/python3.9/email/parser.py", line 56, in parse
feedparser.feed(data)
File "/usr/lib/python3.9/email/feedparser.py", line 175, in feed
self._input.push(data)
File "/usr/lib/python3.9/email/feedparser.py", line 103, in push
self._partial.write(data)
TypeError: string argument expected, got 'bytes'
chris$

Line 18 is the "for msg in maildir:".

Presumably the program worked as posted under python 2, all I have
done is to change the shebang line to use python 3 (I no longer have
python 2 on my system, otherwise I'd have used it as this is only a
quick and dirty conversion script to be used once).

--
Chris Green
·

On Fri, 25 Jun 2021 09:19:49 +0100, Chris Green wrote:

> TypeError: string argument expected, got 'bytes'

couple things comes to mind:

1. find py2 as archive, put it somewhere and run it from that

2. convert the bytes to str (find and replace)

3. run the py2>py3 conversion script

--
Gilmeh

Gilmeh Serda <gilmeh.serdah@nothing.here.invalid> wrote:
> On Fri, 25 Jun 2021 09:19:49 +0100, Chris Green wrote:
>
> > TypeError: string argument expected, got 'bytes'
>
> couple things comes to mind:
>
> 1. find py2 as archive, put it somewhere and run it from that
>
Hmm! :-)

> 2. convert the bytes to str (find and replace)
>
How? It's failing at "for msg in maildir:", you can't change it to
"for str(msg) in maildir" (well you can but it gives a syntax error).

> 3. run the py2>py3 conversion script
>
That does nothing, says "No files need to be modified"

--
Chris Green
·

On Sat, Jun 26, 2021 at 12:28 AM Chris Green <cl@isbd.net> wrote:
>
> Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
> > On 25/06/21 7:06 am, Chris Green wrote:
> > > In python 2 one can do:-
> > >
> > > for msg in maildir:
> > > print msg # or whatever you want to do with the message
> > >
> > >
> > > However in python 3 this produces "TypeError: string argument
> > > expected, got 'bytes'".
> > >
> > > How should one iterate over a maildir in python3?
> >
> > You're already iterating over it just fine. Your problem is
> > actually how to *print* a mail message.
> >
> > The answer to this will depend on what your purpose is. If
> > you just want a rough-and-ready idea of what the message
> > contains, you could do this:
> >
> > print(repr(msg))
> >
> > However, that won't work very well if the message doesn't
> > consist of mostly text in an ASCII-compatible encoding.
> >
> > If you want something better, Python comes with some standard
> > library code for dealing with mail messages. Check out the
> > 'email' module.
> >
> The error comes from the line "for msg in maildir:", not the print.
>
> Here's the full program where I'm encountering the error (yes, I
> should have posted this first time around) :-
>
> #!/usr/bin/python3
>
> import mailbox
> import sys
> import email
>
>
> # open the existing maildir and the target mbox file
> maildir = mailbox.Maildir(sys.argv [-2], email.message_from_file)
> mbox = mailbox.mbox(sys.argv[-1])
>
> # iterate over messages in the maildir and add to the mbox
> for msg in maildir:
> mbox.add(msg)
>

Maildir says that the factory has to take a binary file object. It
looks like email.message_from_file is expecting a text file object,
but there's a very similar function message_from_binary_file that
might be what you want.

Haven't tested it, but worth a try.

ChrisA

Chris Angelico <rosuav@gmail.com> wrote:
> On Sat, Jun 26, 2021 at 12:28 AM Chris Green <cl@isbd.net> wrote:
> >
> > Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
> > > On 25/06/21 7:06 am, Chris Green wrote:
> > > > In python 2 one can do:-
> > > >
> > > > for msg in maildir:
> > > > print msg # or whatever you want to do with the message
> > > >
> > > >
> > > > However in python 3 this produces "TypeError: string argument
> > > > expected, got 'bytes'".
> > > >
> > > > How should one iterate over a maildir in python3?
> > >
> > > You're already iterating over it just fine. Your problem is
> > > actually how to *print* a mail message.
> > >
> > > The answer to this will depend on what your purpose is. If
> > > you just want a rough-and-ready idea of what the message
> > > contains, you could do this:
> > >
> > > print(repr(msg))
> > >
> > > However, that won't work very well if the message doesn't
> > > consist of mostly text in an ASCII-compatible encoding.
> > >
> > > If you want something better, Python comes with some standard
> > > library code for dealing with mail messages. Check out the
> > > 'email' module.
> > >
> > The error comes from the line "for msg in maildir:", not the print.
> >
> > Here's the full program where I'm encountering the error (yes, I
> > should have posted this first time around) :-
> >
> > #!/usr/bin/python3
> >
> > import mailbox
> > import sys
> > import email
> >
> >
> > # open the existing maildir and the target mbox file
> > maildir = mailbox.Maildir(sys.argv [-2], email.message_from_file)
> > mbox = mailbox.mbox(sys.argv[-1])
> >
> > # iterate over messages in the maildir and add to the mbox
> > for msg in maildir:
> > mbox.add(msg)
> >
>
> Maildir says that the factory has to take a binary file object. It
> looks like email.message_from_file is expecting a text file object,
> but there's a very similar function message_from_binary_file that
> might be what you want.
>
> Haven't tested it, but worth a try.
>
Even simpler, just remove email.message_from_file! :-)

The following works fine:-

#!/usr/bin/python3

import mailbox
import sys
import email

# open the existing maildir and the target mbox file
maildir = mailbox.Maildir(sys.argv [-2])
mbox = mailbox.mbox(sys.argv[-1])

# lock the mbox
mbox.lock()

# iterate over messages in the maildir and add to the mbox
for msg in maildir:
mbox.add(msg)

# close and unlock
mbox.close()
maildir.close()

Thank you for pointing me in the right direction.

--
Chris Green
·

On Fri, 25 Jun 2021 09:19:49 +0100, Chris Green <cl@isbd.net> declaimed the
following:

>
>Here's the full program where I'm encountering the error (yes, I
>should have posted this first time around) :-
>
> #!/usr/bin/python3
>
> import mailbox
> import sys
> import email
>
>
> # open the existing maildir and the target mbox file
> maildir = mailbox.Maildir(sys.argv [-2], email.message_from_file)

Is there some reason you are overriding whatever the default "factory"
is for Maildir?

https://docs.python.org/3/library/mailbox.html#maildir
"""
itervalues()
__iter__()
values()

Return an iterator over representations of all messages if called as
itervalues() or __iter__() or return a list of such representations if
called as values(). The messages are represented as instances of the
appropriate format-specific Message subclass unless a custom message
factory was specified when the Mailbox instance was initialized.
"""
Emphasis: messages are represented as instances of the appropriate
format-specific Message subclass

"""
A subclass of Mailbox for mailboxes in Maildir format. Parameter factory is
a callable object that accepts a file-like message representation (which
behaves as if opened in binary mode) and returns a custom representation.
If factory is None, MaildirMessage is used as the default message
representation. If create is True, the mailbox is created if it does not
exist.
"""
Emphasis: If factory is None, MaildirMessage is used as the default message
representation.

>Presumably the program worked as posted under python 2, all I have
>done is to change the shebang line to use python 3 (I no longer have
>python 2 on my system, otherwise I'd have used it as this is only a
>quick and dirty conversion script to be used once).
>
Don't know if
https://docs.python.org/3/library/email.policy.html#email.policy.compat32
applies.

However, you do have to take into account that, in Python 2, strings
WERE bytes, and Unicode was special. In Python 3, strings are Unicode,
while lots of internet protocols define simple bytes -- you have to
encode/decode if the contents are UTF-8 or something else.
..

--
Wulfraed Dennis Lee Bieber AF6VN
wlfraed@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/

Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:
> On Fri, 25 Jun 2021 09:19:49 +0100, Chris Green <cl@isbd.net> declaimed the
> following:
>
> >
> >Here's the full program where I'm encountering the error (yes, I
> >should have posted this first time around) :-
> >
> > #!/usr/bin/python3
> >
> > import mailbox
> > import sys
> > import email
> >
> >
> > # open the existing maildir and the target mbox file
> > maildir = mailbox.Maildir(sys.argv [-2], email.message_from_file)
>
> Is there some reason you are overriding whatever the default "factory"
> is for Maildir?
>
Only that it was 'as copied' off the blog where I found it. As you
can see elsewhere in the thread that was exactly the problem, simply
removing the email.message_from_file fixed the error.

--
Chris Green
·