offlineimap crashes with python 2.7.2rc1

Bug #792043 reported by Scott Moser
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
offlineimap (Debian)
Fix Released
Unknown
offlineimap (Ubuntu)
Fix Released
High
Scott Moser
Oneiric
Fix Released
High
Scott Moser

Bug Description

Binary package hint: offlineimap

sometime in this past week or so, on oneiric offlineimap began to crash for me.

I modified /usr/share/pyshared/offlineimap/accounts.py.dist to raise the error rather than just logging it, and below is the trace.

Thread 'Account sync Canonical' terminated with exception:
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/offlineimap/threadutil.py", line 149, in run
    Thread.run(self)
  File "/usr/lib/python2.7/threading.py", line 505, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/pymodules/python2.7/offlineimap/accounts.py", line 186, in syncrunner
    self.sync(siglistener)
  File "/usr/lib/pymodules/python2.7/offlineimap/accounts.py", line 261, in sync
    remoterepos.holdordropconnections()
  File "/usr/lib/pymodules/python2.7/offlineimap/repository/IMAP.py", line 71, in holdordropconnections
    self.dropconnections()
  File "/usr/lib/pymodules/python2.7/offlineimap/repository/IMAP.py", line 74, in dropconnections
    self.imapserver.close()
  File "/usr/lib/pymodules/python2.7/offlineimap/imapserver.py", line 348, in close
    imapobj.logout()
  File "/usr/lib/python2.7/imaplib.py", line 537, in logout
    self.shutdown()
  File "/usr/lib/python2.7/imaplib.py", line 1187, in shutdown
    self.file.close()
  File "/usr/lib/python2.7/imaplib.py", line 214, in __getattr__
    raise AttributeError("Unknown IMAP4 command: '%s'" % attr)
AttributeError: Unknown IMAP4 command: 'file'

I'm hesitant to attach other output because it has email addresses in it and such, but I see other warnings in offlineimap like:
WARNING: ERROR attempting to copy message 1655 for account Canonical:Unknown IMAP4 command: 'file'

WARNING: ERROR attempting to copy message 1656 for account Canonical:command: UID => unexpected response: 'Return-Path: <email address hidden>'

WARNING: ERROR attempting to copy message 1657 for account Canonical:command: UID => unexpected response: 'X-Original-To: <email address hidden>'

Using 'python2.6 /usr/bin/offlineimap' will make the bug go away.

ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: offlineimap 6.2.0.2ubuntu1
ProcVersionSignature: Ubuntu 2.6.39-3.10-generic 2.6.39
Uname: Linux 2.6.39-3-generic x86_64
Architecture: amd64
Date: Thu Jun 2 15:54:35 2011
EcryptfsInUse: Yes
InstallationMedia: Ubuntu 10.04 "Lucid Lynx" - Beta amd64 (20100318)
PackageArchitecture: all
ProcEnviron:
 LANGUAGE=en_US:en
 PATH=(custom, user)
 LANG=en_US.utf8
 LC_MESSAGES=en_US.utf8
 SHELL=/bin/bash
SourcePackage: offlineimap
UpgradeStatus: Upgraded to oneiric on 2010-11-15 (198 days ago)

Revision history for this message
Scott Moser (smoser) wrote :
description: updated
Revision history for this message
Barry Warsaw (barry) wrote :

The exception message is a lie I think, so ignore that. ;)

It is for some reason getting an AttributeError in __getattr__(), but the question is why `self.file` doesn't exist.

Based on the traceback I'm guessing you're using SSL, and in particular IMAP4_SSL class. This is fairly different between 2.6 and 2.7. Whereas in 2.6 there is no self.file attribute, in 2.7 there is. That doesn't really explain why 2.7 tracebacks though! self.file gets assigned in IMAP4_SSL.open() so, is it at all possible that the open method isn't getting called? (Seems unlikely).

Any possibility you have a reproducible test case? Have you looked upstream for any related problems?

Revision history for this message
Scott Moser (smoser) wrote :

i did a bit of a search on upstream archives, and nothing.
then, tried with current git trunk, and I see the same set of issues. I'll debug a bit.

Scott Moser (smoser)
summary: - offlineimap crashes with python 2.7
+ offlineimap crashes with python 2.7.2rc1
Revision history for this message
Scott Moser (smoser) wrote : bug when using offlineimap with python v2.7.2rc1

Hi all,
   I'm using Ubuntu Oneiric (development branch), and this week brought
python version 2.7.2rc1 into the archive.
   That broke offlineimap with SSL for me. I've opened bug 792043 [1],
and then done some debugging. The breakage is caused by a change that
went into python imaplib for python issue 1441530 [2]. You can see the
specific patch that caused the problem at [3].

   Locally, I've tested the fix below. It seems to work both with
python2.6 and python2.7.

--
[1] http://pad.lv/792043
[2] http://bugs.python.org/issue1441530
[3] http://bugs.python.org/file22063/imaplib_recv_27.diff

diff --git a/offlineimap/imaplibutil.py b/offlineimap/imaplibutil.py
index a60242b..21ee3d2 100644
--- a/offlineimap/imaplibutil.py
+++ b/offlineimap/imaplibutil.py
@@ -172,6 +172,7 @@ def new_open_ssl(self, host = '', port = IMAP4_SSL_PORT):
             # FIXME
             raise socket.error(last_error)
         self.sslobj = ssl_wrap(self.sock, self.keyfile, self.certfile)
+ self.file = self.sslobj.makefile('rb')
         self.sslobj = sslwrapper(self.sslobj)

 mustquote = re.compile(r"[^\w!#$%&'+,.:;<=>?^`|~-]")

Revision history for this message
Scott Moser (smoser) wrote :

previous comment was also posted to offlineimap mailing list:
http://article.gmane.org/gmane.mail.imap.offlineimap.general/3748

Revision history for this message
Scott Moser (smoser) wrote :

Well, my fix was apparently not complete. now UsefulIMAP4_SSL:read is hanging for me on copying [some] messages.

Revision history for this message
Scott Moser (smoser) wrote :

OK, the patch I'm attaching here has worked for me for the 3 accounts that I use.

It just basically gets rid of the WrappedIMAP4_SSL class. I've left 'UsefulIMAP4_SSL' in hoping that what is there will still satisfy the issue that it is present for on darwin.

It also, "works for me" on python2.6 at the moment.

tags: added: patch
Changed in offlineimap (Ubuntu Oneiric):
status: New → Triaged
importance: Undecided → High
Changed in offlineimap (Debian):
status: Unknown → New
Revision history for this message
Scott Moser (smoser) wrote :

On Fri, 3 Jun 2011, Scott Moser wrote:

> Hi all,
> I'm using Ubuntu Oneiric (development branch), and this week brought
> python version 2.7.2rc1 into the archive.
> That broke offlineimap with SSL for me. I've opened bug 792043 [1],
> and then done some debugging. The breakage is caused by a change that
> went into python imaplib for python issue 1441530 [2]. You can see the
> specific patch that caused the problem at [3].
>
> [1] http://pad.lv/792043
> [2] http://bugs.python.org/issue1441530
> [3] http://bugs.python.org/file22063/imaplib_recv_27.diff

Ok.. So, a lot more digging.

This issue appears to be fixed in offlineimap trunk, and is largely a dupe
of http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=569068.
, and in released version 6.3.3. Additionally, debian has 6.3.3-3, so I
will get that pulled into Ubuntu. I obviously should have checked there
first, and have requested sync from debian under bug 793574.

Previously, I had arrived at https://github.com/jgoerzen/offlineimap/wiki
(its the top hit on google), and then at
https://github.com/jgoerzen/offlineimap and, as a result was looking at
the wrong trunk. Apparently, I can't read a file named 'README'. Sorry
for that.

The only thing I would suggest is to remove or update the obsolete comment
in offlineimap/imaputil.py:WrappedIMAP4_SSL, that says:

    It provides a better readline() implementation as impaplib's
    readline() is extremly inefficient. It can also connect to IPv6
    addresses."""

This class no longer has a 'readline', and python's socket performance
issues may have been fixed under current releases of 2.5, 2.6, 2.7, and
3.X (http://bugs.python.org/issue2632).

Again, sorry for the spam, and thanks for offlineimap.

Revision history for this message
Scott Moser (smoser) wrote :

I'm marking as fix-released based on bug 793574 being fixed.
Offlineimap 6.3.3-3 is now in the archive and does not exhibit this issue.

Changed in offlineimap (Ubuntu Oneiric):
assignee: nobody → Scott Moser (smoser)
status: Triaged → Fix Released
Changed in offlineimap (Debian):
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.