MemeCode - Blog

Scribe Import/Export

Date: 12/11/2009

I rewrote the mail encoder API to handle arbitrary streams of bytes for quoted-printable, plain-text and base64 data. This is now used by the mail export code that directly interfaces with the store3 api that each of the mail store back ends use. This means that you get a fairly exact rfc822 message out of the export process. However these changes were initially buggy, so I'm in the process of verifying the changes with actual email. The way I'm doing this is for a given folder, export all the email to a rfc822 message on disk, immediately re-import it to a new object and then compare the mail objects in memory. This has already highlighted a few bugs in the code. Ideally I should be able to do this with every email in my mail store without any comparison errors. So far I've got it working for about the first 10 messages in my inbox :) To help with the debugging I wrote a small app that loads a mime message into a tree control and lets me look at the formatting of the mime segments.

Then I have to verify that this works with the 3 different back ends. Then I have to check that I can copy mail and folders between the 3 different back ends... then... then...

If you wonder why I go silent for weeks on end? It's because I'm working on the code.

The other thing I'm working on are the strange little bugs that happen in the IMAP back end, like when you run a filter that moves an IMAP email, and then sometimes crashes. Basically a IMAP "move" equates to a "copy" and a "delete". So when the delete event propagates back to the UI sometime after the move it nukes the object while other parts of the app still have a pointer to it. Unsurprisingly this is bad for stability. And I still haven't worked out the best way to deal with this.

Comments:

fret
12/11/2009 11:14pm

Just found and fixed a MIME parsing bug using the import/export test harness. Basically if a segment declares a MIME boundary in it's headers and then never uses it, the following segments were becoming the data for that segment rather than being parsed out into their own segments. It's not kosher MIME input really but my parser should handle it because sooner or later someone will send that down the line.

fret
12/11/2009 11:57pm

Yay, now I have a 1:1 mapping in the import / export of my all the mail in my inbox... some 668 email at present. Looks like import and export is working reasonably well now.

From:

Email (optional):

(Will be HTML encoded to evade harvesting)

Message:

Remember username and/or email in a cookie. Notify me of new posts in this thread via email.	BBcode: [q]text[/q] [url=link]description[/url] [img]url_to_image[/img] [pre]some_code[/pre] [b]bold_text[/b]

Sitemap