Perenial Design Issues In An Email Client
Date: 14/2/2005
By most accounts Scribe is good. Some even say excellent. But there are some "grey" areas that still need addressing. And what I'd like to do is put forth the problems and ask my knowledgeable readers their opinion on possible solutions and expectations.

Temporary Files Ahhh, the age old question of temporary files. Or Scribe's use in a particular situation to be more precise. When a user receives an attachment they often want to open the attachment, usually in the default viewer for that file type. Because Scribe stores all data in a single folders.mail2 file it has to write the attachment out to disk for another application to be able to receive the data. So it writes the attachment into the temp folder and starts the associated application to open the file. Now the catch is when the user edits the file they usually don't realize where the file is, i.e. in the temp folder. Then they save there changes into a temp file.

At the moment Scribe produces a new unique filename every time the user opens the attachment to save losing the changes they may have made to the original temp file. It doesn't clean up after the user because they may want the temp file changes. So we end up with a ever increasing set of saved attachments in the temp folder.

I want a system that a) lets the user open an attachment with out specifying a folder to save to, b) doesn't blot over any changes to the file once it's saved somewhere and c) allows changes to the attachment be incorporated back into the email client when appropriate (say in a reply or forward to the original attachment's email).


Rich Editing Everyone at some point wants to do some basic styling or layout of text in an email, Scribe users included. Now Iíve tried several times to write a decent rich text editor based on the existing compose control. And well I suck at it. So Iíll lay down my parameters for a rich edit control and see if anyone knows of something that will fit:
  • Must have a license compatible with closed source commercial software.
  • Must have a small compiled footprint. Iím not adding a 300kb DLL to Scribe.
  • Must be cross platform: Windows, Linux and preferable BeOS as well. Or have a very limited drawing API that I can shoe horn into Lgi calls.
  • Should be fast with large chunks of plain text, e.g. 1mb.
  • Should have a rich event interface and command API or a built in toolbar for changing styles.
  • Should support HTML input / output.
So far nothing comes close to meeting all those. Iíd like to know if there is something in that vein available. Doesnít have to be open source, but if itís not it still has to meet the cross platform criteria. The other option is that some people help me write a native Lgi rich edit control. I can provide all the interfaces and even a stubbed out implementation.

Summary Listings One of the major speed bottlenecks of Scribe at the moment is when it loads a folder of objects (esp email) it expects to have access to the entire object (email). Including the body of the message. The problem with this is that itís bound by disk speed and on large folders that bogs it down a lot. Large message are read in entirety (sans attachments) into memory, costing Scribe time and RAM. Now itís going to be really hard to make it not do that because I would have to rewrite lots of the architecture to make it not read the whole message. Now I could move the body of the message into an attachment of the message, which would mean the size of the ďmailĒ would be much smaller and thus folder read speed could be vastly improved. As a nice side effect compatibility with other mail formats (like MBOX and .EML) could be improved at the same time.

The question is whether to do this now in the v1 code base or leave it to v2. If I do it now with the v1 code base itíll mean staying with v1 for a lot longer, as itís the worst of the architecture problems. If I do it in v2 it might not happen for a long time. Gee Iíve only been talking about v2 for 3 years. V2 brings with it a whole swath of nice features. The less work I have to do on v1.xx the more time I have for working on the v2 prototype code base. Is there some overriding sentiment in the user community either way?
Carlos Rocha
14/02/2005 7:48pm

As I told you before, I think that migrating the file system from .mail2 to SQLite would solve all your present issues but Rich Editing:

1. Attachements could easily be saved in the db as a blob field. You can forget problems with file corruption and concentrate your skill where it really matters. SQLite is the perfect filesystem.

2. It would be easy (to you ) to load only the message list presently in view with simple SQL queries.

3. About Rich Editing...give me some time to look around.

4. Finally, about v2: I really think that v1 is already good enough and you should concentrate yourself in v2

14/02/2005 11:23pm
Carlos: I don't disargee with using Sqlite as a backend. I've even written a tech demo using it to run Sqlite through it's paces.

It's not lack of a suitable backend that is stopping v2 work, but simply lack of time to work on it. I suspect that unless a critical mass of development is completed on the v2 codebase it won't ever get developed into a fully fledged client. My thoughts on this basically turn to fixing the architecture in v1 instead (and putting up with the current storage format), if I'm not going to ever get that critical mass happening.
17/02/2005 2:25pm
About temporary files, not all of them are editable ones, such as images in an HTML email. Those images are stored in %TEMP% just by reading the mail... I guess it would be an improvement to delete them on exit (or even on closing the email) and leaving only those actually opened by the user.
Maybe they could be stored in a folder called Scribe inside %TEMP% that would be whiped on exit.
Email (optional): (Will be HTML encoded to evade harvesting)
Remember username and/or email in a cookie.
Notify me of new posts in this thread via email.