Blog
Scribe Mail3/Sqlite Folders
Date: 17/11/2009
I don't know about you, but the shine of a different back end to store mail in for Scribe v2 has well and truely worn off for me. By that I mean I expect performance to be considerably better than it is. I can even live with the bad write/commit speed, considering that locks need to be made and there are certain overheads to having atomic changes made to the database. However that said I was expecting positively snappy read/query times and well? Yeah it's not all that. Maybe it's the fact I'm using Sqlite rather than a big iron style SQL server. I regularly see SELECT statements take in order of 15-20 seconds to pull up a simple list of email in a folder, esp when the App is starting. I've added an index of the FolderId field so that in theory doing something like:
select * from Mail where FolderId=[inbox_id]
should be a no brainer super fast query right? Well it seems that it's not, and it takes an awfully long time sometimes.

It's not the end of the world. Now that I have an architecture that is flexible I can experiment with a range of back end storage solutions till I hit on something that meets my fast/powerful/portable/small requirements.

One idea that I've got in the back of my head is generalizing the SQL backend to handle a number of different databases. For install, in the "portable" mode you'll probably be storing reasonably small amounts of mail and the existing Sqlite is most likely enough to do the job, but for desktop installs something with more grunt (and more memory / install footprint) could be used instead, trading portability for speed. And with a decent replication tool you wouldn't be tied to that back end forever, you could migrate easily to the next big thing, or just your DB to a different portable format.

Or... and heres a big idea... I make the back end API open and people can plug their own storage in. Of course thats only open to programmers but if someone can produce a good back end implementation I would seriously consider making it available in the default install. The basic API is very controlled and there is already some documentation in the headers. And the existing back ends could be used as reference, as in you'd get source to them as well. If anyone wants a serious crack at that then email me.
 
Reply
From:
Email (optional): (Will be HTML encoded to evade harvesting)
Message:
 
Remember username and/or email in a cookie.
Notify me of new posts in this thread via email.
BBcode:
[q]text[/q]
[url=link]description[/url]
[img]url_to_image[/img]
[pre]some_code[/pre]
[b]bold_text[/b]