Page: 0 ... 5 ... 10 ... 15 18 19 20 21 22 23 24 25 26 ... 30 ... 35 ... 40 ... 45 ... 50 ... 55 ... 60 ... 65 ... 70 ... 75 ... 80 ... 85
Mpeg2 Parser
Date: 19/10/2009
Last night I got the latest refactor of my mpeg2 parser working on non-trivial amounts of data. I'm working on that with a view towards replacing PVAstrumento in my private stock video re-encoding app that compresses a lot of DTV files down to H264/AAC for semi long term storage. The issue with most demuxers is that they are not good at recovering from errors or keep good sync between the video and audio. PVAStrumento is the best demuxer for damaged and chopped up mpeg2 as it has a raft of error recovery logic. However it's painful to use and is not perfect either. Sometimes there are files that even it can't handle. So for most of this year I've been working on and off on a replacement parser.

As of yesterday I have a basic parser working that allows access to the raw streams. No error correcting yet, but it's a great foundation. The trick is mostly in chopping up the audio packets to match the the valid video frames using the PTS (timestamp) values in the program stream packets. At this point I have access to all the PS packets, their PTS values and all the frames of video and audio. So it's just a matter of working out which to keep and which to chuck. And I can use PVAStrumento as a reference as to what I should be doing (it works great 98% of the time) and then move on to the "problem" files and see where PVAStrumento is falling over and make my parser work for those edge cases.

Well thats the theory anyway.

Strangely I was starting with a 2MB memory buffer for the top level program stream. I was getting 15mb/s parse speed which is considerably lower than the HD's maximum transfer rate and I confirmed that the algorithm was CPU bound by checking the CPU usage.. 1 core was maxed. So I experiemented with different buffer sizes, 4mb? speed dropped to 13mb/s, 1mb? Speed up to 20mb/s, hmmmm thats odd. 512kb? 40mb/s wow... thats nice... 256kb? 54mb/s... I ended up settling on 32kb, which tops out about 59mb/s. I think thats largely because the more buffer I keep track of the most PS packets I have to keep track of and I think somewhere I have a non-optimal algorithm working on the list of packets. Anyway, I think it's most HD bound now, which is how is should be.
(0) Comments | Add Comment

Mac Fonts
Date: 15/10/2009
This morning I fixed the issue that has been blocking Mac development for the last 'n' months. Finally the fonts are looking correct and I can get on with other things. Also the advent of valgrind on Mac might make it my preferred platform for dev work. If only the debugger was a bit faster. At least I think the Mac port of Scribe will get a lot more in the way of releases in the coming months, and will definitely by available on v2.00 launch day.

As for the current Linux build, it does work. But it limps along in Ubuntu 8/9 with a host of issues since the rewrite to XCB/Cairo. Drawing is very slow, crashes often, d'n'd is broken, menus have borders when they shouldn't, dialogs can get lost behind their parent window etc etc. *sigh* However on a good note, the current Ubuntu 9 release runs it without having to recompile any of the libraries, unlike Ubuntu 8 where you need to compile support for Cairo into XCB (or something).
(1) Comment | Add Comment

IMAP in Scribe
Date: 14/10/2009
So for the last few weeks I've been using IMAP(full) in Scribe for one of my accounts and it's now getting to the point where it's usable day to day for reading and organizing mail. Which is probably a first. It does cache all the mail locally on disk so it operates offline as well. However in the cold hard light of day I expect there is still a long way to go till it's production ready. In that there are going to be lots of little things that you expect to work but just er... don't.

For instance I'm not allowing you to move folders at the moment, because it's messy and painful code to write. Basically you have to make a new IMAP folder and then copy all the items across and delete the old folder. I don't thing there is such thing as "move" in the IMAP protocol unfortunately. Also copying mail between IMAP folders and Mail2/Mail3 folders is not working at the moment, just because I haven't had time to fix it yet, when I get that sorted out I'll post a release (i.e. in a few days to a week). It also remains to be seen how well it copes with slow connections and lots of emails/folders.

But anyway I feel like I'm really getting somewhere at the moment. So I thought I'd share. Btw it doesn't totally choke on a mailbox of 21,000 email. It's slow but it doesn't die completely. I guess thats a good sign.
(0) Comments | Add Comment

Why There Isn't A New Scribe Release
Date: 25/8/2009
So if you look at the change log you'll see that I'm still making changes and fixes to the code base for v2 but there is no release. This is mainly because I know there are still bugs that I can't release into the wild. For instance, when you close the mail store manage window... it deletes your mail2 folders. Yeah... you don't want that happening to you. I had it happen to me, twice... because I didn't really investigate and fix the bug the first time. Maybe I'm just slow, but a bug that deletes your whole folders is kinda important right? I took the first time as a sign I should be using the mail3 back end (which is of itself was great for the product) but I didn't fix it. So now I've done it to myself again.

Anyway, for the most part I think the show stoppers are dealt with and I'll just sit on it for a week or so and see if it's stable. New build sometime next week.
(2) Comments | Add Comment

Mail3 Storage
Date: 25/5/2009
As of today I'm officially using the Mail3 database for mail storage. It came about primarily because I managed to overwrite one of my Mail2 folders with an empty folder by accident while testing the code that manages folders. (I will make sure that you can't do that before shipping). Anyway seeing as I had to start from scratch, the logical thing to do was dogfood the Mail3 database and get it solid. I know it's like about 4 years after I started talking about it but it's now actually here and stable enough for everyday use. (Which is not to say it's production ready, more like "beta" rather than the previous release being decidedly "alpha")

Currently it's (much?) faster to load than Mail2, has a better memory footprint, but is slower to write to. However I may be able to fix that with better use of transactions. Also there is no easy way to migrate from Mail2 to Mail3, although you can drag and drop between the mail stores while you have both open. Currently the Mail3 format supports Email (duh), Attachments, Contacts and Filters. Support will be added for Calendar Events and Contact Groups shortly.
(0) Comments | Add Comment

Online Version Update
Date: 1/5/2009
I'm planning on rolling out online checks for new versions via HTTP for most of my apps staring with Scribe. There are some things that need to be ironed out in terms of how it will actually work.

My current plan is to:
  • Have a software update component in Lgi that knows how to check for updates online using HTTP (and auto-detection of any proxy).
  • Once or week (or some appropriate time frame) the software calls the update component and it will download any new versions and prompt you to restart to install the update. The time/date of the check will be some multiples of the install time, so that the server doesn't get swamped with HTTP connections all at the same time.
  • A mechanism for manually triggering the update check will be available which will show a user interface. Otherwise the auto check will be pretty silent.
  • There will be an option to turn off auto-updates, however it will default to on.
  • When an update is downloaded to a temp folder the application will start it and then quit. The user will then have to click through the installer... which is like 2 clicks for Scribe. The install folder will now default to the current install location (if known) so even if you have installed into a non-default location it should be fairly smooth. The reason for making the user click through the install is a) to give them control and b) give the app time to quit. The problem with NSIS silent installs is they don't warn you if the file can't be written as is the case when the application is still running. At least in non-silent mode you get a warning dialog come up saying it can't write the file and there is "Abort", "Retry" options.
  • I'm hoping to extend the mechanism to handle sub-components so that I can install on demand items like plugins, internation language support and SSL dlls.

It's almost at a point where I can run an update manually. I haven't started on any auto-update functionality. But if anyone has serious reservations about it, I'm giving you a chance to speak up now. I expect there might be a vocal minority that don't like applications connecting to the internet, "phoning home" so to speak, for any reason at all. But the vast majority of application have some sort of update warning or ability these days and it almost always defaults to on so I feel like it's time I joined them.

I'm playing with the idea of collecting some basic platform and version stats via the online update script. Nothing personal of course, just the applications name, version and OS/platform. The information would be interesting to me, and I might even make some high level stats available from time to time via the blog. It might help in choosing where to focus my time.

If that all sounds scary, I'm also planning on semi-automatically sending crash reports back via the website.

So brace yourself ;)
(4) Comments | Add Comment