Calling Msys and MingW commands from outsite their environment (i.e. CreateProcess)
16/11/2014
So I wanted to check in on how Lgi apps built on Msys / MingW these days, particularly to have a look at implementing a GDB wrapper in the LgiIde application so that I can debug stuff directly in Linux without having to drop down to the command line. But never mind that convoluted explanation.

The problem that I initially saw when trying to call Msys's 'make' was this:
0 [main] make.exe" 4408 fhandler_base::dup: dup(some disk file) failed, handle 0, Win32 error 6
/bin/sh: line 1: /c/Code/Lgi/trunk/LgiIde/C:/MinGW/msys/1.0/bin/make.exe: No such file or directory
Then after I started using a different CreateProcess wrapper I got this:
-f: Nothing to be done for `Makefile.windows'.
I checked and re-checked that the path pointed to both the Msys and MingW bin directories in make's environment. But it still wouldn't work. From the command line it was fine, but doing a CreateProcess on make failed.

It would appear that Msys make really needs to be run inside a shell of some sort. Even if it's just windows build in cmd.exe. So I ended up with code that looks like this:
CreateProcess("C:\\Windows\\System32\\cmd.exe", "/C make.exe -f Makefile.windows", ...);
Which works well enough for me.

Tags: mingw | (1) Comment | Add Comment

The Abomination Of Wingdings And The Shortfalls Of Outlook
24/3/2014
It's fairly well accepted that the font WingDings should not be used in webpages. Many modern cross platform browsers don't even support it. One of the reasons is that supporting WingDings means supporting creating symbol fonts which requires adding the SYMBOL_CHARSET define to the call to CreateFont instead of the normal ANSI_CHARSET. Due to their cross platform nature they want as much code as possible to be the same regardless of platform. So instances like this where a Windows specific font needs some special handling are often unimplemented to keep things simple. It may even be a strategic design decision made to deliberately weaken the support for it hoping that it's usage would fade away.

In my email client I have two places that I have to special case code just for WingDings. Firstly the above mentioned CreateFont call needs a special flag added so that when rendering HTML email that uses WingDings the correct glyphs are placed on the screen/page. The second place is during conversion from HTML to (unicode) text. Of course the actual bytes to render in the HTML in the document are not unicode or some easily known charset, so you have to convert from "WingDingsCharset" to Unicode using some hacked together table.

Now the question is "Who is creating all this content that uses the WingDings font?"

That's an excellent question dear reader, and the answer is: Outlook. In 2014 Outlook still happily converts:
:)
Into a WingDings HTML font tag containing a single uppercase 'J'. That will be rendered as a smiley face by Outlook and a capital J by most other software. Especially software on Macs and Linux that don't have access to that font by default. Never mind that Unicode has a perfectly functional smiley face character (☺) which could be rendered correctly by pretty much any software these days.

Outlook's character set issues don't stop there though. Take the case that I ran into recently where Outlook was used to send an email containing a URL that included a Unicode character. Now that in itself is somewhat dubious and indeed the URL referred to a Microsoft content server that should've known better than to allow a unicode character in the URL (especially when a perfectly suitable ascii character was available). Now that Unicode URL was specified used proper HTML entity encoding, however the meta charset of the HTML aaaaand the Content-Type of the MIME segment both stated the document was in us-ascii. And well, that Unicode character got converted to garbage at some point. Instead Outlook should make the charset of the HTML "utf-8" or something so that the character can exist in the specified charset. I've taken the approach of assuming the attributes of tags like 'src' and 'href' are in unicode despite the prevailing HTML charset.

Another aspect of the poor charset implementation in Outlook is the tendency to use undefined characters in the specified charset. Take for instance an Outlook generated email says it's using the ISO-8859-1 charset and then goes on to use characters in the 0x80 to 0x9F address space which is according to ISO-8859-1, undefined. Internally Outlook is using the Windows-1252 charset, which DOES define characters in that range. However instead of marking the email as "Windows-1252" it puts the incorrect ISO-8859-1 charset in the headers (they are essentially the same outside of that range). This commonly manifests as smart quotes (0x91 and 0x92) getting rendered incorrectly. Scribe of course has the ability to let the user override the charset which fixes the problem.

Now I'm in no way saying Scribe is perfect and lets all bash on Outlook. But I feel these bugs have been there for over 10 years and still haven't been addressed and something needs to be said. I doubt they will ever fix their software. Which is sad, because it makes the rest of the email software community look bad when we have to deal with the broken emails coming out of Outlook. I try and take a very literal interpretation of the incoming data. So that the user sees the content warts and all. I'm not a fan of sweeping bugs in other clients under my carpet. That has had a very poor history in the web-browser world. Ending in content bugs lasting far longer than they should have.

And if this ever gets back to someone working on the Outlook team: fix thy bugs. It's not even hard... these are really basic problems and it would most likely take a day or less to address.

Tags: email charsets Outlook | (1) Comment | Add Comment

Very high Windows 7 idle hard disk utilization
15/3/2014
For months now I've noticed that when I leave my Windows 7 machine alone for over 30 minutes the hard disk starts grinding away relentlessly. Of course if you go to investigate what it's doing all activity ceases immediately. Preventing you from finding out exactly what is going on.

I had these thoughts of possible malware or a virus infection cross my mind and so downloaded and ran Process Monitor and then left the machine sit for 50 minutes. When I came back to it sure enough the hard disk was grinding away. But now I've been logging the activity! So what's the secret process using so much disk I/O?
9:49:49.6359004 PM	MsMpEng.exe	948	ReadFile	
9:49:49.6465209 PM	MsMpEng.exe	948	ReadFile	
9:49:49.6465457 PM	MsMpEng.exe	948	ReadFile	
9:49:49.6540933 PM	MsMpEng.exe	948	ReadFile	
9:49:49.6541138 PM	MsMpEng.exe	948	ReadFile	
9:49:49.6577645 PM	MsMpEng.exe	948	ReadFile	
9:49:49.6600897 PM	MsMpEng.exe	948	ReadFile	
Oh just Microsoft's anti-malware service checking all my files. "Alright... everything seems to be in order here... as you were".

(0) Comments | Add Comment

Gear Unicorns
5/12/2013
It seems that my standards are too high, because I've been researching models of various pieces of gear that I'd like to buy at some point and I end up realizing that nothing in the market meets my needs, or if it does it's out of my budget.

Unicorn: Findings:
A: The colour laser printer.
  • ~$250.
  • Reasonably priced toner (haha.. ha.. *sigh*).
  • Good print quality.
  • Robust wireless.
  • Full size A4 tray.
  • Reasonable foot print.
Pretty much everything falls flat on the toner price. But if I remove that filter it's still hard to make all the other checkboxes fit at the same time.
At this point I've given up for a while.
B: The still / video camera.
  • Great DSLR style stills.
  • Good low light performance.
  • Longish zoom... x5.. x8?
  • Fast auto-focus in stills and video modes (inc zooming).
  • <$1000? (Maybe)
  • Smaller is better (might help with longer zoom).
  • Tethered shooting of video.
  • HDMI output.
  • Mic input.
There are a number of micro 4/3rds that kinda come close. And the EOS-M is again, kinda close. But I don't see any of them ticking all the boxes.
Pretty much decided on a Panasonic GH3 for this.
C: Replacement Mac laptop.
  • >= 8GB RAM.
  • USB3.
  • Firewire (I still regularly use a firewire audio interface).
  • At least 1000 vertical lines of display resolution.
  • Upgradable hard disk and RAM (current laptop lasted 7 years because I could fix things myself).
  • Dedicated graphics card.
  • ~$1000
The non-retina mid-2012 models seem to fit this criteria if they have the optional display upgrade, which makes them rare as hens teeth. Which has the side effect of making them much more expensive. Might just wait a while till the used prices drop.
So the old Macbook died an untimely death at my hands and so while I didn't have all the cash to spring for a mid-2012 model I decide that if I dropped the USB3 requirement I could get a much older / cheaper Macbook Pro that still ticked a lot of the boxes.

Or near enough... in fact the display is only 900px high but thats a lot better than the 13" Macbook's display. Ended up getting an early 2008 for $250 + shipping. Probably will need a RAM upgrade but at that price I can afford it.
  • 2.4ghz C2D
  • 2GB presently, 6GB when upgraded
  • 1440 by 900px screen
  • 7200rpm WD black HD (from old machine)
  • 2 x Firewire
  • 2 x USB2
  • Wireless A/G/N
  • GeForce 8600M GT (Dunno how good that'll end up being?)
I'll probably update this as times goes on and I refine criteria.

(0) Comments | Add Comment

Scribe v2.00 Final
2/12/2013
Unless someone finds some major problem with the Windows v2.00 Beta46 release I'm going to be calling that the final official release of v2.00 in 2 weeks. It's not perfect but most of the issues are minor or only happen very rarely. I'll be trying to get the Mac port up to scratch in the mean time. There are a few known issues and that I need to sort out. (The filter bar is a mess, the edit and checkbox controls should REALLY be native). Then I'll post that hopefully before Christmas.

What that'll mean for customers of InScribe is that your v1.xx key that has been working for all the v2.00 beta's is not going to work for the v2.00 final. For existing customers the upgrade pricing will be $5 USD (50% off the normal price).

Once the v2.00 final is out and the upgrade process is nice and smooth I'll be turning my attention towards the security side of email. I've been watching the news lately and to be honest the revelations coming from Snowden have been eye opening. I think it's time to make privacy the main point, and as the author of an email client I feel it's my responsibility to make that happen. So the stable v2.00 branch will increasingly be focused on seamless encryption of email, and the surrounding RFC standards.

As a side note I'm not super happy about the performance of the Sqlite database layer that makes up the mail3 folder format. So I'm actively playing with options in that space. The client can easily support plug and play storage systems via a simple API (currently implementations are mail2, mail3 and IMAP). So adding more is feasible. Maybe some experimental implementations will appear in the v2.xx releases.

Tags: scribe releases | (0) Comments | Add Comment

Why I Use Version Control
21/8/2013
Apart from the replication functionality of version control I got to use the ability to rollback code for real this week. Normally it's a stop gap that you never really need to use, but in this case I came across a problem that had me stumped. Clearly in the past my software has worked fine, and so in the case where you literally have no idea whats wrong, or even what change you made broke things there is version control.

In a nut shell the Mac port of Scribe was broken, it would start and then crash somewhere inside the Carbon library. I tried valgrinding it and that basically led nowhere useful. So I put it off for a bit before deciding to bite the bullet and walk back through my SVN commits and isolate the change that causes the bug. I started by writing a small python script that took a date as a command line parameter, that checks out the source as it was on that date. I started early in the year, and checked the 1st of each month. Compile each checkout and test it for that crash. I worked my way through till the 1/7/2013, where is started crashing... then I went back to 15/6/2013... runs fine. Forward to 23/6/2013, crashes. So some commit between 15/6 and 23/6 was to blame. The revisions in Lgi were 906 to 931 or so.

I began walking the Lgi checkout forward one revision at a time with the command:
svn up -r [revision_number]
Then recompiling after each update and running the software to test for the crash. It didn't take long to figure out that revision 912 was to blame.

Now I was getting somewhere!

So I started looking at all the source code changes in that commit and quickly ruled out all but two. So to test which part of the commit caused the crash I checked out r911 into a temporary location on my Macbook, while having r912 checked out on my PC. Then I proceeded to WinMerge changes one by one over to the r911 checkout on the Mac. That isolated the change to GList.cpp as the cause of the crash.

The final analysis is that a mismatched call to GSurface::ClipRgn(NULL) was doing something bad to the Carbon API and it would all go south from there. Reasonably easy to fix once you know what the issue is.

Yay... for source control! ;-)

Tags: coding crash | (0) Comments | Add Comment

Visual Studio 2005 crashes when debugging an application
1/8/2013
I've just spent the better part of a day trying to troubleshoot this crash, so I'm going to post this here so that Google indexes it for the next poor soul that stubs their toe on it. Firstly I'll just outline the possibly relevant pre-conditions for this problem:
  • Windows 7 64bit
  • Visual Studio 2005 sp1 (with Vista update)
  • Subsystem for UNIX-based Applications (SUA) previously installed, but currently not installed.
What happens is that after uninstalling SUA I could no longer debug any application in VS2005. The IDE would simply crash as soon as debugging started. The event log would contain the follow entry:
Faulting application name: devenv.exe, version: 8.0.50727.867, time stamp: 0x45d2c842
Faulting module name: KERNELBASE.dll, version: 6.1.7601.18015, time stamp: 0x50b83c8a
Exception code: 0xe0434f4d
Fault offset: 0x0000c41f
Faulting process id: 0x22e0
Faulting application start time: 0x01ce8e70ca7eef80
Faulting application path: C:\Program Files (x86)\Microsoft Visual Studio 8\Common7\IDE\devenv.exe
Faulting module path: C:\Windows\syswow64\KERNELBASE.dll
Report Id: 0fb5f65e-fa64-11e2-8627-e05815f5174c
No one seemed to have the same issue as me. I tried lots of different things. Anyway I eventually stumbled on the problem by mere blind luck. In the add-in manager for VS2005 there was a "VSAddin" plugin that describes itself as an Extension to the VS debugger to support debugging UNIX based applications. And since I had recently uninstalled SUA to fix an unrelated issue I immediately twigged that this add-in was probably causing the crash. So I disabled it and restarted the IDE.

Now it doesn't crash. *sigh*

Tags: vs2005 crash | (0) Comments | Add Comment

Have you ever needed to know what type of memory stick you have?
11/7/2013
Well Wikipedia has a nice SVG file that will tell you. Hold the piece of ram up to the screen and it will match one of the outlines perfectly (if your screen DPI is correct).

I had a piece of DDR2 it turns out.

I'm sorry, but that's just cool.

Tags: hack | (0) Comments | Add Comment

LED Ring Working Prototype
2/3/2013
Here is my LED ring from the previous post working in real hardware:




RGB LED ring PCB
24/1/2013
I've been working towards my own RGB LED ring PCB:



Tags: axefx mc1 led | (0) Comments | Add Comment

Intel HD4000 has no DVI output
14/1/2013
Symptoms: Windows 7 boots up, shows the splash and then the monitor goes to sleep.

Problem: The DVI display is not detected, however there is output available on the HDMI port.

Solution: Unplug and re-plug the DVI port.

(omg indeed)

Tags: HD4000 dvi | (0) Comments | Add Comment

Python's .py Windows Association
11/10/2012
Posting for future reference:
Windows is not passing command line arguments to Python programs executed from the shell.

This solved my problem with passing arguments to python scripts without invoking the python binary first.

Tags: python | (0) Comments | Add Comment