Thursday, July 20, 2006

Moderating comments

I just deleted the first comment spam to appear on this blog. Since I have no plans on wasting much time on moderating the comments I have decided to change the comments policy in two ways:
1. I have enabled word verification option so posting will require entering CAPTCHA displayed by the blog site. This will raise the bar on ease of commenting but I hope it will be acceptable.
2. I have enabled posting comments by non-members of blogspot. This is intended to balance the raised bar on ease of commenting due to the 1st policy change.

On the move...

On Sunday I moved to new house. It's my first house and I'm very happy for it. But only today my ISP connected me back to Internet so I'm finally back working at home! You should have seen me hunting down some very weak WiFi signal the other day at 5AM.... but on the second thought it's probably better that no one saw me!

Saturday, July 15, 2006

Eric Sink on coding addiction

Hi, my name is Ivan, and I'm a coding addict.

Eric's post is really about WPF but he starts with admiting his addiction so I thought I do the same. I have to admit that I feel ill when I don't write enough code during a day. Couple of days like that and I'm ready to go nuts. I remember after my 8th grade (I was 14 years old at the time) I went to a vacations with my family and each day there I wrote Zilog Z80 assembler programs on paper (and of course I read loads of SF) The absolutely first thing I did when we came back home was to turn on my faithful ZX Spectrum and type it all in. I think it was an intro (as demos were called back then) that did some cool image manipulation (or it seemed cool at the time :) IIRC it translated correcly and worked right from the start but I had to tweak it some more to get an even cooler effect (or it seemed cooler at the time :)

Monday, July 10, 2006

Cryptonomicon 2 and 3?!

Yesterday a friend pointed out to me that Cryptonomicon 2 and 3 were available on Amazon. Since Cryptonomicon is one of my favorites books this left me with my mouth watering almost literaly.... until I realized that this is really a publishing strategy imployed by the publisher of Cryptonomicon in Spanish language: splitting one book into three and selling them separately. I heard about this before but didn't remember it immediately. It did however remind me on just how hard it was to come by SF and other reading material (say books on programming?) before the advent of Amazon.com and credit cards (I'm talking about situation in then Yugoslavia in the '80s and '90s) Most of people used to totally depend on the few and sorry translations of local publishers (I was one of the lucky ones - but more on that in another post) From what I hear situation is much better these days exactly because the local publishers these days have to compete with Amazon.com and the likes.

Wednesday, July 05, 2006

Hitting the maximum simultaneously open files at the stdio - Part II

As you may have read in the I part of this post in certain cases we have hit a limit of maximum simultaneously open files in Visual C++ stdio library. Since axLog engine is implemented in Visual C++ and uses iostream library to abstract IO this means that we had to raise this limit somehow and _setmaxstdio helped there. However, the hard upper limit of open files is 2,048 and since during recovery we are opening one file per table, recovery for databases with more than 2,048 tables would fail (actually less than that since there are always other open files like standard streams but you get my meaning). The solution was to use Win32 API for reading files and thus deal with OS and not library limitations. In both Windows 2000 and Windows XP the default number of open user handles per process is set to 10,000 which seemed a much more reasonable limit. And if this limit is ever reached it can be changed as described here and here (as both articles warn this should not be done lightly)

During implementation phase the most important thing to us was keeping IO stream library abstraction intact so that we didn't have to change anything elsewhere in the code (which uses throughout IO stream library interfaces) This left us with only once choice: implementing IO streams classes for files using Win32 API file managment functions. The clearest inspiration came from the handle_stream library that I found researching previous art. We couldn't use this library directly since it didn't support wide characters, it wasn't working correctly in VC++ 7.1 (VS.NET 2003) and its license status was not clear (it seems public domain but we couldn't confirm it) But it was enough to understand that what we really wanted to do is implement IO streams abstraction based on Window's handle concept. This allowed us to move opening of files to a higher level (conceptually) and not complicate further the interface of our IO stream classes (e.g. with all parameters for CreateFile) I will post implementation details in the next post.

Tuesday, July 04, 2006

The Declaration of Independence

My warmest regards and best wishes for their Republic to my US friends and colleagues. I'm neither US citizen nor resident but I'm deeply grateful to be living in the world where political ideals set in the Declaration are prevaling. To the gentlemen that wrote and ratified this document and pledged their Lives, Fortunes and sacred Honour goes my eternal gratitude and respect.

Hitting the maximum simultaneously open files at the stdio

We are working on new version of axLog that will sport some rather important changes in recovery model. Probably the biggest one is switching from single recovery file to multiple recovery files (one per table) We have a customer with immediate need for this so I sent them a modified version that was opening one file per recovery table. And it worked fine until it would suddenly fail to open a file with well formed file name, with no limitations on the directory and with enough free space on the drive. After taking several shots at solving the issue in my own code ("when you see hoofs think horses, not zebras") I finally had a moment of clarity to realize that this could be a library limitation (Log's engine is implemented in Visual C++) on the number of simultaneously open files - so I tested for it. And sure enough by default I couldn't create more than 508 file streams. I checked the results using CreateFile and that of course worked fine for any number of files that I threw at it. I started searching on the net for similar problems and soon found the following function _setmaxstdio from Microsoft Run-Time Library. As you can see this function allows setting "new maximum for the number of simultaneously open files at the stdio level." Great... except that the hard upper limit is 2,048 files and there is no going beyond that. We lucked out with this particular customer as they had to recovery a bit above 1,700 tables. But to solve this generically we had to come up with something else - more on that in next post.

Btw, since this is Microsoft Visual C++ Run-Time library specific limitation, if you use a development tool that uses STDIO from MSVCRT internally - you have this same limitation as well.

Saturday, July 01, 2006

Mounting CD-ROM images in Windows XP

If you ever need to mount a CD-ROM image as a virtual drive instead of burning it, try this unsupported Microsoft utility. The utility supports ISO and several other formats and works great. You can find a short description on how to use it here.

You can also check DAEMON Tools. I haven't tested it but seems to support more formats, works with DVDs and even works on Windows 2000. It's freeware for personal use.