Ruminati

20 years of IT: The Adventures of Me

My career has been so long and varied that I've done just about anything a typical software developer might do, using a wide variety of tools and technologies. More a software developer than just a coder, here are some of the more significant things I've done over the years.

• General programming with C#, Java, VB and Delphi
• Extensive database schema design
• Extensive database implementation
• Some database administration
• Custom GIS tools
• Reports for both electronic and paper delivery
• Wizards for guided data entry in very complex systems
• Context sensitive help
• Custom implementation of various internet protocols
• High-concurrency server-based services
• Design of low-cost warm fail-over for high-traffic systems
• Setupkits

Database design, implementation and administration

In various projects I have used Microsoft SQL Server in every version since 4.2, and on occasionally Oracle 8i. For single-user applications I have used Paradox, Jet (the engine behind Microsoft Access) and dBase.

What did I do with databases? Everything from online superannuation self-management web interfaces for large corporate clients to accounting systems, inventory systems and - rather more interestingly - GPS based asset tracking systems.

Sometimes I designed them. Most often I implemented them. You can't do implementation without doing a certain amount of administration, although my experience in this regard leans away from production environments toward development environments, which place somewhat different demands on the administrator. You don't have to worry so much about high-availability but you are much more often required to restore earlier versions of a database or even maintain several instances in various stages of change and be able to offer instant guidance to developers as to which databases have which versions of structure and test data.

GIS

The GIS involved overlaying asset movements (and various optional detail layers) on maps of cities, highways, mine-sites and suchlike. Often the maps were bought in but sometimes we had to prepare them from aerial photography with trigonometry and matrix math. Plotting asset movements to play back long journeys required adaptation of skills more normally associated with modern computer games, for example the use of KD trees to cull non-visible playback position samples, not to mention the use of DirectX to exploit modern graphics capability in a relatively platform independent manner.

Arguably the wizards fall into the domain of general programming. However, to allow the code to deduce what next to ask of the user, dynamic validation necessary. This is very much more difficult to implement than programmers expect. You end up building a mini application framework involving a number of the more complex design patterns; that or a big mess. Developers who have done it a couple of times are far more likely to produce a robust, reliable wizard in a reasonable time than those who haven't, and so it rates independent mention.

Context-sensitive help

Context-sensitive help is less exciting than it sounds; basically it's program documentation with a structural and stylistic bias toward quick reference invoked for a context. There is a tutorial or how-to element, a reference element and a requirement for brevity.

Reports

Having gone to so much trouble to collect data into a database, people often want to examine it. Generally they don't want to wade through all of it, and often they want the same summary for a different period. Reporting is the business of defining a domain of data, aggregating it in some defined way and then presenting it in a structured layout.

Various tools exist. I have used Crystal Reports in several versions. I will not use it again; there are equivalent tools that combine better quality from the developer's perspective, higher performance from the user's perspective and lower cost from the business perspective. I can think of no reason in any context to prefer Crystal Reports. I'm trying to be polite here but frankly it's expensive rubbish. The only thing I will use it for is examining old reports to convert them to a more worthy medium.

If Microsoft SQL Server is in use then the SQL Reporting engine is a good choice; it can actually do all the things that Crystal Reports claims to be good for, and it scales well unlike Crystal Server. This sounds like an ad but I'm not selling anything. The Microsoft SQL Reporting engine is free if you already licensed SQL Server.

You can deliver SQL Reports over a web server to a browser or you can embed a widget in a window for print-preview or onscreen use. I've done both. There were a few hiccups as Microsoft remembered that most of the world doesn't use inches or Letter paper but it's a rapidly maturing technology that's a pleasure to use especially considering the alternative.

I've gone on a bit about SQL Reports because you can use it irrespective of the programming language in use for application development. There are certainly other options of acceptable quality, and if SQL Server is not in use they should be seriously considered, since practically anything is better than Crystal Reports.

Custom implementations of internet protocols

There is diminishing need for DIY with protocols but occasionally it is useful especially for direct integration into applications of several protocols in concert. For example, one might use a web based report to provide HTML formatted dynamically generated content that forms the payload for bulk email. I have done this, and in the process implemented a subset of the SMTP protocol as well as implementing a simple HTTP client directly in code.

High concurrency

Multi-threading in applications and services is my normal mode of operations. There is no other way to guarantee an application will always respond immediately to the user, and services bespeak concurrent access. For very high concurrency requirements the overheads of thread management may be too high. In such cases it is better to switch to a non-blocking asynchronous callback model with a pool of shared threads rather than have a private thread for each logical context.

Low-cost warm fail-over

In the ultimate hardware failure scenario so beloved of backup equipment salesmen, the question seldom asked is "Onto what do we restore our wonderful backup?" My answer is "the warm fail-over machine at the alternate location". If you don't have one of these there isn't much point in having an offsite backup. So, set one up. And then answer the never asked question "how do we know for sure that the backup is valid and viable?" by taking the backup to Site B and restoring it onto the failover machine. Physical transfer of the backup media both addresses the question of getting the copy offsite as soon as possible, and solves the problem of large data: never underestimate the bandwidth of a briefcase full of DVDs.

In these latter days of removal high capacity disk arrays there's no reason you couldn't have a warm fail-over update twice a day even for high-traffic like a bank.

Where now?

There are embedded systems under development at work and I think I'l get involved because it isn't something I've done. Also, I'm going to have to get into C++ because although .NET is wonderful after the prerequisites are all in place, until then you can't make any assumptions about the runtime environment, and so any support code for the setupkits I'm building will have to be written in C++ and MFC.

Also there's a little project called Empire.