Archive for the ‘software engineering’ Category


Home Cooked vs Open Source. Or, Don’t Build Your Own Workflow.

First thing’s first. I love open source. I think that it’s the best thing since sliced bread. That thing that we were always told about since computer science, that of the open marketplace for components to be shared and reused HAS happened. Just not in the “buy this billing component” kind of way. It’s even better! It’s free (ish)! You download what you want and plug it in to your application. The quality varies, but if you keep your ear to the ground and do your homework, it will save you a lot of time. And time is money.

But just how much money?

I recently discovered Ohloh . It’s like professional networking, but not exactly, and it’s for open source. It has some cool features, but the one that got me instantly was that it trawls through open source repositories and gives you some very cool stats. Like just how much effort it would take to build an equivalent version of something, and how much it would cost given a yearly salary for a programmer. It uses lines of code, which are not a good metric, but rather an OK litmus test.

Another one of my favourite sites is Java-Source.net. Pick a particular category of software that you need, and it lists you a suite of open source options in Java – ready for you to do with as you will.

So let’s pick a category. Here’s one that I prepared earlier – workflow engines. Pretty much every place that I have ever worked has rolled their own in this regard. For some reason it’s considered a low hanging fruit (even though people write postgraduate theses on them). In some cases, off the shelf offerings are seen as overkill, too restrictive or just too complex. Most of the time the analysis is little more than gut feel, a kind of “Hmm… looks too hard, must have been over-engineered.” So a couple of guys get together and do a “bake at home” version. Pretty soon, the reality takes over and it doesn’t do what it’s supposed to, the use case was misunderstood, you need a management console, version control of process flows, different flows in different environments… uh oh! Suddenly, you spend a lot of time maintaining this beast.

So let’s do the stats and take the first handful of products listed on both sites (there are many others). These vary in the amount of activity going on, have quite varied features and uses – some plug in to applications, others are standalone orchestration engines. It’s not exactly scientific, but it’s interesting for illustration purposes. Let’s take the average programmer salary as $55000 (dollars, euro, pounds – it’s all about the sameish worldwide, so doesn’t really matter):

LOC = Lines Of Code

  1. Apache ODE. 108,547 LOC, 55 Person Years, $1,498,221.
  2. Taverna. 134,334 LOC, 33 Person Years, $1,832,157.
  3. jBPM. 286,618 LOC, 74 Person Years, $4,081,422.
  4. Enhydra Shark. 255,101 LOC, 65 Person Years, $3,576,525.
  5. OpenSymphony OSWorkflow. 48,303 LOC, 11 Person Years, $627,203.
  6. ObjectWeb Bonita. 67,916 LOC, 16 Person Years, $894,118.
  7. OpenWFE. 187,176 LOC, 47 Person Years, $2,608,592.
  8. WfMOpen, 152,557 LOC, 38 Person Years, $2,084,413.

The average cost of building a workflow engine?

155,069 LOC. 42 Person Years, $2,150,333.

Once again, this is completely unscientific. I have no idea whether the cost is over the lifetime of the product or initial development cost, whether management costs are included, we aren’t comparing apples with apples, and these are general purpose engines rather than a thing that does only the thing you want. But it does make for a very interesting question. Have you got the time and money to do this, or would you rather get on with the business problem at hand?

Enterprise software is a complex business. There are no shortcuts. There are no easy decisions. The landscape changes all the time. You have to weigh up support costs, training, extensibility, maintenance and skills.

My take on it? Someone did the heavy lifting already. Do yourself a favour and take advantage of it. If it doesn’t do exactly what you want, then the code is right there to change. You can always contribute it back, and if it’s good enough then it becomes the property of the community. The numbers are compelling.

Visio 2003 UML is The Bomb

I have worked with a number of different employer-provided UML tools in the past and have often been left underwhelmed. Rational Rose is a complex memory-hogging beast, ArgoUML seems clunky (although I’m happy to work with it at home since it’s free), and older versions of Visio have needed Pavel Hruby’s stencil to provide good, if fairly basic UML support.

Which is why I have been so pleasantly surprised using Visio 2003′s native UML Model Diagram – it has the CASE like features of the others (Java code generation excluded – surprise, surprise) and it “Just Works”. Within half an hour I was happily churning out structure and collaboration diagrams. The defaults are pretty intuitive (4/5) and standard actions such as moving methods between classes/interfaces, and repackaging classes are as simple as drag and drop. Change the structural details of your classes in the Model Explorer and all your diagrams update just the way you expected them to.

Java support is non-existent out of the box (it supports VB, IDL, C# and C++), but the C# native types are close enough that I’m not that fussed.

Credit where credit is due – any tool that makes me this productive also makes me very happy.

Becoming a Programmer

I came across an article posted a while ago by Rob Walling (http://www.softwarebyrob.com/articles/Software_Training_Sucks_Roll_it_Back.aspx) proposing that the best way to learn how to program is via an apprenticeship with a more experienced programmer.

I love the idea, it just sounds right to have someone shortcut you through a lot of the unnecessary pain. It is something that I have seen that could have been incredibly beneficial – especially when you consider just how many people there are working in the industry who are not up to scratch because of the "Here’s a piece of work. Now do it." mentality, which provides fresh developers with no guidance and assumes that people will simply pick up information as they need it. The problem is the lack of a feedback mechanism. No one to point out mistakes, challenge the existing design, discuss which parts are right, or point you to a shortcut.

There are two areas of the programming profession where working with someone senior could be beneficial:
- good practices. Not everyone out there knows about Code Complete, The Pragmatic Programmer et al. and the good stuff within. Having someone even point out the texts to you and help you to work through the ideas within makes a big difference. There’s good stuff there, and you can spot those who know versus those who don’t.

- conceptual versus practical real-world programming. Make a duck quack in different ways versus implement a web service that back-ends onto a 20 year old COBOL program with dubious documentation. It is the practical side of programming where an apprentice-journeyman relationship would make a huge difference. There is no substitute for experience, and learning from other people’s mistakes is a lot smarter than making them yourself.

It is not a substitute for higher education, which provides an important foundation in the field, but there is a gap between the goal of a degree which is to teach you concepts versus the practice of programming in industry. Rightly so. I would rather work with someone who understands concepts rather than someone who simply knows how to use technology X. You want people to be flexible and pick up new technologies as needed. The two should be combined in much the way that a medical student does an internship.

The natural question to ask is how you go about doing this in a deadline-driven environment, but I think this question is wrong one to be asking. Rather, how does the environment need to change to allow for this type of interaction to develop? You need an acknowledgement that the time of the both developers needs to be accounted for and set it aside. You cannot book 100% of a senior developer’s time on billable work and expect them to also coach a junior member of the team. Some companies do this type of thing incredibly well, but I would like to see this approach adopted more widely.

A problem worth solving

I came across this article in Technology Review, which talks about a completely green-field approach to software development – Intentional Software. Getting the domain experts to write their own code. There have been less-than great implementations of this sort of thing in the past, but a guy like Charles Simonyi (Microsoft Word/Hungarian notation fame) just might pull it off.

http://www.technologyreview.com/InfoTech/18047/

Know your toolset

Out of sheer curiosity I checked out the RadRace results for 2006 from Javapolis (is there a running theme here?) to see what toolkits the guys who seriously churn stuff out quickly are using. The toolsets were as diverse as anything you’re likely to see, some proprietary, some big-vendor, some open-source. What I thought was interesting was the conclusion at the end of the keynote presentation – Whatever toolset you choose, get to know it in detail and stick to it. Makes sense. It’ll always be faster if you’re not reaching for the manual every time you need to do achieve something that’s fairly standard.

However, the only way to learn a toolset is to do up a system with it. Maybe in evaluating a new tool, which seems to be a regular thing these days, we need an example project that you have to work out. By coding it up, you learn what the strengths and weaknesses of the toolkit are, and have an objective basis for comparison. If you can exclude the domain knowledge factor, such as by having the database design already done and well understood, it could be a useful learning tool. And far better than the standard handholding projects covered in most framework books, since you have to work out stuff for yourself.

Can it be done by Friday?

The biggest problem I’ve encountered in the past is something that sends shivers down the spine of most IT professionals – the dreaded question of how long a piece of work is going to take. I’m not talking about quoting for year long projects – they’re a whole different kettle of fish – but about the tasks that you have to do that are on the project plan. Software development is inherently dealing with unknown quantities – new technologies, unencountered bugs, changing requirements… the list seems to be unending. No two projects ever seem to be the same yet you are expected to give an accurate estimate every time. Even the meaning of the word changes depending on who is using it – programmer “best guess based on available understanding of the problem”, client/PM “fixed quote cast in concrete”.

The Software Engineering Profession has waxed lyrical on the topic for years, yet there is still no easily applicable model. Yes, there are tools such as function points and Cocomo, but they are no help for the most part. Heavy, complex, don’t take the nuances of the technology into account and they’re not really useful at the day-to-day scale of estimating. Is a web database system going to take the same amount of time to complete whether you use standalone JDBC, Spring templates, Hibernate or Ruby on Rails? Is it going to take the same amount of time to complete regardless of whether you are using the stable version of your favourite open-source package, or the new one with those brand new time-saving features you just have to use? No.

I approach the problem as per the XP boys. Implement a slice of functionality using the software stack, work out your “velocity” and extrapolate to a larger scale. In order to work out how long the work is taking I use a tool called ToDoList (http://www.abstractspoon.com/tdl_resources.html). Break down a task into it’s various steps and time how long each one takes. You can then base estimates of future work using the same technology by looking back on the task list for previous projects. It’s a variation on estimation
by experience, but slightly more formal since you’re not relying on memory. It would be interesting to see whether a larger resource that would let you see how long other people took to complete similar tasks would help, or whether it’s an inherently personal thing. My guess is the latter, which makes keeping a record of previous work all the more important. It’s not a panacea, but it will help.

10 Reasons Why You Are A Programmer

Sometimes, through the drudgery and stress of software development we forget why we do what we do.