Tuesday, 18 May 2010

Staying Efficient

The software in this post is open source and free for download:
Staying Efficient

As leader of a small team of developers, I am constantly working to keep inefficiency out of our workflow. In this post, I will write about the things I have learned about efficiency during the past years and what you can do to help your team Stay Efficient.

In all teams, small inefficiencies will trickle in over time and gradually slow your down. If these inefficiencies are left unchecked for long enough, your team will be reduced to a dead weight; everybody will put in their due 8 hours per day, but little or nothing comes of it. While these kinds of teams can thrive for years in big corporations (and indefinitely in government), it is a situation professional programmers are desperate to avoid.

But how can we avoid it? Since this "disease" is so widespread, it must be hard to combat, right? That was a rhetorical question - it isn't hard at all. Let's divide team efficiency into three categories and have a detailed look at them:
  1. Rock Stars - A rare and super efficient species of team, these guys seem to get everything done in less time than you thought possible. Need a new button in the application? Done!
  2. Code Monkeys - Steady but slow. Sometimes an hour can go by with no new code written. They will estimate 1 week to put an extra button in the application, and usually spend at least 2 weeks.
  3. Dead Weights - They will spend 2 months putting the wrong button in the wrong place. You need the sheltering wings of a large corporation to keep these teams alive; they sure as hell aren't going to make anything you can sell for a profit.
Like I hinted at above, teams tend to start out as 1 but sink gradually down to 2, or even 3 given time. It will be useful to discuss how this happens to understand how to avoid it.

From type 1 to type 2: This is always the result of complacency. (You're #1, so why try harder?) Each new inefficiency is small and unnoticeable: Build times rise from 30 seconds to 50 seconds, lunches extend from 30 minutes to 35 minutes, etc, until build times are at 40+ minutes and lunches are an hour long.

From type 2 to type 3 takes a lot of neglect, and hence a lot of time for the neglect to happen in. Most teams will tend to stay at type 2 for years because they do inefficiency cleanup in fits and bursts; they always seem to have one or two developers that get fed up with the mounting inefficiencies and take initiative to remove the worst ones. Eventually, though, these developers will get burnt out and leave. After that, you'll reach type 3 within the year.

By now it should be obvious to anyone how we can avoid this sad spiral:

Continuous vigilance

Just spending a small amount of work each week - even when (especially when!) you know your team is already super effective - to detecting and combating inefficiencies is enough to keep your team lean and mean indefinitely.

I speak from experience here. About two years ago I was given lead of a type 2 team, and have been keeping a constant vigil for big and small inefficiencies to squash. Since then, we've not only kept from slowing down, we've even increased efficiency by such a huge margin that we are sporadically acting as a type 1 team!

Go for the big things first

But! Fight your temptation to rush ahead and spend valuable time fixing the first inefficiency you can think of.

You should start out removing the most glaring inefficiencies first, then go for the small ones later. This should really go without saying, but you'd be surprised at how often managers mull around fixing small inefficiencies before facing up to the real problems. They buy faster hardware, replace old office chairs, move the team closer together, upgrade development tools, etc. without even taking a look at what the team members actually do at work. The fastest development hardware in the world doesn't help much if your developers spend half their workday not working.

While "not working" sounds like a serious neglect of duty, it is usually not due to a conscious choice by the developers. They may stop to think about a problem, and just zone out and not notice that their mind has been wandering all over the place for the past half hour before getting back to work. I call this "Attention Drift" and it is the single biggest cause of inefficiency in development teams around the globe. Therefore, it is the thing you should combat first.

Combating Attention Drift

The reason so few teams are willing to acknowledge - and much less deal with - Attention Drift is because developers are normally conflict averse. Developers will not realize their own problem; they have convinced themselves they are spending the wasted time thinking about problems and code design, and will argue forcefully and angrily that spending ten minutes staring into space instead of coding is not only acceptable, but useful. (There seem to be a taboo for insinuating that developers are just plain goofing off.) For this reason, most companies unofficially allow for extended periods of inattention as fringe benefits.

Ignoring Attention Drift like that is completely acceptable and might work out for your company, but it will not allow your team to rise to type 1 efficiency. In my opinion, every company needs a few type 1 teams to be truly successful. How can you grab new customers if you can't cater to their needs fast enough? How can you enter and dominate a new market if your competitors develop a product quicker than you?

Also, doesn't it just appear more honest to pay people when and only when they're actually working to create value for you?

Now that it's obvious that you should do something about it, the next question is: What can you actually do? Constantly walking around and poking at people who are not coding works, but it is tiresome and frightfully boring. You don't want to burn yourself out just by getting other people to do the work they're paid for.

For me, the words "repetitive" and "boring" ring a bell labeled "automation". "Poking at people who are not coding" is a task that is surprisingly easy to automate!

The "Productivity Tool" application

Simply put, I needed a program that could remind me and my team whenever our work slowed down for too long. Measuring work is simple: for a developer, working is coding, and coding is typing. So in more concrete terms, I needed a program to measure my average typing speed for the past, say, 60 seconds.

Thanks to fellow programming blogger Stephen Toub's excellent posts on hardware-level keyboard hooks and likewise mouse gesture hooks, this was a breeze to program. In just a few days, everyone on my team had this little app running as a minimized service:

(Icons courtesy of  MySiteMyWay.)

Since it is usually minimized, it wouldn't be very useful unless I integrated it with the Windows 7 desktop bar:

The service icon on the left there will always display the current activity level, so you can always keep an eye on it to see how you're doing. If it tends to stay in the low end, you are going at a very slow pace; get another coffee or beverage to pick you up.

And, to really combat the dreaded attention drift, we need to wake someone up when they appear to have zoned out completely:

This should be enough to keep honest people busy, but to be really sure nobody feel tempted to ignore the application, I decided to add a web service interface to it: Anyone connecting to port 1337 on the computer running the app will get to see your current efficiency rating in percent (0-100%).

Want to try it out? Download the installer here.

The Employee Overview tool

Constantly accessing the 6 different web services is, of course, too tiresome to work for long. We need a monitor tool:

Really simple, this program allows you to enter in the names and computer names of your team members, and it will read their current performance from their web service and display it as a simple ascii face: ":-(" means they're going slow, ":-)" means they're doing well, "???" means their app is not running, etc.

Sounds useful? You can download the installer here.

Does it work? Can it work?

Yes! It works wonders. With attention drift practically eliminated, our code base has doubled in just the few months we've been running with this setup!

Despite the fact that it is obviously working, some people keep saying "But it can't work! Your developers can cheat this system!" And, yes, that is true. It can also be detected and handled. As an actual example, my team member Name Withheld had been complaining a lot on the new tool, so I naturally kept a close eye on him. By observing closely from afar, I noticed Name Withheld hitting buttons at random while he was reading gossip on Hacker News (it's like Reddit, but uglier). I gave him a public reprimand via our mailing list, followed by a stern warning to everyone that such behavior could lead to termination of employment, and complaints and resistance would lead to a negative mark on the next employee evaluation. There have been no further incidents, and the general attitude towards the new tools has steadily improved. Several team members have even approached me and told me they think it's a great idea!

The code

To go in detail about all of it is a bit too much for one blog post, and this one is getting big enough already, so I'll leave you to download the code from here and try it out for yourself.

In further posts, I will talk about how the code works, specifically:
  • How to create and access a web service.
  • How to initiate hardware keyboard hooks. 
  • Integrating with the Windows 7 desktop toolbar.
  • Making a balloon popup warning. 
  • Utilizing Linear Memory on the overview tool for safe memory handling.
  • And probably more!
That's it for this time. Thanks for reading, and stay tuned for more!