Agile

Gregg and Stefanie have described some management perspective on Agile programming.  As a participant in several of the team rooms they mentioned, I would like to make a few comments.

Likes:           

• Active involvement with customers:  the more the developers know about what end users will want, the better.
• Emphasis on refactoring: rearranging code to avoid duplication while adding capabilities is critical for any developer.
• Retrospective: with any important activity it is important to take some time to think about what you are doing right and what could be done better.

Mixed:

• Pair programming.

When done correctly, pair programming is exhausting.  One especially rewarding session (where Mayank and I coded up how quad trees are intersected with face boundaries) produced very good code, but left us hoarse every day for a week!  What made that session work is that we both challenged each other’s intuitions freely.  The end result was reasonably well tested and reliable.  We had enough clash to write code better than either of us would have alone.

Unfortunately, a pace like that cannot be sustained for long.  It is much easier to develop a hierarchy, where someone “knows the most” about a particular area of the code, and the other partner either watches for typos, or is supervised by the more knowledgeable person.  Even this mode of pairing is tiring.

Anything that makes a team’s efforts better than the sum of individual efforts (if you just divide the work by N and give everyone something to do) is good.  But pairing requires continuous effort, and won’t improve the code without everyone’s sustained efforts.  There is a lot of middle ground between pairing and not.  Code reviews and having people frequently bounce ideas off each other gets a lot of the benefit with less stress.

• Unit testing

If tools are set up correctly, most of the errors are semantic (i.e., the code looks good and compiles, but it’s not doing what it is supposed to).  Unit testing only helps when you know the right tests to apply.  It can’t catch poor scaling (e.g, using an n-squared algorithm where an O(n) or O(nln(n)) would work. )  I have become a big fan of writing out the contract for a function before I write the code, then placing assertions to specify pre and post conditions.

I think the big take away should be: if you are testing your code correctly, mistakes should be obvious.  When you do something wrong,

• Your code should fail to compile
• Half a dozen or more unit tests should fail
• Assertions should be going off all over the place, etc.

Long undiscovered bugs cost more than those found early in testing.  Good programming demands a high level of focus on details: the more time you have to forget the code you wrote, the harder it is to fix.

• Thin Vertical Slices/Design as you go.

Positives: Thin vertical slices make sense because there is business value in quickly getting small but usable pieces functionality to customers.  If they like it, you follow up and develop it further until it meets their needs fully.  If no one buys it, the project stops and you haven’t really lost that much (because you didn’t develop more than you needed).

Negatives: The notion that software can be redesigned on the fly is only partially true.   The more people are using something the more risk there is in changing it.  No amount of testing eliminates regression risk.  If customers aren’t on board with iterative development, having a new drop every few weeks could cost you some credibility (why didn’t they do it right the first time?).  Finally, it takes a lot of skill and good judgment to balance the goal of refactoring to get better code quality with regression risks.

What do you think?  I was reading a recent survey on Agile and the results seemed largely positive.  Does this fit with your experience?

To finish up this series of posts; what Gregg's post described happened a few years ago. Since that first team room, we've kept and dropped some Agile practices, and each development team has evolved their own processes. But the original idea has stuck. He did have to coerce developers into the room the first time. But it worked. Why? Because he correctly identified the criteria for making them successful, and well, I guess people liked that.  

Now most Spatial developers still work in teams rooms . . . but by choice. Some occasionally, some full time.  I guarantee that if you walk past 3 empty developer offices, they are huddled together over laptops somewhere in a dark conference room - and Gregg is often with them, skipping a meeting. The rest of the company got so fed up with not having any conference rooms, so they built a custom team room for the die-hards. In that first team room, Spatial had its first real breakthrough on getting a multi-threaded ACIS API to scale well (api_entity_point_distance), something we'd been trying to do for years. And I believe there may have been a few unit tests written too . . .

Spatial Developers in a Team RoomSo Stefanie's right; but a slight introduction before the criteria. We were floundering big time with Agile. Our original belief was a take-no-prisoners, do-everything-the-book-said, all-R&D strategy. What we ended up with were endless meetings and mind-numbing philosophical debates on “what Agile really was”. It didn’t help when we heard outside sources say, “well, I’ve seen it before and it was like this”. But these outside sources were hapless in helping us install it at Spatial. I personally felt we were chasing a ghost.

So this is what we did; first, we carved out one significant 'delivery' of R&D that needed to be done by the next release (later to be called an epic). I let the rest of the R&D activities, the smaller miscellaneous ones (bug fixes, minor enhancements), go on their merry way un-encumbered with Agile. I assigned six people to this epic delivery, for a six week duration (later to be called a sprint), and gave them the following conditions and promises:

#1 You will work on this epic and this epic only.
No more having a delivery team schedule a bunch of disparate activities for an iteration; minor bug fixes, major project work, along with all the other crap that happens in the daily life of developers. If someone in the group was a specialist that only knew a certain area of code, I would prioritize whatever needed to done on that code later. Or, I would MAKE someone else (not on the epic) learn on the job. (This is why, largely, we only did this with a subset of our R&D staff.) But the major point was to allow them to do Agile on one specific epic, and to not be interrupted!

#2 You will all go in one room and you will not work from your offices or cubes.
BUT, and this is very important, we will never, ever, take your private office or cube away. (I had to get a personal promise from our CEO.) It was this condition that started our notion of a Team Room.

#3 The epic will be well defined and relatively narrow or specific.
I wouldn’t let an epic (or team room) start unless we had some reasonable definition and a prototype worked out. These prototypes might have been done individually or by a small group of people, but we had a pretty good idea of how we wanted to solve the problem before the team room started. Hence, it was reasonably well-defined and the group could start running on day one.

#4 All the resources needed for the epic will be in the team room.
There were to be NO external dependencies. I didn’t want to hear, “we’re blocked, Fred from the blah-blah group needs to deliver this code before we can move on”. If Fred had something that needed to be done for that epic he was in the team room for the entire six weeks. (This meant it was his ass on the line as well). This included QA, documentation and a position we created called, 'Team Room PM'. The Team Room PM was the priority man. He was on the team and made all final decisions. (But to be sure, the epic was planned and scheduled by the higher level PM group outside of development.) This did mean we had to well think-out who was to be in the room. (Hence, number three is important).

#5 It ends in six weeks.
After its over, you can spend time working independently; prototyping future epics, scanning code, fixing bugs, reading, and most importantly, thinking. You can go back into a team room when the next one starts and you’re ready.

#6 If you fail, fail quickly and decisively.
I’m a big believer in having an environment where people feel 'safe to fail'. It’s not that I wanted epics or team rooms to go bust, its more I wanted transparency. We work on a very complex and difficult piece of software. I’ve had what I thought were great ideas that didn’t pan out, and if they didn’t, you had to be man enough to say, “well, rats, that didn’t work”; and management needs to know when to stop feeding a dead horse. (Of course, the horse could be the project or you!).

#7 Lastly, the rest of XP and Agile is up to you.
Pair program, don’t pair program. Unit test, don’t unit test. Play planning poker, don’t play planning poker. Decide your own iteration schedule, one day iterations, four day iterations, one hour iterations . . . I don’t care! This might have been exhaustion on my part, but this is where it all got interesting . . . all the things that we were trying to force earlier, especially pair programming, teaminess (not a word); now happened naturally. We didn’t have to force them or set up goofy metrics to measure how much we were adhering to. (One brain-dead idea from the first year, was to award an iPad to the developer who logged the most pairing hours.)

P.S. The one aspect of Agile (or XP, if you like) I didn’t address was 'vertical slicing'. Okay, there is a lot of XP/Agile that I didn’t address, but I’m a big believer that vertical slicing is a most central and important concept. I didn’t want the team rooms to ignore this, but again, I didn’t want to force it. The question in my head was . . . if conditions #1 - 7 were in place, would vertical slicing become a natural practice, like pair programming did?

You’ll have to wait for the next post to find out!

New Year, time for some resolutions that I'm surely not going to keep. One for work - I really need to make a bigger contribution to the blog. I have a few new ideas, but before getting too ambitious, I need to use one that has been in the, well, backlog, you might say, for a while.

Let me turn my memory back to a time when I worked in the Spatial office (I'm remote now, more on that another time), and I would wander into Gregg's office where we'd solve all the problems of the world, compare notes on our sons, and make an earnest pact to get more serious about the blog. We'd list out 20 or 30 entries which I'm sure will never get written. There is one that stands out, which really Gregg should write, but since he is never going to, I'm going to force his hand.

Let me go back a more few years to a time when we were trying to improve our development practices by going through a major transition to Agile. It wasn't going too well yet, for various reasons… skepticism, an imperfect fit, resistance to change? Not sure. My particular sore point was unit testing. As manager of QA, I was very interested to see us give it a real go, but that wasn't happening. During a long discussion with our then boss about why Agile wasn't sticking, he asked, "Well, what would you do?" I said without thinking, "I'd just stick them all in a room and MAKE them do it!!!!!" (Managing a development team a year later was a humbling and probably beneficial experience for me.)

But wait, this post shouldn't be about me, Gregg was really the one with the ideas . . .

The next day we had a meeting where the boss suggested . . . just that, albeit slightly more politely. To everyone's surprise, Gregg took the idea and ran with it. Literally, I think he may have run out of the room and come back with a list on a napkin five minutes later. What was on it?

The TEAM ROOM (Gregg loves coming up with new catch phrases)

Mandatory criteria for running a team room . . .

Want to find out more about what was on it? Come back soon… I'd like to see him try to get out of this one!