Waterfall to Agile – Phase 1

I’m working with a development group that wants to make improvements to their process – to produce software faster, and with higher quality.

Their current process is pretty much waterfall. The analysts write the specs and “throw them over the wall” to the developers. The developers write all the code, and then pass the project on to testing. The testers do their testing (along with having the developers making fixes/changes), and the program is finally delivered to the people who requested it.

It’s the typical process for software development that is used at most places around the world.

However, it also has the typical problems that most places encounter.

Specifications, no matter how well done, always have some things missing or unclear to the developers and testers. Developers miss things, since they have different assumptions from the analysts and testers (and, most importantly, the users). When the product is eventually delivered, the user says, “That isn’t what I wanted.”

Agile development is supposed to prevent many of these problems, and I’ve seen it happen on Agile projects I’ve worked on. However, in order to change your development process, you need to change the minds of both the development team and the users/clients. That’s where the problems arise.

“Change is scary”

Almost no one likes to change. Everyone on the planet is currently doing whatever they’re doing because they believe (at some deep level) it’s the best thing for them to do. Change might make things better, but it could also make things worse. And, since humans are more worried about losing what they already have than excited about what they might get, they don’t want to change.

So, I’m going to move slowly, and with as delicate a touch as possible.

I don’t want to say, “OK, here’s a book on Scrum/Agile/Lean/whatever. Read it this week because we’re doing everything next week,” or some similarly abrupt statement. [Well, I kind of want to, but that’s not going to do anyone any good in the long run]. Instead, I’m going to try to identify the biggest gains, with the smallest efforts, and make those changes to the development process.

My two main criteria for which changes to include are: which changes are likely to provide the biggest improvement, with the least time/effort, and which changes can start building good habits.

I want to have enough changes that we can see an impact. However, I think we also need to keep the number of changes low, so we can also successfully make them. Adding in too many changes at once will lead to not being able to succeed, which I believe will have a huge negative impact on willingness for future changes. I want to get some “wins” in with these changes, and build momentum.

“When you do things right, people won’t be sure you’ve done anything at all.” – God (from Futurama)

At first, I drew up my “ideal” development process to start with. This took into account that fact that the company I’m working with is writing software for external clients, and they can only make a limited amount of change within their clients’ processes.

Even as a list of short bullet pointed descriptions, this was over four pages long. Obviously, that’s a huge amount of change to accept at once.

Now, I will say, if you’ve got an excited, motivated team that really wants to change, going with the complete change can work well. One company I was at did a very good job that way – except for the fact that everything outside of development (e.g., project planning, scheduling, deployment, etc.) was still done in a very “command and control” method.

Therefore, for a somewhat reluctant company, I’m going with a light touch and on a small project (about 1000 hours).

The Agile changes I want to implement first

Use case/user story review

The first thing I want to do is add in a quick review of the use cases.

Currently, the analysts talk with the users and write up specifications, which are eventually passed to the developers and testers. One significant problem I’ve seen with this is missing, or unclear, specifications.

So, after the analyst has written up all the use cases, I want them to meet with the team and go over each one. This meeting should probably take around two hours, for a relatively small project. The purpose of these meetings is not to go into every detail of every field, but to bring up potentially overlooked scenarios, or clarify any unclear parts of the use case.

If this meeting leads to any unanswered questions, the analyst can go back to the users and get the information they need.

Why: This should reduce the number of situations when a developer is writing code, encounters a scenario that doesn’t have documented behavior, and needs to go back to the analyst (who hasn’t thought about the project since they wrote the documentation three months ago).

Continuous integration

I want this project to be in a continuous integration build.

Even if we don’t have any other automated tests yet, I believe this will start to build an attitude of continuous quality. Knowing that some level of quality checking occurs after each check-in, even if it’s just, “does the build complete”, it’s a huge step from nothing.

Why: Just like exercise, it’s much harder to go from no push-ups every day to one push-up, than to go from one to two push-ups daily. It starts to build a positive habit, and provides a foundation for making more improvements in the future.

Visible status (Kanban board)

There’s an old joke in software development, “The first 90 percent of a project takes 90% of the time. The last 10% of the project takes the remaining 90% of the time.”

A publicly visible chart makes it more difficult to pretend you’re on schedule, when you are nowhere close.

This is especially bad when we assume everything that needs to happen in the future will go exactly as planned. Most projects schedule testing as if it’s something that will encounter no problems – scheduling only time for the QA people. They don’t allow for any time for developers to fix bugs, analysts to answer questions, etc.

Many “pure Agile” people say a project should have automated tests in place for everything, so it should never fail in testing. I’m going to deal with the reality that this project is a stepping-stone to a better process, and not the final, “perfect” solution.

Why: The current process makes it easy to lose track of what is completed, what stage different tasks are at, and what is remaining. A Kanban board makes progress (or, the lack of progress) very visible.

Regular, and frequent, demos to the users

Why: There are three big reasons for the regular demos to users.

First, the team will discover if they’re going off-track early. This will prevent them from needing to make a huge number of changes at the delivery of the project, when the users often say, “This isn’t what I wanted.”

Second, it forces developers to make regular progress on completing things. You can’t really demo a half-done page. And if you don’t have anything new to show, it’s obvious you’re not making progress.

Third, it gets some buy-in from the users. After they see, and approve the screens and reports each week, they feel a bit of ownership and a need to be consistent with what they said earlier. So, when you deliver the final project, they’re should be many fewer problems with users being unhappy with what they got.

Regular code reviews

I expect this will be the most difficult change to implement.

Code reviews are often seen as, “point out how stupid the other developers are”, or, “a waste of developer time.”

Why: Code reviews actually do a couple of immensely good things.

The first, obviously, is identify bugs or potential problems with the code. In fact, when done well, they are the best way of identifying problems with source code. They’re also more likely to catch “edge case” problems that may not be discovered in manual tests.

They also are a great way to share knowledge – of the particular program, and general programming techniques. As good a programmer as I think I am, I’ve learned some great new techniques while attending code reviews. I’ve heard the same comment from many other programmers – junior and senior.

The big problems will be to prevent damage to peoples’ egos, and keep management convinced that code reviews are worth the time. Even though most people have seen the charts that show catching a bug in QA is ten times (or some other large multiple) more expensive than catching it in development, they tend to believe that future costs are never going to occur, or be a problem.

Project retrospective

At the end of the project, have a meeting with the team (including the users, if possible) to discuss how the process changes helped, or hurt.

I’m trying to figure out a way to track each problem, or question, that comes up during development. The programmer in me wants to make a web-based app to handle communications between the team members. Then, at the end of the project, we can use it to review the problems, look for patterns, and try to find solutions to preventing them in the next project. However, I think that may be too involved.

For now, I’m going to ask the team members to keep a list of problems. Maybe create a simple wiki page for their lists, but nothing too formal.

Why: I believe there is always something that can be improved. Review and improve one of the main points of any Lean/Kaizen/Theory of Constraints process.

The review will give them a chance to modify, or drop, things that didn’t help, and add in new parts that should help with their next project.

By having the team review the process, they’ll start to feel a sense of ownership of the process, and make their own suggestions for improvements – I hope.

Summary

Those are the six process changes I want to make for this upcoming project.

Two of them are one-time events (the use case/user story review and the project retrospective), so I don’t think they’ll be difficult to add.

The continuous integration and Kanban board should be easy to start, but will need to be monitored frequently – until they become habits.

The demos and code reviews are where I expect to see some difficulties. I can see how those may be pushed back because, “we don’t have time.” Unfortunately, if you don’t have time for quality in the middle of the project, you’re not going to have the time needed to fix things at the end of the project – which is usually two or three times longer than it would have taken you if you maintained a constant level of quality and monitoring during development.

We’ll see how things go.

Leave a Reply

Your email address will not be published. Required fields are marked *