Managing an Agile Software Project

Everything right or wrong with a software project is management’s fault.

Either management staffed the right people or the wrong people. Management was absent or involved. Management is hard, and there are numerous factors that can cause success or failure of a project. In the best situation you have great people who do great work. A software manager can even succeed despite themselves if they happen to staff a top-notch team even though the managers, themselves, might not be very competent. The success that a top-notch team achieves is still the manager’s fault. Failure, however, is harder to blame on the team because a manager must be able to solve problems as they come along. This article will focus on tips and knowledge to use when managing an agile software project.

Aiming for Agile

Lean, Extreme Programming (XP), and Scrum are probably the three most well-known agile frameworks. I don’t find any of these three to be sufficient on its own. Probably the most well-known at this point (at least in my region) is Scrum. I hear folks referring to iterations as “sprints” more and more. Managers are sending their folks to Scrum training left and right, and yet, Scrum is naked without XP. Scrum doesn’t offer much guidance on engineering practices, and as long as the project involves software, engineering practices are key. When managing your team, ensure you provide adequate engineering guidance. I’ve found that you can get great results when you manage according to Lean principles, run the project inside the Scrum framework, and create the software through Extreme Programming. They are quite complimentary.

Instead of going into an exhaustive explanation on each agile method, I’ll leave that research to you and talk specifically about what things to pay attention to while managing your project and striving for agility. Just think of agility as the ability to change course and adapt continuously to customer or product owner needs. The software should remain just as maintainable as the second day of its life and the pace of the project and subsequent releases should remain stable.

The Fallacy of Fixed Scope

One of the reasons “scope creep” is such a well-known term in our industry is because scope hardly ever stays the same. Even in fixed-scope projects, scope changes. This is because customers or users don’t understand 100% of the scope they need to solve their business problem. They might know 50% or 80%. In highly-regulated environments, that knowledge might even be in the 90s, but then there is the communication factor. How do you communicate scope? Do users or developers or managers attempt to write it down in prose? Do they have long verbal conversations? Even if the software customer theoretically understood 100% of the scope required to solve a problem, they cannot communicate 100% of that understanding up front.

Agile project management understands that scope is a moving target. The larger the scope, the more it will change. It’s not that the scope itself changes so much, but the understanding of that scope changes. Even with the theoretical 100% understanding, the information transition to the development team is a variable and not a perfect medium; therefore, to the team, scope will appear to have changed even if it has not.

Now, let’s move away from the theoretical 100% understanding to the more realistic 50% understanding. Imagine you have a common environment where a customer has a business problem that needs solving, and he thinks he has a pretty good idea what will solve it; hence, the 50% understanding. To manage this environment, both the team and the customer must understand the core of the business problem and be completely committed to solving it. The scope the customer has in mind might be relevant, or it may not. The software manager should foster an environment where questioning assumptions is welcome. In this environment, the software team will apply a critical eye to every story on the table and evaluate whether that story will contribute to the end goal. Every story must stand up to criticism in order to prove worthy of the team’s time. Working on the right thing is as important as making the thing worked on right. Work with the customer to determine together what scope will solve the customer’s problem the best, fastest, and cheapest. In most cases the team will be able to cut scope from the project.

The Unspoken Quality Variable

When managing a team, it’s important to understand who you have. You know how much you pay your people, and you should understand how much you are getting from them. All software engineers are not created equally, and there is no such thing as a project that will take “5 headcount”. Let’s take, for example, a team of six developers who you pay $10,000 each. Now, compare the output to a team of three more experienced developers who you pay $20,000 each. (I’m intentionally giving unrealistic salaries since salary numbers are not relevant here, just the relative size.) Studies referenced in Code Complete, by Steve McConnell, show that you can get developers who are roughly ten times as productive for only twice the price. The key is discerning each developer’s skill. It’s entirely possible that a smooth-talking developer on par with your team of six has negotiated a higher salary without the skill to back it up. That would be your mistake and his gain. Hopefully if you’ve made your way into management, you can discern people’s skills.

In an agile project, you tend to employ more generalists than specialists. With a team of generalists, you want folks who are skilled in a variety of areas. They will all work together to meet the commitment for each iteration. Staffing a team of specialists limits communication and creates bottlenecks, which then create queues. Refer to Lean software principles for information on queuing theory.

Key Performance Indicators (KPIs)

When managing an agile software project, you’ll be interested in tracking some metrics to help you gauge how the project is going. Just remember, whatever metrics you track will improve. Make sure you don’t track the wrong things. For instance, if you track number of lines of code as a productivity measure, you will see productivity soar, at least according to your metric. It’s natural to improve the measure, or game the system, as you might say; therefore, track what you really want: software delivered and customer satisfaction. Start with the following metrics and hone your tracking over time:

Defect-free stories delivered. Each story represents a unique behavior in the software from which the customer benefits. Stories delivered with defects aren’t as valuable, so don’t count those.
Customer satisfaction. Ultimately, you are creating software for a customer. Without the customer, you wouldn’t be creating the software. Keeping them happy is a great metric. It’s simple. Just ask their satisfaction level on a scale from one to ten.
Consistent velocity. Iteration over iteration, velocity should remain constant. If it continues to change, something is wrong. After about the sixth iteration, treat any significant change in velocity as a cause for alarm. If the team make-up is consistent, the velocity should remain constant. A slowing of velocity could be a sign of a codebase that is less than maintainable.

Managing the Customer

Part of your job is managing the development team, but you must also manage the customer. Customer satisfaction depends so much on managing expectations. Customers don’t like surprises. You shouldn’t provide surprises. The customer should know what to expect in the iteration demo, and if you deliver less than your team committed to, the customer is presented with an unpleasant surprise. If, for example, a team member must be out for paternity leave, inform the customer that it will affect the velocity of the next iteration.

A great way to keep the customer informed and to avoid surprises is by inviting them to the daily stand-up meeting. Every morning at 9:00 am, have the team go over three items:

What I accomplished yesterday.
What I hope to accomplish today.
What barriers are impeding progress.

Armed with this daily knowledge, the customer will start to feel like part of the team instead of an outsider. The customer will have up-to-date knowledge and won’t be greeted with any surprises. The daily stand-up meeting, weekly iteration demos, and constant communication will keep the customer happy and involved. It’s important to pull the customer into the process. If the customer isn’t engaged properly, there will be an us vs. them environment, and that is dysfunctional. When your team is in competition with the customer, neither will win.

Bugs vs. Defects

Defects are gaps between what your team committed to and what they delivered. If the team commits to a story and they find a problem with the functionality of the story, you have a defect on your hands, and the story isn’t finished. You should only count stories as “done” when your customer accepts them without defects. It does no one any good if you merely log a defect and take credit for the story anyhow.

A bug is anything that bugs the user. I’m not sure we’ll ever get away from bugs, but constant communication sure helps. In managing the software process, it’s important to understand the difference between a bug and a defect. Bugs are not negotiable. If the behavior of the software bugs a user, you have a bug, end of story. If your team and your customer agreed upon the behavior beforehand, then you have a defect, not a bug, and your team should correct defects immediately.

A defect is a gap in the story contract and acceptance criteria. A bug is not a defect since it’s an annoyance that has yet to be discussed. Bugs pop up all the time, and if the customer is motivated enough, he’ll craft a story to get rid of the bug and put that story on the backlog. A customer shouldn’t have to think about defects, however. When someone finds a defect, the team should drop everything and fix the defect right away. A defect signifies that a story which the team has already labeled “done” is, in fact, not finished. Finish the story and move on. Institute a zero-defect policy. Don’t count a story as done until it meets all acceptance criteria. You can’t get away from bugs, but you sure can eliminate defects.

Retrospectives

At the end of every iteration, have a short time where the team can reflect and craft ways to improve even more. A fun way to do this is to list feedback under three headings:

Good (what you would like to continue).
Bad (what you would like to stop).
Ugly (a fun category if an issue was particularly messy).

Consider all items action items. Your team should continue and enhance “good” items. Your team should stop, change, or overhaul bad and ugly items. A good manager will try to remedy the items listed under “bad” and “ugly”. Most of the time, you are the only one who can change these items. The team will remedy the items under their control, and the items that remain are more organizational problems. Don’t just ignore these organizational problems. It may be difficult, but as management, it’s your job to handle issues like poor facilities, equipment, software licenses, etc. Who said management was easy?

Keep retrospectives short. Thirty minutes is a good cap on a retrospective meeting. You don’t want it to linger on into a discussion on agile theory. All-in-all, you are in charge, so if your team starts a long discussion on how to move forward, you may have to make the call.

Architecture

The design of a software system can enable or cripple your team. Let pain and developer annoyance be your guide. If there is an aspect of the software that is painful or annoying, invest the time to fix it. Treat that time as an investment. That investment should yield returns. For instance, a shared development database often causes pain because one developer who alters the schema for a new feature will often break the system temporarily for the rest of the team. Moving to have a database instance on each developer machine quickly solves that particular problem. Be sensitive to pain and don’t take “we can’t do anything about that” as an answer.

Iteration Length-One Week

An iteration is the length of time a team works before demonstrating the new state of the system to the product owner. Scrum advocates one month. XP advocates one or two weeks. I strongly advocate one week. Iterations are different than releases. Iterations represent a cycle in software delivery. Lean stresses the reduction of cycle time as a means to improve throughput. Shortening cycle time to one week will help keep the team focused on exactly what they need to deliver. It’s hard to get distracted from essential stories when there is only one week to deliver them. Some folks might think that you can’t get anything done in only a week. If that is true, then you have some serious other problems that you need to tackle. If you are just beginning to embrace agile values, then you might be working with systems where even fixing a typo and releasing to production is a two-month ordeal. If that is the case, then you need to improve your build and deployment process so that it can be done in an hour, not two months. If you have some monumental issues you are battling, you need to invest effort into fixing those problems. As the manager, everything right or wrong in your organization is your fault. Take the initiative to mitigate the problems.

One-week iterations squeeze your process and force the dysfunction to pop up to the surface. With short cycle times, anything slowing down your organization will become very obvious. Problems can’t hide when you employ short cycle times. Use one-week iterations as a tool to discover existing problems. Tackle them one by one until you have a smooth-running team. You’ll then be able to deliver significant value in a single one-week iteration.

Hiring and Firing

As a manager you want to do more hiring than firing. Staffing is your responsibility. I’ll now discuss what I call a “net-negative developer.” This developer does not have low productivity. This is the developer no one misses when he is out sick. This developer might even have good intentions, but the velocity of the team is slower because of him. In fact, the team could move faster if this person went on vacation. Have the courage to make the staffing changes necessary to improve your team. Hold high standards for your team members. I understand that large corporate environments might have policies against firing without egregious performance. As a manager, it is your responsibility to navigate the political landscape to remove things that hinder your team’s performance. A net-negative developer is a constant tax on your team’s productivity. Have the courage to do what is right not only for your team but your customer or product owner.

In contrast, recognize when you have a “multiplier”. In performance reviews, you might not be able to list specific items directly attributable to this person, but it seems that whatever project the multiplier is on goes very well. This developer multiplies the efforts of the rest of the team. Perhaps this person removes roadblocks or works to streamline your process. Either way, this person might double the productivity of the entire team just by doing what they do. Recognize and reward this person. They are a valuable asset to your organization. Work to identify this type of person in your hiring process as well.

Agile Values and Principals

Before I conclude, I would like to walk through the values and principles in the Agile Manifesto (www.agilemanifesto.org). As a manager, it is your responsibility to set the culture for your department. You must define your department values and principles to operate by, and it is important to differentiate between practices and principles. Practices can be prescribed. Scrum and XP prescribe practices. They also prescribe principles. Agile prescribes NO practices. Agile is a classification of software methods based on values and principles. The practices of XP and Scrum are examples of ways to implement agile principles.

The four values of agile software development are:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Agile Manifesto places a heavy emphasis on people over tooling. Because software teams must deliver software, the teams that follow agile processes value delivering software over other artifacts that are not. . . software. These four values define an agile mindset. You can find the twelve principles of agile software development at http://agilemanifesto.org/principles.html. I would like to go through each one individually:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.Delivering software is the valuable service that your team provides to your customers. A status report is nice, but in the absence of working software, it is hardly a substitute. An architecture diagram is often pretty, but there is no ROI in the marketplace if users cannot use the system. You should elevate delivering software to the top of the priority list. Whatever else also has to be delivered can be delivered after the working software. We also emphasize continuous delivery of software. This means we do not do a single release. We release early and often so that the customer can use the software as it evolves.
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.A traditional software team might take a full year to deliver a large software project. Twelve months into the project should not be the first time the customer can change his mind. In contrast, an agile software team will demonstrate the state of the software early and often, allowing the customer to evaluate whether or not it perfectly meets their needs. The software team can incorporate any feedback early and often. If the customer has made a big mistake, the software team should enable the business by being able to react quickly and change direction. Not predicting the future should not be an unrecoverable mistake on the part of the customer.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.Short cycle time. Agile software teams deliver full releases frequently. Customers can use full releases, whether it is for internal use or releasing to real users. The more real feedback the customer can give, the quicker the agile software team can make essential changes. Often a customer does not know what will work until they can use a software feature. As they use released features, they’ll come up with ideas for improvement.
Business people and developers must work together daily throughout the project. There is no substitute for humans working with other humans. Relationships between humans are key to good working environments. Business folks and developers who do not know each other have a huge disadvantage when compared to teams where all involved see each other on a daily basis.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.You cannot manage software with assembly-line principles. Software is a specialized form of knowledge work, and folks on a project team need to be highly trained and experienced. You cannot swap team members out like standardized parts on the Army’s M-16 assault rifle. Each team member is unique and has unique ideas. As a manager, you should motivate them and set them loose. Remove any environmental roadblocks like inadequate tools. Give them the responsibility for delivering the project. Then give them the authority to do what is necessary.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.Some studies have concluded that only 7% of communication is verbal, or in the words themselves. Tone of voice accounts for 38% and non-verbal clues accounts for 55%. Therefore, if you choose e-mail as your only means of discussion, you are losing 93% of available communication bandwidth. That is no way to run a software project. Using face-to-face communication leverages the full power of communication.
Working software is the primary measure of progress.There is no substitute for showing the real software to the customer. A status report has no teeth.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.The architecture of the software system should promote an environment where technical complexity rises linearly. In contrast, many software systems have exponential complexity graphs. This causes future changes to be more and more difficult to make. Over time, a system may become unchangeable because of its complexity. You should keep the complexity curve linear for sustained development.
Continuous attention to technical excellence and good design enhances agility.Keeping the software clean will enable you to change it as necessary. Duplication is a particularly egregious software sin. As logic is duplicated, changes to business rules might require the same type of change is multiple places. This hinders agility.
Simplicity-the art of maximizing the amount of work not done-is essential.Companies often face the buy vs. build decision. Even after you have decided to build a custom software system, you also have the opportunity to purchase components as opposed to crafting every line of code yourself. A common buy vs. build decision today is to use an object-relational mapping library instead of writing every line of database access code by hand. Equally important is to help the customer simplify the proposed solution if the problem at hand can be solved with a software system of significantly less complexity. Keep everything as simple as possible. Be wary of developers who are proud of some code because of how complex it is. On the contrary. Rewrite complex code and be proud of it when it is simple.
The best architectures, requirements, and designs emerge from self-organizing teams.You have presumably hired good people. Put their brains together and have them work together to refine the process. Unilaterally prescribing a process with ultimate rigidity will ultimately frustrate your people and will not lead to maximum throughput.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.Use retrospectives to discover how to become more effective. Harness the intelligence of every team member in this process.

Conclusion

It is not easy to manage an Agile software project. You will never know it all, so expect to be a continuous learner. Never hesitate to listen to someone who has an idea for improvement. As a manager, every good idea does not have to be yours. In fact, if every good idea IS yours, you probably have some staffing problems. Always delegate authority along with responsibility and empower your people to succeed. Reach beyond your company into the community and other parts of the industry. Share ideas and do not be afraid to admit fault. Change immediately when you discover you are going in the wrong direction, but do not change any part of your process without evidence that it is not working. Hire and keep good people. Give them the tools and authority to succeed. Then get out of their way.