As this issue of CODE Magazine goes to print, the project I�ve been talking about is code complete. Code complete is the phase of the application lifecycle where all the features of the application have been developed and the final testing process begins. This phase for our project is known as User Acceptance Testing, or UAT for short.
Additional Articles
Part 1: Act 1: Into the Great Unknown
Part 2: Act II
Part 3: Act 3: Merrily We Roll Along
How long this testing phase will take is anyone's guess. UAT is the most unpredictable phase of the development process because you don't know what you don't know. Issues that arise can be simple: an incorrect caption, a missing hot key, or an incorrectly formatted field. Issues can also be more complex: code that performs calculations incorrectly, code that doesn't get called at all, code that gets called too many times, etc. The combination of things that can go wrong is infinite, which explains the unpredictable nature of this phase. Before I discuss the Tenets of UAT, let's talk about metrics.
One of the fun things to follow during a project are the metrics. I know our development team has been following the metrics and has had some fun guessing what various metrics would be as the project progressed (I believe some bets may have been placed along the way
From this image you can deduce the following:
- Lines of application code: 149,070 (87,390 services/61,680 front end)
- Lines of test code: 78,334
- Code coverage: 63% and 73% (services/front-end)
Other metrics not shown here:
- 11,185 unit and integration tests
- 1,500+ SQL Scripts
- 13,664 commits
- 28 contributors
It's amazing how much work this team has done in a relatively short amount of time. Now what?
Tenets of UAT
While the application makes its way through the UAT process, what are developers supposed to do with their time? The three areas where developers should focus their time are:
- Fixing issues
- Adding code coverage
- Refactoring code
Fixing Issues
The first responsibility developers have during UAT is to fix any issues that arise. There are two techniques that can be employed when fixing issues. The first is to just fix the issue, run your tests, check in the code, and move on. The second - and more desirable - technique is to fix the issue and add test code that will prevent the issue found from ever re-occurring.
Tenet of UAT: Fixing issues results in more tests.
Adding Code Coverage
When perusing the metrics above, you'll see that this project has achieved 73% code coverage for the front-end project and 63% coverage for the services project. For those of you unfamiliar with code coverage, it means that the 11,000+ unit and integration tests are exercising around 100,366 of 149,070 lines of application code.
Although that's not too bad, these numbers could use some improvement. During the UAT process, we should spend time adding more test coverage to the application code. This additional coverage is an investment in the long-term viability of the project and will pay dividends as the code base evolves.
How far should these metrics go? I've found that around 8590% of code coverage is a reasonable amount of code coverage to try to achieve. Getting your coverage into the 90% range can be difficult and you'll experience diminishing returns on investment as you approach 100%.
Tenet of UAT: Increasing code coverage will result in more tests.
Refactoring Code
The final area of concentration is refactoring code. A major benefit of adding code coverage is that it requires developers to revisit their old code while adding new tests. With any project, code that you write early in the process may not be as good as code you write later in the process. Along with code quality, you may find dead code or code that can take advantage of services that weren't built when you started. This is a good time to fix past sins in your code as the pressure to “git-r-done” isn't there now.
Tenet of UAT: Adding code coverage results in more refactoring.
Here's a cheat sheet for your new tenets of UAT:
- Fixing issues results in more tests.
- Increasing code coverage results in more tests.
- Adding code coverage results in more refactoring.
Resist Summoning the I.T.I.O.T.
When a project gets close to completion, there's a tendency to try and “just get the thing out the door.” This is where the I.T.I.O.T get summoned. The I.T.I.O.T is “In the Interest of Time.” At some point, the powers that be will try to establish a hard deadline for shipping an application. This is where shortcuts get taken. This is where it's up to us as developers to resist summoning the I.T.I.O.T and follow the tenets of UAT at all times.
