How does requirements management work in the long term with Agile projects?

Question

Requirements Management in the short term for Agile projects seems like a solved problem to me.

From the Scrum angle new requirements or changes to existing requirements are delivered through User Stories. And User Stories grouped under an Epic or Feature facilitate the delivery of larger more complex requirements.

Of course, a User Story isn't technically a requirements document. It is a manageable grouping of work which maps to what is often called a Vertical Slice of functionality. And the scope of these stories can be defined unambiguously through the use of Acceptance Criteria (AC).

So, although User Stories aren't formal requirements, browsing through them can give you a pretty clear idea of their underlying requirements. In the short term.

I say in the short term because, as a project progresses, the number of User Stories increases. Thus, browsing through an ever increasing list of Stories to find a Requirement becomes less and less efficient over time.

This problem is compounded when you consider User Stories that expand on, supersede, or even negate previous Stories.

Now, imagine a 2 year gap between development iterations on a project (stable in production). The original team is gone and so is all their knowledge.

If the original team knew this was going to happen (eg, it's the nature of the business), then what measures could they take to help subsequent teams?

Sure, the backlog will provide some information, but it's hardly in an easily browsable form.

So, what can be done to help subsequent teams understand the state of the project, including why and how it got there?

In my experience, the following things don't work:

• Backlog grooming to delete or update previous User Stories so that the Backlog can be read as a requirements document.
• Documentation Sprints where team members are tasked with documenting the current state of the system.
• Documentation through Behaviour Tests. This approach is the only one I have seen come close to working. Unfortunately, Coded Behaviour tests are victims the Naming Problem. Although the tests might properly document the system, getting a fluctuating team of developers to write their tests following the same Domain terminology, wording, and style is almost impossible.

So, to reiterate:

How does one manage Agile project Requirements in the long term?

Answer 1

This seems to me to be the unspoken elephant in the room with Agile projects: how do you prevent them from evolving into chaos?

Let's look at the Agile Manifesto for a moment. Agile desires:

1. Early and continuous delivery
2. Embracing changing requirements
3. Delivering working software frequently
4. Developers and business stakeholders working together daily
5. Building projects around motivated individuals, giving them the environment and support they need, and trusting them to get the job done
6. Face to face conversation as the primary mode of communication
7. Working software as the primary measure of progress
8. Sustainable development
9. Continuous attention to technical excellence and good design
10. Simplicity - Maximizing the work you don't have to do
11. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly

I've highlighted the last four. Why? Because they're the ones that will prevent the project from collapsing under its own weight.

Sustainable development means that you (hopefully) have someone that oversees the overall development process, someone who can steer the ship beyond growing the software a little bit at a time, someone who has an overarching vision of the entire project, someone with a keen knowledge of software architecture, system design, business logic principles and UI ergonomics. An architect, in other words.

The architect is the one that says "I know you asked for this, but if we build this other thing, we can avoid these other three features that are just confusing, and focus on the core requirement." He's the one that gives the team time to reduce technical debt. He's the one that brings an overarching structure and organization to the project. He's the one that calls meetings where the team gets together and asks "How can we do things better?"

And he's the one that maintains the master requirements document.

In the book Mastering the Requirements Process, the authors maintain that there are three kinds of customers, three kinds of software projects: Rabbits, Horses and Elephants. Rabbits are the small software projects that only need informal requirements; you work directly with the customer in small sprints, the scope of the project is reasonably constrained, and the software gets done within a relatively short time frame. Elephants are those mammoth projects that require a lot of up-front planning, a tremendous amount of documentation, and a long time horizon. They are arguably not agile, although you can break them up into smaller projects that can be built using agile.

It's the Horse projects that are somewhat confounding from an agile perspective. They can still be built iteratively, you can still use short sprints, but without some discipline in the requirements gathering and planning processes, they can easily run off the rails. These are the projects that can benefit from a disciplined requirements gathering process, and then careful adaptation and modification of those requirements as the project grows, while still maintaining an overarching vision for the entire project.

---

From a personal perspective, I work with a small team of about a dozen developers. At any given moment, we might be working on three software projects at the same time, ranging anywhere from a few thousand lines of code to over 100,000. Our customer thinks they are a rabbit, but they're really a horse. The customer is engaged, but not on a daily basis.

By far our weakest area is gathering specific, testable, measurable requirements and managing the customer's expectations relative to project scope. But we're working on that.

Answer 2

The question is not completely clear but it sounds like you are concerned with requirements traceability. One idea that tends to get lost in Agile in my experience is that business requirements are what the customer wants the system to do, while user stories are really functional requirements that state how the system works. "As a user, I want to be able to X." But that is done to satisfy a requirement, which may be lost in the user story.

What works well in my experience is linking the business requirements and user stories in both directions. BR #1 may be realize by stories A and B. Story C might cover BRs #2 and #3. Put this in your project tracker, spreadsheet, whatever you are using.

Long term, this helps link what the customer is asking for (BRs) with what the system does (stories) to achieve those requirements. This should be fairly minimal documentation that is not onerous to maintain, keeping with the Agile mindset of not producing documentation that nobody needs and is a chore to keep up to date.

It also provides a way for new project members to get to speed since they have something to review to get the history behind why the software does what it does.

Answer 3

I am actually working in a kanban maintenace project of a big web shop where the original developpers are not available any more.

every userstory/requirement/bugfix has a ticketnumber and every sourcecode-change is linked to a ticketnumber in the sourcecontrol-comment-field.

sourcecontrol-checkin-s (svn) atomatically update the coressponding ticket so that in the ticket i have a link to all sourceconrtol-changesets.

If a modul/function is newly implemented there are ticketnumbers in the sourcecode, too.

In the ticket-system (redmine) the tickets are linked among each other as (is duplicate, is bug, is refinement of, ....)

so you can follow the tickets and see requirement changes over time.

Example: i have to chage something in the "orderConfirmation-Web-page". In the sourcecode of the page i see a comment: 'created for "#4711"' so i can link my new ticket to the old ticket "4711" or one of its successors.

Answer 4

Personally, I discard the user stories as a source of documentation on what the system can do. Unless active steps have been taken to create documentation (which we have done for some of our more traditional clients) the application base is really the best documentation there is.

Although, that's nothing new.

If you are using a more scaled version of Agile (like SAFe), you'll have other backlogs that are not the implementation backlog. It basically looks like this:

1. Portfolio Backlog (Strategic planning of epics and budgets)
2. Program/Release Backlog (Features and Epics)
3. Project/Team Backlog (Stories, Spikes, Bugs)

I would not recommend documenting at the Team Backlog level. It's too granular and rarely does anybody want to go to that level of detail. Not to mention that there may be stories within a Release that contradict previous stories in the Team backlog.

Instead, I would recommend working at your Release Backlog level to provide Release-level documentation. These stories rarely blow out once assigned to a particular release, and can be a stable and quick way to review what is being worked on within a given release.