Requirements for software – a dysfunction?

by , under Uncategorized

Traditionally software development is driven by requirements – descriptions of functions the product should perform once done.

In the past those requirements were handed down to the development teams as a bulky specification document. Now, with agile methods being widely used they are usually put on a backlog. This backlog is managed by someone – sometimes called a Product Owner – who can move requirements up and down at any time etc. We all know this. This change alone did bring a lot of improvement. It enabled flexibility in responding to change on the product level and helped reduce waste from implementing unnecessary features.

The problem is I think this is not good enough. I think the problem is the requirements themselves. I think we could even call them dysfunctional.

Here is why.

Requirements – as they are practiced by most teams now – are deeply rooted in the sequential, cascading way of building products, with its clearly divided silos of narrow specialists. In each silo those specialists perform on the product the single activity that is their specialty, then pass it on to another silo. Understanding of the product’s purpose, understanding it as a whole is unnecessary – or at least not crucial and expected. Everyone is supposed to do well the one thing that is within their competence – and such are obviously also the limits of responsibility and expectations.

If a piece of software is needed to solve a business problem a group of “business specialists” creates and describes the needs (sometimes already selling the result to clients), then passes this description on the silo with “analysts” who in turn produce requirements, that are in turn passed to the silo with “programmers” as a description of what they are supposed to build – and so on.

The key dysfunction of this model is not even the specialization of workers (an idea stemming from 19th century industrial revolution and its 20th century success – good for making repeatable physical products in mass quantities, not for creative intellectual work), that fails to engage their full potential. The key problem here is the alienation of workers from both the product and other silos. The result is lack of shared responsibility for the end product, especially for it being indeed a solution to the original problem.

This is why a typical employee feels he has no influence on the product nor the company as a whole. This feeling – so common in larger organizations – is killing engagement and creativity. Instead, it fosters an attitude typical of the “deferred life plan”: I sit at work doing something I don’t care about only to get paid.

Agile changed a lot within the development silo. In most companies we no longer have “internal silos” – analysts, programmers and testers working in separate departments (companies that still don’t get it will change or die out). Teams share responsibility for the product, but they share that responsibility only on the technical level, not on the product level. So yes, it helped, there is more engagement etc. but the alienation between business and development continues. Common attitude in the ‘development silo’ is now “we are blacksmith trolls down here, we forge whatever is asked of us, we do it well because we pride our craft – but whether that makes sense, solves client problems etc.? We don’t care. It’s the problem of the business trolls”.

I think part of the problem here is requirements in their traditional form. What does it change if we put requirements handed down the ‘development silo’ from ‘business silo’ on a “product backlog” and call someone “The Product Owner”? Very little. This still doesn’t full potential, brainpower of people in the organization. It still the old waterfall, just one level up. In fact, once we have backlog etc. in place it may make the problem less visible, because now that “we are agile” everything seems ok.

I think that what agile is all about – at least as much as I understand it – is creating teams that will collaborate to deliver products having shared responsibility for them – and influence over them.

To achieve that we need to break one more silo – the ‘business silo’. We should build true crossfunctionality by including the “business specialists” on the teams. And teams should get problems to solve or ideas to validate, not requirements.

It seems like a small change but it is, in my opinion, fundamental. Implementing it means shifting everyone’s thinking. Implementation is no longer a goal in itself for the team – solving the original problem is. Everyone is involved, everyone’s ideas count not only regarding the technical solution but the overall solution. And responsibility is shared too – no more finger pointing blaming the other silo – “stupid busines” or “lazy developers”.

Can a backlog and a role akin to Scrum’s Product Owner still help if we have problems/ideas instead of requirements? Yes. Keeping a list and deciding which ones to address first is helpful. Backlog makes the problems/ideas waiting to be addressed visible, helps gather more information about them, serves as a communication device for other teams. A role of PO can help if it is hard to decide on the order on the backlog – if consensus can’t be reached someone has to step in and decide.

Conclusion is: want true agility? Get rid of requirements! Instead create really crossfunctional teams that include non-technical talent you have and have them work a backlog of ideas/problems.

Note: I’m fully aware what I call for is already how things are done in best, most innovative companies. I’m also aware this is what at least some of “agile gurus” had in mind when they created the backlogs. Yet still I frequently see companies who claim to be agile yet persist in keeping development and business silos separated. And they stick to requirements. I truly believe changing from requirements to ideas/problems could help transition further.

  1. Łukasz Nalepa

    But isn’t Product owner part of the scrum team? Isn’t he responsible for managing requirements, but WITH the help and collaboration of the dev team? Isn’t that, what you had in mind?

    I believe it’s prescribed in scrum guide, but often the implementation of this idea is twisted.

    Reply
  2. Mike Cohn

    Hi Andy–
    I think you raise a good point but I don’t think it is requirements that cause the problem. I think requirements are a symptom of the problem. The problem is that organizations view the product owner and development team separately. Because they’re separate, product owners need to communicate what they want built and they do it through requirements, even in the form of a product backlog.

    Product owners and dev teams need to be viewed as a single thing, all in pursuit of an excellent product. A common dysfunctionality I see is a development team that can do rated highly on an annual review even though the product they built didn’t sell. ("That was the PO’s job.") Development teams need to be accountable for the success of the products they work on.

    The same goes both ways, though, and product owners need to be held accountable for the long term technical viability of the product. If a product starts to suffer from technical debt, that is as much the fault of the product owner as any other team member.

    Reply
    • Andy Brandt

      That’s the sad state of things I also see in many companies. However, if team is to be accountable it must also has an influence on the product. Same goes for PO being accountable also for the technical side of the product. But this could be achieved only if the whole team collaboratively worked on solutions for a backlog of problems – not requirements.

      Words shape thought (just like spaces shape behavior), that’s why I think "requirements" don’t help, but rather tend to perpetuate the thinking, that the "business silo" "requires" something to be done from the "development silo". Hence the problems with POs just bringing descriptions of functions to be implemented – at best with some background as to what purpose they serve, but with no room for the team to do anything more than just dutifully implement them.

      Reply
  3. Pawel Brodzinski

    I believe that pointing requirements means that you bark at the wrong tree.

    I sort of agree with your train of thought, that having this isolation between builders and business leaders is a dysfunction. I would, however, argue with a hypothesis that requirements are the source of evil. They are simply a token of knowledge exchange and can’t be inherently good or bad.

    You can use requirements in any form to steer discussion about what you want to build or you can use them as a means to isolate business from blacksmith trolls. Don’t blame requirements though. In either case.

    By the way, thing that you don’t directly mention and one that usually stems from deep Kanban implementations is that the bigger part of value stream you map and the bigger part of the org understands this value stream, the better. I would however argue that formal organization, e.g. teams, has to follow.

    The ultimate goal was never to have teams organized in this or that way. The goal was to have shared understanding of how the work is done and shared responsibility for making it so. Cross-functional teams is one, for me preferred, way of achieving that, but by no means the only one.

    Reply
  4. Micaël Masse

    The way I understood User Stories is that when defined properly, they state the need and reason behind the requirement with a specific action to answer that need.

    Sometime, the action may not be the best one and I agree that the team should be allowed to challenge "requirements". The team should care about business value, otherwise as you said, you have bigger employee motivation and innovation issues.

    It’s mainly a culture issue or a change management issue to get people out of their old "Silo" paradigm when they only care about their personal agenda. Recognize team success and evaluate individual on team success is a good way to start I think.

    Although I never see project 100% R&D, we all have some "more known" type of work where having a solution not questioned is totally fine. I would not make it a process to always brainstorm all new ideas or issues to have the perfect solution. Balance is the key and have a dedicated person who finally decide what is the most important to do solves many never ending discussions when they not always worth it for well understood, small and clear stories. Otherwise, I agree it may worth some spike time to discover a good enough solution that can be improved overtime in next sprints anyway.

    Reply

Leave a Reply