In defense of retrospectives


Bob Marshall wrote on his popular blog, that retrospectives make no sense if they are not about a hypothesis – or in other words, if they are not about analyzing why things didn’t go as we envisioned. This was followed with others voicing their agreement (example). While I agree with Bob on many things this time I think he missed the point of retrospectives as such.
Retrospectives for me are not a part of the process of “manufacturing” software

  • per se
  • , as they are not related to the product (software). Their purpose is not testing some pre-formulated hypotheses about the process – their point is discovery leading to self improvement. This discovery is achieved by stepping aside from the process, stopping and taking a look back to reflect on what happened, then considering our present state as a team and finally looking into our future. And key focus here should be us as a group of people – humans – being (experiencing our existence) together: how we related to one another, how we related to those not on the team, how we felt about what we were doing and – lastly – how we were doing it. By discussing this as a group we also develop stronger bonds – retrospectives are an important part of the true team-building process (as opposed to silly, artificial “team building” trips and parties).

    Retrospectives – at least the way I treat them – are akin to group therapy sessions for teams. I’m not using this analogy lightly, I think some elements of retrospectives are therapeutic. That’s why I see value in them being led – at least from time to time – by outsiders (coaches). Paradoxically it is easier for groups to open up with the assistance of a stranger, who is not part of teams’ internal dynamics nor local politics.

    No matter who leads a retrospective any pre-formulated hypotheses or (worse) outcomes are an impediment to the discovery process. We should approach a retrospective totally open-minded, curious maybe as to what will surface this time.

    That is why we should retrospect regularly, using varied techniques to avoid routine and stagnation. Retrospecting only when there is a hypothesis to test (as a lady from Target Processes said they do) is, in my humble opinion, a suboptimal practice – it is loosing most of the value retrospectives can bring.

    Breaking complexity with consistency


    Something I wrote the other day to a team I was coaching:

    As you may remember at the end of the second day I made the observation that if I were to describe you with one word I would choose “everything”. This is so because many times you collectively tried to cover everything in your designs, plans and other considerations in an attempt to get it perfect on the first try. That’s why you were unable to complete your sprint planning within the timebox.

    It is easy to get drawn into discussing all the possibilities, options and potential risks in the end producing nothing. Our minds get overwhelmed with complexity and paralyzed with the desire to get the right solution on first attempt. Agile approach is to break big, complex problems into small chunks and then fully solve them within small iterations producing working software. So, the agile way is to do something rather than everything – but to do it well and keep on doing something each and every sprint all the time improving both the product and ourselves. By keeping doing something well each day, week, month you can build anything in the long run.

    That is how agile breaks complexity with consistency.

    Ethical obligation – legal obligation?


    When I talk to software developers[1] I always stress that as professionals they have an ethical obligation to deliver good code quality. I’m not alone in this – people far more known and respected, like Ken Schwaber for example, keep on saying the very same thing for years now: if you are software professionals you have to act like professionals.
    This obligation is very much linked to the so-called structural quality (how well the code is built), because this aspect of software has a great effect on our clients and future users of the software we produce, yet it is completely hidden from the clients. Clients can, for the most part, detect visible functional defects (application no doing what was intended, system ill-behaving etc.), but they are usually not equipped to assess how well the software they pay for has been built internally. However, in the long run it is the structural quality that has a higher impact on the total cost of ownership and other financial parameters of a software project. The customers are obviously not professionals in the field of software development, but they trust that developers they hire are. Therefore, clients have all the right to expect good quality (understood as well built, testable, extendable, maintainable code) even if it was not explicitly requested in the contract. However, for now software developers yield to pressure and produce horrendous code just to meet time constraints (commonly known as “deadlines”).

    A good parallel is civil engineering. If we hire a construction company to build a building for us we expect it to be structurally safe, built from safe materials and adhering to widely accepted engineering standards. Clients are not expected to be civil engineers or construction foremans capable of personally ensuring it is the case – clients trust builders will do their job properly and by virtue of clients’ trust it is the builders’ ethical obligation. This ethical obligation is so widely recognized in societies, that is has became a legal obligation almost everywhere in the world.

    It is only logical then that the same will happen to the software development profession at some point. What I wait for now is a lawsuit that will claim damages from a software development company for poor structural quality of a working software product “delivered on time and within scope”. A client will finally come to claim damages after they find out two years after delivery that while the product works its code is unreadable, it doesn’t have an automatic test harness (unit testes, automated acceptance tests etc.) and it is therefore very costly or impossible to extend it further. I am not a lawyer, but I do think it would be a winnable case.

    I wonder if those in the software development profession really need such a wake-up call from the legal system before they will understand that it has their obligation to do things right – not just “on time, within scope”.

    [1] By “developers” I understand everyone involved in building a software product – not only programmers, but also test engineers, DB gurus, usability designers etc. etc.

    Success is a state


    Our culture is conditioning us to instinctively think of success as a one-time event. In movies when lovers finally overcome obstacles and get together the story ends – “they lived happily ever after” (the interesting questions is: “how?”). In sports an athlete runs, jumps, swims – arrives first or jumps the farthest so he is given a gold disc symbolizing his success. Done. No one can take it away from him. And so on.
    This thinking spills into business. Many startups are created because people want to follow in the footsteps of others who cashed in big selling one (looking at some of them the thinking seems to be: “create a cool service, get noticed, sell to Google/MS/whatever, live happily ever after”). Also, the whole concept of a project is built on the idea that it is an effort with a definite end occurring at a predictable moment – and we can say it was successful when it reached this end as predicted. Done. Successful, closed.

    The problem is all of this is incompatible with the current reality. This reality can be characterized by two key aspects:

    • Increasing pace of change – technology, social customs, fashions, regulations, markets – all of that changes much faster than it used to and the rate of that change still accelerates. We can safely say we live now in constant change.
    • Everything becomes more and more complex. This is largely a function of connectivity. Everything is now more connected and interrelated. Globalization means supply chains are long and complicated – a flood in Philipines impacts now manufacturers in Europe, China and consumers in the US.

      In IT specifically we build much more complex applications and we create way more complicated systems out of them than, say, 10 years ago. Much of this complexity comes from interconnectedness – when we connect systems together their complexity increases exponentially. We also use programing languages and environments that are much more abstract than 20 years ago – this may simplify programmers work, but in the end the applications produced are much more complex.

      To make things worse constant change seems to have just one direction: towards greater complexity. This means increasing complexity.

    Clearly, if we consider it the notion of success as an event is visibly absurd in a constantly changing environment of increasing complexity. The basic assumption of the classic project definition is also clearly wrong. Completing a project as planned can’t be considered a success in this environment, because the assumptions made when creating the initial plan will in most cases be long invalid by the time the project ends.

    This means we have to re-define success. My proposal is as follows:

    Success is a state in which an organization (or a system, a team etc.) adequately responds to its environment.

    By adequate response I understand delivering what the environment needs/expects while meeting internal goals/visions. Maintaining that state over time is much harder than attaining it once. To do it an organization must constantly analyze both the external conditions and itself then if necessary readjust itself as quickly as possible. In other words, in a highly complex and unstable environment the only way to maintain the state I call “success” is through empirical control. The problem now is that modern world as a whole is now such an environment.

    This has deep consequences. The old management model in which someone bright at the top (a prince, a president or a CEO) would think, invent, plan then communicate to others who will just do as they are told doesn’t work anymore. The level of complexity we face now overwhelms the brightest minds, so we need all the brain power we can harness. This means we must form cells (teams?) and tackle the complexity in small chunks – just like we do when building systems using agile methods.

    It would seem then that what we know as “agile” is in fact the only way to deal with the modern world and stay successful in it.

    CFO – Chief Fun Officer


    Software development is a very peculiar industry. If work is not fun for those doing it the products will be mediocre at best and so will be the company – it can make money but it will never be a great company attracting talented people.
    This is so because software development is not really engineering – it only looks like it because it is so technical. If done right it is in fact a fusion of art and technology – very much like a craft only requiring mental, not manual abilities. If people are not emotionally attached to their craft (like those who code&test solely for the money) they will not care if they produce a mess of spaghetti code rather than an elegant solution, they will not care what the user experience will be and they will not really care what happens with the product after they no longer work there. The only way to make them care is to make sure work is fun for them and they see a reason for the product’s existence other than the revenues it will bring.

    Of course, this rule is more universal – happy people work better in general. For example Southwest Airlines’ happy flight crews deliver a better passenger experience, happier dialysis providers at DaVita provide better treatment etc. However, in software development the difference has a more profound effect – an unhappy flight crew will get you from A to B as effectively, keeping development teams unhappy will in the long run ruin the products/systems they create if not the whole company. This is so, because the less fun their work is the more technical debt there is (of all kinds: bugs, low code readability, C&P programming, suboptimal ad-hock solutions limiting scalability etc.) – and technical debt is as lethal for a business as any other unpaid accumulating debt.

    Why it is so? I could bring up some theories, but I think it is less important than realising it and taking notice. Big players do. Take Google for example. They go to great lengths to make the experience of working there as fun as possible. The most visible aspect of this is their offices (each is different, BTW) but it goes way deeper – the way they treat their people, the way they allow them to access all of their code for example, all of that shows that people running this company have a profound understanding of what makes development teams tick. And they are not alone in this, almost every leading business now goes to great lengths to ensure the work itself and environment make the experience as enjoyable as possible for the employees.

    Just to be sure – “fun” means a lot more here than just a nice working environment, cool perks and good atmosphere. It also means a shared sense of purpose in what you are doing as a company and as a team. And it also means dedication to technical excellence – a policy of zero tolerance for makeshift solutions and lack of craftsmanship. This mixture means people working in such a company can be rightly proud of what they do, proud of where they work and want to continue investing their time&energy there.

    The notion of seeing the workplace as fun is sometimes dismissed as childish. It is so because there are many jobs that simply can’t be made to be fun (think of garbage collectors, people working in slaughterhouses or on assembly lines – by now mostly Chinese – assembling over and over again same parts) and historically practically all work was “serious” – not fun at all. Luckily for us that idea is slowly eroding as people search for self-fulfilment. Spending at least 8 hours a day on a job you hate is definitely not self-fulfilment.

    Globalization and shortage of talent, especially in software development and other high tech sectors, make it very easy to run away from jobs where the “fun” part is gone (or wasn’t there ever).

    Therefore one of the key duties of executives in an IT company is to make sure people working there have fun at work. And that means not that they can play computer (or traditional) games in a special room or have fancy office furniture – this means ensuring they are having fun doing the actual work as I’ve already explained above. I’m joking sometimes that there are companies that badly need a CFO – Chief Fun Officer – to shake the boat and bring some fun back. But seriously I don’t think one such guy can make a difference – I think that rather this attitude must be at the very core of company’s culture.

    How this looks at your current place? Is working there fun? If not – take my advice: don’t waste your life, move on.

    An interesting experience in servant leadership


    I have been leading a two day workshop a couple of months ago and it was an interesting experience in servant leadership that I can now share. The workshop was about the overall systems architecture at the company I was working with at the time. My mission there was transforming their IT department into an agile organization and the workshop took place three months after the change effort started.
    The catch was that I was doing it as the company’s CIO – effectively being the boss of all the other people in the room. Given the culture that existed there before it meant there was a real risk they will stay silent and expect me to tell them what architecture I had in mind rather than discuss their ideas. Therefore I had to adopt a very different approach to leading it than when I’m acting as an external consultant/coach. The fact that I genuinely had no idea how to solve the problem at hand helped me do it – I had no architecture to push for.

    One cert to rule them all


    What I suspected for a long time will happen just did: PMI has announced its agile certificate. This is a significant development for many reasons.
    First of all it officially confirms agile’s position amongst respected management methods – as part of the mainstream. I wrote about it at length recently, so I’ll just point out here that PMI’s move spells the end of the “agile revolution” in one sense. Just like the Linux revolution before it agile came with a promise of radically reshaping the workplace. It will in some places, but overall (and just like Linux) it will not completely eradicate its older alternatives, but rather become one of them – another respectable, mature tool for managing projects/teams.

    The Future of Agile


    With 2010 ending soon it is a good moment to think about the future of agile. First – lets define it: agile is a set of principles, methods and practices that emphasise short turnaround times, high flexibility (also as a way of dealing with risk through adaptation), focus on quality and teams. Agile has been around for a decade now and it is now widely known and accepted – long gone are days when barely anyone knew about it.
    With agile now mainstream it is clearly loosing its initial momentum and freshnes. While many would object I think this is a sign of maturity. Agile as a phenomenon merely follows a natural path of methods/approaches from new to commonplace. With that its position changes. From a separate specialty driven by a few “gurus” and a crowd of active followers (and consultants) it is now becoming a part of every good manager’s methods&approaches portfolio. From something that few people specialized in agile will now be transitioning into something everyone must know to a degree appropriate for their seniority and specialty.

    This is good news, because it means the agile movement has succeeded in changing the industry – even if it means that I can envision agile practices becoming part of the Project Management Body of Knowledge (PMBOK) in three to four years. This is the book despised by many agilists (most of whom probably didn’t even read it) yet agile’s inclusion there will be a sign of its success – and an end to an era.

    In hope that in the near future there will be much less need for dogmatic “agile coaches” and much more for good pragmatic managers, who will be able to use and apply both agile and traditional project management methods – as well as manage the operational, financial and human part of their businesses/units/teams. Key here is to use methods, tools and tricks that are appropriate to get things done – as opposed to trying to squeeze every situation to fit a method one happens to know best. Dogmatism that I crticized in my last post doesn’t help here.

    Three things I don’t like in the agile community


    1. Dogmatism
    Agile is about adaptive, creative approach to complex work yet amazingly average agilists are the most dogmatic people I know. If you read their blogs and follow their tweets you will soon see dogmas being proclaimed and anathemas being cast on heretics who don’t agree.

    The irony is that those dogmas can be pretty obvious observations, just repackaged to look like great discoveries. A good example I’ve seen on Twitter recently: “If you haven’t met you are not a team”. Well, that’s pretty obvious that it is much harder for team cohesion to occur when people don’t meet – it has been known for years that colocated teams are more productive than dispersed teams. However, to say that such a team can’t be a team and can’t do anything meaningful is turning an insight into a dogma.

    Is Agile for everyone?


    Almost three years ago at the Agile Development Practices conference Mary Poppendieck took the stage and announced to the assembled agilists that Agile has become mainstream. It was met with applause.
    This moment reflects very well the mood of those involved in the agile movement back then. Everyone was sure that agile approach and practices will now take the industry by storm and reshape the way we work on software projects. For some time it indeed looked like Scrum, XP and other less known practices and methodologies will replace the dreaded waterfall and the poor quality it consistently delivered in software. Alas, three years later it is clear that even though almost everyone now claims to be ‘agile’ not everything turned out so great. In fact, it turned out that implementing Agile in teams is very hard and in large companies with many teams even harder. There were many success stories – but an also a great number of (mostly untold) stories of agile failing to deliver its promises. Clearly, Agile was working as expected only in some places.