Work Stream

I’ve alluded to “THE Plan” before. I hope it’s clear at this point what level of contempt I have for this dinosaur of BDUF (big design up front) and command and control style IT. Today I want to talk about what I perceive as being better: Work Stream, or Flow.

The first major characteristic of work streams for IT delivery is the absence of a “project.” At its essence, a project is just an arbitrary collection of “stuff” that is time boxed or cost boxed (or both!). Arbitrariness increases rather than deceases complexity, forcing teams to not only to focus on the contents of the box, but the box itself. It takes focus off of business value as the prime determinant of success, installing in its place slavish conformance to schedule and “completing the project.”

Instead, imagine a world where we stay laser focused on business value and do little (if any) long-term planning. The walls are unadorned with Work Breakdown and Gantt charts. Obnoxious stoplight reports that are never any color but red are nowhere to be found. Teams spend time doing rather than planning. I can tell, inside the company operating in your head, the inmates are running the asylum. That may be your preconceived notion, but let me explain why this notion is wrong.

In work streams, a continuous flow of value-delivering tasks are ideated by the team, worked, tested, completed, accepted, and deployed, one at a time. Prioritization is based on feedback from real users, the ones who will use the product to deliver business value. The feature in process of being worked is the one decided on by the team, including its embedded users, to have the highest return of value to the business relative to the costs of development. Success is easy to judge: did deploying the feature in question net out the desired business value?

Why is that the correct (and arguably only) definition of success? Because users cannot use software that isn’t deployed. It is better to build, deploy, and get feedback on a simple version, refining as necessary. Capture what business value you can as quickly as possible, and improve it with future iterations. This is true agility. The team can quickly react to not only improve existing lines of business, but to entirely new lines resulting from changes in the organization’s competitive landscape. The application’s features and capabilities are tightly aligned with the needs of today’s users, not with a snapshot of their needs from six, twelve, or eighteen months ago.

Gone is the need for “synchronization” meetings and onerous status reports. The unit of work is so small that its status is apparent without any additional scaffolding or ceremony. Value is always flowing from the team’s efforts, and that value is nearly instantly captured.

This is a vast departure from the old ways, where we were locked in a cycle of chasing buzzwords, missing the opportunity to create real value. We were so tied in knots worried about cost containment, schedule pressures, and scope creep that we got stuck in a Fauxgile rut, beating our heads against gates and review processes that are essentially the same Waterfall hurdles our grandfathers used to lament.

Using outdated development methodologies isn’t working for us. We fall into the control trap, trying to “manage projects” and limit costs. The trap is a vicious cycle. We watch budgets to the penny, track efforts to the second to balance against estimates, and fight tooth and nail against the dreaded “scope creep”. All of this leads to under-delivering value. Yet what if what the business needs most it isn’t cost containment? What if even more value could be obtained by increasing rather than decreasing costs (i.e. more investment)?

Stop and think for a minute… in your project where specifications were laid down by the business, estimates were made, requirements were gathered in excruciating detail, milestones were set; “THE Plan” is perfect. Until it isn’t.

You gather the team together because the stoplight report your executives love so much is deep “in the red”. You’re eighty five percent through your time box and your budget. The backlog shows that the team has only “completed” (whatever that means…) sixty percent of the specified functionality. You’re on the road to time and budget overruns, what can you do? Exhort the team to “work harder” and “catch up”? Compromise (cut) scope? Forge ahead, accepting the overruns? Curl up into a ball and cry?

That last one, obviously, is a joke.

It doesn’t have to be like that because work streams have an ace up their sleeve. Not only do they have the economics on their side, but they actually offer stronger control of the delivery process than BDUF. Yes, you read that right. Agility AND control, coming from a process that looks like chaos? Yup.

In a traditional projects, the only way to clear the hurdle rate and reap value from your investment in the project is to “complete” the entire project (potentially with compromised scope). In work streams, value is captured after each feature is completed and deployed.

In traditional projects, there is a heavy investment and high levels of risk that grow over its life cycle. In work streams, the individual investments are smaller, and the risk never grows beyond the extent of one “dud” feature.

In traditional projects, you have to muck around to establish the team’s definition of “done” (code complete? tested? accepted?). In work streams, there’s no ambiguity, you’re done when the feature is in production and generating business value; the only metric is success, and measuring it is easy!

In traditional projects, as you come to the end of your time box or cost box, you still have to deploy in order to recoup any value. In work streams, if the next feature up won’t justify the time or cost or fit within the budget, you can move on to something else in good conscience because you’ve already delivered numerous value-driving features.

In traditional projects “scope creep” is the enemy, and calls into question the beauty and perfection of “THE plan”. In work streams, new feature requests represent opportunities to adapt to competition or changing market conditions; that new idea might be valuable enough to jump to the front of the queue!

These economics are incredibly powerful, and they give the business what it was looking for all along: limiting risk, investing prudently, moving on from failures with minimal fallout, and capturing value quickly. Which is a more responsible use of organizational resources, the illusion of control (in the form of BDUF bullshitting) or real control?

Hopefully I’ve convinced you of the value of work streams over traditional projects.

What are your experiences with projects? Have you tried work streams or flow-based development? Did you move to work streams full time, or go back to projects? Please share in the comments section below.

“Catching Up” is a Vicious Cycle

If you’ve been involved in providing IT solutions, and especially been on the development team for a software project, you’ve probably experienced something very much like the following narrative:

Some executive, or their delegate, the dreaded “Project Manager” has shown up at your desk holding the latest project status report. That report, with its horrible stop light color scheme shows that your project is yellow trending toward red, or even worse, is already in the red. Specifications were laid down by the business, estimates were made, requirements were gathered in excruciating detail, milestones were set, and the project was chartered to execute the vision. “THE Plan” is perfect.

Because the project is failing on its milestones, and deviating from “THE Plan” the project is troubled, and by definition, out of control. The project team must be loafing around or it must be missing critical skills if they cannot deliver what was promised on the schedule that was promised.

To get back to work and avoid any further schedule slippage, you have to give your inquisitor an answer that demonstrates that you have been appropriately cowed; that you will go faster and allow them to regain control of your “runaway” project. So you blurt out:

“Sure we’re behind, but we have a handle on the problem, and we’ll catch up in the next [phase / sprint / iteration].”

I know, I’ve been guilty of it too. It makes you sick to your stomach to lie. First to yourself, and second to the business. Tasks have taken longer than expected, performance was unacceptable and need re-work, you lost time trying to track down a memory leak, a patch to a third-party library changed the API and requires a bunch of new coding to re-integrate. It’s a lie because you have no basis to assume that future iterations will go any better than past iterations. In all reality, they’ll likely go even worse.

Side Bar:
When in your extensive history working on IT projects have you ever “gone faster” and “caught up” when a project was behind? It was probably the exception, and not the rule, right? I would guess that it probably required several all-nighters and binges of caffeine and pizza. Why do we work like this?

That lie. That’s the first step into hell; the first step down the “project velocity death spiral.” We’ve all seen how this plays out, so many times:

The team has a bad iteration or two. The schedule starts to slip. Management pressures the team to “go faster” to “catch up.” The team works quickly, and though there is the appearance of dedication, heroic efforts are sloppy. For a time, they “complete” many features but delay, or fail to catch entirely, many bugs. The team devolves into poor habits like copy/paste coding and loses focus on solid architecture. Testing, if it was on the schedule to begin with, has fallen off due to lack of time. The schedule slips further. Breakages happen in UAT and features are rejected by customers. Management swoops in to “get quality under control.” The project team slows down to improve quality. However, the project schedule is now in worse shape than before the “catch up” effort, and the loop is closed.

Even if you don’t get yourself into this demoralizing situation, it is still disheartening to ponder that even in projects that have “succeeded” based on schedule, budget, and scope metrics, can be a failure when measured against the only metric that matters: did you deliver the business value that users needed and wanted. Wasn’t that the impetus for starting the project in the first place… delivering functionality to aid users in creating or enhancing business value?

So how do we stop the death spiral? How do we change the conversation around the metrics used to evaluate success? I have found answers in the communities around software development, specifically Agile, Lean, and DevOps. Stay tuned for future posts on how I think these ideas have insight to add to the conversation around projects and business value.

What are your experiences with the death spiral? Do you have a war story that helps further clarify the issue? Please share in the comments section below.