Tag: Automation

Posted on

Fauxgile is Self-Deception

I talk to a lot of developers, and I read a lot about teams and teamwork on the Internet. I absorb a lot of information about “projects” and I have to say that there’s a lot of people out there who are deceiving themselves:

“We are an Agile/Scrum shop; we have daily stand-ups, two week sprints, with sprint retrospectives, incident postmortems, and we release every six months.”

No you’re not. You’re lying to yourself. You’ve latched on to a fad, and like cargo cults in the post-WW2 Pacific theater, you’re using the right vocabulary, saying the words, performing the ceremonies, and going through the motions. The problem is that your executive team despite buying in to this “Agile Transformation” you sold them is starting to get antsy that the results aren’t what was promised. Sure, you’re moving faster, but your defect rate is still too high, and you’re still delivering features that fail to fully capture the intended business value.

The fad isn’t Agile. The fad is what I like to call Fauxgile.

The Problem

The saddest part, is that while teams usually do see improvements from dabbling around the edges of Agile, there’s way more below the surface. Somehow, they cannot shake the vestiges of “THE Plan”. It’s still there, in all of its ugliness, just sliced and diced into two-week mini plans. Because they haven’t truly addressed the constraint of the system, they’re going to slip, and they can never catch up. They’re lulled into complacency, it’s easier to go along than fight it. They let inertia become the system’s new constraint; they fall back into old habits, pushing features, and slaving away in subservient conformance to “THE Plan”.

They dress up their old waterfall SDLC in some new clothes, replacing a few meetings there, adding a few other meetings over here, and for a time, things get better. But I’m here to tell you that the Emperor is STILL naked!

It’s OK to admit that you have not fully embraced Agile principles. But for organizations that want to improve, I believe that there are four main areas in which Fauxgile organizations fall short: User Engagement, Engineering, Managing Effort, and Corporate Leadership.

User Engagement

How often is the team working in direct contact with actual users of the application? What is the cadence of deliveries? Do users have to wait months, quarters, or years to harvest value from the application? How is user feedback connected to ongoing effort? How do users learn when their feedback has been received and acted upon by the team? Is the team observing users directly? Do they asked pointed questions to drill down and bring to the surface latent value?

Fauxgile organizations tend to suffer from critically low User Engagement. This is sad because you cannot have read the Agile Manifesto and its backing principles without tripping over the following:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

http://agilemanifesto.org/principles.html

Literally the first principle is about User Engagement. And it’s only by engaging directly with the end users who will use the application to create business value that the application team knows what to build. Tight feedback loops where the application team works side by side with the business, incrementally delivering features requested by real users.

Engineering

How do you connect contributors to avoid siloism, and ensure that the application benefits from the expertise of the entire team? Is excellence, and in particular good architecture and design, a priority? Does the team weigh speed of delivery against ongoing costs, using technical debt effectively? How is the application tested, does the team still rely on manual testing? Do you still have a buildmaster, or are you using CI and CD tools? Have you ruthlessly questioned and attacked manual processes, eliminating costly and error-prone tasks by leveraging automation and process improvement? Have you adopted “______ as Code”?

Fauxgile teams fall into the chaos that has been unfortunately conflated with Agile. To achieve agility requires doing the small things right, each and every time. That is only possible by rigorous engineering: Identify the task that is the biggest source of error or inefficiency. Make every improvement possible to automate or optimize the task in question. Adjust adjacent tasks to improve flow, offloading effort to other resources if possible. Re-analyze the set of tasks, and apply the steps recursively on the task that is the new biggest source of error or inefficiency.

Automation is the future of software development. Gone are the days where development teams throw “Feature Complete” applications over the wall to QA and Operations to test and run in production. Everyone on the application team from developers to test engineers to server administrators needs to know how to code. Applications are code. Servers are provisioned and configured with code. Code is written to test other code. Applications are installed onto servers with code.

Managing Effort

Is the application’s feature set and capabilities malleable, reacting to changes in business environment? Is the team free to experiment, changing, or even abandoning, processes and procedures? How frequently do you carry out retrospectives and postmortems? When and what was the last big revelation from a retrospective? What did the team learn from that discussion? Did that new knowledge spawn experimentation, feeding back into and strengthening the team?

Caught up in the ceremony, Fauxgile organizations kicked project managers to the curb, installed Project Owners, and hired Scrum Masters. They still hold meetings where they solicit mountains of requirements. Projects turn into giant wishing wells where every manager’s special pet pony initiative gets a spot in the parade. They cannot see the forest because all of those damn trees are in the way.

Agility is about building something quickly that is “good enough” and iterating on enhancements, capturing changes to enhance competitive advantage. Agile teams are faithful to a collaborative cycle of learning and improvement. The future of project managers lies in servant leadership, helping the team to overcome obstacles, and shine a light on blind spots.

Corporate Leadership

How does the application affect the organization’s long-term strategy? Have you shared that vision to drive team members to more deeply engage with the mission? What are you doing to ensure that what is being delivered drives real business value rather than “checking the boxes” on some requirements to-do list? What is the primary performance indicator for the application team? How does changes in that indicator affect prioritization? Can that indicator be used to predict or detect problems within the team? Have you empowered the team to self-organize and make adjustments to its behavior to reinforce positive feedback loops?

Leaders in Fauxgile organizations still treat IT, and especially software development, like a captive consultancy. They “negotiate” estimates as if they were prices quoted in fixed-bid contracts. They are skeptical about these new hyped-up project management philosophies, but have caved because their IT peers have convinced them the only way they’ll be able to recruit developers.

In Agile organizations leaders are concerned with cultivating the fire that burns inside their employees; those employees were hired because they had that passion for the mission of the organization. Agile Leaders understand that their vantage point outside “the trenches” make them poor candidates for knowing what features and capabilities will yield the best outcomes. Application teams are empowered to work directly with their users to ensure that users get tools that will really improve their daily work.

There is one metric, one indicator, that matters: did the team deliver an incremental enhancement to the application that will result in increased capture of business value going forward?

Demonstrating progress and success is as simple as answering a yes/no question. Even as a lagging indicator, past performance is a great predictor of future success; keep cycle times short and re-prioritize as necessary. Put the team in the driver’s seat, and set the expectation that they should not only win, but learn how to win bigger and better in the future.

The Conclusion

Can you be successful in a Fauxgile? Sure, I guess… But once you’ve experienced what Agile can be like, Fauxgile will leave you disappointed, always thirsty for more.

What are your experiences with Fauxgile? Did your organization ever complete its transformation? What were the hardest steps on the journey? Please share in the comments section below.

Posted on

There is No Place in IT for Pretenders Who Don’t Code

I know that the title of this post is click-baity and inflammatory. Before you go all Gaston and LeFou on me, put down your implements of late-night gardening, and let me clarify…

IT is at its very best, the Shangri-La, when IT harnesses the broad world of technology to help the business do business better; gaining familiarity with new hardware, software, and techniques and bringing that knowledge to the business to drive adoption of technologies that produce business value or enhance existing processes. IT should be in the driver’s seat, making possible the most effective use of technology to help the business.

Servers need administering. They need operating system updates. They need configuration management. Quality assurance is critical to the delivery of IT services. Third-party software needs patching. Identity management and role-based access control are important. Penetration testing finds defects that would have otherwise left the business vulnerable. There are a myriad of responsibilities that land on the shoulders of IT professionals.

There are jobs… good jobs… with a purpose within IT.

Despite all that, the writing is on the wall. With the advent of so called “Infrastructure as Code”, containerized workflows using overlay networks, (a crude form of “Software Defined Networking” or “Networks as Code”), and “Configuration as Code”, software is again eating the world. In modern, cloud-centric workflows, there’s no racking and stacking; there’s no cabling; forget ISOs and install media; a couple of clicks in a portal, or a few commands in a remote shell, and BAM! A server appears. With all of this outsourcing to cloud providers, your typical IT professional is left with what looks like an ever-shrinking set of responsibilities.

So what is a poor IT professional to do? What he or she thought was a stable and lucrative career is now evaporating. Those damn software geeks are ruining everything!

If that’s how you feel, it’s time to level up your career, and learn to code.

Stop! I can see you reaching for the torches and the pitchforks again! I didn’t say anything about learning to be a software engineer. There are many coding tasks that have nothing to do with building applications:

  • That script you wrote to install the Expense Report application on user terminals after hours? CODE.
  • That JavaScript bookmarklet that fills in the form on that screen you were testing? CODE.
  • That set of cURL commands you used during penetration testing to verify proper access control? CODE.

People have this mistaken notion that code has to be this big complicated voodoo practiced only by those graduated from the ivory towers of computer science academia. Bull shit! If it executes a repeatable, automated set of steps, and those steps result in value creation for the business… IT’S CODE!

Those VMs that you rent from Microsoft (or Amazon, or Google)? They still need administering! They still need operating system updates! They still need configuration management! QA is still critical! Third-party software still needs patching! Identity management and role-based access control is more important than ever! Penetration testing still finds vulnerabilities!

What’s changed is how you interact with those assets. How you do your work needs to change to accommodate. Coding is how work in IT is going to get done in the future, and there’s no indication that the trend is going to anything but accelerate.

Infrastructure as code allows you to describe dozens (if not hundreds) of pieces of hardware in a few lines of declarative code. Networks as code will in the future describe, declaratively in some format, the topology of your network that will either be overlaid in software, or pushed as configuration to real hardware firewalls and switches. Configuration as code will allow you to describe, again declaratively, what software packages to install, what operating system modules to enable, and what access controls to impose.

Cultural movements like DevOps are sweeping the industry. Your stakeholders in IT projects expect you to deliver servers in minutes, not weeks. They expect OS updates and third-party patches to happen automatically in rolled deployments to eliminate downtime. They expect nodes to join and leave clusters in response to peaks and valleys in demand, each node an exact configuration duplicate of all of its brethren. Tests for functionality, defects, code quality, and vulnerabilities should run automatically after every build, not when someone “gets around to it”. User accounts should be quickly and easily provisioned, and they should have access to the right things from jump.

This is all achievable, and IT’S ALL CODE. So be a professional, and learn your discipline’s version of coding. It just might save your career.

What are you experiences with coding within your organization? What sort of things have you coded? How do you share that code? What was easy, and where did you experience problems? Please share in the comments section below.

Posted on

Careful with those Policies and Processes Eugene!

I sat through “one of those” meetings yesterday. The language being spoken was English, and the vocabulary sounded like words from the Agile dictionary. Yet I couldn’t parse what was going on, the words were stolen concepts, failing to match the actions and attitudes of the participants. So many attitudes from the old command and control IT mentality were surfacing that you’d swear that an “Agile transformation” was not more than a year underway, but instead had just begun.

This was a process meeting. A member of the DevOps team had asked on a number of occasions for RDP access to servers in various projects to enable deployments from Azure DevOps. Operations and Support, who are responsible for the care and feeding of virtual machines, were raising the concern that they didn’t know specifically what this DevOps team member was doing, and why they needed the access in question. So a meeting was called to develop a process for granting access and documenting server configuration changes.

The solution that the team created on the whiteboard was a set of paperwork to document the access and the changes: the who, what, and why; the preparation of a “run book” to document the configuration changes so they can be reproduced in downstream environments; and lastly, two layers of approvals. Future work included the possible addition of a configuration management product to monitor access and changes.

My skin bristled at this conversation. Why do you suppose? Was it because this new process creates more (tedious, manual) work than a simple touch of automation and a leaner, simpler, policy one-liner? In part, yes, but really what was the most disheartening was the lack of willingness to tear down the walls between project teams and Operations and Support. Participants still wanted to “own” tasks, and parts of the DevOps pipeline.

How would I solve this problem? I’m so glad you asked… Step One, a new policy:

“For application servers deployed by continuous deployment pipelines, no configuration change is to be made at any time except by scripting the change, committing that change to source control, and adding execution of that script as a step to the deployment plan.”

Bonus… there is no Step Two!

What are the consequences of choosing a simply worded policy over a set of policies and processes that are more complex?

  • It deepens the experience and skill sets of everyone involved. The project team learns what sort of issues arise in misconfigured VMs, and Operations and Support get a chance to increase their skill in development and use of development tools.
  • It adds auditability of who made the change, what was changed, and with proper commenting policies on check-in, why the change was made. Using a fully-integrated product like Azure DevOps, the check-in can be associated with a work item for the reconfiguration effort, further increasing auditability.
  • In time, the project team can develop a trust and rapport with Operations and Support, eventually allowing the project team to self-serve for application specific configuration changes.
  • If Operations and Support wish to maintain oversight of configuration changes made to servers, they can require pull requests for scripted changes as a manner of peer code review.
  • Repeat after me: “The deployment plan is the run book.” One that is instantly updated without any effort or anyone needing to remember to change it. Some documentation is better than no documentation, but self-updating documentation is even yet better!
  • Every time code is committed, the deployment plan is tested. For a development team that is following Agile principles regarding code commits, the deployment is tested dozens (hundreds?) of times a day!
  • Instead of an engineer following a run book, and manually making changes, the deployment process takes over, reducing chances for error.
  • Because the deployment plan handles change management, and you are leveraging Infrastructure as Code (ARM templates, or your Cloud provider’s equivalent) then spinning up a blank slate VM, configuring that VM properly, and installing your application should be as simple as cloning the deployment plan for a new environment and updating a few variables.

The moral of the story is that you don’t have to design complicated multi-phase processes with gates and approvals to achieve a sensible and secure state. Develop a process, diagramming it on a whiteboard. Find the step where automation gives you the biggest bang for your buck, and plan to automate it instead. Keep evaluating and planning for automation until it the effort to automate is no longer cost-effective. Distill the policies down to the simplest language that enforce the processes you’ve developed.

What are you experiences with policies and processes within your organization? Where have you had success with automation, and where did you experience problems? What sort of policies and processes work best in your environment? Please share in the comments section below.