Barebones Agile DevOps (BAD) – A Simplified Hybrid Agile DevOps Approach for Delivering IT Cloud Solutions

By John Conley III

CEO, Samsona Digital

A division of Samsona Corporation

“Transforming your world one task at a time”

Tips for implementing a hybrid approach to solution delivery

You can have the best DevOps engineering pipeline in the world, but if you don’t control the flow of work before requirements reach the sprint backlog, chaos can wreak havoc on the dev team’s sprint velocity.

Having been a consultant at several companies during the digital cloud revolution (and before this era), I can safely say that most companies do not implement a pure agile approach. That’s fine. Life happens. Not all companies implement a pure Agile approach to solution delivery. Even those who use SAFe still mix in some familiar, waterfall or even unconventional methods. No matter how they slice it, almost all companies who pursue an Agile DevOps approach to IT delivery mix in unorthodox, home grown, semi-waterfall methods. If your company is one of these, then this content is for you.

Your company most likely sent teams to one of the many agile devops trainings available online and in person. Everyone is pumped up, ready to do some sprint planning. But the corporate technology culture is not sharing that enthusiasm. The “corporate culture tech debt” still wants the old ways of deploying technology.

Pure Agile and DevOps practitioners such as myself have to acknowledge that, despite the fact the Agile Manifesto was written almost 20 years ago, most organizations still fall back, by instinct, to waterfall approaches. Why? It’s mostly due to pressure from market forces, executive deadlines, and budget constraints. Human nature drives teams to quickly capture requirements and turn out code with little time for thought. Approaches such as Scaled Agile and DevOps have tried to balance this knee jerk reaction with some lightweight guiding principles, but most organizations still mix in pressure-driven waterfall regardless. Mixing waterfall with agile devops is what we notoriously call “WAgile.” We have to live with this grassroots behavior whether we want to or not.

Rule of Thumb: The more that IT shops veer away from an ideal Agile approach, the more chaotic their IT workflow becomes. Team members generally want to work their butts off to deliver quality solutions on time and within budget if organizations manage the chaos better.

Tip: When you hire new tech workers, be sure to document and train them on the aspects of your technology culture that are not part of industry standard methodologies or CI CD platforms. Just because a cloud developer knows Azure DevOps (the technology), for instance, doesn’t mean they understand all of the crazy complicated DevOps scripts you created (the technology culture). This also includes training senior technology leaders on the kinds of presentation templates that C-level executives want to see for status reports. These executive reports can vary widely per organization – I’ve literally had to have executive admins help me understand what style works best for certain C-suite execs. Separate the technology culture from the technology itself when onboarding team members. These are two different animals. Failure to adequately plan the training of technical team members such as architects and developers on unique homegrown practices is an antipattern I’ve observed in some organizations, including some mature agile organizations.

Figure 1. The Backlog Pipeline (High Level)

The Team Backlog

At a minimum, each team role needs its own backlog. Thus, not only should there be a developer sprint backlog and a product backlog, but there should also be an enterprise solution arch backlog, a security team backlog, a cloud platform engineer backlog, and QA backlog. If you use pure SAFe to manage the flow of work in and out of these backlogs, great. If not, at least when you manage your “wagile” pipeline—wagile being a waterfall-agile hybrid, just have these backlogs in place, at a minimum, so that the work can be organized and prioritized at the beginning of each sprint.

Neglecting any of these backlogs will increase the chaos of the solution delivery cycle, which will impact sprint velocity, and may lead to team member burnout and turnover. For instance, upper management will throw random work on the member and expect deliveries within 24 hours, despite the fact that the team already has backed up work that was due yesterday. This is where “office politics” usually increases because of the resulting personality clashes, escalations, tensions, etc. The more organized your delivery workflow, the more satisfied the team members will be, and the better the expected outcomes. And we all know that top level executives mainly care about results, so efficient backlog management is the best way towards achieving measurable results and value per enterprise capability.

Rule of thumb: It’s seldom the people who cause project failures, it’s actually a flawed workflow process.

Tip: Improve the process, improve the chances of success.

The Barebones Agile DevOps (BAD) Workflow

I’m proposing what I call a Barebones Agile DevOps (BAD) workflow as a way to help organizations who tend to do a hybrid between waterfall and agile methods such as SAFe. This approach is just a lightweight process for hybrid organizations to streamline the flow of work from the Product Owner to the Architects and finally to the Implementers (Business Analysts/User Story Writers, Developers, Testers, Cloud Engineers, DBAs/Data Architects, Technical Writers). By no means is this approach intended to replace any existing methodologies. It is simply a slimmed down method that is realistic about the ways organizations execute hybrid waterfall, Agile and DevOps. Therefore, this content will not be long, exhaustive, or overly detailed. I’m simply presenting just enough info so that your project teams can rapidly adopt it, in order to reduce and control the chaos in the current IT workflow.

So how does it all work? Every organization has a set of capabilities that support a value proposition to external customers and internal workers. The workers execute certain tasks, which are tied to those capabilities, to deliver a product or service to the customer. Capabilities are often aligned with Epics in Agile, but based on my consulting experience, some organizations struggle in this area.

I won’t address those struggles here. But long story short, there are certain functional business requirements that need to be collected in a Product Backlog. They are usually realized as Epics and Features. Again, Epics are loosely aligned with organizational Capabilities. Conversely, epics sometimes evolve from large user stories, which is kind of a bottom-up approach. The alignment of bottom-up epics to capabilities is a challenge for organizations. The difficulty is in using non agile approaches to identify and map such capabilities to various lines of business functionalities, and finally to IT projects (NFRs, architecture runways, etc.

Figure 2. The 6 Backlog Verbs

The 6 Backlog Verbs

Nonetheless, the barebones approach is focused on 6 verbs to manage the backlogs and push items to the next backlog in the IT workflow pipeline. They are:

Identify

Rationalize

Prioritize

Assign

Deliver

Learn

To explain each, I’ll use the Product Backlog as an example. The other backlogs will follow a similar pattern.

Identify

As digital transformations of organizations sweep many industries these days, the activities of workers to provide value to customers is defined by requests from customers, executives and even workers to improve the various solutions in the enterprise. These requests are captured as functional requirements and queued up in the Product Backlog. Assuming you’ve created a Product Owner team in your DevOps tool of choice (I tend to use Azure DevOps), you can manage these incoming backlog items in a Kanban board, which is useful for rationalizing the items as a team.

Rationalize

At this step, the PO team leader, which would likely be a Product Manager, would organize a regular meeting to go over the new backlog items. Ideally, host this meeting every 1-2 weeks. The purpose is to determine if the new PBI is a new requirement, is similar to an existing PBI, or is not a true requirement but a request for product training or awareness. The goal is to filter out the backlog items so that you’re left with a viable set of functional requirements.

Prioritize

Once you’ve rationalized your PBIs, it’s time to prioritize them based on input from executives and the level of effort to document them. The PO can act on behalf of executives, with Product Managers sometimes weighing in on mission critical PBIs. The level of effort can be T shirt sized like this:

• Small effort
• Medium effort
• Large effort
• X Large effort

If an effort is deemed Large or X Large, there’s a possibility it may align to an enterprise Capability which can then become a basis for an Epic. The goal of this step is to have a better idea of which higher priority PBIs can be assigned to POs and/or Business Analysts. You’ll likely categorize the PBIs into Epics, Features, and User Stories.

Note: Sometimes, when a Solution or Enterprise Architect is involved in prioritizing PBIs, some NFRs (technical nonfunctional requirements) may come up which will be rationalized in the Architect Backlog later. Don’t rationalize and prioritize those in this step. Just label them so that they can be moved to the Architect Backlog.

Assign

Continuing from the last step, the team leader assigns each PBI to the team members based on their current workload capacity. A round robin approach is usually used here, based on my experience with several companies. An even distribution of PBIs is the goal. The level of effort should be factored in when assigning PBIs. An X large PBI might be the equivalent of 4 Small PBIs, for instance. Use a Kanban board to track progress in daily standups (or whatever meeting frequency you choose). For teams of Product Managers and Product Owners, the Product Manager would assign items to POs. Some IT shops also have Business Analysts who work with POs.

Tip: The preceding 4 steps can be done in one meeting or two if team member calendars are tight.

Note: In place of “Business Analysts,” some shops may have “user story writers” or other kinds of roles that work the PBIs into more detail.

Deliver

Finally, each team member works their PBI until it is ready to be delivered to the next backlog in the pipeline. In the case of the Product Backlog, the next one is the Architect Backlog. Since the focus is only on documenting functional requirements in each PBI, backlog items that are T shirt sized as Small shouldn’t take longer than a day, Medium probably 2 days, Large 3-5 days, and X Large up to 10 business days. If needed, break up X Large backlog items into smaller ones to do the work concurrently and deliver it faster.

Learn

At the conclusion of every backlog pipeline should be a meeting to discuss lessons learned. What went well? Where can the team improve? Do we have too many meetings or not enough meetings? Does each team member feel empowered and supported to perform their tasks? Do they feel they can be open and honest with leadership? Use this step as a way to calibrate the backlog workflow to improve the quality and the output of the backlog pipeline. A Scrum Master, Agile Coach or similar leader can facilitate the improvements needed to make this learning step effective.

And that’s it. The next backlog in the pipeline is the Architect Backlog. It follows the same 6 verbs, but from a technical and architecture governance standpoint.

Tip: Any methodology has to feel natural to a team. The waterfall approach, though not always favorable, feels natural to most teams because when the pressure is on, it feels like a natural response. Likewise, implementing an Agile DevSecOps approach should feel equally as natural and easy to learn in order for it to replace waterfall.

The Importance of the Architect Backlog

Interestingly, many organizations overlook the need for the critical Architect Backlog (AB) as they go straight from the Product Backlog (PB) to the Sprint Backlog (SB), incorporating architecture input as an afterthought, or just another task. Bridging the gap between these two backlogs is actually not trivial. Smaller organizations can get away with skipping the architect backlog and letting analysts and developers figure things out on their own (which is the nature of self-organizing teams).

For larger enterprises, this is not an option.

Developers are usually overworked and don’t have time to consider if a PBI is a repeat of something already done, whether it truly makes technical sense, whether regulatory or corporate policy requirements need to be applied, strategic backup and disaster recovery designs, etc. My experience shows that this backlog is just as important as the PB and Sprint backlog, making for a “Big Three” backlog strategy at a minimum.

Solution and Enterprise Architects have to rationalize each incoming PBI to make sure they are technically feasible. They make a best guesstimate as to the level of technical effort involved in doing the high level design. They leave the low level design guesstimates to the Sprint Planning process (basically, the beginning of the next backlog, the developer Sprint Backlog). The lead architect, usually an Enterprise Architect, will assign AB items to each Solution Architect on the team. What is delivered is usually a Solution Architecture Design (SAD) document. This document should not be more than 5 to 10 pages long and should come with a one page recommendations to guide developers as they take this input into the Sprint Backlog.

The Developer Sprint Backlog usually is setup pretty well, from my experience. No need to go into much detail here. With a smooth PB and Architect Backlog, the developers can focus on sizing the level of effort using a Story Points scoring system. Items they deem above average (usually higher than 5 on a scale of 1 to 10) would involve Solution Architects having to approve the code at a DevOps tollgate before it is pushed to the next step in the CI CD pipeline. Architects generally verify that best design practices are being used, and if not why not, also looking for regulatory compliance, corporate policy compliance, performance and other NFRs, etc.

The remaining roles of Testers, Cloud Engineers, Security, Data, have more streamlined Backlogs as they don’t need to be as involved once the technical rigor of the Architect and Developer backlogs have been performed. The main backlog verbs here (steps) are Rationalize, Assign, Deliver and Learn.

Conclusion

Hopefully this scaled down, barebones approach is easy enough to help hybrid environments make the most of their unique mix of waterfall and Agile DevOps. If you think your teams need an outside voice to help adopt this easy approach, feel free to reach out to me. Until the next video, stay agile.

Understanding the Relationship Between the Product Owner, Solution Architect and Cloud Development-Deployment Teams

By John Conley

Photo Credit: Mimi Thian from Unsplash.com

Note: This article is also available on Medium.com.

As organizations launch digital transformation projects using Agile and DevOps techniques, it’s easy for them to overlook the dynamic of the teams they assemble to make the transformation a reality. The transformation team needs a technical vision as well as a business vision. Without the clarity provided by the vision, the team can break down into a basket case of ineffective sprints and unproductive relationships. The Product Owner role helps drive that vision into a set of strategic initiatives that then are vetted and technically prioritized, with the help of the Solution Architect, into sprints. Of all the relationships on a given team, the most misunderstood perhaps is that of the Product Owner, the Solution Architect, and the developers and deployers who make the digital solution a reality.

One of the reasons for this misunderstanding is the underestimating of the complexity of a project. With smaller organizations, this may or may not be a big deal. But for larger corporations, especially those that are publicly traded, this is often inexcusable.

Within any organization, there are two figurative sandboxes that employees work in:

• The Business sandbox and
• The Technology sandbox

In the Business sandbox, employees carry out tasks that directly align with the organization’s marketing and business support operations. In the technology sandbox, employees are tasked with ensuring that the best and most updated technology is being used to support the Business sandbox. In a digital transformation, technology is no longer an afterthought, or “those IT nerds over there,” but an essential engine that drives the business forward. For every IT project, the leading voice of the business sandbox is the Product Owner (PO).

Photo: Nesa by Makers at Unsplash.com

For the Technology sandbox, the Solution Architect (SA) is the main voice. The Product Owner, as the voice of the customer, internal employee, and executive stakeholder, has the responsibility to manage a Product Backlog of requests for new features to existing solution, brand new solutions, and bug/defect requests. Since the Product Owner is generally nontechnical (which is usually a good thing), the Enterprise Architecture team assigns a Solution Architect to that PO. The SA and PO meet regularly (usually weekly or biweekly) to go over the Product Backlog (PB) to vet each Product Backlog Item (PBI).

Depending on how large the PB is, the meeting can be either quick or tedious. The latter is especially true for larger organizations who are introducing the concept of a PB for the first time. Vetting a the PBIs in a backlog is a multistep process:

• PO Guesstimate
• SA Guesstimate
• PBI Priority Ranking
• Executive Budget Approval
• EA Backlog

PO Guesstimate

The PO t-shirt sizes (e.g., Small, Medium, Large) the business importance/impact of each PBI based on business strategies and capabilities being addressed. No technical input involved. Sometimes, the PO will recognize that a new PBI was already implemented in part or in full by another solution already in production, or is just a matter of training. In this case, the PBI can be marked “Resolved” or “Closed.” There will be times when the PO needs further clarification on the business rationale for a PBI from executive stakeholders. These can be marked as “Pending further business review” or something similar. These items should be time tracked so that they don’t age too long in the product backlog.

SA Guesstimate

After gaining business insight into the PBI, the SA t-shirt sizes the effort to analyze and implement technologically. No business input involved. The most popular t-shirt sizes used are Small, Medium and Large. Some teams use X Large for really big epics, but that’s up to you what works. Sometimes, the SA will recognize that a new PBI was already implemented in part or in full by another solution already in production, or is just a matter of training. In this case, the PBI can be marked “Resolved” or “Closed.” There are also times the SA may need a second pair of technical eyes to vet a PBI, so these should be marked “Needing further technical review.”

PBI Priority Ranking

If a PBI survives 1 and 2, then the business and technical rankings together should guide the PO and SA to prioritize the PBI relative to the others. Ranking PBIs is mostly subjective, but a good rule of thumb is to give weight to those items that are quick wins or have been heavily requested among users and executive stakeholders for some time.

Executive Budget Approval

Once the PBIs are prioritized, the PO gets budgetary approval of the top 10 (or whatever the threshold is) from executive stakeholders. Once a PBI is approved, the PO marks the PBI as “Ready for Implementation” or whatever status is agreed upon.

Photo: Icons8 Team from Unsplash.com

EA Backlog

There are different schools of thought regarding how to the technology team proceeds after a PBI has been approved. The Scaled Agile Framework offers some good ideas around value trains and solutions trains that have helped many organizations. Plain old Scrum, Agile and DevOps have good ideas here. Choose the approach that works for you. Here, I am focusing mostly on the pure process itself without getting bogged down in methodology wars. Once a PBI is approved, some technology leaders directly assign them to a sprint backlog. That might work for smaller organizations, but for larger ones, there is a need for a technology “brain trust” to determine the next step before it goes to any particular team’s sprint backlog. This is where Enterprise Architecture (EA) comes into play.

Often, SAs are under the direction of an EA Brain Trust who assigns them to one or more IT projects. So it would make since that there needs to be an EA Backlog the mediates between the PO’s Product Backlog and each team’s Sprint Backlog. This becomes especially true for organizations going through a digital transformation that involves implementing cloud computing. In the cloud, there are three layers that a PBI can intersect with:

• Software (SaaS)
• Platform (PaaS)
• Infrastructure (IaaS)

Each of these tiers would have a corresponding technology team of technical development and deployment specialists to implement the incoming PBI within the organizations IT environment. With the typical sprint backlog setup, the default assumption is that the incoming PBI is focused on developing software. But what if it only involves standing up a SQL database or a Linux or Windows Server VM? The determination about what kind of team to engage for the solution needs to be at the EA level. This technology brain trust, the EA Team, should have leaders from each area to help engage the right team.

The PBI would then be routed to that respective team’s Sprint Backlog to work. The EA team would also do a more technical score of each incoming PBI to determine if an SA is needed to guide the sprint team. A good starting point for a scoring system is 1 to 50, where anything 26 or higher means the PBI is strategically important enough to warrant assigning an SA to the team. Less strategic PBIs can usually be handled by sprint teams with just a normal tech lead or dev lead. If an SA is needed, the SA would then work with the team to do normal sprint planning and backlog refinement activities. The SA creates necessary design docs that are within the guidelines of the master EA doc.

Considerations for Consulting Engagements

The process of initiating a consulting engagement for a prospective customer has its own set of steps not unlike a software development project. After a consulting sales pitch, there has to be a customer assessment phase to determine the level of effort both parties can expect before a full engagement commitment is made. Both roles, PO and SA, are needed at a minimum to determine the effort.

During the assessment, the PO and SA develop and manage a backlog of items the customer would need to address before committing to a digital transformation initiative. Depending on the size and complexity of the organization, this assessment can take anywhere from a week or two, to several weeks or longer. The less mature the organization is technologically, the longer this assessment phase will take. It’s similar to when you hire house cleaning service. If it’s your first time, and your house is big, it will take longer than a similar house that already has regular house cleaning.

The successful completion of the engagement setup has, at a minimum, the following micro sprints as key milestones:

• Request for info, quote, proposal from the prospective client
• Sales pitch by the consultant
• High level needs assessment by both the client and consultant
• Service Level Agreement (SLA)
• Product backlog creation and evolution based on user stories and other feature requirements

This set of milestones is by no means exhaustive but is meant to provide a set of guardrails to manage the micro sprints for the engagement setup process.

Wrapping Up

Hopefully this white paper helped in understanding the important relationship between a Product Owner and a Solution Architect. The former is business oriented and is the voice of the customer, employee user and executive stakeholder. The latter is technically oriented, and is the voice of the technology teams and the mediator between the functional requirements (usually user stories and other PBIs) and the nonfunctional requirements that the technology teams implement to satisfy the business. The EA team helps determine how to map incoming PBIs to the right technology teams, and will assign an SA if the PBI is architecturally and strategically significant.

About the Author

John Conley is a Dallas, TX, based Digital Solution Architect and Cloud Engineer Consultant. This is an excerpt from his forthcoming eBook “2020 Business Guide to Digital Transformation Governance.” Feel free to reach out to him if you have any questions or need consulting for your enterprise engagements.