Digital Transformation Tech Talk: Data Governance and Management

With AI technologies taking the digital revolution to the next level so quickly, it can be easy to bypass common sense best practices regarding sensitive data. Agile companies are in a race to take advantage of the AI part of the digital movement, which is why core principles such as data governance and data management might take a back seat. There’s nothing sexy about:

1. putting together a data governance framework

2. hosting a few sessions to hammer out a set of data management practices and operations based on that strategic framework, then

3. socializing it to the various IT cloud teams that handle data

Even though data governance and management may not seem exciting, it’s crucial to have policies and agreed-upon practices in place to handle complicated situations involving data integrity. One such scenario arises when there are conflicting versions of entity data after recovering it from a damaged backup database. When data is considered the master, it is assigned a special identifier called a gold ID. If this master record is replicated to other databases, it’s imperative to ensure that all copies remain consistent, even in backup databases. In the event that one of these replicated databases becomes corrupted, it becomes necessary to have a defined process for accurately and safely restoring that database.

So what does data governance look like? There are some templates online to get started. I put together this governance guiding document based on previous consulting work I did as part of a strike force of architects with financial and healthcare corporate clients. I also found an outline on the AHIMA website a good point of reference for structuring a governance document, which I used as an outline below. In subsequent articles, I’ll dive into more details and provide the basics for cloud data management principles and operations, especially when it comes to data I/O ingestion in a multitenant architecture.

Here’s a healthcare industry example of a core data governance document to lay the foundation for a framework in any industry:

The characteristics of data quality can be summarized as follows:

· Accuracy: Data should be error-free and correct.

· Accessibility: Measures should be in place to ensure data availability when needed.

· Comprehensiveness: Data should contain all the necessary elements.

· Consistency: Data should be reliable and consistent throughout the entire patient encounter.

· Currency: Data should be up to date and current.

· Definition: All data elements should have clear definitions.

· Granularity: Data should be at an appropriate level of detail.

· Precision: Data should be collected in its exact form with precision.

· Relevancy: Data should be relevant to its intended purpose of collection.

· Timeliness: Documentation should be entered promptly, updated regularly, and available within specified time frames.

Many healthcare organizations understand the need for data governance, but may not understand where to begin or how to establish a robust data governance program. One potential obstacle to implementing healthcare data governance at an organizational level is a lack of understanding among key stakeholders regarding data as an asset. This can lead to data silos and delays in developing a comprehensive program.

Healthcare data governance should encompass the entire organization, involving interdisciplinary teams comprising subject matter experts. The primary aim of healthcare data governance is to foster an organizational culture that prioritizes data security, reliability, and accessibility for authorized individuals. When the entire organization is engaged, a data governance culture is established, resulting in a strong program. To cultivate a healthcare data governance culture, it’s good to start with baby steps that demonstrate the value and benefits of data governance.

In order to implement a healthcare data governance plan or program, the first step is to define the concept of data governance and determining its scope. This entails establishing a foundational framework for collecting, retaining, utilizing, accessing, and sharing healthcare data. This framework encompasses various aspects such as policies, procedures, standards, ownership, decision rights, roles and responsibilities, and accountability associated with the data.

To effectively establish healthcare data governance plans or programs, organizations should form a dedicated team called the Data Governance Management Team (or a similar name). This team should include key individuals such as the Chief Data Officer (or an equivalent position) working in collaboration with the Chief Medical Information Officer. Together, they will spearhead the efforts to develop and implement robust healthcare data governance initiatives.

If interested in more healthcare data governance, check out the PDF file on the AHIMA website here.

DATA RETENTION POLICY

A compliant data retention policy defines the duration for which data must be retained to meet regulatory and/or organizational requirements. On top of that, it specifies the actions to be taken once the retention period has expired. Organizations may choose to delete (or destroy) or archive data based on their retention guidelines.

These policies and procedures, or standard operating procedures (SOPs), provide a structured framework for efficient Data Governance, guaranteeing data integrity, appropriate access, privacy compliance, secure data sharing, and quality data retention.

Conclusion

To restate, in the fast-paced digital landscape IT finds itself today, driven by AI technologies like ChatGPT, it is crucial for technical organizations to not overlook fundamental best practices concerning sensitive data. While agile companies strive to harness the power of AI within the digital revolution, core principles such as data governance and data management can sometimes be neglected. Establishing a robust data governance framework and defining data management practices and operations based on a strategic framework may not seem glamorous. However, it is a necessary step for organizations to take. This involves conducting sessions to develop a comprehensive data governance framework and subsequently sharing and integrating it with the various IT cloud teams responsible for data handling.

Digital Agile Strategies: When the Sales, Product and Sprint Teams Work in Tandem

The way a company makes money is by selling products and services to the public. Without sales, no revenue comes in. Revenue, and especially profits, are the lifeblood of an organization. Cut that off, and the company dies. Therefore, the sales staff must constantly sell, sell, sell at all costs to keep the company alive. It’s really that cut and dry.

Or is it?

Side note: This article also appears on Medium.com

True, the sales team is like the priesthood of capitalism that keeps the company’s future alive by sacrificing itself regularly. They should be applauded for this effort. But there is a limit on how much capacity the product team has to support a highly aggressive sales team. If you’ve been in technology long enough, even beyond IT, you know the delicate balancing act between aggressive sales on one hand, vs aggressive timelines to push out quality products based on features in the product backlog and tasks in the sprint backlog.

Product teams are caught in the pinch between the successful sales team, and the quality driven sprint team. If the sales team is too aggressive, product quality may suffer. If, on the other hand, sprint teams are behind schedule, the product team risks losing a sale that cuts the sales team commissions. Both of these scenarios can result in reputational damage as potential customers can get frustrated, terminate any sales agreements, and buy from a competitor.

Balancing the motives and goals of the sales, product and sprint teams is never an easy exercise as business forces can change dynamics rapidly. However, despite the temptation to revert back to ego driven knee jerk reactions to business events, it is crucial for

1. The sales team to be in alignment with the agile product team to ensure that the product quality meets customer expectations
2. The product team to be in alignment with the agile sprint team for the same reason

Aligning Sales and Product Teams

The following graphic highlights some strategies the sales and product teams can implement to ensure that product quality meets customer expectations:

Collaborate on product requirements

Working closely together at the sales and product team level improves the chances that customer requirements are well understood and documented. The sales team can provide valuable insights into the customers’ pain points, preferences, and expectations, which in turn can be used to prioritize quarterly product features and enhancements.

Attend sprint demos

The sales team should attend sprint demos to witness firsthand the progress made by the product team. They can also provide real-world feedback on how well the product is aligning with customer needs. This will allow the sales team to have some visibility into the product development process, as well as help identify any gaps or areas that need improvement.

Provide customer feedback

The sales team should be the primary source of customer feedback and possibly user story input for the product team. They should provide surveys or plain text feedback on customer satisfaction, feature requests (hopefully in the form of user stories), and pain points to help the product team adjust priorities and make product improvements.

Attend retrospectives

Some product teams may hesitate to invite the sales team to retrospectives, but there is some value at times in doing so. The sales team, once it understands the nature and importance of this exercise, should attend retrospectives to provide feedback on the product development process and collaborate with the product team to identify areas for improvement. This will help the product team to continuously improve the quality of the product and ensure it meets customer expectations.

Develop a shared understanding of quality

The sales team and product team should develop a common understanding of what constitutes high-quality products. This shared understanding should include customer expectations, usability, and performance metrics, among others.

In summary, by collaborating on product requirements, attending sprint demos and retrospectives, providing customer feedback, and developing a shared understanding of quality, the sales team can align with the agile product team to ensure that product quality meets customer expectations.

Aligning Product and Sprint Teams

Continuing, the chain of collaboration between the sales and product teams extend to the collaboration between the product and sprint teams. To maintain product quality, the product team must be in alignment with the agile sprint team continuously. A few ways the product team can align with the agile sprint team to achieve continuity include:

Collaborate on sprint goals

The product team should work closely with the sprint team to define sprint goals that are aligned with the product roadmap and customer needs. By setting clear goals, both teams can ensure that the work being done during the sprint is focused and contributes to maintaining product quality.

Define acceptance criteria

The product team should define acceptance criteria for each user story or feature being developed during the sprint. The acceptance criteria should outline the conditions that must be met before a user story or feature is considered complete and of acceptable quality. This ensures that everyone on the sprint team understands what is required to deliver a quality product.

Conduct high level code reviews

The product team should participate in code reviews, with the guidance of the solution architect assigned to the product backlog or at least the most senior technical leader of the sprint team. This approach is a best practice, helping to make sure the code being developed during the sprint meets the product team’s quality standards. This can include reviewing code for readability, maintainability, and adherence to coding standards.

Provide effective feedback and business guidance

The product team should provide feedback and guidance to the sprint team throughout the sprint. This can include reviewing and providing feedback on user stories, features, and design decisions, as well as providing guidance on how to address any quality issues that arise during development.

Collaborate in sprint reviews and retrospectives

The product team should participate in sprint reviews and retrospectives to provide feedback on the quality of the product and the development process. This will help identify areas for improvement and enable the team to continuously improve the quality of the product.

By collaborating on sprint goals, defining acceptance criteria, conducting code reviews, providing feedback and guidance, and participating in sprint reviews and retrospectives, the product team can align with the agile sprint team to maintain product quality. As a result, the emphasis on optimal quality will ensure that the product meets customer needs and expectations while adhering to high quality standards.

How agile sprint planning and sprint retrospectives can improve crucial conversations

Agile sprint planning and sprint retrospectives can improve crucial conversations between C-level executives, product managers, and sprint teams in several ways:

1. Encourages collaboration
2. Creates transparency
3. Facilitates alignment
4. Encourages continuous improvement

Collaboration

Agile sprint planning and sprint retrospectives are collaborative processes that involve input from all stakeholders, including C-level executives, product managers, and sprint teams. This might be surprising for some agile practitioners, but sprint planning and retrospectives are actually important business process in the digital era. In working together, they can gain a deeper understanding of each other’s perspectives, identify potential roadblocks, and find solutions to address them.

Transparency

Agile sprint planning and sprint retrospectives together promote transparency by providing regular updates on the progress of the project. Providing continuous updates can help C-level executives and product managers gain insight into the sprint development process, and identify areas where they can provide support or resources. The basis for C-level support of the team is what can be the basis of what’s called “technical empathy.”

Alignment

Agile sprint planning and sprint retrospectives help align the goals and priorities of all stakeholders involved in the project. By having a shared understanding of the project’s objectives and progress, C-level executives, product managers, and sprint teams can work together more effectively towards a common goal.

Continuous Improvement

Agile sprint planning and sprint retrospectives provide opportunities for continuous improvement. During the sprint retrospective, the sprint team reflects on what worked well, followed by what didn’t. The team then identifies areas for improvement. This crucial feedback can help C-level executives and product managers understand the team’s needs, as well as identify ways to support the team in achieving their goals.

Conclusion

The product team finds themselves caught between the competing interests of the successful sales team and the quality-driven sprint team. If the sales team is too aggressive, it could lead to a decrease in product quality. Conversely, if the sprint teams fall behind schedule, the product team risks losing a sale, which in turn could reduce the commissions of the sales team. In both scenarios, the company’s reputation may suffer as frustrated customers could terminate sales agreements and turn to competitors.

Maintaining a balance between the objectives and goals of the sales, product, and sprint teams can be a challenging task as market forces can change rapidly. Despite the temptation to react impulsively to business events, it is crucial for:

1. The sales team to work in concert with the agile product team to ensure that the product quality meets customer expectations
2. The product team to collaborate with the agile sprint team to achieve the same goal.

Generally speaking, both agile sprint planning and sprint retrospectives create a collaborative and transparent environment that fosters (1) open communication and (2) continuous improvement. Avoiding breaks in collaboration keeps the sprint team alert and ready to respond to business events. By encouraging crucial conversations between C-level executives, product managers, and sprint teams, an organization can help ensure that each member of each team is working towards the same goals and priorities, which is not exactly an easy proposition in an ever changing digital world. At the end of the day, the project is has to be successful in meeting customer needs and business objectives for the given product.

Understanding the Relationship Between the Product Owner, Solution Architect and Cloud Development-Deployment Teams

By John Conley

Photo Credit: Mimi Thian from Unsplash.com

Note: This article is also available on Medium.com.

As organizations launch digital transformation projects using Agile and DevOps techniques, it’s easy for them to overlook the dynamic of the teams they assemble to make the transformation a reality. The transformation team needs a technical vision as well as a business vision. Without the clarity provided by the vision, the team can break down into a basket case of ineffective sprints and unproductive relationships. The Product Owner role helps drive that vision into a set of strategic initiatives that then are vetted and technically prioritized, with the help of the Solution Architect, into sprints. Of all the relationships on a given team, the most misunderstood perhaps is that of the Product Owner, the Solution Architect, and the developers and deployers who make the digital solution a reality.

One of the reasons for this misunderstanding is the underestimating of the complexity of a project. With smaller organizations, this may or may not be a big deal. But for larger corporations, especially those that are publicly traded, this is often inexcusable.

Within any organization, there are two figurative sandboxes that employees work in:

• The Business sandbox and
• The Technology sandbox

In the Business sandbox, employees carry out tasks that directly align with the organization’s marketing and business support operations. In the technology sandbox, employees are tasked with ensuring that the best and most updated technology is being used to support the Business sandbox. In a digital transformation, technology is no longer an afterthought, or “those IT nerds over there,” but an essential engine that drives the business forward. For every IT project, the leading voice of the business sandbox is the Product Owner (PO).

Photo: Nesa by Makers at Unsplash.com

For the Technology sandbox, the Solution Architect (SA) is the main voice. The Product Owner, as the voice of the customer, internal employee, and executive stakeholder, has the responsibility to manage a Product Backlog of requests for new features to existing solution, brand new solutions, and bug/defect requests. Since the Product Owner is generally nontechnical (which is usually a good thing), the Enterprise Architecture team assigns a Solution Architect to that PO. The SA and PO meet regularly (usually weekly or biweekly) to go over the Product Backlog (PB) to vet each Product Backlog Item (PBI).

Depending on how large the PB is, the meeting can be either quick or tedious. The latter is especially true for larger organizations who are introducing the concept of a PB for the first time. Vetting a the PBIs in a backlog is a multistep process:

• PO Guesstimate
• SA Guesstimate
• PBI Priority Ranking
• Executive Budget Approval
• EA Backlog

PO Guesstimate

The PO t-shirt sizes (e.g., Small, Medium, Large) the business importance/impact of each PBI based on business strategies and capabilities being addressed. No technical input involved. Sometimes, the PO will recognize that a new PBI was already implemented in part or in full by another solution already in production, or is just a matter of training. In this case, the PBI can be marked “Resolved” or “Closed.” There will be times when the PO needs further clarification on the business rationale for a PBI from executive stakeholders. These can be marked as “Pending further business review” or something similar. These items should be time tracked so that they don’t age too long in the product backlog.

SA Guesstimate

After gaining business insight into the PBI, the SA t-shirt sizes the effort to analyze and implement technologically. No business input involved. The most popular t-shirt sizes used are Small, Medium and Large. Some teams use X Large for really big epics, but that’s up to you what works. Sometimes, the SA will recognize that a new PBI was already implemented in part or in full by another solution already in production, or is just a matter of training. In this case, the PBI can be marked “Resolved” or “Closed.” There are also times the SA may need a second pair of technical eyes to vet a PBI, so these should be marked “Needing further technical review.”

PBI Priority Ranking

If a PBI survives 1 and 2, then the business and technical rankings together should guide the PO and SA to prioritize the PBI relative to the others. Ranking PBIs is mostly subjective, but a good rule of thumb is to give weight to those items that are quick wins or have been heavily requested among users and executive stakeholders for some time.

Executive Budget Approval

Once the PBIs are prioritized, the PO gets budgetary approval of the top 10 (or whatever the threshold is) from executive stakeholders. Once a PBI is approved, the PO marks the PBI as “Ready for Implementation” or whatever status is agreed upon.

Photo: Icons8 Team from Unsplash.com

EA Backlog

There are different schools of thought regarding how to the technology team proceeds after a PBI has been approved. The Scaled Agile Framework offers some good ideas around value trains and solutions trains that have helped many organizations. Plain old Scrum, Agile and DevOps have good ideas here. Choose the approach that works for you. Here, I am focusing mostly on the pure process itself without getting bogged down in methodology wars. Once a PBI is approved, some technology leaders directly assign them to a sprint backlog. That might work for smaller organizations, but for larger ones, there is a need for a technology “brain trust” to determine the next step before it goes to any particular team’s sprint backlog. This is where Enterprise Architecture (EA) comes into play.

Often, SAs are under the direction of an EA Brain Trust who assigns them to one or more IT projects. So it would make since that there needs to be an EA Backlog the mediates between the PO’s Product Backlog and each team’s Sprint Backlog. This becomes especially true for organizations going through a digital transformation that involves implementing cloud computing. In the cloud, there are three layers that a PBI can intersect with:

• Software (SaaS)
• Platform (PaaS)
• Infrastructure (IaaS)

Each of these tiers would have a corresponding technology team of technical development and deployment specialists to implement the incoming PBI within the organizations IT environment. With the typical sprint backlog setup, the default assumption is that the incoming PBI is focused on developing software. But what if it only involves standing up a SQL database or a Linux or Windows Server VM? The determination about what kind of team to engage for the solution needs to be at the EA level. This technology brain trust, the EA Team, should have leaders from each area to help engage the right team.

The PBI would then be routed to that respective team’s Sprint Backlog to work. The EA team would also do a more technical score of each incoming PBI to determine if an SA is needed to guide the sprint team. A good starting point for a scoring system is 1 to 50, where anything 26 or higher means the PBI is strategically important enough to warrant assigning an SA to the team. Less strategic PBIs can usually be handled by sprint teams with just a normal tech lead or dev lead. If an SA is needed, the SA would then work with the team to do normal sprint planning and backlog refinement activities. The SA creates necessary design docs that are within the guidelines of the master EA doc.

Considerations for Consulting Engagements

The process of initiating a consulting engagement for a prospective customer has its own set of steps not unlike a software development project. After a consulting sales pitch, there has to be a customer assessment phase to determine the level of effort both parties can expect before a full engagement commitment is made. Both roles, PO and SA, are needed at a minimum to determine the effort.

During the assessment, the PO and SA develop and manage a backlog of items the customer would need to address before committing to a digital transformation initiative. Depending on the size and complexity of the organization, this assessment can take anywhere from a week or two, to several weeks or longer. The less mature the organization is technologically, the longer this assessment phase will take. It’s similar to when you hire house cleaning service. If it’s your first time, and your house is big, it will take longer than a similar house that already has regular house cleaning.

The successful completion of the engagement setup has, at a minimum, the following micro sprints as key milestones:

• Request for info, quote, proposal from the prospective client
• Sales pitch by the consultant
• High level needs assessment by both the client and consultant
• Service Level Agreement (SLA)
• Product backlog creation and evolution based on user stories and other feature requirements

This set of milestones is by no means exhaustive but is meant to provide a set of guardrails to manage the micro sprints for the engagement setup process.

Wrapping Up

Hopefully this white paper helped in understanding the important relationship between a Product Owner and a Solution Architect. The former is business oriented and is the voice of the customer, employee user and executive stakeholder. The latter is technically oriented, and is the voice of the technology teams and the mediator between the functional requirements (usually user stories and other PBIs) and the nonfunctional requirements that the technology teams implement to satisfy the business. The EA team helps determine how to map incoming PBIs to the right technology teams, and will assign an SA if the PBI is architecturally and strategically significant.

About the Author

John Conley is a Dallas, TX, based Digital Solution Architect and Cloud Engineer Consultant. This is an excerpt from his forthcoming eBook “2020 Business Guide to Digital Transformation Governance.” Feel free to reach out to him if you have any questions or need consulting for your enterprise engagements.