Agility for Regulated Industries

Agile brings a lot of benefits to the table over traditional project management. It encourages working in small batches so that products can be delivered earlier, and more often. It promotes connecting the business and developers more often so that they can agree to the same level of understanding. Agile also promotes a “just-in-time” way of requirements collection, and the level to which things are detailed is also aligned with working in small batches. The focus of Agile is based on iteratively delivering high value, with the appropriate amount of effort and overhead, while retaining high quality and responsiveness.

Looking Back

The idea of linear sequential design, such as traditional project management, has been around since the 50s. Construction companies used it to build structures one floor at a time, which makes a lot of sense because you can’t build the third floor before the first and second floors.

Later in the 70s, a man named Winston Royce was the first to describe the approach of linear sequential design for software development. Interestingly though, Royce describes the approach of linear sequential design to be “flawed” for software development. Regardless, later in the 80s, the government chose to use the approach to build computer systems for the military.

Following suit for decades, most regulated industries such as finance and insurance, have employed linear sequential design for their software development. Regulators and the rules by which they govern have also been largely created around a linear sequential process. Bodies such as the SEC, Federal Reserve, Health and Human Services, not to mention the obligatory Sarbanes-Oxley have long based their auditing and compliance on this model.

In most cases, these regulatory entities focus on ensuring companies follow the rules that protect consumers, employees, stockholders, and the company itself. It’s natural to see a slowing effect on throughput when more process and oversight are mandated, and yet we can’t completely remove these things. A minimum level of process and oversight are required to refrain from complete chaos. This leads us to an important question.

Can regulated companies still operate in ways that make them competitive in today’s fast-paced markets?

The fortunate thing for companies in regulated industries is, all companies that fall into their space are regulated. Therefore, the rules that impact one company, impacts them all to the same degree.

But should regulation stop you from operating with agility? The controls put in place by governing bodies should not dictate how you arrive at compliance; instead, they should only focus on the outcome of being compliant. Somehow though, I feel like the compliance rules are so heavily based on linear sequential process, that agility has been elusive for most regulated industries. There is certainly a need to update regulatory rules to better handle how the world works today. Companies that are regulated are typically bogged down by the regulation itself, and usually, the result is increased cost, with little innovation. These regulatory bodies are slow to change and lumbering government entities. I’m afraid getting impactful change will take some time, and Agile enthusiasts will need to continue fighting the good fight for change in these industries.

But momentum is in favor of agility. More and more companies are moving towards iterative and incremental development methods within regulated industries. If you look on job boards, typically most Scrum Master and Agile Coach jobs are in the banking industry. Even the government has started changing the way they spin up projects. They are moving towards iterative and incremental, and away from linear sequential design because they realized that predicting projects will go exactly as planned, and end on time and on budget, is like looking into a crystal ball to predict the future. Plus, software is malleable, and not ridge like constructing a building. Agile is slowly becoming the standard for developing software, even when regulation is mandated.

Moving Forward

Agile is the culmination of many practitioners (17 officially) that individually created their own way of doing things differently after realizing the shortcomings of linear sequential software projects. They got together back in 2001 and shared their experiences from their own methods, frameworks, and processes and agreed on a manifesto to guide others on how to build better software.

You’re probably most familiar with Agile in the form of Scrum, as 90% of Agile teams say they are using Scrum to employ agility within their business according to the 2016 State of Agile report by Scrum Alliance. Implementing any form of Agile in regulated industries is not without its inherent challenges. I’ve laid out some challenges and solutions that might help you get started below.

Potential Challenges/Solutions:

Where should we start? Which teams/ applications should we start with?

• Since regulated companies are often very large, find projects that are favorable to iterative and incremental patterns of work. Preferably start with small projects or projects that have been done before. Something where there is little inherent risk involved

• Work in small batches, and be deliberate about it. Smaller projects are more successful. If your project can’t be broken smaller, maybe say due to tight integrations, then at the very least don’t ignore the integrations between Agile development teams and the non-Agile teams they integrate with. This will be important until all teams are Agile.

• Pilot Agile teams, fail small, learn and improve quickly. Then scale to other teams.

• Build Agile Teams from people that want to work in an Agile way, don’t force anyone into it. Time and success will work things out.

• If you can, separate Agile Teams from the existing process, focus instead on the outcome that is required and keep them removed from the bureaucracy and red tape that weigh down other teams. Allow them to be compliant in an Agile way.

We have hierarchical teams with separated duties (i.e., BA team, Dev team, QA team, Deployment team)

• To move quickly, you need to reduce dependencies. Put every role needed to complete work to a functional state on the same team. Think vertical teams instead of horizontal ones. “Team Membership” can be informal if needed so that people don’t have to change management structure.

• Work towards automated deployments. Removing human error from promotions by having “push of a button” or fully automated deployments should make your compliance department very happy.

We regularly have hard deadlines that are mandated.

• You can still work in small batches. Deploy and store functioning increments to an environment just shy of production so that you can get regular timely feedback. Then deploy fully to production based on your mandated deadline… Get started sooner… Get feedback faster… Reduce risk

To be compliant, we are required to document everything extensively for traceability.

• Documentation comes at various times within a project. From requirements to testing, development decisions, and deployment. With Agile the work is completed incrementally; therefore it makes sense to also incrementally document. No need to waste time on collecting all those requirements upfront as requirements are guaranteed to change with time.

• Fundamentally the same documentation that is involved in lengthy BRDs still exist in Agile. It’s just spread out and less wasteful. Requirements are documented via the backlog, testing is preferably automated and recorded as part of a testing tool, development decisions are documented alongside the small batched work requirements, and deployments are automated and detailed via that tool. We’re still checking all the same boxes here folks.

We do manual QA testing/deployments, and that process takes a considerable amount of time.

• Manual testing does take a while. You can’t apply manual testing to Agile and expect great outcomes. Teaching and training current QA workers to manage the automation of QA test will benefit your overall process and help you speed up delivery.

• Automating deployments by focusing on continuous integration and continuous delivery will help to remove human mistakes, and keep the Change Management group happy. Plus it will further speed things up and deploying will become trivial.

How do we implement the Product Owner role since that role doesn’t currently exist?

• Product Owners must be able to understand the needs of the business as it pertains to their products. This doesn’t mean they must be experts, just means they must be good at soliciting information, making tradeoffs, and understanding the outcomes of their decisions.

• Product Owners need to be available to the team when they are needed. Don’t pick a Product Owner that is already overworked in their current role, and preferably dedicate a person to the Product Owner role full time.

• Recognize that Product Owners are not Program Managers, Product Managers, nor Project Managers. The role is a unique skill set that works both with the business, the development team, and the product itself. Some of the people in these traditional roles might fit the PO role, but they are not the same roles.

Our auditors will never go for this way of doing things.

• Educate internal compliance auditors to know the difference between the process to arrive at the outcome, and the outcome itself. They should only be concerned with the latter. They should understand the difference between actions that embrace agility, and the current standard company or industry practices which might not be aligned with Agile.

The unfortunate part about the constraints that come from regulations is that  people usually apply more process/red tape/politics (read as “control”) when regulations are involved, not less. The challenge is not just to implement Agile practices; if that is all you do, you will most certainly struggle. The real challenge is to change the thoughts and behavior of those who serve in roles that have the greatest impact on the current way of doing things. Start with identifying their current pain points, and hopefully, that conversation leads to further conversations about behaving with more Agility.

To learn more about Agile project management practices, contact a member of CCG at 813.265.3239 or info@ccganalytics.com.