Engineering

Why DevOps Often Fails in FinServ

By | | 8 min read


Summary
A chasm remains between IT experts who live and breathe DevOps, and FinServ firms that struggle to embrace digital transformation. But by adopting pragmatic solutions, FinServ can make DevOps work.

Financial services organisations are often late adopters of new technologies and methodologies. There are a number of reasons for this, some related to the way regulators and auditors operate in the industry, others to the difficulties of replacing old systems and practices, which often scale to such levels that it’s not always feasible to replace them without significant disruption.

Furthermore, banks and other FinServ companies are often some of the oldest and largest businesses around, making technological innovation much more challenging than it might be for smaller and newer companies.

Despite these hurdles, large FinServ firms have been investing heavily in digital transformation programmes over the last few years, as modernising the technology stack becomes a competitive necessity. These programmes are aimed at aligning the bank’s technology with recent industry standards, as well as looking for new ways to improve the use of said technology. They often involve the adoption of cloud services, improved automation, better scalability, usage of IaaS/PaaS, and adoption of DevOps methodologies.

A transformative journey is challenging in so many ways. This blog will take a closer look at the DevOps adoption hurdles facing FinServ organisations in their digital transformation.

DevOps Challenges

Why is it so difficult to introduce DevOps in large FinServ companies? I have already highlighted some of the reasons why digitalising large financial services can be challenging, but what specific difficulties do organisations face?

Processes and Structure

By the nature of their business, financial services are heavily process-driven, relying on bureaucracy and structure established over a long period of time. They are also heavily regulated and audited and as such, must carefully consider potential changes such as new methodologies.

One area that might limit the success of DevOps methodologies is the segregation between development and production services. This segregation is driven by role-based access control, which in many cases doesn’t allow software developers to access production environments. As a result, DevOps teams—comprised mostly of developers—can’t access the very software they’re expected to operate in production.

Segregation is caused not only by regulations, but also by historical structure across many FinServ companies, where management of the support groups and development teams is shared only at the CIO level, which means that connecting the Dev and Ops is not always feasible.

Another common theme that separates Dev and Ops is the company’s budget structure. Here’s where two acronyms come into play: Change the Bank (CTB) refers to development projects and budgets assigned for short delivery durations (usually annual projects), while Run the Bank (RTB) refers to operational budgets that get renewed annually.

RTB (or business-as-usual, BAU) support teams are usually handed over completed projects/products that are ready for operations. They will start familiarising themselves with the new products, whereas the CTB teams will most likely be dissolved or move on to develop new products. This approach can conflict with the expected continuity of DevOps methodologies, where a team shepherds products through their entire lifecycle.

Here’s another interesting angle about the way budgets are processed in financial services: budget approvals often require committed financial benefits, which means that newly funded projects are expected to drive future savings. This approach doesn’t always align well with the actual need for DevOps, which is mostly about enabling growth and continuation, and not always cost reduction.

ITIL vs. DevOps

ITIL is a set of detailed practices for IT service management (ITSM). Many articles have addressed the ITIL vs. DevOps debate—whether IT organisations should choose one over the other, or whether DevOps can work together with ITSM disciplines. I’m not taking sides on whether the two are friends or enemies; however, the way ITSM/ITIL is often implemented in FinServ might limit the benefits of DevOps.

One example is the frequency of releases, a core tenet of Agile and DevOps. It’s difficult to operate DevOps methodologies where there are frequent release freezes, or expectations that release cycles adhere to approval processes that may take weeks before a release can go ahead.

One FinServ organisation I worked with was aiming to drive their digital transformation programme as a standalone development entity with full autonomy. The teams were highly innovative and started developing products using cutting-edge technology and DevOps practices.

However, when the programme started looking into the practicalities of releasing products to production, the teams had to take several steps back and reconsider their technology decisions, some of which weren’t approved as part of the organisation’s technology stack. The teams were also required to prepare support handover documentation and follow existing change management procedures. Furthermore, they weren’t allowed access to production environments.

This example highlights the fact that even cutting-edge technology programmes must consider the existing technologies and methodologies used by the organisation, as they’re not likely to be allowed to ignore them.

Skipping the Agile Phase

Since FinServ firms are often late adopters of new technology, they have opportunities to skip, or leapfrog, incremental stages that other enterprises and industries have gone through. But while there are advantages to leapfrogging a few steps, these maturity phases are often necessary for successful implementation.

In DevOps, many organisations and industries have experienced long periods of introducing, fine-tuning and mastering agile development methodologies, which were later complemented by DevOps. While FinServ orgs are trying to move from structured, waterfall-based methodologies straight to DevOps, this transition introduces maturity risks that are not easy to overcome.

When speaking with colleagues in financial services on this topic, I often hear quotes like: “The terms ‘Agile’ and ‘DevOps’ are used as excuses for lack of planning.” I’ve also witnessed a few examples where projects were approved and funded with no clear target deliveries and dates—all on the basis that “This is Agile.”

How FinServ Can Make DevOps Work

Looking at the challenges described above, successful introduction and adoption of DevOps methodologies in FinServ sounds difficult. But being aware of these challenges—and carefully considering them as part of the digital transformation programme—can help prevent most known issues. Here are some suggestions and shared experiences.

Consider a hybrid approach: Don’t get caught up in a quest for DevOps purity, but instead focus on achieving successful agile development. Introduce operational methodologies that contribute to improved agility, mostly where role-based access control and segregation of duties are required by regulators.

Bring Dev and Ops closer: Focusing on DevOps can alienate production services teams, as their roles are theoretically under threat. However, the support of these teams is often critical to the success of new methodologies.

There are a few ways to achieve better relationships between existing support teams and new DevOps teams. Examples include:

  • Integrate support staff into the DevOps pods.

  • Introduce site reliability engineering (SRE) teams that, in addition to providing support services, also focus on DevOps autonomy and reduction of overhead. These teams can eventually integrate with production services.

  • Invest in skilled operations: For Ops teams to become agents of transformation, they need to have the right experience, exposure and skills. In addition to investing in experienced staff, you must provide existing staff with development opportunities.

Another way to introduce DevOps methodologies in FinServ is to identify independent business areas, projects and products that can be delivered and operated using DevOps practices.

There are a few examples in the industry where senior managers gave full backing to DevOps methods in certain projects or product lines. In these successful adoption examples, the DevOps teams managed to isolate their deliveries from the wider technology and process-related dependencies.

One caveat: Having such isolation during development, delivery and operations is rarely feasible in FinServ, so these examples are not common.

Investment in Tools

While the general use of technology in FinServ may lag behind that of other industries, budgeting for some of the best tech tools has never been a problem, and that includes DevOps tools.

The use of tools for development, deployment, testing automation, pipeline automation, analytics, etc., is a must-have for DevOps enablement. But it’s not only about the right tools, it’s also about the successful adoption and use of these tools.

A mature CI/CD process can encompass a significant part of ITSM requirements, and provide governance and confidence in the release process. And with good, well-adopted deployment tools, developers won’t need to access production servers during the release process. Successful adoption of the right monitoring and analytics tools will also provide the information that DevOps teams need to capture and troubleshoot potential faults—again, removing the need for access to production environments.

Summary

A significant gap remains between IT industry gurus who often live and breathe DevOps, and financial services enterprises which—despite their best efforts—face many hurdles when trying to introduce these methodologies. The good news is that FinServ can make DevOps work by embracing a pragmatic approach to DevOps, improving relationships between existing support teams and new DevOps teams, investing in the right tools for DevOps enablement, and other steps.

AppDynamics has been used by many of the top financial services organisations in their own digital transformation journeys. Learn more about how AppDynamics can help accelerate your own DevOps adoption.