Cloud FinOps, 2nd Edition

Chapter 1. What Is FinOps?

In the simplest terms, FinOps brings financial accountability to the variable spend model of cloud. But that description merely hints at the outcome. The cultural change of running in the cloud moves ownership of technology and financial decision making out to the edges of the organization, from procurement to engineering, architecture, and product teams. It empowers technology, finance, and business professionals to work together in more efficient ways.

Organizations around the world are moving aggressively to cloud, whether they are ready or not. Like the transition from mainframes to client/server, to the web, or to mobile, the shift to cloud (and from projects to products) brings with it massive changes to the way traditional business functions need to account for, request, and manage technology costs.

FinOps demands you acknowledge that the old ways of managing infrastructure aren’t just ineffective; they create situations where runaway cloud costs can threaten the business. While traditional infrastructure management methodologies will still be needed for owned infrastructure in the future, no level of adherence to these methodologies—which don’t handle the variable consumption model of cloud well—will allow organizations to manage cloud effectively without the introduction of FinOps.

Defining the Term “FinOps”

As of early 2023, the FinOps Foundation defines it as such:

FinOps is an evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology, and business teams to collaborate on data-driven spending decisions.

At its core, FinOps is a cultural practice. It’s the way for teams to manage their cloud costs, where everyone takes ownership of their cloud usage supported by a central best-practices group. Cross-functional teams in engineering, finance, product, procurement, etc. work together to enable faster product delivery, while at the same time gaining more financial control and predictability.

Tip

FinOps is a portmanteau of the words “Finance” and “DevOps,” stressing the communications and collaboration of business and engineering teams. Cloud FinOps is sometimes incorrectly referred to as “Cloud Financial Operations,” but that term is falling out of favor due to its ambiguity with the more traditional “Financial Operations” role that exists in finance. Other synonyms for the practice include “Cloud Financial Management,” “Cloud Financial Engineering,” “Cloud Cost Management,” or “Cloud Optimization.” Whatever you call it, FinOps is the practice of getting the most business value out of your cloud spend.

This chapter discusses the core principles of FinOps, how the associated cultural transformations began, and why every organization needs to embrace the discipline for cloud success.

But first, let’s set the stage for defining FinOps with a typical story of an individual practitioner’s journey.

The FinOps Hero’s Journey

Today’s FinOps leader often comes out of a world of managing, planning, and accounting for traditional IT and virtualized servers. Here’s a typical story that is an amalgamation of those we’ve heard over the years. It’s likely you are about to embark on a similar journey, are currently on it, or have completed parts of it already. The hero in this story is called Finn.

Things were pretty straightforward for Finn: backward-looking financial reports were done quarterly, and capacity planning meant extrapolating usage trends to guess the production needs of the organization for the next few quarters to meet changing demands for its products. There weren’t a lot of surprises in spending.

Then Finn noticed an increasing number of AWS or Google Cloud payables coming in without purchase orders attached. One of his cloud-savvy colleagues, Ana, explained the highly variable nature of cloud and how it’s just, well, different than on-premises data centers—and that there’s an entirely new way of managing it, as well as a new professional discipline emerging to do so.

Finn carefully considered Ana’s words. It did sound interesting, and appealing. But then Finn remembered how well the processes he had in place for the organization’s 8,000 on-premises servers work. Cloud couldn’t be that different. Surely if he just applied his existing processes more diligently, cloud costs could be managed as well.

The next quarter, cloud spending doubled unexpectedly, and Finn went back to Ana with his tail between his legs. He committed to trying a new set of processes that look more frequently at spend and increasing his interface time with the engineering teams creating the spend.

All of a sudden the finance leader, who previously never cared about cloud spending, began pushing Finn to go back to quarterly reporting, saying the real-time approach he was taking didn’t fit with the company’s other processes. The technology leader was pushing back, saying she couldn’t consider cost and also make her product delivery deadlines. Finn’s executive team encouraged a top-down control methodology. Finn again went back to Ana for help, and she led him to fellow journeyers at the FinOps Foundation. Learning from their mistakes and wins, Finn began to lay out a plan for how to reset the company’s processes and, more ambitiously, effect cultural change.

It was go time. Finn rolled out a new cloud spend allocation strategy, tagging guidelines, criteria for rightsizing (i.e., resizing cloud resources to better match workload requirements), and an initial rate optimization commitment to his cloud provider. There was finally a path forward that seemed to allow him to account for the spend and the teams to get the tech they needed without friction. Cloud migration began to soar.

Right when things seemed to be going well, a new CFO came in and said the cloud was too expensive at scale, advocating for a widespread return to the data center. It was time for Finn’s biggest test: working with his cloud-savvy colleagues to show that cloud is more than a cost center. They had to show how cloud can enable innovation and velocity in ways that on-premises data centers cannot, driving competitive advantage for the company. The CEO saw the bigger picture and agreed, paving the way for a cloud-first strategy.

Newly confident, Finn forged ahead, breaking down silos between teams and helping to drive real change in the organization. But he faced one last battle: cloud spend was now hitting material levels, affecting the bottom line. The CFO stepped in to stop the upward trend by any means necessary to ensure the organization’s margins were not affected. Finn moved beyond the one-dimensional view of looking only at cloud spend and shifted to a unit economics model that tied the spend back to business value, giving him the context to clearly demonstrate that cloud spend was on the right path.

In the end, Finn realized that this journey is just the beginning. Cloud is constantly evolving, and FinOps with it. Finn and Ana will have the most influence in this new field by helping to define its best practices, something they see as a key way to give back to the community that helped them on their own journeys.

Some aspects of this journey may resonate with you, and others may be part of your journey ahead. Now that you’ve heard a typical story, let’s look at where FinOps emerged.

Where Did FinOps Come From?

Trailblazers like Adobe and Intuit, early scalers in public cloud, provided the first glimpse to the coauthors of what would become FinOps as far back as 2012 in the Bay Area. In the mid-2010s, they interacted with larger companies like GE and Nike as they began their own early FinOps journey tied to their rapid expansion of cloud usage. A couple of years later, Mike saw forward-looking enterprises in Australia, like his own Atlassian as well as others like Qantas and Tabcorp, begin similar practices. Finally, during J.R.’s two-year tour of duty in London from 2017 to 2019, he was a firsthand witness to enterprises like BP, HSBC, and Sainsbury’s as they developed this new approach across their company cultures. FinOps came into being slowly and simultaneously, all over the world, as the financial and accountability challenges of cloud presented themselves at scale everywhere, as various industries and regions began to adopt material amounts of cloud. Figure 1-1 traces the increasing prevalence of the term.

Figure 1-1. Search requests for the term “FinOps” between 2018 and 2022 (source: speakeasystrategies.com)

“FinOps” is a term that has come late to the party. In the early days, companies simply called the practice “cloud cost management.” Later, “cloud cost optimization” began to take hold, although it didn’t speak to the allocation challenges of cloud. AWS and other cloud providers began using the phrase “cloud financial management,” a catch-all title that is increasingly being replaced by “FinOps” based on search and analyst mentions referenced in the figures. Others, like the retailer Target, call their internal practice “Engineering Efficiency.” Figure 1-2 shows increasing growth in “FinOps” mentions by industry analysts.

Figure 1-2. Analyst articles mentions of different names for FinOps (source: speakeasystrategies.com)

In Figure 1-2, you can see the recent increase in cloud analyst articles using the term “FinOps” to describe this field of practice, as older terms like “cloud financial management” reduce in mindshare. Choosing this compound term, which purposely echoes DevOps, brings the vital cross-functional and agile aspect of the movement to the forefront.

And now FinOps is becoming a standalone profession worldwide. Figure 1-3 shows the growth in FinOps being a skill listed by members of the LinkedIn platform. The 2022 State of FinOps data showed that nearly every major industry, from Financial Services to Retail to Manufacturing and beyond, is now running a FinOps practice.

Figure 1-3. Count of people on LinkedIn listing FinOps as a skill set

Note

State of FinOps data in 2021 indicated that 95% of those working in the field saw it as their likely career path.

Public job listings for FinOps practitioners at Fortune 500 enterprises ranging from Apple to Disney to CVS Health to Nike are popping up frequently on LinkedIn and the FinOps Foundation site.

The challenge today is less of finding a job in FinOps and more of companies finding enough skilled candidates to fill the roles they have, driving an explosion of those seeking FinOps certification and related skills.

Stories from the Cloud—J.R.

I first spoke about the concept of FinOps in a DevSecOps talk¹ with Emil Lerch from AWS at the AWS Public Sector Summit in Washington, DC, back in 2016. We started with the definition of DevOps from the venerable Gene Kim, author of The Phoenix Project:

The term “DevOps” typically refers to the emerging professional movement that advocates a collaborative working relationship between development and IT operations, resulting in the fast flow of planned work (i.e., high deploy rates) while simultaneously increasing the reliability, stability, resilience, and security of the production environment.

Then, with a bit of hubris, or as an homage, I crafted a definition of FinOps based on Kim’s:

The term “FinOps” typically refers to the emerging professional movement that advocates a collaborative working relationship between DevOps and finance, resulting in an iterative, data-driven management of infrastructure spending (i.e., lowering the unit economics of cloud) while simultaneously increasing the cost efficiency and, ultimately, the profitability of the cloud environment.

Since then, the definition of FinOps has evolved and broadened, but it has retained the all-important principles of driving a collaborative working relationship between teams, making iterative changes using data-driven insights, and improving unit economics.

Data-Driven Decision Making

With FinOps, each operational team (workload, service, product owner) can access the near-real-time data they need to influence their spend and help them make data-driven decisions that result in efficient cloud costs balanced against the speed/performance and quality/availability of services.

If you can’t out-experiment and beat your competitors in time to market and agility, you are sunk.... So the faster you can get those features to market and test them, the better off you’ll be. Incidentally, you also pay back the business faster for the use of capital, which means the business starts making money faster, too.

Gene Kim, The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win (IT Revolution Press, 2013)

If it seems like FinOps is about saving money, then think again. FinOps is about making money. Cloud spend can drive more revenue, signal customer base growth, enable more product and feature release velocity, or even help shut down a data center.

FinOps is all about removing blockers; empowering engineering teams to deliver better features, apps, and migrations faster; and enabling a cross-functional conversation about where to invest and when. Sometimes a business will decide to tighten the belt; sometimes it’ll decide to invest more. But now teams know why they’re making those decisions.

Real-Time Feedback (aka the “Prius Effect”)

There are three parts to a successful FinOps practice:

$${\displaystyle \mathrm{Real-time\; reporting}+\mathrm{just-in-time\; processes}+\mathrm{teams\; working\; together}=\mathrm{FinOps}}$$

We’ll get into the second two later in the book. Right now, let’s look at the first part.

The feedback loop of real-time reporting is a powerful influence on human behavior. In our experience, you should provide engineers with feedback on the impacts of their actions as close as possible to the time those actions occur. This tends to create automatic behavioral changes for the better.

Anyone who’s driven an electric car has probably experienced the Prius Effect. When you put your foot down heavily on the pedal, an electric car’s display shows energy flowing out of the battery into the engine. When you lift your foot up, energy flows back into the battery. The feedback loop is obvious and instantaneous. You can see how the choice you’re making in the moment—one that in the past may have been unconscious—is impacting the amount of energy you’re using.

Note

Without explicit recommendations or guidance to do so, the real-time feedback loop instantly influences driving behavior.

These visual cues typically create an immediate effect. You start to drive a bit more sensibly and step down a little less hard on the accelerator. You begin to realize you don’t need to accelerate quite so fast to get where you’re going. Or, if you’re running late, you decide that hitting the gas harder is worth the extra energy consumption. In either case, you can now make an informed decision to use the appropriate amount of energy to get where you need to go based on the environment in which you’re operating.

Ask yourself: how can we provide information that teams need to make a better decision?

Ron Cuirle, Senior Engineering Manager at Target²

This real-time data-driven decision enablement is what FinOps is all about. In the data center world, engineers take individual actions that can’t easily be traced to their financial impact on the company due to long-term hardware purchases and associated depreciation schedules.

Stories from the Cloud—J.R.

During a previous visit to one of the world’s largest cloud spenders, I learned that, despite nine figures a year of annual cloud spend, no one was sharing those costs with the engineers who were incurring them. When they did finally reveal those costs to the engineering team, it was typically 30–60 days later, during an aggregate cost center review. They had the raw data available, but because they hadn’t adopted a culture of FinOps, they hadn’t implemented a feedback loop directly to those making decisions about how much cloud to use.

Once they did, the results were immediate and dramatic. One of the first teams provided with this visibility discovered that they had been spending over $200,000 per month running development environments that they really didn’t need. With only three hours of engineering effort, they shut down those unnecessary resources and saved enough money to hire an additional team of engineers.

The accountability provided by this new visibility revealed something quite interesting. No one gave a specific instruction or recommendation to make a change. Once the engineering manager was able to see the cost of individual environments, she made an informed decision that the team could do without the extra dev environments. She hadn’t previously taken action because she didn’t have enough information to understand just how large the business impact of leaving the environment running ultimately was. This early FinOps process turned cost into another efficiency metric that engineering teams began to consider alongside other performance and reliability metrics. Engineers tend to be averse to inefficiency and once given the right data (and the leadership support discussed in Chapter 24) tend to optimize cost in the same ways they do other critical metrics.

FinOps is about helping the business make better decisions while moving more quickly. The source of this increased velocity is the frictionless conversations it encourages between teams.

What made those teams great is that everyone trusted one another. It can be a powerful thing when that magic dynamic exists.

Gene Kim, The Phoenix Project

Teams that previously spoke different languages and kept each other at arm’s length (e.g., engineering and finance) now build more collaborative relationships focused on what’s best for the business. This is FinOps in action. By setting best practices and defining a common lexicon on cloud spending as we’ll discuss in Chapter 4, businesses enable productive trade-off conversations to happen.

Core Principles of FinOps

As we have spoken with practitioners who are successfully introducing FinOps culture in their organizations, a common set of values or principles has emerged. Defining FinOps values and ensuring all the process, tooling, and people align to FinOps core principles will help lead you to success. FinOps teams that embrace these principles will be able to establish a self-governing, cost-conscious culture within their organizations that promotes both cost accountability and business agility to better manage and optimize costs while maintaining the velocity and innovation benefits of cloud. These FinOps values are:

• Teams need to collaborate.

  • Finance and technology teams work together in near real time as the cloud operates on a per-resource, per-second basis.

  • Teams work together to continuously improve for efficiency and innovation.

• Decisions are driven by the business value of cloud.

  • Unit economic and value-based metrics demonstrate business impact better than aggregate spend.

  • Make conscious trade-off decisions among cost, quality, and speed.

  • Think of cloud as a driver of innovation.

• Everyone takes ownership of their cloud usage.

  • Accountability of usage and cost is pushed to the edge, with engineers taking ownership of costs from architecture design to ongoing operations.

  • Individual feature and product teams are empowered to manage their own usage of cloud against their budget.

  • Decentralize the decision making around cost-effective architecture, resource usage, and optimization.

  • Technical teams must begin to consider cost as a new efficiency metric from the beginning of the software development lifecycle.

• FinOps reports should be accessible and timely.

  • Process and share cost data as soon as it becomes available.

  • Real-time visibility autonomously drives better cloud utilization.

  • Fast feedback loops result in more efficient behavior.

  • Consistent visibility into cloud spend is provided to all levels of the organization.

  • Create, monitor, and improve real-time financial forecasting and planning.

  • Trending and variance analysis helps explain why costs increased.

  • Internal team benchmarking drives best practices and celebrates wins.

  • Industry peer-level benchmarking assesses your company’s performance.

• A centralized team drives FinOps.

  • The central team encourages, evangelizes, and enables best practices in a shared accountability model, much like security, which has a central team yet everyone remains responsible for their portion.

  • Executive buy-in for FinOps and its practices and processes is required.

  • Rate, commitment, and discount optimization are centralized to take advantage of economies of scale.

  • Remove the need for engineers and operations teams to think about rate negotiations, allowing them to stay focused on usage optimization of their own environments.

• Take advantage of the variable cost model of the cloud.

  • The variable cost model of the cloud should be viewed as an opportunity to deliver more value, not as a risk.

  • Embrace just-in-time prediction, planning, and purchasing of capacity.

  • Agile iterative planning is preferred over static long-term plans.

  • Embrace proactive system design with continuous adjustments in cloud optimization over infrequent reactive cleanups.

When Should You Start FinOps?

Determining when to start FinOps has changed considerably since the first edition of this book. A few years ago, the practice typically began once a company had a spend panic moment where they inadvertently spent considerably more than they had budgeted. Adoption of the practice has now shifted considerably earlier in the cloud maturity lifecycle. It’s now common to see companies at the earliest stages of cloud adoption beginning the practice. This is driven partly by a better understanding of the challenges that cloud billing presents and also a large push by the big three cloud providers themselves to encourage their customers to get ahead of any cost problems early on. All three have also aggressively matured the cost tools and data provided to allow engineers and architects to consider cost as early as possible when engineering cloud-based solutions.

Unfortunately, many still think FinOps is about saving money or reducing consumption exclusively; therefore, those people tend to believe that the right time to implement FinOps should be measured by the amount of their cloud spend. To be fair, this does make sense on some level. For example, a massive cloud spender could immediately find a lot of potential savings. However, creating accountability for cloud spend and the ability to fully allocate costs are critical components of a high-functioning FinOps practice, which often don’t get implemented if you think about it from a retroactive money-saving perspective.

Further, the cultural and skill set changes that FinOps requires from both engineering teams and finance teams can take years to implement. The larger the organization and the more complex their environments, the more they will enjoy compounded benefits from starting the practice earlier.

A successful FinOps practice doesn’t require sizable cloud deployments or a multimillion-dollar cloud bill. Starting FinOps early will make it much easier for an organization to make informed decisions about cloud spend, even as its operations are just starting to scale. Therefore, it is essential to understand FinOps maturity; the correct approach to creating a FinOps practice is for organizations to start small and grow in scale, scope, and complexity as business value warrants maturing an activity.

As you’ll learn in Chapter 6, it’s not possible to install a fully formed FinOps practice from scratch, regardless of your past expertise. It takes time to transform the way engineers work and educate finance people about how cloud operates and to get executives onboard with the right reasons to use cloud. It’s a cultural change in ways of working, and the muscle of data-driven accountability and decision making must be built over time.

No one team does FinOps; the central FinOps team works to enable the many various teams around the organization that must work together to manage the business value of cloud. Further, teams across a large organization may be at very different places in their FinOps journey for a long time as each reaches tipping points where individual adoption makes sense.

Although this book can guide an organization to successful practices while helping teams avoid common pitfalls, no organization can proceed directly from zero FinOps to efficient FinOps. Every organization and team must implement FinOps processes incrementally, taking the time to learn from each other as they go.

Tip

Like DevOps before it, FinOps is a cultural shift, and the earlier it starts, the sooner an organization will benefit.

Our experience has taught us that you do FinOps from day one, but you engage more of the processes as you scale up. Over the years we have seen companies start adopting FinOps at one of two typical times:

• Formerly, the most common approach was to implement when things went off the rails. In this scenario, spending hits a point where an executive forces a hard stop on cloud growth and demands that a new managing model be implemented. Despite it being a common driver, this is not the ideal way to approach FinOps adoption. Innovation and migrations slow down—or even stop temporarily—during this executive fire drill.

• The wiser approach is taken by executives who have seen the cloud journey play out and align to the FinOps maturity model on FinOps.org. The FinOps practice needs to develop at a pace that matches the company’s position in the FinOps maturity cycle and the company’s maturing use of cloud overall. Initially, a single person might be assigned to manage commitments to cloud providers. They set out to implement an initial account, label, and tagging hierarchy. From there, as the practice gets larger, each part of the process can be scaled up. We now see most new practitioners joining the FinOps Foundation with this adoption story.

But no matter how a company arrives at the decision to implement FinOps, the first critical step is to begin providing visibility of cloud spend to relevant teams in a timely manner, so everyone can begin to understand what’s happening and determine how to address cost overruns before they become too large. In a mature practice, cost overruns are prevented via thoughtful architecture design before they even begin. While that level of visibility is achieved, the FinOps team starts educating the larger business stakeholders. As cross-functional teams work together, finance people will learn more of the language of cloud, while engineers begin to grasp relevant financial concepts, and businesspeople can make better decisions about where to invest resources.

The value of starting as early as possible on this cultural shift cannot be overstated, and the benefits of a FinOps practice can be felt and measured almost immediately.

Starting with the End in Mind: Data-Driven Decision Making

One of the most important concepts in FinOps is using unit economic metrics to enable data-driven decision making by engineering teams. The idea is to measure cloud spend against business output or value metrics and then take action based on the insights they provide. This is the real-time feedback loop, discussed earlier as the Prius Effect, in action.

Choosing the right business metrics to use for each part of the infrastructure is a complex process that’s covered in Chapter 26. For now, the main thing to remember is that providing unit economic metrics to enable data-driven decision making relies on almost every aspect of FinOps, including tagging, cost allocation, cost optimization, connectivity to other finance frameworks, and FinOps operations.

The business metric is important, because it allows you to change the conversation from one that is just about dollars spent to one about efficiency and the value of cloud spend. Being able to say, “It costs $X to serve customers who bring in $Y in revenue” brings a context that helps you make the decision whether $X and $Y are reasonable investments for the organization. Then, as products evolve or change entirely with new features, companies are able to measure the impact of these changes via these business metrics.

The result is being able to determine the difference between good cloud spend and bad—and trend those decisions over time. You should keep business value metrics in mind throughout the book and as you implement FinOps inside an organization.

The nirvana state we are building toward is for your organization to continuously make data-driven business decisions about the value you are getting from your cloud spend.

Stories from the Cloud—Jason Fuller

Jason Fuller, who runs Cloud at HERE Technologies, a multinational organization providing global mapping services, shared a story at a FinOps Foundation meeting that illustrates the point well:

We had a team way over budget using 9 billion lambda functions a month. I can’t influence that directly. But what I can do is sit with the team and understand the quality of the algorithm that you’re writing in lambda and determine if it can be tighter.

Do we need that millisecond accuracy you’re providing? Yes. OK. The business now understands it’s that valuable.

Now we can look at the pricing model for selling the service. We can make a business decision based on how valuable we think the offering is and how much we actually think we can get for it as a product on the market.

As a result, we don’t fight about infrastructure anymore—we have a conversation about its business value.

Conclusion

In this chapter we’ve defined FinOps and described the core principles and values that guide any organization starting the practice to ensure their success as they mature on the long journey of FinOps transformation.

To summarize:

• FinOps is a cultural change that drives collaboration between all teams inside an organization.

• Everyone has a part to play and should become cost-aware: from engineers to finance to procurement to executives.

• Real-time feedback loops on spending encourage continuous improvements to spend efficiency.

• Use unit economic and business value metrics to move beyond talking just about the cost of cloud to making data-driven decisions about cloud investments.

• Implement FinOps in your organization as early as possible and grow incrementally.

Now that you have a basic understanding of what FinOps is, let’s look at what the cloud enables inside organizations, and how you can avoid implementing processes that will hamper the good results of a successful FinOps practice.

¹ Emil Lerch and J.R. Storment, “Leveraging Cloud Transformation to Build a DevOps Culture,” AWS Public Sector Summit, June 20, 2016, https://oreil.ly/DCqfg.

² Ron Cuirle and Rachel Shinn, “Better Insight: Measuring the Cost of an Application on GCP,” Google Cloud Next ’19, April 9, 2019, YouTube video, 43:55, https://oreil.ly/FJ1SP.