Continuous Delivery in Java

Chapter 4. Deployment Platforms, Infrastructure, and Continuous Delivery of Java Apps

In this chapter, you will explore the various deployment options available to you for continuously delivering web-based Java applications. Along the way, you will learn about each platform’s components, its various strengths and weaknesses, and the core areas of focus in relation to Java and continuous delivery. Upon completion of this chapter, you should have the knowledge and tools to choose a platform for your next project, and be aware of the best practices and pitfalls.

Functionality Provided by a Platform

In the context of modern web-based Java applications, or distributed Java applications (based on the microservices architectural style), a platform provides the following functionality:

• A physical (or virtual) location to host your applications from, which is accessible to your users

• A Java runtime

• Persistent storage—either a block store or database

• Access to (or the ability to install) required middleware, such as ESBs or message queues

• Self-healing or resilience—automated restart of failed applications

• Service discovery—a mechanism to locate your additional application services and third-party services

• Fundamental security—hardened machine images, restricted ports, and strong authentication required to access any control plane

• A clear mechanism to understand costs incurred from operation

The emergence of the cloud, containers, and DevOps has been empowering for software developers. You can now create and manage custom platforms at a scale that was only dreamed of 10 years ago. However, with this power comes great responsibility. It is all too tempting for Java development teams to build their own custom platform. Not only does this take time and resources away from coding the application that was going to deliver the actual functionality required by the business, but often these platforms are poorly designed, fragile, and cost-ineffective.

The Dangers of Building Your Own Platform

Sure, it may initially be a lot of fun assembling your own platform. In addition, teams often (incorrectly) assume that their project is somehow “special” and needs a custom platform. But before starting to build your own custom platform, always ask yourself the following:

• Have you clearly identified that the most important and urgent problem for your team to be working on is building a bespoke platform to deploy software?

• Have you discussed the impact of the decision to build a custom platform with core stakeholders?

• Do you (and your team) have the appropriate knowledge and skillset to build an effective and maintainable platform?

• Have you assessed existing (open source and commercial) platforms and determined that there is no clear long-term cost/benefit trade-off in using or buying an existing platform?

Designing and operating a platform for Java applications is a specialist skill, and the engineers behind platforms like AWS ECS, Google App Engine, Azure App Service, and Cloud Foundry have a lot of expertise that you can leverage by using their offerings.

Essential Development Processes

Regardless of what platform you ultimately deploy your Java application to, you and your team must implement a series of fundamental first steps when starting your journey with continuous delivery:

• Ensure that all application code and associated configuration is under version control, and that this is the single source of truth.

• Create a continuous delivery pipeline that takes code from version control, builds and tests the application, and deploys this onto a test (or, ideally, a production) environment.

• Encode any manual deployment steps, such as changing configuration or upgrading a database into the automated deployment process that is triggered by the CD pipeline.

• Package your applications as standard Java artifacts (e.g., a WAR or fat JAR) and remove any nonstandard deployment process (e.g., manual patching of class files). This allows standard deployment mechanisms to be used (e.g., uploading a WAR via an application server API or copying a fat JAR onto the filesystem) and will facilitate further migration to additional or new platforms in the future.

The Value of Developer Experience: Daniel’s Perspective

The topic of developer experience, sometimes referred to as DevEx or DX, is a rapidly emerging area of development for open source and commercial tooling. The topic focuses on the workflow and tooling that is required to convert an idea into code, and ultimately through to delivering value to users in production. DevEx is fundamentally about minimizing friction for engineers and making tasks such as debugging and observability easier. Many people, including me, are currently working in this space, and some early thoughts can be found in my July 2018 CNCF webinar “Creating an Effective Developer Experience on Kubernetes”.

Traditional Infrastructure Platforms

Many of you have undoubtedly deployed Java applications on what can now be referred to as traditional infrastructure—application servers running within an on-premises data center that is managed by a separate sysadmin team. Many teams are also undoubtedly still deploying here. Although this book focuses on continuously delivering Java applications onto modern platforms, there is still much here that can be learned that is directly transferable to traditional environments.

Traditional Platform Components

Within traditional infrastructure, you (as a Java developer of web applications) typically interact with the following components:

Application server

A software framework that provides facilities to create web applications and a server environment to run them. For example, JBoss Application Server, GlassFish, or Tomcat.

Middleware

Software that sits “in the middle” between application software that may be working on different operating systems, with the goal of supporting and simplifying complex distributed applications. Examples include ESB technology, like IBM WebSphere ESB, and MQ offerings, like ActiveMQ.

Database

An organized collection of data, which typically in a traditional infrastructure stack is a relational database management system (RDBMS). For example, Oracle RDBMS, Microsoft SQL Server, or IBM DB2.

Challenges with Traditional Infrastructure Platforms

The core challenge with traditional infrastructure is that there is no standardized or broadly accepted continuous delivery out-of-the-box complete solution. This is because the platform and deployment procedures are typically bespoke, and the underlying infrastructure is not programmable. This can make the life of a developer deploying onto this platform quite challenging. Often the deployment and operation of these platforms is captured via “tribal knowledge”; the constantly evolving customized processes and mechanisms are not captured formally, but are instead shared through stories and partially correct wiki pages.

Platform Automation Is Useful for Continuous Delivery

One of the biggest technical challenges with implementing CD by using a traditional infrastructure stack is the difficulty in automating the creation of infrastructure configuration. You should strive to make your tests and deployments as deterministic and reliable as possible, and you must therefore work closely with the sysadmin team to explain the need for consistent and reproducible infrastructure environments and deployment mechanisms.

Typically, deploying an application onto traditional infrastructure requires some form of artifact “hand-off” that may or may not include deployment instructions—potentially, a text file with a few snippets of SQL that are required to be run against the associated database. The lack of codification and repeatability of these instructions can lead to errors and friction between dev and ops. Another core challenge with this type of platform is that it can fall into disrepair over time (particularly if the sysadmin or operator who created the platform leaves the organization), and generally speaking, when something major goes wrong, it quickly becomes a murder-mystery-style debugging adventure, which is not fun.

Benefits of Being Traditional

The benefits of running traditional infrastructure is that the sysadmin and operational teams responsible for management typically have full control (unless, of course, a third-party is maintaining the platform!). This full control allows rapid changes and configuration—sometimes for better or worse—as well as full visibility into the underlying hardware. Knowledge and skills are required for understanding these inner workings, but this ability to look “behind the curtain” can save many hours of debugging in comparison with using a blackbox third-party service.

Sysadmins—Even Developers Need Heroes

Although the relationship between dev and ops can sometimes be like that of cats and dogs, we can’t stress enough the benefits of increased collaboration between these two teams. As it states on the T-shirt, “sysadmins: even developers need heroes.” Over our careers, we have learned so much from sysadmins and operations teams, and this has helped us to understand more about infrastructure, and ultimately to become a better developer.

CI/CD on Traditional Infrastructure Platforms

Introducing continuous delivery to this type of platform typically provides many benefits, such as improved build reliability and faster deployment times. Here is what you should focus on:

• Once an application is being built and deployed via a pipeline, this is a great opportunity to begin augmenting the application test suite with assertions that test for well-known (and repeated) issues. This is also a good opportunity to include nonfunctional testing services within the pipeline, such as code quality analysis, code test coverage, and security testing. You may not be able to fix it all straight away, but you will have a baseline from which to demonstrate continual improvement.

• Creation of a pipeline can also trigger the discussion with the ops team as to characteristics of the underlying infrastructure, and can lead to the formulation of SLAs. The creation of SLAs allows all parties—developers, QA, and ops—to agree on what the platform should provide, and, in turn, what operational functionality the application should have (i.e., should the app boot within 30 seconds, or be tolerant of disk failures, or handle intermittent network failures?).

• Once the pipeline is in place, you will also be able to gain more visibility into areas of your workflow that require improvement. For example, code merge conflicts may indicate a need to change commit policies or implement something like Gitflow, slow test runs may indicate the need to parallelize tests, or repeated failed deployments may mean extra effort should be invested within the deployment scripts.

Cloud (IaaS) Platform

Infrastructure-as-a-service (IaaS) cloud infrastructure and platforms currently come in many forms:

Private cloud

This is an evolution from traditional infrastructure, and, in essence, is virtualized hardware that is managed on premises or within a private data center. The line can be drawn between traditional infrastructure and private cloud by requiring that the private cloud must be manageable via an API or SDK—e.g., VMware vSphere, OpenStack, Cisco Open Network Environment (ONE), and Scality RING storage.

Public cloud

Technology that the majority of people reference when they talk about the cloud, which is typically provided by vendors like AWS, Microsoft Azure, GCP, and others. 

Hybrid cloud

This is a blend of traditional private virtualized infrastructure with public cloud offerings. Major cloud vendors are starting to offer products that will allow organizations to run hybrid clouds (or help with migration), such as Azure’s Stack (and AWS’s VM Import/Export and Direct Connect).

Looking Inside the Cloud

The cloud is different from traditional infrastructure in that you, as a developer, are exposed to more fundamental infrastructure components:

Compute

A machine, VM, or other type of computing instance. This is directly analogous to your desktop or laptop computer, and examples include AWS EC2, vSphere VM, and OpenStack Nova.

Storage

Storage for your application data. This can be block storage that can be used to boot a machine and host application binaries, much like the hard drive on your local machine, or object storage that is not usable for booting a machine and can be used to store binary or other style data objects. Examples of block storage include AWS EBS, GCP Persistent Disk, and VMware local storage. Examples of object storage include Amazon S3, Azure Storage Blobs, and OpenStack Swift.

Network

Connections, routers, firewalls, and other communication infrastructure. Examples include Amazon VPC, security groups/firewall rules, and OpenStack Neutron.

Services

Database and external middleware services that are directly analogous to that of traditional infrastructure, but these are typically fully managed services offered by the cloud vendor (e.g., Amazon RDS, Google Cloud Spanner, or Azure Service Bus).

Continuous delivery services

Increasingly, you will see that virtualization and cloud vendors are providing out-of-the-box CD solutions, such as an integrated Jenkins solution or a bespoke solution such as VMware vRealize Code Stream or Amazon CodePipeline.

Additional Resources for Cloud Best Practices

If you are completely new to cloud technologies, we recommend reading The Practice of Cloud System Administration: Designing and Operating Large Distributed Systems (Addison-Wesley Professional) by Thomas Limoncelli et al.

The public cloud can be a brave new world for many engineers, so the majority of cloud vendors have created best-practice documentation, which discusses everything from recommended architecture principles, security practices, and approaches to managing infrastructure. Many engineers we talk to on our consulting travels believe that vendor-created content can often be biased, but this is becoming less of an issue. The current vendor-driven cloud best practices are very much recommended reading if you are working within their ecosystem:

• Amazon Web Services white papers, including “AWS Well-Architected Framework” and “Architecting for the Cloud: AWS Best Practices”
• “Azure Architecture Center” and “Azure Security Best Practices”
• “How to Use Google Cloud Platform” and “Best Practices for Enterprise Organizations”

Cloud Challenges

The core challenge for you, as a developer moving to cloud technologies, is dealing with the learning curve needed to understand not only the new infrastructure building blocks, but also the characteristics and performance of these technologies. You will need to spend time understanding the impact of being exposed to  networked computers; you will nearly always be building a distributed system. You also will need to ensure that you design your applications to perform as well within a networked environment as they do on your local development machine.

Mechanical Sympathy: Daniel’s First Cloud Experiences

When I first started using cloud technologies, this was one of the biggest learning hurdles for me, as it means that the characteristics and performance of infrastructure on your local machine may not match that of production—in a big way! For one particular project in the earlier 2010s, I spent a week building a product feed parsing application that wrote high volumes of temporary data to local storage. This worked great on my development machine (with a locally connected SSD drive), but as soon as I deployed this code to production, the performance dropped by nearly two orders of magnitude as the block storage on the cloud instance communicated over the network, which had very different performance characteristics!

Because of the nature (and scale) of the hardware used within IaaS platforms, there is a relatively high probability that individual components will fail. Therefore, you will need to design, build, and test resilient applications that can handle known failure modes; for example, most cloud compute instances are ephemeral and can stop unexpectedly, and your application must handle this gracefully. Werner Vogels, CTO at Amazon, is famous for saying that “everything fails all of the time in the cloud.” Initially, this might appear a strange thing for the CTO of a cloud vendor to say, but he was referring to the benefits of the public cloud—on-demand and cost-effective access to incredible infrastructure that can run at large scale—coming at a cost, and this cost is the reliability of individual components.

The Ephemeral Nature of the Cloud: Daniel’s Experience

Another one of my core learning experiences when I started developing Java applications for the cloud was the ephemeral nature of the compute instances. My applications would randomly stop when an instance was terminated, and because I was using elastic configurations to ensure that I always had a set number of instances running, a new instance would be started. I was used to creating Java applications that were started only once a month, or even less often, and so I didn’t worry about long startup times (and the JVM and Java applications are almost famous for their long initialization times!). I was also used to applications rarely being terminated in an uncontrolled fashion, and so wrote only basic graceful restart or cleanup functionality. This all had to change!

The benefit of applying continuous delivery when deploying Java applications onto cloud infrastructure is that you can capture what you’ve learned in the build pipeline. For example, if you discover strange performance characteristics that affect your application negatively, you can create an integration test that simulates this and asserts the correct handling. This test can be run on every build in order to prevent regression from future modifications.

Many cloud vendors provide options to choose and provision infrastructure that has low levels of baseline CPU, network, and disk performance, but has “credits” to allow time-limited burstable performance that is considerably over the baseline. An initial allowance of credits is given, and credits can be accumulated when the infrastructure is being used below the baseline. This allows for the creation of cost-effective systems; the starting credit balance allows the infrastructure to initialize rapidly, and if the system usage patterns truly are burstable, then a healthy level of credit can be maintained to burst as required. However, if the usage patterns of an application are not burstable, the credit balance can soon become depleted, and the corresponding application’s performance will drop, or potentially the application may fail.

Watch for the Effects of “Burstable” Infrastructure

Short development and test runs of an application deployed onto this type of infrastructure can lead to the development team believing that longer-term performance will be better than it actually is. Accordingly, the baseline performance of such infrastructure must be understood, and simulations created within a build pipeline that test for system performance when running at the baseline infrastructure performance. 

Benefits of the Cloud

The core benefit for developers working with cloud infrastructure is that everything is programmable: you can now apply all of your programming know-how and skills to infrastructure. The cloud technologies are generally more standardized than traditional infrastructure, too, and although a lot of variation exists across public cloud vendors, it is possible to transfer your infrastructure knowledge as you move teams or even organizations.

Immutable Infrastructure

Cloud computing (or, really, the introduction of programmable virtualization) made the vision of immutable infrastructure possible. Immutable infrastructure is not modifiable after creation. Instead of updating, you re-create the infrastructure, either by using the same template (guaranteeing that it is identical to the infrastructure being replaced) or using a new “updated” template. This generally makes your life as a developer much easier, as you can provide stronger guarantees of a packaged deployment artifact that works in QA, and staging will work in production.

Another core benefit with the cloud is the flexibility of instantiating infrastructure. The cloud makes it possible for potentially each developer to create their own test environment, or to run large-scale tests (but watch the cost!).

Developers Can’t (or Shouldn’t) Do Everything with the Cloud

As the cloud allows most infrastructure to be manipulated programmatically, it can be tempting for developers to take on more and more responsibility for the operational side of delivering software. For small-scale organizations or startups, this can be useful, allowing a small development team to be self-sufficient and move fast. However, this can also mean that developers become overworked, or must learn increasing amounts of new skills (sometimes outside their expertise or comfort zone), or become blind to the inherent trade-offs and friction between the dev and ops roles. This can lead to individual or team burnout, or lots of operational issues with applications in production. This should be avoided at all costs.

Continuously Delivering into the Cloud

Introducing CD within a cloud environment is generally easier in comparison to a traditional infrastructure platform, as CD practices largely co-evolved with cloud technologies (and vice versa). The introduction of APIs and SDKs by virtualization and public cloud vendors meant that codifying and reproducing environments (for testing and QA, etc.) became much easier. In addition to the steps mentioned previously for practicing CD with traditional infrastructure, the following guidelines also apply to the cloud:

• Conduct analysis on which type of cloud deployment paradigm your current project will fit best. Large cloud vendors like AWS, GCP, and Azure offer many forms of cloud platform, from containers (covered later) like AWS ECS, and Azure AKS, to PaaS-style offerings like AWS Elastic Beanstalk and Azure App Service, in addition to the IaaS building blocks of compute, storage, and networking.

• Focus on the creation of a CD pipeline that delivers your application (or even a proof-of-concept application) from your local development environment to production. This will highlight any technical or organizational issues with using cloud platforms.

• Codify as much of the cloud platform as possible (ideally, everything) by using tools like HashiCorp Terraform, Ansible, or Puppet.

• Codify what you’ve learned (and your mistakes) into the pipeline—for example, load testing, security testing, and chaos/resilience testing.

The cloud offers many possibilities for deploying Java applications at a rapid pace, but speed is nothing without control, and control in the sense of CD is the ability to repeatedly and reliably verify that your application is fit for production usage.

Platform as a Service

Platform as a service (PaaS) has a longer history than most people realize. The JavaScript-supporting Zimki was the first recognized PaaS released in 2005. Within the Java space, the launch of Google’s App Engine and Cloud Foundry have been the most influential events.

Diving Deeper into PaaS Platforms

Many of the core takeaways from this book can be directly applied to PaaS technologies, but because they often provide an opinionated (preset) workflow, we also recommend consulting specific PaaS-flavored books if you are deploying here. For example, if you are deploying applications on Google App Engine, Programming Google App Engine with Java (O’Reilly) by Dan Sanderson is an interesting read. If you are working with Cloud Foundry (specifically, Pivotal Cloud Foundry), then Cloud Native Java (O’Reilly) by Josh Long and Kenny Bastani provides a comprehensive guide.

Peeking Inside a PaaS

PaaSs generally have similar primitives to cloud computing, although some of the abstractions might be at a slightly higher level. For example, within Cloud Foundry, you don’t directly spin up an instance to deploy your code onto, and instead you focus on creating a “droplet” application artifact:

Compute

A machine, VM, or container environment.

Storage

Storage for your application data. This can be block storage that can be used to boot a machine and host application binaries, much like the hard drive on your local machine, or object storage that is not usable for booting a machine and can be used to store binary or other style data objects.

Network

Connections, routers, firewalls, and other communication infrastructure.

Services

Database and external middleware services that are directly analogous to that of traditional infrastructure, but these are typically fully managed services offered by the PaaS vendor.

Continuous delivery services

Many PaaS vendors provide a complete solution for taking code from a local development environment all the way through to production. For example, Cloud Foundry has the integrated cf CLI tool and Concourse CD platform, and Red Hat’s OpenShift has an integrated Jenkins solution.

PaaS Challenges

Many of the challenges revolving around the use of PaaS are concerned with the learning curve and usage models. Generally speaking, most PaaSs are quite opinionated in regards to both developer workflow and deployment models. Some projects will not fit into the models provided by a PaaS, and this is where it can become challenging to bend the PaaS to work with your application. Having said this, if you are building relatively typical web-based applications, then this should not be a problem.

Don’t Automatically Assume That PaaS Is Over-Opinionated

Many developers assume a PaaS like Cloud Foundry is over-opinionated in regards to workflow, or that the project they are working on is “special” or unique in some way, and won’t be deployable onto a PaaS. Generally speaking, if you are creating a web application for an e-commerce business, then this shouldn’t require a bespoke developer workflow. In fact, we even know of many large-scale banking and data processing organizations that use (for example) Cloud Foundry very successfully.

Another possible challenge imposed by a PaaS is the potentially constrained Java JDK or runtime environment.  If you don’t know about these differences between your local JDK and development environment, or you are relatively inexperienced with Java development, this can be a steep learning curve! Examples of PaaS JDK modifications include only whitelisted access to the JDK (i.e., some APIs within the standard OpenJDK may be unavailable); the use of a Java security manager to restrict operations available; the use of a sandbox that restricts access to resources like the Java threads, the network, or filesystem; and differing class-loading strategies. Some PaaS providers also encourage (or force) you to use their proprietary SDKs for accessing databases or middleware.

Understand How Your PaaS Implements and Exposes the JDK: Daniel’s Experience

If your PaaS provider provides a native Java environment (you don’t have to install a Java runtime), please take time to understand how the JDK and associated runtime have been implemented. This was a big learning curve when I created my first Java application on Google App Engine. The platform offered many great things (particularly in relation to operation and scaling), but the sandboxed Java runtime environment took away a lot of the tools I was used to working with. Ultimately, the application I created met the business requirements, but my estimation was way off, as I didn’t budget for the learning curve!

The runtime and OS exposed via a PaaS can also be modified or restricted. In addition to the access restriction to resources like the network and filesystems (which can be implemented within the Java runtime or the OS), resources can also be throttled or controlled—for example, CPU and usage are common targets for this—and the platform instances (and other infrastructure such as network routers or IP addresses) can often be ephemeral. Once this has been identified, the learning can be codified into the build pipeline; for example, asserting that your application does not produce OutOfMemoryExceptions when under heavy load, or that your application gracefully handles forced restarts.

Modern PaaS Documentation Is (Generally Speaking) Excellent

In my experience, most popular Java-friendly PaaSs provide great documentation, both about the developer workflow and about the runtime environment and any caveats or limitations. For example, the Google App Engine team has written extensive notes about its (now deprecated) Java 7 runtime as well as its Java 8 runtime. The Cloud Foundry team has created similar notes about the corresponding Java build packs and runtime, and each CF commercial vendor usually provides additional documentation on any details unique to their environment.

If you do decide that a hosted PaaS is the best fit for your project, one final thing to be aware of is how hosted offerings are priced. Most of the Java PaaSs are priced based on memory per application instance, and the number of application instances, and other resources (such as CPU speed and cores and network bandwidth) are scaled proportionally. This can mean that some CPU- or memory-intensive applications can be expensive to run, and you often cannot pack applications on infrastructure to accommodate this. With traditional or cloud infrastructure, you can run two resource-intensive applications—one CPU-bound and one memory-bound—side by side on the same hardware, and this way, you are not “wasting” (or being charged for wasting) the resources that may go unused if just one application were running here.

Benefits of PaaS

Once you have taken the time to learn the core principles (and let’s be honest: every new technology requires some learning), you can often become very productive, and your knowledge is directly transferable to the next project that uses PaaS. This leads to a (potentially massive) increase in developer productivity. If you are using a PaaS that is based on open standards, such as Cloud Foundry, you also can operate your PaaS across multiple infrastructures—for example, a hybrid of on-premises Cloud Foundry, public-cloud-operated Pivotal Cloud Foundry, and public-cloud-operated IBM Bluemix. Transitioning from one infrastructure/vendor to another should not impact you greatly as a developer, because the platform provides all the core abstractions you care about.

In addition, if your team decides to use a hosted PaaS offering, the operational burden of running the platform, and maintaining and patching the underlying infrastructure, is removed. This can free up time for other value-adding development activities!

CI/CD and PaaS

Introducing continuous delivery with applications deployed into a PaaS environment is usually a no-brainer, and often it is not optional; the only way to deploy your application is via a build pipeline. The following list highlights several areas you should prioritize investing your time in when implementing CD with a PaaS:

• Focus on the creation of a CD pipeline that delivers your application (or even a proof-of-concept application) from your local development environment to production. This will highlight any technical or organizational issues with using the PaaS you have chosen to deploy to.

• Store all PaaS and application configuration and metadata within version control; for example, any service discovery configuration or custom build packs you have created.

• Codify your learning (and mistakes) into the pipeline—for example, load testing, security testing, and chaos/resilience testing.

Containers (Docker)

Many developers use the words “container” and “Docker” interchangeably. This isn’t strictly correct; a container is an abstract concept of OS-level virtualization or process isolation, and Docker is an implementation of this technology by the company Docker, Inc.  Container technology existed before the emergence of Docker in Linux Containers (LXC), but Docker provided the missing user experience and mechanism to centrally share, pull, and push container images. There are many other container technologies such as CoreOS’s rkt, Microsoft’s Hyper-V containers (and Windows containers), and Canonical’s LXD.

Learning More About Docker

Because the container technology space is currently still evolving (and often changes at a rapid pace), the best resources can often be found online. For engineers who like to get the basics of a technology from a book, we recommend Docker Deep Dive by Nigel Poulton.

Container Platform Components

A typical container-based deployment platform has multiple components:

The container technology

Fundamentally, a container provides OS-level virtualization, and although all of the running containers on a single machine share an OS kernel, they each have separate namespaces for processes and networking, control groups (cgroups) for allocating and controlling resources like CPU and memory, and a root filesystem (rootfs) that is not the same as the underlying host’s rootfs.

Container scheduler/orchestrator

This component is responsible for starting, stopping, and managing the container processes. This technology is often referred to as container infastructure as a service (CIaaS), and in the Docker ecosystems this is typically provided by Docker Swarm or Kubernetes.

Storage

Storage for your application data. This can be block storage that can be mapped into a container, or object storage that can be used to store binary or other style data objects. This is often managed by the CIaaS component.

Network

Connections, routers, firewalls, and other communication infrastructure. This is often managed by the CIaaS component.

Services

Database and external middleware services that are directly analogous to that of traditional infrastructure, but these are typically fully managed services offered by the container platform vendor.

Continuous delivery services

Many container platform vendors provide a complete solution for taking code from a local development environment all the way through to production. For example, Docker Enterprise provides a solution integrated with its Docker Hub, Azure provides Azure Pipelines, and AWS ECS provides CodeBuild and CodePipeline.

In Chapter 10, you will learn how to deploy Docker containers onto AWS’s ECS.

Container Challenges

The biggest challenge with container technology is the learning curve. Not only is the technology quite different from that of the cloud or PaaS, but it is very much an emerging and evolving technology. It can sometimes be almost a full-time job keeping up-to-date with the latest container innovations! Another core challenge is that older technologies (including the JVM itself) may not always work correctly or as expected when running within a container, and you will often require (at least basic) operational knowledge to identify, understand, and fix issues.

Running Java Within Docker: Common Gotchas Daniel Experienced

I’ve worked on several projects with Java and Docker, and my team and I had to overcome quite a few issues. Steve Poole, Developer Advocate for IBM Cloud Runtimes, and I talked about “Debugging Java Apps In Containers: No Heavy Welding Gear Required” at JavaOne in 2015, and here is a summary of some of the challenges and issues:

• Any Java runtime pre Java 9 is not control group (cgroup) aware. A call to obtain the number of CPU cores available to an application will return the underlying host CPU resource information, which, in reality, may be shared across multiple containers running on the same host. This also affects several Java runtime properties, as the fork-join pool and garbage-collection parameters are configured based on this information. This can be overcome by specifying resource limits as command-line flags when starting the JVM.

• Containers typically share a single source of entropy (/dev/random) on the host machine, and this can be quickly exhausted. This manifests itself with Java applications unexpectedly stalling/blocking during cryptographic operations such as token generation on the initialization of security functionality. It is often beneficial to use the JVM option -Djava.security.​egd=file:/dev/urandom, but be aware that this may have security implications.

It is also appropriate to remember that not all developers are operationally aware, nor do they want to be; they simply want to write code that solves business problems. For this reason, it can be beneficial to hide some of the container technology from developers and provide them with a CD process that manages the building and deploying of Java applications within this space. As we’ve mentioned earlier, this still does not remove the requirement for developers to at least understand the operational characteristics and impact of running their applications by using this technology; always think about mechanical sympathy!

Container Technology Is Still Rapidly Evolving

Although the speed of innovation has slowed somewhat over the past year, the container technology ecosystem is still very much a pioneering space. Technologies are evolving rapidly, tooling is emerging, and a better understanding of development practices is also occurring. However, the trade-off with access to new (better) ways of packaging and deploying applications means that there can be a steep learning curve—involving much experimentation—and things change rapidly. Be sure that you and your team are happy to commit to this trade-off before embracing container technology!

Container Benefits

The core benefits of container technology are the standardization of deployment packaging—the container image—and the flexibility of execution. The standardization of deployment packaging makes it easier to deploy and operate a platform because the container image becomes the new abstraction/interface between dev and ops. As long as you can package your code within a container, ops can run this on a platform; they don’t have to worry (to some degree) exactly how you have configured the application.

Containers Enable Easier Immutable Infrastructure

We mentioned that virtualization and cloud technologies enabled the creation and deployment of immutable artifacts (and infrastructure), and containers really are an extension of this. It is generally easier and quicker to create an immutable container image artifact than a full VM image, and this allows you to iterate and deploy faster.

Continuously Delivering Containers

Introducing continuous delivery with applications deployed into a container platform is usually relatively easy:

• Focus on the creation of a CD pipeline that packages your application (or even a proof-of-concept application) as a container image and takes this from your local development environment to production. This will highlight any technical or organizational issues with using the platform you have chosen to deploy to.

• If you are running your own container platform, capture all configuration within version control, and automate all installation and upgrading via infrastructure as code (IaC) tooling like Terraform or Ansible.

• Store any application configuration and metadata within version control; for example, any service discovery configuration or custom build containers you have created.

• Codify your learning (and mistakes) into the pipeline—for example, load testing, security testing, and chaos/resilience testing.

Kubernetes

Kubernetes is an open source orchestrator for deploying containerized applications that was originally developed by Google. Google has been running containerized applications for many years, and this led to the creation of the Borg container orchestrator that is used internally within Google and was the source of inspiration for Kubernetes.

If you are not familiar with this technology, a few core concepts may appear alien at first glance, but they hide great power. The first is that Kubernetes embraces the principles of immutable infrastructure. Once a container is deployed, the contents (i.e., the application) are not updated by logging into the container and making changes. Instead, a new version is deployed. Second, everything in Kubernetes is declaratively configured. The developer or operator specifies the desired state of the system through deployment descriptors and configuration files, and Kubernetes is responsible for making this happen; you don’t need to provide imperative, step-by-step instructions.

These principles of immutable infrastructure and declarative configuration have several benefits: it is easier to prevent configuration drift, or “snowflake” application instances; declarative deployment configuration can be stored within version control, alongside the code; and Kubernetes can be largely self-healing, so if the system experiences failure like an underlying compute node failure, the system can rebuild and rebalance the applications according to the state specified in the declarative configuration.

Core Concepts of Kubernetes

Kubernetes provides several abstractions and APIs that make it easier to build these distributed applications, such as those based on the microservice architectural style:

Pods

This is the lowest unit of deployment within Kubernetes, and is essentially a group of containers. A pod allows a microservice application container to be grouped with other “sidecar” containers that may provide system services like logging, monitoring, or communication management. Containers within a pod share a filesystem and network namespace. Note that a single container can be deployed, but it is always deployed within a pod

Services

Kubernetes services provide load balancing, naming, and discovery to isolate one microservice from another. Services are backed by Deployments, which, in turn, are responsible for details associated with maintaining the desired number of instances of a pod to be running within the system. Services, deployments, and pods are connected together in Kubernetes through the use of labels, both for naming and selecting.

In Chapter 10, you will learn how to deploy Docker containers into a Kubernetes cluster.

Kubernetes: Up and Running

The container and Kubernetes space is rapidly evolving, and therefore the best learning resources are often online. In fact, the Kubernetes online documentation is generally excellent. If you are looking for a book to bootstrap your learning, we recommend Kubernetes: Up and Running (O’Reilly), which is written by Kelsey Hightower et al., a group of Kubernetes creators and experts.

Kubernetes Challenges

As with container technology in general, the biggest challenge for developers using Kubernetes is the learning curve. Not only do you have to understand Docker (or another container technology), but you also have to come to grips with concepts like scheduling, orchestration, and service discovery. On top of this, many development teams like the idea of running their own Kubernetes cluster, perhaps within on-premises infrastructure, and the operational learning curve for this can be steep—particularly if you do not have experience in running distributed or container-based platforms.

Don’t Run Your Own Kubernetes Cluster (Unless Essential)

All of the major cloud vendors now offer a hosted and fully managed Kubernetes offering, and so we strongly recommend against running your own Kubernetes cluster—particularly if you are a relatively small team, say, under 100 developers. The cloud vendors have much more expertise and operational experience of running the platform, taking on the burden of the platform SLA, and handling issues with Kubernetes or the underlying infrastructure when they (inevitably) go wrong. In our opinion, running your own Kubernetes cluster is viable only if you have a dedicated operations team or strong compliance and governance requirements.

The core concepts of Kubernetes are all learnable, but it will take time, and not every developer wants to deep-dive into the operational side of a technology like Kubernetes. They simply want to code applications that deliver business value.

Although the speed of innovation has slowed somewhat over the past few months, the Kubernetes ecosystem (much like the container ecosystem in general) is still very much an evolving space. Technologies are evolving rapidly, tooling is emerging, and a better understanding of development practices is also occurring. However, the trade-off with access to new (better) ways of packaging and deploying applications means that there can be a steep learning curve—involving much experimentation—and things change rapidly.

Kubernetes Is Still Evolving (into a PaaS?)

Kubernetes is increasingly being thought of as a lower-level platform component, as vendors like Red Hat and Apprenda are building PaaS offerings on top of this technology. The core idea here is to encapsulate and hide some of the more operationally focused aspects of Kubernetes, and to expose abstractions that will be more useful to developers. Be sure that you and your team are happy to commit to the trade-off of using the raw (and evolving) Kubernetes offering before building an associated platform.

Benefits of Kubernetes

Kubernetes offers many benefits, and the technology encompasses many thousands of hours learning how to run applications at scale within a company like Google. The core benefit is that many of the difficult challenges with running applications within containers are handled for you: you simply specify in a declarative manner how your application is composed of services, the resources that you want your service components to have (CPU, memory, storage, etc.), and the number of service instances that should be running at any given time. Kubernetes provides a framework for you to specify resource and security boundaries (using network policies and ACLs, etc.), creates and manages a cluster from underlying infrastructure resources that handles the allocation of resources, and starts and stops instances according to infrastructure failure or some other issues.

Continuous Delivery on Kubernetes

Introducing continuous delivery with applications deployed into a container platform is usually relatively easy:

• Focus on the creation of a CD pipeline that packages your application (or even a proof-of-concept application) as a container image and then takes this from your local development environment to a production Kubernetes cluster. This will highlight any technical or organizational issues with using the platform you have chosen to deploy to.

• If you are running your own Kubernetes platform, capture all configuration within version control, and automate all installation and upgrading via IaC tooling like Terraform or Ansible. However, generally it is most likely more cost-effective (and requires less learning) to not run your own Kubernetes platform, and instead use a hosted offering from one of the cloud vendors.

• Store any application configuration and metadata within version control; for example, any service discovery configuration or custom build containers you have created.

• Store any sensitive configuration and metadata within security and key management applications like HashiCorp’s Vault.

• Codify your learning (and mistakes) into the pipeline—for example, handling service failures gracefully, load testing, security testing, and chaos/resilience testing.

• Ensure that you load-test not only your applications, but also the ability of the underlying platform to scale on demand.

Function-as-a-Service/Serverless Functions

FaaS is a compute service that lets you run code without provisioning or managing servers. FaaS platforms executes your code only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time you consume. There is no charge when your code is not running. With FaaS, you can run code for virtually any type of application or backend service—all with zero administration. FaaS vendors run your code on a high-availability compute infrastructure and perform all the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code monitoring, and logging. All you need to do is supply your code in one of the languages that the platform supports.

Additional Learning

The topics of FaaS and serverless are evolving crazy fast, but for those who are looking for a general introduction to the core principles and associated architectural requirements, we recommend Serverless Architectures on AWS (Manning) by Peter Sbarski.

FaaS Concepts

You can use FaaS to run your code in response to events, such as changes to data in an blob storage bucket or a managed NoSQL database table, or to run your code in response to HTTP requests using an API Gateway. You can also invoke your code by using API calls made using vendor-specific SDKs. With these capabilities, you can use functions to easily create comprehensive data processing pipelines or reactive event-driven applications that can scale on demand, and you are charged only when a function is running.

Eventing-All-the-Things: A Completely New Paradigm

Don’t underestimate the learning curve for adapting to the event-driven model imposed by FaaS serverless platforms, as this can be completely new to many developers. We have seen several developers creating applications by chaining functions together with synchronous (blocking) calls, much as they would with traditional procedural development, and this causes the creation of fragile and inflexible applications. We definitely recommend learning more about event-driven architecture, asynchronous programming, and designing reactive systems.

When using FaaS, you are responsible only for your code. The FaaS platform manages the compute (server) fleet that offers a balance of memory, CPU, network, and other resources. This is in exchange for flexibility, which means you cannot log in to compute instances, or customize the operating system or language runtime. These constraints enable FaaS platforms to perform operational and administrative activities on your behalf, including provisioning capacity, monitoring fleet health, applying security patches, deploying your code, and monitoring and logging your Lambda functions.

Challenges of FaaS

As with many of the other modern deployment platforms, the biggest challenge with FaaS technologies is the learning curve. Not only is the platform and Java runtime often restricted (in much the same way as PaaS), but the development paradigm is skewed toward event-driven architecture (EDA). FaaS applications are generally composed of reactive functions; the application functions are triggered by events like the arrival of a message in an MQ or a file upload notification from the object store, and the execution of the function may result in side effects (persistence of data in a database, etc.) and the generation of additional events.

FaaS platforms are available to run on your own infrastructure, often on top of an orchestration system like Kubernetes (e.g., kubeless, fission, and OpenFaaS), standalone (like Apache OpenWhisk), or as a hosted service (such as AWS Lambda, Azure Functions, or Google Cloud Functions). If you choose to use a hosted offering—and generally speaking, this is the best option because of the reduction in operational burden and close integration with the vendor’s other services—then you must be aware of how resources are provided and the control you have over this. For example, with AWS Lambda and Azure Functions, you specify the required memory, and the CPU resources available are scaled in relation to this.

Understand the FaaS Java Runtime

In much the same way with PaaS, the Java runtime that is made available in FaaS platforms can be a modified version of the JRE. Common restrictions imposed by running the JRE in a sandbox include limited access to filesystem and network. The functions also typically have a time limit for execution, and this means that the code within your function will be forcefully terminated if this limit is exceeded.

In addition, although FaaS functions are modeled as ephemeral deployment units that are instantiated and terminated on demand, the reality of the implementation details (typically, the function is executed within an OS container) may leak, and this means a function executed twice sequentially may use the same instantiation of the function. This can be bad—for example, if you have left global state like a public class variable set incorrectly, this will persist between executions. Alternatively, it can be good—allowing you to avoid repeated resource instantiation costs by opening a database connection or prewarming a cache within a static initializer and storing the result in a public variable.

Testing FaaS applications can also be challenging, both from a functional and nonfunctional perspective, as events must be created to trigger functionality, and often many functions have to be composed together to provide a meaningful unit of functionality. For functional testing, this means that triggering test events have to be curated, and a variety of functions (of a specific version) instantiated. For nonfunctional tests, you typically have to run the tests on the platform itself (perhaps in a “staging” environment) with the same caveats applying in regard to curating events and composing applications.

Be Aware of (Account-Wide) FaaS Limits

If you are using a fully managed hosted FaaS environment, be aware of various limits that may apply; for example, the maximum number of simultaneous concurrent executions of a function. These limits can also apply account-wide, so if you have a runaway function, this may impact your entire organization’s systems!

FaaS Benefits

The core benefit of FaaS is that the model allows the rapid creation and deployment of new features. Applications with low-volume and spiky usage patterns can also be operated at low cost.

Worth-Based Development: A Powerful Concept

Simon Wardley has written much about the powerful model that FaaS provides, and a core theme is worth-based development. The essence of this concept is that design choices are made to maximize the value provided by these systems in relation to their costs. This implies that you, as a developer, will need to understand how “value” is defined within your organization and for the component you are working on (for example, it may be revenue generated, or the number of sign-ups), as well as the costing model of the underlying platform and infrastructure.

Hosted FaaS platforms make the second part of this challenge much easier as you are billed per function execution, and you have limited (or no) fixed costs of maintaining infrastructure if your application is not used.

CI/CD and FaaS

Introducing continuous delivery with applications deployed within a FaaS platform is usually a no-brainer, and may even be enforced by the vendor:

• Focus on the creation of a CD pipeline that packages your application (or even a proof-of-concept application) as an artifact and takes this from your local development environment through to a production FaaS environment. This will highlight any technical or organizational issues with using FaaS technologies.

• If you are running your own FaaS platform, capture all configuration within version control, and automate all installation and upgrading via IaC tooling like Terraform or Ansible. However, generally it will be more cost-effective to not operate your own platform and instead use a hosted offering from one of the cloud vendors.

• Store any application configuration and metadata within version control; for example, any service discovery configuration or custom build containers you have created. This metadata should also include the current version of the function that is considered “master” or the most current.

• Ensure that you are exposing and capturing core metrics for each function deployed, such as memory used and execution time.

• It may be beneficial to conduct “right-sizing” experiments as part of the load testing within your pipeline: you deploy your function to a variety of container/instance/runtime sizes and evaluate throughput and latency. Load testing FaaS applications is generally challenging, as the underlying platform typically autoscales based on load, but the size (CPU and memory) of the container/instance/runtime will impact the individual and aggregate execution speed.

• Codify your learning (and mistakes) into the pipeline—for example, handling service failures gracefully, security testing, and chaos/resilience testing.

• Although FaaS platforms abstract away the servers (and associated security patching), don’t forget that you are responsible for identifying and resolving security issues with your code and all of your dependencies.

• Don’t forget to decommission or delete old functions. It is all too easy to forget about an old unused function, or to create a new replacement function and leave the existing one. However, this can not only be confusing for other engineers (wondering which functions are actively used), but also increase the attack surface from a security perspective.

Working with Infrastructure as Code

If you are creating your own platform or working closely with an operations team that is doing this, we recommend learning more about programmable infrastructure and infrastructure as code (IaC). This will not only allow you to create or customize your own infrastructure, but also help you to cultivate mechanical sympathy for the platform.

Infrastructure as Code: Learning More from Kief Morris

This book focuses on Java development, so IaC is not covered in much detail because of space limitations. However, if you are interested in learning more, then Infrastructure as Code by Kief Morris (O’Reilly) is highly recommended.

Our preferred tooling in this space is HashiCorp’s Terraform for specifying cloud environments and related configuration (block storage, networking, additional cloud vendor services, etc.), and Red Hat’s Ansible for configuration management and machine instance installation.

Summary

In this chapter, you learned about the functionality that is provided by platforms for deployment of Java-based web applications, and how the choice of platform affects CI/CD.

• A platform provides an entry point for your users to access your application; a Java runtime (which you may have to manage); access to CPU, memory, and block storage; access to middleware or data stores; resilience (e.g., restarts of failed applications); service discovery; security; and cost management.

• Regardless of the platform used, all code and configuration must be stored within version control, and a continuous delivery build pipeline created that can take code from a local machine, build and test, and deploy.

• Packaging and deploying Java applications as standard Java artifacts, such as the JAR or WAR, provides more flexibility and safety than using a bespoke process (e.g., zipping code artifacts or class patching).

• The four main platform types—traditional infrastructure platforms, cloud (IaaS), PaaS, and FaaS—all have differing benefits and challenges for implementing CI/CD for your application.

• You can augment your skills as a Java developer by learning more about infrastructure as code (IaC) and using tools like Terraform and Ansible. This will allow you to create test environments and to cultivate a better understanding of the underlying platform you are using.

Now that you have a good understanding of Java deployment platforms, let’s look into building Java applications in more detail.