Chapter 1. Introduction

The Grain of a Programming Language

Like wood, a programming language has a grain. In both carpentry and programming, when you work with the grain, things go smoothly. When you work against the grain, things are more difficult. When you work against the grain of a programming language, you have to write more code than necessary, performance suffers, you are more likely to introduce defects, you usually have to override convenient defaults, and you have to fight against the tooling every step of the way.

Going against the grain involves constant effort with an uncertain payoff.

For example, it has always been possible to write Java code in a functional style, but few programmers did before Java 8—for good reasons.

Here’s Kotlin code that calculates the sum of a list of numbers by folding the list with the addition operator:

val sum = numbers.fold(0, Int::plus)

Let’s compare that to what was required in Java 1.0 to do the same.

The mists of time will pass over you while transporting you to 1995…

Java 1.0 does not have first-class functions, so we have to implement functions as objects and define our own interfaces for different types of function. For example, the addition function takes two arguments, so we have to define the type of two-argument functions:

public interface Function2 {
    Object apply(Object arg1, Object arg2);
}

Then we have to write the fold higher-order function, hiding the iteration and mutation required by the Vector class. (The 1995 Java standard library doesn’t yet include the Collections Framework.)

public class Vectors {
    public static Object fold(Vector l, Object initial, Function2 f) {
        Object result = initial;
        for (int i = 0; i < l.size(); i++) {
            result = f.apply(result, l.get(i));
        }
        return result;
    }

    ... and other operations on vectors
}

We have to define a separate class for every function we want to pass to our fold function. The addition operator can’t be passed around as a value, and the language has no method references, lambdas, or closures at this time, not even inner classes. Nor does Java 1.0 have generics or autoboxing—we have to cast the arguments to the expected type and write the boxing between reference types and primitives:

public class AddIntegers implements Function2 {
    public Object apply(Object arg1, Object arg2) {
        int i1 = ((Integer) arg1).intValue();
        int i2 = ((Integer) arg2).intValue();
        return new Integer(i1 + i2);
    }
}

And, finally, we can use all that to calculate the sum:

int sum = ((Integer) Vectors.fold(counts, new Integer(0), new AddIntegers()))
    .intValue();

That’s a lot of effort for what is a single expression in a mainstream language in 2020.

But that’s not the end of it. Because Java doesn’t have standard function types, we can’t easily combine different libraries written in a functional style. We have to write adapter classes to map between the function types defined in different libraries. And, because the virtual machine has no JIT and a simple garbage collector, our functional code has worse performance than the imperative alternative:

int sum = 0;
for (int i = 0; i < counts.size(); i++) {
    sum += ((Integer)counts.get(i)).intValue();
}

In 1995, there is just not enough benefit to justify the effort of writing Java in a functional style. Java programmers found it easier to write imperative code that iterated over collections and mutated state.

Writing functional code goes against the grain of Java 1.0.

A language’s grain forms over time as its designers and users build a common understanding of how language features interact and encode their understanding and preferences in libraries that others build upon. The grain influences the way that programmers write code in the language, which influences the evolution of the language and its libraries and programming tools, changing the grain, altering the way that programmers write code in the language, on and on in a continual cycle of mutual feedback and evolution.

For example, as we move forward through time, Java 1.1 adds anonymous inner classes to the language, and Java 2 adds the Collections Framework to the standard library. Anonymous inner classes mean that we don’t need to write a named class for each function we want to pass to our fold function, but the resulting code is arguably harder to read:

int sum = ((Integer) Lists.fold(counts, new Integer(0),
    new Function2() {
        public Object apply(Object arg1, Object arg2) {
            int i1 = ((Integer) arg1).intValue();
            int i2 = ((Integer) arg2).intValue();
            return new Integer(i1 + i2);
        }
    })).intValue();

Functional idioms still go against the grain of Java 2.

Fast-forward to 2004, and Java 5 is the next release that significantly changes the language. It adds generics and autoboxing, which improve type safety and reduce boilerplate code:

public interface Function2<A, B, R> {
    R apply(A arg1, B arg2);
}
int sum = Lists.fold(counts, 0,
    new Function2<Integer, Integer, Integer>() {
        @Override
        public Integer apply(Integer arg1, Integer arg2) {
            return arg1 + arg2;
        }
    });

Java developers often use Google’s Guava library to add some common higher-order functions over collections (although fold is not among them), but even the authors of Guava recommend writing imperative code by default, because it has better performance and is usually easier to read.

Functional programming still goes largely against the grain of Java 5, but we can see the start of a trend.

Java 8 adds anonymous functions (aka lambda expressions) and method references to the language, and the Streams API to the standard library. The compiler and virtual machine optimize lambdas to avoid the performance overhead of anonymous inner classes. The Streams API fully embraces functional idioms, finally allowing:

int sum = counts.stream().reduce(0, Integer::sum);

However, it isn’t entirely plain sailing. We still can’t pass the addition operator as a parameter to the Streams reduce function, but we have the standard library function Integer::sum that does the same thing. Java’s type system still creates awkward edge cases because of its distinction between reference and primitive types. The Streams API is missing some common higher-order functions that we would expect to find if coming from a functional language (or even Ruby). Checked exceptions don’t play well with the Streams API and functional programming in general. And making immutable classes with value semantics still involves a lot of boilerplate code. But with Java 8, Java has fundamentally changed to make a functional style work, if not completely with the grain of the language, at least not against it.

The releases after Java 8 add a variety of smaller language and library features that support more functional programming idioms, but nothing that changes our sum calculation. And that brings us back to the present day.

In the case of Java, the grain of the language, and the way programmers adapted to it, evolved through several distinct programming styles.

An Opinionated History of Java Programming Style

Like ancient poets, we divide the development of Java programming style into four distinct ages: Primeval, Beans, Enterprise, and Modern.

Primeval Style

Originally intended for use in domestic appliances and interactive TV, Java only took off when Netscape adopted Java applets in its hugely popular Navigator browser. Sun released the Java development kit 1.0, Microsoft included Java in Internet Explorer, and suddenly everyone with a web browser had a Java runtime environment. Interest in Java as a programming language exploded.

The fundamentals of Java were in place by this time:

  • The Java virtual machine and its bytecode and class file format

  • Primitive and reference types, null references, garbage collection

  • Classes and interfaces, methods and control flow statements

  • Checked exceptions for error handling, the abstract windowing toolkit

  • Classes for networking with internet and web protocols

  • The loading and linking of code at runtime, sandboxed by a security manager

However, Java wasn’t yet ready for general-purpose programming: the JVM was slow and the standard library sparse.

Java looked like a cross between C++ and Smalltalk, and those two languages influenced the Java programming style of the time. The “getFoo/setFoo” and “AbstractSingletonProxyFactoryBean” conventions that programmers of other languages poke fun at were not yet widespread.

One of Java’s unsung innovations was an official coding convention that spelled out how programmers should name packages, classes, methods, and variables. C and C++ programmers followed a seemingly infinite variety of coding conventions, and code that combined multiple libraries ended up looking like a right dog’s dinner somewhat inconsistent. Java’s one true coding convention meant that Java programmers could seamlessly integrate strangers’ libraries into their programs, and encouraged the growth of a vibrant open source community that continues to this day.

Bean Style

After Java’s initial success, Sun set out to make it a practical tool for building applications. Java 1.1 (1996) added language features (most notably inner classes), improved the runtime (most notably just-in-time compilation and reflection), and extended the standard library. Java 1.2 (1998) added a standard collections API and the Swing cross-platform GUI framework, which ensured Java applications looked and felt equally awkward on every desktop operating system.

At this time, Sun was eying Microsoft’s and Borland’s domination of corporate software development. Java had the potential to be a strong competitor to Visual Basic and Delphi. Sun added a slew of APIs that were heavily inspired by Microsoft APIs: JDBC for data base access (equivalent to Microsoft’s ODBC), Swing for desktop GUI programming (equivalent to Microsoft’s MFC), and the framework that had the greatest influence on Java programming style, JavaBeans.

The JavaBeans API was Sun’s answer to Microsoft’s ActiveX component model for low-code, graphical, drag-and-drop programming. Windows programmers could use ActiveX components in their Visual Basic programs or embed them in office documents or web pages on their corporate intranet. Despite how easy it was to use ActiveX components, they were notoriously difficult to write. JavaBeans were much easier; you merely had to follow some additional coding conventions for your Java class to be considered a “bean” that could be instantiated and configured in a graphical designer. The promise of “Write once, run anywhere” meant you would also be able to use—or sell—JavaBean components on any operating system, not just Windows.

For a class to be a JavaBean, it needed to have a constructor that took no arguments, be serializable, and declare an API made up of public properties that could be read and optionally written, methods that could be invoked, and events that objects of the class would emit. The idea was that programmers would instantiate beans in a graphical application designer, configure them by setting their properties, and connect events emitted by beans to the methods of other beans. By default, the Beans API defined properties by pairs of methods whose names started with get and set. This default could be overridden, but doing so required the programmer to write more classes of boilerplate code. Programmers usually went to the effort only when retrofitting existing classes to act as JavaBeans. In new code, it was much easier to go with the grain.

The drawback of Beans style is that it relies heavily on mutable state and requires more of that state to be public than plain old Java objects do, because visual builder tools could not pass parameters to an object’s constructor, but instead had to set properties. User interface components work well as beans, because they can safely be initialized with default content and styling and adjusted after construction. When we have classes that have no reasonable defaults, treating them in the same way is error-prone, because the type checker can’t tell us when we have provided all the required values. The Beans conventions make writing correct code harder, and changes in dependencies can silently break client code.

In the end, graphical composition of JavaBeans did not become mainstream, but the coding conventions stuck. Java programmers followed the JavaBean conventions even when they had no intention of their class being used as a JavaBean. Beans had an enormous, lasting, and not entirely positive influence on Java programming style.

Enterprise Style

Java did eventually spread through the enterprise. It didn’t replace Visual Basic on the corporate desktop as expected, but rather unseated C++ as the server-side language of choice. In 1998, Sun released Java 2 Enterprise Edition (then known as J2EE, now JakartaEE), a suite of standard APIs for programming server-side, transaction processing systems.

The J2EE APIs suffer from abstraction inversion. The JavaBeans and applets APIs also suffer from abstraction inversion—they both disallow passing parameters to constructors, for example—but it is far more severe in J2EE. J2EE applications don’t have a single entry point. They are composed of many small components whose lifetime is managed by an application container, and are exposed to one another through a JNDI name service. Applications need a lot of boilerplate code and mutable state to look up the resources they depend on. Programmers responded by inventing dependency injection (DI) frameworks that did all the resource lookup and binding, and managed lifetimes. The most successful of these is Spring. It builds upon the JavaBeans coding conventions and uses reflection to compose applications from Bean-like objects.

In terms of programming style, DI frameworks encourage programmers to eschew direct use of the new keyword and instead rely on the framework to instantiate objects. The Android APIs also exhibit abstraction inversion, and Android programmers also turn to DI frameworks to help them write to the APIs. DI frameworks’ focus on mechanism over domain modeling leads to enterprisey class names such as Spring’s infamous AbstractSingletonProxyFactoryBean.

On the plus side, though, the Enterprise Era saw the release of Java 5, which added generics and autoboxing to the language, the most significant change to date. This era also saw a massive uptake of open source libraries in the Java community, powered by the Maven packaging conventions and central package repository. The availability of top-notch open source libraries fueled the adoption of Java for business-critical application development, and led to more open source libraries, in a virtuous circle. This was followed by best-in-class development tools, including the IntelliJ IDE, which we use in this book.

Modern Style

Java 8 brought the next big change to the language—lambdas—and significant additions to the standard library to take advantage of them. The Streams API encouraged a functional programming style, in which processing is performed by transforming streams of immutable values rather than changing the state of mutable objects. A new date/time API ignored JavaBeans coding conventions for property accessors and followed coding conventions common to the Primeval Age.

The growth of the cloud platforms meant that programmers didn’t need to deploy their servers into JavaEE application containers. Lightweight web application frameworks let programmers write a main function to compose their applications. Many server-side programmers stopped using DI frameworks—function and object composition were good enough—so DI frameworks released greatly simplified APIs to stay relevant. With no DI framework or mutable state, there’s less need to follow JavaBean coding conventions. Within a single codebase, exposing fields of immutable values works fine, because the IDE can encapsulate a field behind accessors in an instant if they’re needed.

Java 9 introduced modules, but so far they have not seen widespread adoption outside the JDK itself. The most exciting thing about recent Java releases has been the modularization of the JDK and removal of seldom-used modules, such as CORBA, from the JDK into optional extensions.

The Future

The future of Java promises more features to make Modern Style easier to apply: records, pattern matching, user-defined value types, and eventually the unification of primitive and reference types into a uniform type system.

However, this is a challenging effort that will take many years to complete. Java started off with some deep-seated inconsistencies and edge cases that are hard to unify into clean abstractions while staying backward compatible. Kotlin has the benefit of 25 years of hindsight, and a clean slate from which to start afresh.

The Grain of Kotlin

Kotlin is a young language, but it clearly has a different grain than Java.

When we wrote this, the “Why Kotlin” section of the Kotlin home page gave four design goals: concise, safe, interoperable, and tool-friendly. The designers of the language and its standard library also encoded implicit preferences that contribute to these design goals. These preferences include:

Kotlin prefers the transformation of immutable data to mutation of state.

Data classes make it easy to define new types with value semantics. The standard library makes it easier and more concise to transform collections of immutable data than to iterate and mutate data in place.

Kotlin prefers behavior to be explicit.

For example, there is no implicit coercion between types, even from smaller to larger range. Java implicitly converts int values to long values, because there is no loss of precision. In Kotlin, you have to call Int.toLong() explicitly. The preference for explicitness is especially strong when it comes to control flow. Although you can overload arithmetic and comparison operators for your own types, you cannot overload the shortcut logical operators (&& and ||), because that would allow you to define different control flow.

Kotlin prefers static over dynamic binding.

Kotlin encourages a type-safe, compositional coding style. Extension functions are bound statically. By default, classes are not extensible and methods are not polymorphic. You must explicitly opt in to polymorphism and inheritance. If you want to use reflection, you have to add a platform-specific library dependency. Kotlin is designed from the outset to be used with a language-aware IDE that statically analyzes the code to guide the programmer, automate navigation, and automate program transformation.

Kotlin doesn’t like special cases.

Compared to Java, Kotlin has fewer special cases that interact in unpredictable ways. There is no distinction between primitive and reference types. There is no void type for functions that return but do not return a value; functions in Kotlin either return a value or never return at all. Extension functions allow you to add new operations to existing types that look the same at the call point. You can write new control structures as inline functions, and the break, continue, and return statements act the same as they do in built-in control structures.

Kotlin breaks its own rules to make migration easier.

The Kotlin language has features to allow idiomatic Java and Kotlin code to coexist in the same codebase. Some of those features remove guarantees provided by the type checker and should only be used to interoperate with legacy Java. For example, lateinit opens a hole in the type system so that Java dependency injection frameworks that initialize objects by reflection can inject values through the encapsulation boundaries that are normally enforced by the compiler. If you declare a property as lateinit var, it’s up to you to ensure the code initializes the property before reading it. The compiler will not catch your mistakes.

When we, Nat and Duncan, revisit the earliest code we wrote in Kotlin, it tends to look like Java dressed in Kotlin syntax. We came to Kotlin after years writing a lot of Java and had ingrained habits that affected how we wrote Kotlin code. We wrote unnecessary boilerplate, didn’t make good use of the standard library, and avoided using null because we weren’t yet used to the type checker enforcing null safety. The Scala programmers on our team went too far the other way—their code looked like Kotlin trying to be Scala, cosplaying as Haskell. None of us had yet found the sweet spot that comes from working with the grain of Kotlin.

The path to idiomatic Kotlin is complicated by the Java code we have to keep working along the way. In practice, it is not enough just to learn Kotlin. We have to work with the different grains of Java and Kotlin, being sympathetic to both as we gradually transition from one to the other.

Refactoring to Kotlin

When we started our journey to Kotlin, we were responsible for maintaining and enhancing business-critical systems. We were never able to focus only on converting our Java codebase to Kotlin. We always had to migrate code to Kotlin at the same time as changing the system to meet new business needs, maintaining a mixed Java/Kotlin codebase as we did so. We managed the risk by working in small changes, making each easy to understand and cheap to discard if we found out it broke something. Our process was first to convert Java code to Kotlin, giving us a Java-esque design in Kotlin syntax. We then incrementally applied Kotlin language features to make the code increasingly easy to understand, more type safe, more concise, and with a more compositional structure that is easier to change without unpleasant surprises.

Small, safe, reversible changes that improved the design: we refactored from idiomatic Java to idiomatic Kotlin.

Refactoring between languages is usually harder than refactoring within a single language because refactoring tools do not work well across the boundaries between the languages, if they work at all. Porting logic from one language to another must be done manually, which takes longer and introduces more risk. Once multiple languages are in use, the language boundary impedes refactoring because when you refactor code in one language, the IDE does not update dependent code written in other languages to be compatible.

What makes the combination of Java and Kotlin unique is the (relatively) seamless boundary between the two languages. Thanks to the design of the Kotlin language, the way it is mapped to the JVM platform, and JetBrains’ investment in developer tooling, refactoring Java to Kotlin and refactoring a combined Java/Kotlin codebase is almost as easy as refactoring in a single codebase.

Our experience has been that we can refactor Java to Kotlin without affecting productivity, and that productivity then accelerates as we convert more of the codebase to Kotlin.

Refactoring Principles

The practice of refactoring has come a long way since its initial popularization in Martin Fowler’s book, Refactoring: Improving the Design of Existing Code (Addison-Wesley), published in 1999. This book had to detail manual steps for even simple refactorings like renaming identifiers but notes that some state-of-the-art development environments were beginning to provide automated support to reduce such drudgery. Nowadays we expect our tools to automate even complicated scenarios such as extracting an interface or changing function signatures.

These individual refactorings rarely stand alone though. Now that the building-block refactorings can be performed automatically, we have the time and energy to combine them to make larger-scale changes to our codebase. When the IDE does not have distinct user-interface actions for a large-scale transformation we wish to do, we have to perform it as a sequence of more granular refactorings. We use the IDE’s automatic refactoring whenever we can, and fall back on text editing when the IDE does not automate a transformation we need.

It’s tedious and error-prone to refactor by editing text. To reduce the risk, and our boredom, we minimize the amount of text editing we have to do. If we must edit text, we prefer that edit to affect a single expression. So we use automatic refactorings to transform the code so that is possible, edit one expression, and then use automatic refactorings to tidy back up to the final state we’re aiming for.

The first time we describe a large-scale refactoring, we’ll go through it step by step and show how the code changes at each step. This takes quite a lot of space on the page and will take a bit of reading time to follow. In practice, however, these large refactorings are quick to apply. They typically take a few seconds, a few minutes at most.

We expect the refactorings published here to date quite quickly as tools improve. The individual IDE steps may be renamed, and some combinations might be implemented as single refactorings in their own right. Experiment in your context to find ways of gradually and safely transforming your code that are better than those we present, and then share them with the world too.

We Assume Good Test Coverage

As Martin Fowler says in Refactoring: Improving the Design of Existing Code: “[I]f you want to refactor, the essential precondition is having solid tests.” Good test coverage ensures that the code transformations we want to only improve design have not inadvertently changed our system’s behavior. In this book, we assume that you have good test coverage. We do not cover how to write automated tests. Other authors have addressed these topics in more detail than we could in this book, for example: Test-Driven Development By Example by Kent Beck (Addison-Wesley) and Growing Object-Oriented Software Guided By Tests by Steve Freeman and Nat Pryce (Addison-Wesley). We do, however, show how to apply Kotlin features to improve our tests.

As we walk through multistep code transformations, we won’t always say when we run the tests. Assume that we run our tests after every change that we show that compiles, no matter how small.

If your system does not already have good test coverage, it can be difficult (and expensive) to retrofit tests to the code because the logic you want to test is entangled with other aspects of the system. You’re in a chicken and egg situation: you have to refactor to be able to add tests so that you can safely refactor. Again, other authors have addressed these topics in more detail than we could, for example: Working Effectively with Legacy Code by Michael Feathers (Pearson).

We’ve listed more books about these topics in the Bibliography.

We Commit for Git Bisect

Just as we don’t explicitly state when we run our tests, nor do we explicitly state when we commit our changes. Assume we commit our changes whenever they have added value to the code, no matter how small.

We know our test suite isn’t perfect. If we accidentally break something that is not caught by our tests, we want to find the commit that introduced the fault and fix it as quickly as we can.

The git bisect command automates that search. We write a new test that demonstrates the error, and git bisect does a binary search of the history to find the first commit that makes that test fail.

If the commits in our history are large, and contain a mishmash of unrelated changes, git bisect won’t help as much as it could. It cannot tell which of the source changes within a commit introduced the error. If commits mix refactoring and changes to behavior, reverting a bad refactoring step is likely to break other behavior in the system.

Therefore, we commit small, focused changes that separate refactorings from each other, and from changes to behavior, to make it easy to understand what changed and fix any erroneous change. For the same reason, we very rarely squash commits.

Note

We prefer to commit changes straight onto the mainline branch—“trunk-based development”—but changing code in a sequence of small, independent commits is just as beneficial when working in branches and merging less frequently.

What Are We Working On?

In the chapters that follow, we take examples from the codebase of Travelator, a fictional application for planning and booking international surface travel. Our (still fictional) users plan routes by sea, rail, and road; search for places to stay and sights to see; compare their options by price, time, and spectacle; and finally book their trips, all from web and mobile frontends that invoke backend services via HTTP.

Each chapter pulls an informative example from a different part of the Travelator system, but they share common domain concepts: money, currency conversion, journeys, itineraries, bookings, and so on.

Our aim is that, like our Travelator application, this book will help you plan your journey from Java to Kotlin.

Let’s Get Started!

Enough chitchat. You’re probably itching to convert all that Java to Kotlin. We’ll start in the next chapter by adding Kotlin support to our project’s build file.

Get Java to Kotlin now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial