Dependencies are a formidable challenge, whether they are dependencies in a Java build, in a Ruby project, or in any other kind of system. For one thing to depend on another means accepting the liability that the depended-on thing might not be available when we need it, might be expensive to obtain, or might not work in the way we expected. Programmers are painfully familiar with these costs.
Dependencies can seem like an annoyance—indeed, much of the undeserved criticism commonly directed against Maven focuses on dependency management—but they are an unavoidable fact of a complex ecosystem and are a sign that the actors in that ecosystem are trying to divide up their labor in worthwhile ways. Therefore, Gradle embraces dependency management.
We will consider the problem of dependency management primarily from a Java perspective, since the Java community has excelled both in creating an enormous dependency management problem, and in solving it effectively. Java’s early embrace of open source caused thousands of software modules to proliferate, and typical projects soon came to depend on dozens of these individual modules. The modules, in turn, depend heavily on one another.
Of course, Java dependency management solutions also apply to related languages of the JVM. Groovy and Scala builds will apply the exact same techniques shown in this chapter.
You may have worked with a build that constructed the compile classpath for your compile step by blindly grabbing all the JAR files in a certain directory (usually called “lib”). Inevitably, this becomes a problem over time as the software grows. It becomes more and more costly to change or remove dependencies as it becomes increasingly difficult to determine the impact of doing so. By managing (i.e., declaring) your dependencies as part of the definition of how your software builds, the dependencies can be understood and analyzed. Having your JAR files in the lib directory is not the problem. It’s that they are being used indiscriminately and that the relationships between them are completely opaque. Dependency management solves this problem.
It is common to see “dependency management” inaccurately equated with the automated fetching of dependencies from a remote source or even the automated fetching of transitive dependencies (i.e., dependencies of dependencies). This is a benefit of dependency management, not its essence. By declaring and modelling dependencies, tooling such as Gradle can automate working with dependencies by leveraging this information. This includes automatically fetching dependencies, detecting transitive dependency conflicts, and so on.
Gradle embraces dependency management at its core and provides excellent support for dependency automation.
In a Gradle build script, you declare dependencies via a DSL (Example 4-1).
Example 4-1. An example of declaring a dependency
repositories{mavenCentral()}configurations{compile}dependencies{compile'org.springframework:spring-core:3.0.5'}
The preceding example shows three key concepts to Gradle dependency management: configurations (compile), dependencies (org.springframework:spring-core:3.0.5), and repositories (mavenCentral()). A “configuration” is a named grouping of dependencies. A Gradle build can have zero or more of them. A “repository” is a source of dependencies. Dependencies are often declared via identifying attributes, and given these attributes, Gradle knows how to find a dependency in a repository.
In a typical Gradle build, configuration declaration is done implicitly, and configurations fade in to a background role whose details are managed by plug-ins. However, you will not fully understand Gradle dependency management without understanding configurations thoroughly, so we’ll begin with them.
Configurations are fundamentally named buckets of files that are filled up with dependencies. A Configuration is a special type of FileCollection. Recall that a FileCollection is a lazy specification of files that when queried (i.e., when the actual files are asked for), turns that specification into a concrete list of files. Configurations are used in a similar way, except that rather than being a specification of plain files on the filesystem, they are a specification of “dependencies” that may exist locally, on the network, or in some other abstract location. Different kinds of dependencies are resolved into files in different ways. While this is the fundamental role of configurations, they also provide methods for querying the declared dependencies and customizing the resolution process.
The Java plug-in introduces six configurations:
-
archives -
default -
compile -
runtime -
testCompile -
testRuntime
Note
The archives and default configurations are actually created by the base plug-in, that is itself implicitly applied by the java plug-in. We can ignore this detail for the rest of this discussion.
The compile configuration contains all dependencies required to compile the code. When the Java plug-in’s compileJava task invokes the javac compiler, it must provide a classpath. In a completely imperative build in which it’s your job to build up all those command-line parameters by hand (such as might be the case in a typical Ant build), you’d have to collect all of the JAR files the compiler might need to resolve import statements in the compiled code. Assigning compile-time dependencies to the compile configuration accomplishes this same goal, since the compile classpath used by compileJava task is formed internally from the compile configuration (Example 4-2).
Example 4-2. Declaring the default configurations
configurations{compileruntimetestCompile.extendsFrom('compile')testRuntime.extendsFrom('runtime','testCompile')default.extendsFrom('runtime')}
Automated tests have different dependency requirements than the main set of source files. They will typically need every single external dependency required to compile the main sources, and will need several other modules in addition: a unit testing framework, a mocking framework, a test-friendly database driver, or others in this vein. Not only that, but automated tests also depend on the compiled output of the main source set. The Java plug-ins introduces the testCompile configuration to handle this collection of files. When the compileTestJava task runs the javac compiler, it looks to this configuration to set the compiler’s classpath.
Often, projects will depend on modules that must be available at runtime, but need not be fetched or made available on the compile-time classpath. A common example is a JDBC driver, in which the application code is compiled against interfaces in the Java standard libraries, while the implementation is provided by a vendor. The java plug-in provides the runtime and testRuntime configurations for this purpose. These are used to create deployment archives and execute test code, but do not add to the compile-time classpath at all. The runtime configuration does contain the output of the compileJava task, of course, since a project’s .class files are necessarily required to run the project.
The default configuration is almost never directly used in a build, but participates in project-level dependency declaration. When one build depends on another build as a subproject, Gradle’s default behavior is to include all of the files in this configuration. It extends runtime by default.
Note
The Gradle war plug-in adds providedCompile and providedRuntime configurations that can be used to achieve the same effect as the Apache Maven dependency scope of provided. See the Gradle user guide for more information.
When we say that default extends runtime as shown in Example 4-2, we might intuitively understand the mechanism, but it’s still worth some explanation. To see the details, let’s look back at another case: how the testCompile configuration provides dependencies to the compileTestJava task. When compiling unit tests, we are likely to need access to every single compile-time dependency that the main source code needs, plus several other things (e.g., non-production mocks or stubs, a testing framework, etc.). Strictly speaking, testCompile is a superset of compile. Gradle supports this relationship by allowing one configuration to extend another.
You can see the configuration extension syntax in Example 4-2. Calling the extendsFrom() method on a configuration indicates that it will contain all of the files of the extended configuration automatically, plus any other files you add to the extending configuration explicitly.
The “extends” relationship between configurations is entirely mutable. If for some reason you do not want the runtime configuration to extend from the compile configuration you can simply remove it from the extendsFrom collection (Example 4-3).
Now that we have a scheme for organizing dependencies, let’s turn our attention to the most common kind of dependency in a Gradle build: the module. Module dependencies are JAR files built by some other build external to the project in question. Various open-source Java projects are good examples of modules, like Commons Logging, Hibernate, or the Spring Framework.[16]
Modules are typically identified by a vector of three parameters: group, name, and version. The group specifies the organization responsible for the module, and is often—but not always—a reverse domain name (like org.apache.solr, or org.springframework, or an outlier like junit). The name is a unique label for the module itself, and is often the same as the project name (like solr-core, spring-web, or junit). The version identifies which release of the project you are depending on (like 1.4.1, 3.0.2, or 4.8). These three vector elements can be expressed together in a string delimited by colons, as shown in Example 4-4.
Example 4-4. Declaring dependencies with group-name-version strings
dependencies{compile'org.apache.solr:solr-core:1.4.1'compile'org.springframework:spring-core:3.0.5'testCompile'junit:junit:4.8'}
Alternatively, the three elements can be expressed as a Groovy map, with their names explicitly called out. Example 4-5 illustrates this format. Note that only the name field is required for a dependency declaration; group and version are optional.
Example 4-5. Declaring dependencies with Groovy map syntax
dependencies{compilegroup:'org.apache.solr',name:'solr-core'version:'1.4.1'compilegroup:'org.springframework',name:'spring-core',version:'3.0.5'testCompilegroup:'junit',name:'junit',version:'4.8'}
By default, Gradle dependencies are transitive. Transitive dependencies are described more in Maven Repositories, but for now, all you need to know is that one module may depend on other modules, and Gradle can discover those dependencies-of-dependencies when it resolves the declared dependency against a repository. This is almost always an enormous time-saver, but sometimes it can create problems. If you depend on version 1 of module A and version 2 of module B, and module A transitively depends on version 3 of module B, you may not want Gradle to resolve that final dependency. The wrong version of a JAR file might make it into your compile or runtime classpath—and most Java developers know what a frustrating experience that can be.
Happily, you can alter transitive dependency resolution in three separate ways: you can disable it altogether (Example 4-6), you can disable individual transitive dependencies from being resolved (Example 4-7), and you can force a particular dependency to be the victor when two versions are in conflict (Example 4-8). Let’s take a look at the syntax for each of these.
Example 4-6. Fetching Spring Core 3.0.6 without any of its dependencies
dependencies{compile('org.springframework:spring-core:3.0.6'){transitive=false}}
Specifying a particular version like 3.0.6 in a dependency declaration guarantees that Gradle downloads the same module every time it runs the build, assuming that all repositories are properly maintained. This is bedrock functionality, but sometimes we’d like to make a minor adjustment to the way it works: sometimes we want Gradle to download the most recent version it can find of a given module.
Using dynamic dependencies can be risky, since a new release to a declared repository will cause a future execution of the build to produce different output artifacts—even though the build’s sources didn’t change at all. However, build masters may decide to use this technique for principled reasons, and Gradle supports it fully (Example 4-9). This feature interacts with the dependency cache in important ways. You should understand the cache well if you plan on using dynamic versions.
Declaring a dependency on a Maven-style group:name:version vector implies that the module is probably going to be resolved against some external repository like Maven Central. However, even in a conventional project where most dependencies are managed in this way, you might still come up with a module you need to include in your build that is not available in any managed repository. It might be a vendor JAR that is only available by direct download, or a binary patch of a module that you can’t conveniently deploy to any internal binary artifact repository. Regardless of the reason, Gradle is ready to help with file dependencies.
Since a dependency configuration is fundamentally a collection of files, it’s easy enough to add more files to it by brute force. In this situation, you are likely to create a lib directory at the root of your project and place the files in there (Example 4-10).
Example 4-10. Declaring a dependency explicitly on a locally managed module
dependencies{compilefiles('lib/hacked-vendor-module.jar')}
Of course, this practice may get out of hand, and you may soon find that you have a whole lib directory full of modules, just like the pattern typically followed by Ant builds. This isn’t necessarily to be encouraged, but you can grab all of those files in one declaration, even if you’ve created a nested directory structure under the top-level lib directory (Example 4-11).
Separate builds in a multi-project build may depend on one another. A multi-project build is an arbitrary configuration of projects that are all evaluated together by Gradle at build time. Each build’s Project object is related to other builds’ Project objects in a project graph. Gradle doesn’t prescribe the exact nature of these relationships, but rather provides you with a syntax to express them. When you are expressing a relationship between projects, you are really indicating that one project depends on another.
Looking at the multiproject example code, you recall that the project was structured as one top-level command line driver utility, plus one subproject that provided text content, and another that provided text encoding services. In this multiproject system, the top-level project depended on the two subprojects, and each of those subprojects was fully independent of each other. Note Example 4-12, and the section of the build file in the top-level project.
Example 4-12. Declaring dependencies on subprojects
dependencies{compileproject(':codec')compileproject(':content')}
The code in Example 4-12 indicates that the top-level project has a compile-time dependency on both the :codec and the :content subprojects. Of course, a project is not a collection of files, and a dependency must eventually resolve to a file collection in order to be added to a named configuration like compile or runtime. When you depend on a project as shown in the example, Gradle assumes you mean the default configuration, which includes all of the compiled output of the project and all of its compile-time and runtime module dependencies. The default dependency configuration is covered in more detail in Configurations.
Of course, you might not want to grab all of the files in default. You might have designed a subproject’s dependency configurations intentionally to publish only certain files to external projects. In Example 4-13, the project() method takes a map with parameters for the project path and the desired dependency configuration.
The Gradle dependency DSL offers a couple of convenience methods that are particularly useful when writing code to extend Gradle itself. These are the gradleApi() method and the localGroovy() method.
If you are writing custom tasks or a custom plug-in, you will be programming Gradle’s APIs directly, and will therefore have a compile-time dependency on them. This sort of dependency is similar in principle to the compile-time dependency you might have on any API you are programming, like what you might see if you are writing MapReduce jobs for execution on a Hadoop cluster or writing enterprise integration code with Apache Camel.
However, since the classes you are importing happen to be available in the same Gradle runtime package that’s interpreting and executing the build, Gradle provides a method you can use to access those classes directly without having a true external dependency (Example 4-14).
Example 4-14. Depending on the Gradle API
dependencies{compilegradleApi()// other dependencies...}
Also, while you can extend Gradle in any one of a variety of JVM languages, many Gradle developers have historically preferred do so in Groovy. Plug-ins and custom tasks written in Groovy share a common syntax with Gradle build files, and as a high-productivity dynamic language, Groovy serves the build developer well.
Gradle’s Groovy plug-in makes building code in Gradle easy, but it does require one extra step: configuring a dependency on a particular version of Groovy (Example 4-15). This is normally done by assigning the dependency vector of a particular version of Groovy to a special dependency configuration called groovy.
Example 4-15. The configuration of a typical Groovy build
applyplugin:'groovy'// Repository declarationdependencies{groovy'org.codehaus.groovy:groovy:2.0.5'// other dependencies...}
If you’re developing code to extend Gradle itself, you don’t have to declare a particular version of Groovy to use. Since Gradle’s build files are Groovy scripts, every Gradle installation already contains a version of Groovy. Your custom task, plug-in, or other Gradle extension will necessarily run using the same version of Groovy as the version of Gradle for which you are writing the extension. Gradle therefore provides the localGroovy() method to make that local Groovy version available to your build without creating a true external dependency (Example 4-16).
Dependencies are, by definition, things external to the build. Once they are declared, this means Gradle must go get them from somewhere. In Gradle, the place dependencies are fetched from is called a repository.
Every Gradle project has an internal list of repositories. By default, this list is empty, but we can add to it by using the repositories block. Gradle uses the repositories at build execution time to fetch the build’s dependencies and store them in a cache. There are three kinds of repositories currently supported by Gradle: Maven repositories, Ivy repositories, and static directories. We’ll describe each one in turn, paying particular attention to its configuration DSL.
One of Maven’s most valuable innovations was the Maven Central repository. Not only does it serve as a publicly accessible repository of thousands of open-source Java modules, but its structure and access protocol are open as well. Many other Maven-style repos exist on the Internet and inside corporate networks all over the world. Gradle supports these as first-class citizens.
As described in the section on module dependencies, artifacts in a Maven repository are usually identified by three coordinates: group, name, and version. This self-describing coordinate tuple was originally introduced to the Java ecosystem by the Maven repository format. In addition, modules may be qualified by their classifier and type. The classifier usually differentiates between the code, JavaDocs, and source archives. The type indicates whether the module is packaged as a JAR or some other archive format. classifier and type are often omitted from declarations, but are a supported part of the Maven repository format.
The primary value of Maven repositories is not that they offer a standard means of storing and retrieving executable code, source, and documentation over the Internet, but that they describe the dependencies between those modules as well. Each Maven coordinate, whether it corresponds to any code, source, or JavaDocs at all, always describes an XML descriptor file called a POM (Project Object Model). The POM file contains various metadata about the module, not least is the list of other modules required for the POM’s module to function. By default, Gradle recursively fetches these dependencies from the repository and caches them on disk.
Since a Maven repository is nothing more than a website containing downloadable POM files and modules in a predictable directory structure, declaring a Maven repository in Gradle begins with telling Gradle the URL of the repo. The most common Maven repository is the Central repository at http://repo1.maven.org/maven2/; however, Gradle provides a convenience method to avoid scripting that URL directly into your build (Example 4-17).
Central may be the most-frequently used Maven repository, but it is hardly the only one. Many other open-source Maven repositories exist on the Internet, and still more exist within the private networks of companies using Gradle for their enterprise builds. Sonatype’s Nexus and JFrog’s Artifactory are two examples of products that can be used to deploy Maven-style binary repositories internally. Declaring such a repository in Gradle is a simple matter of providing the URL (Example 4-18).
Example 4-18. Declaring Maven repositories in general
repositories{maven{url='http://download.java.net/maven/2'}maven{name='JBoss Repo'//optional nameurl='https://repository.jboss.org/nexus/content/repositories/releases'}}
A Maven repository is merely a website that serves POMs and artifacts through a well-defined URL naming scheme. In some cases, we might want to obtain POMs from one website (say, a centralized corporate repo), but download build artifacts from a mirrored copy of the repo somewhere else in the network (Example 4-19). We might be motivated to do this to reduce bandwidth consumption or increase download speeds between the build server and the main repository. Regardless of our reason, Gradle makes this easy.
Example 4-19. Setting the artifact URL as distinct from the default POM URL
repositories{// Overriding artifacts for an internal repomaven{url='http://central.megacorp.com/main/repo'artifactUrls=['http://dept.megacorp.com/local/repo']}// Obtain Maven Central artifacts locallymavenCentralartifactUrls:['http://dept.megacorp.com/maven/central']}
Maven famously caches all of its dependencies in the ~/.m2 directory, which is sometimes called the local Maven repository. Gradle maintains its own caching mechanism, but some builds might want to be able to dip into Maven’s local cache to resolve dependencies. The most common use case for this is when a build needs to access a custom version of some other module, especially when that customization is itself under development and is not ready to publish to the outside world. Asking Gradle to attempt to resolve dependencies against the local Maven repo is simple (Example 4-20).
Example 4-20. Resolving dependencies against the ~/.m2 local Maven cache
repositories{mavenLocal()}
Maven repositories can also host snapshots, or what Gradle calls changing modules. A changing module is one whose version is presently in a state of flux, and can’t meaningfully be pegged to a single version number. Changing modules can represent code that is under active development during the workday, or code that is being rebuilt nightly given whatever state the code happens to be in at the end of the day.[17] They are still managed dependencies, but because they are subject to change, their caching semantics are different.
Normally Gradle caches a module once and never reloads it from its source repository again, since versioned modules are supposed to be immutable. However, changing modules explicitly disclaim immutability, so cached copies of those modules must be expired after some time. By default, changing modules are refetched after 24 hours in the cache. You can override this default by configuring the global dependency resolution strategy.
Apache Ivy was developed as an extension to the Ant build tool to provide Maven-style binary artifact management. It was never deployed as widely as Maven-style repos have been, but it sees some continued use at the time of this writing in connection with enterprise Ant builds.
In principle, Ivy repositories function very much like Maven repositories. Artifacts are identified by a vector composed of a group name, a module name, and a version. Repositories reside either at HTTP-addressable URLs or in the file system. Repository metadata, held in ivy.xml files, expresses transitive dependency relationships. And of course, declaring an Ivy repository in Gradle is trivial (Example 4-21).
Example 4-21. Declaring an Ivy repository using the default Maven layout
repositories{ivy{url'http://build.megacorp.com/ivy/repo'}}
Ivy differs from Maven in that its repositories do not always follow the same directory structure as one another. Maven repos have a fixed structure that follows the group, artifact, or version hierarchy, but Ivy repos may be organized along different lines. If we provide no artifact or Ivy mappings (as in Example 4-21), Gradle assumes that the Ivy repo uses the default Maven structure. However, we can define a custom layout easily enough.
Example 4-22 shows a repository that hosts all of its artifacts and ivy.xml files at a particular URL, but locates the JAR and Ivy files at different paths on the server. The portions inside square brackets (e.g., [module], [artifact], etc.) are substituted for the value of the dependency being resolved. The strings in Table 4-1 are supported.
Table 4-1. The mapping of Ivy repository variables to dependency parameters
| Ivy repository variable | Dependency parameter |
|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Example 4-22. Declaring an Ivy repository using a named custom layout
repositories{ivy{url'http://build.megacorp.com/ivy/repo'layout'custom',{artifact"artifacts/[module]/[revision]/[artifact].[ext]"ivy"ivy/[module]/[revision]/ivy.xml"}}}
Moreover, a single Ivy repo may place its ivy.xml files and its binary modules at entirely separate URLs. If there is no common base URL between artifacts and Ivy files, use the syntax in Example 4-23.
Example 4-23. Declaring an Ivy repository using custom artifact and ivy.xml URLs
repositories{ivy{artifactPattern"http://build.megacorp.com/ivy/repo/artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"ivyPattern"http://build.megacorp.com/ivy/repo/ivy-xml/[organisation]/[module]/[revision]/ivy.xml"}}
Most Internet-based binary repositories exist to serve open-source artifacts, so they are open to everyone without authentication. Likewise, most corporate repositories exist inside trusted networks where company employees can access them freely. However, sometimes repositories have to be secured with basic username and password authentication. Maven and Ivy repositories both have mechanisms for doing so, and so Gradle has a common configuration DSL to provide these credentials in the build (Example 4-24).
Example 4-24. Adding credentials to a repository declaration, insecurely
repositories{maven{url='http://central.megacorp.com/main/repo'credentials{username'repouser'password'badly-protected-secret'}}}
The problem with the naive implementation is that the password is in the build script in plain text, and is therefore checked into source control. This is, putting it mildly, not prudent password management. A better option is to put the password in an environment variable or an external properties file. Gradle automatically reads the gradle.properties file at startup, making build script variables out of each of the properties defined therein (Example 4-25). This is a convenient place to put passwords, as long as there is no other configuration in the file that needs to be checked into source control (Example 4-26).
Example 4-25. An example gradle.properties file containing a password
megacorpRepoPassword=well-protected-secret
Example 4-26. Adding credentials to a repository declaration, accessing the password through a properties file
repositories{maven{url='http://central.megacorp.com/main/repo'credentials{username'repouser'passwordmegacorpRepoPassword}}}
Using gradle.properties is viable only if it can be kept out of source control though a mechanism like the .gitignore file or the svn:ignore property, or a similar technique appropriate for the version control system in use. If you can’t afford to ignore this file due to other content you have decided to place in it, you can create a custom properties file and read it explicitly in your build script using the java.util.Properties API and Gradle’s native Groovy scripting.
Most contemporary open-source and enterprise builds use transitive dependency management through Maven-style repos, but not every build in the world does. An older, Ant-based system might use static dependency management, where a single lib directory holds all of the JAR files the build needs. This might not be the preferred practice for new designs, but a prudent build migration process might only choose to attack one problem at a time, preferring to migrate core build functionality to Gradle before redesigning dependency management as well.
Other builds might use managed dependencies as much as possible, but occasionally fall back to static management for pragmatic reasons. For example, if you need to hack an old, unsupported vendor JAR at the bytecode level, or otherwise have a highly custom build of some external module, it might make more sense to control that JAR with your source code rather than publish it to a binary repository. This scenario is most likely when you don’t have your own internal binary repository and module publishing mechanism established.
Whether you are managing static dependencies due to old technology or pragmatic necessity, Gradle is ready to support you. It is able to resolve dependencies against a local directory, just as if that directory were a source of Maven- or Ivy-style artifacts. Example 4-27 shows how to declare a flat directory repository.
Example 4-27. Declaring a flat directory repository
repositories{flatDirdirs:"${projectDir}/lib"}
That declaration will cause Gradle to look in a directory called lib located at the project root, using the idiom common to the Ant builds of a decade ago. However, this simple declaration raises the question of how Gradle will construct filenames, since the group, name, and version metadata is now missing from the ad-hoc “repository.” The answer is that Gradle builds filenames using the [group]-[name]-[version]-[classifier].[type] formula. Some examples of how dependency declarations would be converted into strings are in Table 4-2.
Table 4-2. Examples of dependency declaration to flatDir filename mappings
| Dependency declaration | Filename |
|---|---|
|
|
|
|
|
|
|
|
As you can see, the more compact 'group:name:version' declaration, which is common when resolving dependencies against Maven repositories, might not serve as well in the flatDir case. If you have versions in your JAR filenames, you will most often use the combination of name and version in your dependency declarations. If you don’t have versions in your JAR filenames, declare your dependencies with only a name attribute, and they’ll resolve properly against the flat dir.
In rare cases, you might have multiple directories of static files you’ll want Gradle to check when it’s resolving dependencies. The syntax for declaring multiple directories is shown in Example 4-28. Note that this example also shows how to apply a name to a flatDir repo, just like we have with Maven and Ivy declarations.
Most dependency and repository declarations in a build are concerned with the dependencies of the code being built: what modules are required to compile the code, to compile the tests, to run the code, and so forth. However, since a Gradle build is a Groovy program, we might choose to introduce external dependencies into the build script itself.
Let’s consider an example. Suppose we have a build that has a collection of Markdown files in it, and one of the build’s tasks is to convert them into HTML. This is a perfect job for a Copy task with a filter attached to it, but that filter has a lot of work to do! It has to be a full-featured Markdown parser and HTML renderer. We definitely don’t want to write that code ourselves.
Happily, the MarkdownJ project will do it for us, but now we need MarkdownJ classes to be available to us not during our project’s compilation or execution, but during the execution of the build script itself. The resulting build looks like Example 4-29.
Example 4-29. Using buildscript to add to the build classpath
buildscript{repositories{mavenCentral()}dependencies{classpath'org.markdownj:markdownj:0.3.0-1.0.2b4'}}importcom.petebevin.markdown.MarkdownProcessorclassMarkdownFilterextendsFilterReader{MarkdownFilter(Readerinput){super(newStringReader(newMarkdownProcessor().markdown(input.text)))}}taskmarkdown(type:Copy){from'src/markdown'include'*.md'into'build/labs'rename{it-'.md'+'.html'}filterMarkdownFilter}
Note that the repositories and dependencies declarations inside the buildscript block are identical to their conventional uses as described earlier in this chapter. The one difference is the new dependency configuration: classpath. This is the only configuration you’ll use inside the buildscript block. It indicates that the dependencies you’re providing will be available to the classloader during the rest of the build script execution, which is exactly what we need. The line following the buildscript block is an import of the MarkdownProcessor class, which is later used to configure the copy task’s filter. We could bring any resolvable modules we’d like into the build in this same way.
We can apply this same technique to load external plug-ins. As described in Chapter 2, plug-ins are merely JARs containing code that programs the Gradle plug-in API. To apply a plug-in to a Gradle build, the plug-in has to be available in the classpath of the build itself. If a plug-in is properly packaged and deployed to a binary repository like Central, we can use the buildscript block to make it visible to the build script (Example 4-30).
It is one thing to be able to declare dependencies with a concise syntax and resolve them against a variety of artifact repositories. It is another thing entirely to do this while retaining the ability to execute builds quickly or run them when disconnected from the network. To support these goals, Gradle offers a sophisticated dependency cache, containing features no dependency management system before it has offered.
Binary repository formats like Ivy and Maven expose both metadata (e.g., pom.xml and ivy.xml files) and binary artifacts themselves (e.g., JAR files, source archives, JavaDoc archives, etc.). In an important innovation, Gradle caches the metadata and artifacts separately. This allows the dependency cache to avoid a significant category of build problems that can lead to non-repeatable builds in enterprise environments and elsewhere.
The trivial implementation of dependency caching would group all downloaded content by its identifier, which is ostensibly the standard group:artifact:version vector used to declare the dependency. However, the naive approach ignores the case in which different binary modules are uploaded to separate repositories with the same version number. This is always a pathological condition—the release management process should have bumped up the version number in one of those two builds—but it is nevertheless a reality in practical build environments.
Gradle caches dependency metadata (pom.xml and ivy.xml) by the group ID, artifact ID, version number, and the repository from which the dependency was resolved. Thus for a dependency to be “resolved” means that it has been resolved against a particular repository.[18] To fetch the metadata for commons-codec:commons-codec:1.6 from your internal Artifactory repository and from Central are not the same operation as far as Gradle’s cache is concerned. This will lead to an increase in network requests during a build, but only for small metadata files; it will only lead to more artifact downloading when such downloading is the right thing to do.
There is, in effect, one metadata cache per repository, but Gradle maintains just one artifact cache for all dependencies. Binary artifacts are stored by the SHA-1 hash code of their contents, not the repo they came from or their group:artifact:version metadata. Thus, all of the metadata caches share common access to a local, disk-based store of downloaded binary artifacts. In the previous example, an internal Artifactory repo may have a copy of commons-codec:commons-codec:1.6, and Maven Central may have the same. If they are both legitimately serving the same JAR—as defined by both files sharing the same SHA-1 hash code—then Gradle will only download the binary artifact once, and store it in one place on disk. Gradle relies heavily on hashes to optimize download performance; when resolving a dependency, it will first attempt to download the much-smaller SHA from the repo, skipping the download if it determines that it already has that content in the artifact cache.
There are a couple of important command line switches that impact the way Gradle uses its dependency cache. The --offline switch tells Gradle to use cached dependencies, and not to attempt to re-resolve anything (e.g., changing dependencies) against networked repositories. This is useful when working without a network connection or when a needed repository is offline. The --refresh-dependencies switch, by contrast, forces Gradle to go to the network to re-evaluate and re-fetch all dependency metadata. It will not download redundant modules if an artifact with the matching SHA-1 is already present in the artifact cache. This can be useful to ensure that you are building against the very latest of any changing or dynamic dependencies.
Every dependency configuration has a resolution strategy associated with it. In most builds, this strategy operates behind the scenes and out of the view of build masters and developers alike, but dependency problems can arise that require its customization. At present, there are three parts of the strategy that you can change: what to do when a version conflict arises, whether to force versions of certain artifacts, and what caching semantics to apply to changing and dynamic dependencies. Let’s look at these in turn.
Since Gradle resolves dependencies transitively, two independently declared dependencies may result in conflicting versions of some module they both depend on. For example, a web form processing API may depend on commons-collections:commons-collections:3.2.1, whereas an older vendor-specific enterprise integration module may rely on commons-collections:commons-collections:2.1. Gradle will detect that different versions of the same module are in the dependency graph, and by default it will choose the newer version. However, building the enterprise integration module with Commons Collections 3.2.1 might not work. You might want to know about the conflict and fail the build. You can do this, either globally or on a per-configuration basis, as in Example 4-31.
Following the prior example, you may determine that the modules that declare the newer dependency are simply trying to stay up-to-date in their dependency metadata, but are in fact backward compatible with the older version of Commons Collections. Rather than defaulting to the new version or failing the build, you might want to force Gradle to use version 2.1 instead of 3.2.1 (Example 4-32).
Statically declared releases like commons-collections:commons-collections:3.2.1 and commons-collections:commons-collections:2.1 are immutable artifacts. They are published to repositories on the network, and can be downloaded from those repositories through the mechanisms we’ve covered so far. Once downloaded, they are highly amenable to caching, since they can be identified uniquely and they never change. However, dynamic dependencies and changing dependencies are not immutable, and so their caching semantics are subject to more customization.
Dynamic and changing modules are still cached, but since the artifact can become out-of-date with respect to their identifiers, Gradle invalidates the cached versions after some period of time. By default, both types of cached artifacts expire after 24 hours, but both timeouts can be set to arbitrary periods (including zero) using the resolutionStrategy block. Again, you can change these settings on a per-configuration basis, or uniformly to all configurations as shown in Example 4-33.
Gradle has rich and customizable support for the current state of the art in dependency management as it has evolved in the world of the JVM. It fully supports Maven and Ivy repositories, and pragmatically adapts to legacy builds that use static dependency management. Moreover, its dependency caching feature helps solve difficult build repeatability problems that afflict real-world enterprise builds. There are few dependency management scenarios it can’t handle natively or be adapted to through a few easy customizations.
[16] Hibernate and the Spring Framework are actually composed of several distinct modules each. Some of those modules depend on one another, and some stand alone, depending only on modules outside of their respective frameworks.
[17] In organizations with globally distributed teams, words like “workday” and “nightly” begin to lose crisp definition, but these remain the terms normally used to discuss builds of this kind.
[18] A repository is defined as a URL, a type (e.g., Maven, Ivy, etc.), and a layout. All Maven repos have the same layout, but Ivy repos may differ.
Get Gradle Beyond the Basics now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
