Chapter 1. Overview of Drupal

This chapter contains overviews of the Drupal software and open source project, how Drupal has evolved from its beginnings through the latest version (Drupal 8), how Drupal performs its core function (handling HTTP requests), the Drupal cache system, and Drupal’s automatic class loading system. You will need to be familiar with this information in order to understand and use the rest of the book, so you’ll probably want to at least skim through it, if not read it in depth.

What Is Drupal?

Depending on who you talk to, you’ll hear Drupal called a content management system (CMS) or a content management framework (CMF), a platform that you can use to build a custom CMS—and both are accurate. It can be called a CMS because after installing only the base Drupal software, you can create a basic website and manage the content online. On the other hand, it can be called a flexible CMF because most people choose to add additional modules to Drupal in order to build more complicated websites with more features, and Drupal also allows you to create fully custom modules.

Drupal is free and open source software (FOSS), governed by the GNU General Public License (GPL) version 2 (or, at your option, any later version). The licensing means that Drupal is free for you, or anyone else in the world, to download, use, and modify, as long as you comply with the license terms.

Tip

If you have never read the GPL and plan to use Drupal, you would be well advised to do so (even more so if you plan to do any Drupal programming, for yourself or others). The GPL governs not only what you can do with Drupal software itself, but also what you can do with any add-ons you download from drupal.org, code you find on drupal.org documentation pages, and any derivative work (work that contains GPL-licensed work, verbatim or with modifications) that you or others create. It’s also written in plain English and is quite a good read (for programmer types anyway); you can find it in the LICENSE.txt file distributed with Drupal core, or at http://gnu.org.

And finally, Drupal is also a project and a community. Unlike some FOSS software that is developed primarily by one company that later releases the source code to the public, Drupal is continually evolving due to the efforts of a worldwide community of individuals and companies who donate their time and money to create and test Drupal software, write the documentation, translate it into other languages, answer support questions, keep the drupal.org web servers running, and organize get-togethers on a local and worldwide scale.

Drupal Core

Drupal core is the name of the package you can download from https://www.drupal.org/project/drupal, consisting of a set of PHP scripts (some with embedded HTML markup), JavaScript, CSS, and other files. This software interacts with a web server (typically, Apache), a database (MySQL, PostgreSQL, and SQLite are supported by Drupal core versions 7 and 8, and others are supported by add-on modules), and a web browser to provide the basics of a CMS:

A URL request dispatch system
A user account management system with flexible permissions and roles
Online content editing
An API for Drupal programmers
A module system, which lets you enable and disable functionality in a modular way and download additional code to expand functionality
A theme (template) system, which lets you override how everything from a button to an entire page is displayed
An installation profile system, which governs what modules and themes are installed when you first install Drupal and allows downloading of complete Drupal distributions for special-purpose sites
An internationalization system, which allows you to translate your site into other languages
A block system that allows you to place chunks of content in various regions of a site’s pages
A navigation menu builder
A flexible taxonomy system that supports categories, tags, and user-defined taxonomy vocabularies
Optional modules supporting commenting, content fields, RSS aggregation, search, lists of content and other site components, and site features such as forums and polls (depending on the Drupal version, some of these may require downloading add-on modules instead of being part of Drupal core)
The ability to set up a site in different languages and translate content (depending on the Drupal version, some add-on modules may be required to make a multi-lingual or non-English site)
Logging of system events and errors
An automatic class loading system
In Drupal 8, a file-based configuration management system
In Drupal 8, a plugin API, which is used in many Drupal systems
In Drupal 8, the services concept and the dependency injection container, which allow overrides of basic Drupal functionality

Drupal Add-Ons: Modules, Themes, Distributions, and Translations

Drupal is modular software, meaning that you can turn site features and functionality on and off by enabling and disabling modules. In Drupal 7 and earlier versions, you can disable a module to turn off its functionality while retaining its data so it can be enabled again. However, in Drupal 8, this is not allowed: to turn off a module you have to uninstall it completely, losing its data. Drupal core comes with a few required modules and several optional modules; you can download thousands of additional contributed modules from https://www.drupal.org/project/modules. Most modules have configuration options that you can modify from the Drupal administration interface, by logging in to the Drupal-based site using an account that has been given appropriate permissions. The permission system is flexible: you can define named roles, which are granted specific permissions (the permissions are defined by modules), and you can assign one or more roles to each user account.

Drupal’s theme system separates the content from the specific HTML markup and styling. This means that if you want to redesign the site’s layout or styling, you can do so by downloading a new theme from https://www.drupal.org/project/themes, purchasing a commercially available theme, or creating one yourself. Once a theme is installed and enabled, it takes effect immediately to change the look of your site without the necessity of editing your content pages. The theme system has a cascading system of inheritance and overrides, including the ability to define sub-themes, which allows you to use the default display or the display from a theme you have downloaded for some aspects of the output, and override the parts you want to change; the overrides can be at anything from the lowest level (e.g., the presentation of buttons) to the full page.

You can also download Drupal in a distribution, which consists of Drupal core, an installation profile, and a collection of contributed modules and themes that work together to provide a more functional site for a specific purpose. Distributions are available at https://www.drupal.org/project/distributions for ecommerce, government, nonprofits, and many other purposes.

And finally, you can download translations for Drupal and its contributed modules, themes, and distributions from https://localize.drupal.org. This allows you to build a non–English language site fairly easily with both Drupal 7 and Drupal 8. Multilingual sites can also be built using Drupal; in Drupal 7, this requires downloading several contributed modules and is fairly complex, but Drupal 8 core has much more multilingual functionality built in and is much more streamlined. See https://www.drupal.org/documentation/multilingual for more on building multilingual sites.

Finding Drupal add-ons

Here are the main ways to find Drupal add-ons (modules, themes, or distributions):

To find a specific add-on that you know the name of, visit https://www.drupal.org and type the name into the search box.
If the add-on that you’re looking for is not in the first few results, try restricting the search to modules or themes, using the filters in the right sidebar (there is no way to restrict to distributions from a drupal.org search, as of summer 2015).
Alternatively, start by navigating to https://www.drupal.org/project/modules, https://www.drupal.org/project/themes, or https://www.drupal.org/project/distributions, and searching from there. You can also search from these pages by keyword, Drupal version compatibility, or category (modules only).
You can try guessing the URL, which is always drupal.org/project/, followed by the machine name of the project. The machine name is composed of lowercase letters, numbers, and underscores, but because the machine names are chosen by developers, some are hard to guess and they may take a couple of tries. For example, the Panels module is at https://www.drupal.org/project/panels; the Omega theme is at https://www.drupal.org/project/omega; the XML Sitemap module is at https://www.drupal.org/project/xmlsitemap.
Of course, you can always try your favorite web search engine, with keywords “Drupal module” and the name of the module.

The Evolution of Drupal

Drupal started out as a message board, created by founder Dries Buytaert at the University of Antwerp in 1999. Once this message board was established, its members started exploring ideas for its improvement, and these were added to the software. Eventually, Dries named the software Drupal, and he released it on January 15, 2001, as an open source project so others could work on it. By June 2002, when version 4.0.0 was released, Drupal had expanded to a more flexible CMS, with new modules handling blogs, polls, taxonomy, and generic content, and a system for add-on modules and themes. The 4.x.y, 5.x, and 6.x versions continued the process of adding new CMS functionality and refining the API.

Note

Don’t worry if you don’t know what all of the terminology in this section means yet! It will be explained later in the book, where programming with each of the related subsystems of Drupal is explored.

January 2011 saw the first full release of Drupal version 7, which brought several major changes to the Drupal software and its API:

There was a major usability push. A group formed to do usability testing, and its findings resulted in the redesign and reorganization of administrative pages.
The accessibility of the administrative interface was also vastly improved, due to the efforts of a dedicated group of people interested in accessibility, and the recognition in the Drupal community as a whole of its importance.
A completely new database API, based on the PHP Data Objects (PDO) library, was added. This took Drupal 6’s concept of database portability, which was accomplished through a set of wrapper functions, and made it object-oriented, more robust, more secure, and more industry-standard.
A class autoloader was created and added to Drupal core.
The concept of entities was introduced, encompassing content items from the Node module, user accounts, taxonomy terms, and comments. The system also allowed add-on modules to define their own entity types.
In conjunction with entities, a system for fields on entities was added to Drupal core (previously, this had been in the Content Construction Kit, or CCK, contributed module, and it only worked on node content items). Many field modules were also added to Drupal core, including modules to handle file and image uploads, and output images in different sizes. Because nearly every Drupal site needs this functionality, getting it into Drupal core was a key Drupal 7 milestone.
A new Render API was introduced, with the goal of delaying rendering until later in the page-generating process, so that the page can be altered before it is rendered. This was based on the Form API that existed in earlier versions.
An automated testing system was introduced, which had previously been in a contributed module. Tests were added to Drupal core for much of the basic functionality, and these tests were run on every proposed Drupal core patch, to guard against regression. This greatly improved the quality of the Drupal core source code and of contributed modules that made use of it.
The API for defining installation profiles was revised and streamlined.

Drupal 8 introduces even more changes:

The usability and accessibility teams have continued to improve the usability and accessibility of the administrative UI.
A mobile team was formed, which has made sure that both the administrative UI and the core themes work well on mobile devices (phones and tablets).
Underneath the UI, the internals of Drupal have been radically transformed from a mostly ad-hoc, specific-to-Drupal, procedural-code-based system into a mostly object-oriented system that incorporates code from many outside open source projects. To do this, the Drupal 8 core development community has had to overcome the “not invented here” attitude seen in many open source projects; the idea is that the Drupal project should use code from other projects where possible, and improve that code if necessary (to the benefit of other open source projects), rather than trying to invent everything itself. In addition, some of the core systems of Drupal were rewritten as portable PHP classes without Drupal-specific dependencies, with the idea that they could be used by other open source projects.

The internal systems and APIs that are radically changed in Drupal 8 include:

The system for handling URL requests, which was native to Drupal in previous versions, has been replaced by the URL request handler from the Symfony project. Symfony is an open source PHP web development framework.
A system for swappable services has been added, and many aspects of Drupal core functionality have been converted to services (allowing contributed modules to replace them). The service framework also comes from the Symfony project.
An object-oriented plugin system has been added, and many of the hooks from previous versions of Drupal have been converted to plugins. This means that modules have increasing amounts of object-oriented code and reduced amounts of procedural code. The plugin framework comes from the Symfony project and the Doctrine project.
Some informational hooks from previous versions of Drupal have been converted to using YAML-formatted files to provide the information.
The system for storing configuration information, which was interconnected with other data and somewhat ad hoc in Drupal 7 and previous versions, has been replaced by a new API. Configuration is now easy to export and import, making it fairly straightforward to share configuration among sites or store it in a revision control system.
The theme system has been converted to use the templating system from the Twig project, instead of the Drupal-specific PHP-based templates and functions used in Drupal 7 and previous versions.
The class loader from the open source Composer project replaces the Drupal-specific class loader from Drupal 7.
Some functionality previously in contributed modules, such as the Views module, has been added to Drupal core. On the other hand, some of the less-used Drupal core modules, such as the Poll module, have been converted to contributed modules and removed from Drupal core.

How Drupal Handles HTTP Requests

When Drupal is installed properly and the web server receives an HTTP request that corresponds to the Drupal site, the main Drupal index.php file is loaded and executed by the server to handle the request. It is important for Drupal programmers to understand how Drupal handles such requests. This process changed significantly between Drupal 7 and Drupal 8, so it is discussed in two separate sections.

HTTP Request Handling in Drupal 7

Here is an overview of the HTTP request processing sequence in Drupal 7:

Drupal determines which settings.php file to use for the HTTP request (you can set up Drupal to serve multiple sites, each with its own settings.php file), and this file is loaded and executed.
The database connection and configuration/variable system are initialized.
If a request is coming from an anonymous user (a site visitor who is not logged in), the page cache is checked to see if output has previously been cached for the same requested URL. If so, the cached output is returned to the web server, and Drupal is done. Drupal page caching does not apply to authenticated (logged-in) users.
PHP session variables are initialized.
The language system is initialized, and various files are loaded and executed (core include files and enabled modules’ .module files).
Drupal determines whether the site is offline (also known as being in maintenance mode) or online.
If the site is offline, Drupal retrieves the offline message stored by an administrator as the page content. Other functions are called to generate some sections of the page content.
If the site is online, or if an authorized user is accessing a page while the site is offline, Drupal determines which functions need to be called to generate the content for the request, and it calls these functions. They ideally return render arrays that include the page data and other information necessary to render the page, but they could also return rendered or partially rendered content. During this process, if the page request includes a Drupal Form API form submission, the submitted data is processed and sent to form validation and/or submission handler functions.
Drupal determines what delivery method to use for the page and calls the appropriate delivery function.
For HTML page requests, the default page-delivery function prints HTTP headers, uses the theme to render the render array elements into HTML, prints the HTML output (which effectively sends it to the web server), saves user session information to the database, and exits. The Ajax request-delivery function is similar, but it renders into JavaScript Object Notation (JSON) output instead of using the theme system to render to HTML. Modules can also define custom page-delivery methods.

HTTP Request Handling in Drupal 8

The major change in Drupal 8’s HTTP request handling from all previous versions of Drupal is that Drupal 8 makes use of many aspects of the Symfony framework, and other third-party libraries. Here is an overview of the HTTP request processing sequence in Drupal 8:

The dynamic class loader and error handlers are started. This class loader comes from the Composer project.
A \Drupal\Core\DrupalKernel object is initialized, which sets up the Drupal services environment, so that needed services (such as the database, configuration, settings, etc.) can be loaded dynamically when they are needed.
A \Symfony\Component\HttpFoundation\Request object is built, which holds the request data (URL, cookies, etc.) in a usable format.
The handle($request) method is called on the DrupalKernel object, which allows Drupal to process the request and generate page output in the form of a \Symfony\Component\HttpFoundation\Response object.
The send() method is called on the Response object, which sends headers and response data to the web server, which in turn sends them to the browser.

The DrupalKernel::handle() method performs the following steps:

Loads the core/includes/bootstrap.inc include file.
Determines which settings.php file to use for the HTTP request (you can set up Drupal to serve multiple sites, each with its own settings.php file). Loads and executes this file.

Note that the setup for multiple sites in Drupal 8 requires a sites.php file; in Drupal 7, you could get by with special names for the site directories.
Initializes the dependency injection container for the services.
Initializes PHP session variables and cookies.
Initializes the middleware, which is a stacked set of \Symfony\Component\HttpKernel\HttpKernel objects. Each middleware object defines a way to respond to an HTTP request, such as banning IP addresses, returning cached page output from previous requests, and normal request handling.
Calls the handle($request) methods on each middleware object, until a response is generated.
If the normal request handling is being used, calls the DrupalKernel::preHandle() method. This loads and executes the core include files and enabled modules’ .module files.
Processes the page using Symfony’s HTTP request system.

The main step in Symfony’s HTTP request system processing is to determine which controller method has registered with the routing system to handle the request, and then instantiate the controller class and invoke the method. Drupal provides a system for modules to register routes in the routing system, which map URLs and other context to controller methods; Symfony determines the best match to the context in order to choose which controller to invoke. Symfony expects a controller to return a Response object, but Drupal provides some wrappers. One allows controller methods to return a render array, which is processed by the theme into HTML. Another allows controller methods to return a PHP data structure, which is transformed into JSON data for output.

The Symfony response handling system also dispatches a series of events during request processing, which allows Drupal core and modules to register event subscribers to intercept and override various steps in the response process. Drupal core uses event subscribers for many purposes, including authenticating site users, translating URL aliases to system URL paths, enabling maintenance mode, handling language selection, and providing dynamic URL routes.

The Drupal Cache

Drupal has a cache system, which allows modules to precalculate data or output and store it so that the next time it is needed it doesn’t have to be calculated again. This can save a lot of time on page loads, at the expense of some added complexity: any module that uses caching needs to take care to clear its cached data whenever the data is invalidated due to changes in dependent data.

Both Drupal core and add-on modules cache information using this system. Here are a few examples (not all of them apply to all versions of Drupal):

Page output (page caching can be turned off from the Performance configuration page)
Block output (block caching can be turned off from the Performance page in Drupal 7 but is always on in Drupal 8 for cacheable blocks)
Information collected from hooks; for example, Drupal 7 entity type and field definitions
Lists of available plugins in Drupal 8 plugin managers
Theme information, including the list of theme regions and theme-related information from modules and themes
Form arrays

Programmers and site builders new to Drupal quickly learn that the first thing to try, if they are having trouble with a site or if programming changes they have recently made are not being recognized, is to clear the cache. You can clear the cache by visiting the Performance configuration page and clicking the cache clear button, or by using Drush. The Drush command to clear the cache in Drupal 7 is drush cc all; in Drupal 8, use drush cr (which clears the cache and also rebuilds the dependency injection container; more about that later in the book).

Tip

The rest of this section provides details of the cache API, which you can skip for now and return to when you need them.

Drupal 7 Cache API

The Drupal 7 cache system has a fairly simple API, consisting of the cache_set() and cache_get() functions (with a few variations), as well as cache_clear_all() and drupal_flush_all_caches() to clear cache information. Modules can register to have their caches cleared by implementing a hook (hooks are module entry points to altering Drupal) called hook_flush_caches(). All cached data is stored in the database, usually in tables whose names start with cache_.

Drupal 8 Cache API

The Drupal 8 cache API uses services, so that different sets of cached data, or bins, can use different storage mechanisms. In Drupal 8, all cache functionality starts by calling \Drupal::cache($bin), passing in the name of the cache storage bin you want to use, to get an instance of the correct cache class; this class will implement \Drupal\Core\Cache\CacheBackendInterface. Many cache applications use the default bin. Alternatively, if you have a $container variable in a class method, you’ll call:

// The name of the service for a particular cache bin is
// 'cache.' . $bin.
$cache_class = $container->get('cache.' . $bin);

Once you have a cache class, your module can call methods such as set(), get(), and invalidate() to store, retrieve, and invalidate your cached data. If your module uses a custom cache system, it should implement hook_cache_flush() to flush the data when a cache clear is requested via the drupal_flush_all_caches() function, and you may need to implement hook_rebuild() as well. However, if you store your data in cache bins that Drupal core defines, your cached data will automatically be flushed.

For efficiency, the Drupal 8 cache system also has a tagging mechanism, which is used to flush parts of the cache (rather than the whole cache) when data they’re related to is changed or removed. For instance, if your module is caching data related to a particular node content item, you should tag it with the node ID when you add it to the cache, by calling:

// $cid is a unique cache ID key for your data, which is $data.
// $nid is the ID of the node whose data is cached.
$cache_class->set($cid, $data, CacheBackendInterface::CACHE_PERMANENT,
   array('node:' . $nid));

Tags are passed in as the fourth argument; here, the node tag is given a value equal to the node ID related to this cached data. Then, when calls to the core Node module modify this node content item, it will call cache methods to invalidate all cached data that was tagged with this node ID. There is also a node_list cache tag, which indicates your data is a list of node content items; this tag is invalidated whenever any node is changed, added, or deleted.

You can define custom tags for your module’s caches and use them to invalidate groups of cached data when appropriate by calling:

\Drupal\Core\Cache\Cache::invalidateTags($tags);

There is one more concept in caching, which is used in the request-rendering pipeline. The idea is that the content of a page (or other request result) may depend on various variables, such as the user who is viewing the page, the user’s roles, the page language, the time zone, or other factors; these factors are collectively known as the cache context. For details on how to add cache information to a render array, see the “Render API Overview” topic page on https://api.drupal.org.

Automatic Class Loading in Drupal

Drupal core versions 7 and 8 both include systems for automatic loading of files containing PHP class, interface, and trait declarations (in the rest of this section, class refers to either a class or an interface, or a trait in Drupal 8). The basic idea of both systems is the same:

Integrate with PHP’s native class-autoloading system, in which functions can be registered as class loaders.
Reduce the load on the server by loading files containing class definitions only if those classes are being used in a particular page request.
Reduce the burden on developers by automatically loading files containing class definitions, instead of making them load the classes explicitly in code.

Drupal 7’s class-loading system is a homegrown (specific to Drupal) system called the registry. Here is how it works:

Drupal determines when the class registry needs to be updated; for example, when you enable or disable modules, update a module version, or clear the Drupal cache.
When the registry needs to be updated, Drupal makes a list of files it needs to scan. These include files explicitly listed in enabled modules’ .info files, as well as .inc files in the Drupal core includes directory.
Each of these files is scanned for PHP class declarations.
The resulting list of which files contain which class declarations (the registry) is saved in the database.
During start-up, Drupal registers its dru⁠pal_autoload_class() and dru⁠pal_autoload_interface() functions as PHP autoload functions. When an undefined class is accessed in PHP code, the PHP engine calls these functions, which read the registry database to locate the file the class is defined in, and then include the file so that the class is defined.

Warning

Drupal 7 class names have to be globally unique. Always name your classes with a prefix that makes them unique, such as your module name.

Early in the development cycle for Drupal version 8, a decision was made to stop using the homegrown registry system developed for Drupal 7, and instead incorporate the class loader from the Composer open source project, which is based on the PSR-0 and PSR-4 standards, and PHP namespaces. Here is how this system works:

Classes are declared to be in a PHP namespace with a PHP namespace declaration. Drupal uses namespaces beginning with the name \Drupal, and the namespace(s) for a particular module mymodule should begin with \Drupal\mymodule.
Files that depend on classes outside their own namespace have PHP use declarations for the outside classes.
Each class declaration is in its own file.
The directory that each class file goes in depends on its namespace. Examples:
- An add-on module’s \Drupal\mymodule\subnamespace\Foo class would go in the src/subnamespace/Foo.php file under the main module directory.
- Classes that are for Drupal core broadly, and not part of a particular Drupal core module, are found under the core/lib directory; for example, the \Drupal\Component\Datetime\DateTimePlus class is located in the core/lib/Drupal/Component/Datetime/DateTimePlus.php file.
- Classes for testing Drupal are found under the core/test directory.
- Classes that have been adopted from outside projects are found under the vendor directory; for example, the \Symfony\Component\DependencyInjection\Container class is located in the vendor/symfony/dependency-injection/Container.php file.
Drupal provides information about its namespace and directory conventions to the Composer class-loading system. For instance, it tells Composer to look in enabled modules’ src directories for class files, but not in disabled modules’ directories.

Note

In PHP, namespaces in namespace and use declarations do not start with a backslash, although the fully qualified namespace, used in other places in code, always starts with a backslash. In this book, Drupal 8 classes are always referred to with their fully qualified namespace, to avoid ambiguity.

When a class is needed, the Composer class loader automatically locates and loads the include file so that the class is defined.

Warning

As a side effect of the Drupal 8 class-loading system, each module needs to be in its own directory or subdirectory in Drupal 8 (in Drupal 7, you could put several modules into the same directory). The name of the directory that directly contains the main module files needs to match the module’s machine name exactly (in Drupal 7, this did not need to be the case).

Programmer's Guide to Drupal, 2nd Edition by Jennifer Hodgdon

Chapter 1. Overview of Drupal

What Is Drupal?

Tip

Drupal Core

Drupal Add-Ons: Modules, Themes, Distributions, and Translations

Finding Drupal add-ons

The Evolution of Drupal

Note

How Drupal Handles HTTP Requests

HTTP Request Handling in Drupal 7

HTTP Request Handling in Drupal 8

The Drupal Cache

Tip

Drupal 7 Cache API

Drupal 8 Cache API

Automatic Class Loading in Drupal

Warning

Note

Warning

Don’t leave empty-handed

It’s yours, free.

Check it out now on O’Reilly