Linux Device Drivers, Second Edition

Compiling and Loading

The rest of this chapter is devoted to writing a complete, though typeless, module. That is, the module will not belong to any of the classes listed in Section 1.3 in Chapter 1. The sample driver shown in this chapter is called skull, short for Simple Kernel Utility for Loading Localities. You can reuse the skull source to load your own local code to the kernel, after removing the sample functionality it offers.^[8]

Before we deal with the roles of init_module and cleanup_module, however, we’ll write a makefile that builds object code that the kernel can load.

First, we need to define the __KERNEL__ symbol in the preprocessor before we include any headers. As mentioned earlier, much of the kernel-specific content in the kernel headers is unavailable without this symbol.

Another important symbol is MODULE, which must be defined before including <linux/module.h> (except for drivers that are linked directly into the kernel). This book does not cover directly linked modules; thus, the MODULE symbol is always defined in our examples.

If you are compiling for an SMP machine, you also need to define __SMP__ before including the kernel headers. In version 2.2, the “multiprocessor or uniprocessor” choice was promoted to a proper configuration item, so using these lines as the very first lines of your modules will do the task:

 #include <linux/config.h>
 #ifdef CONFIG_SMP
 # define __SMP__
 #endif

A module writer must also specify the -O flag to the compiler, because many functions are declared as inline in the header files. gcc doesn’t expand inline functions unless optimization is enabled, but it can accept both the -g and -O options, allowing you to debug code that uses inline functions.^[9] Because the kernel makes extensive use of inline functions, it is important that they be expanded properly.

You may also need to check that the compiler you are running matches the kernel you are compiling against, referring to the file Documentation/Changes in the kernel source tree. The kernel and the compiler are developed at the same time, though by different groups, so sometimes changes in one tool reveal bugs in the other. Some distributions ship a version of the compiler that is too new to reliably build the kernel. In this case, they will usually provide a separate package (often called kgcc) with a compiler intended for kernel compilation.

Finally, in order to prevent unpleasant errors, we suggest that you use the -Wall (all warnings) compiler flag, and also that you fix all features in your code that cause compiler warnings, even if this requires changing your usual programming style. When writing kernel code, the preferred coding style is undoubtedly Linus’s own style. Documentation/CodingStyle is amusing reading and a mandatory lesson for anyone interested in kernel hacking.

All the definitions and flags we have introduced so far are best located within the CFLAGS variable used by make.

In addition to a suitable CFLAGS, the makefile being built needs a rule for joining different object files. The rule is needed only if the module is split into different source files, but that is not uncommon with modules. The object files are joined by the ld -r command, which is not really a linking operation, even though it uses the linker. The output of ld -r is another object file, which incorporates all the code from the input files. The -r option means “relocatable;” the output file is relocatable in that it doesn’t yet embed absolute addresses.

The following makefile is a minimal example showing how to build a module made up of two source files. If your module is made up of a single source file, just skip the entry containing ld -r.

 # Change it here or specify it on the "make" command line
 KERNELDIR = /usr/src/linux

 include $(KERNELDIR)/.config

 CFLAGS = -D__KERNEL__ -DMODULE -I$(KERNELDIR)/include \
   -O -Wall

 ifdef CONFIG_SMP
  CFLAGS += -D__SMP__ -DSMP
 endif

 all: skull.o

 skull.o: skull_init.o skull_clean.o
   $(LD) -r $^ -o $@

 clean:
   rm -f *.o *~ core

If you are not familiar with make, you may wonder why no .c file and no compilation rule appear in the makefile shown. These declarations are unnecessary because make is smart enough to turn .c into .o without being instructed to, using the current (or default) choice for the compiler, $(CC), and its flags, $(CFLAGS).

After the module is built, the next step is loading it into the kernel. As we’ve already suggested, insmod does the job for you. The program is like ld, in that it links any unresolved symbol in the module to the symbol table of the running kernel. Unlike the linker, however, it doesn’t modify the disk file, but rather an in-memory copy. insmod accepts a number of command-line options (for details, see the manpage), and it can assign values to integer and string variables in your module before linking it to the current kernel. Thus, if a module is correctly designed, it can be configured at load time; load-time configuration gives the user more flexibility than compile-time configuration, which is still used sometimes. Load-time configuration is explained in Section 2.6 later in this chapter.

Interested readers may want to look at how the kernel supports insmod: it relies on a few system calls defined in kernel/module.c. The function sys_create_module allocates kernel memory to hold a module (this memory is allocated with vmalloc; see Section 7.4 in Chapter 7). The system call get_kernel_syms returns the kernel symbol table so that kernel references in the module can be resolved, and sys_init_module copies the relocated object code to kernel space and calls the module’s initialization function.

If you actually look in the kernel source, you’ll find that the names of the system calls are prefixed with sys_. This is true for all system calls and no other functions; it’s useful to keep this in mind when grepping for the system calls in the sources.

Version Dependency

Bear in mind that your module’s code has to be recompiled for each version of the kernel that it will be linked to. Each module defines a symbol called __module_kernel_version, which insmod matches against the version number of the current kernel. This symbol is placed in the .modinfo Executable Linking and Format (ELF) section, as explained in detail in Chapter 11. Please note that this description of the internals applies only to versions 2.2 and 2.4 of the kernel; Linux 2.0 did the same job in a different way.

The compiler will define the symbol for you whenever you include <linux/module.h> (that’s why hello.c earlier didn’t need to declare it). This also means that if your module is made up of multiple source files, you have to include <linux/module.h> from only one of your source files (unless you use __NO_VERSION__, which we’ll introduce in a while).

In case of version mismatch, you can still try to load a module against a different kernel version by specifying the -f (“force”) switch to insmod, but this operation isn’t safe and can fail. It’s also difficult to tell in advance what will happen. Loading can fail because of mismatching symbols, in which case you’ll get an error message, or it can fail because of an internal change in the kernel. If that happens, you’ll get serious errors at runtime and possibly a system panic—a good reason to be wary of version mismatches. Version mismatches can be handled more gracefully by using versioning in the kernel (a topic that is more advanced and is introduced in Section 11.3 in Chapter 11).

If you want to compile your module for a particular kernel version, you have to include the specific header files for that kernel (for example, by declaring a different KERNELDIR) in the makefile given previously. This situation is not uncommon when playing with the kernel sources, as most of the time you’ll end up with several versions of the source tree. All of the sample modules accompanying this book use the KERNELDIR variable to point to the correct kernel sources; it can be set in your environment or passed on the command line of make.

When asked to load a module, insmod follows its own search path to look for the object file, looking in version-dependent directories under /lib/modules. Although older versions of the program looked in the current directory, first, that behavior is now disabled for security reasons (it’s the same problem of the PATH environment variable). Thus, if you need to load a module from the current directory you should use ./module.o, which works with all known versions of the tool.

Sometimes, you’ll encounter kernel interfaces that behave differently between versions 2.0.x and 2.4.x of Linux. In this case you’ll need to resort to the macros defining the version number of the current source tree, which are defined in the header <linux/version.h>. We will point out cases where interfaces have changed as we come to them, either within the chapter or in a specific section about version dependencies at the end, to avoid complicating a 2.4-specific discussion.

The header, automatically included by linux/module.h, defines the following macros:

UTS_RELEASE

The macro expands to a string describing the version of this kernel tree. For example, "2.3.48".

LINUX_VERSION_CODE

The macro expands to the binary representation of the kernel version, one byte for each part of the version release number. For example, the code for 2.3.48 is 131888 (i.e., 0x020330).^[10] With this information, you can (almost) easily determine what version of the kernel you are dealing with.

KERNEL_VERSION(major,minor,release)

This is the macro used to build a “kernel_version_code” from the individual numbers that build up a version number. For example, KERNEL_VERSION(2,3,48) expands to 131888. This macro is very useful when you need to compare the current version and a known checkpoint. We’ll use this macro several times throughout the book.

The file version.h is included by module.h, so you won’t usually need to include version.h explicitly. On the other hand, you can prevent module.h from including version.h by declaring __NO_VERSION__ in advance. You’ll use __NO_VERSION__ if you need to include <linux/module.h> in several source files that will be linked together to form a single module—for example, if you need preprocessor macros declared in module.h. Declaring __NO_VERSION__ before including module.h prevents automatic declaration of the string __module_kernel_version or its equivalent in source files where you don’t want it (ld -r would complain about the multiple definition of the symbol). Sample modules in this book use __NO_VERSION__ to this end.

Most dependencies based on the kernel version can be worked around with preprocessor conditionals by exploiting KERNEL_VERSION and LINUX_VERSION_CODE. Version dependency should, however, not clutter driver code with hairy #ifdef conditionals; the best way to deal with incompatibilities is by confining them to a specific header file. That’s why our sample code includes a sysdep.h header, used to hide all incompatibilities in suitable macro definitions.

The first version dependency we are going to face is in the definition of a "make install" rule for our drivers. As you may expect, the installation directory, which varies according to the kernel version being used, is chosen by looking in version.h. The following fragment comes from the file Rules.make, which is included by all makefiles:

VERSIONFILE = $(INCLUDEDIR)/linux/version.h
VERSION  = $(shell awk -F\" '/REL/ {print $$2}' $(VERSIONFILE))
INSTALLDIR = /lib/modules/$(VERSION)/misc

We chose to install all of our drivers in the misc directory; this is both the right choice for miscellaneous add-ons and a good way to avoid dealing with the change in the directory structure under /lib/modules that was introduced right before version 2.4 of the kernel was released. Even though the new directory structure is more complicated, the misc directory is used by both old and new versions of the modutils package.

With the definition of INSTALLDIR just given, the install rule of each makefile, then, is laid out like this:

install:
	install -d $(INSTALLDIR)
	install -c $(OBJS) $(INSTALLDIR)

Platform Dependency

Each computer platform has its peculiarities, and kernel designers are free to exploit all the peculiarities to achieve better performance in the target object file.

Unlike application developers, who must link their code with precompiled libraries and stick to conventions on parameter passing, kernel developers can dedicate some processor registers to specific roles, and they have done so. Moreover, kernel code can be optimized for a specific processor in a CPU family to get the best from the target platform: unlike applications that are often distributed in binary format, a custom compilation of the kernel can be optimized for a specific computer set.

Modularized code, in order to be interoperable with the kernel, needs to be compiled using the same options used in compiling the kernel (i.e., reserving the same registers for special use and performing the same optimizations). For this reason, our top-level Rules.make includes a platform-specific file that complements the makefiles with extra definitions. All of those files are called Makefile. platform and assign suitable values to make variables according to the current kernel configuration.

Another interesting feature of this layout of makefiles is that cross compilation is supported for the whole tree of sample files. Whenever you need to cross compile for your target platform, you’ll need to replace all of your tools (gcc, ld, etc.) with another set of tools (for example, m68k-linux-gcc, m68k-linux-ld). The prefix to be used is defined as $(CROSS_COMPILE), either in the make command line or in your environment.

The SPARC architecture is a special case that must be handled by the makefiles. User-space programs running on the SPARC64 (SPARC V9) platform are the same binaries you run on SPARC32 (SPARC V8). Therefore, the default compiler running on SPARC64 (gcc) generates SPARC32 object code. The kernel, on the other hand, must run SPARC V9 object code, so a cross compiler is needed. All GNU/Linux distributions for SPARC64 include a suitable cross compiler, which the makefiles select.

Although the complete list of version and platform dependencies is slightly more complicated than shown here, the previous description and the set of makefiles we provide is enough to get things going. The set of makefiles and the kernel sources can be browsed if you are looking for more detailed information.

---

^[8] We use the word local here to denote personal changes to the system, in the good old Unix tradition of /usr/local.

^[9] Note, however, that using any optimization greater than -O2 is risky, because the compiler might inline functions that are not declared as inline in the source. This may be a problem with kernel code, because some functions expect to find a standard stack layout when they are called.

^[10] This allows up to 256 development versions between stable versions.