Network Security with OpenSSL

Chapter 4. Support Infrastructure

The OpenSSL library is composed of many different packages. Some of the lower-level packages can be used independently, while the higher-level ones may make use of several of the lower-level ones. To use the OpenSSL library effectively, it is important to understand the fundamental concepts of cryptography that we’ve already introduced, and to gain familiarity with the more important supplemental package offerings.

In this chapter, we concentrate on the lower-level APIs that are most useful with the higher-level APIs that we discuss through the rest of this book. We’ll start by demonstrating what is necessary when using the OpenSSL library in a multithreaded environment by developing a small “drop-in” library for Windows and Unix platforms that use POSIX threads. We’ll also examine OpenSSL’s error handling and its input/output interface, which are both quite different from how most other development libraries deal with the same things. OpenSSL also provides packages for arbitrary precision math and secure random number generation, as we already mentioned. These packages are both fundamental to strong crypto, and we’ll cover them as well.

For all of the packages that we examine in this chapter, we’ll discuss how to use them and provide examples. Additionally, we’ll discuss some of the common pitfalls that developers often encounter.

Note that if some of the material in this chapter doesn’t seem immediately relevant and interesting, it is safe to skip it, and come back to this chapter when necessary.

Multithread Support

Most modern operating systems provide support for multithreaded applications, and it is becoming increasingly more common for applications to take advantage of that support. OpenSSL can certainly be used in a multithreaded environment; however, it requires that the developer do some work in order to make a program thread-safe. A common mistake that many developers make with OpenSSL is that they assume the library is thread-safe without requiring anything special to be done in the application. This is most certainly an incorrect assumption, and failing to set up OpenSSL for use in a multithreaded environment can result in unpredictable behavior and seemingly random crashes that are very difficult to debug.

OpenSSL uses many data structures on which operations must be atomic. That is, it must be guaranteed that only one thread will access them at a time. If two or more threads are allowed to modify the same structure concurrently, there is no way to predict which one’s changes will be realized. What’s worse, the operations could end up mixed—part of the first thread’s changes could be made, while part of the second thread’s changes could also be made. In either case, the results are unpredictable, so steps must be taken to make the structures thread-safe.

OpenSSL provides for the thread safety of its data structures by requiring each thread to acquire a mutually exclusive lock known as a mutex that protects the structure before allowing it to be accessed. When the thread is finished with the data structure, it releases the mutex, allowing another thread to acquire the lock and access the data structure. Because OpenSSL is designed for use on multiple platforms that differ in their implementation of threading, OpenSSL doesn’t make direct calls to create, destroy, acquire, and release mutexes: it requires the application programmer to perform these operations in a manner appropriate for the platform it’s running on by making callbacks to functions that the application registers with OpenSSL for this purpose.

There are two different sets of callbacks that an application is expected to provide to safely operate in a multithreaded environment. Static locks provide a fixed number of mutexes available for OpenSSL’s use. Dynamic locks allow OpenSSL to create mutexes as it needs them. OpenSSL does not currently make use of dynamic locks, but reserves the right to do so in the future. If you want your applications to continue working with a minimal amount of effort in the future, we recommend that you implement both static and dynamic locks now.

Static Locking Callbacks

The static locking mechanism requires the application to provde two callback functions. In addition to providing an implementation for the functions, the application must tell OpenSSL about them so that it knows to call them when appropriate. The first callback function is used either to acquire or release a lock, and is defined like this:

void locking_function(int mode, int n, const char *file, int line);

mode: Determines the action that the locking function should take. When the CRYPTO_LOCK flag is set, the lock should be acquired; otherwise, it should be released.
n: The number of the lock that should be acquired or released. The number is zero-based, meaning that the first lock is identified by 0. The value will never be greater than or equal to the return from the CRYPTO_num_locks function.
file: The name of the source file requesting the locking operation to take place. It is intended to aid in debugging and is usually supplied by the _ _FILE_ _ preprocessor macro.
line: The source line number requesting the locking operation to take place. Like the file argument, it is also intended to aid in debugging, and it is usually supplied by the _ _LINE_ _ preprocessor macro.

The next callback function is used to get a unique identifier for the calling thread. For example, GetCurrentThreadId on Windows will do just that. For reasons that will soon become clear, it is important the value returned from this function be consistent across calls for the same thread, but different for each thread within the same process. The return value from the function should be the unique identifier. The function is defined like this:

unsigned long id_function(void);

Example 4-1 introduces two new OpenSSL library functions: CRYPTO_set_id_callback and CRYPTO_set_locking_callback . These two functions are used to tell OpenSSL about the callbacks that we’ve implemented for the static locking mechanism. We can either pass a pointer to a function to install a callback or NULL to remove a callback.

Example 4-1. Static locking callbacks for WIN32 and POSIX threads systems

int THREAD_setup(void);
int THREAD_cleanup(void);
 
#if defined(WIN32)
    #define MUTEX_TYPE HANDLE
    #define MUTEX_SETUP(x)   (x) = CreateMutex(NULL, FALSE, NULL)
    #define MUTEX_CLEANUP(x) CloseHandle(x)
    #define MUTEX_LOCK(x)    WaitForSingleObject((x), INFINITE)
    #define MUTEX_UNLOCK(x)  ReleaseMutex(x)
    #define THREAD_ID        GetCurrentThreadId(  )
#elif _POSIX_THREADS
    /* _POSIX_THREADS is normally defined in unistd.h if pthreads are available
       on your platform. */
    #define MUTEX_TYPE       pthread_mutex_t
    #define MUTEX_SETUP(x)   pthread_mutex_init(&(x), NULL)
    #define MUTEX_CLEANUP(x) pthread_mutex_destroy(&(x))
    #define MUTEX_LOCK(x)    pthread_mutex_lock(&(x))
    #define MUTEX_UNLOCK(x)  pthread_mutex_unlock(&(x))
    #define THREAD_ID        pthread_self(  )
#else
    #error You must define mutex operations appropriate for your platform!
#endif
 
/* This array will store all of the mutexes available to OpenSSL. */
static MUTEX_TYPE mutex_buf[] = NULL;
 
static void locking_function(int mode, int n, const char * file, int line)
{
    if (mode & CRYPTO_LOCK)
        MUTEX_LOCK(mutex_buf[n]);
    else
        MUTEX_UNLOCK(mutex_buf[n]);
}
 
static unsigned long id_function(void)
{
    return ((unsigned long)THREAD_ID);
}
 
int THREAD_setup(void)
{
    int i;
 
    mutex_buf = (MUTEX_TYPE *)malloc(CRYPTO_num_locks(  ) * sizeof(MUTEX_TYPE));
    if (!mutex_buf)
        return 0;
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_SETUP(mutex_buf[i]);
    CRYPTO_set_id_callback(id_function);
    CRYPTO_set_locking_callback(locking_function);
    return 1;
}
 
int THREAD_cleanup(void)
{
    int i;
 
    if (!mutex_buf)
        return 0;
    CRYPTO_set_id_callback(NULL);
    CRYPTO_set_locking_callback(NULL);
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_CLEANUP(mutex_buf[i]);
    free(mutex_buf);
    mutex_buf = NULL;
    return 1;
}

To use these static locking functions, we need to make one function call before our program starts threads or calls OpenSSL functions, and we must call THREAD_setup , which returns 1 normally and 0 if it is unable to allocate the memory required to hold the mutexes. In our example code, we do make a potentially unsafe assumption that the initialization of each individual mutex will succeed. You may wish to add additional error handling code to your programs. Once we’ve called THREAD_setup and it returns successfully, we can safely make calls into OpenSSL from multiple threads. After our program’s threads are finished, or if we are done using OpenSSL, we should call THREAD_cleanup to reclaim any memory used for the mutex structures.

Dynamic Locking Callbacks

The dynamic locking mechanism requires a data structure (CRYPTO_dynlock_value ) and three callback functions. The structure is meant to hold the data necessary for the mutex, and the three functions correspond to the operations for creation, locking/unlocking, and destruction. As with the static locking mechanism, we must also tell OpenSSL about the callback functions so that it knows to call them when appropriate.

The first thing that we must do is define the CRYPTO_dynlock_value structure. We’ll be building on the static locking support that we built in Example 4-1, so we can use the same platform-dependent macros that we defined already. For our purposes, this structure will be quite simple, containing only one member:

struct CRYPTO_dynlock_value
{
    MUTEX_TYPE mutex;
};

The first callback function that we need to define is used to create a new mutex that OpenSSL will be able to use to protect a data structure. Memory must be allocated for the structure, and the structure should have any necessary initialization performed on it. The newly created and initialized mutex should be returned in a released state from the function. The callback is defined like this:

struct CRYPTO_dynlock_value *dyn_create_function(const char *file, int line);

file: The name of the source file requesting that the mutex be created. It is intended to aid in debugging and is usually supplied by the _ _FILE_ _ preprocessor macro.
line: The source line number requesting that the mutex be created. Like the file argument, it is also intended to aid in debugging, and it is usually supplied by the _ _LINE_ _ preprocessor macro.

The next callback function is used for acquiring or releasing a mutex. It behaves almost identically to the corresponding static locking mechanism’s callback, which performs the same operation. It is defined like this:

void dyn_lock_function(int mode, struct CRYPTO_dynlock_value
                      *mutex, const char *file, int line);

mode: Determines the action that the locking function should take. When the CRYPTO_LOCK flag is set, the lock should be acquired; otherwise, it should be released.
mutex: The mutex that should be either acquired or released. It will never be NULL and will always be created and initialized by the mutex creation callback first.
file: The name of the source file requesting that the locking operation take place. It is intended to aid in debugging and is usually supplied by the _ _FILE_ _ preprocessor macro.
line: The source line number requesting that the locking operation take place. Like the file argument, it is also intended to aid in debugging, and it is usually supplied by the _ _LINE_ _ preprocessor macro.

The third and final callback function is used to destroy a mutex that OpenSSL no longer requires. It should perform any platform-dependent destruction of the mutex and free any memory that was allocated for the CRYPTO_dynlock_value structure. It is defined like this:

void dyn_destroy_function(struct CRYPTO_dynlock_value *mutex,
                         const char *file, int line);

mutex: The mutex that should be destroyed. It will never be NULL and will always have first been created and initialized by the mutex creation callback.
file: The name of the source file requesting that the mutex be destroyed. It is intended to aid in debugging and is usually supplied by the _ _FILE_ _ preprocessor macro.
line: The source line number requesting that the mutex be destroyed. Like the file argument, it is also intended to aid in debugging, and it is usually supplied by the _ _LINE_ _ preprocessor macro.

Using the static locking mechanism’s code from Example 4-1, we can easily build a dynamic locking mechanism implementation. Example 4-2 shows an implementation of the three dynamic locking callback functions. It also includes new versions of the THREAD_setup and THREAD_cleanup functions extended to support the dynamic locking mechanism in addition to the static locking mechanism. The modifications to these two functions are simply to make the appropriate OpenSSL library calls to install and remove the dynamic locking callback functions.

Example 4-2. E xtensions to the library to support the dynamic locking mechanism

struct CRYPTO_dynlock_value
{
    MUTEX_TYPE mutex;
};
 
static struct CRYPTO_dynlock_value * dyn_create_function(const char *file,
                                                         int line)
{
    struct CRYPTO_dynlock_value *value;
 
    value = (struct CRYPTO_dynlock_value *)malloc(sizeof(
                                                  struct CRYPTO_dynlock_value));
    if (!value)
        return NULL;
    MUTEX_SETUP(value->mutex);
    return value;
}
 
static void dyn_lock_function(int mode, struct CRYPTO_dynlock_value *l,
                              const char *file, int line)
{
    if (mode & CRYPTO_LOCK)
        MUTEX_LOCK(l->mutex);
    else
        MUTEX_UNLOCK(l->mutex);
}
 
static void dyn_destroy_function(struct CRYPTO_dynlock_value *l,
                                 const char *file, int line)
{
    MUTEX_CLEANUP(l->mutexp);
    free(l);
}
 
int THREAD_setup(void)
{
    int i;
 
    mutex_buf = (MUTEX_TYPE *)malloc(CRYPTO_num_locks(  ) * sizeof(MUTEX_TYPE));
    if (!mutex_buf)
        return 0;
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_SETUP(mutex_buf[i]);
    CRYPTO_set_id_callback(id_function);
    CRYPTO_set_locking_callback(locking_function);
 
    /* The following three CRYPTO_... functions are the OpenSSL functions
       for registering the callbacks we implemented above */
    CRYPTO_set_dynlock_create_callback(dyn_create_function);
    CRYPTO_set_dynlock_lock_callback(dyn_lock_function);
    CRYPTO_set_dynlock_destroy_callback(dyn_destroy_function);
 
    return 1;
}
 
int THREAD_cleanup(void)
{
    int i;
 
    if (!mutex_buf)
        return 0;
    CRYPTO_set_id_callback(NULL);
    CRYPTO_set_locking_callback(NULL);
    CRYPTO_set_dynlock_create_callback(NULL);
    CRYPTO_set_dynlock_lock_callback(NULL);
    CRYPTO_set_dynlock_destroy_callback(NULL);
    for (i = 0;  i < CRYPTO_num_locks(  );  i++)
        MUTEX_CLEANUP(mutex_buf[i]);
    free(mutex_buf);
    mutex_buf = NULL;
    return 1;
}

Internal Error Handling

OpenSSL has a package, known as the ERR package, devoted to the handling and processing of errors. When an OpenSSL function encounters an error, it creates an error report and logs the information to an error queue. Because the information is logged to a queue, if multiple errors occur, information can be gathered for each of them. It is our responsibility as developers to check the error queue to obtain detailed information when a function returns an error so that we can handle error conditions appropriately. The OpenSSL error handling mechanism is more complex than most other libraries of similar stature, but that also means more information is available to help resolve the error condition.

Let’s suppose for a moment that OpenSSL didn’t log errors onto a queue. Consider, for example, a rather common case in which an application calling into a high-level OpenSSL library function causes OpenSSL to make several successive calls into various lower-level packages that make up OpenSSL. If an error were to occur at a low level, that error would be propagated back up the call stack to the application. The problem is that by the time the application gets the information, it’s likely to have changed to something less detailed than the initial error as each function in the chain causes a new error to be generated all because of the initial low-level error.

Manipulating Error Queues

When an error occurs in the OpenSSL library, a significant amount of information is logged. Some of the information can be useful in attempting to recover from an error automatically, but much of it is for debugging and reporting the error to a user.

The ERR package provides six basic functions that are useful for obtaining information from the error queue. Each function always retrieves the oldest information from the queue so that errors are returned in the order that they were generated. The most basic piece of information that is logged is an error code, which describes the error that occurred. The error code is a 32-bit integer that has meaning only to OpenSSL. That is, OpenSSL defines its own unique error codes for any error condition that it could possibly encounter. It does not rely on error codes defined by any other library, including the standard C runtime. For each of the six basic functions, this error code is the return value from the function. If there is no error in the queue, the return from any of them will be 0, which also tells us that 0 is never a valid error code.

This first function retrieves only the error code from the error queue. It also removes that error report from the queue, so the next call will retrieve the next error that occurred or possibly 0 if there are no more errors in the queue:

unsigned long ERR_get_error(void);

The second function also retrieves only the error code from the error queue, but it does not remove the error report from the queue, so the next call will retrieve the same error:

unsigned long ERR_peek_error(void);

The third function builds on the information returned by ERR_get_error and ERR_peek_error. In addition to returning the error code, it also returns the name of the source file and source line number that generated the error. Like ERR_get_error, it also removes the error report from the queue:

unsigned long ERR_get_error_line(const char **file, int *line);

file: Receives the name of the source file that generated the error. It is usually supplied to the error handler from the _ _FILE_ _ preprocessor macro.
line: Receives the source line number that generated the error. It is usually supplied to the error handler from the _ _LINE_ _ preprocessor macro.

The fourth function returns the same information as ERR_get_error_line, but like ERR_peek_error, it does not remove the error report from the queue. Its arguments and their meanings are identical to ERR_get_error_line:

unsigned long ERR_peek_error_line(const char **file, int *line);

The fifth function builds on the information returned by ERR_get_error_line and ERR_peek_error_line. In addition to returning the error code, source filename, and line number, it also returns extra data and a set of flags that indicate how that data should be treated. The extra data and flags are supplied when the error is generated. Like ERR_get_error and ERR_get_error_line, this function also removes the error report from the queue:

unsigned long ERR_get_error_line_data(const char **file, int *line,
                                      const char **data, int *flags);

file: Receives the name of the source file that generated the error. It is usually supplied to the error handler from the _ _FILE_ _ preprocessor macro.
line: Receives the source line number that generated the error. It is usually supplied to the error handler from the _ _LINE_ _ preprocessor macro.
data: Receives a pointer to the extra data that was included with the error report. The pointer that is returned is not a copy, and so it should not be modified or freed. See below.
flags: Receives a set of flags that define the attributes of the extra data.

The sixth function returns the same information as ERR_get_error_line_data, but like ERR_peek_error and ERR_peek_error_line, it does not remove the error report from the queue. Its arguments and their meanings are identical to ERR_get_error_line_data:

unsigned long ERR_peek_error_line_data(const char **file, int *line,
                const char **data, int *flags);

ERR_get_error_line_data and ERR_peek_error_line_data both retrieve the optional piece of data that can be associated with an error report. This optional piece of data can be anything, but most frequently, it’s a string. Stored along with the data is a bit mask of flags that describe the data so that it can be dealt with appropriately by the error handling package. If the flag ERR_TXT_MALLOCED is set, the memory for the data will be freed by a call to OpenSSL’s OPENSSL_free function. If the flag ERR_TXT_STRING is set, the data is safe to be interpreted as a C-style string.

Note that the file and data information that can be obtained from the queue is returned as a pointer to the information on the queue. It is not a copy, so you should not attempt to modify the data. In the case of the file information, it is usually a constant string from the _ _FILE_ _ preprocessor macro. For the data information, if you need to store the information for any reason, you should make a copy and not store the pointer that is returned. When you use the “get” family of functions to obtain this data, the data remains valid for a short period, but you should be sure to make a copy before any other error handler function is called if you need to preserve it. Example 4-3 demonstrates how to print out the error information that is in the calling thread’s error queue.

Example 4-3. Accessing error information on the error queue

void print_errors(void)
{
    int           flags, line;
    char          *data, *file;
    unsigned long code;
 
    code = ERR_get_error_line_data(&file, &line, &data, &flags);
    while (code)
    {
        printf("error code: %lu in %s line %d.\n", code, file, line);
        if (data && (flags & ERR_TXT_STRING))
            printf("error data: %s\n", data);
        code = ERR_get_error_line_data(&file, &line, &data, &flags);
    }
}

There is one last queue-manipulation function that we’ll discuss here: the function for clearing the error queue. It will delete all errors currently in the queue. In general, there is no need to call this function unless we are trying to reset the error status for the current thread and don’t care about any other errors that are on the queue. There is no way to recover the previous errors once it’s been called, so use it judiciously:

void ERR_clear_error(void);

Human-Readable Error Messages

In some cases, the most appropriate way to handle an error condition is to display or log the error so that the user of your application can take the necessary steps to resolve the error. To do that, it’s best to display a human-readable error message rather than an error code. The error handling package provides standard error messages for its error codes for just this purpose, but before they can be used, they must be loaded.

There are two sets of error messages: one for the errors generated by libcrypto , and one for the errors generated by libssl. The function ERR_load_crypto_strings loads the errors generated by libcrypto, and the function ERR_load_SSL_strings loads the errors generated by libssl. There is an additional function, SSL_load_error_strings , which will load both sets of error messages.

Once the error strings are loaded, ERR_error_string and ERR_error_string_n can be used to translate an error code into an error message that is more meaningful to humans. Particularly in a multithreaded application, ERR_error_string should never be used. It is always best to use ERR_error_string_n. Both functions always return a pointer to the start of the buffer into which the translated error message was written.

char *ERR_error_string(unsigned long e, char *buf);

e: The error code that will be translated.
buf: The buffer into which the error message is written. The buffer must be at least 256 bytes in size, or it can be specified as NULL, in which case an internal buffer will be used. Use of this buffer is never thread-safe.

char *ERR_error_string_n(unsigned long e, char *buf, size_t len);

e: The error code that will be translated.
buf: The buffer into which the error message is written. It must never be NULL.
len: The size of the buf argument in bytes. It should include space for the NULL terminating character.

The resultant error message is formatted into a colon-separated list of fields. The first field is always the word “error”, and the second field is always the error code represented in hexadecimal. The third field is the name of the package that generated the error, such as “BIO routines” or “bignum routines”. The fourth field is the name of the function that generated the error, and the fifth field is the reason why the error was generated. The function name is taken from an internal table that is actually rather small, and may very likely be represented as func(<code>), in which code is a number representing the function.

To get information about an error, ERR_get_error_line_data and ERR_error_string should be used. Armed with all of the information from these two functions, we can emit rather detailed error information. The OpenSSL library provides us with two functions that ease this process for us, however. ERR_print_errors will produce an error listing and write it to a BIO. ERR_print_errors_fp will produce an error listing and write it to a standard C runtime FILE object. The error listings are produced by iterating through each error report in the error queue and removing them as it goes. For each error report, ERR_get_error_line_data and ERR_error_string are used to obtain the information necessary to produce the listing:

void ERR_print_errors(BIO *bp);

bp: The BIO that the error listing should be written to.

void ERR_print_errors_fp(FILE *fp);

fp: The FILE object that the error listing should be written to.

Threading and Practical Applications

A common concern of developers is the handling of errors produced by a library when using threaded code, and rightly so. With a few exceptions that can be easily avoided, OpenSSL’s error handling is completely thread-safe. Each thread is assigned its own error queue, which is one of the reasons why the id_function callback that we described earlier in the chapter must return a different identifier for each thread. Each error queue will contain only errors that were caused by that thread. This is convenient for threaded applications because the programmer doesn’t need to do anything special to handle errors correctly.

By creating a separate error queue for each thread, it would seem that all the bases are covered for error handling, but that’s not entirely true. OpenSSL does not use thread-local storage for the error queues, and so there is no way for each queue to be automatically destroyed when a thread terminates. Thread-local storage is a great feature to have in a multithreaded environment, but unfortunately, it is not supported on all platforms. The bottom line is that the application is responsible for destroying a thread’s error queue when a thread terminates because OpenSSL has no way of knowing on its own when a thread has terminated.

OpenSSL provides a function to destroy a thread’s error queue called ERR_remove_state . It should be called by a thread just before it terminates, or it may be called by another thread within the process after the thread has terminated. The function requires a single argument that is the identifier of the thread as it would be returned by the id_function callback that we described earlier in the chapter.

Until now, we have overlooked the implications of loading the strings for error processing. These strings do take up memory, and it isn’t always appropriate to load them. It should be mentioned that all of the error handling routines work properly without the strings loaded. The translated error messages will merely have internal OpenSSL codes inserted instead of the more meaningful strings. If we do choose to load the error strings, we should also be sure to free them when they’re no longer needed by calling ERR_free_strings. For most applications, this should happen after the program is done making calls into the OpenSSL library.

Abstract Input/Output

The BIO package provides a powerful abstraction for handling input and output. Many different types of BIO objects are available for use, but they all fall into one of two basic categories: source/sink and filter, both of which will be described in detail in upcoming sections. BIOs can be attached together in chains, allowing the data to flow through multiple BIO objects for processing as it is read or written. For example, a BIO chain can be created that causes data to be base64-encoded as it is written out to a file and decoded as it is read from a file. This feature of BIOs makes them very flexible and powerful. A single function with a BIO parameter can be written to read or write some data, and just by setting up a BIO chain, it is possible for that one function to deal with all kinds of different types of data encoding.

The OpenSSL library provides a variety of functions for creating and destroying BIOs, chaining them together, and reading or writing data. It’s important to note that the BIO package is a low-level package, and as such, you must exercise care in using it. Many of the functions will allow you to perform operations that could later lead to unpredictable behavior and even crashes.

BIO_new function is used to create a new BIO. It requires a BIO_METHOD object to be specified, which defines the type of BIO the new object will be. We’ll discuss the available BIO_METHOD objects in the next two sections. If the BIO is created successfully, it will be returned. If an error occurred in creating the BIO, NULL will be returned.

The BIO *BIO_new(BIO_METHOD *type);

Once a BIO is created, its BIO_METHOD can be changed to some other type using the BIO_set function, which will return 0 if an error occurs; otherwise, the return will be nonzero to indicate success. You should take care in using BIO_set, particularly if the BIO is part of a chain because the call will improperly break the chain.

int BIO_set(BIO *bio, BIO_METHOD *type);

When a BIO is no longer needed, it should be destroyed. The function BIO_free will destroy a single BIO and return nonzero if it was successfully destroyed; otherwise, it will return 0.

int BIO_free(BIO *bio);

The BIO_vfree function is identical to BIO_free except that it does not return a value.

void BIO_vfree(BIO *bio);

The BIO_free_all function can be used to free an entire chain of BIOs. When using BIO_free_all, you must ensure that you specify the BIO that is the head of the chain, which is usually a filter BIO. If the BIO that you wish to destroy is part of a chain, you must first remove it from the chain before calling BIO_free or BIO_vfree; otherwise, the chain will end up with a dangling pointer to the BIO that you’ve destroyed.

void BIO_free_all(BIO *bio);

The BIO_push and BIO_pop functions are poorly named because they imply that a stack is being operated on, but in fact, there is no stack.

The BIO_push function will append a BIO to a BIO, either creating or lengthening a BIO chain. The returned BIO will always be the BIO that was initially specified as the head of the chain. In other words, the return value will be the same as the first argument, bio.

BIO *BIO_push(BIO *bio, BIO *append);

bio: The BIO that should have another BIO, typically a filter BIO, appended to its chain.
append: The BIO that should be appended to the chain.

The BIO_pop function will remove the specified BIO from the chain that it is part of and return the next BIO in the chain or NULL if there is no next BIO.

BIO *BIO_pop(BIO *bio);

bio: The BIO that should be removed from the chain of which it is a part.

BIO_read behaves almost identically to the C runtime function read. The primary difference between the two is in how the return value is interpreted. For both functions, a return value that is greater than zero is the number of bytes that were successfully read. A return value of 0 indicates that no data is currently available to be read. For the C read function, a return value of -1 indicates that an error occurred. Often, this is the case with BIO_read as well, but it doesn’t necessarily mean that an error has occurred. We’ll talk more about this in a moment.

int BIO_read(BIO *bio, void *buf, int len);

bio: The first BIO in a chain that will be used for reading data. If there is no chain, this is a source BIO; otherwise, it should be a filter BIO.
buf: The buffer that will receive the data that is read.
len: The number of bytes to read. It may be less than the actual buffer size, but it should never be larger.

Another function that is provided for reading data from a source BIO is BIO_gets , which usually behaves almost identically to its C runtime counterpart, fgets. In general, you should probably avoid using this function if you can, because it is not supported by all types of BIOs, and some types of BIOs may behave differently than you might expect. Normally, though, it will read data until it finds an end-of-line character or the maximum number of bytes are read, whichever happens first. If an end-of-line character is read, it will be included in the buffer. The return value from this function is the same as for BIO_read.

int BIO_gets(BIO *bio, char *buf, int len);

bio: The first BIO in a chain that will be used for reading data. If there is no chain, this is a source BIO; otherwise, it should be a filter BIO.
buf: The buffer that will receive the data that is read.
len: The maximum number of bytes to read. This length should include space for the NULL terminating character, and of course, should never exceed the size of the buffer that will receive the data.

Corresponding to BIO_read for reading from a source BIO is BIO_write , which writes data to a sink BIO. It behaves almost identically to the C runtime function write. The primary difference between the two is in how the return value is interpreted, as is true for BIO_read, as we just described. The return value is interpreted in much the same way as it is for BIO_read and BIO_gets, with the difference being that a positive value indicates the number of bytes that were successfully written.

int BIO_write(BIO *bio, const void *buf, int len);

bio: The first BIO in a chain that will be used for writing data. If there is no chain, this is a sink BIO; otherwise, it should be a filter BIO.
buf: The buffer that contains the data to be written.
len: The number of bytes from the buffer that should be written. It may be less than the actual buffer size, but it should never be larger.

BIO_puts interprets the specified buffer as a C-style string and attempts to write it out in its entirety. The buffer must contain a NULL terminating character, but it will not be written out with the rest of the data. The return value from this function is interpreted the same as it is for BIO_write.

int BIO_puts(BIO *bio, const char *buf);

bio: The first BIO in a chain that will be used for writing data. If there is no chain, this is a sink BIO; otherwise, it should be a filter BIO.
buf: The buffer that contains the data to be written.

We mentioned that for each of the four reading and writing functions, a 0 or -1 return value may or may not necessarily indicate that an error has occurred. A suite of functions is provided that allows us to determine whether an error really did occur, and whether we should retry the operation.

If BIO_should_retry returns a nonzero value, the call that caused the condition should be retried later. If it returns 0, the actual error condition is determined by the type of BIO. For example, if BIO_read and BIO_should_retry both return 0 and the type of BIO is a socket, the socket has been closed.

int BIO_should_retry(BIO *bio);

If BIO_should_read returns nonzero, the BIO needs to read data. As an example, this condition could occur when a filter BIO is decrypting a block cipher, and a complete block has not been read from the source. In such a case, the block would need to be completely read in order for the data to be successfully decrypted.

int BIO_should_read(BIO *bio);

If BIO_should_write returns nonzero, the BIO needs to write data. This condition could possibly occur when more data is required to satisfy a block cipher’s need to fill a buffer before it can be encrypted.

int BIO_should_write(BIO *bio);

If BIO_should_io_special returns nonzero, an exceptional condition has occurred, and the meaning is entirely dependent on the type of BIO that caused the condition. For example, with a socket BIO, this could mean that out-of-band data has been received.

int BIO_should_io_special(BIO *bio);

The function BIO_retry_type returns a bit mask that describes the condition. Possible bit fields include BIO_FLAGS_READ, BIO_FLAGS_WRITE, and BIO_FLAGS_IO_SPECIAL. It is conceivable that more than one bit could be set, but with the types of BIOs that are currently included as part of OpenSSL, only one will ever be set. The functions BIO_should_read, BIO_should_write, and BIO_should_io_special are implemented as macros that test the three bits corresponding to their names.

int BIO_retry_type(BIO *bio);

The function BIO_get_retry_BIO will return a pointer to the BIO in the BIO chain that caused the retry condition. If its second argument, reason, is not NULL, it will be loaded with the reason code for the retry condition. The retry condition doesn’t necessarily have to be caused by a source/sink BIO, but can be caused by a filter BIO as well.

BIO *BIO_get_retry_BIO(BIO *bio, int *reason);

The function BIO_get_retry_reason returns the reason code for the retry operation. The retry condition must be a special condition, and the BIO passed must be the BIO that caused the condition. In most cases, the BIO passed to BIO_get_retry_reason should be the BIO that is returned by BIO_get_retry_BIO.

int BIO_get_retry_reason(BIO *bio);

In many cases, BIO_flush will do nothing, but in cases in which buffered I/O is involved, it will force any buffered data to be written. For example, with a buffered file sink, it’s effectively the same as calling fflush on the FILE object attached to the BIO.

int BIO_flush(BIO *bio);

Source/Sink BIOs

A BIO that is used for reading is known as a source BIO, and a sink BIO is one that is used for writing. A source/sink BIO is attached to a concrete input/output medium such as a file, a socket, or memory. Only a single source/sink BIO may exist in a chain. It is possible to conceive of situations in which it might be useful to have more than one, particularly for writing, but the source/sink types of BIOs provided by OpenSSL do not currently allow for more than one source/sink BIO to exist in a chain.

OpenSSL provides nine source/sink types of BIOs that can be used with BIO_new and BIO_set. A function is provided for each that simply returns a BIO_METHOD object suitable for passing to BIO_new or BIO_set. Most of the source/sink types of BIOs require additional setup work beyond just creating a BIO with the appropriate BIO_METHOD. We’ll cover only the four most commonly used types in any detail here due to space limitations and the huge number of individual functions that are available to operate on them in various ways.

Memory sources/sinks

A memory BIO treats a memory segment the same as a file or socket, and can be created by using BIO_s_mem to obtain a BIO_METHOD object suitable for use with BIO_new and BIO_set. As an alternative, the function BIO_new_mem_buf can be used to create a read-only memory BIO, which requires a pointer to an existing memory segment for reading as well as the size of the buffer. If the size of the buffer is specified as -1, the buffer is assumed to be a C-style string, and the size of the buffer is computed to be the length of the string, not including the NULL terminating character.

When a memory BIO is created using BIO_new and BIO_s_mem, a new memory segment is created, and resized as necessary. The memory segment is owned by the BIO in this case and is destroyed when the BIO is destroyed unless BIO_set_close prevents it. BIO_get_mem_data or BIO_get_mem_ptr can be used to obtain a pointer to the memory segment. A memory BIO created with BIO_new_mem_buf will never destroy the memory segment attached to the BIO, regardless of whether BIO_set_close is used to enable it. Example 4-4 demonstrates how to create a memory BIO.

Example 4-4. Creating a memory BIO

/* Create a read/write BIO */
bio = BIO_new(BIO_s_mem(  ));
 
/* Create a read-only BIO using an allocated buffer */
buffer = malloc(4096);
bio = BIO_new_mem_buf(buffer, 4096);
 
/* Create a read-only BIO using a C-style string */
bio = BIO_new_mem_buf("This is a read-only buffer.", -1);
 
/* Get a pointer to a memory BIO's memory segment */
BIO_get_mem_ptr(bio, &buffer);
 
/* Prevent a memory BIO from destroying its memory segment when it is destroyed
 */
BIO_set_close(bio, BIO_NOCLOSE);

File sources/sinks

Two types of file BIOs are available: buffered and unbuffered. A buffered file BIO is a wrapper around the standard C runtime FILE object and its related functions. An unbuffered file BIO is a wrapper around a file descriptor and its related functions. With the exception of how the two different types of file BIOs are created, the interface for using them is essentially the same.

A buffered file BIO can be created by using BIO_s_file to obtain a BIO_METHOD object suitable for use with BIO_new and BIO_set. Alternatively, BIO_new_file can be used the same way as the standard C runtime function, fopen, is used, or BIO_new_fp can be used to create a BIO around an already existing FILE object. Using BIO_new_fp, you must specify the FILE object to use and a flag indicating whether the FILE object should be closed when the BIO is destroyed.

An unbuffered file BIO can be created by using BIO_s_fd to obtain a BIO_METHOD object suitable for use with BIO_new and BIO_set. Alternatively, BIO_new_fd can be used in the same way that BIO_new_fp cis used for buffered BIOs. The difference is that a file descriptor rather than a FILE object must be specified.

For either a buffered or an unbuffered file BIO created with BIO_new or BIO_set, additional work must be done to make the BIO usable. Initially, no underlying file object is attached to the BIO, and any read or write operations performed on the BIO always fail. Unbuffered file types of BIOs require that BIO_set_fd be used to attach a file descriptor to the BIO. Buffered file types of BIOs require that BIO_set_file be used to attach a FILE object to the BIO, or one of BIO_read_filename, BIO_write_filename, BIO_append_filename, or BIO_rw_filename be used to create an underlying FILE object with the appropriate mode for the BIO. Example 4-5 shows how to create a file BIO.

Example 4-5. Creating a file BIO

/* Create a buffered file BIO with an existing FILE object that will be closed
   when the BIO is destroyed. */
file = fopen("filename.ext", "r+");
bio = BIO_new(BIO_s_file(  ));
BIO_set_file(bio, file, BIO_CLOSE);
 
/* Create an unbuffered file BIO with an existing file descriptor that will not
   be closed when the BIO is destroyed. */
fd = open("filename.ext", O_RDWR);
bio = BIO_new(BIO_s_fd(  ));
BIO_set_fd(bio, fd, BIO_NOCLOSE);
 
/* Create a buffered file BIO with a new FILE object owned by the BIO */
bio = BIO_new_file("filename.ext", "w");
 
/* Create an unbuffered file BIO with an existing file descriptor that will be
   closed when the BIO is destroyed. */
fd = open("filename.ext", O_RDONLY);
bio = BIO_new_fd(fd, BIO_CLOSE);

Socket sources/sinks

There are three types of socket BIOs. The simplest is a socket BIO that must have an already existing socket descriptor attached to it. Such a BIO can be created using BIO_s_socket to obtain a BIO_METHOD object suitable for use with BIO_new and BIO_set. The socket descriptor can then be attached to the BIO using BIO_set_fd . This type of BIO works almost like an unbuffered file BIO. Alternatively, BIO_new_socket can be used in the same way that BIO_new_fd works for unbuffered file BIOs.

The second type of BIO socket is a connection socket. This type of BIO creates a new socket that is initially unconnected. The IP address and port to connect to must be set, and the connection established before data can be read from or written to the BIO. BIO_s_connect is used to obtain a BIO_METHOD object suitable for use with BIO_new and BIO_set. To set the address, either BIO_set_conn_hostname can be used to set the hostname or BIO_set_conn_ip can be used to set the IP address in dotted decimal form. Both functions take the connection address as a C-style string. The port to connect to is set using BIO_set_conn_port or BIO_set_conn_int_port . The difference between the two is that BIO_set_conn_port takes the port number as a string, which can be either a port number or a service name such as “http” or “https”, and BIO_set_conn_int_port takes the port number as an integer. Once the address and port are set for making a connection, an attempt to establish a connection can be made via BIO_do_connect . Once a connection is successfully established, the BIO can be used just as if it was a plain socket BIO.

The third type of BIO socket is an accept socket. This type of BIO creates a new socket that will listen for incoming connections and accept them. When a connection is established, a new BIO object is created that is bound to the accepted socket. The new BIO object is chained to the original BIO and should be disconnected from the chain before use. Data can be read or written with the new BIO object. The original BIO object can then be used to accept more connections.

In order to create an accept socket type of socket BIO, use BIO_s_accept to obtain a BIO_METHOD object suitable for use with BIO_new and BIO_set. The port used to listen for connections must be set before the BIO can be placed into listening mode. This can be done using BIO_set_accept_port , which accepts the port as a string. The port can be either a number or the name of a service, just like with BIO_set_conn_port . Once the port is set, BIO_do_accept will place the BIO’s socket into listening mode. Successive calls to BIO_do_accept will block until a new connection is established. Example 4-6 demonstrates.

Example 4-6. Creating a socket BIO

/* Create a socket BIO attached to an already existing socket descriptor.  The
   socket descriptor will not be closed when the BIO is destroyed. */
bio = BIO_new(BIO_s_socket(  ));
BIO_set_fd(bio, sd, BIO_NOCLOSE);
 
/* Create a socket BIO attached to an already existing socket descriptor.  The
   socket descriptor will be closed when the BIO is destroyed. */
bio = BIO_new_socket(sd, BIO_CLOSE);
 
/* Create a socket BIO to establish a connection to a remote host. */
bio = BIO_new(BIO_s_connect(  ));
BIO_set_conn_hostname(bio, "www.ora.com");
BIO_set_conn_port(bio, "http");
BIO_do_connect(bio);
 
/* Create a socket BIO to listen for an incoming connection. */
bio = BIO_new(BIO_s_accept(  ));
BIO_set_accept_port(bio, "https");
BIO_do_accept(bio); /* place the underlying socket into listening mode */
for (;;)
{
    BIO_do_accept(bio); /* wait for a new connection */
    new_bio = BIO_pop(bio);
    /* new_bio now behaves like a BIO_s_socket(  ) BIO */
}

BIO pairs

The final type of source/sink BIO that we’ll discuss is a BIO pair. A BIO pair is similar to an anonymous pipe,^[1] but does have one important difference. In a BIO pair, two source/sink BIOs are bound together as peers so that anything written to one can be read from the other. Similarly, an anonymous pipe creates two endpoints, but only one can be written to, and the other is read from. Both endpoints of a BIO pair can be read to and written from.

A BIO pair can be formed by joining two already existing BIO objects, or two new BIO objects can be created in a joined state. The function BIO_make_bio_pair will join two existing BIO objects created using the BIO_METHOD object returned from the BIO_s_bio function. It accepts two parameters, each one a BIO that will be an endpoint in the resultant pair. When a BIO is created using BIO_s_bio to obtain a BIO_METHOD suitable for use with BIO_new, it must be assigned a buffer with a call to BIO_set_write_buf_size , which accepts two parameters. The first is the BIO to assign the buffer to, and the second is the size in bytes of the buffer to be assigned.

New BIO objects can be created already joined with the convenience function BIO_new_bio_pair, which accepts four parameters. The first and third parameters are pointers to BIO objects that will receive a pointer to each newly created BIO object. The second and fourth parameters are the sizes of the buffers to be assigned to each half of the BIO pair. If an error occurs, such as an out of memory condition, the function will return zero; otherwise, it will return nonzero.

The function BIO_destroy_bio_pair will sever the pairing of the two endpoints in a BIO pair. This function is useful when you want to break up a pair and reassign one or both of the endpoints to other potential endpoints. The function accepts one parameter, which is one of the endpoints in a pair. It should only be called on one half of a pair, not both. Calling BIO_free will also cleanly sever a pair, but will only free the one endpoint of the pair that is passed to it.

One of the useful features of BIO pairs is their ability to use the SSL engine (which requires the use of BIO objects) while maintaining control over the low-level IO primitives. For example, you could provide an endpoint of a BIO pair to the SSL engine for reading and writing, and then use the other end of the endpoint to read and write the data however you wish. In other words, if the SSL engine writes to the BIO, you can read that data from the other endpoint and do what you wish with it. Likewise, when the SSL engine needs to read data, you write to the other endpoint, and the SSL engine will read it. Included in the OpenSSL distribution is a test application (the source file is ssl/ssltest.c) that is a good example of how to use BIO pairs. It implements a client and a server in the same application. The client and the server talk to each other within the same application without requiring sockets or some other low-level communication mechanism. Example 4-7 demonstrates how BIO pairs can be created, detached, and reattached.

Example 4-7. Creating BIO pairs

a = BIO_new(BIO_s_bio(  ));
BIO_set_write_buf_size(a, 4096);
b = BIO_new(BIO_s_bio(  ));
BIO_set_write_buf_size(b, 4096);
BIO_make_bio_pair(a, b);
 
BIO_new_bio_pair(&a, 8192, &b, 8192);
 
c = BIO_new(BIO_s_bio(  ));
BIO_set_write_buf_size(c, 1024);
BIO_destroy_bio_pair(a); /* disconnect a from b */
BIO_make_bio_pair(a, c);

Filter BIOs

A filter BIO by itself provides no utility. It must be chained with a source/sink BIO and possibly other filter BIOs to be useful. The ability to chain filters with other BIOs is perhaps the most powerful feature of OpenSSL’s BIO package, and it provides a great deal of flexibility. A filter BIO often performs some kind of translation of data before writing to or after reading from a concrete medium, such as a file or socket.

Creating BIO chains is reasonably simple and straightforward; however, care must be taken to keep track of the BIO that is at the end of the chain so that the chain can be manipulated and destroyed safely. If you destroy a BIO that is in the middle of a chain without first removing it from the chain, it’s a safe bet that your program will crash shortly thereafter. As we mentioned earlier, the BIO package is one of OpenSSL’s lower-level packages, and as such, little error checking is done. This places the burden on the programmer to be sure that any operations performed on a BIO chain are both legal and error-free.

When creating a chain, you must also ensure that you create the chain in the proper order. For example, if you use filters that perform base64 conversion and encryption, you would probably want to perform base64 encoding after encryption, not before. It’s also important to ensure that your source/sink BIO is at the end of the chain. If it’s not, none of the filters in the chain will be used.

The interface for creating a filter BIO is similar to creating source/sink BIO. BIO_new is used to create a new BIO with the appropriate BIO_METHOD object. Filter BIOs are provided by OpenSSL for performing encryption and decryption, base64 encoding and decoding, computing message digests, and buffering. There are a handful of others as well, but they are of limited use, since they are either platform-specific or meant for testing the BIO package.

The function shown in Example 4-8 can be used to write data to a file using the BIO package. What’s interesting about the function is that it creates a chain of four BIOs. The result is that the data written to the file is encrypted and base64 encoded with the base64 encoding performed after the data is encrypted. The data is first encrypted using outer triple CBC DES and the specified key. The encrypted data is then base64-encoded before it is written to the file through an in-memory buffer. The in-memory buffer is used because triple CBC DES is a block cipher, and the two filters cooperate to ensure that the cipher’s blocks are filled and padded properly. Chapter 6 discusses symmetric ciphers in detail.

Example 4-8. Assembling and using a BIO chain

int write_data(const char *filename, char *out, int len, unsigned char *key)
{
    int total, written;
    BIO *cipher, *b64, *buffer, *file;
 
    /* Create a buffered file BIO for writing */
    file = BIO_new_file(filename, "w");
    if (!file)
        return 0;
 
    /* Create a buffering filter BIO to buffer writes to the file */
    buffer = BIO_new(BIO_f_buffer(  ));
 
    /* Create a base64 encoding filter BIO */
    b64 = BIO_new(BIO_f_base64(  ));
 
    /* Create the cipher filter BIO and set the key.  The last parameter of
       BIO_set_cipher is 1 for encryption and 0 for decryption */
    cipher = BIO_new(BIO_f_cipher(  ));
    BIO_set_cipher(cipher, EVP_des_ede3_cbc(  ), key, NULL, 1);
 
    /* Assemble the BIO chain to be in the order cipher-b64-buffer-file */
    BIO_push(cipher, b64);
    BIO_push(b64, buffer);
    BIO_push(buffer, file);
 
    /* This loop writes the data to the file.  It checks for errors as if the
       underlying file were non-blocking */
    for (total = 0;  total < len;  total += written)
    {
        if ((written = BIO_write(cipher, out + total, len - total)) <= 0)
        {
            if (BIO_should_retry(cipher))
            {
                written = 0;
                continue;
            }
            break;
        }
    }
 
    /* Ensure all of our data is pushed all the way to the file */
    BIO_flush(cipher);
 
    /* We now need to free the BIO chain. A call to BIO_free_all(cipher) would
       accomplish this, but we'll first remove b64 from the chain for
       demonstration purposes. */
    BIO_pop(b64);
 
    /* At this point the b64 BIO is isolated and the chain is cipher-buffer-file.
       The following frees all of that memory */
    BIO_free(b64);
    BIO_free_all(cipher);
}

Random Number Generation

Many functions throughout the OpenSSL library require the availability of random numbers. For example, creating session keys and generating public/private key pairs both require random numbers. To meet this requirement, the RAND package provides a cryptographically strong pseudorandom number generator (PRNG). This means that the “random” data it produces isn’t truly random, but it is computationally difficult to predict.

Cryptographically secure PRNGs, including those of the RAND package, require a seed . A seed is essentially a secret, unpredictable piece of data that we use to set the initial state of the PRNG. The security of this seed is the basis for the unpredictability of the output. Using the seed value, the generator can use mathematical and cryptographic transforms to ensure that its output cannot be determined. Ideally, the seed should be high in entropy . Entropy is a measurement of how random data is. To illustrate, let’s consider generating a bit of data by flipping a fair coin. The resulting bit would have a 50% chance of being 0, and a 50% chance of being 1. The output can be said to have one bit of entropy. We can also say that the value of the bit is truly random. If the coin flip was not fair, then we would have less than a bit of entropy, indicating that the resulting output isn’t truly random.

It is difficult for a deterministic machine like a computer to produce true entropy. Often, entropy is collected in small bits from all sorts of unpredictable events such as the low-order bits of the time between keystrokes, thread exits, and hard-disk interrupts. It’s hard to determine how much entropy actually exists in a piece of data, though. It’s fairly common to overestimate how much entropy is available.

In general, entropy is unpredictable data, whereas pseudorandom numbers generated by a PRNG are not unpredictable at all if both the algorithm and the seed are known. Aside from using entropic data to seed the PRNG, it’s also a good idea to use pure entropy for generating important keys. If we generate a 256-bit key using a pseudorandom number generator that has a 128-bit seed, then our key does not contain 256-bits of strength, despite its length. At most, it has 128 bits. Similarly, if multiple keys are generated using the same seed, there will be correlations between the keys that are undesirable. The security of the keys should be independent.

For all other random number requirements, pseudorandom numbers generated by the PRNG are suitable for use.

Seeding the PRNG

A common security pitfall is the incorrect seeding of the OpenSSL PRNG. There are functions that seed the generator easily enough, but the problems occur when a developer uses some predictable data for the seed. While the internal routines can quantify the amount of “seed” data, they can do nothing to determine the quality of that data (i.e., how much entropy the data contains). We’ve stated that the seed is an important value, but we haven’t explicitly looked at why this is so. For example, when using a session key to secure a connection, the basis for security is both the encryption algorithm used to encrypt the messages and the inability of the attacker to simply guess the session key. If an insecure seed is used, the PRNG output is predictable. If the output is predictable, the keys generated are predictable; thus the security of even a correctly designed application will be compromised. Clearly, a lot depends on the PRNG’s output and as such, OpenSSL provides several functions for manipulating it. It’s important to understand how to use these functions so that security can be assured.

The function RAND_add seeds the PRNG with the specified data, considering only the specified number of bytes to be entropic. For example, suppose the buffer contained a pointer to the current time as returned by the standard C runtime function, time. The buffer size would be four bytes, but only a single byte of that could be reasonably considered entropic because the high bytes don’t change frequently and are extremely predictable. The current time by itself is never a good source of entropy; we’ve only used it here for clarity.

void RAND_add(const void *buf, int num, double entropy);

buf: The buffer that contains the data to be used as the seed for the PRNG.
num: The number of bytes contained in the buffer.
entropy: An estimate of the quantity of entropy contained in the buffer.

Like RAND_add, the function RAND_seed seeds the PRNG with the specified data, but considers it to contain pure entropy. In fact, the default implementation of RAND_seed is simply to call RAND_add using the number of bytes in the buffer as the amount of entropy contained in the buffer’s data.

void RAND_seed(const void *buf, int num);

buf: The buffer that contains the data to be used as the seed for the PRNG.
num: The number of bytes contained in the buffer.

Two additional functions are provided for use on Windows systems. They’re not the best sources of entropy, but lacking a better source, they’re better than what most programmers would typically use or devise on their own. In general, it’s a good idea to avoid using either of these two functions unless there is no other entropy source available, especially if your application is running on a machine that ordinarily has no user interaction, such as a server. They’re intended to be a last resort, and you should treat them as such.

int  RAND_event(UINT iMsg, WPARAM wParam, LPARAM lParam);

RAND_event should be called from message handling functions and pass each message’s identifier and parameters. The current implementation uses only the WM_KEYDOWN and WM_MOUSEMOVE messages for gathering entropy.

void RAND_screen(void);

RAND_screen can be called periodically to gather entropy as well. The function will take a snapshot of the contents of the screen, generate a hash for each scan-line, and use the hash value as entropy. This function should not be called too frequently for a couple of reasons. One reason is that the screen won’t change much, which can lead to predictability. The other reason is that the function is not particularly fast.

A common misuse of the PRNG seeding functions is to use a static string as the seed buffer. Most often, this is done for no reason other than to silence OpenSSL because it will generate warning messages whenever the PRNG is not seeded and an attempt to use it is made. Another bad idea is to use an uninitialized memory segment, assuming its contents will be unpredictable enough. There are plenty of other examples of how not to seed the PRNG, but rather than enumerate them all here, we’ll concentrate on the right way. A good rule of thumb to determine whether you’re seeding the PRNG correctly is this: if you’re not seeding it with data from a service whose explicit purpose is to gather entropy, you’re not seeding the PRNG correctly.

On many Unix systems, /dev/random is available as an entropy-gathering service. On systems that provide such a device, there is usually another device, /dev/urandom. The reason for this is that the /dev/random device will block if there is not enough entropy available to produce the output requested. The /dev/urandom device, on the other hand, will use a cryptographic PRNG to assure that it never blocks. It’s actually most accurate to say that /dev/random produces entropy and that /dev/urandom produces pseudorandom numbers.

The OpenSSL package provides a function, RAND_load_file , which will seed the PRNG with the contents of a file up to the number of bytes specified, or its entirety if the limit is specified as -1. It is expected that the file read will contain pure entropy. Since OpenSSL has no way of knowing whether the file actually does contain pure entropy, it assumes that the file does; OpenSSL leaves it to the programmer. Example 4-9 shows some example uses of this function and its counterpart, RAND_write_file . On systems that do have /dev/random available, seeding the PRNG with RAND_load_file from /dev/random is the best thing to do. Be sure to limit the number of bytes read from /dev/random to some reasonable value, though! If you specify -1 to read the entire file, RAND_load_file will read data forever and never return.

The RAND_write_file function will write 1,024 bytes of random bytes obtained from the PRNG to the specified file. The bytes written are not purely entropic, but they can be safely used to seed an unseeded PRNG in the absence of a better entropy source. This can be particularly useful for a server that starts running immediately when a system boots up because /dev/random will not have much entropy available when the system first boots. Example 4-9 demonstrates various methods of employing RAND_load_file and RAND_write_file.

Example 4-9. Using RAND_load_file( ) and RAND_write_file( )

int RAND_load_file(const char *filename, long bytes);
int RAND_write_file(const char *filename);
 
/* Read 1024 bytes from /dev/random and seed the PRNG with it */
RAND_load_file("/dev/random", 1024);
 
/* Write a seed file */
RAND_write_file("prngseed.dat");
 
/* Read the seed file in its entirety and print the number of bytes obtained */
nb = RAND_load_file("prngseed.dat", -1);
printf("Seeded the PRNG with %d byte(s) of data from prngseed.dat.\n", nb);

When you write seed data to a file with RAND_write_file, you must be sure that you’re writing the file to a secure location. On a Unix system, this means the file should be owned by the user ID of the application, and all access to group members and other users should be disallowed. Additionally, the directory in which the file resides and all parent directories should have only write access enabled for the directory owner. On a Windows system, the file should be owned by the Administrator and allow no permissions to any other users.

One final point worth mentioning is that OpenSSL will try to seed the PRNG transparently with /dev/urandom on systems that have it available. While this is better than nothing, it’s a good idea to go ahead and read better entropy from /dev/random, unless there is a compelling reason not to. On systems that don’t have /dev/urandom, the PRNG will not be seeded at all, and you must make sure that you seed it properly before you attempt to use the PRNG or any other part of OpenSSL that utilizes the PRNG. For systems that have /dev/random, Example 4-10 demonstrates how to use it to seed the OpenSSL PRNG.

Example 4-10. Seeding OpenSSL’s PRNG with /dev/random

int seed_prng(int bytes)
{
    if (!RAND_load_file("/dev/random", bytes))
        return 0;
    return 1;
}

Using an Alternate Entropy Source

We’ve discussed /dev/random and /dev/urandom as entropy sources at some length, but what about systems that don’t have these services available? Many operating systems do not provide them, including Windows. Obtaining entropy on such systems can be problematic, but luckily, there is a solution. Several third-party packages are available for various platforms that perform entropy-gathering services. One of the more full-featured and portable solutions available is EGADS (Entropy Gathering and Distribution System). It’s licensed under the BSD license, which means that it’s free and the source code is available. You can obtain a copy of EGADS from http://www.securesw.com/egads/.

As we mentioned, there are other entropy solutions available in addition to EGADS. EGD is an entropy-gathering daemon that is written in Perl by Brian Warner and is available from http://egd.sourceforge.net/. Because it is written in Perl, it requires a Perl interpreter to be installed. It provides a Unix domain socket interface for clients to obtain entropy. It does not support Windows at all. PRNGD is another popular entropy-gathering daemon written by Lutz Jänicke. It provides an EGD-compatible interface for clients to obtain entropy from it; like EGD itself, Windows is not supported. Because neither EGD nor PRNGD support Windows, we’ll concentrate primarily on EGADS, which does support Windows. Where appropriate, we will also discuss EGD and PRNGD together, because all three use the same interface.

Before we can use EGADS to obtain entropy, we must first initialize it. This is done with a simple call to egads_init. Once the library is initialized, we can use the function egads_entropy to obtain entropy. Like /dev/random on systems that make it available, egads_entropy will block until enough entropy is available to satisfy the request. Example 4-11 shows how to use EGADS to seed OpenSSL’s PRNG.

Example 4-11. Seeding OpenSSL’s PRNG with EGADS

int seed_prng(int bytes)
{
    int       error;
    char      *buf;
    prngctx_t ctx;
 
    egads_init(&ctx, NULL, NULL, &error);
    if (error)
        return 0;
 
    buf = (char *)malloc(bytes);
    egads_entropy(&ctx, buf, bytes, &error);
    if (!error)
        RAND_seed(buf, bytes);
    free(buf);
 
    egads_destroy(&ctx);
    return (!error);
}

EGADS, EGD, and PRNGD all provide a Unix domain socket that allows clients to obtain entropy. EGD defines a simple protocol for clients to communicate with that both EGADS and PRNGD have mimicked. Many cryptographic applications, such as GnuPG and OpenSSH, provide support for obtaining entropy from a daemon using the EGD protocol. OpenSSL also provides support for seeding its PRNG using the EGD protocol.

OpenSSL provides two functions for communicating with a server that speaks the EGD protocol. Version 0.9.7 of OpenSSL adds a third. In addition, Version 0.9.7 will attempt to automatically connect to four different commonly used names for EGD sockets in the following order: /var/run/egd-pool, /dev/egd-pool, /etc/egd-pool, and /etc/entropy.

RAND_egd attempts to connect to the specified Unix domain socket. If the connection is successful, 255 bytes of entropy will be requested from the server. The data returned will be passed in a call to RAND_add to seed the PRNG. RAND_egd is actually a wrapper around the next function, RAND_egd_bytes .

int RAND_egd(const char *path);

RAND_egd_bytes will attempt to connect to the specified Unix domain socket. If the connection is successful, the specified number of bytes of entropy will be requested from the server. The data returned will be passed in a call to RAND_add to seed the PRNG. Both RAND_egd and RAND_egd_bytes will return the number of bytes obtained from the EGD server if they’re successful. If an error occurred connecting to the daemon, they’ll both return -1.

int RAND_egd_bytes(const char *path, int bytes);

Version 0.9.7 of OpenSSL adds the function RAND_query_egd_bytes to make a query for data from an EGD server without automatically feeding the returned data into OpenSSL’s PRNG via RAND_add. It attempts to connect to the specified Unix domain socket and obtain the specified number of bytes. The data that is returned from the EGD server is copied into the specified buffer. If the buffer is specified as NULL, the function works just like RAND_egd_bytes and passes the returned data to RAND_add to seed the PRNG. It returns the number of bytes received on success; otherwise, it returns -1 if an error occurs.

int RAND_query_egd_bytes(const char *path, unsigned char *buf, int bytes);

Example 4-12 demonstrates how to use the RAND functions to access an EGD socket and seed the PRNG with the entropy that is obtained from a running entropy-gathering server, whether it’s EGADS, EGD, PRNGD, or another server that provides an EGD socket interface.

Example 4-12. Seeding OpenSSL’s PRNG via an EGD socket

#ifndef DEVRANDOM_EGD
#define DEVRANDOM_EGD "/var/run/egd-pool", "/dev/egd-pool", "/etc/egd-pool", \
                      "/etc/entropy"
#endif
 
int seed_prng(int bytes)
{
    int  i;
    char *names[] = { DEVRANDOM_EGD, NULL };
 
    for (i = 0;  names[i];  i++)
        if (RAND_egd(names[i]) != -1)  /* RAND_egd_bytes(names[i], 255) */
            return 1;
    return 0;
}

Arbitrary Precision Math

To implement many public key encryption algorithms, the library must have support for mathematical operations on large integers. Use of standard C or C++ data types is not adequate in these situations. To alleviate this problem, OpenSSL provides the BN package. This package declares routines on the aggregate type BIGNUM , which have virtually no limits on the upper bounds of numbers. More specifically, the size of the number that a BIGNUM-typed variable can hold is limited only by available memory.

It’s likely that direct exposure to the BN package in developing an SSL-enabled application will be limited since it is a very low-level package, and the higher-level packages generally hide the details. However, because the package is so widely used and integral to public key cryptography, we’ve briefly covered it here.

The Basics

To use the BIGNUM package in your programs, you’ll need to include the header file openssl/bn.h. Before we can use a BIGNUM , we must first initialize it. The BN package provides support for both statically allocated and dynamically allocated BIGNUMs. The function BN_new will allocate a new BIGNUM and initialize it for use. The function BN_init will initialize a statically allocated BIGNUM. When you’re done using a BIGNUM, you should always be sure to destroy it, even if it is allocated on the stack, because internally, a BIGNUM dynamically allocates memory. The function BN_free will destroy a BIGNUM. Example 4-13 shows some examples of how these three functions are used.

Example 4-13. Creating, initializing, and destroying BIGNUMs

BIGNUM static_bn, *dynamic_bn;
 
/* Initialize a statically allocated BIGNUM */
BN_init(&static_bn);
 
/* Allocate an initialize a new BIGNUM */
dynamic_bn = BN_new(  );
 
/* Destroy the two BIGNUMs that we just created */
BN_free(dynamic_bn);
BN_free(&static_bn);

A BIGNUM is implemented as an opaque structure that contains dynamically allocated memory. The functions provided to operate on a BIGNUM allow the programmer to remain blissfully unaware of what’s going on for the most part, but it is important that the programmer does understand that there is much more going on internally. One such instance is when the value of one BIGNUM needs to be assigned to another BIGNUM. The natural inclination is to perform a shallow copy of the structure, but this should never be done! Deep copies must be performed, and the BN package provides functions for doing just that. Example 4-14 demonstrates the right way and the wrong way to make a copy of a BIGNUM.

Example 4-14. The wrong way and the right way to copy a BIGNUM

BIGNUM a, b *c;
 
/* First, the wrong way to copy a BIGNUM */
a = b;
*c = b;
 
/* Now the right way to copy a BIGNUM */
BN_copy(&a,&b);    /* Copies b to a */
c = BN_dup(&b);    /* Creates c and initializes it to the same value as b */

It is important that we copy BIGNUMs properly. If we don’t, we’re likely to experience unpredictable behavior or crashes. When programming with BIGNUMs, there will likely be situations in which you’ll need to make copies, so it’s best to learn this lesson early.

Another operation that is similar in nature is the comparison of two BIGNUM s. We cannot simply compare two BIGNUMs using normal C comparison operators like =, <, or >. Instead, we must use the BN_cmp function to compare two BIGNUMs. This function will compare the two values, a and b, and return -1 if a is less than b, 0 if they are equal, and 1 if a is greater than b. The function BN_ucmp will perform the same type of comparison on the absolute values of a and b.

It may be useful to convert a BIGNUM into a flat binary representation for the purpose of storing it in a file or sending it over a socket connection to a peer. Since a BIGNUM contains pointers to internal, dynamically allocated memory, it is not a flat structure, so we must convert it to one before we can send it anywhere. Conversely, we must be able to convert a flat representation of a BIGNUM back into a BIGNUM structure that the BN package can use. Two functions are provided for performing these respective operations. The first, BN_bn2bin , will convert a BIGNUM to a flat binary representation in big-endian form. The second, BN_bin2bn , performs the inverse operation, converting a flat binary representation of a big-endian number into a BIGNUM.

Before converting a BIGNUM into a flat binary representation, we need to know the number of bytes of memory that will be required to hold the converted data. It’s also important to know how big the binary representation is before converting it back into a BIGNUM. The number of bytes required to represent a BIGNUM in flat binary form can be discovered using the BN_num_bytes function. Example 4-15 demonstrates converting between BIGNUM and flat binary representations.

Example 4-15. Converting between BIGNUM and binary representations

/* Converting from BIGNUM to binary */
len = BN_num_bytes(num);
buf = (unsigned char *)malloc(len);
len = BN_bn2bin(num, buf);
 
/* Converting from binary to BIGNUM */
BN_bin2bn(buf, len, num);
num = BN_bin2bn(buf, len, NULL);

When BN_bn2bin performs its conversion, it will return the number of bytes that were written out into the supplied buffer. If an error occurs in the conversion, the return will be 0. When BN_bin2bn performs its conversion, the result is placed into the BIGNUM specified as the third argument, overwriting any value that it may have previously held. If the third argument is specified as NULL, a new BIGNUM is created and initialized with the value from the binary representation. In either case, the BN_bin2bn will always return a pointer to the BIGNUM that received the value or NULL if an error occurred during the conversion.

Binary-encoded numbers are fine when we want to transfer the data over a medium that supports it—for example, a binary file or a socket. However, for circumstances in which we need a text-based representation, such as printing the number on the screen, it is inadequate. We can always base64-encode the data before emitting it, but the BN package provides more intuitive methods.

The function BN_bn2hex converts a BIGNUM into a hexadecimal representation stored in a C-style string. The C-style string is allocated dynamically using OPENSSL_malloc , which must then be freed by the caller using OPENSSL_free .

char *BN_bn2hex(const BIGNUM *num);

The function BN_bn2dec converts a BIGNUM into a decimal representation stored in a C-style string. The C-style string is allocated dynamically using OPENSSL_malloc, which must then be freed by the caller using OPENSSL_free.

char *BN_bn2dec(const BIGNUM *num);

The function BN_hex2bn converts a hexadecimal representation of a number stored in a C-style string to a BIGNUM. The resulting value is stored in the supplied BIGNUM, or a new BIGNUM is created with BN_new if the BIGNUM is supplied as NULL.

int BN_hex2bn(BIGNUM **num, const char *str);

The function BN_dec2bn converts a decimal representation of a number stored in a C-style string to a BIGNUM. The resulting value is stored in the supplied BIGNUM, or a new BIGNUM is created with BN_new if the BIGNUM is supplied as NULL.

int BN_dec2bn(BIGNUM **num, const char *str);

Mathematical Operations

With few exceptions, the majority of the remainder of the functions that make up the BN package all perform mathematical operations such as addition and multiplication. Most of the functions require at least two BIGNUM operands and store their result in a third BIGNUM. It is often safe to use one of the operands to store the result, but it isn’t always, so you should exercise care in doing so. Consult Table 4-1, which lists the most common arithmetic functions, if you’re not sure. Unless otherwise noted in the table, the BIGNUM that will receive the result of the operation may not be the same as any of the operands. Each of the functions returns nonzero or zero, indicating success or failure, respectively.

Many of the functions in Table 4-1 are shown as accepting an argument labeled “ctx”. This argument is a pointer to a BN_CTX structure. This argument may not be specified as NULL, but instead should be a context structure returned by BN_CTX_new. The purpose of the context structure is to store temporary values used by many of the arithmetic operations. Storing the temporary values in a context increases the performance of the various functions. When a context structure is no longer needed, it should be destroyed using BN_CTX_free .

Table 4-1. Arithmetic functions for BIGNUMs

Function	Comments
BN_add(r, a, b)	(r = a + b) r may be the same as a or b.
BN_sub(r, a, b)	(r = a - b)
BN_mul(r, a, b, ctx)	(r = a x b) r may be the same as a or b.
BN_sqr(r, a, ctx)	(r = pow(a, 2)) r may be the same as a. This function is faster than BN_mul(r, a, a).
BN_div(d, r, a, b, ctx)	(d = a / b, r = a % b) Neither d nor r may be the same as either a or b. Either d or r may be NULL.
BN_mod(r, a, b, ctx)	(r = a % b)
BN_nnmod(r, a, b, ctx)	(r = abs(a % b))
BN_mod_add(r, a, b, m, ctx)	(r = abs((a + b) % m))
BN_mod_sub(r, a, b, m, ctx)	(r = abs((a - b) % m))
BN_mod_mul(r, a, b, m, ctx)	(r = abs((a x b) % m)) r may be the same as a or b.
BN_mod_sqr(r, a, m, ctx)	(r = abs(pow(a, 2) % m))
BN_exp(r, a, p, ctx)	(r = pow(a, p))
BN_mod_exp(r, a, p, m, ctx)	(r = pow(a, 2) % m)
BN_gcd(r, a, b, ctx)	Finds the greatest common divisor of a and b. r may be the same as a or b.

Generating Prime Numbers

One of the functions provided by the BN package that is most import to public key cryptography is BN_generate_prime . As its name implies, the function generates prime numbers, but more importantly, it generates pseudorandom primes. In other words, it repeatedly chooses numbers at random until one of the choices it makes is a prime number. Such a function can be quite useful for other applications as well, which is one of the reasons why we’ve chosen to pay so much attention to it in this chapter. Another reason is because its parameter list is rather large and complex, which can make using the function seem to be a daunting task.

BIGNUM *BN_generate_prime(BIGNUM *ret, int bits, int safe, BIGNUM *add,
               BIGNUM *rem, void (*callback)(int, int, void *), void *cb_arg);

ret: Used to receive the prime number that is generated. If it is specified as NULL, a new BIGNUM will be created, initialized with BN_new, and returned.
bits: The number of bits that should be used to represent the generated prime.
safe: Either zero or nonzero, indicating whether the generated prime should be safe or not. A safe prime is defined as a prime, p, in which (p-1)/2 is also prime.
add: Used to specify additional properties that the generated prime should have. If it is specified as NULL, no additional properties will be required. Otherwise, the generated prime must satisfy the condition that when divided by this value, the remainder is one.
rem: Used to specify additional properties that the generated prime should have. If it is specified as NULL, no additional properties will be required. Otherwise, the generated prime must satisfy the condition that when divided by add, the remainder must be this value. If add is specified as NULL, this argument is ignored.
callback: A function that is called during the generation of the prime to report the status of the operation. Generating a prime can often be a rather time-consuming task, so this provides some means of advising a user that work is being done and that the program hasn’t crashed or hung.
cb_arg: A value that is used only to pass to the callback function if one is specified. OpenSSL does not use this argument for anything else and will never attempt to interpret its value or meaning.

If one is used, the callback function should accept three arguments and return no value. The third argument to the callback function is always the cb_arg argument to BN_generate_prime. The first argument passed to the callback function is a status code indicating which phase of the prime generation has just completed. The status code will always be 0, 1, or 2. The meaning of the second argument depends on the status code. When the status code is 0, it indicates that a potential prime has been found, but it has not yet been tested to ensure that it conforms to the criteria specified in the call to BN_generate_prime. The callback can be called with a status code of 0 many times, and each time the second argument will contain a counter of the number of primes that have been found so far, not including the current one. When the status code is 1, the second argument indicates the number of Miller-Rabin probabilistic primality tests that have been completed. Finally, when the status code is 2, a conforming prime has been found, and the second argument indicates the number of candidates that were tested before it. Example 4-16 demonstrates how to use the BN_generate_prime function with a callback for displaying the status of the process.

Example 4-16. Generating a pseudorandom prime number with BN_generate_prime( )

static void prime_status(int code, int arg, void *cb_arg)
{
    if (code == 0)
        printf("\n  * Found potential prime #%d ...", (arg + 1));
    else if (code == 1 && arg && !(arg % 10))
        printf(".");
    else
        printf("\n Got one!\n");
}
 
BIGNUM *generate_prime(int bits, int safe)
{
    char   *str;
    BIGNUM *prime;
 
    printf("Searching for a %sprime %d bits in size ...", (safe ? "safe " : ""),
           bits);
 
    prime = BN_generate_prime(NULL, bits, safe, NULL, NULL, prime_status, NULL);
    if (!prime)
        return NULL;
 
    str = BN_bn2dec(prime);
    if (str)
    {
        printf("Found prime: %s\n", str);
        OPENSSL_free(str);
    }
 
    return prime;
}

Using Engines

OpenSSL has built-in support for cryptographic acceleration. Using the ENGINE object type, an application can get a reference to a changeable, underlying representation, most often a hardware device. This support was built in the 0.9.6 versions of OpenSSL that included the name engine ; it will be incorporated into the main branch of OpenSSL beginning with Version 0.9.7. While 0.9.7 will have a much more robust feature specification for the ENGINE package, 0.9.6-engine contains some simple functions to set up an ENGINE object. These functions do not appear to have changed at the time of writing. If they do, we’ll update our web site with the relevant information.

The general idea is simple: we retrieve an object representing the type of hardware we wish to utilize, then we tell OpenSSL to use the device we chose. Example 4-17 shows a small code example of how we would perform this operation.

Example 4-17. Enabling use of a hardware engine

ENGINE *e;
 
if (!(e = ENGINE_by_id("cswift")))
    fprintf(stderr, "Error finding specified ENGINE\n");
else if (!ENGINE_set_default(e, ENGINE_METHOD_ALL))
    fprintf(stderr, "Error using ENGINE\n");
else
    fprintf(stderr, "Engine successfully enabled\n");

The function call ENGINE_by_id will look up an implementation from the built-in methods available and return an ENGINE object. The single argument to this function should be the string identifier of the underlying implementation we wish to use. Table 4-2 shows the available methods for supported cryptographic hardware and software.

Table 4-2. Supported hardware and software engines

ID string	Description
openssl	The engine uses the normal built-in functions for cryptographic operations. This is the default.
openbsd_dev_crypto	On the OpenBSD operating system, this engine will use the kernel level cryptography built into the operating system.
cswift	Used for CryptoSwift acceleration hardware.
chil	Used for nCipher CHIL acceleration hardware.
atalla	Used for Compaq Atalla acceleration hardware.
nuron	Used for Nuron acceleration hardware.
ubsec	Used for Broadcom uBSec acceleration hardware.
aep	Used for Aep acceleration hardware.
sureware	Used for SureWare acceleration hardware.

The ENGINE object that we receive from the lookup should be used in the call to ENGINE_set_default to allow cryptographic functions to utilize the capabilities of the specific ENGINE. The second parameter allows us to specify constraints on what we allow the engine to implement. For example, if we had an engine that implemented only RSA, making a call like the one in Example 4-17 would allow RSA to be handled by the engine. On the other hand, if we called ENGINE_set_default with our RSA engine and ENGINE_METHOD_DSA, OpenSSL would not use the engine for any cryptographic calls, since this flag allows the engine to work only on DSA functions. Table 4-3 provides a complete list of the restraints we can use. They can be combined with the logical OR operation.

Table 4-3. Flags for ENGINE_set_default

Flag	Description
ENGINE_METHOD_RSA	Limit engine usage to only RSA operations.
ENGINE_METHOD_DSA	Limit engine usage to only DSA operations.
ENGINE_METHOD_DH	Limit engine usage to only DH operations.
ENGINE_METHOD_RAND	Limit engine usage to only random number operation.
ENGINE_METHOD_CIPHERS	Limit engine usage to only symmetric ciphers operations.
ENGINE_METHOD_DIGESTS	Limit engine usage to only digest operations.
ENGINE_METHOD_ALL	Allow OpenSSL to use any of the above implementations.

Aside from setting the default engine, ENGINE objects are typically used in several other places in OpenSSL Version 0.9.7. For instance, the function EVP_EncryptInit has been deprecated and replaced by EVP_EncryptInit_ex . This “ex” function takes one additional parameter: the ENGINE object. In general, these replacement functions can be passed a NULL argument for the ENGINE, which will cause OpenSSL to use the default engine. Recall that the default engine is changed when a call to ENGINE_set_default is made; if no such call is made, the built-in software implementation is used.

The purpose of these new “ex” functions is to allow a more fine-grained control over which underlying cryptographic device is used for each call. This is particularly useful for circumstances in which we have multiple cryptographic accelerators, and we wish to utilize them differently depending on application code.

^[1]An anonymous pipe is a common operating system construct in which two file descriptors are created, but no file is created or socket opened. The two descriptors are connected to each other where one can be written to and the other read from. The data written to one half of the pipe can be read from the other half of the pipe.

Get Network Security with OpenSSL now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Network Security with OpenSSL by John Viega, Matt Messier, Pravir Chandra