Python Cookbook

Coding the Methods of a Python Class in C

Credit: Brent Burley

Problem

You have a Python class and want to recode it as a C extension for speed while keeping all client-code unchanged, so it must remain a class.

Solution

One hardly ever sees Python class objects built in C extensions. And yet it’s anything but difficult, and, in fact, it’s quite handy:

#include <Python.h>

static PyObject* Foo_init(PyObject *self, PyObject *args)
{
    printf("Foo._ _init_ _ called\n");
    Py_INCREF(Py_None);
    return Py_None;
}

static PyObject* Foo_doSomething(PyObject *self, PyObject *args)
{
    printf("Foo.doSomething called\n");
    Py_INCREF(Py_None);
    return Py_None;
}

static PyMethodDef FooMethods[] =
{
    {"_ _init_ _", Foo_init, METH_VARARGS, "doc string"},
    {"doSomething", Foo_doSomething, METH_VARARGS, "doc string"},
    {0, 0},
};

static PyMethodDef ModuleMethods[] = { {0, 0} };

#ifdef _ _cplusplus
extern "C"
#endif
void initFoo(  )
{
    PyMethodDef *def;

    /* create new module and class objects */
    PyObject *module = Py_InitModule("Foo", ModuleMethods);
    PyObject *moduleDict = PyModule_GetDict(module);
    PyObject *classDict = PyDict_New(  );
    PyObject *className = PyString_FromString("Foo");
    PyObject *fooClass = PyClass_New(NULL, classDict, className);
    PyDict_SetItemString(moduleDict, "Foo", fooClass);
    Py_DECREF(classDict);
    Py_DECREF(className);
    Py_DECREF(fooClass);

    /* add methods to class */
    for (def = FooMethods; def->ml_name != NULL; def++) {
        PyObject *func = PyCFunction_New(def, NULL);
        PyObject *method = PyMethod_New(func, NULL, fooClass);
        PyDict_SetItemString(classDict, def->ml_name, method);
        Py_DECREF(func);
        Py_DECREF(method);
    }
}

Discussion

This recipe shows how to define a new Python class from a C extension module. The class’s methods are implemented in C, but the class can still be instantiated, extended, and subclassed from Python. The same technique can also be used with inheritance to extend an existing Python class with methods written in C. In this recipe, the first argument to PyClass_New is passed as NULL, indicating that the new class has no base classes. Pass the tuple of base classes in this spot, and you’ll get normal Python inheritance behavior, even though your new class is being built in a C extension rather than in Python source code.

The usual method of creating new types in an extension module is to define a new instance of PyTypeObject and provide callbacks to the various C functions that implement the type. However, it may be better to define the new type as a Python class, so that the type can be instantiated and subclassed from Python. In some cases, when defining a custom exception type, for example, it is required that the new type be a Python class.

The methods in this recipe are coded as C functions and are described by a table of PyMethodDef statements in the same way that a module’s methods (functions) are described. The key fact that allows these functions to become unbound methods is that each of them is first wrapped in a PyCFunction object and then in a PyMethod object. The PyCFunction turns the C function into a Python object, and the PyMethod associates the function with a particular class as an unbound method. Finally, the methods are added to the class’s dictionary, which makes them callable on instances of the class.

Note that base classes can be specified for the new class by passing a tuple of class objects as the first argument to PyClass_New. These can be existing Python classes. The second argument passed to PyCFunction_New becomes the self argument passed to the C function. This can be any Python object, but it’s not very useful in most cases since you can just as easily keep a static C variable. However, it can be very handy when you want to use the same C function, associated with different data, to implement more than one Python function or method. Also note that the class instance is passed to the C functions as the first argument in the args tuple.

See Also

The Extending and Embedding manual is available as part of the standard Python documentation set at http://www.python.org/doc/current/ext/ext.html; documentation on the Python C API at http://www.python.org/doc/current/api/api.html.