Python Cookbook

Using XML-RPC with Medusa

Credit: Jeff Bauer

Problem

You need to establish a distributed processing system and want to use the XML-RPC protocol.

Solution

The medusa package lets you implement lightweight, highly scalable servers, even with old versions of Python. An XML-RPC handler is included in the Medusa distribution. Here is how you code a server with Medusa:

# xmlrpc_server.py
from socket import gethostname
from medusa.xmlrpc_handler import xmlrpc_handler
from medusa.http_server import http_server
from medusa import asyncore

class xmlrpc_server(xmlrpc_handler):
    # initializes and runs the server
    def _ _init_ _(self, host=None, port=8182):
        if host is None:
            host = gethostname(  )
        hs = http_server(host, port)
        hs.install_handler(self)
        asyncore.loop(  )

    # an example of a method to be exposed via the XML-RPC protocol
    def add(self, op1, op2):
        return op1 + op2

    # the infrastructure ("plumbing") to expose methods
    def call(self, method, params):
        print "call method: %s, params: %s" % (method, str(params))
        if method == 'add':
            return apply(self.add, params)
        return "method not found: %s" % method

if _ _name_ _ == '_ _main_ _':
    server = xmlrpc_server(  )

And here is an xmlrpclib-based client of this server:

# xmlrpc_client.py
from socket import gethostname
from xmlrpclib import Transport, dumps

class xmlrpc_connection:
    def _ _init_ _(self, host=None, port=8182):
        if host is None:
            host = gethostname(  )
        self.host = "%s:%s" % (host, port)
        self.transport = Transport(  )

    def remote(self, method, params=(  )):
        """ remote invokes the server with the method name and an optional set
        of parameters. The return value is always a tuple. """

        response = self.transport.request(self.host, '/RPC2',
                                          dumps(params, method))
        return response

if _ _name_ _ == '_ _main_ _':
    connection = xmlrpc_connection(  )
    (answer,) = connection.remote("add", (40, 2))
    print "The answer is:", answer

Discussion

This recipe demonstrates remote method calls between two machines (or two processes, even on the same machine) using the XML-RPC protocol. A complete example of working client/server code is provided. XML-RPC is one of the easiest ways to handle distributed processing tasks. There’s no messing around with the low-level socket details, nor is it necessary to write an interface definition. The protocol is platform- and language-neutral. The XML-RPC specification can be found at http://www.xml-rpc.com and is well worth studying.

With Medusa (http://www.nightmare.com), you implement an XML-RPC server by subclassing the xmlrpc_handler class and passing an instance of your class to the install_handler method of an instance of http_server. HTTP is the transport-level protocol, and http_server handles all transport-level issues on your behalf. You need to provide only the handler part by customizing xmlrpc_handler through subclassing and method overriding. Specifically, you must override the call method, which the Medusa framework calls on your instance with the name of the XML-RPC method being called, along with its parameters, as arguments. This is exactly what we do in this recipe, in which we expose a single XML-RPC method named add which accepts two numeric parameters and returns their sum as the method’s result.

The sample XML-RPC client uses xmlrpclib in a more sophisticated way than Recipe 13.2 by using the Transport class explicitly. This lets you see what happens under the covers of an XML-RPC method call a bit more transparently and also lets you control things in a finer-grained way, although we don’t use that fine-grained-control potential in this recipe (and you will need it only rarely in XML-RPC clients that you actually deploy, anyway).

xmlrpclib can also be used on its own, without separately downloading and installing Medusa, and comes with similar client and server program examples. However, the asynchronous operation of Medusa can significantly enhance performance, particularly scalability. Medusa (and asyncore and asynchat) are applicable to client- and server-side programming, but this recipe does not use the asynchronous approach in its example client, only in its example server. Of course, the benefits of the asynchronous approach come when a program does several network operations at once (in such cases, asynchronous Medusa operations can give you substantial performance benefits when compared to alternatives such as multiprocessing and multithreading). This is almost always the case for servers, which need to be able to field several requests arriving simultaneously from different clients. It’s certainly not unheard of for clients, too, if the client needs to make several requests at once.

See Also

The XML-RPC (xmlrpclib) library ships with recent versions of Python; if it isn’t in your version of Python, you can get it from http://www.pythonware.com/products/xmlrpc/; the Medusa library at http://www.nightmare.com; recent Python releases include the asyncore and asynchat modules from Medusa as parts of the Python standard library (not, however, other parts of Medusa, such as xmlrpc_handler).