Palm OS Network Programming Writing Connected Applications for the Palm By Greg Winton September 2001 0-596-00005-7, Order Number: 0057 400 pages, $39.95

Chapter 5
A Brief Tour of the Net Library

There are two layers to the Net Library: the network protocol stack and the application programming interface. The protocol stack is on the bottom, handling the nitty gritty details of network services. The programming interface sits on top, providing a way for your application to take advantage of these services. This architecture, and the details of its implementation, are the result of years of research and development on a wide variety of platforms. As a developer, you benefit from all this work. Understanding this work and the reasons behind the design decisions will help you write more efficient and stable network code.

In this chapter, we explore the Net Library from a distance. Don't worry, we dive into the details of Palm Network Programming in Chapter 6. If you're already an expert (or just really impatient), feel free to jump on ahead. We'll catch up. But if you persevere through this chapter, you'll be rewarded with a view of the architectural and functional beauty of the Net Library.

The Design of the Net Library

The Net Library provides networking services for the Palm OS. The standard implementation supports TCP and UDP over an IP network. In this section, we explore the architecture and design of the external interface presented to the networking applications, as well as the internal system that provides the networking functionality.

Architecture

The Network Library can be divided into two parts: the API and the protocol stack. The API is the set of functions that an application invokes to access the Net Library's functionality. The protocol stack handles all communication with the network: connecting, sending, and receiving. These components are connected by a mailbox queue. Figure 5-1 illustrates these relationships.

Figure 5-1. Net Library architecture

In this diagram, we see the client application invoking functions in the API. The API translates these into requests that are placed into the mailbox queue. The protocol stack reads the requests and talks to the network. The protocol stack then places a response into the mailbox queue. The API reads the mailbox queue and returns a result to the client application.

This is a fairly standard division of labor in network architectures. Mapped against the OSI network model, the protocol stack provides transport, network, and data link services. TCP and UDP are transport-layer protocols, IP is a network-layer protocol, and PPP and SLIP are data link-layer protocols. Your application needs to provide the application, presentation, and session services of the OSI model.

The Net Library API provides the interface between these layers. This is the same role that is played by the Berkeley sockets API on Unix platforms and the Winsock API on Microsoft Windows platforms. The mailbox queue is an operating system construct that allows the user interface task, under which the API is running, to communicate with the protocol stack's task.

The Protocol Stack

The Palm OS protocol stack runs as a task, separate from the UI task under which all applications run. As it turns out, the Net Library API is mostly responsible for packaging up requests for the protocol stack and putting them in the mailbox queue, and then taking responses off the mailbox queue and translating them into return values that the application expects. The protocol stack contains the meat of the network handling code. The mailbox queue sits between these two modules.

When an API call is made, it packages the relevant information into a request and adds it to the end of the mailbox queue. The protocol stack reads requests from the queue, processes them, and then adds responses to the mailbox queue. The API call waits until it receives this response to be returned by the protocol stack.

Avoiding Network Traffic Jams

Networks, as a rule, operate much more slowly than the devices they connect. In the case of a nonmultitasking environment, such as DOS or Windows 3.1, this means that everything comes to a stop for the duration of any network call. This is bad.

So TCP/IP protocol stacks provide asynchronous operation. The network call is made (packets are sent and received) but the application doesn't have to wait for the answer. Every so often, it peeks to see if there is data to send or receive

Also, modern operating systems provide multitasking. So you can create a subtask that patiently sits and waits for the network to respond. The rest of the application goes along, occasionally checking for signs that the network has replied. When the network does respond, the subtask signals that it is ready for the next network function. On the Protocol Stack side, different tasks can be connecting, reading, and writing, all at the same time. And this is even better.

On the Palm, things are a little different. Today's PC's are computing powerhouses running sophisticated operating systems on hundreds of megabytes of memory and gigabytes of disk space, all made possible by the wire that connects them to the electrical outlet. Palms run on batteries, and small ones at that. Once you switch to batteries, well, your priorities just change a little. Efficiency is really important. So is simplicity.

So the Palm Engineers simplified. They provided multitasking internally to the protocol stack. The protocol stack runs as a separate task in the operating system. This allows tremendous efficiencies in the networking code. These tasks run separate from the single application task. This simplifies things for the application developer.

Figure 5-2 illustrates the addition of task boundaries to our architectural diagram.

Figure 5-2. Net Library architecture with task boundaries

The application calls functions from the Net Library API. These functions are performed in the application task; the API writes to and reads from the queue, and at the same time, the protocol stack tasks reads from and writes to the queue, and talks to the network.

Avoiding Lengthy Redials

When your application exits, it should be a good Palm citizen and shut down its resources, including the network. But what if you are surfing the Net and want to look up the URL of a best friend's adventure travel web site? Press the address book button. The first thing that happens is the web browser has to stop. And when the web browser stops, it closes down the Net Library, which closes down the Net protocol stack. So you copy the URL of a best friend's adventure travel web site and switch back to your browser. Paste the URL into the browser, and wait while the browser loads the protocol stack.

I used to do this with my email. Every 15 minutes or so, Eudora would wake up, dial my ISP, check for mail, and hang up. Whenever I really needed something, I'd dial up the network, launch a browser, and hit the Web. I had to really need it though, to go through all that work. Nowadays I stay connected to the Internet all day and all night. I find that because the connection is up, I'm much more likely to check things out on the Web. A constant connection encourages me to use my browser application.

You want to encourage your users to use your network application. But, if every time your user switches away to look up the URL of his best friend's adventure travel web site, he loses the network connection and then has to wait while your application reconnects to the Internet before he can find out about exciting trips, well, let's just say your application will not be popular.

Fortunately, the Palm engineers have already solved this. When an application closes down the Net Library, it can select to have the protocol stack live a little before shutting itself down. So now I can switch to address book, find the URL I need, switch back to my browser, paste the URL, and find about exciting trips all without dropping my connection. I just have to be really fast.

You might ask, "Why doesn't my web browser just leave the protocol stack running when I switch to another application?" Well it could, but how would it know that you intend to switch back again? If you never switch back again, then your web browser never gets the chance to unload the protocol stack. The protocol stack would run forever (or until your Palm shutdown), consuming resources and tying up the processor with all those protocol stack tasks.

Calling the Net Library

The API functions act as if they are directly linked into the application. They execute in the applications task, and if there is not a lot of overhead in the function calls, they share the same memory with the application. But they are not directly linked into the application. If they were, their code would repeat in every network-aware application. This overhead applies even if the application never actually accessed the network.

So the Palm engineers simplified. The Net Library API is packaged as a system library. The client application attempts to load this library. If it cannot, the application can continue safely without networking, or it can exit. In short, it can handle this failure gracefully. On the other hand, if it is successful, then the client application has gained entrance to the wonderful world of the Net Library.

Using the Net Library

This is all very well and good, but what does the application do in this wonderful world of the Net Library? Let's take a look.

Before we dive into the functions themselves, let's talk briefly about system libraries. When your compiler comes to normal function calls, it embeds the symbolic function name into your code. Then the linker comes along and resolves this symbolic name to the actual address of the function's implementation. If the linker can't find the implementation of the function, then it generates a linker error.

Palm OS System Libraries work differently. They operate according to something known as the 68K Trap Dispatch mechanism. 68K refers to the series of processors from Motorola that run the Palm, the MAC, and a bunch of other devices. What we care about, though, is the Trap Dispatch Mechanism. When your compiler comes across a call to a function in a system library, instead of inserting a symbolic name, it puts in assembly instructions that generate an exception that is caught by the Palm OS's system trap dispatcher. The system trap dispatcher uses the first parameter to select the correct system library to invoke a function from. As a result, every call to the NetLib API takes a reference number as its first parameter (well, actually there are a few byte-ordering functions that don't, but they turn out to be macros).

Finding the Net Library

The very first thing you must do is to find the Net Library. This is a three-step process. First, ask the operating system for its version. The Net Library appeared for the first time with the PalmPilot Professional, and it simply cannot be used on any device that is not running Palm OS Version 2 or above. The version of the OS is stored in the sysFtrNumROMVersion feature. If you don't know how to retrieve features, don't panic. We describe this in Chapter 6, when we discuss how to load, initialize, and open the Net Library.

Once you determine that you're running the right OS, you need to check if the Net Library is in fact there. Remember, system libraries such as the Net Library are not required for a Palm Device. Even though it has been included with every device since the PalmPilot Professional, there are no guarantees that it will always be available. The easiest way to check if the Net Library is installed is to try to get its version number. The version of the Net Library is stored in the netFtrNumVersion feature. If the Net Library is not installed, you get an error when you try to retrieve this feature.

Now, if you've gotten this far, it's time to initialize the Net Library. System libraries are always loaded, you just have to find them. Call SysLibFind and it returns the reference to the Net Library. This is the reference number that you'll need to pass to all subsequent Net Library API calls, so keep it handy. Once you have the Net Library's reference number, you're ready to go. The process for finding the Net Library is shown in Figure 5-3.

Figure 5-3. Finding the Net Library

Configuring the Net Library

It's a fact of life that network interfaces need to be configured. Thankfully, configuration is an infrequent occurrence. Rather than have each network application bear the responsibility of supporting network interface configuration, the Palm engineers provided an application in which all preferences should be set. The network panel of the Preferences application allows the user to configure one or more services. Services describe how the device operates on the network: connection types, baud rates, flow control, and all sorts of settings that I usually only set when I have a tech support person near at hand.

There might be a time when you find yourself needing to adjust the configuration for your application. For this, the Palm engineers also provided a programmatic API for setting these configurations. In fact, they used this same API in implementing the Network Preferences Panel. The Net Library setup and configuration functions allow you to configure the protocol stack from your application. This is not recommended except in extreme circumstances. We do not cover the setup and configuration protocol in this book.

Opening the Net Library

Once you have finished configuring the Net Library, it's time to use it. Remember, the Palm engineers set up the Net Library so that the protocol stack could be loaded only when needed. Your application must therefore tell the Net Library when you want to use the protocol stack and when you are done. The process for opening the Net Library is illustrated in Figure 5-4.

Figure 5-4. Opening the Net Library

Initializing the protocol stack

To open the Net Library and initialize the protocol stack, call NetLibOpen. Be sure you've completed any setup and configuration required. NetLibOpen reads the current configuration from the Network Preferences database. Once the Net Library is open, most of these configurations cannot be changed. Also, be careful about calling any nonconfiguration functions before calling NetLibOpen; they mostly return the error netErrNotOpen.

The first thing NetLibOpen does is check if the protocol stack has already been loaded. If so, NetLibOpen simply increments the open count and returns. Why would the protocol stack have already been loaded? The most common reason is that an application has closed the Net Library, but specified that the protocol stack should hang out for a while in case it's needed again soon. This could happen in the case described earlier, in which you want to switch from your web browser to your address book to get a URL and switch back again. Or, you might switch from your web browser to your email reader.

If the protocol stack is not already loaded, NetLibOpen creates a new task and loads the protocol stack on this task. The protocol stack then allocates memory for internal use, and brings up the network connection. At this point, the configuration settings are read and applied. From now on, any configuration changes are only reflected if the Net Library is shut down and reopened.

Loading network interfaces

If the selected network interface involves dialing into a server and connecting using the Point-to-Point Protocol (PPP) or the Serial Link Internet Protocol (SLIP), at the time the NetLibOpen connection is made. As you can imagine, this might take a while.

Fortunately, the Palm engineers provided the user interface for communicating the connection progress to the user; a progress dialog is displayed with the current step in the process. Because of this, NetLibOpen must be called from the main user interface task, which is the task that runs your application's event loop. It may not be called from a separate background task.

It is possible that the Net Library will open successfully, but that one or more of the Network Interface components could fail to load. The Net Library allows for more than one network interface to run from the Net Library at the same time. The call to NetLibOpen might succeed even if one of these interfaces fails to load.

Checking for errors

Your application should always check the error code returned by NetLibOpen. It contains the error code of the first interface that fails to load. These might fail due to a bad modem setting, an unavailable service, or some other reason. To determine the exact interface that failed and why it failed, you can then iterate through each of the attached interfaces and check their state. We discuss how to do this in Chapter 6.

NetLibOpen may fail completely and the protocol stack may not be loaded. In this case, the return value indicates an error and the specific error code is returned through a parameter. The Net Library may fail to open if there is insufficient memory, if there are no interfaces configured for it, or if there is some other error in the configuration.

A Quick Word About Berkeley Sockets Once the Net Library's Protocol Stack has been initialized, you can access the network to connect to remote hosts, to send data to the remote host, or to receive data from the remote host. This part of the Net Library API corresponds very closely to the Berkeley Sockets API. In fact, an application could actually use the Berkeley Sockets API, with a little glue here and there. This is good news because there are hundreds of books that talk about how to write networking programs with the Berkeley Sockets API. Some of them are even good. This book, however, is written for programmers who want to use the Palm OS Net Library API and don't necessarily want to go out searching for the good books on Berkeley sockets. If you do, see the bibliography section at the end of this book -- all the books are worth buying. Most Net Library functions take two parameters that the corresponding Berkeley functions do not: a timeout and an error. Most implementations of the Berkeley Sockets API run on multitasking operating systems. If a network function blocks for a long time, the rest of the running applications and services can continue without interruption. If the function blocks for too long, the user can manually kill the process. In the Palm environment, there is only a single user interface task. As long as a network function is blocking, no other user interface code can run. And there is no way to kill any processes, short of resetting the device. If a network function takes too long, the device appears to freeze. The timeout parameter limits the amount of time a network function can take. If the function does not complete before the timeout expires, the function returns the error netErrTimeout. In the Berkeley Sockets API, errors are accessed through a global error variable, errno. In the Palm environment, this is dangerous because there are times when the code does not have access to global variables. As a result, Net Library functions take an additional parameter that allows them to return the error code. These functions are discussed in Chapters through .

Opening the Socket

Once the protocol stack is loaded, you can connect to a remote host. To do this, you need a socket. NetLibSocketOpen creates a socket and returns a reference to it. This reference, also known as a socket descriptor or a NetSocketRef, should be treated as an unknown; you should not depend on it being or not being a specific value. The only value you should check it against is -1, which, when returned from NetLibSocketOpen, indicates an error.

When you open a socket, specify whether it is to be a stream or datagram socket. Stream sockets use the Transmission Control Protocol (TCP) to provide a "virtual circuit" between the two connected hosts. Packets are guaranteed to arrive in order, exactly once. This protocol is geared towards a world where the network is relatively fast but not necessarily reliable. TCP is a "chatty" protocol, in which the chattiness is designed to ensure reliability.

Datagram sockets use the User Datagram Protocol (UDP). This protocol is connectionless. Packets may arrive in order or they may not. They may arrive once, more than once, or not at all. This protocol is good for situations where the network reliability is not a question or where the protocol that sits on top of UDP ensures the correct delivery of packets. UDP is common in wireless applications because of its low overhead.

When you call NetLibSocketOpen, the Net Library puts an open socket request into the mailbox queue and waits. The protocol stack reads this request and allocates the internal memory structure to hold the socket information. It then puts a reference to this structure into the mailbox queue. The Net Library, which is hopefully still waiting, reads the response and returns the socket reference to the caller.

If a timeout was specified and the protocol stack does not handle the request before this timeout expires, NetLibSocketOpen returns an error. There are not many other reasons for this function to fail. The Net Library could be closed, or one of the parameters could be invalid. These failures would all be detected without involving the protocol stack. Once the request has been put in the mailbox, the main cause of failure would be that no more sockets are available. Remember, the protocol stack only supports four open sockets at a time.

Configuring the Socket

When the socket is opened, the socket is not connected to anything; we've just allocated the resources for the connection. Now is the time for the application to configure the socket. An application calls NetLibSocketOptionSet to configure an option for the socket. The Net Library supports the following options:

TCP No Delay

Enables or disables the Nagel Algorithm, which is designed to reduce network congestion.

Socket Keep Alive

Enables or disables the keep-alive mechanism, which detects failure on an idle connection and reports the error to the application.

Socket Linger

Controls how the socket behaves when closing with data waiting to be sent or acknowledged.

Socket Blocking

Controls whether a socket function call should wait (or block) until it has completed.

All these options must be set while the socket is not connected to a remote host or bound to a particular address. Once the socket is connected or bound, trying to set these options has no effect.

The value of these options can be read using the API function NetLibSocketOptionGet. There are three more read-only options:

TCP Maximum Segment

Returns the maximum size a segment can be.

Socket Error Status

Returns the most recent error that occurred on the socket, and clears it.

Socket Type

The type of the socket, Datagram or Stream.

As a rule, we will use the default values for socket options. In those cases where we need to change from the defaults, such as in Chapter 12, we will discuss the relevant options and their possible values.

NetLibSocketOptionSet places a request to set an option into the mailbox queue and waits. The protocol stack reads the request and attempts to set the option in the appropriate socket structure. When it is finished it places a response in the mailbox queue. NetLibSocketOptionSet reads this response and passes the appropriate return code to the application.

If a timeout has been specified and the protocol stack does not complete the operation before this timeout expires, this NetLibSocketOptionSet returns an error. This is unlikely since it is a fairly simple operation with no network access involved. The function may also fail if either the Net Library or the socket is closed, if there is a bad parameter, or if the requested option is not supported or does not apply to the socket because of its type. For example, the Socket Keep Alive option has no meaning for a datagram-based socket.

Naming the Socket

There are three attributes to a socket name: protocol, IP address, and port number. The protocols available for Palm OS sockets are TCP for stream-based sockets and UDP for datagram-based sockets. Even if you've never written a line of network code before, you probably can recognize the IP address. It is written as a series of four numbers separated by periods (or dots); for example, 129.92.157.102. It is represented to the Net Library as four bytes packed into a long integer; none of the individual values can be greater than 255. The port number indicates which service is supported by the socket. Well-known services such as HTTP, FTP, or SMTP have well-known ports. Custom services define custom ports; both sides of a connection need to know in advance what those ports are.

If you are going to act as a client in the connection and initiate the connection request, you don't need to bind an address to your socket; the protocol stack automatically performs this binding when you attempt to connect to a remote host. UDP sockets are not required to connect to remote hosts before sending data. In this case, the implicit binding occurs with each send operation.

NetLibSocketBind binds addresses to sockets. Since your Palm only has one network adapter connected to it, the IP address will be common to all sockets. The distinguishing factor will be the port. In the case of implicit binding, the protocol stack assigns the next available port number to the socket.

However, if you are going to act as a server in the connection and listen for incoming, it is essential that you bind an address to your socket. Connection requests and incoming datagram packets apply to specific services on the respective port. Binding your socket to that port informs the protocol stack that you will handle requests for these services. If there is already a socket bound to the requested address and port, NetLibSocketBind returns an error.

When you make a call to NetLibSocketBind, the Net Library places a request to associate the specified TCP/IP address with the socket into the mailbox queue, and waits. The protocol stack reads the request and assigns the name in the associated socket structure. When this is finished, the protocol stack places a response into the mailbox queue. NetLibSocketBind reads this response and returns the appropriate value to the application.

If NetLibSocketBind was invoked with a timeout, and the protocol stack fails to complete the bind operation before this timeout expires, an error is returned. Just as with NetLibSocketOpen and NetLibSocketOptionSet, this is unlikely to happen since this is a fairly straightforward operation that requires no network access. NetLibSocketBind may also fail if the Net Library or the requested socket is not open, if one or more of the parameters is invalid, or if the socket is already connected to a remote host. Remember, a connection is uniquely identified on the entire network by the combination of the address and ports of the two endpoints and the protocol. Once the connection is made, none of these attributes can be changed.

Making a Connection

There are three steps in establishing a connection. First, the server application prepares to receive connection requests. Then the client initiates a connection by sending a request to the server. When the server receives the request and grants it, the connection is established.

Before we try to connect, we need to make a design decision: will our application act as a client or a server? A server application waits for clients to connect to it; the client initiates the connection to the server. Most Palm applications act as clients in this regard. Servers must constantly check for incoming connection requests, which does not fit well with the Palm paradigm of rapidly changing between applications, automatic device shutdown, and conservation of resources. Remember, just keeping the serial port open is a significant drain on the batteries. Also, server applications must be able to handle multiple clients simultaneously. They depend on multitasking, interprocess communications, and other sophisticated techniques to service these clients, techniques that are not available to the Palm application developer.

Initiating a connection

Given the constraints of the Palm OS, you're probably going to write a client application. With the socket open and configured, the application can now make a connection to a remote host. To do so, the application calls NetLibSocketConnect to associate the address of a remote host with the socket. In the case of a stream-based connection, NetLibSocketConnect also initiates a three-way handshake with the remote host. In this, the client sends a request for a connection, the server sends an acceptance of this request, and the client sends back an acknowledgment of the server's acceptance.

For Datagram sockets, connecting is optional. In general, you definitely want to call NetLibSocketConnect if you are going to be exchanging data with the same remote host. If you are going to be dealing with a series of remote hosts, each for a significant set of exchanges, you could call NetLibSocketConnect each time you began communicating with a different host. If you are exchanging data with a lot of different remote hosts, you would provide the address information with every send and receive call instead.

When you call NetLibSocketConnect on a socket, this function places a connection request into the mailbox queue and waits. The protocol stack reads the request. It retrieves the remote hosts address from the connection request, as well as the local hosts connection information and address from the relevant socket structure, and creates a TCP control packet requesting a connection. This packet is sent to the remote host. If the remote host accepts the connection, it sends back a control packet acknowledging this acceptance. Finally, the local host responds with another control packet acknowledging its receipt of the remote hosts acknowledgment. This is known as three-way handshaking.

When all this is complete or if an error occurs, the protocol stack places a response into the mailbox queue. NetLibSocketConnect, which has been waiting all this time, retrieves this response and returns the appropriate value to the application. If a timeout was specified and this fails to complete before that timeout expires, NetLibSocketConnect returns an error. The connection request might also fail because the Net Library or the socket is not open, the socket is busy or already connected, the remote host refuses the connection, or there are no TCP connections available.

Accepting a connection

Even though you are writing a client application, you still might need to accept incoming connections on occasion. An FTP client, for example, connects to the server for sending commands, but the server connects back to the client for sending data. Managing incoming connections is more complicated than initiating them, but it's not difficult.

Once you have bound an address to your socket, you can begin "listening" for incoming connection requests. NetLibSocketListen puts a stream-based socket into a passive listening mode. This prepares the socket to accept a connection attempt from a TCP client. When an incoming connection arrives, call NetLibAccept to complete the connection. NetLibAccept returns a reference to a new socket. Remember, the Net Library supports only four open sockets at a time. Also, it is rare that you would be writing a server application to run on a Palm OS device. So you probably want to close down the listening socket once you accepted an incoming connection.

WARNING:   It bears repeating that connecting the socket is not required for datagram sockets, and they cannot accept connections at all; any attempt to call NetLibListen or NetLibAccept on a datagram socket will fail. The address associated with the socket is used when reading packets. Stream sockets, on the other hand, must be connected before writing or reading data on the network.

When you call NetLibListen, it passes a request through the mailbox queue to the protocol stack to put the associated, stream-based socket into passive listening mode. The protocol stack also allocates structures to handle incoming connection requests for this socket. When the protocol stack completes this, it passes its response back through the mailbox queue to the waiting NetLibListen.

If a timeout was specified and the protocol stack cannot fulfill the request before this timeout expires, then an error is returned. Because this function just changes the state of the socket, a timeout error is unlikely. This function also returns an error if the Net Library or the specified socket is not open, if another socket is already listening on the same port, if the specified socket is already connected, if the specified socket is not a stream-based socket, if there are insufficient resources, or if one or more parameters is invalid.

Assuming NetLibListen returns success, you then call NetLibAccept. This function passes a request to accept any pending connection requests through the mailbox queue to the protocol stack. The protocol stack first checks to see if there are any pending connection requests. If there are not, the protocol stack waits until there are. When the protocol stack finds a pending connection request, it opens a new socket and initializes it with information from the listening socket. It then sends an accept acknowledgment back to the remote host that initiated the connection, and waits for the expected acknowledgment.

When the protocol stack receives the acknowledgment and the three-way handshaking is complete, it passes a response back through the mailbox queue to the application's task. This response includes both the outcome of the accept and the address of the remote host, if the operation is successful. This information is then returned to the calling application. This process is illustrated in Figure 5-5.

Figure 5-5. Accepting a connection

Sending Data

The most common way of sending data is to call NetLibSend. For datagram sockets that were not connected to a specific address, you need to pass the destination address as a parameter. For all other types of sockets, this parameter is ignored. When you send data, the protocol stack stores the data in buffers to be sent out later, then returns. If there is insufficient room in these buffers for the amount of data you send, the protocol stack stores as much of the data as it can fit.

For stream-based sockets, NetLibSend returns the amount of data accepted by the protocol stack to be sent. It is the application's responsibility to keep track of how much data was sent. For datagram sockets, the data must all be sent. In either case, if there is no buffer available for the data, the function waits until the space is available.

The Net Library also provides a send function, NetLibSendPB, that allows you to send data from a "scatter gather array." If you don't know what a scatter gather array is, you probably don't want to use this function. If you do know, you probably still won't want to use this function.

When you call NetLibSend or one of the other Net Library send functions, the protocol stack attempts to write the data to the specified socket. This data is first stored in the protocol stack's internal buffers. The data may be too big to fit into the internal buffers. If the socket in question is stream-based, then as much data as will fit is put into the buffers; the size that was buffered is returned. If, on the other hand, we are dealing with a datagram-based socket, then an error is returned and no data is buffered. If there is no space for the data to be buffered, the protocol stack will wait until the space becomes available. When this is complete, the protocol stack queues a response for the application.

After the protocol stack places a response into the mailbox queue, it begins to process the buffered data. The data is first encoded as TCP packets, then as IP packets, then finally as PPP, SLIP, or other data link protocol packets. Once the data is completely packetized, the protocol stack writes these packets to the network interface hardware.

For a stream-based socket, a single send operation can generate multiple packets. The remote host sends an acknowledgment for each packet sent. If the acknowledgment is not received within a certain amount of time, the protocol stack resends the data. Until the protocol stack receives this acknowledgment, it will not send the next packet.

Datagram-based sockets generate exactly one packet for each send operation. Once the packet is sent out, the protocol stack releases the buffers used to store the data; the UDP protocol does not support acknowledgments of packets. If the data from a send operation is too big for a UDP packet, no data is sent and an error is returned.

If a timeout is specified and the protocol stack is unable to buffer the data before the timeout expires, then an error is returned. Given the complexity of this operation, this is a distinct possibility. When the protocol stack waits for buffer space to become available, it is waiting for the transmission of that data on the network to complete. Other reasons for a send operation to fail include the Net Library or the socket being closed, an error in the IP-routing information, the remote host closing terminating the connection, or an invalid parameter.

TIP:   It is essential that you check the return value of a send operation. This value indicates the number of bytes actually sent, or, more accurately, the number of bytes buffered to be sent by this operation. For stream-based sockets, it is possible that the protocol stack was unable to buffer all this data. In the case of large amounts of data being sent, it is even likely. It is the application's responsibility to continuously send the remaining data until it has all been sent or an error is returned.

Receiving Data

For the most part, receiving data is the inverse of sending it. Generally, you call NetLibReceive. For unconnected datagram sockets, you pass a buffer for a network address; NetLibReceive populates this with the sender's address. For all other types of sockets, this parameter is ignored. NetLibReceive returns the actual amount of data read into the data buffer, which was passed as a parameter. For datagram-based sockets, the entire datagram is read into the buffer. If the datagram is larger than the buffer, any extra data is discarded. It is important, therefore, to be very careful when using datagram-based sockets.

In addition to a function that reads into a scatter gather array, NetLibReceivePB, there is also a function that allows you to read directly from the network into a database record, NetLibDmReceive. Aside from a few parameter differences that relate to how data is written to database records, this function behaves the same as NetLibReceive.

Before you even call NetLibReceive or its equivalents, the protocol stack is already working for you. While your application is off displaying user interface, waiting for input, or doing whatever processing your application does, the protocol stack is busy accepting packets from the network, sending back acknowledgments for them and routing the packets to the appropriate socket.

When a packet comes in, the protocol stack reads its address information to see which connection it is for. It then strips off the TCP/IP header information and stuffs the remaining data into a buffer. Next it goes back to processing requests from applications including yours, and processing more incoming packets.

When you call NetLibReceive, a request to retrieve data is passed through the mailbox queue to the protocol stack. The protocol stack checks to see if any data has been received for your socket. If it has, it transfers the data from its internal buffers to the buffers you pass as parameters. The receive functions work rather differently depending on whether the socket is stream-based or datagram-based.

For stream-based sockets, the protocol stack transfers any available data to the buffers, up to the size of the buffers. If there is more data than the application requested, this data is left in the buffers for subsequent calls to one of the receive functions. If there is no data available, the protocol stack waits until at least one byte of data arrives.

For datagram-based sockets, the function always transfers the contents of exactly one datagram. If the amount of data requested by the application is smaller than the size of the next available datagram, the protocol stack discards any excess data. If there is no data available, the protocol stack waits until the next datagram arrives.

The receive functions return the number of bytes transferred to the buffers; the application should repeatedly call these functions until that return value is 0. Also, you should be careful to call a receive function fairly frequently in order to keep the buffers cleared. If packets arrive and there are no buffers to hold them, they are not accepted. Since you share the protocol stack with other applications, you might cause them to lose data as well.

If a timeout is specified for a receive function, and the protocol stack is unable to fulfill the receive request before this timeout expires, the function returns an error. As you can imagine, network latency or processing time by the remote host makes timing out a common event. In Chapter 12, we discuss how to handle this efficiently using nonblocking sockets.

If there is data available, however, the receive functions generally are successful. The other possible errors involve the Net Library or the socket being closed and bad parameter values being passed. These are all errors that can be avoided by careful programming.

Disconnecting

Once you are done exchanging data with the remote host, it's time to close down the socket. In the case of a datagram-based socket, this is a fairly straightforward operation; the application simply calls NetLibClose. The socket resources are returned to the protocol stack.

Stream-based sockets are a bit more complicated. To provide the level of service that TCP guarantees, the protocol stack sends an acknowledgment for every packet it receives. If the sender does not receive the packet in a certain amount of time (known as the timeout), it resends the packet. When closing a socket, it is possible that packets have arrived but not been acknowledged. If you precipitously close the socket, you might leave the remote socket in the uncomfortable state of having sent packets for which it has not received acknowledgments. On the other hand, you may have a good reason for closing, and you might not want to wait around while the remote host keeps sending data.

Fortunately, the Network designers solved this with the "graceful close." First, invoke NetLibShutdown. This informs both sides of the connection that the end of their connection is near. Then invoke NetLibReceive until you have received all pending data. This ensures that there are no packets waiting to be acknowledged. Finally, call NetLibClose to close the socket.

When your application calls NetLibShutdown, the protocol stack does a "partial close" of the TCP connection. It does not release any system resources used by the socket. This function takes a parameter indicating whether the shutdown should occur for sends, receives, or both. The operation indicated by this parameter is then disallowed; if the shutdown only sends, your application can continue to receive packets. If the shutdown only receives, your application can continue to send packets.

When you shut down sending packets, the protocol stack sends a control packet to the remote host indicating that shutdown is imminent. After you shut down receiving, the protocol stack can legally discard incoming packets without informing the remote host; your application can no longer call receive functions to empty the buffers.

In general, it is recommended that you only shut down sending packets. The remote host is informed of your intention to close the connection, but any outstanding packets will be received and handled. The protocol stack will continue to process these incoming packets and buffer the contained data. You should repeatedly invoke receive functions until no more data is available.

Closing the Net Library

When you are finished with communicating over the network, it's time to close down the Net Library, unload the protocol stack, and return the resources to the operating system. Usually, you are finished communicating over the network under one of two circumstances. The first is that the user has switched to another application, so your program is exiting. Or, your application may only use the network infrequently; it will continue to run, it just no longer needs the network. For example, your email client dials up the network to check if you have mail, then disconnects when it is finished.

If your application is exiting, call NetLibClose to close down the Net Library. You won't know what application the user is going to launch next. If your program is an email client and the user is switching to a web browser, the wise programmer will tell the Net Library to close, but wait a little bit before unloading the protocol stack. This way, if the user does launch another application that requires a network connection, he won't have to wait while the device reconnects to the network.

Even if your application is not exiting, you still call NetLibClose to close down the Net Library. You might want to have the protocol stack linger for a while, too. If your application is likely to need network access again in the near future, or if you worry that the user might switch to another network application in the very near future, then all the same reasons mentioned previously still apply. On the other hand, you might want to connect to a different network, or you might be concerned about the high cost of the dial-up connection. One of the ISP's I use is a toll call away, so I try to keep my connection time down. NetLibClose also supports immediate closing.

When your application calls NetLibClose, the Net Library puts a request in the mailbox queue to unload the protocol stack and waits. Part of this request is whether or not to close immediately. The protocol stack reads this request from the mailbox queue. It shuts down any outstanding connections for the session and decrements the open count. If the open count is still greater than zero, the protocol stack places a response in the mailbox queue. If the open count reaches zero, the protocol stack begins shutting itself down.

In the case where you requested an immediate shutdown, the protocol stack shuts down all logical connections to the network, disconnects the physical network interface, and frees the internal buffers and memory structures. It then posts a response into the mailbox queue. When all these operations are complete, the protocol stack exits and its task shuts down.

If your application requested a delayed shutdown, then the protocol stack starts a timer running. The exact value of this timer is configurable via the network preferences panel. When this timer expires, the protocol stack unloads itself from memory and ends its task, as just described.

All this time, NetLibClose has been waiting, occasionally checking the mailbox queue for a response. When one finally arrives, NetLibClose reads it from the queue and translates it into a return value, and potentially, into an error code which it then returns to the application.

Summary

Using the Net Library is fairly straightforward. You must, of course, find the library and load the protocol stack. Once the protocol stack is loaded, you can connect to a remote host, then send data to and receive data from that host. When you are done exchanging data, close down the connection. When you are finished with all network access, close the library and unload the protocol stack. The devil, as they say, is in the details. We face this devil over the next several chapters, as we build a sample application based on the File Transfer Protocol (FTP).

Back to: Palm OS Network Programming