Java is the first programming language designed from the ground up with networking in mind. As the global Internet continues to grow, Java is uniquely suited to build the next generation of network applications. Java provides solutions to a number of problems—platform independence, security, and international character sets being the most important—that are crucial to Internet applications, yet difficult to address in other languages. Together, these and other Java features allow web surfers to quickly download and execute untrusted programs from a web site without worrying that the program may spread a virus, steal their data, or crash their systems. Indeed, the intrinsic safety of a Java applet is far greater than that of shrink-wrapped software.
One of the biggest secrets about Java is that it makes writing network programs easy. In fact, it is far easier to write network programs in Java than in almost any other language. This book shows you dozens of complete programs that take advantage of the Internet. Some are simple textbook examples, while others are completely functional applications. One thing you’ll note in the fully functional applications is just how little code is devoted to networking. Even in network-intensive programs like web servers and clients, almost all the code handles data manipulation or the user interface. The part of the program that deals with the network is almost always the shortest and simplest.
In short, it is easy for Java applications to send and receive data across the Internet. It is also possible for applets to communicate across the Internet, though they are limited by security restrictions. In this chapter, you’ll learn about a few of the network-centric applets and applications that can be written in Java. In later chapters, you’ll develop the tools you need to write these programs.
Networking adds a lot of power to simple programs. With networks, a single program can retrieve information stored in millions of computers located anywhere in the world. A single program can communicate with tens of millions of people. A single program can harness the power of many computers to work on one problem.
But that sounds like a Microsoft advertisement, not the start of a technical book. Let’s talk more precisely about what network programs do. Network applications generally take one of several forms. The distinction you hear about most is between clients and servers. In the simplest case, clients retrieve data from a server and display it. More complex clients filter and reorganize data, repeatedly retrieve changing data, send data to other people and computers, and interact with peers in real time for chat, multiplayer games, or collaboration. Servers respond to requests for data. Simple servers merely look up some file and return it to the client, but more complex servers often do a lot of processing before answering an involved question. Beyond clients and servers, the next generation of Internet applications almost certainly includes mobile agents, which move from server to server, searching the Web for information and dragging their findings home. And that’s only the beginning. Let’s look a little more closely at the possibilities that open up when you add networking to your programs.
At the most basic level, a network client retrieves data from a server and shows it to a user. Of course, many programs did just this long before Java came along; after all, that’s exactly what a web browser does. However, web browsers are limited. They can talk to only certain kinds of servers (generally web, FTP, gopher, and perhaps mail and news servers). They can understand and display certain kinds of data (generally text, HTML, and a few standard image formats). If you want to go further, you’re in trouble: a web browser cannot send SQL commands to a database to ask for all books in print by Elliotte Rusty Harold published by O’Reilly & Associates, Inc. A web browser cannot check the time to within a hundredth of a second with the U.S. Naval Observatory’s[1] super-accurate hydrogen maser clocks using the network time protocol. A web browser can’t speak the custom protocol needed to remotely control the High Resolution Airborne Wideband Camera (HAWC) on the Stratospheric Observatory for Infrared Astronomy (SOFIA).[2]
A Java program, however, can do all this and more. A Java program can send SQL queries to a database. Figure 1.1 shows part of a program that communicates with a remote database server to submit queries against the Books in Print database. While something similar could be done with HTML forms and CGI, a Java client is more flexible because it’s not limited to single pages. When something changes, only the actual data needs to be sent across the network. A web server would have to send all the data as well as all the layout information. Furthermore, user requests that change only the appearance of data rather than which data is displayed (for example, hiding or showing a column of results) don’t even require a connection back to the database server because presentation logic is incorporated in the client. HTML-based database interfaces tend to place fairly heavy loads on both web and database servers. Java clients move all the user interface processing to the client side, and let the database focus on the data.
A Java program can connect to a network time-server to synchronize itself with an atomic clock. Figure 1.2 shows an applet doing exactly this. A Java program can speak any custom protocols it needs to speak, including the one to control the HAWC. Figure 1.3 shows an early prototype of the HAWC controller. Even better: a Java program embedded into an HTML page (an applet) can give a Java-enabled web browser capabilities the browser didn’t have to begin with.
Furthermore, a web browser is limited to displaying a single complete HTML page. A Java program can display more or less content as appropriate. It can extract and display the exact piece of information the user wants. For example, an indexing program might extract only the actual text of a page while filtering out the HTML tags and navigation links. Or a summary program can combine data from multiple sites and pages. For instance, a Java servlet can ask the user for the title of a book using an HTML form, then connect to 10 different online stores to check the prices for that book, then finally send the client an HTML page showing which stores have it in stock sorted by price. Figure 1.4 shows the Amazon.com (née Junglee) WebMarket site showing the results of exactly such a search for the lowest price for an Anne Rice novel. In both examples, what’s shown to the user looks nothing like the original web page or pages would look in a browser. Java programs can act as filters that convert what the server sends into what the user wants to see.
Figure 1-4. The WebMarket site at http://www.webmarket.com/ is written in Java using the servlet API
Finally, a Java program can use the full power of a modern graphical user interface to show this data to the user and get a response to it. Although web browsers can create very fancy displays, they are still limited to HTML forms for user input and interaction.
Java programs are flexible because Java is a fully general programming language, unlike HTML. Java programs see network connections as streams of data, which can be interpreted and responded to in any way that’s necessary. Web browsers see only certain kinds of data streams and can interpret them only in certain ways. If a browser sees a data stream that it’s not familiar with (for example, a response to an SQL query), its behavior is unpredictable. Web sites can use CGI programs to provide some of these capabilities, but they’re still limited to HTML for the user interface.
Writing Java programs that talk to Internet servers is easy. Java’s core library includes classes for communicating with Internet hosts using the TCP and UDP protocols of the TCP/IP family. You just tell Java what IP address and port you want, and Java handles the low-level details. Java does not support NetWare IPX, Windows NetBEUI, AppleTalk, or other non-IP-based network protocols; but this is rapidly becoming a nonissue as TCP/IP becomes the lingua franca of networked applications. Slightly more of an issue is that Java does not provide direct access to the IP layer below TCP and UDP, so it can’t be used to write programs such as ping or traceroute. However, these are fairly uncommon needs. Java certainly fills well over 90% of most network programmers’ needs.
Once a program has connected to a server, the local program must understand the protocol that the remote server speaks and properly interpret the data the server sends back. In almost all cases, packaging data to send to a server and unpacking the data received is harder than simply making the connection. Java includes classes that help your programs communicate with certain types of servers, most notably web servers. It also includes classes to process some kinds of data, such as text, GIF images, and JPEG images. However, not all servers are web servers, and not all data is text, GIF, or JPEG. Therefore, Java lets you write protocol handlers to communicate with different kinds of servers and content handers that understand and display different kinds of data. A Java-enabled web browser can automatically download and install the software needed by a web site it visits. Java applets can perform tasks similar to those performed by Netscape plug-ins. However, applets are more secure and much more convenient than plug-ins. They don’t require user intervention to download or install the software, and they don’t waste memory or disk space when they’re not in use.
Web browsers retrieve data on demand; the user asks for a page at a URL and the browser gets it. This model is fine as long as the user needs the information only once, and the information doesn’t change often. However, continuous access to information that’s changing constantly is a problem. There have been a few attempts to solve this problem with extensions to HTML and HTTP. For example, server push and client pull are fairly awkward ways of keeping a client up to date. There are even services that send email to alert you that a page you’re interested in has changed.[3]
A Java client, however, can repeatedly connect to a server to keep an updated picture of the data. If the data changes very frequently—for example, a stock price—a Java application can keep a connection to the server open at all times, and display a running graph of the stock price on the desktop. Figure 1.5 shows only one of many such applets. A Java program can even respond in real time to changes in the data: a stock ticker applet might ring a bell if IBM’s stock price goes over $100 so you know to call your broker and sell. A more complex program could even perform the sale without human intervention. It is easy to imagine considerably more complicated combinations of data that a client can monitor, data you’d be unlikely to find on any single web site. For example, you could get the stock price of a company from one server, the poll standings of candidates they’ve contributed to from another, and correlate that data to decide whether to buy or sell the company’s stock. A stock broker would certainly not implement this scheme for the average small investor.
As long as the data is available via the Internet, a Java program can track it. Data available on the Internet ranges from weather conditions in Tuva to the temperature of soft drink machines in Pittsburgh to the stock price of Sun Microsystems to the sales status of this very book at amazon.com. Any or all of this information can be integrated into your programs in real time.
Web browsers are optimized for retrieving data. They send only limited amounts of data back to the server, mostly via forms. Java programs have no such limitations. Once a connection between two machines is established, Java programs can send data across that connection just as easily as they can receive from it. This opens up many possibilities.
Applets often need to save data between runs; for example, to store the level a player has reached in a game. Untrusted applets aren’t allowed to write files on local disks, but they can store data on a cooperating server. The applet just opens a network connection to the host it came from and sends the data to it. The host may accept the data through a CGI interface, ftp, SOAP, a custom server or servlet, or some other means.
Since Java applets are secure, individual users can safely offer the use of their spare CPU cycles to scientific projects that require massively parallel machines. When part of the calculation is complete, the program makes a network connection to the originating host and adds its results to the collected data.
So far, efforts such as SETI@home’s[4] search for intelligent life in the universe and distributed.net’s[5] RC5/DES cracker have relied on native code programs written in C that have to be downloaded and installed separately, mostly because slow Java virtual machines have been at a significant competitive disadvantage on these CPU-intensive problems. However, Java applets performing the same work do make it more convenient for individuals to participate. With a Java applet version, all a user would have to do is point the browser at the page containing the applet that solves the problem.
The Charlotte project from New York University and Arizona State is currently developing a general architecture for using Java applets for supporting parallel calculations using Java applets running on many different clients all connected over the Internet. Figure 1.6 shows a Charlotte demo applet that calculates the Mandelbrot set relatively quickly by harnessing many different CPUs.
Java’s AWT has all the user interface components available in HTML forms, including text fields, checkboxes, radio buttons, pop-up lists, buttons, and a few more besides. Thus with Java you can create forms with all the power of a regular HTML form. These forms can use network connections to send the data back to the server exactly as a web browser does.
However, because Java applets are real programs instead of mere displayed data, these forms can be truly interactive and respond immediately to user input. For instance, an order form can keep a running total including sales tax and shipping charges. Every time the user checks off another item to buy, the applet can update the total price. A regular HTML form would need to send the data back to the server, which would calculate the total price and send an updated version of the form—a process that’s both slower and more work for the server.
Furthermore, a Java applet can validate input. For example, an applet can warn users that they can’t order 1.5 cases of jelly beans, that only whole cases are sent. When the user has filled out the form, the applet sends the data to the server over a new network connection. This can talk to the same CGI program that would process input from an HTML form, or it can talk to a more efficient custom server. Either way, it uses the Internet to communicate.
The previous examples all follow a client/server model. However, Java applications can also talk to each other across the Internet, opening up many new possibilities for group applications. Java applets can also talk to each other, though for security reasons they have to do it via an intermediary proxy program running on the server they were downloaded from. (Again, Java makes writing this proxy program relatively easy.)
Combine the ability to easily include networking in your programs with Java’s powerful graphics and you have the recipe for truly awesome multiplayer games. Some that have already been written are Backgammon, Battleship, Othello, Go, Mahjongg, Pong, Charades, Bridge, and even strip poker. Figure 1.7 shows a four-player game of Hearts in progress on Yahoo! Plays are made using the applet interface. Network sockets send the plays back to the central Yahoo!Yahoo! server, which copies them out to all the participants.
Java lets you set up private or public chat rooms. Text that is typed in one applet can be echoed to other applets around the world. Figure 1.8 shows a basic chat applet like this on Yahoo! More interestingly, if you add a canvas with basic drawing ability to the applet, you can share a whiteboard between multiple locations. And as soon as browsers support Version 2.0 of the Java Media Framework API, writing a network phone application or adding one to an existing applet will become trivial. Other applications of this type include custom clients for Multi-User Dungeons (MUDs) and Object-Oriented (MOOs), which could easily use Java’s graphic capabilities to incorporate the pictures people have been imagining for years.
Java programs aren’t limited to sending text and data across the network. Graphics can be sent too. A number of programmers have developed whiteboard software that allows users in diverse locations to draw on their computers. For the most part, the user interfaces of these programs look like any simple drawing program with a canvas area and a variety of pencil, text, eraser, paintbrush, and other tools. However, when networking is added to a simple drawing program, many different people can collaborate on the same drawing at the same time. The final drawing may not be as polished or as artistic as the Warhol/Basquiat collaborations, but it doesn’t require all the participants to be in the same New York loft either. Figure 1.9 shows several windows from a session of the IBM alphaWorks’ WebCollab program.[6] WebCollab allows users in diverse locations to display and annotate slides during teleconferences. One participant runs the central WebCollab server that all the peers connect to while conferees participate using a Java applet loaded into their web browsers.
Java applications can listen for network connections and respond to them. This makes it possible to implement servers in Java. Both Sun and the W3C have written web servers in Java designed to be as fully functional and fast as servers written in C. Many other kinds of servers have been written in Java as well, including IRC servers, NFS servers, file servers, print servers, email servers, directory servers, domain name servers, FTP servers, TFTP servers, and more. In fact, pretty much any standard TCP or UDP server you can think of has probably been ported to Java.
More interestingly, you can write custom servers that fill your specific needs. For example, you might write a server that stored state for your game applet and had exactly the functionality needed to let the players save and restore their games, and no more. Or, since applets can normally communicate only with the host from which they were downloaded, a custom server could mediate between two or more applets that need to communicate for a networked game. Such a server could be very simple, perhaps just echoing what one applet sent to all other connected applets. The Charlotte project mentioned earlier uses a custom server written in Java to collect and distribute the computation performed by individual clients. WebCollab uses a custom server written in Java to collect annotations, notes, and slides from participants in the teleconference and distribute them to all other participants. It also stores the notes on the central server. It uses a combination of the normal HTTP and FTP protocols as well as its custom WebCollab protocol.
As well as classical servers that listen for and accept socket connections, Java provides several higher-level abstractions for client/server communication. Remote Method Invocation (RMI) allows objects located on a server to have their methods called by clients. Servers that support the Java Servlet API can load extensions written in Java called servlets that give them new capabilities. The easiest way to build your multiplayer game server might be to write a servlet, rather than writing an entire server.
Java programs can wander through the Web, looking for crucial information. Search programs that run on a single client system are called spiders. A spider downloads a page at a particular URL, extracts the URLs from the links on that page, downloads the pages referred to by the URLs, and then repeats the process for each page it’s downloaded. Generally, a spider does something with each page it sees, ranging from indexing it in a database to performing linguistic analysis to hunting for specific information. This is more or less how services like AltaVista build their indices. Building your own spider to search the Internet is a bad idea, because AltaVista and similar services have already done the work, and a few million private spiders would soon bring the Net to its knees. However, this doesn’t mean that you shouldn’t write spiders to index your own local intranet. In a company that uses the Web to store and access internal information, building a local index service might be very useful. You can use Java to build a program that indexes all your local servers and interacts with another server program (or acts as its own server) to let users query the index.
Agents have purposes similar to those of spiders (researching a stock, soliciting quotations for a purchase, bidding on similar items at multiple auctions, finding the lowest price for a CD, finding all links to a site, etc.). But whereas spiders run on a single host system to which they download pages from remote sites, agents actually move themselves from host to host and execute their code on each system they move to. When they find what they’re looking for, they return to the originating system with the information, possibly even a completed contract for goods or services. People have been talking about mobile agents for years, but until now, practical agent technology has been rather boring. It hasn’t come close to achieving the possibilities envisioned in various science fiction novels, like John Brunner’s Shockwave Rider and William Gibson’s Neuromancer. The primary reason for this is that agents have been restricted to running on a single system—and that’s neither useful nor exciting. In fact until 2000, there’s been only one widely successful (to use the term very loosely) true agent that ran on multiple systems, the Morris Internet worm of 1988.
The Internet worm demonstrates one reason developers haven’t been willing to let agents go beyond a single host. It was destructive; after breaking into a system through one of several known bugs, it proceeded to overload the system, rendering it useless. Letting agents run on your system introduces the possibility that hostile or buggy agents may damage that system—and that’s a risk most network managers haven’t been willing to take. Java mitigates the security problem by providing a controlled environment for the execution of agents. This environment has a security manager that can ensure that, unlike the Morris worm, the agents won’t do anything nasty. This allows systems to open their doors to these agents.
The second problem with agents has been portability. Agents aren’t very interesting if they can run on only one kind of computer. That’s like having a credit card for Nieman Marcus; it’s somewhat useful and has a certain snob appeal, but it won’t help as much as a Visa card if you want to buy something at Sears. Java provides a platform-independent environment in which agents can run; the agent doesn’t care if it’s visiting a Linux server, a Sun workstation, a Macintosh desktop, or a Windows PC.
An indexing program could be implemented in Java as a mobile agent: instead of downloading pages from servers to the client and building the index there, the agent could travel to each server and build the index locally, sending much less data across the network. Another kind of agent could move through a local network to inventory hardware, check software versions, update software, perform backups, and take care of other necessary tasks. Commercially oriented agents might let you check different record stores to find the best price for a CD, see whether opera tickets are available on a given evening, or more. A massively parallel computer could be implemented as a system that assigned small pieces of a problem to individual agents, which then searched out idle machines on the network to carry out parts of the computation. The same security features that allow clients to run untrusted programs downloaded from a server let servers run untrusted programs uploaded from a client.
Shopping sites have proven to be one of the few real ways to make money from consumers on the Web. Although many sites accept credit cards through HTML forms, the mechanism is clunky. Shopping carts (pages that keep track of where users have been and what they have chosen) are at the outer limits of what’s possible with HTML and forms. Building a server-based shopping cart is difficult, requires lots of CGI and database work, and puts a huge CPU load on the server. And it still limits the interface options. For instance, the user can’t drag a picture of an item across the screen and drop it into a shopping cart. Java can move all this work to the client and offer richer user interfaces as well.
Applets can store state as the user moves from page to page, making shopping carts much easier to build. When the user finishes shopping, the applet sends the data back to the server across the network. Figure 1.10 shows one such shopping cart used on a Beanie Babies web site. To buy a doll, the user drags and drops its picture into the grocery bag.
Even this is too inconvenient and too costly for small payments of a couple of dollars or less. Nobody wants to fill out a form with name, address, billing address, credit card number, and expiration date every day just to pay $0.50 to read today’s Daily Planet. Imagine how easy it would be to implement this kind of transaction in Java. The user clicks on a link to some information. The server downloads a small applet that pops up a dialog box saying, “Access to the information at http://www.greedy.com/ costs $2. Do you wish to pay this?” The user can then click buttons that say “Yes” or “No”. If the user clicks the No button, then he doesn’t get into the site. Now let’s imagine what happens if the user clicks “Yes”.
The applet contains a small amount of information: the price, the URL, and the seller. If the client agrees to the transaction, then the applet adds the buyer’s data to the transaction, perhaps a name and an account number, and signs the order with the buyer’s private key. Then the applet sends the data back to the server over the network. The server grants the user access to the requested information using the standard HTTP security model. Then it signs the transaction with its private key and forwards the order to a central clearinghouse. Sellers can offer money-back guarantees or delayed purchase plans (No money down! Pay nothing until July!) by agreeing not to forward the transaction to the clearinghouse until a certain amount of time has elapsed.
The clearinghouse verifies each transaction with the buyer’s and seller’s public keys and enters the transaction in its database. The clearinghouse can use credit cards, checks, or electronic fund transfers to move money from the buyer to the seller. Most likely, the clearinghouse won’t move the money until the accumulated total for a buyer or seller reaches a certain minimum threshold, keeping the transaction costs low.
Every part of this can be written in Java. An applet requests the user’s permission. The Java Cryptography Extension authenticates and encrypts the transaction. The data moves from the client to the seller using sockets, URLs, CGI programs, servlets, and/or RMI. These can also be used for the host to talk to the central clearinghouse. The web server itself can be written in Java, as can the database and billing systems at the central clearinghouse; or JDBC can be used to talk to a traditional database such as Informix or Oracle.
The hard part of this is setting up a clearinghouse and getting users and sites to subscribe. The major credit card companies have a head start, though none of them yet use the scheme described here. In an ideal world you’d like the buyer and the seller to be able to use different banks or clearinghouses. However, this is a social problem, not a technological one; and it is solvable. You can deposit a check from any American bank at any other American bank where you have an account. The two parties to a transaction do not need to bank in the same place. Sun is currently developing a system somewhat like this as part of Java Wallet.
Java makes it possible to write many kinds of applications that have been imagined for years but haven’t been practical until now. Many of these applications would require too much processing power if they were entirely server-based; Java moves the processing to the client, where it belongs. Other application types (for example, mobile agents) require extreme portability and some guarantees that the application can’t do anything hostile to its host. While Java’s security model has been criticized (and yes, some bugs have been found), it’s a quantum leap beyond anything that has been attempted in the past and an absolute necessity for the mobile software we will want to write in the future.
Networked devices don’t have to be tied to particular physical locations, subnets, or IP addresses. Jini is a framework that sits on top of Java for easily and instantly connecting all sorts of devices to a network. For example, when a group of coworkers gather for a meeting, they generally bring with them a random assortment of personal digital assistants, laptops, cell phones, pagers, and other electronic devices. The conference room where they meet may have one or two PCs, perhaps a Mac, a digital projector, a printer, a coffee machine, a speaker phone, an Ethernet router, and assorted other useful tools. If these devices include a Java virtual machine and Jini, they form an impromptu network as soon as they’re turned on and plugged in. (With wireless connections, they may not even need to be plugged in.) Devices can join or leave the local network at any time without explicit reconfiguration. They can use one of the cell phones, the speaker phone, or the router to connect to hosts outside the room.
Participants can easily share files and trade data. Their computers and other devices can be configured to recognize and trust each other regardless of where in the network one happens to be at any given time. Trust can be restricted, though, so that, for example, all the laptops of company employees in the room are trusted, but those of outside vendors at the meeting aren’t. Some devices, such as the printer and the digital projector, may be configured to trust anyone in the room to use their services but to not allow more than one person to use them at once. Most importantly of all, the coffee machine may not trust anyone, but it can notice that it’s running out of coffee and email the supply room that it needs to be restocked.
Before the Web took the world by storm, Java was intended for the cable TV set-top box market. Five years after Java made its public debut, Sun’s finally got back to its original plans, but this time those plans are even more network-centric. PersonalJava is a stripped-down version of the rather large Java API that’s useful for set-top boxes and other devices with restricted memory, CPU power, and user interfaces, such as Palm Pilots. The Java TV API adds some television-specific features such as channel changing, and audio and video streaming and synchronization. Although PersonalJava is missing a lot of things you may be accustomed to in the full JDK, it does include a complete complement of networking classes. TV stations can send applets down the data stream that allow channel surfers to interact with the shows. An infomercial for spray-on hair could include an applet that lets the viewer pick a color, enter his credit card number, and send the order through the cable modem, back over the Internet using his remote control. A news magazine could conduct a viewer poll in real time and report the responses after the commercial break. Ratings could be collected from every household with a cable modem instead of merely the 5,000 Nielsen families.
Peer-to-peer networked Java programs can allow multiple people to collaborate on a document at one time. Imagine a Java word processor that two people, perhaps in different countries, can pull up and edit simultaneously. Imagine the interaction that’s possible when you attach an Internet phone. For example, two astronomers could work on a paper while one’s in New Mexico and the other’s in Moscow. The Russian could say, “I think you dropped the superscript in Equation 3.9”, and then type the corrected equation so that it appears on both people’s displays simultaneously. Then the astronomer in New Mexico might say, “I see, but doesn’t that mean we have to revise Figure 3.2 like this?” and then use a drawing tool to make the change immediately. This sort of interaction isn’t particularly hard to implement in Java (a word processor with a decent user interface for equations is probably the hardest part of the problem), but it does need to be built into the word processor from the start. It cannot be retrofitted onto a word processor that was not originally designed with networking in mind.
[2] SOFIA will be a 2.5-meter reflecting telescope mounted on a Boeing 747. When launched in 2001, it will be the largest airborne telescope in the world. Airborne telescopes have a number of advantages compared to ground-based telescopes—one is the ability to observe phenomena obscured by Earth’s atmosphere. Furthermore, rather than being fixed at one latitude and longitude, they can fly anywhere to observe phenomenon. For information about Java-based remote control of telescopes, see http://pioneer.gsfc.nasa.gov/public/irc/. For information about SOFIA, see http://www.sofia.usra.edu/.
[3] See, for example, the URL-minder at http://www.netmind.com/.
Get Java Network Programming, Second Edition now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
