Perl Cookbook

Preface

The investment group eyed the entrepreneur with caution, their expressions flickering from scepticism to intrigue and back again.

“Your bold plan holds promise,” their spokesman conceded. “But it is very costly and entirely speculative. Our mathematicians mistrust your figures. Why should we entrust our money into your hands? What do you know that we do not?”

“For one thing,” he replied, “I know how to balance an egg on its point without outside support. Do you?” And with that, the entrepreneur reached into his satchel and delicately withdrew a fresh hen’s egg. He handed over the egg to the financial tycoons, who passed it amongst themselves trying to carry out the simple task. At last they gave up. In exasperation they declared, “What you ask is impossible! No man can balance an egg on its point.”

So the entrepreneur took back the egg from the annoyed businessmen and placed it upon the fine oak table, holding it so that its point faced down. Lightly but firmly, he pushed down on the egg with just enough force to crush in its bottom about half an inch. When he took his hand away, the egg stood there on its own, somewhat messy, but definitely balanced. “Was that impossible?” he asked.

“It’s just a trick,” cried the businessmen. “Once you know how, anyone can do it.”

“True enough,” came the retort. “But the same can be said for anything. Before you know how, it seems an impossibility. Once the way is revealed, it’s so simple that you wonder why you never thought of it that way before. Let me show you that easy way, so others may easily follow. Will you trust me?”

Eventually convinced that this entrepreneur might possibly have something to show them, the skeptical venture capitalists funded his project. From the tiny Andalusian port of Palos de Moguer set forth the Niña, the Pinta, and the Santa María, led by an entrepreneur with a slightly broken egg and his own ideas: Christopher Columbus.

Many have since followed.

Approaching a programming problem can be like balancing Columbus’s egg. If no one shows you how, you may sit forever perplexed, watching the egg—and your program—fall over again and again, no closer to the Indies than when you began. This is especially true in a language as idiomatic as Perl.

This book had its genesis in two chapters of the first edition of Programming Perl. Chapters 5 and 6 covered “Common Tasks in Perl” and “Real Perl Programs.” Those chapters were highly valued by readers because they showed real applications of the language—how to solve day-to-day tasks using Perl. While revising the Camel, we realized that there was no way to do proper justice to those chapters without publishing the new edition on onionskin paper or in multiple volumes. The book you hold in your hands, published two years after the revised Camel, tries to do proper justice to those chapters. We trust it has been worth the wait.

This book isn’t meant to be a complete reference book for Perl, although we do describe some parts of Perl previously undocumented. Having a copy of Programming Perl handy will allow you to look up the exact definition of an operator, keyword, or function. Alternatively, every Perl installation comes with over 1,000 pages of searchable, online reference materials. If those aren’t where you can get at them, see your system administrator.

Neither is this book meant to be a bare-bones introduction for programmers who’ve never seen Perl before. That’s what Learning Perl, a kinder and gentler introduction to Perl, is designed for. (If you’re on a Microsoft system, you’ll probably prefer the Learning Perl for Win32 Systems version.)

Instead, this is a book for learning more Perl. Neither a reference book nor a tutorial book, the Perl Cookbook serves as a companion book to both. It’s for people who already know the basics but are wondering how to mix all those ingredients together into a complete program. Spread across 20 chapters and more than 300 focused topic areas affectionately called recipes, this book contains thousands of solutions to everyday challenges encountered by novice and journeyman alike.

We tried hard to make this book useful for both random and sequential access. Each recipe is self-contained, but has a list of references at the end should you need further information on the topic. We’ve tried to put the simpler, more common recipes toward the front of each chapter and the simpler chapters toward the front of the book. Perl novices should find that these recipes about Perl’s basic data types and operators are just what they’re looking for. We gradually work our way through topic areas and solutions more geared toward the journeyman Perl programmer. Every now and then we include material that should inspire even the master Perl programmer.

Each chapter begins with an overview of that chapter’s topic. This introduction is followed by the main body of each chapter, its recipes. In the spirit of the Perl slogan of TMTOWTDI, “There’s more than one way to do it,” most recipes show several different techniques for solving the same or closely related problems. These recipes range from short-but-sweet solutions to in-depth mini-tutorials. Where more than one technique is given, we often show costs and benefits of each approach.

As with a traditional cookbook, we expect you to access this book more or less at random. When you want to learn how to do something, you’ll look up its recipe. Even if the exact solutions presented don’t fit your problem exactly, they’ll give you ideas about possible approaches.

Each chapter concludes with one or more complete programs. Although some recipes already include small programs, these longer applications highlight the chapter’s principal focus and combine techniques from other chapters, just as any real-world program would. All are useful, and many are used on a daily basis. Some even helped us put this book together.

What’s in This Book

The first quarter of the book addresses Perl’s basic data types, spread over five chapters. Chapter 1, covers matters like accessing substrings, expanding function calls in strings, and parsing comma-separated data. Chapter 2, tackles oddities of floating point representation, placing commas in numbers, and pseudo-random numbers. Chapter 3, demonstrates conversions between numeric and string date formats and using timers. Chapter 4, covers everything relating to list and array manipulation, including finding unique elements in a list, efficiently sorting lists, and randomizing them. Chapter 5, concludes the basics with a demonstration of the most useful data type, the associative array. The chapter shows how to access a hash in insertion order, how to sort a hash by value, and how to have multiple values per key.

Chapter 6, is by far the largest chapter. Recipes include converting a shell wildcard into a pattern, matching letters or words, matching multiple lines, avoiding greediness, and matching strings that are close to but not exactly what you’re looking for. Although this chapter is the longest in the book, it could easily have been longer still—every chapter contains uses of regular expressions. It’s part of what makes Perl Perl.

The next three chapters cover the filesystem. Chapter 7, shows opening files, locking them for concurrent access, modifying them in place, and storing filehandles in variables. Chapter 8, discusses watching the end of a growing file, reading a particular line from a file, and random access binary I/O. Finally, in Chapter 9, we show techniques to copy, move, or delete a file, manipulate a file’s timestamps, and recursively process all files in a directory.

Chapters 10 through 13 focus on making your program flexible and powerful. Chapter 10, includes recipes on creating persistent local variables, passing parameters by reference, calling functions indirectly, and handling exceptions. Chapter 11, is about data structures; basic manipulation of references to data and functions are demonstrated. Later recipes show how to create record-like data structures and how to save and restore these structures from permanent storage. Chapter 12, concerns breaking up your program into separate files; we discuss how to make variables and functions private to a module, replace built-ins, trap calls to missing modules, and use the h2ph and h2xs tools to interact with C and C++ code. Lastly, Chapter 13, covers the fundamentals of building your own object-based module to create user-defined types, complete with constructors, destructors, and inheritance. Other recipes show examples of circular data structures, operator overloading, and tied data types.

The next two chapters are about interfaces: one to databases, the other to display devices. Chapter 14, includes techniques for manipulating indexed text files, locking DBM files and storing data in them, and a demonstration of Perl’s SQL interface. Chapter 15, covers topics such as clearing the screen, processing command-line switches, single-character input, moving the cursor using termcap and curses, and platform independent graphical programming using Tk.

The last quarter of the book is devoted to interacting with other programs and services. Chapter 16, is about running other programs and collecting their output, handling zombie processes, named pipes, signal management, and sharing variables between running programs. Chapter 17, shows how to establish stream connections or use datagrams to create low-level networking applications for client-server programming. Chapter 18, is about higher-level protocols such as mail, FTP, Usenet news, and Telnet. Chapter 19, contains recipes for processing web forms, trapping their errors, avoiding shell escapes for security, managing cookies, shopping cart techniques, and saving forms to files or pipes. The final chapter of the book, Chapter 20 , covers non-interactive uses of the Web. Recipes include fetching a URL, automating form submissions in a script, extracting URLs from a web page, removing HTML tags, finding fresh or stale links, and processing server log files.