The 20-Minute SNMP Tutorial - Automating System Administration with Perl

by David N. Blank-Edelman

The Simple Network Management Protocol (SNMP) is the ubiquitous protocol used to manage devices on a network. Unfortunately, as mentioned at the beginning of Chapter 12, SNMP, SNMP is not a particularly simple protocol (despite its name). This longish tutorial will give you the information you need to get started with version 1 of SNMP.

Automating System Administration with Perl, Second Edition book cover

This excerpt is from Automating System Administration with Perl, Second Edition . Thoroughly updated and expanded in its second edition to cover the latest operating systems, technologies, and Perl modules, Automating System Administration with Perl will help you perform your job with less effort. The second edition not only offers you the right tools for your job, but also suggests the best way to approach particular problems and securely automate pressing tasks.

buy button

SNMP is predicated on the notion of a management station polling SNMP agents running on remote devices for information. An agent can also signal the management station if an important condition arises, such as a counter exceeding a threshold. When we programmed SNMP in Perl in Chapter 12, SNMP, we essentially acted as a management station, polling the SNMP agents on other network devices.

We’re going to concentrate on version 1 of SNMP in this tutorial. Seven versions of the protocol (SNMPv1, SNMPsec, SNMPv2p, SNMPv2c, SNMPv2u, SNMPv2*, and SNMPv3) have been proposed; v1 is the one that has been most widely implemented and deployed, though v3 is expected to eventually ascend thanks to its superior security architecture.

Perl and SNMP both have simple data types. Perl uses a scalar as its base type. Lists and hashes are just collections of scalars in Perl. In SNMP, you also work with scalar variables. SNMP variables can hold any of four primitive types: integers, strings, object identifiers (more on this in a moment), or null values. And just like in Perl, in SNMP a set of related variables can be grouped together to form larger structures (most often tables). This is where their similarity ends.

Perl and SNMP diverge radically on the subject of variable names. In Perl, you can, given a few restrictions, name your variables anything you’d like. SNMP variable names are considerably more restrictive. All SNMP variables exist within a virtual hierarchical storage structure known as the management information base (MIB). All valid variable names are defined within this framework. The MIB, now at version MIB-II, defines a tree structure for all of the objects (and their names) that can be managed via SNMP.

In some ways the MIB is similar to a filesystem: instead of organizing files, the MIB logically organizes management information in a hierarchical tree-like structure. Each node in this tree has a short text string, called a label, and an accompanying number that represents its position at that level in the tree. To give you a sense of how this works, let’s go find the SNMP variable in the MIB that holds a system’s description of itself. Bear with me; we have a bit of a tree walking (eight levels’ worth) to do to get there.

Figure G.1, “Finding sysDescr(1) in the MIB” shows a picture of the top of the MIB tree.

Figure G.1. Finding sysDescr(1) in the MIB

Finding sysDescr(1) in the MIB

The top of the tree consists of standards organizations: iso(1), ccitt(2), joint-iso-ccitt(3). Under the iso(1) node, there is a node called org(3) for other organizations. Under this node is dod(6), for the Department of Defense. Under that node is internet(1), a subtree for the Internet community.

Here’s where things start to get interesting. The Internet Architecture Board has assigned the subtrees listed in Table G.1, “Subtrees of the internet(1) node” under internet(1).

Table G.1. Subtrees of the internet(1) node

Subtree

Description

directory(1)

OSI directory

mgmt(2)

RFC standard objects

experimental(3)

Internet experiments

private(4)

Vendor-specific

security(5)

Security

snmpV2(6)

SNMP internals


Because we’re interested in using SNMP for device management, we will want to take the mgmt(2) branch. The first node under mgmt(2) is the MIB itself (this is almost recursive). Since there is only one MIB, the only node under mgmt(2) is mib-2(1).

The real meat (or tofu) of the MIB begins at this level in the tree. We find the first set of branches, called object groups, which hold the variables we’ll want to query:

system(1)
interfaces(2)
at(3)
ip(4)
icmp(5)
tcp(6)
udp(7)
egp(8)
cmot(9)
transmission(10)
snmp(11)

Remember, we’re hunting for the “system description” SNMP variable, so the system(1) group is the logical place to look. The first node in that tree is sysDescr(1). Bingo—we’ve located the object we need.

Why bother with all this tree-walking stuff? This trip provides us with sysDescr(1)’s object identifier (OID), which is the dotted set of the numbers from each label of the tree we encountered on our way to this object. Figure G.2, “Finding the OID for our desired object” shows this graphically.

So, the OID for the Internet tree is 1.3.6.1, the OID for the system object group is 1.3.6.1.2.1.1, and the OID for the sysDescr object is 1.3.6.1.2.1.1.1.

When we want to actually use this OID in practice, we’ll need to tack on another number to get the value of this variable. That is, we will need to append a .0, representing the first (and only, since a device cannot have more than one description) instance of this object.

Let’s do that now, to get a sneak preview of SNMP in action. In this appendix we’ll be using the command-line tools from the Net-SNMP package for demonstration purposes. This package is an excellent free SNMPv1 and v3 implementation. We’re using this particular implementation because one of the Perl modules links to its library, but any other client that can send an SNMP request will do just as nicely. Once you’re familiar with command-line SNMP utilities, making the jump to the Perl equivalents is easy.

Figure G.2. Finding the OID for our desired object

Finding the OID for our desired object

The Net-SNMP command-line tools allow us to prepend a dot (.) if we wish to specify an OID/variable name starting at the root of the tree. Here are two ways we might query the machine solarisbox for its systems description (note that the second command should appear on one line; it’s broken here with a line continuation marker for readability):

$ snmpget -v 1 -c public solarisbox .1.3.6.1.2.1.1.1.0

$ snmpget -v 1 -c public solarisbox \
.iso.org.dod.internet.mgmt.mib-2.system.sysDescr.0

These lines both yield:

system.sysDescr.0 = Sun SNMP Agent, Ultra-1

Back to the theory. It is important to remember that the P in SNMP stands for Protocol. SNMP itself is just the protocol for the communication between entities in a management infrastructure. The operations, or “protocol data units” (PDUs), are meant to be simple. Here are the PDUs you’ll see most often, especially when programming in Perl:[148]

get-request

get-request is the workhorse of the PDU family: it is used to poll an SNMP entity for the value of some SNMP variable. Many people live their whole SNMP lives using nothing but this operation.

get-next-request

get-next-request is just like get-request, except it returns the item in the MIB just after the specified item (the “first lexicographic successor” in RFC terms). This operation comes into play most often when you are attempting to find all of the items in a logical table object. For instance, you might send a set of repeated get-next-requests to query for each line of a workstation’s ARP table. We’ll see an example of this in practice in a moment.

get-bulk-request

get-bulk-request is an SNMPv2/v3 addition that allows for the bulk transfer of information. With other PDUs, you typically ask for and receive one piece of information. get-bulk lets you make one query and receive a whole set of values. This can be a much more efficient way to transfer chunks of information (like whole tables).

set-request

set-request does just what you would anticipate: it attempts to change the value of an SNMP variable. This is the operation used to change the configuration of an SNMP-capable device.

trap/snmpV2-trap

trap is the SNMPv1 name, and snmpV2-trap is the SNMPv2/3 name. Traps allow you to ask an SNMP-capable box to signal its management entity about an event (e.g., a reboot, or a counter threshold being reached) without being explicitly polled. Traps report events right when they happen, rather than when the agent is polled.

inform-request

inform-request is an SNMPv2/3 addition to the PDU list. It provides trap-like functionality with the addition of confirmation. (With normal trap requests, the agent sends a notification but has no way of knowing if that notification was received. Informs provide this mechanism.)

response

response is the PDU used to carry back the response from any of the other PDUs. It can be used to reply to a get-request, signal if a set-request succeeded, and so on. You rarely reference this PDU explicitly when programming, since most SNMP libraries, programs, and Perl modules handle SNMP response receipt automatically. Still, it is important to understand not just how requests are made, but also how they are answered.

If you’ve never dealt with SNMP before, a natural reaction to this list might be, “That’s it? Get, set, tell me when something happens, that’s all it can do?” But simple, as SNMP’s creators realized early on, is not the opposite of powerful. If the manufacturer of an SNMP device chooses his variables well, there’s little that cannot be done with the protocol. The classic example from the RFCs is the rebooting of an SNMP-capable device. There may be no “reboot-request” PDU, but a manufacturer could easily implement this operation by using an SNMP trigger variable to hold the number of seconds before a reboot. When this variable is changed via set-request, a reboot of the device can be initiated in the specified amount of time.

Given this power, what sort of security is in place to keep anyone with an SNMP client from rebooting your machine? In earlier versions of the protocol, the protection mechanism was pretty puny. In fact, some people have taken to expanding the acronym as “Security Not My Problem” because of SNMPv1’s poor authentication mechanism. To explain the who, what, and how of this protection mechanism, we have to drag out some nomenclature, so bear with me.

SNMPv1 and SNMPv2c allow you to define administrative relationships between SNMP entities called communities. Communities are a way of grouping SNMP agents that have similar access restrictions with the management entities that meet those restrictions. All entities that are in a community share the same community name. To prove you are part of a community, you just have to know the name of that community. That is the who can access? part of the scheme.

Now for the what can they access? part. RFC 1157 calls the parts of a MIB applicable to a particular network entity an SNMP MIB view. For instance, an SNMP-capable toaster[149] would not provide all of the same SNMP configuration variables as an SNMP-capable router.

Each object in an MIB is defined by its accessibility: read-only, read-write, or none. This is known as that object’s SNMP access mode. If we put an SNMP MIB view and an SNMP access mode together, we get an SNMP community profile that describes the type of access available to the applicable variables in the MIB by a particular community.

When we bring together the who and what parts, we have an SNMP access policy that describes what kind of access members of a particular community offer each other.

How does this all work in real life? You configure your router or your workstation to be in at least two communities, one controlling read and the other controlling read/write access. People often refer to these communities as the public and private communities, named after popular default names for these communities. For instance, on a Cisco router you might include this as part of the configuration:

! set the read-only community name to MyPublicCommunityName
snmp-server community MyPublicCommunityName RO

! set the read-write community name to MyPrivateCommunityName
snmp-server community MyPrivateCommunityName RW

On a Solaris machine, you might include this in the /etc/snmp/conf/snmpd.conf file:

read-community  MyPublicCommunityName
write-community MyPrivateCommunityName

SNMP queries to either of these devices would have to use the MyPublicCommunityName community name to gain access to read-only variables or the MyPrivateCommunityName community name to change read/write variables on those devices. In other words, the community name functions as a pseudo-password used to gain SNMP access to a device. This is a poor security scheme. Not only is the community name passed in clear text in every SNMPv1 packet, but the overall strategy is “security by obscurity.”

Later versions of SNMP—in particular, v3—added significantly better security to the protocol. RFCs 3414 and 3415 define a User Security Model (USM) and a View-Based Access Control Model (VACM): USM provides crypto-based protection for authentication and encryption of messages, while VACM offers a comprehensive access-control mechanism for MIB objects. We won’t be discussing these mechanisms here, but it is probably worth your while to peruse the RFCs since v3 is increasing in popularity. I’d also recommend reading the SNMPv3 tutorials provided with the Net-SNMP distribution. If you are interested in USM and VACM and how they can be configured, the SNMP vendor NuDesign Technologies has also published a good tutorial on the subject (http://www.ndt-inc.com/SNMP/HelpFiles/v3ConfigTutorial/v3ConfigTutorial.html).

SNMP in Practice

Now that you’ve received a healthy dose of SNMP theory, let’s do something practical with this knowledge. You’ve already seen how to query a machine’s system description (remember the sneak preview earlier), so now let’s look at two more examples: querying the system uptime and the IP routing table.

Until now, you just had to take my word for the location and name of an SNMP variable in the MIB. Querying information via SNMP is a two-step process:

  1. Find the right MIB document. If you are looking for a device-independent setting that could be found on any generic SNMP device, you will probably find it in RFC 1213.[150] If you need a vendor-specific variable name (e.g., the variable that holds the color of the blinky-light on the front panel of a specific VoIP switch) you will need to contact the switch’s vendor and request a copy of the vendor’s MIB module. I’m being pedantic about the terms here because it is not uncommon to hear people incorrectly say, “I need the MIB for that device.” There is only one MIB in the world; everything else fits somewhere in that structure (usually off of the private(4) branch).

  2. Search through MIB descriptions until you find the SNMP variable(s) you need.

To make this second step easier for you,[151] let me help decode the format.

MIB descriptions aren’t all that scary once you get used to them. They look like one long set of variable declarations similar to those you would find in source code. This is no coincidence, because they are variable declarations. If a vendor has been responsible in the construction of its module, that module will be heavily commented like any good source code file.

MIB information is written in a subset of Abstract Syntax Notation One (ASN.1), an Open Systems Interconnection (OSI) standard notation. A description of this subset and other details of the data descriptions for SNMP are found in the Structure for Management Information (SMI) RFCs that accompany the RFCs that define the SNMP protocol and the current MIB. For instance, the latest (as of this writing) SNMP protocol definition can be found in RFC 3416, the latest base MIB manipulated by this protocol is in RFC 3418, and the SMI for this MIB is in RFC 2578. I bring this to your attention because it is not uncommon to have to flip between several documents when looking for specifics on an SNMP subject.

Let’s use this knowledge to address the first task at hand: finding the system uptime of a machine via SNMP. This information is fairly generic, so there’s a good chance we can find the SNMP variable we need in RFC 1213. A quick search for “uptime” in RFC 1213 yields this snippet of ASN.1:

sysUpTime OBJECT-TYPE
              SYNTAX  TimeTicks
              ACCESS  read-only
              STATUS  mandatory
              DESCRIPTION
                      "The time (in hundredths of a second) since the
                      network management portion of the system was last
                      re-initialized."
              ::= { system 3 }

Let’s take this definition apart line by line:

sysUpTime OBJECT-TYPE

This defines the object called sysUpTime.

SYNTAX TimeTicks

This object is of the type TimeTicks. Object types are specified in the SMI I mentioned a moment ago.

ACCESS read-only

This object can only be read via SNMP (i.e., with get-request); it cannot be changed (i.e., with set-request).

STATUS mandatory

This object must be implemented in any SNMP agent.

DESCRIPTION...

This is a textual description of the object. Always read this field carefully. In this definition, there’s a surprise in store for us: sysUpTime only shows the amount of time that has elapsed since “the network management portion of the system was last re-initialized.” This means we’re only going to be able to tell a system’s uptime since its SNMP agent was last started. This is almost always the same as when the system itself last started, but if you spot an anomaly, this could be the reason.

::= { system 3 }

Here’s where this object fits in the MIB tree. The sysUpTime object is the third branch off of the system object group tree. This information also gives you part of the OID, should you need it later.

If we wanted to query this variable on the machine solarisbox in the read-only community, we could use the following Net-SNMP tool command line:

$ snmpget -v 1 -c MyPublicCommunityName solarisbox system.sysUpTime.0

This returns:

system.sysUpTime.0 = Timeticks: (5126167) 14:14:21.67

indicating that the agent was last initialized 14 hours ago.

Note

The examples in this appendix assume our SNMP agents have been configured to allow requests from the querying host. In general, if you can restrict SNMP access to a certain subset of “trusted” hosts, you should.

“Need to know” is an excellent security principle to follow. It is good practice to restrict the network services provided by each machine and device. If you do not need to provide a network service, turn it off. If you do need to provide it, restrict the access to only the devices that “need to know.”

Time for our second and more advanced SNMP example: dumping the contents of a device’s IP routing table. The complexity in this example comes from the need to treat a collection of scalar data as a single logical table. We will have to invoke the get-next-request PDU to pull this off. Our first step toward this goal is to look for an MIB definition of the IP routing table. Searching for “route” in RFC 1213, we eventually find this definition:

-- The IP routing table contains an entry for each route
-- presently known to this entity.
ipRouteTable OBJECT-TYPE
    SYNTAX  SEQUENCE OF IpRouteEntry
    ACCESS  not-accessible
    STATUS  mandatory
    DESCRIPTION
              "This entity's IP Routing table."
    ::= { ip 21 }

This doesn’t look much different from the definition we took apart just a moment ago. The differences are in the ACCESS and SYNTAX lines. The ACCESS line is a tip-off that this object is just a structural placeholder representing the whole table, not a real variable that can be queried. The SYNTAX line tells us this is a table consisting of a set of IpRouteEntry objects. Let’s look at the beginning of the IpRouteEntry definition:

ipRouteEntry OBJECT-TYPE
    SYNTAX  IpRouteEntry
    ACCESS  not-accessible
    STATUS  mandatory
    DESCRIPTION
            "A route to a particular destination."
    INDEX   { ipRouteDest }
    ::= { ipRouteTable 1 }

The ACCESS line says we’ve found another placeholder—the placeholder for each of the rows in our table. But this placeholder also has something to tell us. It indicates that we’ll be able to access each row by using an index object, the ipRouteDest object of each row.

If these multiple definition levels throw you, it may help to relate this to Perl. Pretend we’re dealing with a Perl hash of lists structure. The hash key for the row would be the ipRouteDest variable. The value for this hash would then be a reference to a list containing the other elements in that row (i.e., the rest of the route entry).

The ipRouteEntry definition continues as follows:

ipRouteEntry ::=
    SEQUENCE {
        ipRouteDest
            IpAddress,
        ipRouteIfIndex
            INTEGER,
        ipRouteMetric1
            INTEGER,
        ipRouteMetric2
            INTEGER,
        ipRouteMetric3
            INTEGER,
        ipRouteMetric4
            INTEGER,
        ipRouteNextHop
            IpAddress,
        ipRouteType
            INTEGER,
        ipRouteProto
            INTEGER,
        ipRouteAge
            INTEGER,
        ipRouteMask
            IpAddress,
        ipRouteMetric5
            INTEGER,
        ipRouteInfo
            OBJECT IDENTIFIER
    }

Now you can see the elements that make up each row of the table. The MIB continues by describing those elements. Here are the first two definitions for these elements:

ipRouteDest OBJECT-TYPE
    SYNTAX  IpAddress
    ACCESS  read-write
    STATUS  mandatory
    DESCRIPTION
            "The destination IP address of this route. An
            entry with a value of 0.0.0.0 is considered a
            default route. Multiple routes to a single
            destination can appear in the table, but access to
            such multiple entries is dependent on the table-
            access mechanisms defined by the network
            management protocol in use."
     ::= { ipRouteEntry 1 }

 ipRouteIfIndex OBJECT-TYPE
     SYNTAX  INTEGER
     ACCESS  read-write
     STATUS  mandatory
     DESCRIPTION
             "The index value which uniquely identifies the
             local interface through which the next hop of this
             route should be reached. The interface identified
             by a particular value of this index is the same
             interface as identified by the same value of
             ifIndex."
     ::= { ipRouteEntry 2 }

Figure G.3, “The ipRouteTable structure and its index” shows a picture of the ipRouteTable part of the MIB to help summarize all of this information.

Figure G.3. The ipRouteTable structure and its index

The ipRouteTable structure and its index

Once you understand this part of the MIB, the next step is querying the information. This is a process known as “table traversal.” Most SNMP packages have a command-line utility called something like snmptable or snmp-tbl that will perform this process for you, but they might not offer the granularity of control you need. For instance, you may not want a dump of the whole routing table; you may just want a list of all of the ipRouteNextHops. On top of this, some of the Perl SNMP packages do not have tree-walking routines. For all of these reasons, it is worth knowing how to perform this process by hand.

To make this process easier to understand, I’ll show you up front the information we’re eventually going to be receiving from the device. This will let you see how each step of the process adds another row to the table data we’ll collect. If I log into a sample machine (as opposed to using SNMP to query it remotely) and type netstat -nr to dump the IP routing table, the output might look like this:

default          192.168.1.1       UGS        0    215345  tu0
127.0.0.1        127.0.0.1         UH         8   5404381  lo0
192.168.1/24     192.168.1.189     U          15  9222638  tu0

This shows the default internal loopback and local network routes, respectively.

Now let’s see how we go about obtaining a subset of this information via the Net-SNMP command-line utilities. For this example, we’re only going to concern ourselves with the first two columns of the output (route destination and next hop address). We make an initial request for the first instance of those two variables in the table. Everything in bold type is one long command line and is only printed here on separate lines for legibility:

$ snmpgetnext -v 1 -c public computer \

ip.ipRouteTable.ipRouteEntry.ipRouteDest \
ip.ipRouteTable.ipRouteEntry.ipRouteNextHop
ip.ipRouteTable.ipRouteEntry.ipRouteDest.0.0.0.0 = IpAddress: 0.0.0.0
ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.0.0.0.0 = IpAddress: 192.168.1.1

We need to pay attention to two parts of this response. The first is the actual data: the information returned after the equals sign. 0.0.0.0 means “default route,” so the information returned corresponded to the first line of the routing table output. The second important part of the response is the .0.0.0.0 tacked onto the variable names. This is the index for the ipRouteEntry entry representing the table row.

Now that we have the first row, we can make another get-next-request call, this time using the index. A get-next-request always returns the next item in an MIB, so we feed it the index of the row we just received to get back the next row after it:

$ snmpgetnext -v 1 -c public computer \

ip.ipRouteTable.ipRouteEntry.ipRouteDest.0.0.0.0\
ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.0.0.0.0
ip.ipRouteTable.ipRouteEntry.ipRouteDest.127.0.0.1 = IpAddress: 127.0.0.1
ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.127.0.0.1 = IpAddress: 127.0.0.1

You can probably guess the next step. We issue another get-next-request using the 127.0.0.1 part (the index) of the ip.ipRouteTable.ipRouteEntry.ipRouteDest.127.0.0.1 response:

$ snmpgetnext -v 1 -c public computer \
ip.ipRouteTable.ipRouteEntry.ipRouteDest.127.0.0.1 \
ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.127.0.0.1
ip.ipRouteTable.ipRouteEntry.ipRouteDest.192.168.1 = IpAddress: 192.168.1.0
ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.192.168.11.0 = IpAddress: 192.168.1.189

Looking at the sample netstat output shown earlier, you can see we’ve achieved our goal and dumped all of the rows of the IP routing table. How would we know this if we had dispensed with the dramatic irony and hadn’t seen the netstat output ahead of time? Under normal circumstances, we would have to proceed as usual and continue querying:

$ snmpgetnext -v 1 -c public computer \

ip.ipRouteTable.ipRouteEntry.ipRouteDest.192.168.1.0 \
ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.192.168.1.0
ip.ipRouteTable.ipRouteEntry.ipRouteIfIndex.0.0.0.0 = 1
ip.ipRouteTable.ipRouteEntry.ipRouteType.0.0.0.0 = indirect(4)

Whoops, the response did not match the request! We asked for ipRouteDest and ipRouteNextHop but got back ipRouteIfIndex and ipRouteType. We’ve fallen off the edge of the ipRouteTable table. The SNMP get-next-request PDU has done its sworn duty and returned the “first lexicographic successor” in the MIB for each of the objects in our request. Looking back at the definition of ipRouteEntry in the previous excerpt from RFC 1213, we can see that ipRouteIfIndex(2) follows ipRouteDest(1), and ipRouteType(8) does indeed follow ipRouteNextHop(7).

The answer to the question of how you know when you’re done querying for the contents of a table is “When you notice you’ve fallen off the edge of that table.” Programmatically, this translates into checking that the same string or OID prefix you requested is returned in the answer to your query. For instance, you might make sure that all responses to a query about ipRouteDest contained either ip.ipRouteTable.ipRouteEntry.ipRouteDest or 1.3.6.1.2.1.4.21.1.1.

Now that you have the basics of SNMP under your belt, you may want to turn to Chapter 12, SNMP to see how you can use it from Perl. You should also check out the references at the end of Chapter 12, SNMP for more information on SNMP.



[148] The canonical list of PDUs for SNMPv2 and v3 is found in RFC 3416; it builds upon the list of PDUs in SNMPv1’s RFC 1157. The list in the RFC doesn’t contain many more PDUs than are cited here, so you’re not missing much.

[149] There used to be several SNMP-capable soda machines on the Web, so it isn’t all that far-fetched. Scoff if you will, but the Internet Toaster (controlled via SNMP over a SLIP connection) first made its debut in 1990!

[150] RFC 1213 is marginally updated by RFCs 4293, 4022, and 4113. RFC 3418 adds additional SNMPv2 items to the MIB.

[151] This task can become even easier if you use a good GUI MIB browser like mbrowse or jmibbrowser. You can often get a hunch about the MIB contents by performing an snmpwalk on the device.

If you enjoyed this excerpt, buy a copy of Automating System Administration with Perl, Second Edition .