XML jockeys might well want to avert their eyes for this one. What is herein suggested is something just so preposterous that it just might prove useful—and indeed it does. NoXML is a drop-in alternative to SOAP::Lite. As its name suggests, this home-brewed module doesn't make use of an XML parser of any kind, relying instead on some dead-simple regular expressions and other bits of programmatic magic.

TIP

As any XML guru will attest, there's simply no substitute for an honest-to-goodness XML parser. And they'd be right. There are encoding and hierarchy issues that a regular expression-based parser simply can't fathom. NoXML is simplistic at best. That said, it does what needs doing, the very essence of "hacking."

Best of all, NoXML can fill in for SOAP::Lite with little more than a two-line alteration to the target hack.

The Code

The heart of this hack is NoXML.pm, which should be saved into the same directory as your hacks themselves.

# NoXML.pm
# NoXML [pronounced "no xml"] is a dire-need drop-in 
# replacement for SOAP::Lite designed for Google Web API hacking.

package NoXML;

use strict;
no strict "refs";

# LWP for making HTTP requests, XML for parsing Google SOAP
use LWP::UserAgent;
use XML::Simple;

# Create a new NoXML
sub new {
  my $self = {};
  bless($self);
  return $self;
}

# Replacement for the SOAP::Lite-based doGoogleSearch method
sub doGoogleSearch {
  my($self, %args);
  ($self, @args{qw/ key q start maxResults filter restrict 
  safeSearch lr ie oe /}) = @_;

  # grab SOAP request from _  _DATA_  _
  my $tell = tell(DATA);
  my $soap_request = join '', ; 
  seek(DATA, $tell, 0);
  $soap_request =~ s/\$(\w+)/$args{$1}/ge; #interpolate variables

  # Make (POST) a SOAP-based request to Google
  my $ua = LWP::UserAgent->new;
  my $req = HTTP::Request->new(POST => 'http://api.google.com/search/beta2');
  $req->content_type('text/xml');
  $req->content($soap_request);
  my $res = $ua->request($req);
  my $soap_response = $res->as_string;

  # Drop the HTTP headers and so forth until the initial xml element
  $soap_response =~ s/^.+?(<\?xml)/$1/migs;

  # Drop element namespaces for tolerance of future prefix changes
  $soap_response =~ s!(<\/?)[\w-]+?:([\w-]+?)!$1$2!g;

  # Set up a return dataset
  my $return;

  # Unescape escaped HTML in the resultset
  my %unescape = ('<'=>'<', '>'=>'>', '&'=>'&', '"'=>'"', '&apos;'=>"'"); 
  my $unescape_re = join '|' => keys %unescape;

  # Divide the SOAP response into the results and other metadata
  my($before, $results, $after) = $soap_response =~ 
    m#(^.+)(.+?)(.+$)#migs ;
  my $before_and_after = $before . $after;

  # Glean as much metadata as possible (while being somewhat lazy ;-)
  while ($before_and_after =~ m#([^<]*?)<#migs) {
    $return->{$1} = $3; # pack the metadata into the return dataset
  }

  # Glean the results
  my @results;
  while ($results =~ m#(.+?)#migs) {
    my $item = $1;
    my $pairs = {};
    while ( $item =~ m#([^<]*)#migs ) {
      my($element, $value) = ($1, $2);
      $value =~ s/($unescape_re)/$unescape{$1}/g;
      $pairs->{$element} = $value;
    }
    push @results, $pairs;
  }

  # Pack the results into the return dataset
  $return->{resultElements} = \@results;

  # Return nice, clean, usable results
  return $return;
}

1;

# This is the SOAP message template sent to api.google.com. Variables
# signified with $variablename are replaced by the values of their 
# counterparts sent to the doGoogleSearch subroutine.

_  _DATA_  _
<?xml version='1.0' encoding='UTF-8'?>
<SOAP-ENV:Envelope 
 xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" 
 xmlns:xsi="http://www.w3.org/1999/XMLSchema-instance" 
 xmlns:xsd="http://www.w3.org/1999/XMLSchema">
  <SOAP-ENV:Body>
   <ns1:doGoogleSearch xmlns:ns1="urn:GoogleSearch" 
    SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
     <key xsi:type="xsd:string">$key</key>
     <q xsi:type="xsd:string">$q</q>
     <start xsi:type="xsd:int">$start</start>
     <maxResults xsi:type="xsd:int">$maxResults</maxResults>
     <filter xsi:type="xsd:boolean">$filter</filter>
     <restrict xsi:type="xsd:string">$restrict</restrict>
     <safeSearch xsi:type="xsd:boolean">$safeSearch</safeSearch>
     <lr xsi:type="xsd:string">$lr</lr>
     <ie xsi:type="xsd:string">$ie</ie>
     <oe xsi:type="xsd:string">$oe</oe>
   </ns1:doGoogleSearch>
 </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Here's a little script to show NoXML in action. It's no different, really, from any number of hacks in this book. The only minor alterations necessary to make use of NoXML instead of SOAP::Lite are highlighted in bold.

#!/usr/bin/perl
# noxml_google2csv.pl
# Google Web Search Results via NoXML ("no xml") module
# exported to CSV suitable for import into Excel
# Usage: noxml_google2csv.pl "{query}" [> results.csv]

# Your Google API developer's key
my $google_key='insert key here';

use strict;

# use SOAP::Lite;use NoXML;

$ARGV[0]
  or die qq{usage: perl noxml_search2csv.pl "{query}"\n};

# my $google_search = SOAP::Lite->service("file:$google_wdsl");my $google_search = new NoXML;

my $results = $google_search -> 
  doGoogleSearch(
    $google_key, shift @ARGV, 0, 10, "false", 
    "", "false", "", "latin1", "latin1"
  );
@{$results->{'resultElements'}} or die('No results');

print qq{"title","url","snippet"\n};

foreach (@{$results->{'resultElements'}}) {
  $_->NoXML, Another SOAP::Lite Alternative =~ s!"!""!g; # double escape " marks
  $_->{snippet} =~ s!"!""!g;
  my $output = qq{"$_->NoXML, Another SOAP::Lite Alternative","$_->{URL}","$_->{snippet}"\n};
  $output =~ s!<.+?>!!g; # drop all HTML tags
  print $output;
}