O'Reilly Hacks
oreilly.comO'Reilly NetworkSafari BookshelfConferences Sign In/My Account | View Cart   
Book List Learning Lab PDFs O'Reilly Gear Newsletters Press Room Jobs  


 
Buy the book!
Yahoo! Hacks
By Paul Bausch
October 2005
More Info

HACK
#68
Program Yahoo! with Perl
Grabbing data from Yahoo! Search Web Services can be quite easy with just a little bit of Perl
[Discuss (1) | Link to this hack]

Perl is great for getting things done quickly, and fetching search results from Yahoo! is no exception. This hack shows a simple way to access Yahoo! with Perl, using minimal code. Think of it as a doorway to Yahoo! that you can drop into your own Perl scripts, or that you can use as a starting point for more complex applications. This script will accept a keyword or phrase, contact Yahoo! Search, and print out the first 10 results.

What You Need

In the spirit of keeping things easy, this hack uses two simple modules that may already be installed on your system: LWP::Simple(http://search.cpan.org/~gaas/libwww-perl-5.803/lib/LWP/Simple.pm) makes the HTTP request; XML::Simple (http://search.cpan.org/~grantm/XML-Simple-2.14/lib/XML/Simple.pm) parses the XML response. If you need to install these modules, you can use CPAN for each module:

perl –MCPAN –e shell
	cpan> install XML::Simple

On a Windows system with ActivePerl, you can install these modules from the command line with the Perl package manager, like this:

ppm install LWP-Simple

The only other piece you'll need is a unique application ID from Yahoo!, which you can pick up at http://api.search.yahoo.com/webservices/register_application.

The Code

This code builds a Yahoo! Search Web Services request URL using the keyword passed to it when the script is run. Then it parses the response and prints it out in a readable format. Save the following code to a file named yahoo_search.pl:

	#!/usr/bin/perl
	# yahoo_search.pl
	# Accepts a search term and shows the top results.
	# Usage: yahoo_search.pl <Query>
	#
	# You can create an AppID, and read the full documentation
	# for Yahoo! Web Services at http://developer.yahoo.net/

	use strict;
	use LWP::Simple;
	use XML::Simple;

	# Set your unique Yahoo! Application ID
	my $appID = "insert your app ID";
	
	# Grab the incoming search query 
	my $query = join(' ', @ARGV) or die "Usage: yahoo_search.pl <query>\n";
	# Construct a Yahoo! Search Query with only required options
	my $language = "en";
	my $req_url = "http://api.search.yahoo.com/";

		$req_url .= "WebSearchService/V1/webSearch?";
		$req_url .= "appid=$appID";
		$req_url .= "&query=$query";
		$req_url .= "&language=$language";

	# Make the request
	my $yahoo_response = get($req_url);

	# Parse the XML
	my $xmlsimple = XML::Simple->new();
	my $yahoo_xml = $xmlsimple->XMLin($yahoo_response);

	# Initialize results counter
	my $i;

	# Loop through the items returned, printing them out
	foreach my $result (@{$yahoo_xml->{Result}}) {
		$i++;
		my $title = $result->{Title};
		my $summary = $result->{Summary};
		my $url = $result->{Url};
		print "$i. $title\n$summary\n$url\n\n";
		
	}

The final print command sends the information from Yahoo! to STDOUT. You can change what this script shows by rearranging the variables and making this last line more or less complex.


O'Reilly Home | Privacy Policy

© 2007 O'Reilly Media, Inc.
Website: | Customer Service: | Book issues:

All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners.