I have not noticed that eBay redoes the underlying HTML on its auction results pages. Even if this is the case, a sophisticated regular expression or regular expression builder would be able to pull results out of all but the most convoluted HTML. Anything eBay does you can do better.
How can we scrape the ebay auction results page if ebay keeps changing the screen very regularly?
Any ideas or thoughts or links would be appreciated...
Regards,
I can testify that writing webbots that scrape eBay pages is a very bad idea, especially when the API is so easy to use and available to people who want to develop. Scraping eBay pages is a maintenance nightmare.
Having done it both ways my advaice is to use the API. It's much easier and very stable.
Not only is hacking eBay in this way against eBay's policies, it's a terrible idea from a technical standpoint. Why? Because eBay changes the HTML on its site like clockwork, every two weeks. This means that every two weeks, applications that use this hack to scrape HTML off the site instead of using the API have an opportunity to break. I've talked to lots of developers who have done things the wrong way. They have had to perform literally hundreds of upgrades to their applications to keep up -- in some cases, taking their product off the market -- because they didn't use the API to begin with.
How do I know all this? I'm the guy at eBay responsible for helping software developers write applications that use the eBay API. If you're interested in writing an application that uses eBay, email me at jmcmanus@ebay.com or see our Web site at http://developer.ebay.com. It's free to get started, now, so there's no excuse not to give it a shot.
I'll tell you what the bad idea is. It's eBay changing its HTML every two weeks in an attempt to foil its customers.
Companies like eBay sometimes let their success go to their heads and think they can afford to fight their own customers. Not so. It's a terrible idea from a long term business standpoint. I know dozens of eBay sellers and I can attest to the huge amount of ill will eBay has stirred up with its own sellers.
The API is not limited to Windows; it (and the scripts in eBay Hacks) can be used with Mac OS X, Linux, Unix, or any other platform... all you need is an internet connection.
Note that this particular hack is as much about showing the value of the API as it is about showing alternatives to the API.
