R Web Scraping Quick Start Guide

R Web Scraping Quick Start Guide

by Olgun Aydin
Released October 2018
Publisher(s): Packt Publishing
ISBN: 9781789138733

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Web Scraping techniques are getting more popular, since data is as valuable as oil in 21st century. Through this book get some key knowledge about using XPath, regEX; web scraping libraries for R like rvest and RSelenium technologies.

Key Features

  • Techniques, tools and frameworks for web scraping with R
  • Scrape data effortlessly from a variety of websites
  • Learn how to selectively choose the data to scrape, and build your dataset

Book Description

Web scraping is a technique to extract data from websites. It simulates the behavior of a website user to turn the website itself into a web service to retrieve or introduce new data. This book gives you all you need to get started with scraping web pages using R programming.

You will learn about the rules of RegEx and Xpath, key components for scraping website data. We will show you web scraping techniques, methodologies, and frameworks. With this book's guidance, you will become comfortable with the tools to write and test RegEx and XPath rules.

We will focus on examples of dynamic websites for scraping data and how to implement the techniques learned. You will learn how to collect URLs and then create XPath rules for your first web scraping script using rvest library. From the data you collect, you will be able to calculate the statistics and create R plots to visualize them.

Finally, you will discover how to use Selenium drivers with R for more sophisticated scraping. You will create AWS instances and use R to connect a PostgreSQL database hosted on AWS. By the end of the book, you will be sufficiently confident to create end-to-end web scraping systems using R.

What you will learn

  • Write and create regEX rules
  • Write XPath rules to query your data
  • Learn how web scraping methods work
  • Use rvest to crawl web pages
  • Store data retrieved from the web
  • Learn the key uses of Rselenium to scrape data

Who this book is for

This book is for R programmers who want to get started quickly with web scraping, as well as data analysts who want to learn scraping using R. Basic knowledge of R is all you need to get started with this book.

Table of contents

  1. Title Page
  2. Copyright and Credits
    1. R Web Scraping Quick Start Guide
  3. Dedication
  4. Packt Upsell
    1. Why subscribe?
    2. Packt.com
  5. Contributors
    1. About the author
    2. About the reviewer
    3. Packt is searching for authors like you
  6. Preface
    1. Who this book is for
    2. What this book covers
    3. To get the most out of this book
      1. Download the example code files
      2. Download the color images
      3. Conventions used
    4. Get in touch
      1. Reviews
  7. Introduction to Web Scraping
    1. Learning about data on the internet
    2. Introduction to XPath (XML Path)
    3. Data extraction systems
    4. Web scraping techniques 
      1.  Traditional copy and paste
      2. Text grabbing and regular expression
      3. Document Object Model (DOM)
      4. Semantic annotation recognition
      5. Web scraping tools
        1. JavaScript tools
        2. Web crawling frameworks
      6. Web crawling environment in R
    5. Summary
  8. XML Path Language and Regular Expression Language
    1. XML Path (XPath)
      1. Nodes
      2. Relationships between nodes
        1. Parent
        2. Child
        3. Sibling
        4. Ancestor
        5. Descendant
      3. Predicates
      4. Selecting unknown nodes
      5. Selecting several paths
    2. Regular expression language (Regex)
      1. How to match a single character 
      2. How to match the characters of a set 
      3. How to match words 
    3. Exercises on RegEx and XPath
      1. RegEx exercises
      2. XPath exercises
    4. Summary
  9. Web Scraping with rvest
    1. Introducing rvest 
    2. Step-by-step web scraping with rvest
      1. Writing XPath rules
      2. Writing your first scraping script
      3. Playing with data
    3. Summary
  10. Web Scraping with Rselenium
    1. Advantages and disadvantages of using Selenium for web scraping
    2. RSelenium
    3. Step-by-step web scraping with RSelenium
      1. Collecting data with RSelenium
    4. Summary
  11. Storing Data and Creating Cronjob
    1. Cloud engine models
      1. Infrastructure as a service (IaaS)
      2. Platform as a service (PaaS)
      3. Software as a service (SaaS)
      4. Mobile backend as a service (MBaaS)
      5. Function as a service (FaaS)
    2. Some of the cloud services
      1. Amazon Web Services (AWS)
      2. Google Cloud
    3. Cronjob
    4. Storing data and creating schedule jobs for web scraping
      1. Creating an AWS RDS Instance
      2. Connecting to the PostgreSQL database on AWS
      3. Creating cronjob
    5.  Summary
  12. Other Books You May Enjoy
    1. Leave a review - let other readers know what you think

Product information

  • Title: R Web Scraping Quick Start Guide
  • Author(s): Olgun Aydin
  • Release date: October 2018
  • Publisher(s): Packt Publishing
  • ISBN: 9781789138733