Web Scraping Tutorials

LEARN HOW TO USE WEB SCRAPING TO ENHANCE PRODUCTIVITY AND AUTOMATION

We provide many step-by-step tutorials with source code for web scraping, web crawling, data extraction, headless browsers, etc.

Our web scraping tutorials are usually written in Python using libraries such as LXML, Beautiful Soup, Selectorlib and occasionally in Node.js.

The full source code is also available to download in most cases or available to be easily cloned using Git.

We also provide various in-depth articles about Web Scraping tips, techniques and the latest technologies which include the latest anti-bot technologies, methods used to safely and responsibly gather publicly available data from the Internet.

The community that has coalesced around these tutorials and their comments help anyone from a beginner hobbyist person to an advanced programmer solve some of the issues they face with web scraping.

These tutorials are frequently linked to as StackOverflow solutions and discussed on Reddit.

Please feel free to read and participate in the discussions with your comments.

All Tutorials

How To Install Python Packages for Web Scraping in Windows 10

Web scraping using Python in Windows can be tough. In this tutorial follow the steps to setup python 3 and python packages on your Windows 10 computer for web scraping in Windows 10.

How to Scrape Fandango using Python and LXML

Learn how to scrape movie details from Fandango.com, a movie booking site using Python and LXML in this web scraping tutorial. We will show you how to extract movie details such as movie theatres playing, location, movie name, rating, genre and more from a particular zip code/city and date.

How to Scrape Store Locations from Target.com using Python

You can get a lot of information on store locations. This tutorial will show you how to extract store details such as store timings, address, latitude and longitude and more from Target.com using Python and LXML.

How to Solve Simple Captchas using Python Tesseract

CAPTCHA stands for Completely Automated Public Turing test to tell Computers and Humans Apart. As the acronym suggests, it is a test used to determine whether the user is human or not. A typical captcha consists of a distorted test, which a computer program cannot interpret but a human can (hopefully) still read. This tutorial will […]

How to Parse Addresses using Python and Google GeoCoding API

Web scraping can often lead to you having scraped address data which are unstructured. If you have come across a large number of freeform address as a single string, for example – “9 Downing St Westminster London SW1A, UK”, you know how hard it would be to validate, compare and deduplicate these addresses. To start […]

How to Scrape Expedia using Python and LXML

Learn how to scrape flight details from Expedia.com, a leading travel and hotel site, using Python 3 and LXML in this web scraping tutorial. You’ll learn how to extract flight details such as flight timings, plane names, flight duration and more for a given source and destination.