Learn Web Scraping with Python

How Do You Scrape Data from A Website

How Do You Scrape Data from A Website?

When you run the code for web scraping, a request is sent to the URL that you have mentioned. As a response to the request, the server sends the data and allows you to read the HTML or XML page. The code then, parses the HTML or XML page, finds the data and extracts it. 


To extract data using web scraping with python, you need to follow these basic steps:

  1. Find the URL that you want to scrape
  2. Inspecting the Page
  3. Find the data you want to extract
  4. Write the code
  5. Run the code and extract the data
  6. Store the data in the required format 


Now let us see how to extract data from the Flipkart website using Python.

Libraries used for Web Scraping 

As we know, Python is has various applications and there are different libraries for different purposes.

In our further demonstration, we will be using the following libraries:

  • Selenium:  Selenium is a web testing library. It is used to automate browser activities.
  • BeautifulSoup: Beautiful Soup is a Python package for parsing HTML and XML documents. It creates parse trees that is helpful to extract the data easily.
  • Pandas: Pandas is a library used for data manipulation and analysis. It is used to extract the data and store it in the desired format.