Prepare development environment
Install required libraries
Download the web driver you want and place that driver to your PATH (i.e. /usr/bin, /usr/local/bin for linux and MacOS, $PATH for Windows)
Import required libraries
If we use the “requests” library, we only get the source code from the website, but not the generated content.
Then we can pass the rendered content to Beautiful Soup object and scrap data we want
If you would like to learn Beautiful Soup and Selenium, you can find their documentations below or leave a comment here.
Both of them are powerful libraries for web scrapping and browser automation projects.
Beautiful Soup: https://www.crummy.com/software/BeautifulSoup/bs4/doc/