Python project: Build a Google-like web-crawler
Web crawling is used by many companies for lead generation, competitor pricing analysis and market research and is a must-have skill in a developer or data scientist.
In this project, I will guide you to build a Google-like web-crawler on your own, which starts from a URL and crawls every link within it.
Get a certificate from my company once you complete the project.
- Build a web-crawler on your own using Python
- Step-by-step guidance on tasks to be done
- Curated learning material provided
- Add Project certificate to your CV
What you'll learn
HTTP GET request basics
How to do a web request in Python
How to parse HTML using BeautifulSoup
How to save a file on disk using Python
How to save data to a database
How to query the database
How to use threading to optimize a process
Web crawling gotchas
I'm Ananth SNC. I started my software development journey as a freelance programmer 10 years ago, while I was studying at IIT Kharagpur. During this time, I have learnt many programming languages, built web and Mobile apps for 200+ companies and founded 3 companies on my own.
Who should apply
Anyone looking to use programming knowedge to build a real project that adds value to your CV
Basic programming knowledge