Scrapeghost
lastmod 2023-03-30created 2023-03-301 edits
A web scraping library using GPT
Features
- An experimental library that extracts data from HTML without writing any page-specific code
- Just specify the shape of the data you want to extract and pass a URL, and it extracts it on its own
- Internally handles HTML cleaning, splitting, etc. to cut costs, and also performs a Hallucination check to verify that the extracted data is actually an accurate value present on the page
- Provides a budget setting for GPT API call costs