Status: [LIVE] Endpoints found: https://api.quanticdata.io/v1/scrape Source: https://github.com/quantumproxies/web-scraper-api # Web Scraper API [![ci](https://github.com/quantumproxies/web-scraper-api/actions/workflows/ci.yml/badge.svg)](https://github.com/quantumproxies/web-scraper-api/actions/workflows/ci.yml) ![python](https://img.shields.io/badge/python-3.9%2B-blue) ![license](https://img.shields.io/badge/license-MIT-green) ![dependencies](https://img.shields.io/badge/dependencies-none-brightgreen) Three jobs every scraping project starts with, from Python or the command line: read one page as clean Markdown (or HTML or text), list every URL a site exposes without crawling it, and crawl a site into Markdown pages ready for an LLM or a RAG index. The code is free and open source (MIT). It runs on the QuanticData Data API, so there are no proxies to rotate and no headless browser to keep alive: sign in once and run a command. ```bash pip install git+https://github.com/quantumproxies/web-scraper-api web-scraper login web-scraper scrape https://en.wikipedia.org/wiki/Web_scraping --content-mode article -o page.md ``` `login` opens the QuanticData sign-in page in your browser. New accounts get $2 of free usage every month, no card required: [create one](https://app.quanticdata.io/register?utm_source=github&utm_medium=readme&utm_campaign=web-scraper-api). ## What it scrapes | Command | Input | Returns | Per run | |---|---|---|---| | `web-scraper scrape` | URL | Markdown content, title, status, canonical, language, size, 11 fields per row ([docs and price](https://quanticdata.io/web-scraping-api/?utm_source=github&utm_medium=readme&utm_campaign=web-scraper-api)) | one page per call | | `web-scraper map` | site URL | URL list, site total, count per section, 1 fields per row ([docs and price](https://quanticdata.io/crawl-map/?utm_source=github&utm_medium=readme&utm_campaign=web-scraper-api)) | up to 5,000 URLs | | `web-scraper crawl` | start URL | URL, status, title, depth, Markdown content, 5 fields per row ([docs and price](https://quanticdata.io/crawl-map/?utm_source=github&utm_medium=r