Batch jobs

POST /scrape/batch queues a set of scrapes and returns a job id. Use it when you have a list of URLs and do not need each result the moment it is ready.

curl -X POST 'https://api.helicitylabs.com/scrape/batch' \
  -H 'Content-Type: application/json' \
  -H 'X-API-Key: YOUR_API_KEY' \
  -d '{
    "urls": ["https://example.com/a", "https://example.com/b"],
    "render_js": false,
    "proxy_type": "datacenter"
  }'

The scrape options apply to every URL in the batch — one shape per job. URLs needing different settings go in different jobs.

Following a job

curl 'https://api.helicitylabs.com/jobs/{job_id}' \
  -H 'X-API-Key: YOUR_API_KEY'

Poll it until it reports finished, then fetch the output:

curl -L 'https://api.helicitylabs.com/jobs/{job_id}/result' \
  -H 'X-API-Key: YOUR_API_KEY' \
  -o results.zip

The result is a zip of per-URL responses, each in the same shape a single /scrape returns — so ok, error_class and blocked mean exactly what they mean elsewhere.

A batch is a set of individual runs for billing. Each URL is charged by the same table in Credits, and the ones that fail on our side are still free.

Every URL in a job shares one proxy_type and one render_js. A list mixing pages that need a browser with pages that do not will either overpay for the cheap ones or fail on the expensive ones. Split the list.