Scraping

Sessions and logins

Reuse a browser session, or let us hold the login for you.

Three different things look similar and are not:

What it pinsField
Sticky proxy sessionThe exit IPproxy_session
Stored sessionCookies and storage from a previous runsession_id
Managed accountA login we keep alive for youaccount_id

Stored sessions

Pass a session_id to reuse state from an earlier run under the same id.

{
  "url": "https://example.com/account",
  "proxy_type": "premium",
  "session_id": "acme-user-4417"
}

The two engines use it differently: the browser engine restores the full storage state, the HTTP engine reuses the saved cookies only. The id must be 1–128 characters of [A-Za-z0-9._-].

Inspect or drop one with GET /sessions/{session_id} and DELETE /sessions/{session_id}.

A session can only serve one in-flight request at a time. A second concurrent request against the same session returns session_busy (HTTP 409). Serialise per session, or use more than one.

Managed accounts

An account is a credential we store and log in with, so you never put a password in a scrape request.

curl -X POST 'https://api.helicitylabs.com/accounts' \
  -H 'Authorization: Bearer YOUR_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{ "label": "acme supplier portal", "...": "see GET /accounts/templates" }'

GET /accounts/templates lists the login shapes we already know. Then scrape with the account instead of a session:

{
  "url": "https://supplier.example.com/orders",
  "proxy_type": "premium",
  "account_id": "acct_8f21c0"
}

account_id and session_id are mutually exclusive — sending both is a bad_request. We pick a healthy session belonging to that account and use it.

Two errors are specific to accounts:

  • account_unavailable (409) — the account has no healthy logged-in session. Something needs attention: the password changed, or the site added a step. Check POST /accounts/{account_id}/login.
  • account_busy (409) — every healthy session is in flight. Retry shortly, or add capacity.

Scraping behind a login means handling someone’s credentials. Only connect accounts whose owner has agreed to it, and read Errors for what we return when a login goes stale rather than silently serving you a sign-in page.