instagrab

A Rust CLI that scrapes Instagram profile metadata (full name, biography, follower / following / post counts, recent posts with captions) by attaching to a real Chrome process over the Chrome DevTools Protocol.

Designed to run on a server you own under cron, alongside a long-lived headless Chrome whose user-data-dir holds an authenticated IG session.

It can scan a fixed list of profiles, or fan out over everyone a seed profile follows (re-fetched on demand), writing one JSONL record + the first image per post into a directory.

Instagram's ToS prohibits automated scraping. Use at your own account risk. The defaults (daily cadence, jittered inter-profile sleeps, single account) are conservative for personal-scale use.

Getting started

You'll need a recent Rust toolchain and Chrome.

# 1. Launch Chrome with a dedicated dev profile and sign in to IG manually:
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
  --remote-debugging-port=9222 \
  --user-data-dir="$HOME/.instagrab/CDPProfile"

# 2. Build and write a sample config, then edit it
# (at minimum set output_path and seed_username):
cargo build
./target/debug/instagrab --write-sample-config ./config.toml

# 3. Fetch your follows list (merges into <output_path>/friends.txt):
./target/debug/instagrab --config ./config.toml --fetch-follows

# 4. Scan everyone on that list (writes <output_path>/runs.jsonl + images):
./target/debug/instagrab --config ./config.toml
tail -1 <output_path>/runs.jsonl | jq .

instagrab does two things:

• --fetch-follows merges your seed profile's Following into a follows file you can edit (run it rarely).
• The default run scans everyone in that file for recent posts + first image (run it daily).

For more information, see Friends fan-out and Configuration.

Output shape

A successful run emits one JSON object per username:

{
  "scraped_at": "2026-07-24T02:17:10.765462Z",
  "username": "gjtorikian",
  "user_id": "1018933991",
  "source": "graphql",
  "full_name": "Garen J Torikian",
  "biography": "I inhale and exhale.",
  "is_private": false,
  "followers": 654,
  "following": 577,
  "posts": 497,
  "recent_posts": [
    {
      "shortcode": "DXztvjcCTy8",
      "url": "https://www.instagram.com/p/DXztvjcCTy8/",
      "caption": "Fausto is SEVENTEEN today! This is the best photo he would let me take!!!",
      "taken_at_unix": 1777664290,
      "display_url": "https://scontent-lga3-3.cdninstagram.com/...",
      "local_path": "./images/gjtorikian/DXztvjcCTy8.jpg"
    }
  ],
  "window_days": 90
}

source is "graphql" (the only successful kind), "not_found", "logged_out", or "error". Per-field problems land in the errors array.

window_maybe_truncated: true means paging stopped before reaching the oldest post in the window; bump max_scrolls if you need to go further.

A post may also carry media_count, is_carousel, and has_video (present only when it's a carousel or a video). Due to rate limiting, only the first image is downloaded; one can link out to Instagram for the rest.

Friends fan-out

instagrab's main mode scans everyone a seed profile follows, re-fetching that list on demand.

instagrab --fetch-follows pages seed_username's Following once, at a human pace, and merges the handles into the follows file (<output_path>/friends.txt). This is the only path that touches Instagram's friendship endpoint (isolated in src/follows.rs); it is off by default and detection-sensitive, so run it rarely. It prints progress (fetching follows... N so far) and a summary of what the merge changed. --seed <name> overrides seed_username ad hoc.

The follows file is yours to edit

The harvest merges; it does not overwrite. Your edits survive it, so the file — not Instagram — is the durable record of what gets scraped.

There's one username per line, and a commented-out username is an exclusion: still followed, just never scanned.

# accounts I follow but don't want on the wall
anniebannie
# nononancy           <- excluded; the harvest leaves this comment alone
berryp

Across a --fetch-follows:

Blank lines and multi-word comments are preserved exactly where they sit. A comment holding a lone username-shaped word is read as an exclusion, so keep header comments to more than one word — # archived would be mistaken for a handle, # archived accounts would not.

Deletions are withheld whenever the harvest might be short. Absence from a truncated list isn't evidence of an unfollow, and one rate-limited page would otherwise prune most of the file. The merge stays additive when the fetch errored (e.g. an HTTP 429 mid-pagination) or when a clean harvest came back with under half the entries already on file; either way it emits a follows_partial alert and deletes nothing.

Commands

Configuration

All settings live in the TOML file passed with --config (generate a starter with --write-sample-config). output_path (an output directory) is required; everything else has a default.

The canary scrape (against instagram.com/instagram) has no knob — it always runs first and exits 5 if extraction looks broken.

Alerts

Out-of-band conditions surface as event: "alert" lines and a non-zero exit:

logged_out means: re-run the SSH-tunnel login bootstrap (see deploy/README.md). schema_drift means: IG renamed/removed a JSON path; update EXPECTED_PROFILE_PATHS and the parser in src/parse.rs. follows_partial is not a failure — the follows file is intact and simply a merge behind. It exits 0 (or 4, if the fetch returned nothing at all).

Deploying

See deploy/README.md for a full deployment walkthrough, including the SSH-tunnel login bootstrap that mints cookies (so IG's first-login flag fires once, during setup, not during cron).

How it works

1. A persistent --headless=new Chrome runs on the host with --remote-debugging-port=9222 --user-data-dir=…/CDPProfile. You log in to Instagram once into that profile.
2. instagrab connects to that Chrome via CDP, navigates each profile in its own tab, then actively fetches /api/v1/users/web_profile_info/ from inside the page (with the X-IG-App-ID header IG's web app sends). Passively listening for that endpoint isn't reliable — the logged-in React shell doesn't always trigger it.
3. Posts come from the mobile-API user-feed endpoint (/api/v1/feed/user/<id>/), fetched from inside the page and paged via max_id. If time_window_days > 0, instagrab keeps paging until the oldest captured post is older than the window.
4. Each kept post's display_url is downloaded over plain HTTPS to <output_path>/images/<username>/<shortcode>.jpg (idempotent — existing files are kept).
5. One JSONL line per username is appended to <output_path>/runs.jsonl.

License

MIT