flq 1.0.0

# flq

Fetch the web as lean, token-efficient text. `HTML` → `.flq`

Raw HTML wastes 70-90% of LLM tokens on markup. flq strips a page down to
its content — headings, text, lists, tables, code — in a format that
tokenizes tight and reads clean.

## Install

```sh
cargo install flq
```

## Usage

```sh
flq https://example.com                          # fetch one page
flq https://example.com/1 https://example.com/2  # multiple (=== separated)
curl -s https://example.com | flq -              # HTML from stdin
flq page.html                                    # local file
```

### curl-compatible flags

```sh
flq URL -o page.flq                    # output to file
flq URL -H "Authorization: Bearer x"   # custom header
flq URL -A "my-agent/1.0"              # user-agent
flq URL -m 10                          # timeout (seconds; k/m suffix ok, e.g. 1k)
flq URL -b "session=abc"               # cookies
flq URL -I                             # response headers only
flq URL -s                             # silent
```

### Extraction

```sh
flq URL -f headings                    # headings only
flq URL -f code                        # code blocks only
flq URL -f tables,links                # combine filters
flq URL -g "install"                   # grep body lines
flq URL --select "article.post"        # scope by CSS selector
flq URL --raw                          # raw HTML passthrough
```

Filters: `headings` `code` `tables` `links` `img` `text`

### Agent workflow

```sh
flq URL --max-tokens 2k                # token budget (k/m suffix), truncates cleanly
flq URL --no-meta                      # body only, no frontmatter
flq URL -j                             # JSON: {title,url,body,tokens,truncated}
cat urls.txt | flq --pipe              # NDJSON stream, one page per line
flq URL --depth 1 --max-pages 10       # crawl same-host links
flq URL --cache                        # cache locally (~/.cache/flq)
flq URL --cache --cache-ttl 86400      # custom TTL
flq URL --diff                         # diff against cached version
```

## Format

```
title: Page Title
url: https://example.com
lang: ja
desc: meta description
---
# heading
paragraph text
- list item
  - nested item
> quote
@ https://link | link text
! https://img.png | alt text
table[3]{name,price,stock}:
Widget A,1200,45
Widget B,800,120
Gadget C,500,300
```

Prose stays markdown — LLMs parse it natively and every marker is one
token. Tables use schema-once TOON-style headers: field names are
declared once, rows stream as CSV. Links (`@`) and images (`!`) are
off by default; enable with `-f links` / `-f img`.

## JSON output

```sh
flq https://example.com -j
```

```json
{
  "title": "Page Title",
  "url": "https://example.com",
  "body": "# heading\nparagraph...",
  "tokens": 142,
  "truncated": false
}
```

`tokens` is an estimate (~4 ASCII chars or ~1.3 CJK chars per token).

## As a library

```rust
use flq::{Opts, process};

let opts = Opts {
    max_tokens: Some(2000),
    no_meta: true,
    ..Default::default()
};
let page = process("https://example.com", &opts)?;
println!("{} ({} tokens)", page.body, page.tokens);
```

## MCP server

flq doubles as a [Model Context Protocol](https://modelcontextprotocol.io)
server, so agents like Claude Code can fetch and parse the web directly.

```sh
flq --mcp                               # run as MCP server (stdio JSON-RPC)
claude mcp add flq -- flq --mcp         # register with Claude Code
```

Tools: `fetch_page`, `fetch_pages`, `crawl_site`, `head`. See
[docs/mcp.md](docs/mcp.md) for the full tool reference and setup, and
[skills/flq/SKILL.md](skills/flq/SKILL.md) for an agent skill that teaches when
and how to reach for flq.

## License

MIT