//! A stateless onetaskgraph source over one GitHub Projects v2 board.
//!
//! **A board is a container of projects, not a project.** Its own `title`,
//! `shortDescription` and `readme` are never read as an item's fields and are never
//! written: nothing in this source can rename the board a user configured.
//!
//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
//! board has none, so a write without [`GitHubProjectsConfig::repository`] is refused
//! naming the field — but that repository is the *fallback*, not the home of every item.
//!
//! <!-- llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] The rule's one
//! executable source is `GitHubProjectsSource::creation_target`; this is where a reader of
//! the module meets it, and `tests/plugin.rs` drives every arm below against the loopback
//! board and asserts on `createIssue`'s own `repositoryId`, so the prose cannot outlive a
//! change to the rule. -->
//! **Which repository an issue is created in is decided by the item's own `repositories`
//! field, under one rule.** Exactly one entry names the repository the issue is created in:
//! a task issue is where a person finds the work from the repository it changes, and one
//! filed in a board's nominated repository is invisible from every other. Zero entries, or
//! two or more, name none, so a task's or a document's issue is created in the repository
//! its parent project's issue lives in — read from the board, or from this process's own
//! record of a project it created earlier in the same command — and a project's issue, or
//! a task or document written with no parent, is created in the configured `repository:`.
//! What that rule refuses, it refuses before `createIssue`, so no issue is half-created. An
//! existing issue is never moved: the update path leaves the issue where it is and records
//! the list in the metadata slot when it differs, so the read side's derivation and the
//! creation rule agree by construction.
//! <!-- llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate] -->
//!
//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
//! board has no document type and nothing but issues to hold one in, so the title is the
//! discriminator and it is the whole of it. The title this source *reports* is the one a
//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
//! the body so `content` is what the person wrote — and writing a document puts the prefix
//! back, so a round trip returns the title that went in.
//!
//! **Telling a document from a project from a task.** The design prefix is read **first**:
//! a document is never a project and never a task, whatever sub-issues it has or does not
//! have. Only then does the rest apply — a board issue is a project when *either* it has
//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
//! necessary: it is what makes an *empty* project — the state a project copy passes
//! through between creating the project and filing its first task — readable as a
//! project, while the sub-issue arm lets a person author a project on the board by hand
//! with no knowledge of this product's metadata at all. Reading the prefix later than the
//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
//! task nor a document and are ignored.
//!
//! **A task's comments are its issue's comments.** They are read off `Issue.comments` and
//! written with `addComment`, `updateIssueComment` and `deleteIssueComment`, and a comment's
//! id is GitHub's own node id for the `IssueComment`. Two things GitHub decides are refused
//! rather than papered over: a board **draft** is not an issue and has no comments at all, so
//! a comment call on one is refused rather than answered with an empty page; and GitHub signs
//! every comment as the account the token belongs to, so a comment handed an author of its
//! own is refused rather than posted under another name. GitHub's comment mutations take the
//! comment's id and nothing else, so an edit or a delete first reads which issue that comment
//! is on, and a comment on some other issue is one this task does not have.
//!
//! **Where an entity is, is a link.** Every project, task and document this source reports
//! carries a [`Location::Url`] naming the issue's own web address — the same address the
//! `url` field already reports, in the shape that says a reader can open it. That is the
//! contrast the location contract exists for: a reader holding an entity from this source
//! is handed something to link to and one holding an entity from a folder of Markdown is
//! handed a path, and neither has to know which plugin answered. It does not replace or
//! derive from `url`; that field goes on reporting what it always reported.
//!
//! **Where metadata lives.** Short typed things go to typed fields and native relations:
//! status to the board's `Status` single-select and the issue's own state, the copy
//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
//! why neither can hold a caller's own prose. Setting one caller key on its own — on a task,
//! a project or a document alike — is one update of the issue body that changes that slot
//! and not one byte outside it, and it is not sent at all when the key already holds the
//! value.
//!
// llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] This public module documentation is a required user-facing description; the loopback plugin tests and shared live journey drive StatusMapping resolution, both mutations, and observed read-back together.
//! **Status.** `status_mapping` is per-instance configuration from a status category to
//! `null` or a board `Status` option name. `done` selects its mapped option and closes the
//! issue as `COMPLETED`; `cancelled` selects its mapped option and closes it as
//! `NOT_PLANNED`. Every open category reopens a closed issue before selecting its option.
//! A missing mapped option refuses the write before either representation changes. Reads
//! give a closed issue's reason precedence over its option, while an open issue's option
//! decides its category. The guarded [`GitHubProjectsSource::status_options`] operation is
//! the one path here that calls `updateProjectV2Field`: GitHub replaces the whole option
//! list, so it preserves every existing option id and verifies the field and item
//! assignments immediately afterwards. It counts a terminal category's mapped option as
//! configured, because a terminal write refuses without it. No ordinary source read or
//! write calls that mutation, whose
//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
//! and a mistake destroys every item's status. A status this board cannot represent is a
//! refusal naming the status and the instance instead.
//!
//! `unknown` is disabled by default because this source cannot preserve an open-ended
//! status word: it writes an existing board option and never
//! creates an option. An operator may map `unknown` to one existing option, in which case
//! every unknown word lands on that option and reads back as `unknown` under the option's
//! name. This differs from `local-md`, which writes and reads the original word itself.
//!
//! The shipped terminal options are exactly `done: Done` and `cancelled: Cancelled`.
//! `done` also closes the issue because GitHub derives `subIssuesSummary.completed`
//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
//! finished tasks were only moved to a "Done" column would read 0% complete forever.
// llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate]
//!
//! # What this source declares, field by field
//!
//! One verdict per field of [`Capabilities`], and what `Native` means when this source
//! says it. *Proven* means a shared journey drives it against the real
//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
//! [`capabilities`](TaskSource::capabilities) from parting.
//!
//! | Field | Verdict |
//! | --- | --- |
//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
//! | `comments` | **Supported and proven,** over the task issue's own comment connection, oldest first and paged by GitHub's own cursor; added, edited and removed through GitHub's comment mutations, paced as every other mutation is. A draft item has no comments on GitHub and is refused, and so is an author, because GitHub records the signed-in account as every comment's author. |
//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
//! | `search_title` | **Supported and proven,** over `Issue.title`. |
//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
//!
//! Nothing here is unsupported. `documents` and `comments` are not predicates — they say this
//! source has documents and that its tasks have comments, both of which hold — and the three
//! facts behind the uniform `Native` on the
//! predicates beside it are recorded below rather than re-derived, because a reader who
//! takes `Native` to mean *the remote service filters* will read that uniformity as a
//! lie.
//!
//! First, the plugin contract defines `Support::Native` as *the source applies this
//! predicate itself*, and says nothing about where it applies it. What the declaration
//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
//! — so that the engine may push it down and apply nothing of its own.
//!
//! Second, this source can keep that promise for every predicate at no additional API
//! cost, because whichever of the three reads below answers a query has already read every
//! item that query could keep before it filters anything. Filtering those items is
//! in-process work over data already in hand.
//!
//! Third, no predicate but `projects` could be pushed into the API even if that were
//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
//! label set, a status column or a substring of a body, and its title qualifier matches
//! tokens where this source — and the local Markdown source beside it — match substrings,
//! so pushing a search down would silently *narrow* the answer. What a project filter has
//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
//! the issue for them is both cheaper and exact. So there is one predicate this source
//! applies by asking a narrower question, six it applies in process, and none it is unable
//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
//! source has already done, and declaring `projects` native while ignoring the filter
//! (which this source once did) silently returns another project's tasks, because the
//! engine trusts the declaration and applies nothing locally.
//!
//! # The three ways this source reaches an item, and what each costs
//!
//! A board read is charged for what its *nested* connections could return rather than for
//! what was asked, so one whole-board read costs the same whether the question was about
//! one project or about all of them. That is why a question about one project is never
//! answered by reading the board:
//!
//! | The question | What is sent | What it costs |
//! | --- | --- | --- |
//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
//! | which board item one issue is, past the page that came with it | [`graphql::ISSUE_BOARD_ITEMS`] — that issue's own `projectItems` | one issue's memberships |
//!
//! The board half of an issue — its board item's id, its `Status` option and this
//! source's origin text field — rides along on `Issue.projectItems` in the first three, so
//! an item reached any of those ways resolves through the same
//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
//! same status, the same labels and the same qualified id. That connection comes back a
//! *page* at a time, at `BOARD_ITEMS_PAGE_SIZE`, so the entry for this board is looked for
//! on the page in hand and — only if that page reports more of the connection — in the
//! last row's read of that one issue's memberships, resumed from the page's own cursor and
//! walked to exhaustion. An issue with no entry for *this* board is not this source's to
//! report, which is what keeps an id naming another repository's issue from being answered
//! as an item of this board; and because the page is where the search starts rather than
//! where it ends, that answer is one about a connection read to exhaustion and never about
//! an unread page. Nothing costs the extra read but an issue on more boards than a page
//! holds: an issue this board really does not hold reports no next page, so its
//! memberships are already exhausted where they arrived.
//!
//! **No document here selects the board's own `Labels` field, and nothing is lost by
//! that.** An item's labels are read from its content alone, wherever that content is
//! reached: the three documents above select `Issue.labels` on the fragment, and
//! [`graphql::BOARD`] selects the same connection on the `... on Issue` arm of its
//! `content`. A board's `Labels` field is not one anybody fills in: it is a built-in
//! `ProjectV2FieldType`, it is absent from `ProjectV2CustomFieldType` so no project can
//! create one, and `ProjectV2FieldValue` — the whole of what
//! `updateProjectV2ItemFieldValue` accepts — offers no way to write one. So GitHub derives
//! it from the content, for every content type it exists on, and there is nothing it can
//! hold that the content does not already say: for an `Issue` it *is* that issue's own
//! labels, so selecting it beside them unions a set with itself.
//!
//! **A draft loses nothing by that either**, which is the reasoning this paragraph once had
//! backwards. `DraftIssue` exposes no `labels` field, and by the three schema facts above
//! it cannot carry a board `Labels` value to be derived from one — so a draft has nothing
//! to select *and nothing to lose*, and reports no labels at all. A `PullRequest` item is
//! discarded by [`GitHubProjectsSource::resolve`] before labels are read. Both halves are
//! held to that by tests in `tests/plugin.rs`: the four ways an item is reached report one
//! label set, and that set is the fixture issue's own, by
//! `an_item_reports_the_same_labels_title_status_and_id_however_it_is_reached`; and a board
//! item whose content is a draft reports an empty set, by
//! `a_board_item_whose_content_is_a_draft_reports_no_labels_at_all`. The absence of the
//! selection is held over [`graphql::DOCUMENTS`] by
//! `no_document_selects_the_boards_own_labels_field`.
//!
//! The last row is still the board's own item connection, and deliberately: a **draft**
//! board item is not an issue, so no search and no node read can reach one, and the reads
//! that have to answer for the whole board are the ones whose cost is the board's size
//! anyway.
//!
//! **What a read may return is capped too, and that cap is on the document rather than on
//! the board.** GitHub limits the number of nodes **one query may return** to
//! [`NODE_COUNT_LIMIT`] and refuses a query above that before executing it: the answer is
//! an error naming the connection the count crossed at, not a slow or a partial result.
//! Every board this source reads is refused the same way, so no board is too big for these
//! documents and none is small enough to save one that is over.
//!
//! The count is arithmetic over the document's own text: each connection contributes the
//! `first:` it asks for, counts **multiply** down a nested path and **sum** across sibling
//! paths. Those are [GitHub's published rules][node-limits] and this workspace does not
//! restate them — `github-graphql-node-count` implements them, and
//! [`worst_case_node_count`] under [`largest_page_sizes`] is where every node count here
//! comes from. `every_document_this_source_sends_stays_under_githubs_node_limit`, in
//! `tests/node_count.rs`, recomputes every document in [`graphql::DOCUMENTS`] from that
//! same text on every run and fails naming any that reaches the limit — so a connection
//! added to a shared fragment is caught there rather than by GitHub.
//!
//! What decides those counts is the page sizes: [`MAX_PAGE_SIZE`] on the outer page,
//! `NESTED_PAGE_SIZE` on the connections hanging off one item, and
//! `BOARD_ITEMS_PAGE_SIZE` on the page of an issue's board memberships a read carries.
//! `$nestedFirst` is spent twice down one path of a board read, so that constant is
//! effectively squared there, which is why it is the one the limit is most sensitive to.
//! `BOARD_ITEMS_PAGE_SIZE` is small for a reason of its own, recorded beside it: what a
//! page of memberships misses is recovered by one further read rather than refused, so it
//! buys a bound every read pays for at the price of a request only a multi-board issue
//! pays.
//!
//! **`nodeCount` and `cost` are two numbers against two limits, and both are computed
//! offline here — per document, one document at a time.** `nodeCount` is the one above: the
//! most nodes one query may return, checked per query and bounded by [`NODE_COUNT_LIMIT`].
//! `cost` is rate-limit points, metered per hour across everything one credential does; it
//! is what the two limiters [`Limiter`] tells apart meter, and a document under
//! [`NODE_COUNT_LIMIT`] still says nothing about its price. [`worst_case_point_cost`] is
//! that second number, and `tests/point_cost.rs` pins every document in
//! [`graphql::DOCUMENTS`] at what it costs — there being no per-call point ceiling to hold
//! one under, the pin itself is the check. The credentialed lane reconciles both figures
//! against GitHub's own, off a probe it already sends.
//!
//! **What is pinned that way is a per-document price and never a session's.** The record in
//! `session-cost.md` measures the two quantities a whole session can be counted in offline —
//! **requests** and **worst-case nodes** — and neither is points. What one whole session
//! consumes of the hourly point allowance is observable only from a credentialed run's own
//! `x-ratelimit-*` headers, which is what [`accounting`] fills its per-budget figures from
//! and what `tests/live.rs` prints at the end of every run.
//!
//! [node-limits]: https://docs.github.com/en/graphql/overview/rate-limits-and-node-limits-for-the-graphql-api
//!
//! **Where a read-after-write guarantee comes from, since a search index cannot supply
//! one.** GitHub's issue search is eventually consistent and answers a write made moments
//! ago with the value from before it. Resolving a node id is not, so a read by id and a
//! project's own sub-issues are already current. What closes the gap for the search is
//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
//! what this process itself wrote, so an item created seconds ago is reported whether or
//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
//! and the record dies with the process.
//!
//! Filtering happens before paging, so a page of a filtered result is a page of the
//! survivors rather than the survivors of a page. Label and text matching answer the same
//! question the same way the local Markdown source's do, so one cross-source expectation
//! holds for both.
//!
//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
//! has one source, `capabilities`, and the note above is the reasoning behind it rather
//! than a second copy of it: without the three facts recorded here a reader takes the
//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
//! crate's own capabilities test, which pins every field of it against a fully spelled-out
//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
//! fails to compile there rather than going unasserted. -->
//! The fixture-server tests above run wherever this crate is selected; the credentialed
//! lane runs in the same required check, beside them, and can fail it — it verifies the
//! current schema, then drives every field of the table above against the real board. It builds its own fixture there — two projects, one task filed under each,
//! one filed under neither, a label on one of the three and a closed status on another —
//! because that shape is what tells an honoured predicate from an ignored one: a board
//! holding a single project answers a project filter the same way whether or not this
//! source applies it, which is exactly how the defect above went unseen.
//!
//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
//! nominated is what keeps a credentialed write lane off a board and a repository nobody
//! nominated; it never asks GitHub which project was updated most recently. Before it
//! starts, the lane also clears any item titled — and any repository label named — the way
//! it titles and names its own artifacts, which is self-healing after an interrupted run:
//! a process killed between its writes and its cleanup leaves artifacts the next run
//! removes.
//!
//! # What a session of requests costs, and where the report is
//!
//! This source records **every** request it sends into [`accounting::Accounting`], at
//! `send_once` — the one place a request leaves this crate, which is why a read path added
//! later is counted without anybody remembering to count it. That is the whole of what this
//! crate adds to the arrangement; [`accounting`] is where what a record carries, how a
//! session's spend is arrived at, and what it deliberately does not know are set out.
//!
//! What one whole session of the live journey costs, counted that way against this crate's
//! loopback fixture board, is written down in `session-cost.md` beside this crate — with the
//! reduction it came out of, and with what it does and does not say about rate-limit points.
//!
//! [`GitHubProjectsSource::accounting`] is the read: a snapshot to hold and compare, which
//! [`accounting::Session::report`] renders the session report from. It is on the ordinary
//! code path — no environment variable, no feature, no build configuration — because an
//! instrument nobody switches on measures nothing, and
//! [`Plugin::build_recording_into`] is how a caller making its own calls beside this
//! source's counts the whole session rather than this source's share. The credentialed lane
//! in `tests/live.rs` does exactly that, and prints the report at the end of every run,
//! passed or failed.
//!
//! **A live session refuses to start unless the account can afford it.** Before it does any
//! of the work it exists to do, the journey makes one request — `GET /rate_limit`, which
//! GitHub documents as not counting against the REST rate limit and which answers both of
//! its budgets at once — and starts only if, for each of them, what remains minus this
//! session's estimated cost is still at least
//! `onetaskgraph_live::RETAINED_BUFFER` — twenty per cent — of that budget's whole
//! allowance. A session that cannot **declines**: it did not run, so it is
//! neither a pass nor a failing assertion, and it says which budget was short, that budget's
//! limit, what remained, the estimate, the buffer and when it resets — then stops, without
//! waiting for the budget to come back. The estimate is derived offline from
//! `tests/fixtures/session-cost.txt` and a cost model stated in `tests/journey/budget.rs`,
//! which is also where the published rule that model rests on is cited; the accounting
//! above records the gate's own read like any other request, and
//! [`accounting::Session::report`] prints the estimate beside what the session really spent.
//!
//! **GitHub is the authority on both of its own numbers, and the credentialed lane goes and
//! asks it.** Everything above computes `nodeCount` and `cost` offline from a document's own
//! text, which is what lets it run on every platform and on a pull request from a fork with
//! no credential — and that is what actually stops a regression merging. But an offline
//! arithmetic can only ever agree with itself: if GitHub changes its rules, this workspace
//! goes on computing the old answer and nothing notices. So `tests/live.rs` reconciles them.
//! GitHub's schema exposes `rateLimit(dryRun: true)`, whose `nodeCount` is *"the maximum
//! number of nodes this query may return"* and whose `cost` is what that document would
//! spend, both for a document **without executing it**, and the lane asks it for every query
//! document this source sends, under the largest bindings this source sends, and fails when
//! GitHub's figure and [`worst_case_node_count`] or [`worst_case_point_cost`] disagree. A
//! mutation is skipped, because `rateLimit` is a field of `Query` and cannot be asked about
//! one; the offline pins still cover it. It records what those calls reported about the
//! account's own allowance, because whether asking is free is a thing to observe rather than
//! to assume. Two quantities, not one: [`NODE_COUNT_LIMIT`] bounds `nodeCount` per query,
//! and `cost` is metered against an hourly allowance the accounting above reads off a
//! credentialed run's own response headers.
//!
//! **GitHub has two rate limiters and this source is refused by both, so nothing here
//! treats them as one thing.** The primary budget is the hourly allowance `gh api
//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
//! one part of.
#![deny(missing_docs)]
use std::collections::BTreeMap;
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use chrono::{DateTime, Utc};
use onetaskgraph_plugin_api::{
Capabilities, Comment, CommentBody, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind,
DependencySupport, Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label,
LabelFilter, Location, MetadataKey, Metering, NativeId, NewComment, Page, PageRequest, Project,
ProjectFilter, ProjectQuery, Repository, SecretResolver, SourceError, SourceName, SourcePlugin,
Status, StatusCategory, Support, Task, TaskQuery, TaskRef, TaskSource, TextFields, TextQuery,
WriteSupport,
};
use reqwest::{Client, StatusCode, Url};
use schemars::{Schema, schema_for};
use secrecy::{ExposeSecret, SecretString};
use serde::{Deserialize, Serialize};
use serde_json::{Value, json};
pub mod accounting;
use accounting::Accounting;
/// The registry name for this plugin.
pub const KIND: &str = "github-projects";
/// GitHub's maximum connection page size.
pub const MAX_PAGE_SIZE: u32 = 100;
/// The most nodes any one document this source sends may be asked to return.
///
/// GitHub's own published per-query ceiling, taken from
/// [`github_graphql_node_count::NODE_LIMIT`] rather than written out again here, so this
/// workspace cannot hold a stale copy of somebody else's number. A query above it is
/// **refused before it is executed**, whoever is asking and whatever board they are
/// asking about — so this is a bound on the documents rather than a budget that runs out.
///
/// This is `nodeCount`, the maximum number of nodes *one query may return*. It is not
/// `cost`, the rate-limit points a call spends against an hourly allowance shared by
/// everything the credential does — two numbers against two limits, and this constant
/// bounds only the first. The second is computed offline too, per document:
/// [`worst_case_point_cost`], pinned for every document in [`graphql::DOCUMENTS`] by
/// `tests/point_cost.rs`, and reconciled against GitHub's own `cost` by the credentialed
/// lane. There is no constant like this one to hold a price under, because points are an
/// hourly allowance rather than a per-call bound.
///
/// Neither is a session's price. What `session-cost.md` records of a whole session is its
/// **requests** and its **worst-case nodes**; what a whole session spends in points is
/// reported only by a credentialed run's own `x-ratelimit-*` headers, through
/// [`accounting`]. The module section on the three ways this source reaches an item says how
/// the count is arrived at, and which of the page sizes below decide it.
pub const NODE_COUNT_LIMIT: u64 = github_graphql_node_count::NODE_LIMIT;
/// Nested connection size for the connections that hang off one item.
///
/// It multiplies through every document that reaches an item under a page — the count
/// rules multiply down a nested path — so it is the constant [`NODE_COUNT_LIMIT`] is most
/// sensitive to. `tests/node_count.rs` is what holds the pair together: it recomputes
/// every document under these constants and fails naming any that reaches the limit, so
/// raising this is caught there rather than by GitHub.
const NESTED_PAGE_SIZE: u32 = 50;
/// How many of one issue's board memberships are read when an issue is reached directly.
///
/// An issue reached through a search or through its own node id carries its board half in
/// `Issue.projectItems`, and only the entry for *this* board is read. This connection sits
/// under a page of issues, so every point of it multiplies through the whole document and
/// is paid for whether or not any issue is on a second board — which is why it is
/// deliberately far smaller than [`NESTED_PAGE_SIZE`].
///
/// **Three, because what a page misses is now recovered rather than refused**, and the
/// recovery is what the value is chosen against. An issue whose entry for this board sits
/// past this page costs one further request — [`graphql::ISSUE_BOARD_ITEMS`], resumed from
/// that page's own cursor — so the value trades a bound every read pays for a request only
/// a multi-board issue pays. At one, a deployment whose issues commonly sit on two or more
/// boards would pay that request *per issue*, which is order N against the one page per
/// hundred issues a read costs today. At three it is only reached by an issue on four or
/// more boards at once, which keeps the recovery path exceptional rather than routine for
/// a plausible deployment.
const BOARD_ITEMS_PAGE_SIZE: u32 = 3;
pub use github_graphql_node_count::{NodeCountError, Variables};
/// The largest value this source can bind to each page-size variable its documents name.
///
/// Every `first:` in [`graphql`] reads one of these three, and each is capped at the
/// constant above it wherever a caller's own limit could reach it — `$first` at
/// [`MAX_PAGE_SIZE`], `$nestedFirst` at `NESTED_PAGE_SIZE`, `$boardItems` at
/// `BOARD_ITEMS_PAGE_SIZE`. So this is the worst case a caller can drive this source to,
/// not one configuration of it, which is what makes a bound computed under it a bound on
/// every read.
pub fn largest_page_sizes() -> Variables {
Variables::from([
("first".to_owned(), MAX_PAGE_SIZE),
("nestedFirst".to_owned(), NESTED_PAGE_SIZE),
("boardItems".to_owned(), BOARD_ITEMS_PAGE_SIZE),
])
}
/// The most nodes `document` could be asked to return, by GitHub's published rules.
///
/// Computed offline from the document's own text under [`largest_page_sizes`] — no
/// network, no credential and no schema — by
/// [`github_graphql_node_count::node_count`], which is where the rules themselves live.
/// A document at or above [`NODE_COUNT_LIMIT`] is one GitHub refuses before executing, so
/// this is what a check holds every document in [`graphql::DOCUMENTS`] below.
///
/// # Errors
///
/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
/// no single operation, or binds a page size this source does not name — each of which is
/// a defect in the document rather than a number.
pub fn worst_case_node_count(document: &str) -> Result<u64, NodeCountError> {
node_count(document, &largest_page_sizes())
}
/// The most rate-limit points one call of `document` could spend, by GitHub's published
/// rules.
///
/// Computed offline from the document's own text under [`largest_page_sizes`] — no
/// network, no credential and no schema — by
/// [`github_graphql_node_count::point_cost`], which is where the rules themselves live.
/// This is `cost`, metered **per hour** against the allowance one credential shares across
/// everything it does; it is not `nodeCount`, which is [`worst_case_node_count`] and is
/// bounded per query by [`NODE_COUNT_LIMIT`]. There is no per-call ceiling to hold this
/// under, so what `tests/point_cost.rs` does with it is pin every document in
/// [`graphql::DOCUMENTS`] at what it costs, and the credentialed lane reconciles those
/// figures against GitHub's own reported `cost`.
///
/// # Errors
///
/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
/// no single operation, or binds a page size this source does not name — each of which is
/// a defect in the document rather than a number.
pub fn worst_case_point_cost(document: &str) -> Result<u64, NodeCountError> {
github_graphql_node_count::point_cost(document, &largest_page_sizes())
}
/// The most nodes `document` could be asked to return under `variables`.
///
/// [`worst_case_node_count`] is this under [`largest_page_sizes`], and the accounting in
/// [`accounting`] is this under the bindings one request really sent — one spelling of the
/// calculation, so a bound checked offline and a cost recorded at run time cannot come to
/// disagree. The rules themselves live in [`github_graphql_node_count::node_count`].
///
/// # Errors
///
/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
/// no single operation, or binds a page size `variables` does not name.
pub fn node_count(document: &str, variables: &Variables) -> Result<u64, NodeCountError> {
github_graphql_node_count::node_count(document, variables)
}
/// The issue-title prefix that makes a board issue a document.
///
/// A GitHub Projects board has no document type — it holds issues — so the discriminator
/// is the title, and this is the whole of it: an issue whose title begins with these bytes
/// is a document and every other issue is the task or project the sub-issue rule makes it.
///
/// It is spelled **once**, here, and read rather than restated everywhere else — including
/// by the shared journeys, which take it from this constant so a board fixture cannot
/// drift from what this source reads. `docs/metadata.md` records the two consequences that
/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
/// Exact GraphQL query documents issued by this plugin.
///
/// Keeping the production documents here lets the pinned-schema test validate the same
/// bytes that are sent to GitHub, rather than a test-only copy which could drift
/// independently. [`STATUS_OPTIONS_UPDATE`] is the sole document that may rewrite a board
/// field, and its guarded caller always supplies the complete existing option set with ids.
pub mod graphql {
/// The board half of one item: the field values every document here reads it from.
///
/// A macro for the same reason [`board_issue!`] below is one, a level further in. This
/// selection is needed by that fragment, by [`BOARD`] under the board's own `items`,
/// and by [`ISSUE_BOARD_ITEMS`] under a membership walk — and all three have to produce
/// *the same value*, because
/// [`GitHubProjectsSource::resolve`](super::GitHubProjectsSource) reads them through
/// one path. Three spellings of it is what would drift, so there is one.
///
/// The `Status` option and this source's own origin text field are the whole of it. It
/// selects no `ProjectV2ItemFieldLabelValue`: GitHub derives that field from the item's
/// content, so it holds nothing the content's own `labels` do not already say, and it
/// would sit a label connection two page sizes deep.
macro_rules! board_item_values {
() => {
r#"fieldValues(first:$nestedFirst){nodes{
... on ProjectV2ItemFieldSingleSelectValue{name field{
... on ProjectV2SingleSelectField{id name options{id name}}
}}
... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
}pageInfo{hasNextPage}}"#
};
}
/// Everything this source reads about one issue, wherever it reaches that issue.
///
/// A macro rather than a constant so the three documents below can `concat!` it: one
/// spelling of these fields is what makes an issue read through the board-scoped
/// search, through its own node id, and through its project's sub-issue relationship
/// resolve to *the same* item, which is the whole of what
/// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
///
/// `projectItems` is what carries the board half of an issue: the board item's own id
/// and the [`board_item_values!`] above — the `Status` option and this source's origin
/// text field — that a `ProjectV2.items` read used to carry. It is asked for on the
/// issue rather than on the board, which is what makes the cost of a read proportional
/// to what was asked for instead of to the board's size.
///
/// It carries a *page* of that connection, at `BOARD_ITEMS_PAGE_SIZE`, and its
/// `endCursor` is what [`ISSUE_BOARD_ITEMS`] resumes from when this board's entry is
/// not on that page: a page here is where the search for the entry starts rather than
/// where it ends.
///
/// It does **not** select the board's `Labels` field value, and that is the whole of
/// what keeps the three documents below under [`NODE_COUNT_LIMIT`](super::NODE_COUNT_LIMIT):
/// a label connection there sits under `fieldValues` under `projectItems` under a page
/// of issues, spending `$nestedFirst` twice down one path, and took
/// [`SEARCH_ISSUES`] and [`SUB_ISSUES`] to 2,556,100 nodes against a limit of 500,000.
/// No label is lost — this is a fragment `on Issue`, whose own `labels` are selected
/// above, and that connection is where every label this source reports comes from. No
/// document in this module selects the board field any longer, [`BOARD`] included; the
/// module documentation records why nothing it could have held is lost.
macro_rules! board_issue {
() => {
concat!(
r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
projectItems(first:$boardItems){nodes{id project{id number}
"#,
board_item_values!(),
r#"}pageInfo{hasNextPage endCursor}}}"#
)
};
}
/// Every issue of one board, found by a search scoped to that board.
///
/// This is how the projects a board holds are listed, and it selects no `items`
/// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
/// container walked page by page, so nothing nested inside a board item is paid for.
/// Which of the issues it returns is a project is then read off `parent` — GitHub
/// accepts `-has:parent` as a search qualifier and silently ignores it, so the
/// discriminator has to be applied to the field, which is a scalar on the issue and
/// costs nothing.
pub const SEARCH_ISSUES: &str = concat!(
r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
search(query:$search,type:$type,first:$first,after:$after){
pageInfo{hasNextPage endCursor}
nodes{__typename ...BoardIssue}
}
}"#,
board_issue!()
);
/// One issue by its own node id, which is what a qualified id names here.
///
/// Strongly consistent, unlike the search above: GitHub's issue search is an index and
/// answers a write made moments ago with the value from before it, and resolving a node
/// id does not.
pub const ISSUE: &str = concat!(
r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
node(id:$id){__typename ...BoardIssue}
}"#,
board_issue!()
);
/// One project's tasks: the sub-issues of the issue that project is.
///
/// The work this costs is the project's own size. Nothing about it grows as the board
/// gains projects, or as those projects gain tasks.
pub const SUB_ISSUES: &str = concat!(
r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
node(id:$id){__typename
... on Issue{subIssues(first:$first,after:$after){
pageInfo{hasNextPage endCursor}
nodes{__typename ...BoardIssue}
}}}
}"#,
board_issue!()
);
/// Reads the board's fields and one page of its items.
pub const BOARD: &str = concat!(
r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
owner:repositoryOwner(login:$owner){
... on ProjectV2Owner{projectV2(number:$number){...Board}}
}
} fragment Board on ProjectV2 { id title
fields(first:$nestedFirst){nodes{
... on ProjectV2SingleSelectField{__typename id name options{id name}}
... on ProjectV2Field{__typename id name}
}pageInfo{hasNextPage}}
items(first:$first,after:$after){nodes{id "#,
board_item_values!(),
r#" content{
... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
... on PullRequest{__typename id}
... on DraftIssue{__typename id title body createdAt updatedAt}
}} pageInfo{hasNextPage endCursor}}
}"#
);
/// One issue's board memberships alone, walked past the page a read of it carried.
///
/// The recovery read behind [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource):
/// every document above carries a *page* of `Issue.projectItems`, and an issue on more
/// boards than that page holds may have this board's entry past its end. This asks that
/// one issue for its memberships and nothing else — the caller already holds the issue —
/// so an answer of "this board does not hold it" is only ever given about a connection
/// read to exhaustion.
///
/// It selects the board item's id, its project number and the same
/// [`board_item_values!`] the fragment does, because what it produces is handed to the
/// very same resolver: an issue recovered this way reports the same title, the same
/// status, the same labels and the same qualified id as one whose entry was on the
/// page.
///
/// `$first` rather than `$boardItems`: this document reads one issue, so nothing
/// multiplies through it and the membership connection can be walked at
/// [`MAX_PAGE_SIZE`](super::MAX_PAGE_SIZE) — which is what keeps the recovery to one
/// further request for any issue a person really keeps.
pub const ISSUE_BOARD_ITEMS: &str = concat!(
r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!){
node(id:$id){
... on Issue{projectItems(first:$first,after:$after){
nodes{id project{id number}
"#,
board_item_values!(),
r#"}
pageInfo{hasNextPage endCursor}}}
}
}"#
);
/// Resolves the configured repository's node id, which creating an issue requires.
pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
/// Reads both dependency directions for one issue, with each far end's own kind — and
/// the issue's own body, which is where an edge to another source is recorded, so that
/// half of a dependency read needs no second read of the issue or of the board.
pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
... on Issue{body
blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
}}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
/// Creates one issue in the configured repository.
pub const CREATE_ISSUE: &str =
r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
/// Puts an existing issue on the configured board.
pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
/// Updates an issue's visible fields and its open or closed state in one call.
pub const UPDATE_ISSUE: &str =
r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
/// Updates an existing draft's user-visible fields.
pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
/// Updates a text or single-select value on one project item.
pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
/// Replaces a single-select field's options. Only the guarded status-options operation
/// may use this document, because GitHub treats the input as the complete option list.
pub const STATUS_OPTIONS_UPDATE: &str = r#"mutation($input:UpdateProjectV2FieldInput!){updateProjectV2Field(input:$input){projectV2Field{... on ProjectV2SingleSelectField{id options{id name color description}}}}}"#;
/// A fresh snapshot of the Status field and every board item's assignment.
pub const STATUS_OPTIONS_SNAPSHOT: &str = r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!){owner:repositoryOwner(login:$owner){... on ProjectV2Owner{projectV2(number:$number){id fields(first:$nestedFirst){nodes{... on ProjectV2SingleSelectField{id name options{id name color description}}}pageInfo{hasNextPage}} items(first:$first,after:$after){nodes{id fieldValues(first:$nestedFirst){nodes{... on ProjectV2ItemFieldSingleSelectValue{name optionId field{... on ProjectV2SingleSelectField{id name}}}}pageInfo{hasNextPage}}}pageInfo{hasNextPage endCursor}}}}}}"#;
/// Files one issue under another as a sub-issue, which is what project membership is.
pub const ADD_SUB_ISSUE: &str =
r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
/// Takes one issue back out of its parent.
pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
/// Adds GitHub's native issue blocked-by relationship.
pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
/// Removes one native issue blocked-by relationship.
pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
/// Deletes one issue, which takes its board item with it.
///
/// The engine sends this in one situation only: undoing a copy that could not finish,
/// over the items that same copy created. Deleting the issue removes the board item
/// too, so there is no second `deleteProjectV2Item` to keep in step with it.
pub const DELETE_ISSUE: &str =
r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
/// Everything this source reads about one issue comment, wherever it reaches one.
///
/// A macro for the reason [`board_issue!`] is one: a comment listed, a comment just added
/// and a comment just edited are handed to one mapper, so they are selected by one
/// spelling. `author` is `Actor`, which GitHub answers `null` for an account that no
/// longer exists, and `login` is the one member every kind of actor carries.
macro_rules! issue_comment {
() => {
"id author{login} createdAt updatedAt body url"
};
}
/// One task's comments: a page of its issue's own `comments` connection.
///
/// **No `orderBy`, and that is what makes the page oldest first.** GitHub's only
/// `IssueCommentOrder` field is `UPDATED_AT`, which would move a comment to the end of the
/// list every time somebody edited it; left unordered the connection answers in the order
/// the comments were written, which is the order GitHub documents for the same collection
/// over REST — ascending id. Nothing multiplies through it, so `$first` is the whole of its
/// node count and the caller's own page size is pushed straight down.
pub const ISSUE_COMMENTS: &str = concat!(
r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename ... on Issue{comments(first:$first,after:$after){nodes{"#,
issue_comment!(),
r#"}pageInfo{hasNextPage endCursor}}}}}"#
);
/// Which issue one comment is on, read before that comment is edited or removed.
///
/// GitHub's comment mutations take the comment's id and nothing else, so without this a
/// comment id given against the wrong task would change a comment on another issue.
pub const COMMENT_ISSUE: &str =
r#"query($id:ID!){node(id:$id){__typename ... on IssueComment{id issue{id}}}}"#;
/// Adds one comment to an issue, signed as the account the token belongs to.
pub const ADD_COMMENT: &str = concat!(
r#"mutation($input:AddCommentInput!){addComment(input:$input){subject{id} commentEdge{node{"#,
issue_comment!(),
r#"}}}}"#
);
/// Replaces the body of one issue comment.
pub const UPDATE_COMMENT: &str = concat!(
r#"mutation($input:UpdateIssueCommentInput!){updateIssueComment(input:$input){issueComment{"#,
issue_comment!(),
r#"}}}"#
);
/// Removes one issue comment. Its payload carries nothing about the comment it removed.
pub const DELETE_COMMENT: &str = r#"mutation($input:DeleteIssueCommentInput!){deleteIssueComment(input:$input){clientMutationId}}"#;
/// Every document above, with what this source is doing when it sends one.
///
/// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
/// name the call that was refused, and a `match` with a catch-all arm would answer a
/// document added later with "talking to GitHub" and never say so.
///
/// `documents_are_all_inventoried` reads this file back and fails naming any `pub
/// const` here that this list omits, so the two cannot part — which is the same guard
/// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
pub const DOCUMENTS: [(&str, &str); 24] = [
(SEARCH_ISSUES, "searching this board's issues"),
(ISSUE, "reading one issue"),
(
ISSUE_BOARD_ITEMS,
"reading one issue's board memberships past the page it came with",
),
(SUB_ISSUES, "reading a project's tasks"),
(BOARD, "reading the board"),
(REPOSITORY, "reading the destination repository"),
(ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
(CREATE_ISSUE, "creating an issue"),
(ADD_TO_BOARD, "adding an issue to the board"),
(UPDATE_ISSUE, "updating an issue"),
(UPDATE_DRAFT, "updating a draft item"),
(UPDATE_FIELD, "writing a board field"),
(
STATUS_OPTIONS_SNAPSHOT,
"snapshotting board Status options and assignments",
),
(
STATUS_OPTIONS_UPDATE,
"safely replacing the board Status option list",
),
(ADD_SUB_ISSUE, "filing an issue under its project"),
(REMOVE_SUB_ISSUE, "taking an issue out of its project"),
(ADD_BLOCKED_BY, "recording a dependency"),
(REMOVE_BLOCKED_BY, "removing a dependency"),
(DELETE_ISSUE, "deleting an issue"),
(ISSUE_COMMENTS, "reading a task's comments"),
(COMMENT_ISSUE, "reading which issue a comment is on"),
(ADD_COMMENT, "adding a comment"),
(UPDATE_COMMENT, "editing a comment"),
(DELETE_COMMENT, "deleting a comment"),
];
}
/// Which of GitHub's two rate limiters refused a request.
///
/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
/// secondary one — so an operator told the wrong one takes the wrong next step, which is
/// the whole reason this is carried rather than collapsed into "rate limited".
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Limiter {
/// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
Primary,
/// The burst limiter over content-generating requests, which nothing reports.
Secondary,
}
/// The wordings GitHub answers a secondary rate limit with.
///
/// It sends them under a forbidden status, under a too-many-requests status, and inside
/// the `errors` of a *successful* response, which is why the text is what this matches on
/// rather than the status. `abuse detection` is the wording GitHub used before the
/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
/// what a burst of content creation is refused with.
///
/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
/// when, and the drift gate reconciles the two lists both ways. Public for that gate
/// alone — a caller has no use for it, and matching on a refusal is this source's job.
pub const SECONDARY_WORDINGS: [&str; 5] = [
"secondary rate limit",
"temporarily blocked from content creation",
"abuse detection",
"submitted too quickly",
"exceeded a secondary",
];
/// The wordings GitHub answers an exhausted primary budget with.
///
/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
/// than looked for in the response text. Pinned and gated exactly as
/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
pub const PRIMARY_WORDINGS: [&str; 3] = [
"api rate limit exceeded",
"rate limit exceeded",
"rate_limited",
];
/// What a response *says about itself*, which is the only place a refusal can be read.
///
/// Deliberately not the whole response body. A board is a place people write about their
/// own work, and a task on it titled "the secondary rate limit" would, matched across the
/// raw text, turn a perfectly good answer into a refusal this source then waited out and
/// reported. So the item data is never read: what is read is GitHub's own REST-style
/// `message` envelope, which is what a forbidden status carries, and the `message` and
/// `type` of each GraphQL error, which is where a *successful* response says it.
///
/// A body that is not JSON at all has nothing structured to read, so only a failing
/// response's own text is taken — a successful response that is not JSON is malformed
/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
fn refusal_wording(status: StatusCode, body: &str) -> String {
let Ok(parsed) = serde_json::from_str::<Value>(body) else {
return if status.is_success() {
String::new()
} else {
body.to_owned()
};
};
let mut said: Vec<&str> = parsed
.get("message")
.and_then(Value::as_str)
.into_iter()
.collect();
if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
for error in errors {
said.extend(
["message", "type"]
.into_iter()
.filter_map(|key| error.get(key).and_then(Value::as_str)),
);
}
}
said.join("; ")
}
impl Limiter {
/// Which limiter refused this response, or `None` when none of them did.
///
/// The wording is read first and the status only decides what carries none of it,
/// because GitHub answers a secondary limit with a forbidden status far more often
/// than with too-many-requests — while a forbidden status saying nothing about a limit
/// really is a credential this token lacks.
///
/// A response is a refusal because of its status or its own wording. A spent budget
/// only ever explains one; it never turns an answer into a refusal.
fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
let normalized = refusal_wording(status, body).to_ascii_lowercase();
if SECONDARY_WORDINGS
.iter()
.any(|wording| normalized.contains(wording))
{
return Some(Self::Secondary);
}
if status == StatusCode::TOO_MANY_REQUESTS {
return Some(Self::Primary);
}
// An exhausted budget *explains* a response that failed; it does not make one that
// succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
// request the budget allowed as well as on the ones it then refuses, so reading
// the header alone threw away a good answer — and, once refusals were retried,
// replayed a request that had already taken effect.
if !status.is_success() && budget_exhausted {
return Some(Self::Primary);
}
// A successful response saying it: GitHub reports a GraphQL rate limit in the
// `errors` of an HTTP 200, where nothing about the status says so at all.
if status.is_success()
&& PRIMARY_WORDINGS
.iter()
.any(|wording| normalized.contains(wording))
{
return Some(Self::Primary);
}
None
}
/// What this limiter is called where an operator can look it up.
const fn name(self) -> &'static str {
match self {
Self::Primary => "GitHub's primary API rate limit",
Self::Secondary => "GitHub's secondary rate limit",
}
}
/// What the endpoint an operator would go and check says about this limiter.
const fn where_to_look(self) -> &'static str {
match self {
Self::Primary => {
"That is the budget `gh api rate_limit` reports, so that endpoint says when it \
comes back."
}
Self::Secondary => {
"That limiter is not the primary API budget: `gh api rate_limit` reports the \
primary budget and does not report this one, so budget showing there says \
nothing about this refusal, and every further attempt extends it."
}
}
}
/// The next step this limiter actually calls for.
const fn what_to_do(self) -> &'static str {
match self {
Self::Primary => {
"wait for the reset `gh api rate_limit` reports, then run the command again."
}
Self::Secondary => {
"leave this board alone for a few minutes, then run the command again — or \
raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
}
}
}
}
/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
#[derive(Debug, Clone, Copy)]
struct Limited {
limiter: Limiter,
hint: Option<u64>,
}
impl Limited {
/// What the caller is told once this source has waited as long as it may.
///
/// Both limiters report as [`SourceError::RateLimited`], because that is what
/// happened: the kind a caller matches on says a rate limit refused this, and nothing
/// about *which* limiter it was makes it a different kind of failure. What differs is
/// the operator's next step, and that is what the message carries — a secondary
/// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
/// budget looks fine, and then back to retry the very burst that was refused.
fn exhausted(
self,
doing: &str,
waits: u32,
waited: Duration,
needed: Duration,
budget: Duration,
) -> SourceError {
SourceError::RateLimited {
retry_after_seconds: self.hint,
message: Some(format!(
"{} refused this source while {doing}; it waited {} out over {} and was refused \
again, and the next wait of {} would take it past the {} one call may spend \
waiting. {} next: {}",
self.limiter.name(),
plural(waits, "refusal"),
seconds(waited),
seconds(needed),
seconds(budget),
self.limiter.where_to_look(),
self.limiter.what_to_do(),
)),
}
}
}
/// One HTTP attempt's result, with what its response said about the rate limit.
///
/// The two travel together so the record and the outcome are written from the same place:
/// what a response said about the budget is only readable while that response is in hand,
/// and what the attempt *meant* is only decidable once its body has been read.
struct Attempted {
result: Result<Value, Attempt>,
limits: accounting::RateLimit,
/// GitHub's own reported cost for this call, for a document that asked for it.
reported_cost: Option<u64>,
}
/// One attempt's outcome: an error to report, or a rate limit to wait out.
enum Attempt {
Failed(SourceError),
Limited(Limited),
}
fn plural(count: u32, thing: &str) -> String {
if count == 1 {
format!("{count} {thing}")
} else {
format!("{count} {thing}s")
}
}
fn seconds(duration: Duration) -> String {
format!("{:.1}s", duration.as_secs_f64())
}
/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
///
/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
/// header, and neither is what makes a response a refusal — so the whole cost of one this
/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
/// instead. Refusing the response over the header would turn a readable refusal into an
/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
value
.and_then(|value| value.to_str().ok())
.and_then(|value| value.trim().parse::<u64>().ok())
}
/// Every mutation this source sends creates content — an issue, a board item, a field of
/// one, a sub-issue link, a dependency, a comment — or edits or removes content of that
/// kind, and no query in [`graphql::DOCUMENTS`] does, so what the secondary limiter counts
/// and what the keyword says are the same set. That is what makes the keyword a sound test
/// rather than a convenient one: pacing an edit or a removal the limiter might not have
/// counted costs a wait, and not pacing one it did count costs the next fifty minutes.
fn is_mutation(query: &str) -> bool {
query.trim_start().starts_with("mutation")
}
/// What this source was doing, for a diagnostic that has to say so.
///
/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
/// a document added without a description is caught by that list's own gate instead of
/// falling through to the vague arm below.
fn operation_description(query: &str) -> &'static str {
graphql::DOCUMENTS
.iter()
.find(|(document, _)| *document == query)
.map_or("talking to GitHub", |(_, doing)| *doing)
}
/// GitHub's published ceiling on content-generating requests, per minute.
///
/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
/// from it, so a pacing value checked only against itself cannot go stale here.
pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
/// governs; it is pinned beside its sibling so the gate would notice either one moving.
pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
/// Shortest interval between two content-creating mutations, in milliseconds.
///
/// GitHub documents two secondary limits on content-generating requests:
/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
/// deliberately *not* what this paces at. An installation that wants the hourly bound
/// honoured for a long sequence of copies says so through
/// `pacing.min_mutation_interval_ms`.
pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
///
/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
/// own advice for a secondary limit — wait, and wait longer each time — without spending
/// the first minute of a transient refusal doing nothing.
pub const RETRY_BACKOFF_MS: u64 = 1_000;
/// Total time one call may spend waiting out rate limits before it reports a failure.
///
/// Two minutes is long enough to ride out the refusals a paced copy still collects and
/// short enough that a command an operator is watching returns. The bound is what makes
/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
/// the limiter, not in a process nobody can tell from a wedged one.
pub const RETRY_BUDGET_MS: u64 = 120_000;
fn default_token_env() -> String {
"GH_PROJECTS_TOKEN".to_owned()
}
fn default_endpoint() -> String {
"https://api.github.com/graphql".to_owned()
}
/// Where one status category lands on this board.
///
/// `null` — an absent value — disables the category for this instance, and using a
/// disabled status is a refusal naming the status and the instance.
#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
#[serde(untagged)]
pub enum StatusTargetConfig {
/// The name of a `Status` single-select option already on the board.
Column(ColumnName),
}
/// The name of a `Status` single-select option on the board.
///
/// Validated on the way in rather than checked later, so a blank option name — which
/// nothing on a board can be — is a state this type cannot hold.
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize, schemars::JsonSchema)]
#[serde(try_from = "String")]
#[schemars(extend("minLength" = 1))]
pub struct ColumnName(String);
impl ColumnName {
/// The option name, as the board spells it.
fn as_str(&self) -> &str {
&self.0
}
}
impl TryFrom<String> for ColumnName {
type Error = String;
fn try_from(name: String) -> Result<Self, Self::Error> {
if name.trim().is_empty() {
return Err("a status_mapping option name cannot be blank".to_owned());
}
Ok(Self(name))
}
}
/// The two closed states this product can mean.
///
/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
/// work nor abandoned work, so nothing here ever writes it.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
#[serde(rename_all = "kebab-case")]
pub enum ClosedState {
/// `COMPLETED` — precisely done.
Completed,
/// `NOT_PLANNED` — precisely cancelled.
NotPlanned,
}
impl ClosedState {
const fn reason(self) -> &'static str {
match self {
Self::Completed => "COMPLETED",
Self::NotPlanned => "NOT_PLANNED",
}
}
}
/// Configuration for one GitHub Projects v2 board.
#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
#[serde(default, deny_unknown_fields)]
pub struct GitHubProjectsConfig {
/// Login of the user or organization which owns the board.
pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
/// The project number shown in the board's GitHub URL.
pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
// llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] This doc is the field's schema description, which is what a person configuring the source reads, so it has to say when the field decides an issue's repository and when the item's own field does; the rule's one executable source is `GitHubProjectsSource::creation_target`, and `tests/plugin.rs` drives each case named here against the loopback board.
/// `owner/name` of the repository this source creates an issue in when the item's own
/// `repositories` field does not decide it.
///
/// An item naming exactly one repository is created there; a task or a document naming
/// none or several is created in its parent project's repository; and a project, or a
/// task or document with no parent, naming none or several is created here. A board
/// has no repository of its own and `createIssue` requires one, so a write without
/// this is refused naming the field. Reads never need it.
pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
// llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate]
/// Environment variable containing a fine-grained token with Projects and Issues
/// read/write plus Pull requests read-only access for every repository represented on
/// the board.
#[serde(default = "default_token_env")]
pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
/// GraphQL endpoint. GitHub Enterprise installations may override it.
#[serde(default = "default_endpoint")]
pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
/// Per-instance mapping from a status category to where it lands on this board.
///
/// A category this does not mention keeps its shipped default: `backlog` to
/// "Backlog", `todo` to "Todo", `queued` to "Queued", `in-progress` to "In Progress",
/// `done` to "Done" plus closed as completed, `cancelled` to "Cancelled" plus closed
/// as not planned, and `draft` and `unknown` disabled. `unknown` may name one existing
/// board option; every unknown word then lands on that option and reads back as
/// `unknown` under its name. Unlike `local-md`, this source cannot keep each unknown
/// word because it never creates board options.
#[serde(default)]
pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
/// How fast this source writes, and how long it waits out a rate-limit refusal.
///
/// Every field keeps its shipped default when it is absent, and the defaults are
/// GitHub's own published limits rather than taste. See [`Pacing`].
#[serde(default)]
pub pacing: PacingConfig,
}
/// How fast this source writes, and how long it waits out a rate-limit refusal.
///
/// Configurable because a GitHub Enterprise installation sets its own limits and an
/// operator who has already been refused may want to go slower still — not because the
/// defaults are guesses.
#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
#[serde(default, deny_unknown_fields)]
pub struct PacingConfig {
/// Shortest interval between two content-creating mutations, in milliseconds.
///
/// Zero sends them as fast as they are asked for, which is what a fixture server on
/// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
/// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
/// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
/// zero while there is a budget to spend, because a schedule of zero-length waits
/// consumes none of it and so never ends.
pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
/// Total time one call may spend waiting out rate limits, in milliseconds.
///
/// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
/// the bound is what makes this a wait rather than a hang.
pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
}
/// The largest any pacing setting may be, in milliseconds.
///
/// One hour. GitHub's own harshest published bound on content-generating requests works
/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
/// anything a real limit asks for, and past it the settings stop describing pacing at all:
/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
/// and an interval beyond it is a command that never sends its second mutation. It also
/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
/// what an `Instant` can hold on every platform.
pub const MAX_PACING_MS: u64 = 3_600_000;
/// [`PacingConfig`] with every default resolved and every value checked, which is what the
/// source holds.
#[derive(Debug, Clone, Copy)]
struct Pacing {
min_mutation_interval: Duration,
retry_backoff: Duration,
retry_budget: Duration,
}
impl Pacing {
/// Resolve one instance's pacing, refusing a configuration that would not pace at all.
fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
let bounded = |value: Option<u64>, default: u64, field: &str| match value {
Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
message: format!(
"pacing.{field} of source {instance} is {value} ms, and the most any pacing \
setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
GitHub's own harshest published limit"
),
}),
Some(value) => Ok(Duration::from_millis(value)),
None => Ok(Duration::from_millis(default)),
};
let retry_backoff = bounded(
config.retry_backoff_ms,
RETRY_BACKOFF_MS,
"retry_backoff_ms",
)?;
let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
if retry_backoff.is_zero() && !retry_budget.is_zero() {
return Err(SourceError::Config {
message: format!(
"pacing.retry_backoff_ms of source {instance} is 0 while \
pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
none of that budget, so it would retry a refusal forever. Set a backoff of \
at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
waiting at all",
retry_budget.as_millis()
),
});
}
Ok(Self {
min_mutation_interval: bounded(
config.min_mutation_interval_ms,
MIN_MUTATION_INTERVAL_MS,
"min_mutation_interval_ms",
)?,
retry_backoff,
retry_budget,
})
}
}
/// Factory for [`GitHubProjectsSource`].
#[derive(Debug, Clone, Copy, Default)]
pub struct Plugin;
impl SourcePlugin for Plugin {
fn kind(&self) -> &'static str {
KIND
}
fn config_schema(&self) -> Schema {
schema_for!(GitHubProjectsConfig)
}
fn build(
&self,
name: &SourceName,
config: &Value,
secrets: &dyn SecretResolver,
) -> Result<Box<dyn TaskSource>, SourceError> {
self.build_recording_into(name, config, secrets, Arc::new(Accounting::new()))
}
}
impl Plugin {
/// Build a source recording every request it sends into an accounting the caller holds.
///
/// [`SourcePlugin::build`] is this with an accounting of its own, which is what the
/// registry gets. This is for a caller that is also calling GitHub itself and wants one
/// session total rather than two — see [`accounting`] and
/// [`GitHubProjectsSource::recording_into`].
///
/// # Errors
///
/// Exactly [`SourcePlugin::build`]'s, with the same source name in front of each:
/// [`SourceError::Config`] for configuration this plugin cannot use and
/// [`SourceError::Auth`] for a credential it cannot find.
pub fn build_recording_into(
&self,
name: &SourceName,
config: &Value,
secrets: &dyn SecretResolver,
ledger: Arc<Accounting>,
) -> Result<Box<dyn TaskSource>, SourceError> {
let config: GitHubProjectsConfig =
serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
message: format!("source {name}: {e}"),
})?;
let source = GitHubProjectsSource::recording_into(name, config, secrets, ledger).map_err(
|error| match error {
SourceError::Config { message } => SourceError::Config {
message: format!("source {name}: {message}"),
},
SourceError::Auth { message } => SourceError::Auth {
message: format!("source {name}: {message}"),
},
other => other,
},
)?;
Ok(Box::new(source))
}
}
/// Where a status category lands on this board, once configuration is resolved.
#[derive(Debug, Clone, PartialEq, Eq)]
enum StatusTarget {
/// Not usable against this instance.
Disabled,
/// The board's `Status` option of this name.
Column(ColumnName),
/// A closed issue, with both its board option and the reason that says which closed it means.
// llmlint: ignore[invalid_states_unrepresentable] The reason is fixed by the category — `done` closes as completed, `cancelled` as not planned — and this private enum is built in one place, `StatusMapping::new`, which pairs each from the category's own slot. Carrying the reason on the target is what lets every write site that holds only a target derive its `stateInput` from that one resolved model rather than re-deriving it from a category and risking a disagreement with the mapping.
Terminal(ColumnName, ClosedState),
}
/// Every status category, in the order the vocabulary declares them.
///
/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
/// added to the shared vocabulary fails to compile until it is named there, and this
/// crate's suite reconciles this list against that enum's own derived schema, which is
/// generated from the variants rather than written beside them. The schema is what
/// catches a list left one short — a list checking only the positions it already holds
/// would pass while every mapping indexed by the new position panicked.
pub const CATEGORIES: [StatusCategory; 8] = [
StatusCategory::Draft,
StatusCategory::Backlog,
StatusCategory::Todo,
StatusCategory::Queued,
StatusCategory::InProgress,
StatusCategory::Done,
StatusCategory::Cancelled,
StatusCategory::Unknown,
];
/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
#[must_use]
pub const fn category_position(category: StatusCategory) -> usize {
match category {
StatusCategory::Draft => 0,
StatusCategory::Backlog => 1,
StatusCategory::Todo => 2,
StatusCategory::Queued => 3,
StatusCategory::InProgress => 4,
StatusCategory::Done => 5,
StatusCategory::Cancelled => 6,
StatusCategory::Unknown => 7,
}
}
/// The spelling a status category is configured and reported under.
fn category_name(category: StatusCategory) -> &'static str {
match category {
StatusCategory::Draft => "draft",
StatusCategory::Backlog => "backlog",
StatusCategory::Todo => "todo",
StatusCategory::Queued => "queued",
StatusCategory::InProgress => "in-progress",
StatusCategory::Done => "done",
StatusCategory::Cancelled => "cancelled",
StatusCategory::Unknown => "unknown",
}
}
/// A shipped default's option name.
///
/// The literals below are this file's own and non-blank, and they are validated by the
/// one constructor a configured name goes through rather than beside it.
fn shipped_column(name: &'static str) -> ColumnName {
ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
}
/// The shipped default for one category, before this instance's configuration.
fn shipped_default(category: StatusCategory) -> StatusTarget {
match category {
StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
StatusCategory::Queued => StatusTarget::Column(shipped_column("Queued")),
StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
StatusCategory::Done => {
StatusTarget::Terminal(shipped_column("Done"), ClosedState::Completed)
}
StatusCategory::Cancelled => {
StatusTarget::Terminal(shipped_column("Cancelled"), ClosedState::NotPlanned)
}
StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
}
}
/// This instance's complete category-to-target mapping, read in both directions.
///
/// One target per category, held at that category's own [`category_position`], so a
/// category missing from the mapping, named twice in it, or filed out of order is a
/// state this type cannot hold rather than one [`Self::target`] has to defend against.
#[derive(Debug, Clone)]
struct StatusMapping {
targets: [StatusTarget; CATEGORIES.len()],
}
impl StatusMapping {
fn resolve(
configured: BTreeMap<String, Option<StatusTargetConfig>>,
instance: &SourceName,
) -> Result<Self, SourceError> {
let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
for (key, value) in configured {
let category = CATEGORIES
.iter()
.find(|category| category_name(**category) == key)
.ok_or_else(|| SourceError::Config {
message: format!(
"status_mapping names {key:?}, which is not a status category of source \
{instance}; the categories are {}",
CATEGORIES
.iter()
.map(|category| category_name(*category))
.collect::<Vec<_>>()
.join(", ")
),
})?;
overrides.insert(category_name(*category), value);
}
// `CATEGORIES[position] == category` for every category — the crate's suite
// asserts it — so mapping the list in order fills each category's own slot.
let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
None => shipped_default(category),
Some(None) => StatusTarget::Disabled,
Some(Some(StatusTargetConfig::Column(option))) => match category {
StatusCategory::Done => StatusTarget::Terminal(option, ClosedState::Completed),
StatusCategory::Cancelled => {
StatusTarget::Terminal(option, ClosedState::NotPlanned)
}
_ => StatusTarget::Column(option),
},
});
let mapping = Self { targets };
for (index, category) in CATEGORIES.into_iter().enumerate() {
let option = match mapping.target(category) {
StatusTarget::Column(option) | StatusTarget::Terminal(option, _) => option,
StatusTarget::Disabled => continue,
};
if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
matches!(mapping.target(**earlier), StatusTarget::Column(name) | StatusTarget::Terminal(name, _)
if name.as_str().eq_ignore_ascii_case(option.as_str()))
}) {
return Err(SourceError::Config {
message: format!(
"status_mapping of source {instance} sends both {} and {} to the board \
option {:?}; one option cannot read back as two categories",
category_name(*other),
category_name(category),
option.as_str()
),
});
}
}
Ok(mapping)
}
fn target(&self, category: StatusCategory) -> &StatusTarget {
&self.targets[category_position(category)]
}
/// The category a board option name reports, or `None` when nothing maps to it.
fn category_of(&self, option: &str) -> Option<StatusCategory> {
CATEGORIES.into_iter().find(|category| {
matches!(self.target(*category), StatusTarget::Column(name) | StatusTarget::Terminal(name, _)
if name.as_str().eq_ignore_ascii_case(option))
})
}
/// The status an item reports, from the three things a read of it says: its board
/// `Status` option, whether its issue is closed, and the reason it was closed with.
///
/// The closed state decides the category and the `Status` option decides the name, so
/// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
/// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
/// duplicate is not finished work, and calling it done is a lie the next copy would
/// write back. `REOPENED`-while-closed is a state this source can never produce, so
/// it is read permissively rather than refused — reads are faithful, and refusals
/// belong on writes.
///
/// One function of those three rather than of a response, so a narrow status write can
/// answer what a re-read would report by applying it to the state it has just written.
fn status(&self, option: Option<&str>, closed: bool, reason: Option<&str>) -> Status {
if closed {
let category = match reason {
None | Some("COMPLETED") => StatusCategory::Done,
Some("NOT_PLANNED") => StatusCategory::Cancelled,
Some(_) => StatusCategory::Unknown,
};
let fallback = match category {
StatusCategory::Done => "Done",
StatusCategory::Cancelled => "Cancelled",
_ => "Closed",
};
return Status {
category,
name: option.unwrap_or(fallback).to_owned(),
};
}
let name = option.unwrap_or("Open").to_owned();
Status {
category: self.category_of(&name).unwrap_or(StatusCategory::Unknown),
name,
}
}
}
// llmlint: ignore-block[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate] Every `createIssue` names one of these, and which one is the rule — a reader who reaches the type from `create_and_file_issue` gets the rule in one sentence here without the method's refusals, which stay on `creation_target`, the rule's one executable source; `tests/plugin.rs` drives every arm of it against the loopback board.
/// One repository this source can create an issue in, as `owner/name`.
///
/// Every `createIssue` this source sends names one of these: the item's own single
/// `repositories` entry, else its parent project issue's repository, else the configured
/// [`GitHubProjectsConfig::repository`]. [`GitHubProjectsSource::creation_target`] makes
/// that choice and says what it refuses before `createIssue`.
// llmlint: ignore-end[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate]
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
struct RepositoryTarget {
owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
}
impl RepositoryTarget {
fn parse(value: &str) -> Result<Self, SourceError> {
let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
message: format!(
"repository must be spelled owner/name; {value:?} names no repository"
),
})?;
if !valid_github_owner(owner) || !valid_github_repository_name(name) {
return Err(SourceError::Config {
message: format!(
"repository must be spelled owner/name with a GitHub login and one \
repository name; {value:?} is not"
),
});
}
Ok(Self {
owner: owner.to_owned(),
name: name.to_owned(),
})
}
/// The one host whose repositories this source creates issues in, spelled once: it is
/// what [`Self::origin`] renders and what [`Self::from_origin`] accepts.
const HOST: &str = "github.com";
fn origin(&self) -> String {
format!("{}/{}/{}", Self::HOST, self.owner, self.name)
}
/// The repository a normalized origin names, or why it is none this source can create
/// an issue in: another host, or more or fewer than `owner/name` under this one.
fn from_origin(origin: &Repository) -> Result<Self, String> {
let not_here = || {
format!(
"{} is not a {}/owner/name repository",
origin.as_str(),
Self::HOST
)
};
let (host, rest) = origin.as_str().split_once('/').ok_or_else(not_here)?;
if host != Self::HOST {
return Err(not_here());
}
Self::parse(rest).map_err(|_| not_here())
}
fn slug(&self) -> String {
format!("{}/{}", self.owner, self.name)
}
}
/// A source which reads GitHub afresh for every operation.
pub struct GitHubProjectsSource {
/// This source's configured name, used both to tell a far end naming this source
/// from one naming a system it knows nothing about, and to name the instance a
/// status refusal is about.
name: SourceName,
owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
repository: Option<RepositoryTarget>,
endpoint: Url,
token: SecretString,
credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
statuses: StatusMapping,
client: Client,
/// Every item this source has created since it was built, in the order it created
/// them.
///
/// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
/// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
/// a copy resolving a dependency on an item it had just created refused it as not
/// found. A board read is completed from this — an item remembered here and absent from
/// the read is added back, because the board really does hold it and only the read is
/// behind.
///
/// It is not a cache of a user's work: nothing is remembered that this process did not
/// itself just write, it lives and dies with the process, and it is never consulted for
/// an item this source did not create.
created: Mutex<Vec<Resolved>>,
/// How fast this source writes, and how long it waits out a refusal.
pacing: Pacing,
/// When the last content-creating mutation finished, or the moment the furthest-out
/// reserved slot releases the next one, whichever is later — so the one after it can be
/// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
/// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
/// what it is measured from.
last_mutation: Mutex<Option<Instant>>,
/// The board as this process last read it, for the length of one command.
///
/// A copy of a project used to re-read the whole board, paged, before writing each of
/// its items, which is by far the largest part of a copy's request count and none of
/// its work. Nothing else changes this board while a command runs — this source's own
/// writes are the only writer — so one read answers them all.
///
/// It is not a store of a user's work and it is not the cache the no-persistence
/// invariant forbids: it lives and dies with the process exactly as `created` does,
/// nothing is written down, and [`Self::board`] still completes it from `created`, so
/// an item this command created and then depends on resolves whether or not GitHub's
/// own eventually-consistent read has caught up. A write to an item already on the
/// board updates the entry here too, so what this holds is the last read plus this
/// process's own writes rather than a snapshot taken before them.
board_cache: Mutex<Option<Board>>,
/// Each destination repository's node id, resolved once per repository
/// rather than per issue created.
///
/// A repository's node id does not change, and re-reading it for every issue of a copy
/// spent one request per item on an answer this source already had. It is a map rather
/// than one entry because a copy files each item in the repository its own
/// `repositories` field names, so a plan across five repositories asks GitHub five
/// times and not once per item.
repository_cache: Mutex<BTreeMap<RepositoryTarget, String>>,
/// What every request this source sends is recorded into.
///
/// Ordinary code path, not a mode: [`Self::send_once`] records into it at the one place
/// a request leaves this crate, so nothing has to be switched on for a session to be
/// counted. It is shared rather than owned so a caller accounting for a whole session —
/// its own schema verification, board lookups, residue sweep and cleanup beside this
/// source's reads and writes — adds up one accounting instead of two. See
/// [`accounting`] for what a record carries and what a session's spend is and is not.
ledger: Arc<Accounting>,
}
/// GitHub's closed single-select color vocabulary.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, schemars::JsonSchema)]
#[serde(rename_all = "SCREAMING_SNAKE_CASE")]
pub enum StatusOptionColor {
/// Gray.
Gray,
/// Blue.
Blue,
/// Green.
Green,
/// Yellow.
Yellow,
/// Purple.
Purple,
/// Red.
Red,
/// Orange.
Orange,
/// Pink.
Pink,
}
/// Whether the guarded operation plans or applies additions.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum StatusOptionsMode {
/// Read without mutation.
Plan,
/// Apply and verify.
Apply,
}
/// The explicit result of the requested operation.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, schemars::JsonSchema)]
#[serde(rename_all = "kebab-case")]
pub enum StatusOptionsOutcome {
/// A read-only plan.
Planned,
/// Apply found nothing missing.
Unchanged,
/// Additions were applied and verified.
Applied,
}
/// A GitHub single-select option's opaque GraphQL node identifier.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, schemars::JsonSchema)]
#[serde(transparent)]
pub struct StatusOptionId(#[schemars(length(min = 1))] String);
impl TryFrom<String> for StatusOptionId {
type Error = String;
fn try_from(id: String) -> Result<Self, Self::Error> {
if id.trim().is_empty() {
return Err("a GitHub Status option id cannot be blank".to_owned());
}
Ok(Self(id))
}
}
/// One existing or proposed option in a guarded Status-field update.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, schemars::JsonSchema)]
pub struct StatusOption {
/// GitHub's stable id.
pub id: StatusOptionId,
/// The visible option name.
pub name: ColumnName,
/// GitHub's single-select color token.
pub color: StatusOptionColor,
/// The option description, including an empty one.
pub description: String,
}
/// One board item's Status assignment, retained as recovery data.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, schemars::JsonSchema)]
pub struct StatusAssignment {
/// The project item id whose assignment this is.
// llmlint: ignore[invalid_states_unrepresentable] This opaque GraphQL node ID is
// carried verbatim as operator recovery data; introducing a semantic type would claim
// validation rules GitHub does not publish and no operation here interprets.
pub item_id: String,
/// The selected option, absent when the item has no status.
#[serde(skip_serializing_if = "Option::is_none")]
pub option: Option<AssignedStatusOption>,
}
/// The inseparable id and name of an assigned option.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, schemars::JsonSchema)]
pub struct AssignedStatusOption {
/// GitHub's stable id.
pub id: StatusOptionId,
/// The visible name.
pub name: ColumnName,
}
/// The plan and verified outcome of reconciling configured Status options.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, schemars::JsonSchema)]
pub struct StatusOptionsReport {
/// The configured source name.
pub source: SourceName,
/// Configured option names absent before the operation.
// llmlint: ignore[invalid_states_unrepresentable] Each value originates from a
// `ColumnName` and has therefore already passed its nonblank validation; retaining the
// serialized string here preserves the report's intentionally simple public contract.
pub missing: Vec<String>,
/// What the requested operation did.
pub outcome: StatusOptionsOutcome,
/// The complete option list observed before any mutation.
pub existing: Vec<StatusOption>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct StatusSnapshot {
// llmlint: ignore[invalid_states_unrepresentable] This private opaque GraphQL ID is
// passed back as the mutation's project identity; a newtype could enforce no stronger
// invariant because GitHub publishes no grammar for it.
board_id: String,
// llmlint: ignore[invalid_states_unrepresentable] This private opaque GraphQL ID is
// passed back as the mutation's field identity; a newtype could enforce no stronger
// invariant because GitHub publishes no grammar for it.
field_id: String,
options: Vec<StatusOption>,
assignments: Vec<StatusAssignment>,
}
impl GitHubProjectsSource {
/// Report missing configured Status options and, when `apply` is true, add them with
/// a whole-list mutation that preserves every existing id and verifies the result.
///
/// # Errors
///
/// Refuses a board without a single-select `Status` field. A post-write difference in
/// any pre-existing option id or item assignment is refused with the complete pre-write
/// assignment snapshot in the diagnostic for recovery.
// llmlint: ignore[changed_behavior_has_e2e] The CLI journeys cover plan, no-op apply,
// successful mutation, both drift refusals, source selection, missing Status, casing,
// and paging. Transport errors remain the shared `graphql` boundary's behavior rather
// than a new status-options behavior, and the pinned-schema test prevents valid GitHub
// responses from entering the defensive malformed-response branches below.
pub async fn status_options(
&self,
mode: StatusOptionsMode,
) -> Result<StatusOptionsReport, SourceError> {
let before = self.status_snapshot().await?;
let configured = self
.statuses
.targets
.iter()
// A terminal category's option is as configured as an open one's: a terminal
// write validates it before closing and refuses when the board lacks it.
.filter_map(|target| match target {
StatusTarget::Column(name) | StatusTarget::Terminal(name, _) => {
Some(name.as_str().to_owned())
}
StatusTarget::Disabled => None,
});
let missing = configured
.filter(|wanted| {
!before
.options
.iter()
.any(|option| option.name.as_str().eq_ignore_ascii_case(wanted))
})
.collect::<Vec<_>>();
let report = StatusOptionsReport {
source: self.name.clone(),
missing: missing.clone(),
outcome: match (mode, missing.is_empty()) {
(StatusOptionsMode::Plan, _) => StatusOptionsOutcome::Planned,
(StatusOptionsMode::Apply, true) => StatusOptionsOutcome::Unchanged,
(StatusOptionsMode::Apply, false) => StatusOptionsOutcome::Applied,
},
existing: before.options.clone(),
};
if mode == StatusOptionsMode::Plan || missing.is_empty() {
return Ok(report);
}
let mut options = before
.options
.iter()
.map(|option| {
json!({
"id": option.id, "name": option.name, "color": option.color,
"description": option.description,
})
})
.collect::<Vec<_>>();
options.extend(missing.iter().map(|name| {
json!({
"name": name, "color": "GRAY", "description": ""
})
}));
self.graphql(
graphql::STATUS_OPTIONS_UPDATE,
json!({"input": {
"projectId": before.board_id, "fieldId": before.field_id,
"singleSelectOptions": options,
}}),
)
.await?;
let after = self.status_snapshot().await?;
let options_preserved = before
.options
.iter()
.all(|old| after.options.iter().any(|new| new == old));
let additions_present = missing.iter().all(|wanted| {
after
.options
.iter()
.any(|option| option.name.as_str().eq_ignore_ascii_case(wanted))
});
if !options_preserved || !additions_present || after.assignments != before.assignments {
let recovery = serde_json::to_string_pretty(&before.assignments).map_err(|error| {
SourceError::Malformed {
message: format!("cannot render pre-write Status recovery snapshot: {error}"),
}
})?;
return Err(SourceError::Refused {
message: format!(
"GitHub changed a pre-existing Status option id, name, color or description, or an item assignment after the guarded update; the pre-write item assignment snapshot is:\n{recovery}"
),
});
}
Ok(report)
}
// llmlint: ignore-block[changed_behavior_has_e2e] Valid snapshot shapes are exercised through
// the real CLI loopback journey, including pagination. The individual malformed guards
// are defensive validation of a schema-pinned third-party response, not separate user
// journeys; drift and missing-field failures cover the operation's recovery behavior.
async fn status_snapshot(&self) -> Result<StatusSnapshot, SourceError> {
let mut after: Option<String> = None;
let mut snapshot: Option<StatusSnapshot> = None;
loop {
let data = self
.graphql(
graphql::STATUS_OPTIONS_SNAPSHOT,
json!({
"owner": self.owner, "number": self.project_number,
"first": MAX_PAGE_SIZE, "after": after, "nestedFirst": MAX_PAGE_SIZE,
}),
)
.await?;
let board = data
.pointer("/owner/projectV2")
.filter(|board| board.is_object())
.ok_or_else(|| SourceError::Refused {
message: format!(
"source {} has no accessible GitHub Projects board",
self.name
),
})?;
if board
.pointer("/fields/pageInfo/hasNextPage")
.and_then(Value::as_bool)
!= Some(false)
{
return Err(SourceError::Malformed {
message:
"GitHub project fields is incomplete or has malformed pageInfo.hasNextPage"
.into(),
});
}
let field = board
.pointer("/fields/nodes")
.and_then(Value::as_array)
.and_then(|fields| {
fields
.iter()
.find(|field| field.get("name").and_then(Value::as_str) == Some("Status"))
})
.ok_or_else(|| SourceError::Refused {
message: format!("source {} board has no Status field", self.name),
})?;
let options = field
.get("options")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub Status field options is not an array".into(),
})?
.iter()
.map(|option| {
Ok(StatusOption {
id: StatusOptionId::try_from(required_str(option, "id")?.to_owned())
.map_err(|message| SourceError::Malformed { message })?,
name: ColumnName::try_from(required_str(option, "name")?.to_owned())
.map_err(|message| SourceError::Malformed {
message: format!("GitHub Status option name is invalid: {message}"),
})?,
color: serde_json::from_value(
option.get("color").cloned().unwrap_or(Value::Null),
)
.map_err(|error| SourceError::Malformed {
message: format!("GitHub Status option color is invalid: {error}"),
})?,
description: optional_str(option, "description")?
.unwrap_or_default()
.to_owned(),
})
})
.collect::<Result<Vec<_>, SourceError>>()?;
let board_id = required_nonblank_str(board, "id")?.to_owned();
let field_id = required_nonblank_str(field, "id")?.to_owned();
let current = snapshot.get_or_insert_with(|| StatusSnapshot {
board_id,
field_id,
options,
assignments: Vec::new(),
});
let items = board
.pointer("/items/nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project items.nodes is not an array".into(),
})?;
for item in items {
let field_values =
item.get("fieldValues")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project item is missing fieldValues".into(),
})?;
if field_values
.pointer("/pageInfo/hasNextPage")
.and_then(Value::as_bool)
!= Some(false)
{
return Err(SourceError::Malformed {
message: "GitHub project item fieldValues is incomplete or has malformed pageInfo.hasNextPage".into(),
});
}
let values = item
.pointer("/fieldValues/nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project item fieldValues.nodes is not an array".into(),
})?;
let status = values.iter().find(|value| {
value.pointer("/field/name").and_then(Value::as_str) == Some("Status")
});
current.assignments.push(StatusAssignment {
item_id: required_nonblank_str(item, "id")?.to_owned(),
option: status
.map(|value| {
Ok(AssignedStatusOption {
id: StatusOptionId::try_from(
required_str(value, "optionId")?.to_owned(),
)
.map_err(|message| SourceError::Malformed { message })?,
name: ColumnName::try_from(required_str(value, "name")?.to_owned())
.map_err(|message| SourceError::Malformed {
message: format!(
"GitHub assigned Status name is invalid: {message}"
),
})?,
})
})
.transpose()?,
});
}
let page = board.get("items").ok_or_else(|| SourceError::Malformed {
message: "GitHub project is missing items".into(),
})?;
let has_next = page
.pointer("/pageInfo/hasNextPage")
.and_then(Value::as_bool)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project items.pageInfo.hasNextPage is not a boolean".into(),
})?;
if !has_next {
break;
}
let next =
required_nonblank_str(page.get("pageInfo").unwrap_or(&Value::Null), "endCursor")?;
validate_cursor_progress(after.as_deref(), next)?;
after = Some(next.to_owned());
}
snapshot.ok_or_else(|| SourceError::Malformed {
message: "GitHub returned no Status snapshot".into(),
})
}
// llmlint: ignore-end[changed_behavior_has_e2e]
/// Validate configuration and capture the named credential without exposing it.
///
/// # Errors
///
/// Returns [`SourceError::Config`] for a configuration this instance cannot use and
/// [`SourceError::Auth`] when the named credential is missing or empty.
pub fn new(
name: &SourceName,
config: GitHubProjectsConfig,
secrets: &dyn SecretResolver,
) -> Result<Self, SourceError> {
Self::recording_into(name, config, secrets, Arc::new(Accounting::new()))
}
/// The same, recording every request it sends into an accounting the caller holds too.
///
/// [`Self::new`] is this with an accounting of its own. A caller that is also making
/// its own calls to GitHub — a lane verifying a schema, sweeping residue or cleaning
/// up — passes the one it records those into, so the session total accounts for the
/// whole session rather than for this source's share of it.
///
/// # Errors
///
/// Exactly [`Self::new`]'s: [`SourceError::Config`] for a configuration this instance
/// cannot use and [`SourceError::Auth`] when the named credential is missing or empty.
pub fn recording_into(
name: &SourceName,
config: GitHubProjectsConfig,
secrets: &dyn SecretResolver,
ledger: Arc<Accounting>,
) -> Result<Self, SourceError> {
if !valid_github_owner(&config.owner) {
return Err(SourceError::Config {
message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
});
}
if config.project_number == 0 || config.project_number > i32::MAX as u32 {
return Err(SourceError::Config {
message: format!("project_number must be between 1 and {}", i32::MAX),
});
}
if !valid_environment_name(&config.token_env) {
return Err(SourceError::Config {
message: "token_env must be a valid environment-variable name".into(),
});
}
let repository = config
.repository
.as_deref()
.map(RepositoryTarget::parse)
.transpose()?;
let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
message: format!("endpoint is not a valid URL: {e}"),
})?;
if endpoint.scheme() != "https"
&& !(endpoint.scheme() == "http"
&& endpoint
.host_str()
.is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
{
return Err(SourceError::Config {
message:
"endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
.into(),
});
}
let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
})?;
Ok(Self {
name: name.clone(),
owner: config.owner,
project_number: config.project_number,
repository,
endpoint,
token,
credential_name: config.token_env,
statuses: StatusMapping::resolve(config.status_mapping, name)?,
client: Client::builder()
.user_agent("onetaskgraph")
.build()
.map_err(|e| SourceError::Config {
message: format!("cannot build HTTP client: {e}"),
})?,
created: Mutex::new(Vec::new()),
pacing: Pacing::resolve(config.pacing, name)?,
last_mutation: Mutex::new(None),
board_cache: Mutex::new(None),
repository_cache: Mutex::new(BTreeMap::new()),
ledger,
})
}
/// A snapshot of every request this source has sent, and what each cost.
///
/// A value to hold and compare rather than a borrow of the accounting itself, so two
/// of them can sit side by side. When this source was built with
/// [`Self::recording_into`] the snapshot is the whole shared session, which is the
/// point of building it that way.
#[must_use]
pub fn accounting(&self) -> accounting::Session {
self.ledger.snapshot()
}
/// Send one GraphQL document, pacing this source's own mutations and waiting out a
/// rate limit rather than handing it straight back as an error.
///
/// Retrying is safe for every document here, including the mutations, and the reason
/// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
/// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
/// this replays has already taken effect. An outcome this source cannot know — the
/// send failed, or the body could not be read, so the mutation may well have landed —
/// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
/// attempt. A duplicate write would come from replaying one of those, and none is
/// replayed.
async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
let doing = operation_description(query);
let mut waited = Duration::ZERO;
let mut waits = 0_u32;
let mut backoff = self.pacing.retry_backoff;
loop {
if is_mutation(query) {
let spacing = self.reserve_mutation_slot();
if !spacing.is_zero() {
tokio::time::sleep(spacing).await;
}
}
let attempt = self.send_once(query, &variables).await;
if is_mutation(query) {
self.finish_mutation();
}
let limited = match attempt {
Ok(data) => return Ok(data),
Err(Attempt::Failed(error)) => return Err(error),
Err(Attempt::Limited(limited)) => limited,
};
// GitHub really does send `retry-after: 0`, and retrying at once is the one
// move that extends a secondary limit, so a hint below the schedule's own next
// wait is raised to it.
let wait = match limited.hint {
Some(hint) => Duration::from_secs(hint).max(backoff),
None => backoff,
};
let remaining = self.pacing.retry_budget.saturating_sub(waited);
// A wait of nothing spends none of the budget, so it is exhaustion rather
// than a retry. `Pacing::resolve` rules out every way of configuring one
// except a budget of zero, where reporting the first refusal is the ask.
if wait.is_zero() || wait > remaining {
return Err(limited.exhausted(
doing,
waits,
waited,
wait,
self.pacing.retry_budget,
));
}
tokio::time::sleep(wait).await;
waited += wait;
waits += 1;
backoff = backoff.saturating_mul(2);
}
}
/// The next moment a content-creating mutation may leave this source, as a wait from
/// now.
///
/// The slot is reserved under the lock and the waiting happens outside it, so two
/// callers take two slots rather than the same one — and no lock is held across an
/// await.
///
/// The moment it is spaced from is the previous mutation's *completion*, which
/// [`Self::finish_mutation`] records. See that method for why the release moment on its
/// own is the wrong thing to measure from.
fn reserve_mutation_slot(&self) -> Duration {
if self.pacing.min_mutation_interval.is_zero() {
return Duration::ZERO;
}
// A poisoned lock here costs pacing, not correctness, and refusing the write over
// it would turn an earlier failure into a second one for no gain.
let mut last = self
.last_mutation
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let now = Instant::now();
// `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
// pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
let at = last.map_or(now, |previous| {
previous
.checked_add(self.pacing.min_mutation_interval)
.map_or(now, |earliest| earliest.max(now))
});
*last = Some(at);
at.saturating_duration_since(now)
}
/// Record that a content-creating mutation has finished, so the next one is spaced
/// from here rather than from the moment this one was released.
///
/// This source can only choose when a request *departs*; the limiter counts when it
/// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
/// departure from the last therefore hands the limiter a gap of the interval less that
/// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
/// exactly how a copy paced well inside a board's threshold was refused by it on a
/// slower machine while passing on a quick one.
///
/// Spacing from completion removes the subtraction rather than budgeting for it. The
/// previous request had already arrived before its response came back, so its arrival
/// is no later than this moment, and the next mutation is released at least the
/// interval after this moment and arrives no earlier than it is released: the gap the
/// limiter measures is therefore at least the interval, whatever transit costs and on
/// whatever platform. The price is that a mutation's own round trip no longer counts
/// towards its spacing, which makes this source slightly slower than the configured
/// rate rather than slightly faster — the safe side of a limit that punishes being
/// wrong by refusing reads for the next fifty minutes.
///
/// A failed attempt is recorded too: a request refused by the limiter still arrived,
/// and one that never left costs only a wait nobody needed.
fn finish_mutation(&self) {
if self.pacing.min_mutation_interval.is_zero() {
return;
}
// A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
let mut last = self
.last_mutation
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
let now = Instant::now();
// `max` rather than an assignment: a concurrent caller may already have reserved a
// slot further out, and completing this request must never pull that slot back in.
*last = Some(last.map_or(now, |reserved| reserved.max(now)));
}
/// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
/// failure that waiting cannot help — and recorded, whichever of the three it was.
///
/// This is the one place a request leaves this crate, which is why the accounting is
/// here rather than at each of the callers: a read path added later is counted without
/// anybody remembering to count it, and
/// `the_session_report_counts_every_request_the_board_served_and_what_each_cost` fails
/// when one is not.
async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
let Attempted {
result,
limits,
reported_cost,
} = self.attempt(query, variables).await;
// No `otherwise` name: every document this source sends is one of its own, and the
// inventory gate on `graphql::DOCUMENTS` is what keeps that true.
let sending = accounting::Request::graphql(query, variables, None, reported_cost);
let outcome = match &result {
Ok(_) => accounting::Outcome::Answered,
Err(Attempt::Limited(_)) => accounting::Outcome::RateLimited,
Err(Attempt::Failed(_)) => accounting::Outcome::Refused,
};
self.ledger.record(sending.finished(outcome, limits));
result
}
/// The attempt itself, with what its response said about the rate limit alongside.
///
/// The two are returned together rather than recorded here because every one of the
/// early exits below is a different outcome, and a record written at each of them is a
/// record one of them can be added without.
async fn attempt(&self, query: &str, variables: &Value) -> Attempted {
let mut limits = accounting::RateLimit::default();
let mut reported_cost = None;
let result = self
.attempted(query, variables, &mut limits, &mut reported_cost)
.await;
Attempted {
result,
limits,
reported_cost,
}
}
/// One HTTP attempt, filling in what its response said about the rate limit as it goes.
async fn attempted(
&self,
query: &str,
variables: &Value,
limits: &mut accounting::RateLimit,
reported_cost: &mut Option<u64>,
) -> Result<Value, Attempt> {
let response = self
.client
.post(self.endpoint.clone())
.bearer_auth(self.token.expose_secret())
.json(&json!({"query": query, "variables": variables}))
.send()
.await
.map_err(|e| {
Attempt::Failed(SourceError::Unavailable {
message: format!("GitHub GraphQL request failed: {e}"),
})
})?;
let status = response.status();
let header = |name: &str| whole_seconds(response.headers().get(name));
*limits = accounting::RateLimit::read(|name| {
response
.headers()
.get(name)
.and_then(|value| value.to_str().ok())
.map(str::to_owned)
});
// Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
// that are not text at all — is "not known to be exhausted". This never makes a
// response a refusal on its own: it says which limiter a refusal is attributed to
// and where its hint comes from, so a value this cannot read costs a hint rather
// than an answer.
let exhausted = response
.headers()
.get("x-ratelimit-remaining")
.and_then(|value| value.to_str().ok())
== Some("0");
// `retry-after` is what GitHub asks for when it asks; when it does not and the
// primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
// which is the same question answered as an absolute time. Nothing else here is a
// hint, and a schedule is what answers a refusal that carries none.
let hint = header("retry-after").or_else(|| {
exhausted
.then(|| header("x-ratelimit-reset"))
.flatten()
.map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
});
// Read before it is parsed, because the evidence which tells a secondary rate
// limit from a rejected credential is in the body of a response whose status says
// only "forbidden" — and a non-success response was never parsed at all.
let body = response.text().await.map_err(|e| {
Attempt::Failed(SourceError::Unavailable {
message: format!("GitHub GraphQL response could not be read: {e}"),
})
})?;
if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
return Err(Attempt::Limited(Limited { limiter, hint }));
}
if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
return Err(Attempt::Failed(SourceError::Auth {
message: format!(
"GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
),
}));
}
if !status.is_success() {
return Err(Attempt::Failed(SourceError::Unavailable {
message: format!("GitHub GraphQL returned HTTP {status}"),
}));
}
// GitHub reports what a call cost only when the document asked it to, and no
// document this source sends does — so this is `None` here and carries the figure
// for a caller whose own document selects `rateLimit { cost }`. What it must never
// pick up is a `dryRun` probe's cost, which is some other document's.
*reported_cost = serde_json::from_str::<Value>(&body)
.ok()
.as_ref()
.and_then(|body| body.pointer("/data/rateLimit/cost"))
.and_then(Value::as_u64);
self.answer(&body).map_err(Attempt::Failed)
}
/// What one successful HTTP response says, once its GraphQL errors are read.
fn answer(&self, body: &str) -> Result<Value, SourceError> {
let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
message: format!("GitHub returned invalid JSON: {e}"),
})?;
let errors = body
.get("errors")
.map(|value| {
value.as_array().ok_or_else(|| SourceError::Malformed {
message: "GitHub response errors is not an array".into(),
})
})
.transpose()?;
if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
let messages = errors
.iter()
.filter_map(|e| e.get("message").and_then(Value::as_str))
.collect::<Vec<_>>()
.join("; ");
let message = if messages.is_empty() {
"GitHub returned GraphQL errors".into()
} else {
messages
};
let normalized = message.to_ascii_lowercase();
if normalized.contains("resource not accessible") || normalized.contains("scope") {
return Err(SourceError::Auth {
message: format!(
"{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
self.credential_name
),
});
}
return Err(SourceError::Refused { message });
}
body.get("data")
.filter(|data| data.is_object())
.cloned()
.ok_or_else(|| SourceError::Malformed {
message: "GitHub response has no data object".into(),
})
}
// llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
// GraphQL cannot independently page them inside the outer item page. This source page is
// deliberately bounded at that published maximum; the live drift journey exercises it.
async fn board_page(
&self,
items_after: Option<&str>,
items_first: u32,
) -> Result<Value, SourceError> {
let data = self
.graphql(
graphql::BOARD,
json!({"owner":self.owner,"number":self.project_number,
"first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
"nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
)
.await?;
data.pointer("/owner/projectV2")
.filter(|v| !v.is_null())
.cloned()
.ok_or_else(|| SourceError::Refused {
message: format!(
"GitHub project {}/{} was not found or is not visible to the token",
self.owner, self.project_number
),
})
}
/// The search that finds the issues of this board, narrowed by `also` when it is
/// given.
///
/// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
/// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
/// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
/// here on purpose — GitHub accepts it and silently ignores it, so a project is told
/// from a task by the `parent` field each issue carries rather than by the search.
fn board_search(&self, also: Option<&str>) -> String {
let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
match also {
Some(also) => format!("{scope} {also}"),
None => scope,
}
}
/// One issue this source reached directly, as the board item a read of the board would
/// have produced — or `None` when this board does not hold it.
///
/// The board half of an issue rides along on `Issue.projectItems`, so the value handed
/// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
/// item's own id, that item's field values, and the issue as its content. One resolver
/// for both routes is what makes an issue read through a search, through its own node
/// id, or through its project's sub-issues report the same title, the same status, the
/// same labels and the same qualified id.
///
/// An issue with no entry for *this* board is not this source's to report, which is
/// what keeps an id naming some other repository's issue from being answered as an item
/// of this board. That answer is given about an **exhausted** connection and never
/// about an unread page: the entry is looked for on the page in hand, and only if that
/// page reports more of the connection, in [`Self::board_membership`]'s walk of the
/// rest of it.
async fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
if optional_str(issue, "__typename")? != Some("Issue") {
return Ok(None);
}
let memberships = issue
.get("projectItems")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue is missing projectItems".into(),
})?;
let nodes = memberships
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue projectItems.nodes is not an array".into(),
})?;
let held = match self.board_entry(nodes) {
Some(held) => held.clone(),
None => {
let info = memberships
.get("pageInfo")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue projectItems has no pageInfo".into(),
})?;
// The page held no entry for this board. Whether that means the issue is
// not on it is a question about the rest of the connection, and only a
// connection with no rest answers it here.
if !required_bool(info, "hasNextPage")? {
return Ok(None);
}
let cursor = required_str(info, "endCursor")?;
validate_cursor_progress(None, cursor)?;
let issue_id = required_str(issue, "id")?;
match self.board_membership(issue_id, cursor).await? {
Some(held) => held,
None => return Ok(None),
}
}
};
let item = json!({
"id": required_str(&held, "id")?,
"project": held.get("project"),
"fieldValues": held.get("fieldValues"),
"content": issue,
});
self.resolve(&item)
}
/// This board's own entry among one page of an issue's `Issue.projectItems`.
///
/// One spelling of *which membership is this board's*, so the page a read carries and
/// the pages [`Self::board_membership`] walks are searched by the same rule.
fn board_entry<'a>(&self, nodes: &'a [Value]) -> Option<&'a Value> {
nodes.iter().find(|node| {
node.pointer("/project/number").and_then(Value::as_u64)
== Some(u64::from(self.project_number))
})
}
/// The rest of one issue's board memberships, from `after`, for this board's entry.
///
/// The recovery read: a page of memberships that holds no entry for this board says
/// nothing about the memberships past it, so the connection is walked to exhaustion
/// before an issue is reported as one this board does not hold. `Ok(None)` is that
/// positive answer — the whole connection was read and no entry named this board —
/// rather than a failure, and the walk is held to
/// [`validate_cursor_progress`] like every other page walk here, so a source answering
/// with a cursor that does not advance is refused instead of spun on.
async fn board_membership(
&self,
issue: &str,
after: &str,
) -> Result<Option<Value>, SourceError> {
let mut after = after.to_owned();
loop {
let data = self
.graphql(
graphql::ISSUE_BOARD_ITEMS,
json!({"id":issue,"first":MAX_PAGE_SIZE,"after":after,
"nestedFirst":NESTED_PAGE_SIZE}),
)
.await?;
let Some(connection) = data
.pointer("/node/projectItems")
.filter(|value| !value.is_null())
else {
// The id resolved to nothing, or to something with no memberships to walk —
// which is the same answer as a connection holding no entry for this board.
return Ok(None);
};
let nodes = connection
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue projectItems.nodes is not an array".into(),
})?;
if let Some(held) = self.board_entry(nodes) {
return Ok(Some(held.clone()));
}
let info = connection
.get("pageInfo")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue projectItems has no pageInfo".into(),
})?;
let next = required_bool(info, "hasNextPage")?
.then(|| required_str(info, "endCursor"))
.transpose()?;
match next {
Some(next) => {
validate_cursor_progress(Some(&after), next)?;
after = next.to_owned();
}
None => return Ok(None),
}
}
}
/// One page of a board-scoped issue search, and where the next page resumes.
async fn search_page(
&self,
search: &str,
first: u32,
after: Option<&str>,
) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
let data = self
.graphql(
graphql::SEARCH_ISSUES,
json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
"after":after,"nestedFirst":NESTED_PAGE_SIZE,
"boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
)
.await?;
let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
message: "GitHub search response has no search connection".into(),
})?;
let mut found = Vec::new();
for node in connection
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub search nodes is not an array".into(),
})?
{
if let Some(resolved) = self.resolve_issue(node).await? {
found.push(resolved);
}
}
let info = connection
.get("pageInfo")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub search connection has no pageInfo".into(),
})?;
let next = required_bool(info, "hasNextPage")?
.then(|| required_str(info, "endCursor"))
.transpose()?
.map(str::to_owned);
if let Some(next) = &next {
validate_cursor_progress(after, next)?;
}
Ok((found, next))
}
/// Every issue this board holds, walked to exhaustion, completed with what this run
/// wrote.
///
/// The completion is not an optimisation and it is not a cache: GitHub's issue search
/// is an index and is eventually consistent, so an issue this run created seconds ago
/// is routinely absent from it, and a project listed straight after being written would
/// otherwise be missing from its own board. What is added back is only what this
/// process itself wrote, out of [`Self::created`], which lives and dies with the
/// process.
async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
let mut after: Option<String> = None;
let mut found = Vec::new();
let search = self.board_search(None);
loop {
let (page, next) = self
.search_page(&search, MAX_PAGE_SIZE, after.as_deref())
.await?;
found.extend(page);
match next {
Some(next) => after = Some(next),
None => break,
}
}
self.completed_with_written(found, |_| true)
}
/// `found`, with everything this run wrote that `keep` accepts and the read did not
/// report.
///
/// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
/// at all: the search index is behind, and a node read of an item filed moments ago can
/// be too.
fn completed_with_written(
&self,
mut found: Vec<Resolved>,
keep: impl Fn(&Resolved) -> bool,
) -> Result<Vec<Resolved>, SourceError> {
for own in self.created()?.iter().filter(|own| keep(own)) {
if !found.iter().any(|item| item.id == own.id) {
found.push(own.clone());
}
}
Ok(found)
}
/// What resolving one node id reached.
///
/// Three answers rather than an `Option`, because a board *draft* is none of the other
/// two: it is not an issue, it has no node of its own this source can read the board
/// half off, and its only home is the board's own item connection — so a read of one
/// is completed from there rather than reported as nothing.
async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
let asked = self
.graphql(
graphql::ISSUE,
json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
"boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
)
.await;
let data = match asked {
Ok(data) => data,
// A string that is not a node id at all is not a failure to report: it is an id
// this board does not hold, which is what every read of one already answers.
Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
Err(error) => return Err(error),
};
let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
return Ok(Reached::Nothing);
};
if optional_str(node, "__typename")? == Some("DraftIssue") {
return Ok(Reached::Draft);
}
Ok(match self.resolve_issue(node).await? {
Some(item) => Reached::Held(Box::new(item)),
None => Reached::Nothing,
})
}
/// One item of this board by its own id, whatever kind it is.
///
/// Resolved from the identifier alone: no search, board-wide or otherwise. What this
/// run wrote is read first, because a node read of an item created moments ago can
/// still be behind the board field values written onto it — see [`Self::created`].
async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
return Ok(Some(own.clone()));
}
match self.reach(id).await? {
Reached::Held(item) => Ok(Some(*item)),
Reached::Nothing => Ok(None),
// The one read that still costs the board: a draft lives nowhere else.
Reached::Draft => Ok(self
.board()
.await?
.items
.into_iter()
.find(|item| item.id == *id)),
}
}
/// Everything filed under one issue of this board, walked to exhaustion — or `None`
/// when that id names nothing here with a sub-issue relationship to walk.
///
/// `None` and an empty answer are different: `None` is *this is not an issue of this
/// GitHub*, which is what sends a project selector on to be read as a name, and an
/// empty vector is a project that holds nothing.
async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
let mut after: Option<String> = None;
let mut children = Vec::new();
loop {
let asked = self
.graphql(
graphql::SUB_ISSUES,
json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
"nestedFirst":NESTED_PAGE_SIZE,
"boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
)
.await;
let data = match asked {
Ok(data) => data,
// A string that is not a node id at all is not a failure to report: it is
// the ordinary answer to a selector naming a project by its name.
Err(error) if unresolvable_node(&error) => return Ok(None),
Err(error) => return Err(error),
};
let Some(connection) = data
.pointer("/node/subIssues")
.filter(|value| !value.is_null())
else {
// No such node, or one with no sub-issue relationship — a board draft is
// the one this board can really hold.
return Ok(None);
};
for node in connection
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub subIssues.nodes is not an array".into(),
})?
{
if let Some(resolved) = self.resolve_issue(node).await? {
children.push(resolved);
}
}
let info = connection
.get("pageInfo")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub subIssues connection has no pageInfo".into(),
})?;
let next = required_bool(info, "hasNextPage")?
.then(|| required_str(info, "endCursor"))
.transpose()?;
match next {
Some(next) => {
validate_cursor_progress(after.as_deref(), next)?;
after = Some(next.to_owned());
}
None => return Ok(Some(children)),
}
}
}
/// Which issue of this board a project *name* is, or `None` when none is.
///
/// One bounded query which filters on that name at the server, rather than a walk of
/// every issue the board holds. The name is compared again here: the qualifier narrows
/// what GitHub sends, and this source decides what it names.
async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
let search = self.board_search(Some(&title_qualifier(name)));
let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
Ok(candidates
.into_iter()
.find(|item| {
item.kind == BoardKind::Work(ItemKind::Project)
&& item.title.eq_ignore_ascii_case(name)
})
.map(|item| item.id))
}
/// Everything filed under one project of this board: the sub-issues of the issue that
/// project is.
///
/// Tasks *and* documents, because a document filed under a project is a sub-issue of it
/// too — the caller keeps the kind it asked for. Nothing about this grows as the board
/// gains projects, or as another project gains tasks.
///
/// A qualified id names the issue and is asked for its sub-issues directly: one
/// request, no search of any kind. Only a selector GitHub cannot resolve that way is
/// read as a project *name*, which costs the one bounded search
/// [`Self::project_by_name`] makes.
async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
let (project, children) = match self.sub_issues(selector).await? {
Some(children) => (selector.clone(), children),
None => match self.project_by_name(&selector.0).await? {
Some(project) => {
let children = self.sub_issues(&project).await?.unwrap_or_default();
(project, children)
}
None => return Ok(Vec::new()),
},
};
self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
}
/// Every item on the board, with the one board identity they all share.
///
/// See [`Self::board_cache`]. The completion from `created` happens on every call
/// rather than once, which is what the cache could otherwise have broken.
async fn board(&self) -> Result<Board, SourceError> {
let cached = self.board_cache()?.clone();
let mut board = match cached {
Some(board) => board,
None => {
let read = self.read_board().await?;
*self.board_cache()? = Some(read.clone());
read
}
};
for own in self.created()?.iter() {
if !board.items.iter().any(|item| item.id == own.id) {
board.items.push(own.clone());
}
}
Ok(board)
}
/// This process's own view of the board, or the refusal a poisoned lock is.
fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
self.board_cache
.lock()
.map_err(|_| SourceError::Unavailable {
message: "this source's view of the board was left inconsistent by an earlier \
failure; next: run the command again"
.into(),
})
}
/// Bring this process's own view of the board up to an item it has just written.
///
/// A created item goes to `created`, which is what completes a board read GitHub's own
/// eventual consistency has left behind. An item that was already there is replaced
/// where it sits, so a second write of it in the same command reads its real parent
/// rather than the one it had before the first write.
///
/// "Where it sits" is two places, and missing the first leaves a stale record that
/// wins: an item this same run created is held in `created` and not in the cached
/// board, and `board` completes the cached board *from* `created`, so replacing only
/// the cached copy of such an item replaces nothing and the read still reports the
/// title it was created with.
fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
if created {
self.created()?.push(item);
return Ok(());
}
{
let mut own = self.created()?;
if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
*held = item;
return Ok(());
}
}
if let Some(board) = self.board_cache()?.as_mut()
&& let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
{
*held = item;
}
Ok(())
}
/// Forget one item this process has just deleted, from both halves of its own view.
fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
self.created()?.retain(|own| own.id != *id);
if let Some(board) = self.board_cache()?.as_mut() {
board.items.retain(|item| item.id != *id);
}
Ok(())
}
/// Every page of the board, read from GitHub.
async fn read_board(&self) -> Result<Board, SourceError> {
let mut after: Option<String> = None;
let mut items = Vec::new();
let mut board;
loop {
let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
for item in page
.pointer("/items/nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project items.nodes is not an array".into(),
})?
{
if let Some(resolved) = self.resolve(item)? {
items.push(resolved);
}
}
let info = page
.pointer("/items/pageInfo")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project items have no pageInfo".into(),
})?;
let has_next = required_bool(info, "hasNextPage")?;
let next = has_next
.then(|| required_str(info, "endCursor"))
.transpose()?;
board = page.clone();
match next {
Some(next) => {
validate_cursor_progress(after.as_deref(), next)?;
after = Some(next.to_owned());
}
None => break,
}
}
Ok(Board {
id: required_str(&board, "id")?.to_owned(),
fields: board.get("fields").cloned().unwrap_or(Value::Null),
items,
})
}
/// The items this source has created, for completing a board read that is behind.
fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
self.created.lock().map_err(|_| SourceError::Unavailable {
message: "this source's record of what it created in this run was left \
inconsistent by an earlier failure; next: run the command again"
.into(),
})
}
/// One board item as this source reports it, or `None` for content it ignores.
///
/// A pull request is neither a project nor a task — it is somebody's change, not a
/// unit of plan — and an item whose content the token cannot see has nothing to
/// report at all.
fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
let content = item.get("content").ok_or_else(|| SourceError::Malformed {
message: "GitHub project item is missing content".into(),
})?;
if content.is_null() {
return Ok(None);
}
let content_kind = match required_str(content, "__typename")? {
"Issue" => ContentKind::Issue,
"DraftIssue" => ContentKind::DraftIssue,
_ => return Ok(None),
};
let field_values = item
.get("fieldValues")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project item is missing fieldValues".into(),
})?;
complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
let nodes = field_values
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project item fieldValues.nodes is not an array".into(),
})?;
if let Some(labels) = content.get("labels") {
complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
}
let raw_body = optional_str(content, "body")?.map(str::to_owned);
let (body, slot) = metadata_body(raw_body.clone())?;
let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
.map(|id| NativeId(id.to_owned()));
// A draft has no sub-issues to summarise, and GitHub's schema gives it no field
// to read one from; it is a task, and never a project.
let sub_issues = match content_kind {
ContentKind::Issue => sub_issue_total(content)?,
ContentKind::DraftIssue => 0,
};
let content_id = required_str(content, "id")?;
let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
message: format!("GitHub issue {content_id}: {message}"),
})?;
let raw_title = required_str(content, "title")?;
// The design prefix is read *first*, before either of the two rules that separate
// a project from a task. A document is not work whatever sub-issues it has and
// whatever marker it carries, and reading the prefix later would make a design
// issue with none of either an empty project.
let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
BoardKind::Document
} else if parent.is_some() {
// Being a sub-issue wins outright, and no marker overrides it: an issue filed
// under a project is that project's task even when it has sub-issues of its
// own.
BoardKind::Work(ItemKind::Task)
} else if sub_issues > 0 || marked == Some(ItemKind::Project) {
BoardKind::Work(ItemKind::Project)
} else {
BoardKind::Work(ItemKind::Task)
};
// The title a person wrote, which for a document is the one without the prefix —
// the same way `content` above is the body without this source's metadata slot.
let title = match kind {
BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
BoardKind::Work(_) => raw_title.to_owned(),
};
let own_repository = content
.pointer("/repository/nameWithOwner")
.and_then(Value::as_str)
.map(|origin| Repository::try_from(format!("{}/{origin}", RepositoryTarget::HOST)))
.transpose()
.map_err(|message| SourceError::Malformed { message })?;
let repositories = if slot.contains_key(Repository::METADATA_KEY) {
Repository::from_metadata(&slot)
.map_err(|message| SourceError::Malformed { message })?
} else {
own_repository.clone().into_iter().collect()
};
let id = NativeId(content_id.to_owned());
// Read only for a task, because only a task has either list: a project or a
// document holding one of these keys holds nothing this source reports, and the
// keys are left out of its caller-visible metadata all the same.
let (delivers, delivered_by) = if kind == BoardKind::Work(ItemKind::Task) {
let listed = |key: &str| {
TaskRef::from_value(key, &id, Some(&self.name), slot.get(key))
.map_err(|message| SourceError::Malformed { message })
};
(
listed(TaskRef::DELIVERS_KEY)?,
listed(TaskRef::DELIVERED_BY_KEY)?,
)
} else {
(Vec::new(), Vec::new())
};
let (option, closed, reason) = Self::status_parts(nodes, content)?;
Ok(Some(Resolved {
item_id: required_str(item, "id")?.to_owned(),
id,
content_kind,
kind,
title,
body: body.filter(|value| !value.is_empty()),
raw_body,
status: self.statuses.status(option, closed, reason),
option: option.map(str::to_owned),
closed,
delivers,
delivered_by,
labels: labels(content)?,
parent,
origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
url: optional_str(content, "url")?.map(str::to_owned),
created_at: optional_time(content, "createdAt")?,
updated_at: optional_time(content, "updatedAt")?,
own_repository,
repositories,
slot,
// Present when the item was reached through its own issue, whose board entry
// names the board; a read of the board's own items has the board already. An
// empty id names nothing a field write could address, so it is read as absent and
// the write goes back to reading the board.
board_id: item
.pointer("/project/id")
.and_then(Value::as_str)
.filter(|id| !id.is_empty())
.map(str::to_owned),
fields: field_definitions(nodes),
}))
}
/// What one board item's status is read from: its `Status` option, whether its issue
/// is closed, and the reason it was closed with. [`StatusMapping::status`] turns the
/// three into the status it reports.
fn status_parts<'a>(
field_values: &'a [Value],
content: &'a Value,
) -> Result<(Option<&'a str>, bool, Option<&'a str>), SourceError> {
let option = field_values
.iter()
.find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
.map(|value| required_str(value, "name"))
.transpose()?;
let closed = optional_str(content, "state")? == Some("CLOSED");
Ok((option, closed, optional_str(content, "stateReason")?))
}
/// The board Status option this write selects, or the refusal that says why not.
///
/// The mapped option is required for both open and terminal targets. A terminal write
/// validates it before changing either representation, so it can never fall back to
/// closing an issue whose board cannot display the matching status.
///
/// Answers the field's id, the option's id, and the option's name as the board spells
/// it — which is the name a read of the item reports once it sits there.
fn column_for(
&self,
board: &Board,
status: &Status,
target: &StatusTarget,
) -> Result<Option<(String, String, String)>, SourceError> {
let wanted = match target {
StatusTarget::Column(wanted) | StatusTarget::Terminal(wanted, _) => wanted.as_str(),
StatusTarget::Disabled => return Ok(None),
};
let missing = |detail: &str| SourceError::Refused {
message: format!(
"status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
category_name(status.category),
self.name,
category_name(status.category)
),
};
let Some(field) = Board::field(&board.fields, "Status")? else {
return Err(missing("this board has no Status field"));
};
if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
return Err(missing(
"this board's Status field is not a single-select field",
));
}
let option = field
.get("options")
.and_then(Value::as_array)
.and_then(|options| {
options.iter().find(|option| {
option
.get("name")
.and_then(Value::as_str)
.is_some_and(|name| name.eq_ignore_ascii_case(wanted))
})
});
match option {
None => Err(missing("this board does not have it")),
Some(option) => Ok(Some((
required_str(field, "id")?.to_owned(),
required_str(option, "id")?.to_owned(),
required_str(option, "name")?.to_owned(),
))),
}
}
/// The refusal a status that closes an issue is answered with over a board draft.
fn closes_a_draft(&self, category: StatusCategory) -> SourceError {
SourceError::Refused {
message: format!(
"status {} of source {} closes the item's issue, and GitHub draft items have \
no open or closed state",
category_name(category),
self.name
),
}
}
/// What a status write to one item needs of the board: the board's id and the
/// definition of its `Status` field, read off the item when the item says both.
///
/// The same reasoning as [`Self::board_for_update`]: a node read of the item names its
/// board, and its `Status` value carries that field's definition, options and all. An
/// item that does not say — no board id, or no `Status` value to read the field off —
/// sends this back to reading the board, as does a board this command has already read.
async fn status_board(&self, item: &Resolved) -> Result<Board, SourceError> {
let defines_status = item
.fields
.iter()
.any(|field| field.get("name").and_then(Value::as_str) == Some("Status"));
if self.board_cache()?.is_none()
&& defines_status
&& let Some(board_id) = &item.board_id
{
return Ok(Board {
id: board_id.clone(),
fields: json!({"nodes": item.fields, "pageInfo": {"hasNextPage": false}}),
items: Vec::new(),
});
}
self.board().await
}
/// Set one task's status and nothing else; see [`TaskSource::set_task_status`].
async fn set_status(
&self,
id: &NativeId,
category: StatusCategory,
) -> Result<Option<Status>, SourceError> {
// Refused before anything is read, in the words a write of the same status is.
let target = self.resolved_target(category)?;
let Some(mut item) = self
.item_by_id(id)
.await?
.filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
else {
return Ok(None);
};
let board = self.status_board(&item).await?;
let wanted = Status {
category,
name: category_name(category).to_owned(),
};
let (field, option, name) =
self.column_for(&board, &wanted, &target)?
.ok_or_else(|| SourceError::Malformed {
message: format!(
"status {} of source {} names no board Status option",
category_name(category),
self.name
),
})?;
match &target {
StatusTarget::Terminal(_, reason) => {
if item.content_kind == ContentKind::DraftIssue {
return Err(self.closes_a_draft(category));
}
self.set_item_field(
&board.id,
&item.item_id,
&field,
json!({"singleSelectOptionId": option}),
)
.await?;
self.update_content(
ContentKind::Issue,
&item.id,
json!({"stateInput": state_input(Some(&target))}),
)
.await?;
item.closed = true;
item.status = self
.statuses
.status(Some(&name), true, Some(reason.reason()));
item.option = Some(name);
}
StatusTarget::Column(_) => {
// An option is what an open item's status is, so a closed issue is reopened
// first — sitting closed in the column, it would read back as closed. A draft has
// no state to reopen.
if item.content_kind == ContentKind::Issue && item.closed {
self.update_content(
ContentKind::Issue,
&item.id,
json!({"stateInput": state_input(Some(&target))}),
)
.await?;
item.closed = false;
}
self.set_item_field(
&board.id,
&item.item_id,
&field,
json!({"singleSelectOptionId": option}),
)
.await?;
item.status = self.statuses.status(Some(&name), false, None);
item.option = Some(name);
}
StatusTarget::Disabled => unreachable!("resolved_target refused a disabled status"),
}
let status = item.status.clone();
self.remember_written(item, false)?;
Ok(Some(status))
}
/// Replace one task's `delivered_by` and nothing else; see
/// [`TaskSource::set_delivered_by`].
///
/// One update of the body, which differs from the body GitHub holds only inside the
/// metadata slot — see [`with_slot`]. A body that would not change is not sent at all.
async fn replace_delivered_by(
&self,
id: &NativeId,
delivered_by: &[TaskRef],
) -> Result<Option<()>, SourceError> {
let entries = TaskRef::listed(
TaskRef::DELIVERED_BY_KEY,
id,
Some(&self.name),
delivered_by.to_vec(),
)
.map_err(|message| SourceError::Refused { message })?;
let Some(mut item) = self
.item_by_id(id)
.await?
.filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
else {
return Ok(None);
};
let mut slot = item.slot.clone();
set_task_list(&mut slot, TaskRef::DELIVERED_BY_KEY, &entries);
self.write_slot(&mut item, &slot).await?;
item.delivered_by = entries;
self.remember_written(item, false)?;
Ok(Some(()))
}
/// Set one caller key of the metadata slot of one issue of `kind`, and nothing else;
/// see [`TaskSource::set_task_metadata`].
///
/// `None` when this board holds no item by that id, or holds one of another kind. The
/// answer is the item as this source now reads it, so what a caller is told the key
/// holds is what the slot holds.
///
/// A key already holding the value is answered without a write, compared as JSON rather
/// than as the body's bytes: a slot a person spelled with other whitespace would
/// otherwise be re-encoded, which is a write that changes nothing the caller asked for.
async fn set_slot_key(
&self,
id: &NativeId,
kind: BoardKind,
key: &MetadataKey,
value: &Value,
) -> Result<Option<Resolved>, SourceError> {
let Some(mut item) = self.item_by_id(id).await?.filter(|item| item.kind == kind) else {
return Ok(None);
};
if item.slot.get(key.as_str()) == Some(value) {
return Ok(Some(item));
}
let mut slot = item.slot.clone();
slot.insert(key.as_str().to_owned(), value.clone());
self.write_slot(&mut item, &slot).await?;
self.remember_written(item.clone(), false)?;
Ok(Some(item))
}
/// Put `slot` in one item's metadata slot with a single update of its body, and bring
/// `item` up to what that write left.
///
/// The body sent differs from the body GitHub holds only inside the slot — see
/// [`with_slot`] — and a body that would not change is not sent at all. It goes through
/// the mutation the item's content takes, so a board draft's body is written with
/// `updateProjectV2DraftIssue` exactly as an issue's is with `updateIssue`.
async fn write_slot(
&self,
item: &mut Resolved,
slot: &BTreeMap<String, Value>,
) -> Result<(), SourceError> {
let held = item.raw_body.clone().unwrap_or_default();
let body = with_slot(&held, slot)?;
if body != held {
self.update_content(item.content_kind, &item.id, json!({"body": body}))
.await?;
}
let (visible, slot) = metadata_body(Some(body.clone()))?;
item.body = visible.filter(|value| !value.is_empty());
item.raw_body = Some(body);
item.slot = slot;
Ok(())
}
/// This instance's target for a category, refusing one it has disabled.
///
/// Nothing here mutates the board's option set to make room for a status. GitHub
/// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
/// overwrite existing options"*, so no addition is additive and a mistake destroys the
/// field and every item's status.
fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
let target = self.statuses.target(category).clone();
if target != StatusTarget::Disabled {
return Ok(target);
}
Err(SourceError::Refused {
message: if category == StatusCategory::Draft {
format!(
"status draft is disabled for source {}: draft is incompatible with this \
integration because GitHub draft issues cannot have sub-issues, and this \
source stores a project's tasks as its issue's sub-issues",
self.name
)
} else if category == StatusCategory::Unknown {
format!(
"status {} is disabled for source {}; set status_mapping.{} of this source \
to one board Status option name; every word classified unknown is written \
to that one option",
category_name(category),
self.name,
category_name(category)
)
} else {
format!(
"status {} is disabled for source {}; set status_mapping.{} of this source \
to a board Status option name",
category_name(category),
self.name,
category_name(category)
)
},
})
}
async fn set_item_field(
&self,
board_id: &str,
item_id: &str,
field_id: &str,
value: Value,
) -> Result<(), SourceError> {
let data = self
.graphql(
graphql::UPDATE_FIELD,
json!({"input":{
"projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
}}),
)
.await?;
let returned = data
.pointer("/updateProjectV2ItemFieldValue/projectV2Item")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub field update returned no project item".into(),
})?;
if required_str(returned, "id")? != item_id {
return Err(SourceError::Malformed {
message: "GitHub field update returned the wrong project item".into(),
});
}
Ok(())
}
async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
let mut after: Option<String> = None;
let mut ids = Vec::new();
loop {
let data = self
.graphql(
graphql::ISSUE_DEPENDENCIES,
json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
)
.await?;
let connection =
data.pointer("/node/blockedBy")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub dependency response has no blockedBy connection".into(),
})?;
ids.extend(
connection
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub dependency response nodes is not an array".into(),
})?
.iter()
.map(|value| required_str(value, "id").map(str::to_owned))
.collect::<Result<Vec<_>, _>>()?,
);
let next = next_cursor(connection)?;
if let Some(next) = &next {
validate_cursor_progress(after.as_deref(), &next.0)?;
}
after = next.map(|cursor| cursor.0);
if after.is_none() {
return Ok(ids);
}
}
}
async fn dependencies(
&self,
id: &NativeId,
near_kind: ItemKind,
direction: Direction,
page: &PageRequest,
) -> Result<Page<DependencyEdge>, SourceError> {
validate_page(page)?;
let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
let recorded = recorded_offset(cursor, direction)?;
// Asked for even in the recorded phase, whose page reads nothing from the
// connection: `__typename` is what says whether this item has a native
// relationship at all, and that is what decides which far ends the reserved key is
// allowed to hold.
let data = self
.graphql(
graphql::ISSUE_DEPENDENCIES,
json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
"after":if recorded.is_some() {None} else {cursor}}),
)
.await?;
let node =
data.get("node")
.filter(|v| !v.is_null())
.ok_or_else(|| SourceError::Refused {
message: format!(
"GitHub item {} was not found or does not support dependencies",
id.0
),
})?;
let connection_name = match direction {
Direction::DependsOn => "blockedBy",
Direction::DependedOnBy => "blocking",
};
// A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
// named natively and the reserved key may hold any far end. An issue's connections
// hold issues, and this source reads them at the near item's own level.
let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
if let Some(offset) = recorded {
return Ok(recorded_page(
self.recorded_edges(id, near_kind, direction, natively_names, node)
.await?,
offset,
limit,
));
}
if natively_names.is_none() {
return Ok(recorded_page(
self.recorded_edges(id, near_kind, direction, natively_names, node)
.await?,
0,
limit,
));
}
let connection = node
.get(connection_name)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub dependency response is missing its connection".into(),
})?;
let nodes = connection
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub dependency response nodes is not an array".into(),
})?;
// `from` depends on `to`, always. GitHub spells the same relationship from either
// end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
// it — so the near item is `from` in one direction and `to` in the other.
let items = nodes
.iter()
.map(|value| {
let related = NativeId(required_str(value, "id")?.into());
let related_kind = related_kind(value)?;
let (from, to) = match direction {
Direction::DependsOn => (
DependencyEndpoint::from_native(id.clone(), near_kind),
DependencyEndpoint::from_native(related, related_kind),
),
Direction::DependedOnBy => (
DependencyEndpoint::from_native(related, related_kind),
DependencyEndpoint::from_native(id.clone(), near_kind),
),
};
Ok(DependencyEdge {
from,
to,
kind: DependencyKind::Blocks,
})
})
.collect::<Result<Vec<_>, SourceError>>()?;
let mut next = next_cursor(connection)?;
if let Some(next) = &next {
validate_cursor_progress(cursor, &next.0)?;
}
if next.is_none()
&& !self
.recorded_edges(id, near_kind, direction, natively_names, node)
.await?
.is_empty()
{
next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
}
Ok(Page { items, next })
}
/// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
/// a far end in another source has to live: no GitHub issue relationship can name one.
///
/// Only forwards. The reverse of a recorded edge is derived from the far end, and this
/// source never writes one down.
///
/// The metadata lives in the item's own body slot, and `node` is the dependency read's
/// own answer, which carries an issue's body — so an issue's recorded edges cost no
/// request beyond the read already made, and reading the board for them would be a
/// walk of every item for one field of one. A draft has no body in that answer, because
/// a draft lives only inside the board, so a draft's are read off the board as before.
async fn recorded_edges(
&self,
id: &NativeId,
near_kind: ItemKind,
direction: Direction,
natively_names: Option<ItemKind>,
node: &Value,
) -> Result<Vec<DependencyEdge>, SourceError> {
if direction != Direction::DependsOn {
return Ok(Vec::new());
}
let slot = match node.get("body") {
Some(body) if natively_names.is_some() => {
metadata_body(body.as_str().map(str::to_owned))?.1
}
_ => {
let Some(item) = self
.board()
.await?
.items
.into_iter()
.find(|item| item.id == *id)
else {
return Ok(Vec::new());
};
item.slot
}
};
DependencyEdge::recorded(&slot, id, near_kind, &self.name, natively_names)
.map_err(|message| SourceError::Malformed { message })
}
fn configured_repository(&self) -> Result<&RepositoryTarget, SourceError> {
self.repository
.as_ref()
.ok_or_else(|| SourceError::Refused {
message: format!(
"source {} has no repository configured, and a GitHub Projects board has no \
repository of its own to create an issue in; set repository: owner/name on \
this source",
self.name
),
})
}
/// The repository one new issue is created in, under the rule [`RepositoryTarget`]
/// states.
///
/// The fallback is demanded first, whichever arm answers: a write without a configured
/// repository is refused naming the field exactly as it was before the rule existed,
/// so a source that could not write before cannot write now, rather than writing for
/// the one item whose own field happens to decide it.
///
/// Everything this refuses is refused before `createIssue`, so a refusal leaves no
/// issue behind: an entry that is not a repository on [`RepositoryTarget::HOST`], an
/// entry owned by someone other than the owner of the parent issue's repository —
/// GitHub accepts a sub-issue from another repository of the same owner and from no
/// other, so `addSubIssue` would refuse it after the issue existed — a parent the
/// board does not hold, and a parent that is a draft, which GitHub gives no sub-issues,
/// both of which `addSubIssue` would likewise refuse too late. Whether the entry exists
/// and is visible to the token is checked where its node id is resolved, still before
/// `createIssue`. The parent is read off `board`, which is completed from
/// this process's own record, so a project created moments ago in this command answers
/// though GitHub's board read has not caught up.
fn creation_target(
&self,
board: &Board,
incoming: &Incoming<'_>,
) -> Result<RepositoryTarget, SourceError> {
let fallback = self.configured_repository()?;
let what = |incoming: &Incoming<'_>| {
format!(
"{} {:?}",
incoming.written.kind().describes(),
incoming.title
)
};
let parent = incoming
.parent
.map(|parent| {
board
.items
.iter()
.find(|item| item.id == *parent)
.ok_or_else(|| SourceError::Refused {
message: format!(
"GitHub project issue {} was not found on the board of source {}, \
so {} cannot be filed under it",
parent.0,
self.name,
what(incoming)
),
})
})
.transpose()?;
let parents_repository = parent
.map(|parent| {
// A draft is on the board and so is found, but it has no repository to
// place a task in and GitHub gives it no sub-issues, so `addSubIssue`
// would refuse the task only once `createIssue` had made it.
if parent.content_kind == ContentKind::DraftIssue {
return Err(SourceError::Refused {
message: format!(
"GitHub project item {} on the board of source {} is a draft, \
which cannot have sub-issues, so {} cannot be filed under it",
parent.id.0,
self.name,
what(incoming)
),
});
}
// An issue's repository is where a sub-issue is placed and whose owner it
// is compared against, so a parent whose repository this source cannot
// spell as `owner/name` — GitHub's login grammar is wider than this
// source's floor — is one nothing can be filed under.
parent
.own_repository
.as_ref()
.and_then(|origin| RepositoryTarget::from_origin(origin).ok())
.ok_or_else(|| SourceError::Malformed {
message: format!(
"GitHub project issue {} on the board of source {} is in {}, which \
is not a {}/owner/name repository this source can place {} in",
parent.id.0,
self.name,
parent
.own_repository
.as_ref()
.map_or("no repository", Repository::as_str),
RepositoryTarget::HOST,
what(incoming)
),
})
})
.transpose()?;
match incoming.repositories {
[named] => {
let target =
RepositoryTarget::from_origin(named).map_err(|_| SourceError::Refused {
message: format!(
"{} names repository {}, which is not a {}/owner/name repository \
source {} can create an issue in; name one that is, or name none",
what(incoming),
named.as_str(),
RepositoryTarget::HOST,
self.name
),
})?;
if let Some(parents) = &parents_repository
&& parents.owner != target.owner
{
return Err(SourceError::Refused {
message: format!(
"{} names repository {}, owned by {}, but its project's issue is in \
{}, owned by {}, and GitHub files a sub-issue only in a repository \
of the same owner as its parent issue; name a repository of {}, or \
name none",
what(incoming),
target.slug(),
target.owner,
parents.slug(),
parents.owner,
parents.owner
),
});
}
Ok(target)
}
_ => Ok(parents_repository.unwrap_or_else(|| fallback.clone())),
}
}
/// The node id of the repository `incoming` is being created in, or the refusal naming
/// the item and the repository the token cannot see.
///
/// Resolved once per command per repository; see [`Self::repository_cache`].
async fn repository_id(
&self,
repository: &RepositoryTarget,
incoming: &Incoming<'_>,
) -> Result<String, SourceError> {
if let Some(id) = self.repository_cache()?.get(repository).cloned() {
return Ok(id);
}
let data = self
.graphql(
graphql::REPOSITORY,
json!({"owner":repository.owner,"name":repository.name}),
)
.await?;
let node = data
.get("repository")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Refused {
message: format!(
"GitHub repository {} was not found or is not visible to the token, so {} \
{:?} cannot be created in it",
repository.slug(),
incoming.written.kind().describes(),
incoming.title
),
})?;
let id = required_str(node, "id")?.to_owned();
self.repository_cache()?
.insert(repository.clone(), id.clone());
Ok(id)
}
fn repository_cache(
&self,
) -> Result<std::sync::MutexGuard<'_, BTreeMap<RepositoryTarget, String>>, SourceError> {
self.repository_cache
.lock()
.map_err(|_| SourceError::Unavailable {
message: "this source's record of the destination repository was left \
inconsistent by an earlier failure; next: run the command again"
.into(),
})
}
/// Create or update one board item, whichever kind it is.
async fn write_item(
&self,
incoming: &Incoming<'_>,
target: Option<&NativeId>,
depends_on: &[DependencyEdge],
) -> Result<NativeId, SourceError> {
// Refused before anything is read or written: a task or a project titled the way
// this board spells a document would land as an issue this same source reads back
// as a document, so the field this destination cannot carry is named rather than
// written and silently reclassified.
if let Written::Work(kind, _) = incoming.written
&& incoming.title.starts_with(DESIGN_TITLE_PREFIX)
{
return Err(SourceError::Refused {
message: format!(
"the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
spells a document, so it would read back as one rather than as a {}; \
retitle it, or copy it as a document",
kind.marker(),
self.name,
kind.marker()
),
});
}
let board = match target {
Some(target) => match self.board_for_update(target, incoming, depends_on).await? {
Some(board) => board,
None => self.board().await?,
},
None => self.board().await?,
};
let status_target = incoming
.written
.status()
.map(|status| self.resolved_target(status.category))
.transpose()?;
let column = match (incoming.written.status(), status_target.as_ref()) {
(Some(status), Some(target)) => self.column_for(&board, status, target)?,
_ => None,
};
let existing = target
.map(|target| {
board
.items
.iter()
.find(|item| item.id == *target)
.ok_or_else(|| SourceError::Refused {
message: format!("GitHub destination item {} was not found", target.0),
})
})
.transpose()?;
let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
if content_kind == ContentKind::DraftIssue {
if let (Some(StatusTarget::Terminal(_, _)), Some(status)) =
(status_target.as_ref(), incoming.written.status())
{
return Err(self.closes_a_draft(status.category));
}
if incoming.parent.is_some() {
return Err(SourceError::Refused {
message: "GitHub draft items cannot be a project's sub-issue".into(),
});
}
}
match existing {
Some(item) if content_kind == ContentKind::Issue => {
if item.labels != incoming.labels {
return Err(SourceError::Refused {
message: "GitHub issue labels differ from the labels being written".into(),
});
}
}
_ => {
if !incoming.labels.is_empty() {
return Err(SourceError::Refused {
message: "GitHub items created by this destination carry no labels".into(),
});
}
}
}
// An existing issue is never moved; a new one is created where the rule says. The
// repository the issue really lives in is what the slot below is written against,
// so a single entry that is where the issue is created travels as no key at all,
// and the read side derives it back from the issue.
let (own_repository, creation_target) = match existing {
Some(item) => (item.own_repository.clone(), None),
None => {
let target = self.creation_target(&board, incoming)?;
let origin = Repository::try_from(target.origin())
.map_err(|message| SourceError::Config { message })?;
(Some(origin), Some(target))
}
};
let (native, fallback) = self
.partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
.await?;
let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
let body = compose_body(incoming.content, &slot)?;
// Read before anything is created, for the reason the field below is: a value
// this destination cannot store has to refuse, and refusing after `createIssue`
// would leave an issue behind that nothing asked for. The engine writes a
// qualified id here; a caller handing this key anything else is told so rather
// than having it silently stored as no origin at all.
// llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
let origin = match incoming.metadata.get(ORIGIN_KEY) {
None => "",
Some(Value::String(origin)) => origin.as_str(),
Some(other) => {
return Err(SourceError::Refused {
message: format!(
"{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
is {other}"
),
});
}
};
// Resolved before anything is created: a board that cannot carry the copy origin
// has to refuse the write, and refusing it after `createIssue` would leave an
// issue behind that nothing asked for.
let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
Some(field) => {
if required_str(field, "__typename")? != "ProjectV2Field" {
return Err(SourceError::Refused {
message: format!(
"GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
),
});
}
Some(required_str(field, "id")?.to_owned())
}
None if incoming.metadata.contains_key(ORIGIN_KEY) => {
return Err(SourceError::Refused {
message: format!(
"GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
the board"
),
});
}
None => None,
};
let (content_id, item_id, url) = match existing {
Some(item) => {
self.update_existing(item, incoming, &body, status_target.as_ref())
.await?;
(item.id.clone(), item.item_id.clone(), item.url.clone())
}
None => {
let target = creation_target
.as_ref()
.ok_or_else(|| SourceError::Malformed {
message: "a new item was decided without a repository to create it in"
.into(),
})?;
self.create_and_file_issue(&board, target, incoming, &body)
.await?
}
};
let written_option = column.as_ref().map(|(_, _, name)| name.clone());
let column = column.map(|(field, option, _)| (field, option));
// Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
// then each board field, the parent and the dependencies — and GitHub can fail at
// any of them. Everything this source can refuse *before* the first of those is
// already checked above, so what is left is GitHub itself failing part way. When it
// does over an item this call created, the issue is taken back: a write that
// refused must not leave an item behind that nobody asked for, and one that does
// makes the retry create a second.
let landed = self
.finish_write(
&board,
incoming,
&content_id,
&item_id,
content_kind,
existing,
origin_field.as_deref(),
origin,
column,
status_target.as_ref(),
&native,
)
.await;
if let Err(error) = landed {
if existing.is_none() {
// Best effort, and the write's own failure is what the caller is told: a
// refusal naming the tidy-up would hide why the write failed at all.
let _ = self.delete_issue(&content_id).await;
}
return Err(error);
}
let written_status = match (incoming.written.status(), status_target.as_ref()) {
(Some(_), Some(StatusTarget::Terminal(_, reason))) => {
self.statuses
.status(written_option.as_deref(), true, Some(reason.reason()))
}
(Some(_), Some(StatusTarget::Column(_))) => {
self.statuses.status(written_option.as_deref(), false, None)
}
(Some(status), _) => status.clone(),
(None, _) => Status {
category: StatusCategory::Unknown,
name: "Open".to_owned(),
},
};
// So the rest of this command reads what it just did rather than what the board
// said before it. See `remember_written` for which half takes it.
let remembered = Resolved {
item_id,
id: content_id.clone(),
content_kind,
kind: incoming.written.kind(),
title: incoming.title.to_owned(),
// The visible half of the body this write composed, split back off it the
// way a read splits it — so what this record reports is what a read of the
// same issue reports, rather than the person's text with the metadata slot
// still on the end of it.
body: metadata_body(body.clone())?.0,
raw_body: body.clone(),
// A document has no status of its own; what it reads back as is whatever
// the issue's own state says, which is what a re-read reports.
status: written_status,
option: written_option.or_else(|| existing.and_then(|item| item.option.clone())),
// What `state_input` asked for: closed for a terminal target, open for any other
// status, and the issue's own state left as it was by a document write.
closed: content_kind == ContentKind::Issue
&& match status_target.as_ref() {
Some(StatusTarget::Terminal(_, _)) => true,
Some(_) => false,
None => existing.is_some_and(|item| item.closed),
},
delivers: incoming.delivers.to_vec(),
delivered_by: incoming.delivered_by.to_vec(),
labels: incoming.labels.to_vec(),
parent: incoming.parent.cloned(),
origin: (!origin.is_empty()).then(|| origin.to_owned()),
// In the update path this is the item's own url, read off `existing` where the
// tuple above was bound, so one expression serves both halves.
url,
created_at: existing.and_then(|item| item.created_at),
updated_at: existing.and_then(|item| item.updated_at),
own_repository,
repositories: incoming.repositories.to_vec(),
slot,
board_id: Some(board.id.clone()),
fields: board
.fields
.get("nodes")
.and_then(Value::as_array)
.cloned()
.unwrap_or_default(),
};
self.remember_written(remembered, existing.is_none())?;
Ok(content_id)
}
/// What an update of an item this board already holds needs of the board, read off that
/// item rather than off the board — or `None` when the item cannot say enough, and the
/// board has to be read after all.
///
/// An update needs the item it updates, the board's own id for a field write, the
/// definitions of the `Status` and origin fields it writes, and every same-source far end
/// its edges name. The first three ride along on the item's own node read — its board
/// entry names the board's id, and each field value on it carries the definition of the
/// field it is a value of — and each far end is read by its own node id. So updating one
/// item costs a read of that item rather than every page of the board, which is what a
/// copy naming one member out of many is for.
///
/// **Nothing the item does not say is guessed.** A field this item holds no value of may
/// still be on the board, and a view that read it as absent would refuse a write the
/// board can take, or skip a field write the board needs. So a board this command has
/// already read is used as it is, and an item that does not name its board, holds no
/// value of the origin field, or holds no `Status` value when the write carries a status,
/// sends the write back to reading the board — exactly as every update read it before.
async fn board_for_update(
&self,
target: &NativeId,
incoming: &Incoming<'_>,
depends_on: &[DependencyEdge],
) -> Result<Option<Board>, SourceError> {
if self.board_cache()?.is_some() {
return Ok(None);
}
let Some(existing) = self.item_by_id(target).await? else {
return Ok(None);
};
let Some(board_id) = existing.board_id.clone() else {
return Ok(None);
};
let defines = |name: &str| {
existing
.fields
.iter()
.any(|field| field.get("name").and_then(Value::as_str) == Some(name))
};
if !defines(ORIGIN_FIELD) || (incoming.written.status().is_some() && !defines("Status")) {
return Ok(None);
}
let fields = json!({"nodes": existing.fields, "pageInfo": {"hasNextPage": false}});
let mut items = vec![existing];
for edge in depends_on {
if !edge
.to
.source()
.is_none_or(|source| source == self.name.as_str())
{
continue;
}
// The first colon, for the reason `partition_edges` gives.
let far = NativeId(if edge.to.is_qualified() {
edge.to
.id()
.split_once(':')
.map_or(edge.to.id(), |(_, native)| native)
.to_owned()
} else {
edge.to.id().to_owned()
});
if items.iter().any(|item| item.id == far) {
continue;
}
match self.item_by_id(&far).await? {
Some(item) => items.push(item),
// Refused against the whole board, in the words that refusal has always had.
None => return Ok(None),
}
}
Ok(Some(Board {
id: board_id,
fields,
items,
}))
}
/// Everything a write does after the item exists: its board fields, its parent, and
/// its dependencies.
///
/// Split out of `write_item` so there is one place a failure past the point of no
/// return is caught, rather than a tidy-up repeated at each `?` above.
// llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
// so there is one place a failure past the point of no return is caught, and its
// arguments are exactly the values that tail already had in scope. Bundling them into a
// struct would describe no concept — it would be "the arguments of this function" — and
// would put the whole of `write_item`'s locals behind one more indirection.
#[allow(clippy::too_many_arguments)]
async fn finish_write(
&self,
board: &Board,
incoming: &Incoming<'_>,
content_id: &NativeId,
item_id: &str,
content_kind: ContentKind,
existing: Option<&Resolved>,
origin_field: Option<&str>,
origin: &str,
column: Option<(String, String)>,
status_target: Option<&StatusTarget>,
native: &[String],
) -> Result<(), SourceError> {
if let Some(field_id) = origin_field {
self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
.await?;
}
if let Some((field_id, option_id)) = column {
self.set_item_field(
&board.id,
item_id,
&field_id,
json!({"singleSelectOptionId":option_id}),
)
.await?;
}
if content_kind == ContentKind::Issue
&& matches!(status_target, Some(StatusTarget::Terminal(_, _)))
{
self.update_content(
ContentKind::Issue,
content_id,
json!({"stateInput":state_input(status_target)}),
)
.await?;
}
if content_kind == ContentKind::Issue {
self.reparent(
existing.and_then(|item| item.parent.clone()),
content_id,
incoming.parent,
)
.await?;
// A document takes part in no dependency graph, so writing one neither reads
// nor changes the issue's own `blockedBy` relationships. Reconciling them
// against the empty list a document write carries would *delete* whatever
// relationships a person had made on that issue, which is a write nobody
// asked for.
if incoming.written.kind() != BoardKind::Document {
self.reconcile_blocked_by(content_id, native).await?;
}
}
Ok(())
}
/// Delete one issue, which takes its board item with it.
async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
let data = self
.graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
.await?;
data.pointer("/deleteIssue/repository")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue deletion returned no repository".into(),
})?;
self.forget(id)?;
Ok(())
}
/// Remove one item this copy created, so a copy that could not finish leaves the board
/// as it found it.
///
/// Deleting the issue takes its board item with it, so there is no second mutation to
/// keep in step. An id the board does not hold is not an error: the item is already
/// gone, which is the state this asks for.
async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
let board = self.board().await?;
let Some(item) = board.items.iter().find(|item| item.id == *id) else {
return Ok(());
};
if item.content_kind == ContentKind::DraftIssue {
return Err(SourceError::Refused {
message: format!(
"GitHub item {} is a draft, and this source removes an item by deleting \
its issue; next: remove it from the board by hand",
id.0
),
});
}
let data = self
.graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
.await?;
data.pointer("/deleteIssue/repository")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue deletion returned no repository".into(),
})?;
self.forget(id)?;
Ok(())
}
/// The issue a comment call on `task` is about, or `None` when this board holds no such
/// task.
///
/// Resolved exactly as [`TaskSource::get_task`] resolves it, so the comment verbs and a
/// read of the task cannot disagree about which ids name one: a project or a document of
/// this board is not a task here either.
///
/// A **draft** is a task with nowhere to keep a comment, because GitHub keeps comments on
/// issues and a draft is not one. It is refused rather than answered with an empty page,
/// which would read as a task nobody has commented on yet.
async fn commented_issue(&self, task: &NativeId) -> Result<Option<NativeId>, SourceError> {
let Some(item) = self
.item_by_id(task)
.await?
.filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
else {
return Ok(None);
};
if item.content_kind == ContentKind::DraftIssue {
return Err(SourceError::Refused {
message: format!(
"task {} of source {} is a draft item on the board, and GitHub keeps \
comments on issues alone, so a draft has none to read or write; next: \
convert the draft to an issue on the board, then comment on the issue it \
becomes",
task.0, self.name
),
});
}
Ok(Some(item.id))
}
/// Whether the comment `comment` is one of `issue`'s own.
///
/// Read before an edit or a removal is sent, because GitHub's comment mutations take the
/// comment's id and nothing else: a comment id given against the wrong task would
/// otherwise change a comment on some other issue entirely. An id that names nothing, or
/// names something that is not an issue comment, is a comment this task does not have —
/// which is what GitHub refusing to resolve it means too.
async fn comment_is_on(
&self,
issue: &NativeId,
comment: &NativeId,
) -> Result<bool, SourceError> {
let asked = self
.graphql(graphql::COMMENT_ISSUE, json!({"id":comment.0}))
.await;
let data = match asked {
Ok(data) => data,
Err(error) if unresolvable_node(&error) => return Ok(false),
Err(error) => return Err(error),
};
let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
return Ok(false);
};
if optional_str(node, "__typename")? != Some("IssueComment") {
return Ok(false);
}
let on = node.get("issue").ok_or_else(|| SourceError::Malformed {
message: format!("GitHub issue comment {} names no issue", comment.0),
})?;
Ok(required_str(on, "id")? == issue.0)
}
/// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
async fn partition_edges(
&self,
board: &Board,
near_kind: BoardKind,
near_content: ContentKind,
depends_on: &[DependencyEdge],
) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
let mut native = Vec::new();
let mut fallback = Vec::new();
for edge in depends_on {
let same_source = edge
.to
.source()
.is_none_or(|source| source == self.name.as_str());
// A qualified id's source segment runs to its *first* colon — `GlobalId` and
// `DependencyEndpoint::source` both read it that way — and a native id may hold
// colons of its own, so the far end is everything after that one separator.
// Splitting at the last would truncate `work:urn:task:7` to `7`.
let far_id = if edge.to.is_qualified() {
edge.to
.id()
.split_once(':')
.map_or(edge.to.id(), |(_, native)| native)
} else {
edge.to.id()
};
let far = if same_source {
Some(
board
.items
.iter()
.find(|item| item.id.0 == far_id)
.ok_or_else(|| SourceError::Refused {
message: format!("GitHub dependency item {far_id} was not found"),
})?,
)
} else {
None
};
// The caller says which kind the far end is, and this board holds the far end
// itself, so a disagreement is settled here rather than stored: recorded, the
// wrong kind would read back as a cross-level edge that never existed; written
// natively, it would name a relationship of a different level than the caller
// asked for.
//
// A far end this board holds as a *document* fails the same comparison and is
// refused by the same sentence: `ItemKind` has no document variant because
// nothing may point at one, so no caller can name it correctly and the refusal
// is the only honest answer.
if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
return Err(SourceError::Refused {
message: format!(
"GitHub dependency item {far_id} is a {} of this board, and this item \
names it as a {}; record the kind it is",
disagreeing.kind.describes(),
edge.to.kind.marker()
),
});
}
// A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
// however the far end is spelled — and one classified native here would be
// written nowhere at all, because a draft's native reconciliation never runs.
let native_here = near_content == ContentKind::Issue
&& far.is_some_and(|far| {
far.content_kind == ContentKind::Issue
&& BoardKind::Work(edge.to.kind) == near_kind
});
if native_here {
native.push(far_id.to_owned());
} else {
fallback.push(edge.clone());
}
}
Ok((native, fallback))
}
async fn update_existing(
&self,
item: &Resolved,
incoming: &Incoming<'_>,
body: &Option<String>,
status_target: Option<&StatusTarget>,
) -> Result<(), SourceError> {
let title = incoming.written_title();
let mut fields = match item.content_kind {
ContentKind::DraftIssue => json!({"title":title,"body":body}),
ContentKind::Issue => json!({"title":title,"body":body,
"stateInput":state_input(status_target)}),
};
if matches!(status_target, Some(StatusTarget::Terminal(_, _))) {
fields
.as_object_mut()
.expect("update fields are an object")
.remove("stateInput");
}
self.update_content(item.content_kind, &item.id, fields)
.await
}
/// Update one board item's content with exactly `fields` beside its id, through the
/// mutation its kind takes: `updateIssue` for an issue, `updateProjectV2DraftIssue` for
/// a draft.
///
/// Every input field either mutation leaves out is a field GitHub leaves as it is, which
/// is what lets a narrow write carry the one thing it changes and nothing else.
async fn update_content(
&self,
kind: ContentKind,
id: &NativeId,
fields: Value,
) -> Result<(), SourceError> {
let (operation, id_key, pointer) = match kind {
ContentKind::DraftIssue => (
graphql::UPDATE_DRAFT,
"draftIssueId",
"/updateProjectV2DraftIssue/draftIssue",
),
ContentKind::Issue => (graphql::UPDATE_ISSUE, "id", "/updateIssue/issue"),
};
let mut input = fields;
input[id_key] = json!(id.0);
let data = self.graphql(operation, json!({"input":input})).await?;
let returned = data
.pointer(pointer)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub item update returned no item".into(),
})?;
if required_str(returned, "id")? != id.0 {
return Err(SourceError::Malformed {
message: "GitHub item update returned the wrong item".into(),
});
}
Ok(())
}
/// Creates one issue, files it on the board, and reports what a read of it would say:
/// its content id, its board item id, and the web address GitHub gave it.
///
/// Two calls rather than one: `createIssue` needs a repository and answers with an
/// issue that is on no board, and `addProjectV2ItemById` is what puts it there. A
/// terminal status is not written here: `finish_write` selects its option first and
/// closes the issue after, so a close never lands on an item whose board cannot show it.
///
/// The address comes back here because this is the only place it is known before
/// GitHub's own board read catches up — an item this run created answers the reads
/// that follow it out of the record below, and one remembered without its address
/// would report no location for the rest of the run.
async fn create_and_file_issue(
&self,
board: &Board,
repository: &RepositoryTarget,
incoming: &Incoming<'_>,
body: &Option<String>,
) -> Result<(NativeId, String, Option<String>), SourceError> {
let repository_id = self.repository_id(repository, incoming).await?;
let data = self
.graphql(
graphql::CREATE_ISSUE,
json!({"input":{
"repositoryId":repository_id,"title":incoming.written_title(),"body":body
}}),
)
.await?;
let created = data
.pointer("/createIssue/issue")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue creation returned no issue".into(),
})?;
let content_id = NativeId(required_str(created, "id")?.to_owned());
// Optional although GitHub's schema makes it non-null: the issue exists by now, so
// a response without it is not worth failing a landed write over — the item simply
// reports no location until the board read catches up, which is what it did before.
let url = optional_str(created, "url")?.map(str::to_owned);
// The issue exists from here on, so a failure filing it on the board takes it
// back: an issue in the repository that is on no board is an item nobody asked for
// and nothing here would find again.
let added = match self
.graphql(
graphql::ADD_TO_BOARD,
json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
)
.await
{
Ok(added) => added,
Err(error) => {
let _ = self.delete_issue(&content_id).await;
return Err(error);
}
};
let item = added
.pointer("/addProjectV2ItemById/item")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub board addition returned no project item".into(),
})?;
Ok((content_id, required_str(item, "id")?.to_owned(), url))
}
/// Move one issue under the project it now belongs to, or out of the one it left.
async fn reparent(
&self,
held: Option<NativeId>,
child: &NativeId,
wanted: Option<&NativeId>,
) -> Result<(), SourceError> {
if held.as_ref() == wanted {
return Ok(());
}
if let Some(held) = &held {
self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
.await?;
}
if let Some(wanted) = wanted {
self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
.await?;
}
Ok(())
}
async fn sub_issue(
&self,
operation: &str,
parent: &NativeId,
child: &NativeId,
root: &str,
) -> Result<(), SourceError> {
let data = self
.graphql(
operation,
json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
)
.await?;
let issue =
data.pointer(&format!("/{root}/issue"))
.ok_or_else(|| SourceError::Malformed {
message: "GitHub sub-issue update returned no issue".into(),
})?;
let sub =
data.pointer(&format!("/{root}/subIssue"))
.ok_or_else(|| SourceError::Malformed {
message: "GitHub sub-issue update returned no sub-issue".into(),
})?;
if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
return Err(SourceError::Malformed {
message: "GitHub sub-issue update returned the wrong issues".into(),
});
}
Ok(())
}
async fn reconcile_blocked_by(
&self,
content_id: &NativeId,
native: &[String],
) -> Result<(), SourceError> {
let current = self.native_dependency_ids(content_id).await?;
for (operation, far_id) in current
.iter()
.filter(|id| !native.contains(id))
.map(|id| (graphql::REMOVE_BLOCKED_BY, id))
.chain(
native
.iter()
.filter(|id| !current.contains(id))
.map(|id| (graphql::ADD_BLOCKED_BY, id)),
)
{
let data = self
.graphql(
operation,
json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
)
.await?;
let root = if operation == graphql::ADD_BLOCKED_BY {
"addBlockedBy"
} else {
"removeBlockedBy"
};
let issue =
data.pointer(&format!("/{root}/issue"))
.ok_or_else(|| SourceError::Malformed {
message: "GitHub dependency update returned no issue".into(),
})?;
let blocker = data
.pointer(&format!("/{root}/blockingIssue"))
.ok_or_else(|| SourceError::Malformed {
message: "GitHub dependency update returned no blocking issue".into(),
})?;
if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
{
return Err(SourceError::Malformed {
message: "GitHub dependency update returned the wrong issues".into(),
});
}
}
Ok(())
}
}
/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
enum Reached {
/// An issue this board holds, resolved into everything this source reports about it.
Held(Box<Resolved>),
/// Nothing this board holds: no such node, or a node on some other board.
Nothing,
/// A board draft, which exists only inside the board's own item connection.
Draft,
}
/// What GitHub says when a string is not a node id it can resolve.
///
/// Matched because it is the ordinary answer to a project selector naming a project by its
/// *name*, and reporting that as a failure would make naming one impossible. It is read
/// off the refusal GitHub sent, never guessed from the shape of the string: this source
/// does not define the syntax of a GitHub node id and would be wrong about it.
const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
/// Whether this refusal is GitHub saying the id names no node at all.
fn unresolvable_node(error: &SourceError) -> bool {
matches!(error, SourceError::Refused { message }
if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
}
/// One project name, as a search qualifier which filters on it at the server.
///
/// Quoted so the whole title is one phrase rather than a bag of words, with the two
/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
/// the way it documents. A title matched here is still compared for equality afterwards:
/// the qualifier narrows what the server sends, and this source decides what it names.
fn title_qualifier(name: &str) -> String {
let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
format!("in:title \"{escaped}\"")
}
/// The board, and every item on it this source reports.
#[derive(Clone)]
struct Board {
id: String,
fields: Value,
items: Vec<Resolved>,
}
impl Board {
fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
let nodes = fields
.get("nodes")
.and_then(Value::as_array)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub project fields.nodes is not an array".into(),
})?;
Ok(nodes
.iter()
.find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
}
}
/// One board item, resolved into everything this source reports about it.
#[derive(Clone)]
struct Resolved {
item_id: String,
id: NativeId,
content_kind: ContentKind,
kind: BoardKind,
title: String,
body: Option<String>,
/// The body exactly as GitHub holds it, metadata slot and all, which is what a write
/// that changes the slot alone has to keep byte for byte outside it.
raw_body: Option<String>,
status: Status,
/// The name of the board `Status` option this item sits in, as the board spells it.
option: Option<String>,
/// Whether this item's issue is closed. A draft has no such state and is never closed.
closed: bool,
/// The tasks this one delivers, read out of its slot. Empty for anything not a task.
delivers: Vec<TaskRef>,
/// Every task that delivers this one, read out of its slot. Empty for anything not a
/// task.
delivered_by: Vec<TaskRef>,
labels: Vec<Label>,
parent: Option<NativeId>,
// llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
origin: Option<String>,
url: Option<String>,
created_at: Option<DateTime<Utc>>,
updated_at: Option<DateTime<Utc>>,
own_repository: Option<Repository>,
repositories: Vec<Repository>,
slot: BTreeMap<String, Value>,
/// The node id of the board this item sits on, when the read that reached it said.
board_id: Option<String>,
/// The definition of every board field this item holds a value of, in the shape a read
/// of the board's own `fields` gives one.
///
/// Only the fields this item has a value in: a field it holds nothing of is not here,
/// which says nothing about whether the board has it.
fields: Vec<Value>,
}
impl Resolved {
/// The metadata a caller sees: their own keys, plus the copy origin this source keeps
/// in a field of its own, and none of the five keys that are only an encoding.
///
/// The two delivery keys are left out for every kind, not only for a task: they are
/// the encoding of [`Task::delivers`] and [`Task::delivered_by`], and a project or a
/// document carrying one holds nothing a caller's own metadata could mean by it.
fn metadata(&self) -> BTreeMap<String, Value> {
let mut metadata = self.slot.clone();
metadata.remove(Repository::METADATA_KEY);
metadata.remove(DependencyEdge::RECORDED_KEY);
metadata.remove(ItemKind::METADATA_KEY);
metadata.remove(TaskRef::DELIVERS_KEY);
metadata.remove(TaskRef::DELIVERED_BY_KEY);
if let Some(origin) = &self.origin {
metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
}
metadata
}
/// Where this item is, as a link a reader can open.
///
/// A board is a hosted place and every issue on it has a web address, so that address
/// is what "where is this?" means here — and [`Location::Url`] is what says which kind
/// of place it is, so a reader knows to open it rather than to read a file out. It
/// does not replace or derive from `url`: the field goes on reporting exactly what it
/// reported before, and this says what that address *is*.
///
/// An item GitHub gave no `url` for — a draft has none — reports no location at all
/// rather than a third variant, which is the contract's "the source did not say". An
/// issue this run created is not one of those: its address comes back from the
/// creating mutation, so it is somewhere a reader can open from the moment it exists
/// rather than from whenever the board read catches up.
fn location(&self) -> Option<Location> {
self.url.clone().map(Location::Url)
}
fn task(&self) -> Task {
Task {
id: self.id.clone(),
title: self.title.clone(),
content: self.body.clone(),
status: self.status.clone(),
labels: self.labels.clone(),
project: self.parent.clone(),
url: self.url.clone(),
location: self.location(),
created_at: self.created_at,
updated_at: self.updated_at,
metadata: self.metadata(),
repositories: self.repositories.clone(),
delivers: self.delivers.clone(),
delivered_by: self.delivered_by.clone(),
}
}
fn project(&self) -> Project {
Project {
id: self.id.clone(),
title: self.title.clone(),
content: self.body.clone(),
status: self.status.clone(),
labels: self.labels.clone(),
url: self.url.clone(),
location: self.location(),
created_at: self.created_at,
updated_at: self.updated_at,
metadata: self.metadata(),
repositories: self.repositories.clone(),
}
}
/// The same issue as a document: the project it is filed under, and no status and no
/// dependencies, because a document is not work.
fn document(&self) -> Document {
Document {
id: self.id.clone(),
title: self.title.clone(),
content: self.body.clone(),
project: self.parent.clone(),
labels: self.labels.clone(),
url: self.url.clone(),
location: self.location(),
created_at: self.created_at,
updated_at: self.updated_at,
metadata: self.metadata(),
repositories: self.repositories.clone(),
}
}
}
/// What one write is, and the status that comes with being it.
///
/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
/// status and a task or a project always has one, so "a document carrying a status" and
/// "a task carrying none" are states a write cannot be in rather than states every use
/// site below has to defend against.
enum Written<'a> {
/// A document, which is not work and so has no status at all.
Document,
/// A task or a project, and the status it is being written with.
Work(ItemKind, &'a Status),
}
impl Written<'_> {
/// Which of the board's three kinds this write is.
const fn kind(&self) -> BoardKind {
match self {
Self::Document => BoardKind::Document,
Self::Work(kind, _) => BoardKind::Work(*kind),
}
}
/// The status this write carries. A document carries none, so a write of one says
/// nothing about the issue's open or closed state and selects no board `Status`
/// option.
const fn status(&self) -> Option<&Status> {
match self {
Self::Document => None,
Self::Work(_, status) => Some(status),
}
}
}
/// The item being written, in the one shape all three write methods reach.
struct Incoming<'a> {
written: Written<'a>,
/// The title a person wrote. A document's goes onto the issue with
/// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
title: &'a str,
content: Option<&'a str>,
labels: &'a [Label],
metadata: &'a BTreeMap<String, Value>,
repositories: &'a [Repository],
parent: Option<&'a NativeId>,
/// [`Task::delivers`], already checked. Empty for a project or a document, which is
/// what keeps either key out of their slot.
delivers: &'a [TaskRef],
/// [`Task::delivered_by`], already checked. Empty for a project or a document.
delivered_by: &'a [TaskRef],
}
impl Incoming<'_> {
/// The title this write puts on the issue.
fn written_title(&self) -> String {
match self.written {
Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
Written::Work(..) => self.title.to_owned(),
}
}
}
#[derive(Clone, Copy, PartialEq, Eq)]
enum ContentKind {
DraftIssue,
Issue,
}
/// What one board issue is: a document, or the work an [`ItemKind`] names.
///
/// A type of this source's own rather than an `ItemKind` with a third variant, because
/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
/// document — the contract keeps a document out of that enum deliberately. Holding the
/// board's three answers in one value is what makes every place that asks "which is this?"
/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
/// two thirds of the board.
#[derive(Clone, Copy, PartialEq, Eq)]
enum BoardKind {
/// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
Document,
/// Every other issue, and every draft.
Work(ItemKind),
}
impl BoardKind {
/// How a refusal names this kind to the person reading it.
const fn describes(self) -> &'static str {
match self {
Self::Document => "document",
Self::Work(kind) => kind.marker(),
}
}
}
/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
///
/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
/// the shared cross-source journeys assert one answer to one question, so two sources
/// that disagree about what "carries the label bug" means fail them.
fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
let holds = |name: &String| {
labels
.iter()
.any(|label| label.name.eq_ignore_ascii_case(name))
};
(filter.any_of.is_empty() || filter.any_of.iter().any(holds))
&& filter.all_of.iter().all(holds)
&& !filter.none_of.iter().any(holds)
}
/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
statuses.is_empty() || statuses.contains(&category)
}
/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
///
/// `content` is the item's own prose — the body with this source's trailing metadata
/// comment already taken off — so a search never matches an encoding the author of the
/// issue never wrote.
fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
let terms = query.terms.to_lowercase();
let in_title = title.to_lowercase().contains(&terms);
let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
match query.fields {
TextFields::Title => in_title,
TextFields::Content => in_content,
TextFields::TitleOrContent => in_title || in_content,
}
}
/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
///
/// The project predicate is passed separately because a read narrowed to one project has
/// already answered it by asking *that project* for its own items — and re-applying it
/// there would compare the caller's selector, which may be a project's **name**, against
/// the id of the project that name resolved to, and keep nothing. Every other read passes
/// `query.project` and applies it here, which is what keeps `projects` a predicate this
/// source really does apply.
fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
labels_match(&task.labels, &query.labels)
&& status_matches(task.status.category, &query.statuses)
&& match project {
ProjectFilter::Any => true,
ProjectFilter::Orphans => task.project.is_none(),
ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
}
&& query
.text
.as_ref()
.is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
}
fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
labels_match(&project.labels, &query.labels)
&& status_matches(project.status.category, &query.statuses)
&& query
.text
.as_ref()
.is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
}
/// The same three predicates a task query carries, minus the status filter.
///
/// A document is not work, so it has no status for one to compare against and the query
/// type carries none. The project predicate is the same one — a design issue filed under a
/// project issue is in that project, and one filed under nothing is in none — so it is
/// spelled the same way here rather than answered differently.
fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
labels_match(&document.labels, &query.labels)
&& match project {
ProjectFilter::Any => true,
ProjectFilter::Orphans => document.project.is_none(),
ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
}
&& query
.text
.as_ref()
.is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
}
#[async_trait::async_trait]
impl TaskSource for GitHubProjectsSource {
fn kind(&self) -> &'static str {
KIND
}
fn capabilities(&self) -> Capabilities {
Capabilities {
projects: Support::Native,
documents: Support::Native,
comments: Support::Native,
orphan_tasks: Support::Native,
filter_by_label: Support::Native,
filter_by_status: Support::Native,
search_title: Support::Native,
search_content: Support::Native,
task_dependencies: DependencySupport::BothDirections,
project_dependencies: DependencySupport::BothDirections,
max_page_size: MAX_PAGE_SIZE,
}
}
async fn health(&self) -> Result<Health, SourceError> {
let board = self.board_page(None, 1).await?;
Ok(Health {
reachable: true,
detail: Some(format!(
"reading GitHub project {}/{} ({})",
self.owner,
self.project_number,
required_str(&board, "title")?
)),
})
}
async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
Ok(self
.item_by_id(id)
.await?
.filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
.map(|item| item.task()))
}
async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
Ok(self
.item_by_id(id)
.await?
.filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
.map(|item| item.project()))
}
async fn query_tasks(
&self,
query: &TaskQuery,
page: &PageRequest,
) -> Result<Page<Task>, SourceError> {
validate_page(page)?;
// A read narrowed to one project asks that project for its own tasks, so nothing
// about it costs what the rest of the board holds. Every other task read is a
// question about the whole board and is answered by reading it.
let (held, membership) = match &query.project {
ProjectFilter::Is(project) => (
self.project_children(project).await?,
// Answered by where these items came from; see `task_matches`.
&ProjectFilter::Any,
),
ProjectFilter::Any | ProjectFilter::Orphans => {
(self.board().await?.items, &query.project)
}
};
// Filtered before paged: a page of a filtered result is a page of the survivors,
// never the survivors of a page.
let tasks = held
.iter()
.filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
.map(Resolved::task)
.filter(|task| task_matches(task, query, membership))
.collect();
Ok(offset_page(
tasks,
numeric_cursor(page.cursor.as_ref())?,
page.limit.min(MAX_PAGE_SIZE) as usize,
))
}
async fn query_projects(
&self,
query: &ProjectQuery,
page: &PageRequest,
) -> Result<Page<Project>, SourceError> {
validate_page(page)?;
// The projects a board holds are found by an issue search scoped to that board,
// never by walking the board's own item connection: what tells a project from a
// task is the `parent` each issue carries, which costs nothing to read.
let projects = self
.board_issues()
.await?
.iter()
.filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
.map(Resolved::project)
.filter(|project| project_matches(project, query))
.collect();
Ok(offset_page(
projects,
numeric_cursor(page.cursor.as_ref())?,
page.limit.min(MAX_PAGE_SIZE) as usize,
))
}
async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
Ok(self
.item_by_id(id)
.await?
.filter(|item| item.kind == BoardKind::Document)
.map(|item| item.document()))
}
async fn query_documents(
&self,
query: &DocumentQuery,
page: &PageRequest,
) -> Result<Page<Document>, SourceError> {
validate_page(page)?;
// Narrowed to one project, this is the same sub-issue read a task list scoped to
// that project makes — a document filed under a project is a sub-issue of it too,
// and which of them come back is the kind this caller asked for.
let (held, membership) = match &query.project {
ProjectFilter::Is(project) => (
self.project_children(project).await?,
// Answered by where these items came from; see `task_matches`.
&ProjectFilter::Any,
),
ProjectFilter::Any | ProjectFilter::Orphans => {
(self.board().await?.items, &query.project)
}
};
// Filtered before paged, exactly as a task read is: a page of a filtered result is
// a page of the survivors, never the survivors of a page.
let documents = held
.iter()
.filter(|item| item.kind == BoardKind::Document)
.map(Resolved::document)
.filter(|document| document_matches(document, query, membership))
.collect();
Ok(offset_page(
documents,
numeric_cursor(page.cursor.as_ref())?,
page.limit.min(MAX_PAGE_SIZE) as usize,
))
}
async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
validate_page(page)?;
let offset = numeric_cursor(page.cursor.as_ref())?;
let mut labels = self
.board()
.await?
.items
.into_iter()
.flat_map(|item| item.labels)
.fold(Vec::new(), |mut all, label| {
if !all.iter().any(|x: &Label| x.id == label.id) {
all.push(label);
}
all
});
labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
Ok(offset_page(
labels,
offset,
page.limit.min(MAX_PAGE_SIZE) as usize,
))
}
async fn task_dependencies(
&self,
id: &NativeId,
direction: Direction,
page: &PageRequest,
) -> Result<Page<DependencyEdge>, SourceError> {
self.dependencies(id, ItemKind::Task, direction, page).await
}
async fn project_dependencies(
&self,
id: &NativeId,
direction: Direction,
page: &PageRequest,
) -> Result<Page<DependencyEdge>, SourceError> {
self.dependencies(id, ItemKind::Project, direction, page)
.await
}
fn writes(&self) -> WriteSupport {
WriteSupport::Supported
}
/// Create or update one task.
///
/// Its `delivers` and `delivered_by` are checked before anything is read or written —
/// neither may name the task itself or name one task twice — and land in the body's
/// metadata slot under their reserved keys, in place of any caller metadata of those
/// names.
async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
let near = write.target.as_ref().unwrap_or(&write.item.id);
for (key, entries) in [
(TaskRef::DELIVERS_KEY, &write.item.delivers),
(TaskRef::DELIVERED_BY_KEY, &write.item.delivered_by),
] {
TaskRef::listed(key, near, Some(&self.name), entries.clone())
.map_err(|message| SourceError::Refused { message })?;
}
self.write_item(
&Incoming {
written: Written::Work(ItemKind::Task, &write.item.status),
title: &write.item.title,
content: write.item.content.as_deref(),
labels: &write.item.labels,
metadata: &write.item.metadata,
repositories: &write.item.repositories,
parent: write.item.project.as_ref(),
delivers: &write.item.delivers,
delivered_by: &write.item.delivered_by,
},
write.target.as_ref(),
&write.depends_on,
)
.await
}
async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
self.write_item(
&Incoming {
written: Written::Work(ItemKind::Project, &write.item.status),
title: &write.item.title,
content: write.item.content.as_deref(),
labels: &write.item.labels,
metadata: &write.item.metadata,
repositories: &write.item.repositories,
parent: None,
delivers: &[],
delivered_by: &[],
},
write.target.as_ref(),
&write.depends_on,
)
.await
}
/// Create or update one document, which is one issue titled the way this board spells
/// a document.
///
/// Everything else is exactly a task write: caller metadata goes to the same canonical
/// JSON slot at the end of the body and comes back with its JSON types intact, a key
/// or a field this board cannot carry is refused by name rather than dropped, a target
/// naming an issue this board does not hold is refused rather than created, and an
/// issue this call created is taken back when the rest of the write fails.
async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
// A document takes part in no dependency graph, so there is no far end to write
// natively and none to record: a caller naming one is told so rather than having it
// stored under the reserved key, where a later read would report an edge the
// contract says cannot exist.
if !write.depends_on.is_empty() {
return Err(SourceError::Refused {
message: format!(
"this write names {} dependencies for a document, and a document takes \
part in no dependency graph; next: put the dependency on the task or \
project the document is about",
write.depends_on.len()
),
});
}
self.write_item(
&Incoming {
written: Written::Document,
title: &write.item.title,
content: write.item.content.as_deref(),
labels: &write.item.labels,
metadata: &write.item.metadata,
repositories: &write.item.repositories,
parent: write.item.project.as_ref(),
delivers: &[],
delivered_by: &[],
},
write.target.as_ref(),
&[],
)
.await
}
/// Set one task's status alone.
///
/// An open target reopens a closed issue with an `updateIssue` carrying only its
/// `stateInput`, then selects the board option with `updateProjectV2ItemFieldValue`; a
/// terminal target selects its mapped option, then closes with its fixed reason. No
/// request carries a title, a body or a label. The status
/// answered is what [`StatusMapping::status`] reads off the state just written, which is
/// what a re-read reports.
async fn set_task_status(
&self,
id: &NativeId,
category: StatusCategory,
) -> Result<Option<Status>, SourceError> {
self.set_status(id, category).await
}
/// Replace one task's `delivered_by` with a single body update that changes the
/// metadata slot and nothing outside it.
async fn set_delivered_by(
&self,
id: &NativeId,
delivered_by: &[TaskRef],
) -> Result<Option<()>, SourceError> {
self.replace_delivered_by(id, delivered_by).await
}
/// Set one key of one task issue's metadata with a single body update that changes the
/// metadata slot and nothing outside it — no title, label, state or board field request —
/// and sends nothing when the task already holds that value under the key.
async fn set_task_metadata(
&self,
id: &NativeId,
key: &MetadataKey,
value: &Value,
) -> Result<Option<Task>, SourceError> {
Ok(self
.set_slot_key(id, BoardKind::Work(ItemKind::Task), key, value)
.await?
.map(|item| item.task()))
}
/// Set one key of one project issue's metadata, on exactly the terms of
/// [`set_task_metadata`](TaskSource::set_task_metadata).
async fn set_project_metadata(
&self,
id: &NativeId,
key: &MetadataKey,
value: &Value,
) -> Result<Option<Project>, SourceError> {
Ok(self
.set_slot_key(id, BoardKind::Work(ItemKind::Project), key, value)
.await?
.map(|item| item.project()))
}
/// Set one key of one design-document issue's metadata, on exactly the terms of
/// [`set_task_metadata`](TaskSource::set_task_metadata).
async fn set_document_metadata(
&self,
id: &NativeId,
key: &MetadataKey,
value: &Value,
) -> Result<Option<Document>, SourceError> {
Ok(self
.set_slot_key(id, BoardKind::Document, key, value)
.await?
.map(|item| item.document()))
}
async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
self.delete_item(id).await
}
async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
self.delete_item(id).await
}
async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
self.delete_item(id).await
}
/// One page of the task issue's own comments, walked by GitHub's own cursor.
///
/// Nothing here filters, so nothing has to be read ahead of the page: the caller's limit is
/// the page GitHub is asked for and GitHub's `endCursor` is the cursor handed back.
async fn task_comments(
&self,
task: &NativeId,
page: &PageRequest,
) -> Result<Option<Page<Comment>>, SourceError> {
validate_page(page)?;
let Some(issue) = self.commented_issue(task).await? else {
return Ok(None);
};
let after = page.cursor.as_ref().map(|cursor| cursor.0.as_str());
let data = self
.graphql(
graphql::ISSUE_COMMENTS,
json!({"id":issue.0,"first":page.limit.min(MAX_PAGE_SIZE),"after":after}),
)
.await?;
// The issue was there a moment ago; one removed since is no longer a task here.
let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
return Ok(None);
};
let connection = node
.get("comments")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: format!(
"GitHub issue {} answered with no comments connection",
issue.0
),
})?;
let items = optional_nodes(Some(connection), "issue comments")?
.into_iter()
.flatten()
.map(comment_from)
.collect::<Result<Vec<_>, _>>()?;
let next = next_cursor(connection)?;
if let Some(next) = &next {
validate_cursor_progress(after, &next.0)?;
}
Ok(Some(Page { items, next }))
}
/// Add one comment to the task's issue, as the account the token belongs to.
///
/// The author is refused before anything is sent — not even the task is read — because
/// no answer GitHub could give would make posting under another name than the one asked
/// for the right outcome.
async fn add_comment(
&self,
task: &NativeId,
comment: &NewComment,
) -> Result<Option<Comment>, SourceError> {
if let Some(author) = &comment.author {
return Err(SourceError::Refused {
message: format!(
"source {} cannot post a comment as {author:?}: GitHub records the account \
the token signs in as the author of every comment; next: leave --author \
out, and the comment is posted as that account",
self.name
),
});
}
let Some(issue) = self.commented_issue(task).await? else {
return Ok(None);
};
let data = self
.graphql(
graphql::ADD_COMMENT,
json!({"input":{"subjectId":issue.0,"body":comment.body.as_str()}}),
)
.await?;
let subject = data
.pointer("/addComment/subject")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub comment addition returned no subject".into(),
})?;
if required_str(subject, "id")? != issue.0 {
return Err(SourceError::Malformed {
message: "GitHub comment addition answered about another issue".into(),
});
}
let added = data
.pointer("/addComment/commentEdge/node")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub comment addition returned no comment".into(),
})?;
comment_from(added).map(Some)
}
async fn edit_comment(
&self,
task: &NativeId,
comment: &NativeId,
body: &CommentBody,
) -> Result<Option<Comment>, SourceError> {
let Some(issue) = self.commented_issue(task).await? else {
return Ok(None);
};
if !self.comment_is_on(&issue, comment).await? {
return Ok(None);
}
let data = self
.graphql(
graphql::UPDATE_COMMENT,
json!({"input":{"id":comment.0,"body":body.as_str()}}),
)
.await?;
let edited = data
.pointer("/updateIssueComment/issueComment")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub comment update returned no comment".into(),
})?;
let edited = comment_from(edited)?;
if edited.id != *comment {
return Err(SourceError::Malformed {
message: "GitHub comment update returned the wrong comment".into(),
});
}
Ok(Some(edited))
}
async fn delete_comment(
&self,
task: &NativeId,
comment: &NativeId,
) -> Result<Option<NativeId>, SourceError> {
let Some(issue) = self.commented_issue(task).await? else {
return Ok(None);
};
if !self.comment_is_on(&issue, comment).await? {
return Ok(None);
}
let data = self
.graphql(graphql::DELETE_COMMENT, json!({"input":{"id":comment.0}}))
.await?;
// The payload says nothing about the comment it removed, so what is checked is that
// GitHub answered the mutation at all rather than leaving it unanswered.
data.get("deleteIssueComment")
.filter(|value| !value.is_null())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub comment deletion returned no payload".into(),
})?;
Ok(Some(comment.clone()))
}
/// Every request this source has recorded, and what each of GitHub's two budgets was
/// attributed — read off the same accounting the session report is rendered from, so
/// the two cannot count one request two ways.
async fn metering(&self) -> Result<Option<Metering>, SourceError> {
Ok(Some(self.ledger.snapshot().metering()))
}
}
/// One issue comment as the contract carries it.
///
/// `author` is absent both when GitHub answers `null` for an account that no longer exists
/// and when it answers an actor with no login, because either way the source did not say who
/// wrote it — which is what an absent author means, rather than an author called nothing.
fn comment_from(value: &Value) -> Result<Comment, SourceError> {
Ok(Comment {
id: NativeId(required_str(value, "id")?.to_owned()),
author: optional_str(value.get("author").unwrap_or(&Value::Null), "login")?
.map(str::to_owned),
created_at: optional_time(value, "createdAt")?,
updated_at: optional_time(value, "updatedAt")?,
body: required_str(value, "body")?.to_owned(),
url: optional_str(value, "url")?.map(str::to_owned),
})
}
/// Where the recorded tail of a dependency walk resumes; see
/// [`GitHubProjectsSource::recorded_edges`].
const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
/// The board text field this source keeps a copy's origin in.
///
/// Named after the key it holds, and held to that name by the guard below rather than by
/// a reader noticing.
const ORIGIN_FIELD: &str = "onetaskgraph.origin";
/// The metadata key that field holds.
///
/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
/// constructs or interprets the qualified id it carries. This source names it only to
/// route it — a short, typed value belongs in a typed field rather than in the body slot
/// a caller's own prose shares.
///
/// Restated rather than imported, because no plugin crate may depend on the engine. What
/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
/// target in `check`: it reads the engine's own literal and fails naming the file and the
/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
/// that creates a second item every run instead of finding the one it wrote — and that is
/// too late to learn it.
const ORIGIN_KEY: &str = "onetaskgraph.origin";
/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
///
/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
/// is derived from the far end, never written down on the near item — so only a forward
/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
/// it did not come from, and it is told so rather than answered with an empty page that
/// reads as a walk which ended.
fn recorded_offset(
cursor: Option<&str>,
direction: Direction,
) -> Result<Option<usize>, SourceError> {
cursor
.and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
.map(|offset| {
if direction != Direction::DependsOn {
return Err(SourceError::Config {
message: format!(
"{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
reverse dependency read never issues; resume it in the direction \
that reported it"
),
});
}
offset.parse().map_err(|_| SourceError::Config {
message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
})
})
.transpose()
}
fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
let mut page = offset_page(edges, offset, limit.max(1));
page.next = page
.next
.map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
page
}
/// The kind of one issue reached through a dependency connection.
///
/// The same questions the board scan asks, over the fields the dependency document
/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
/// then anything with sub-issues or the marker is a project.
///
/// # Errors
///
/// A far end this board holds as a document is refused rather than reported. The two
/// answers that are not refusals would both be wrong: reporting it as a task names an id
/// no task read of this source can find, and reporting it as a project names one no
/// project read can. There is no third value to return — `ItemKind` has no document
/// variant, because nothing may point at a document — so the relationship itself is what
/// the person is told about.
fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
let id = required_str(value, "id")?;
if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
return Err(SourceError::Refused {
message: format!(
"GitHub issue {id} is a document of this board — its title begins \
{DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
on by one; next: remove that issue's blocking relationship on this board"
),
});
}
let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
if parent.is_some() {
return Ok(ItemKind::Task);
}
let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
message: format!("GitHub issue {id}: {message}"),
})?;
let sub_issues = sub_issue_total(value)?;
Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
ItemKind::Project
} else {
ItemKind::Task
})
}
/// The `IssueStateUpdateInput` one status target asks for.
///
/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
/// a currently-closed issue: without that the item would read back `Unknown` and a copy
/// would report a change forever. A document has no status at all, and asks for neither.
fn state_input(target: Option<&StatusTarget>) -> Value {
match target {
Some(StatusTarget::Terminal(_, reason)) => {
json!({"value":"CLOSED","stateReason":reason.reason()})
}
Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
// A document has no status, so a write of one says nothing about the issue's open
// or closed state rather than forcing it open: `stateInput` is what carries that
// instruction, and an explicit null asks for no change to it.
None => Value::Null,
}
}
/// The metadata one write stores in the item's body slot.
///
/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
/// rather than carried: the kind marker so an empty project stays readable, the
/// repository list only when it is not exactly the issue's own repository, and the far
/// ends no relationship here can name.
fn slot_metadata(
incoming: &Incoming<'_>,
own_repository: Option<&Repository>,
fallback: &[DependencyEdge],
) -> BTreeMap<String, Value> {
let mut metadata = incoming.metadata.clone();
metadata.remove(ORIGIN_KEY);
match incoming.written.kind() {
BoardKind::Work(kind) => metadata.insert(
ItemKind::METADATA_KEY.to_owned(),
Value::String(kind.marker().to_owned()),
),
// A document is told by its title, so it carries no kind marker: that key names
// what a dependency endpoint points at, and nothing may point at a document.
BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
};
let derivable = own_repository
.map(|own| incoming.repositories == [own.clone()])
.unwrap_or(incoming.repositories.is_empty());
if derivable {
metadata.remove(Repository::METADATA_KEY);
} else {
metadata.insert(
Repository::METADATA_KEY.to_owned(),
Value::Array(
incoming
.repositories
.iter()
.map(|repository| Value::String(repository.as_str().to_owned()))
.collect(),
),
);
}
// The typed lists are what land, whatever the caller's own metadata held under their
// keys: a key of either name travelling beside the field would otherwise be a second
// answer to the same question, and the field is the one the contract names.
for (key, entries) in [
(TaskRef::DELIVERS_KEY, incoming.delivers),
(TaskRef::DELIVERED_BY_KEY, incoming.delivered_by),
] {
set_task_list(&mut metadata, key, entries);
}
if fallback.is_empty() {
metadata.remove(DependencyEdge::RECORDED_KEY);
} else {
metadata.insert(
DependencyEdge::RECORDED_KEY.to_owned(),
Value::Array(
fallback
.iter()
.map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
.collect(),
),
);
}
metadata
}
/// Every label one item carries, from its content's own connection and nowhere else.
///
/// There is no second place to read one from: no document this source sends selects the
/// board's built-in `Labels` field, because GitHub derives it from the content and a draft
/// cannot carry one at all. The module documentation records the three schema facts that
/// settle it.
fn labels(content: &Value) -> Result<Vec<Label>, SourceError> {
optional_nodes(content.get("labels"), "content labels")?
.into_iter()
.flatten()
.map(|v| {
Ok(Label {
id: NativeId(required_str(v, "id")?.to_owned()),
name: required_str(v, "name")?.to_owned(),
color: optional_str(v, "color")?.map(str::to_owned),
})
})
.collect()
}
/// The definition of each board field one item's values are values of, in the shape a read
/// of the board's own `fields` gives one.
///
/// A value names its field through a fragment on that field's own type, so the type is
/// known from which kind of value it is: a single-select value's field is a
/// `ProjectV2SingleSelectField`, options and all, and a text value's is a `ProjectV2Field`.
/// A value whose field carried no id, or an empty one, says nothing usable and is left out.
fn field_definitions(field_values: &[Value]) -> Vec<Value> {
field_values
.iter()
.filter_map(|value| {
let field = value.get("field")?.as_object()?;
field.get("id")?.as_str().filter(|id| !id.is_empty())?;
let typename = if value.get("text").is_some() {
"ProjectV2Field"
} else if value.get("name").is_some() {
"ProjectV2SingleSelectField"
} else {
return None;
};
let mut defined = field.clone();
defined.insert("__typename".to_owned(), json!(typename));
Some(Value::Object(defined))
})
.collect()
}
fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
let Some(node) = field_values
.iter()
.find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
else {
return Ok(None);
};
Ok(optional_str(node, "text")?.map(str::to_owned))
}
fn valid_github_owner(owner: &str) -> bool {
!owner.is_empty()
&& owner.len() <= 39
&& !owner.starts_with('-')
&& !owner.ends_with('-')
&& !owner.contains("--")
&& owner
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
}
/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
/// neither of the two names a path segment already means.
fn valid_github_repository_name(name: &str) -> bool {
!name.is_empty()
&& name.len() <= 100
&& name != "."
&& name != ".."
&& name
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
}
fn valid_environment_name(name: &str) -> bool {
let mut bytes = name.bytes();
bytes
.next()
.is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
&& bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
}
/// How many sub-issues one issue has.
///
/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
/// absent or non-integer one is a response this source cannot read — and reading it as
/// zero would classify a project as a task, which is exactly the mistake the marker
/// exists to keep from happening quietly.
fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
let summary = issue
.get("subIssuesSummary")
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue is missing subIssuesSummary".into(),
})?;
summary
.get("total")
.and_then(Value::as_u64)
.ok_or_else(|| SourceError::Malformed {
message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
})
}
fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
value
.get(field)
.and_then(Value::as_str)
.ok_or_else(|| SourceError::Malformed {
message: format!("GitHub response is missing string field {field}"),
})
}
fn required_nonblank_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
let found = required_str(value, field)?;
if found.trim().is_empty() {
return Err(SourceError::Malformed {
message: format!("GitHub response has blank string field {field}"),
});
}
Ok(found)
}
/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
/// needs one — Linear spells them too, in its own description field.
///
/// Restated rather than shared, because a plugin crate depends on the contract crate and
/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
/// source round-trips its own writes perfectly well under its own spelling.
const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
const METADATA_CLOSE: &str = "\n-->";
/// The visible body and the metadata slot at the end of it.
///
/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
/// own content and is left alone.
fn metadata_body(
body: Option<String>,
) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
let Some(body) = body else {
return Ok((None, BTreeMap::new()));
};
let Some(slot) = slot_span(&body)? else {
return Ok((Some(body), BTreeMap::new()));
};
let metadata =
serde_json::from_str(&body[slot.encoded_start..slot.encoded_end]).map_err(|error| {
SourceError::Malformed {
message: format!(
"invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
),
}
})?;
let visible = body[..slot.start].trim_end();
Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
}
/// Where the metadata slot sits in one body, as byte offsets into it.
struct SlotSpan {
/// Where [`METADATA_OPEN`] begins.
start: usize,
/// Where the encoded JSON begins, just past [`METADATA_OPEN`].
encoded_start: usize,
/// Where the encoded JSON ends, at the start of [`METADATA_CLOSE`].
encoded_end: usize,
/// Just past [`METADATA_CLOSE`].
end: usize,
}
/// The slot at the very end of `body`, or `None` when it has none.
///
/// The one reading of *where the slot is*, shared by [`metadata_body`], which reads it, and
/// [`with_slot`], which rewrites it — so the two cannot disagree about which comment is the
/// slot.
fn slot_span(body: &str) -> Result<Option<SlotSpan>, SourceError> {
let Some(start) = body.rfind(METADATA_OPEN) else {
return Ok(None);
};
let encoded_start = start + METADATA_OPEN.len();
let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
return Err(SourceError::Malformed {
message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
});
};
let encoded_end = encoded_start + relative_end;
let end = encoded_end + METADATA_CLOSE.len();
if !body[end..].trim().is_empty() {
return Ok(None);
}
Ok(Some(SlotSpan {
start,
encoded_start,
encoded_end,
end,
}))
}
/// `body` with its metadata slot holding exactly `metadata`, and every byte outside the
/// slot as it was.
///
/// A slot that is there has its JSON replaced in place; one that becomes empty is removed
/// together with the one `"\n\n"` separating it from the prose before it. A body with no
/// slot gains one the way [`compose_body`] writes it — after a `"\n\n"`, or alone in an empty
/// body — and a body with no slot that is given no metadata is returned as it is.
fn with_slot(body: &str, metadata: &BTreeMap<String, Value>) -> Result<String, SourceError> {
let encoded = if metadata.is_empty() {
None
} else {
Some(
serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
message: error.to_string(),
})?,
)
};
Ok(match (slot_span(body)?, encoded) {
(Some(slot), Some(encoded)) => format!(
"{}{encoded}{}",
&body[..slot.encoded_start],
&body[slot.encoded_end..]
),
(Some(slot), None) => {
let before = &body[..slot.start];
format!(
"{}{}",
before.strip_suffix("\n\n").unwrap_or(before),
&body[slot.end..]
)
}
(None, None) => body.to_owned(),
(None, Some(encoded)) if body.is_empty() => {
format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
}
(None, Some(encoded)) => format!("{body}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}"),
})
}
/// Hold `entries` under `key` in one slot's metadata, or no such key when there are none.
fn set_task_list(metadata: &mut BTreeMap<String, Value>, key: &str, entries: &[TaskRef]) {
if entries.is_empty() {
metadata.remove(key);
} else {
metadata.insert(
key.to_owned(),
Value::Array(
entries
.iter()
.map(|entry| Value::String(entry.as_str().to_owned()))
.collect(),
),
);
}
}
fn compose_body(
content: Option<&str>,
metadata: &BTreeMap<String, Value>,
) -> Result<Option<String>, SourceError> {
let visible = content.unwrap_or_default();
if metadata.is_empty() {
return Ok((!visible.is_empty()).then(|| visible.to_owned()));
}
let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
message: error.to_string(),
})?;
Ok(Some(if visible.is_empty() {
format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
} else {
format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
}))
}
fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
value
.get(field)
.and_then(Value::as_bool)
.ok_or_else(|| SourceError::Malformed {
message: format!("GitHub response is missing boolean field {field}"),
})
}
fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
match value.get(field) {
None | Some(Value::Null) => Ok(None),
Some(value) => value
.as_str()
.map(Some)
.ok_or_else(|| SourceError::Malformed {
message: format!("GitHub response field {field} is not a string or null"),
}),
}
}
fn optional_nodes<'a>(
connection: Option<&'a Value>,
name: &str,
) -> Result<Option<&'a Vec<Value>>, SourceError> {
match connection {
None | Some(Value::Null) => Ok(None),
Some(value) => value
.get("nodes")
.and_then(Value::as_array)
.map(Some)
.ok_or_else(|| SourceError::Malformed {
message: format!("GitHub {name}.nodes is not an array"),
}),
}
}
fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
let page_info = connection
.get("pageInfo")
.ok_or_else(|| SourceError::Malformed {
message: format!("GitHub {name} has no pageInfo"),
})?;
if required_bool(page_info, "hasNextPage")? {
return Err(SourceError::Malformed {
message: format!(
"GitHub {name} exceeds the supported nested connection size of {size}"
),
});
}
Ok(())
}
fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
optional_str(value, field)?
.map(|timestamp| {
timestamp.parse().map_err(|error| SourceError::Malformed {
message: format!("GitHub response field {field} is not a timestamp: {error}"),
})
})
.transpose()
}
fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
if page.limit == 0 {
Err(SourceError::Config {
message: "page limit must be at least 1".into(),
})
} else {
Ok(())
}
}
fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
let page = connection
.get("pageInfo")
.filter(|value| value.is_object())
.ok_or_else(|| SourceError::Malformed {
message: "GitHub connection is missing pageInfo".into(),
})?;
if required_bool(page, "hasNextPage")? {
let cursor = required_str(page, "endCursor")?;
validate_cursor_progress(None, cursor)?;
Ok(Some(Cursor(cursor.into())))
} else {
Ok(None)
}
}
fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
if next.is_empty() || previous == Some(next) {
Err(SourceError::Malformed {
message: "GitHub pagination cursor is empty or did not advance".into(),
})
} else {
Ok(())
}
}
fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
cursor.map_or(Ok(0), |c| {
c.0.parse().map_err(|_| SourceError::Config {
message: "page cursor is invalid".into(),
})
})
}
fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
if offset > items.len() {
return Page::last(vec![]);
}
let tail = items.split_off(offset);
let mut selected = tail;
let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
selected.truncate(limit);
Page {
items: selected,
next,
}
}