onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so a write without [`GitHubProjectsConfig::repository`] is refused
11//! naming the field — but that repository is the *fallback*, not the home of every item.
12//!
13//! <!-- llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] The rule's one
14//! executable source is `GitHubProjectsSource::creation_target`; this is where a reader of
15//! the module meets it, and `tests/plugin.rs` drives every arm below against the loopback
16//! board and asserts on `createIssue`'s own `repositoryId`, so the prose cannot outlive a
17//! change to the rule. -->
18//! **Which repository an issue is created in is decided by the item's own `repositories`
19//! field, under one rule.** Exactly one entry names the repository the issue is created in:
20//! a task issue is where a person finds the work from the repository it changes, and one
21//! filed in a board's nominated repository is invisible from every other. Zero entries, or
22//! two or more, name none, so a task's or a document's issue is created in the repository
23//! its parent project's issue lives in — read from the board, or from this process's own
24//! record of a project it created earlier in the same command — and a project's issue, or
25//! a task or document written with no parent, is created in the configured `repository:`.
26//! What that rule refuses, it refuses before `createIssue`, so no issue is half-created. An
27//! existing issue is never moved: the update path leaves the issue where it is and records
28//! the list in the metadata slot when it differs, so the read side's derivation and the
29//! creation rule agree by construction.
30//! <!-- llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate] -->
31//!
32//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
33//! board has no document type and nothing but issues to hold one in, so the title is the
34//! discriminator and it is the whole of it. The title this source *reports* is the one a
35//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
36//! the body so `content` is what the person wrote — and writing a document puts the prefix
37//! back, so a round trip returns the title that went in.
38//!
39//! **Telling a document from a project from a task.** The design prefix is read **first**:
40//! a document is never a project and never a task, whatever sub-issues it has or does not
41//! have. Only then does the rest apply — a board issue is a project when *either* it has
42//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
43//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
44//! necessary: it is what makes an *empty* project — the state a project copy passes
45//! through between creating the project and filing its first task — readable as a
46//! project, while the sub-issue arm lets a person author a project on the board by hand
47//! with no knowledge of this product's metadata at all. Reading the prefix later than the
48//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
49//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
50//! task nor a document and are ignored.
51//!
52//! **Where an entity is, is a link.** Every project, task and document this source reports
53//! carries a [`Location::Url`] naming the issue's own web address — the same address the
54//! `url` field already reports, in the shape that says a reader can open it. That is the
55//! contrast the location contract exists for: a reader holding an entity from this source
56//! is handed something to link to and one holding an entity from a folder of Markdown is
57//! handed a path, and neither has to know which plugin answered. It does not replace or
58//! derive from `url`; that field goes on reporting what it always reported.
59//!
60//! **Where metadata lives.** Short typed things go to typed fields and native relations:
61//! status to the board's `Status` single-select and the issue's own state, the copy
62//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
63//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
64//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
65//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
66//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
67//! why neither can hold a caller's own prose.
68//!
69//! **Status.** `status_mapping` is per-instance configuration from a status category to
70//! `null`, a board `Status` option name, or a closed state of `completed` or
71//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
72//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
73//! and a mistake destroys every item's status. A status this board cannot represent is a
74//! refusal naming the status and the instance instead.
75//!
76//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
77//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
78//! finished tasks were only moved to a "Done" column would read 0% complete forever.
79//!
80//! # What this source declares, field by field
81//!
82//! One verdict per field of [`Capabilities`], and what `Native` means when this source
83//! says it. *Proven* means a shared journey drives it against the real
84//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
85//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
86//! [`capabilities`](TaskSource::capabilities) from parting.
87//!
88//! | Field | Verdict |
89//! | --- | --- |
90//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
91//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
92//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
93//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
94//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
95//! | `search_title` | **Supported and proven,** over `Issue.title`. |
96//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
97//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
98//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
99//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
100//!
101//! Nothing here is unsupported. `documents` is not a predicate — it says this source has
102//! documents, which it does — and the three facts behind the uniform `Native` on the
103//! predicates beside it are recorded below rather than re-derived, because a reader who
104//! takes `Native` to mean *the remote service filters* will read that uniformity as a
105//! lie.
106//!
107//! First, the plugin contract defines `Support::Native` as *the source applies this
108//! predicate itself*, and says nothing about where it applies it. What the declaration
109//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
110//! — so that the engine may push it down and apply nothing of its own.
111//!
112//! Second, this source can keep that promise for every predicate at no additional API
113//! cost, because whichever of the three reads below answers a query has already read every
114//! item that query could keep before it filters anything. Filtering those items is
115//! in-process work over data already in hand.
116//!
117//! Third, no predicate but `projects` could be pushed into the API even if that were
118//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
119//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
120//! label set, a status column or a substring of a body, and its title qualifier matches
121//! tokens where this source — and the local Markdown source beside it — match substrings,
122//! so pushing a search down would silently *narrow* the answer. What a project filter has
123//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
124//! the issue for them is both cheaper and exact. So there is one predicate this source
125//! applies by asking a narrower question, six it applies in process, and none it is unable
126//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
127//! source has already done, and declaring `projects` native while ignoring the filter
128//! (which this source once did) silently returns another project's tasks, because the
129//! engine trusts the declaration and applies nothing locally.
130//!
131//! # The three ways this source reaches an item, and what each costs
132//!
133//! A board read is charged for what its *nested* connections could return rather than for
134//! what was asked, so one whole-board read costs the same whether the question was about
135//! one project or about all of them. That is why a question about one project is never
136//! answered by reading the board:
137//!
138//! | The question | What is sent | What it costs |
139//! | --- | --- | --- |
140//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
141//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
142//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
143//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
144//! | which board item one issue is, past the page that came with it | [`graphql::ISSUE_BOARD_ITEMS`] — that issue's own `projectItems` | one issue's memberships |
145//!
146//! The board half of an issue — its board item's id, its `Status` option and this
147//! source's origin text field — rides along on `Issue.projectItems` in the first three, so
148//! an item reached any of those ways resolves through the same
149//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
150//! same status, the same labels and the same qualified id. That connection comes back a
151//! *page* at a time, at `BOARD_ITEMS_PAGE_SIZE`, so the entry for this board is looked for
152//! on the page in hand and — only if that page reports more of the connection — in the
153//! last row's read of that one issue's memberships, resumed from the page's own cursor and
154//! walked to exhaustion. An issue with no entry for *this* board is not this source's to
155//! report, which is what keeps an id naming another repository's issue from being answered
156//! as an item of this board; and because the page is where the search starts rather than
157//! where it ends, that answer is one about a connection read to exhaustion and never about
158//! an unread page. Nothing costs the extra read but an issue on more boards than a page
159//! holds: an issue this board really does not hold reports no next page, so its
160//! memberships are already exhausted where they arrived.
161//!
162//! **No document here selects the board's own `Labels` field, and nothing is lost by
163//! that.** An item's labels are read from its content alone, wherever that content is
164//! reached: the three documents above select `Issue.labels` on the fragment, and
165//! [`graphql::BOARD`] selects the same connection on the `... on Issue` arm of its
166//! `content`. A board's `Labels` field is not one anybody fills in: it is a built-in
167//! `ProjectV2FieldType`, it is absent from `ProjectV2CustomFieldType` so no project can
168//! create one, and `ProjectV2FieldValue` — the whole of what
169//! `updateProjectV2ItemFieldValue` accepts — offers no way to write one. So GitHub derives
170//! it from the content, for every content type it exists on, and there is nothing it can
171//! hold that the content does not already say: for an `Issue` it *is* that issue's own
172//! labels, so selecting it beside them unions a set with itself.
173//!
174//! **A draft loses nothing by that either**, which is the reasoning this paragraph once had
175//! backwards. `DraftIssue` exposes no `labels` field, and by the three schema facts above
176//! it cannot carry a board `Labels` value to be derived from one — so a draft has nothing
177//! to select *and nothing to lose*, and reports no labels at all. A `PullRequest` item is
178//! discarded by [`GitHubProjectsSource::resolve`] before labels are read. Both halves are
179//! held to that by tests in `tests/plugin.rs`: the four ways an item is reached report one
180//! label set, and that set is the fixture issue's own, by
181//! `an_item_reports_the_same_labels_title_status_and_id_however_it_is_reached`; and a board
182//! item whose content is a draft reports an empty set, by
183//! `a_board_item_whose_content_is_a_draft_reports_no_labels_at_all`. The absence of the
184//! selection is held over [`graphql::DOCUMENTS`] by
185//! `no_document_selects_the_boards_own_labels_field`.
186//!
187//! The last row is still the board's own item connection, and deliberately: a **draft**
188//! board item is not an issue, so no search and no node read can reach one, and the reads
189//! that have to answer for the whole board are the ones whose cost is the board's size
190//! anyway.
191//!
192//! **What a read may return is capped too, and that cap is on the document rather than on
193//! the board.** GitHub limits the number of nodes **one query may return** to
194//! [`NODE_COUNT_LIMIT`] and refuses a query above that before executing it: the answer is
195//! an error naming the connection the count crossed at, not a slow or a partial result.
196//! Every board this source reads is refused the same way, so no board is too big for these
197//! documents and none is small enough to save one that is over.
198//!
199//! The count is arithmetic over the document's own text: each connection contributes the
200//! `first:` it asks for, counts **multiply** down a nested path and **sum** across sibling
201//! paths. Those are [GitHub's published rules][node-limits] and this workspace does not
202//! restate them — `github-graphql-node-count` implements them, and
203//! [`worst_case_node_count`] under [`largest_page_sizes`] is where every node count here
204//! comes from. `every_document_this_source_sends_stays_under_githubs_node_limit`, in
205//! `tests/node_count.rs`, recomputes every document in [`graphql::DOCUMENTS`] from that
206//! same text on every run and fails naming any that reaches the limit — so a connection
207//! added to a shared fragment is caught there rather than by GitHub.
208//!
209//! What decides those counts is the page sizes: [`MAX_PAGE_SIZE`] on the outer page,
210//! `NESTED_PAGE_SIZE` on the connections hanging off one item, and
211//! `BOARD_ITEMS_PAGE_SIZE` on the page of an issue's board memberships a read carries.
212//! `$nestedFirst` is spent twice down one path of a board read, so that constant is
213//! effectively squared there, which is why it is the one the limit is most sensitive to.
214//! `BOARD_ITEMS_PAGE_SIZE` is small for a reason of its own, recorded beside it: what a
215//! page of memberships misses is recovered by one further read rather than refused, so it
216//! buys a bound every read pays for at the price of a request only a multi-board issue
217//! pays.
218//!
219//! **`nodeCount` and `cost` are two numbers against two limits, and both are computed
220//! offline here — per document, one document at a time.** `nodeCount` is the one above: the
221//! most nodes one query may return, checked per query and bounded by [`NODE_COUNT_LIMIT`].
222//! `cost` is rate-limit points, metered per hour across everything one credential does; it
223//! is what the two limiters [`Limiter`] tells apart meter, and a document under
224//! [`NODE_COUNT_LIMIT`] still says nothing about its price. [`worst_case_point_cost`] is
225//! that second number, and `tests/point_cost.rs` pins every document in
226//! [`graphql::DOCUMENTS`] at what it costs — there being no per-call point ceiling to hold
227//! one under, the pin itself is the check. The credentialed lane reconciles both figures
228//! against GitHub's own, off a probe it already sends.
229//!
230//! **What is pinned that way is a per-document price and never a session's.** The record in
231//! `session-cost.md` measures the two quantities a whole session can be counted in offline —
232//! **requests** and **worst-case nodes** — and neither is points. What one whole session
233//! consumes of the hourly point allowance is observable only from a credentialed run's own
234//! `x-ratelimit-*` headers, which is what [`accounting`] fills its per-budget figures from
235//! and what `tests/live.rs` prints at the end of every run.
236//!
237//! [node-limits]: https://docs.github.com/en/graphql/overview/rate-limits-and-node-limits-for-the-graphql-api
238//!
239//! **Where a read-after-write guarantee comes from, since a search index cannot supply
240//! one.** GitHub's issue search is eventually consistent and answers a write made moments
241//! ago with the value from before it. Resolving a node id is not, so a read by id and a
242//! project's own sub-issues are already current. What closes the gap for the search is
243//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
244//! what this process itself wrote, so an item created seconds ago is reported whether or
245//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
246//! and the record dies with the process.
247//!
248//! Filtering happens before paging, so a page of a filtered result is a page of the
249//! survivors rather than the survivors of a page. Label and text matching answer the same
250//! question the same way the local Markdown source's do, so one cross-source expectation
251//! holds for both.
252//!
253//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
254//! has one source, `capabilities`, and the note above is the reasoning behind it rather
255//! than a second copy of it: without the three facts recorded here a reader takes the
256//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
257//! crate's own capabilities test, which pins every field of it against a fully spelled-out
258//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
259//! fails to compile there rather than going unasserted. -->
260//! The fixture-server tests above run wherever this crate is selected; the credentialed
261//! lane runs in the same required check, beside them, and can fail it — it verifies the
262//! current schema, then drives every field of the table above against the real board. It builds its own fixture there — two projects, one task filed under each,
263//! one filed under neither, a label on one of the three and a closed status on another —
264//! because that shape is what tells an honoured predicate from an ignored one: a board
265//! holding a single project answers a project filter the same way whether or not this
266//! source applies it, which is exactly how the defect above went unseen.
267//!
268//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
269//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
270//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
271//! nominated is what keeps a credentialed write lane off a board and a repository nobody
272//! nominated; it never asks GitHub which project was updated most recently. Before it
273//! starts, the lane also clears any item titled — and any repository label named — the way
274//! it titles and names its own artifacts, which is self-healing after an interrupted run:
275//! a process killed between its writes and its cleanup leaves artifacts the next run
276//! removes.
277//!
278//! # What a session of requests costs, and where the report is
279//!
280//! This source records **every** request it sends into [`accounting::Accounting`], at
281//! `send_once` — the one place a request leaves this crate, which is why a read path added
282//! later is counted without anybody remembering to count it. That is the whole of what this
283//! crate adds to the arrangement; [`accounting`] is where what a record carries, how a
284//! session's spend is arrived at, and what it deliberately does not know are set out.
285//!
286//! What one whole session of the live journey costs, counted that way against this crate's
287//! loopback fixture board, is written down in `session-cost.md` beside this crate — with the
288//! reduction it came out of, and with what it does and does not say about rate-limit points.
289//!
290//! [`GitHubProjectsSource::accounting`] is the read: a snapshot to hold and compare, which
291//! [`accounting::Session::report`] renders the session report from. It is on the ordinary
292//! code path — no environment variable, no feature, no build configuration — because an
293//! instrument nobody switches on measures nothing, and
294//! [`Plugin::build_recording_into`] is how a caller making its own calls beside this
295//! source's counts the whole session rather than this source's share. The credentialed lane
296//! in `tests/live.rs` does exactly that, and prints the report at the end of every run,
297//! passed or failed.
298//!
299//! **A live session refuses to start unless the account can afford it.** Before it does any
300//! of the work it exists to do, the journey makes one request — `GET /rate_limit`, which
301//! GitHub documents as not counting against the REST rate limit and which answers both of
302//! its budgets at once — and starts only if, for each of them, what remains minus this
303//! session's estimated cost is still at least
304//! `onetaskgraph_live::RETAINED_BUFFER` — twenty per cent — of that budget's whole
305//! allowance. A session that cannot **declines**: it did not run, so it is
306//! neither a pass nor a failing assertion, and it says which budget was short, that budget's
307//! limit, what remained, the estimate, the buffer and when it resets — then stops, without
308//! waiting for the budget to come back. The estimate is derived offline from
309//! `tests/fixtures/session-cost.txt` and a cost model stated in `tests/journey/budget.rs`,
310//! which is also where the published rule that model rests on is cited; the accounting
311//! above records the gate's own read like any other request, and
312//! [`accounting::Session::report`] prints the estimate beside what the session really spent.
313//!
314//! **GitHub is the authority on both of its own numbers, and the credentialed lane goes and
315//! asks it.** Everything above computes `nodeCount` and `cost` offline from a document's own
316//! text, which is what lets it run on every platform and on a pull request from a fork with
317//! no credential — and that is what actually stops a regression merging. But an offline
318//! arithmetic can only ever agree with itself: if GitHub changes its rules, this workspace
319//! goes on computing the old answer and nothing notices. So `tests/live.rs` reconciles them.
320//! GitHub's schema exposes `rateLimit(dryRun: true)`, whose `nodeCount` is *"the maximum
321//! number of nodes this query may return"* and whose `cost` is what that document would
322//! spend, both for a document **without executing it**, and the lane asks it for every query
323//! document this source sends, under the largest bindings this source sends, and fails when
324//! GitHub's figure and [`worst_case_node_count`] or [`worst_case_point_cost`] disagree. A
325//! mutation is skipped, because `rateLimit` is a field of `Query` and cannot be asked about
326//! one; the offline pins still cover it. It records what those calls reported about the
327//! account's own allowance, because whether asking is free is a thing to observe rather than
328//! to assume. Two quantities, not one: [`NODE_COUNT_LIMIT`] bounds `nodeCount` per query,
329//! and `cost` is metered against an hourly allowance the accounting above reads off a
330//! credentialed run's own response headers.
331//!
332//! **GitHub has two rate limiters and this source is refused by both, so nothing here
333//! treats them as one thing.** The primary budget is the hourly allowance `gh api
334//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
335//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
336//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
337//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
338//! one part of.
339#![deny(missing_docs)]
340
341use std::collections::BTreeMap;
342use std::sync::{Arc, Mutex};
343use std::time::{Duration, Instant};
344
345use chrono::{DateTime, Utc};
346use onetaskgraph_plugin_api::{
347 Capabilities, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind, DependencySupport,
348 Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label, LabelFilter, Location,
349 NativeId, Page, PageRequest, Project, ProjectFilter, ProjectQuery, Repository, SecretResolver,
350 SourceError, SourceName, SourcePlugin, Status, StatusCategory, Support, Task, TaskQuery,
351 TaskSource, TextFields, TextQuery, WriteSupport,
352};
353use reqwest::{Client, StatusCode, Url};
354use schemars::{Schema, schema_for};
355use secrecy::{ExposeSecret, SecretString};
356use serde::Deserialize;
357use serde_json::{Value, json};
358
359pub mod accounting;
360
361use accounting::Accounting;
362
363/// The registry name for this plugin.
364pub const KIND: &str = "github-projects";
365/// GitHub's maximum connection page size.
366pub const MAX_PAGE_SIZE: u32 = 100;
367
368/// The most nodes any one document this source sends may be asked to return.
369///
370/// GitHub's own published per-query ceiling, taken from
371/// [`github_graphql_node_count::NODE_LIMIT`] rather than written out again here, so this
372/// workspace cannot hold a stale copy of somebody else's number. A query above it is
373/// **refused before it is executed**, whoever is asking and whatever board they are
374/// asking about — so this is a bound on the documents rather than a budget that runs out.
375///
376/// This is `nodeCount`, the maximum number of nodes *one query may return*. It is not
377/// `cost`, the rate-limit points a call spends against an hourly allowance shared by
378/// everything the credential does — two numbers against two limits, and this constant
379/// bounds only the first. The second is computed offline too, per document:
380/// [`worst_case_point_cost`], pinned for every document in [`graphql::DOCUMENTS`] by
381/// `tests/point_cost.rs`, and reconciled against GitHub's own `cost` by the credentialed
382/// lane. There is no constant like this one to hold a price under, because points are an
383/// hourly allowance rather than a per-call bound.
384///
385/// Neither is a session's price. What `session-cost.md` records of a whole session is its
386/// **requests** and its **worst-case nodes**; what a whole session spends in points is
387/// reported only by a credentialed run's own `x-ratelimit-*` headers, through
388/// [`accounting`]. The module section on the three ways this source reaches an item says how
389/// the count is arrived at, and which of the page sizes below decide it.
390pub const NODE_COUNT_LIMIT: u64 = github_graphql_node_count::NODE_LIMIT;
391
392/// Nested connection size for the connections that hang off one item.
393///
394/// It multiplies through every document that reaches an item under a page — the count
395/// rules multiply down a nested path — so it is the constant [`NODE_COUNT_LIMIT`] is most
396/// sensitive to. `tests/node_count.rs` is what holds the pair together: it recomputes
397/// every document under these constants and fails naming any that reaches the limit, so
398/// raising this is caught there rather than by GitHub.
399const NESTED_PAGE_SIZE: u32 = 50;
400/// How many of one issue's board memberships are read when an issue is reached directly.
401///
402/// An issue reached through a search or through its own node id carries its board half in
403/// `Issue.projectItems`, and only the entry for *this* board is read. This connection sits
404/// under a page of issues, so every point of it multiplies through the whole document and
405/// is paid for whether or not any issue is on a second board — which is why it is
406/// deliberately far smaller than [`NESTED_PAGE_SIZE`].
407///
408/// **Three, because what a page misses is now recovered rather than refused**, and the
409/// recovery is what the value is chosen against. An issue whose entry for this board sits
410/// past this page costs one further request — [`graphql::ISSUE_BOARD_ITEMS`], resumed from
411/// that page's own cursor — so the value trades a bound every read pays for a request only
412/// a multi-board issue pays. At one, a deployment whose issues commonly sit on two or more
413/// boards would pay that request *per issue*, which is order N against the one page per
414/// hundred issues a read costs today. At three it is only reached by an issue on four or
415/// more boards at once, which keeps the recovery path exceptional rather than routine for
416/// a plausible deployment.
417const BOARD_ITEMS_PAGE_SIZE: u32 = 3;
418
419pub use github_graphql_node_count::{NodeCountError, Variables};
420
421/// The largest value this source can bind to each page-size variable its documents name.
422///
423/// Every `first:` in [`graphql`] reads one of these three, and each is capped at the
424/// constant above it wherever a caller's own limit could reach it — `$first` at
425/// [`MAX_PAGE_SIZE`], `$nestedFirst` at `NESTED_PAGE_SIZE`, `$boardItems` at
426/// `BOARD_ITEMS_PAGE_SIZE`. So this is the worst case a caller can drive this source to,
427/// not one configuration of it, which is what makes a bound computed under it a bound on
428/// every read.
429pub fn largest_page_sizes() -> Variables {
430 Variables::from([
431 ("first".to_owned(), MAX_PAGE_SIZE),
432 ("nestedFirst".to_owned(), NESTED_PAGE_SIZE),
433 ("boardItems".to_owned(), BOARD_ITEMS_PAGE_SIZE),
434 ])
435}
436
437/// The most nodes `document` could be asked to return, by GitHub's published rules.
438///
439/// Computed offline from the document's own text under [`largest_page_sizes`] — no
440/// network, no credential and no schema — by
441/// [`github_graphql_node_count::node_count`], which is where the rules themselves live.
442/// A document at or above [`NODE_COUNT_LIMIT`] is one GitHub refuses before executing, so
443/// this is what a check holds every document in [`graphql::DOCUMENTS`] below.
444///
445/// # Errors
446///
447/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
448/// no single operation, or binds a page size this source does not name — each of which is
449/// a defect in the document rather than a number.
450pub fn worst_case_node_count(document: &str) -> Result<u64, NodeCountError> {
451 node_count(document, &largest_page_sizes())
452}
453
454/// The most rate-limit points one call of `document` could spend, by GitHub's published
455/// rules.
456///
457/// Computed offline from the document's own text under [`largest_page_sizes`] — no
458/// network, no credential and no schema — by
459/// [`github_graphql_node_count::point_cost`], which is where the rules themselves live.
460/// This is `cost`, metered **per hour** against the allowance one credential shares across
461/// everything it does; it is not `nodeCount`, which is [`worst_case_node_count`] and is
462/// bounded per query by [`NODE_COUNT_LIMIT`]. There is no per-call ceiling to hold this
463/// under, so what `tests/point_cost.rs` does with it is pin every document in
464/// [`graphql::DOCUMENTS`] at what it costs, and the credentialed lane reconciles those
465/// figures against GitHub's own reported `cost`.
466///
467/// # Errors
468///
469/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
470/// no single operation, or binds a page size this source does not name — each of which is
471/// a defect in the document rather than a number.
472pub fn worst_case_point_cost(document: &str) -> Result<u64, NodeCountError> {
473 github_graphql_node_count::point_cost(document, &largest_page_sizes())
474}
475
476/// The most nodes `document` could be asked to return under `variables`.
477///
478/// [`worst_case_node_count`] is this under [`largest_page_sizes`], and the accounting in
479/// [`accounting`] is this under the bindings one request really sent — one spelling of the
480/// calculation, so a bound checked offline and a cost recorded at run time cannot come to
481/// disagree. The rules themselves live in [`github_graphql_node_count::node_count`].
482///
483/// # Errors
484///
485/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
486/// no single operation, or binds a page size `variables` does not name.
487pub fn node_count(document: &str, variables: &Variables) -> Result<u64, NodeCountError> {
488 github_graphql_node_count::node_count(document, variables)
489}
490
491/// The issue-title prefix that makes a board issue a document.
492///
493/// A GitHub Projects board has no document type — it holds issues — so the discriminator
494/// is the title, and this is the whole of it: an issue whose title begins with these bytes
495/// is a document and every other issue is the task or project the sub-issue rule makes it.
496///
497/// It is spelled **once**, here, and read rather than restated everywhere else — including
498/// by the shared journeys, which take it from this constant so a board fixture cannot
499/// drift from what this source reads. `docs/metadata.md` records the two consequences that
500/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
501/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
502/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
503pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
504
505/// Exact GraphQL query documents issued by this plugin.
506///
507/// Keeping the production documents here lets the pinned-schema test validate the same
508/// bytes that are sent to GitHub, rather than a test-only copy which could drift
509/// independently. No document in this module writes the board itself, and none of them
510/// names `updateProjectV2Field`.
511pub mod graphql {
512 /// The board half of one item: the field values every document here reads it from.
513 ///
514 /// A macro for the same reason [`board_issue!`] below is one, a level further in. This
515 /// selection is needed by that fragment, by [`BOARD`] under the board's own `items`,
516 /// and by [`ISSUE_BOARD_ITEMS`] under a membership walk — and all three have to produce
517 /// *the same value*, because
518 /// [`GitHubProjectsSource::resolve`](super::GitHubProjectsSource) reads them through
519 /// one path. Three spellings of it is what would drift, so there is one.
520 ///
521 /// The `Status` option and this source's own origin text field are the whole of it. It
522 /// selects no `ProjectV2ItemFieldLabelValue`: GitHub derives that field from the item's
523 /// content, so it holds nothing the content's own `labels` do not already say, and it
524 /// would sit a label connection two page sizes deep.
525 macro_rules! board_item_values {
526 () => {
527 r#"fieldValues(first:$nestedFirst){nodes{
528 ... on ProjectV2ItemFieldSingleSelectValue{name field{
529 ... on ProjectV2SingleSelectField{id name options{id name}}
530 }}
531 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
532 }pageInfo{hasNextPage}}"#
533 };
534 }
535
536 /// Everything this source reads about one issue, wherever it reaches that issue.
537 ///
538 /// A macro rather than a constant so the three documents below can `concat!` it: one
539 /// spelling of these fields is what makes an issue read through the board-scoped
540 /// search, through its own node id, and through its project's sub-issue relationship
541 /// resolve to *the same* item, which is the whole of what
542 /// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
543 ///
544 /// `projectItems` is what carries the board half of an issue: the board item's own id
545 /// and the [`board_item_values!`] above — the `Status` option and this source's origin
546 /// text field — that a `ProjectV2.items` read used to carry. It is asked for on the
547 /// issue rather than on the board, which is what makes the cost of a read proportional
548 /// to what was asked for instead of to the board's size.
549 ///
550 /// It carries a *page* of that connection, at `BOARD_ITEMS_PAGE_SIZE`, and its
551 /// `endCursor` is what [`ISSUE_BOARD_ITEMS`] resumes from when this board's entry is
552 /// not on that page: a page here is where the search for the entry starts rather than
553 /// where it ends.
554 ///
555 /// It does **not** select the board's `Labels` field value, and that is the whole of
556 /// what keeps the three documents below under [`NODE_COUNT_LIMIT`](super::NODE_COUNT_LIMIT):
557 /// a label connection there sits under `fieldValues` under `projectItems` under a page
558 /// of issues, spending `$nestedFirst` twice down one path, and took
559 /// [`SEARCH_ISSUES`] and [`SUB_ISSUES`] to 2,556,100 nodes against a limit of 500,000.
560 /// No label is lost — this is a fragment `on Issue`, whose own `labels` are selected
561 /// above, and that connection is where every label this source reports comes from. No
562 /// document in this module selects the board field any longer, [`BOARD`] included; the
563 /// module documentation records why nothing it could have held is lost.
564 macro_rules! board_issue {
565 () => {
566 concat!(
567 r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
568 labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
569 projectItems(first:$boardItems){nodes{id project{number}
570 "#,
571 board_item_values!(),
572 r#"}pageInfo{hasNextPage endCursor}}}"#
573 )
574 };
575 }
576
577 /// Every issue of one board, found by a search scoped to that board.
578 ///
579 /// This is how the projects a board holds are listed, and it selects no `items`
580 /// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
581 /// container walked page by page, so nothing nested inside a board item is paid for.
582 /// Which of the issues it returns is a project is then read off `parent` — GitHub
583 /// accepts `-has:parent` as a search qualifier and silently ignores it, so the
584 /// discriminator has to be applied to the field, which is a scalar on the issue and
585 /// costs nothing.
586 pub const SEARCH_ISSUES: &str = concat!(
587 r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
588 search(query:$search,type:$type,first:$first,after:$after){
589 pageInfo{hasNextPage endCursor}
590 nodes{__typename ...BoardIssue}
591 }
592 }"#,
593 board_issue!()
594 );
595
596 /// One issue by its own node id, which is what a qualified id names here.
597 ///
598 /// Strongly consistent, unlike the search above: GitHub's issue search is an index and
599 /// answers a write made moments ago with the value from before it, and resolving a node
600 /// id does not.
601 pub const ISSUE: &str = concat!(
602 r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
603 node(id:$id){__typename ...BoardIssue}
604 }"#,
605 board_issue!()
606 );
607
608 /// One project's tasks: the sub-issues of the issue that project is.
609 ///
610 /// The work this costs is the project's own size. Nothing about it grows as the board
611 /// gains projects, or as those projects gain tasks.
612 pub const SUB_ISSUES: &str = concat!(
613 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
614 node(id:$id){__typename
615 ... on Issue{subIssues(first:$first,after:$after){
616 pageInfo{hasNextPage endCursor}
617 nodes{__typename ...BoardIssue}
618 }}}
619 }"#,
620 board_issue!()
621 );
622
623 /// Reads the board's fields and one page of its items.
624 pub const BOARD: &str = concat!(
625 r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
626 owner:repositoryOwner(login:$owner){
627 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
628 }
629 } fragment Board on ProjectV2 { id title
630 fields(first:$nestedFirst){nodes{
631 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
632 ... on ProjectV2Field{__typename id name}
633 }pageInfo{hasNextPage}}
634 items(first:$first,after:$after){nodes{id "#,
635 board_item_values!(),
636 r#" content{
637 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
638 ... on PullRequest{__typename id}
639 ... on DraftIssue{__typename id title body createdAt updatedAt}
640 }} pageInfo{hasNextPage endCursor}}
641 }"#
642 );
643
644 /// One issue's board memberships alone, walked past the page a read of it carried.
645 ///
646 /// The recovery read behind [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource):
647 /// every document above carries a *page* of `Issue.projectItems`, and an issue on more
648 /// boards than that page holds may have this board's entry past its end. This asks that
649 /// one issue for its memberships and nothing else — the caller already holds the issue —
650 /// so an answer of "this board does not hold it" is only ever given about a connection
651 /// read to exhaustion.
652 ///
653 /// It selects the board item's id, its project number and the same
654 /// [`board_item_values!`] the fragment does, because what it produces is handed to the
655 /// very same resolver: an issue recovered this way reports the same title, the same
656 /// status, the same labels and the same qualified id as one whose entry was on the
657 /// page.
658 ///
659 /// `$first` rather than `$boardItems`: this document reads one issue, so nothing
660 /// multiplies through it and the membership connection can be walked at
661 /// [`MAX_PAGE_SIZE`](super::MAX_PAGE_SIZE) — which is what keeps the recovery to one
662 /// further request for any issue a person really keeps.
663 pub const ISSUE_BOARD_ITEMS: &str = concat!(
664 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!){
665 node(id:$id){
666 ... on Issue{projectItems(first:$first,after:$after){
667 nodes{id project{number}
668 "#,
669 board_item_values!(),
670 r#"}
671 pageInfo{hasNextPage endCursor}}}
672 }
673 }"#
674 );
675 /// Resolves the configured repository's node id, which creating an issue requires.
676 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
677 /// Reads both dependency directions for one issue, with each far end's own kind.
678 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
679 ... on Issue{
680 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
681 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
682 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
683 /// Creates one issue in the configured repository.
684 pub const CREATE_ISSUE: &str =
685 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
686 /// Puts an existing issue on the configured board.
687 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
688 /// Updates an issue's visible fields and its open or closed state in one call.
689 pub const UPDATE_ISSUE: &str =
690 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
691 /// Updates an existing draft's user-visible fields.
692 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
693 /// Updates a text or single-select value on one project item.
694 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
695 /// Files one issue under another as a sub-issue, which is what project membership is.
696 pub const ADD_SUB_ISSUE: &str =
697 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
698 /// Takes one issue back out of its parent.
699 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
700 /// Adds GitHub's native issue blocked-by relationship.
701 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
702 /// Removes one native issue blocked-by relationship.
703 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
704 /// Deletes one issue, which takes its board item with it.
705 ///
706 /// The engine sends this in one situation only: undoing a copy that could not finish,
707 /// over the items that same copy created. Deleting the issue removes the board item
708 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
709 pub const DELETE_ISSUE: &str =
710 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
711
712 /// Every document above, with what this source is doing when it sends one.
713 ///
714 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
715 /// name the call that was refused, and a `match` with a catch-all arm would answer a
716 /// document added later with "talking to GitHub" and never say so.
717 ///
718 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
719 /// const` here that this list omits, so the two cannot part — which is the same guard
720 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
721 pub const DOCUMENTS: [(&str, &str); 17] = [
722 (SEARCH_ISSUES, "searching this board's issues"),
723 (ISSUE, "reading one issue"),
724 (
725 ISSUE_BOARD_ITEMS,
726 "reading one issue's board memberships past the page it came with",
727 ),
728 (SUB_ISSUES, "reading a project's tasks"),
729 (BOARD, "reading the board"),
730 (REPOSITORY, "reading the destination repository"),
731 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
732 (CREATE_ISSUE, "creating an issue"),
733 (ADD_TO_BOARD, "adding an issue to the board"),
734 (UPDATE_ISSUE, "updating an issue"),
735 (UPDATE_DRAFT, "updating a draft item"),
736 (UPDATE_FIELD, "writing a board field"),
737 (ADD_SUB_ISSUE, "filing an issue under its project"),
738 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
739 (ADD_BLOCKED_BY, "recording a dependency"),
740 (REMOVE_BLOCKED_BY, "removing a dependency"),
741 (DELETE_ISSUE, "deleting an issue"),
742 ];
743}
744
745/// Which of GitHub's two rate limiters refused a request.
746///
747/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
748/// secondary one — so an operator told the wrong one takes the wrong next step, which is
749/// the whole reason this is carried rather than collapsed into "rate limited".
750#[derive(Debug, Clone, Copy, PartialEq, Eq)]
751enum Limiter {
752 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
753 Primary,
754 /// The burst limiter over content-generating requests, which nothing reports.
755 Secondary,
756}
757
758/// The wordings GitHub answers a secondary rate limit with.
759///
760/// It sends them under a forbidden status, under a too-many-requests status, and inside
761/// the `errors` of a *successful* response, which is why the text is what this matches on
762/// rather than the status. `abuse detection` is the wording GitHub used before the
763/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
764/// what a burst of content creation is refused with.
765///
766/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
767/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
768/// when, and the drift gate reconciles the two lists both ways. Public for that gate
769/// alone — a caller has no use for it, and matching on a refusal is this source's job.
770pub const SECONDARY_WORDINGS: [&str; 5] = [
771 "secondary rate limit",
772 "temporarily blocked from content creation",
773 "abuse detection",
774 "submitted too quickly",
775 "exceeded a secondary",
776];
777
778/// The wordings GitHub answers an exhausted primary budget with.
779///
780/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
781/// than looked for in the response text. Pinned and gated exactly as
782/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
783pub const PRIMARY_WORDINGS: [&str; 3] = [
784 "api rate limit exceeded",
785 "rate limit exceeded",
786 "rate_limited",
787];
788
789/// What a response *says about itself*, which is the only place a refusal can be read.
790///
791/// Deliberately not the whole response body. A board is a place people write about their
792/// own work, and a task on it titled "the secondary rate limit" would, matched across the
793/// raw text, turn a perfectly good answer into a refusal this source then waited out and
794/// reported. So the item data is never read: what is read is GitHub's own REST-style
795/// `message` envelope, which is what a forbidden status carries, and the `message` and
796/// `type` of each GraphQL error, which is where a *successful* response says it.
797///
798/// A body that is not JSON at all has nothing structured to read, so only a failing
799/// response's own text is taken — a successful response that is not JSON is malformed
800/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
801fn refusal_wording(status: StatusCode, body: &str) -> String {
802 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
803 return if status.is_success() {
804 String::new()
805 } else {
806 body.to_owned()
807 };
808 };
809 let mut said: Vec<&str> = parsed
810 .get("message")
811 .and_then(Value::as_str)
812 .into_iter()
813 .collect();
814 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
815 for error in errors {
816 said.extend(
817 ["message", "type"]
818 .into_iter()
819 .filter_map(|key| error.get(key).and_then(Value::as_str)),
820 );
821 }
822 }
823 said.join("; ")
824}
825
826impl Limiter {
827 /// Which limiter refused this response, or `None` when none of them did.
828 ///
829 /// The wording is read first and the status only decides what carries none of it,
830 /// because GitHub answers a secondary limit with a forbidden status far more often
831 /// than with too-many-requests — while a forbidden status saying nothing about a limit
832 /// really is a credential this token lacks.
833 ///
834 /// A response is a refusal because of its status or its own wording. A spent budget
835 /// only ever explains one; it never turns an answer into a refusal.
836 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
837 let normalized = refusal_wording(status, body).to_ascii_lowercase();
838 if SECONDARY_WORDINGS
839 .iter()
840 .any(|wording| normalized.contains(wording))
841 {
842 return Some(Self::Secondary);
843 }
844 if status == StatusCode::TOO_MANY_REQUESTS {
845 return Some(Self::Primary);
846 }
847 // An exhausted budget *explains* a response that failed; it does not make one that
848 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
849 // request the budget allowed as well as on the ones it then refuses, so reading
850 // the header alone threw away a good answer — and, once refusals were retried,
851 // replayed a request that had already taken effect.
852 if !status.is_success() && budget_exhausted {
853 return Some(Self::Primary);
854 }
855 // A successful response saying it: GitHub reports a GraphQL rate limit in the
856 // `errors` of an HTTP 200, where nothing about the status says so at all.
857 if status.is_success()
858 && PRIMARY_WORDINGS
859 .iter()
860 .any(|wording| normalized.contains(wording))
861 {
862 return Some(Self::Primary);
863 }
864 None
865 }
866
867 /// What this limiter is called where an operator can look it up.
868 const fn name(self) -> &'static str {
869 match self {
870 Self::Primary => "GitHub's primary API rate limit",
871 Self::Secondary => "GitHub's secondary rate limit",
872 }
873 }
874
875 /// What the endpoint an operator would go and check says about this limiter.
876 const fn where_to_look(self) -> &'static str {
877 match self {
878 Self::Primary => {
879 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
880 comes back."
881 }
882 Self::Secondary => {
883 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
884 primary budget and does not report this one, so budget showing there says \
885 nothing about this refusal, and every further attempt extends it."
886 }
887 }
888 }
889
890 /// The next step this limiter actually calls for.
891 const fn what_to_do(self) -> &'static str {
892 match self {
893 Self::Primary => {
894 "wait for the reset `gh api rate_limit` reports, then run the command again."
895 }
896 Self::Secondary => {
897 "leave this board alone for a few minutes, then run the command again — or \
898 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
899 }
900 }
901 }
902}
903
904/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
905#[derive(Debug, Clone, Copy)]
906struct Limited {
907 limiter: Limiter,
908 hint: Option<u64>,
909}
910
911impl Limited {
912 /// What the caller is told once this source has waited as long as it may.
913 ///
914 /// Both limiters report as [`SourceError::RateLimited`], because that is what
915 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
916 /// about *which* limiter it was makes it a different kind of failure. What differs is
917 /// the operator's next step, and that is what the message carries — a secondary
918 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
919 /// budget looks fine, and then back to retry the very burst that was refused.
920 fn exhausted(
921 self,
922 doing: &str,
923 waits: u32,
924 waited: Duration,
925 needed: Duration,
926 budget: Duration,
927 ) -> SourceError {
928 SourceError::RateLimited {
929 retry_after_seconds: self.hint,
930 message: Some(format!(
931 "{} refused this source while {doing}; it waited {} out over {} and was refused \
932 again, and the next wait of {} would take it past the {} one call may spend \
933 waiting. {} next: {}",
934 self.limiter.name(),
935 plural(waits, "refusal"),
936 seconds(waited),
937 seconds(needed),
938 seconds(budget),
939 self.limiter.where_to_look(),
940 self.limiter.what_to_do(),
941 )),
942 }
943 }
944}
945
946/// One HTTP attempt's result, with what its response said about the rate limit.
947///
948/// The two travel together so the record and the outcome are written from the same place:
949/// what a response said about the budget is only readable while that response is in hand,
950/// and what the attempt *meant* is only decidable once its body has been read.
951struct Attempted {
952 result: Result<Value, Attempt>,
953 limits: accounting::RateLimit,
954 /// GitHub's own reported cost for this call, for a document that asked for it.
955 reported_cost: Option<u64>,
956}
957
958/// One attempt's outcome: an error to report, or a rate limit to wait out.
959enum Attempt {
960 Failed(SourceError),
961 Limited(Limited),
962}
963
964fn plural(count: u32, thing: &str) -> String {
965 if count == 1 {
966 format!("{count} {thing}")
967 } else {
968 format!("{count} {thing}s")
969 }
970}
971
972fn seconds(duration: Duration) -> String {
973 format!("{:.1}s", duration.as_secs_f64())
974}
975
976/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
977///
978/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
979/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
980/// header, and neither is what makes a response a refusal — so the whole cost of one this
981/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
982/// instead. Refusing the response over the header would turn a readable refusal into an
983/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
984fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
985 value
986 .and_then(|value| value.to_str().ok())
987 .and_then(|value| value.trim().parse::<u64>().ok())
988}
989
990/// Every mutation this source sends creates content — an issue, a board item, a field of
991/// one, a sub-issue link, a dependency — and no query in [`graphql::DOCUMENTS`] does, so
992/// what the secondary limiter counts and what the keyword says are the same set. That is
993/// what makes the keyword a sound test rather than a convenient one.
994fn is_mutation(query: &str) -> bool {
995 query.trim_start().starts_with("mutation")
996}
997
998/// What this source was doing, for a diagnostic that has to say so.
999///
1000/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
1001/// a document added without a description is caught by that list's own gate instead of
1002/// falling through to the vague arm below.
1003fn operation_description(query: &str) -> &'static str {
1004 graphql::DOCUMENTS
1005 .iter()
1006 .find(|(document, _)| *document == query)
1007 .map_or("talking to GitHub", |(_, doing)| *doing)
1008}
1009
1010/// GitHub's published ceiling on content-generating requests, per minute.
1011///
1012/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
1013/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
1014/// from it, so a pacing value checked only against itself cannot go stale here.
1015pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
1016/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
1017/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
1018/// governs; it is pinned beside its sibling so the gate would notice either one moving.
1019pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
1020/// Shortest interval between two content-creating mutations, in milliseconds.
1021///
1022/// GitHub documents two secondary limits on content-generating requests:
1023/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
1024/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
1025/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
1026/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
1027/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
1028/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
1029/// deliberately *not* what this paces at. An installation that wants the hourly bound
1030/// honoured for a long sequence of copies says so through
1031/// `pacing.min_mutation_interval_ms`.
1032pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
1033/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
1034///
1035/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
1036/// own advice for a secondary limit — wait, and wait longer each time — without spending
1037/// the first minute of a transient refusal doing nothing.
1038pub const RETRY_BACKOFF_MS: u64 = 1_000;
1039/// Total time one call may spend waiting out rate limits before it reports a failure.
1040///
1041/// Two minutes is long enough to ride out the refusals a paced copy still collects and
1042/// short enough that a command an operator is watching returns. The bound is what makes
1043/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
1044/// the limiter, not in a process nobody can tell from a wedged one.
1045pub const RETRY_BUDGET_MS: u64 = 120_000;
1046
1047fn default_token_env() -> String {
1048 "GH_PROJECTS_TOKEN".to_owned()
1049}
1050fn default_endpoint() -> String {
1051 "https://api.github.com/graphql".to_owned()
1052}
1053
1054/// Where one status category lands on this board.
1055///
1056/// `null` — an absent value — disables the category for this instance, and using a
1057/// disabled status is a refusal naming the status and the instance.
1058#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
1059#[serde(untagged)]
1060pub enum StatusTargetConfig {
1061 /// The name of a `Status` single-select option already on the board.
1062 Column(ColumnName),
1063 /// A closed issue state, whose reason is what tells done from cancelled.
1064 Closed {
1065 /// The `IssueClosedStateReason` to close with.
1066 closed: ClosedState,
1067 },
1068}
1069
1070/// The name of a `Status` single-select option on the board.
1071///
1072/// Validated on the way in rather than checked later, so a blank option name — which
1073/// nothing on a board can be — is a state this type cannot hold.
1074#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1075#[serde(try_from = "String")]
1076pub struct ColumnName(String);
1077
1078impl ColumnName {
1079 /// The option name, as the board spells it.
1080 fn as_str(&self) -> &str {
1081 &self.0
1082 }
1083}
1084
1085impl TryFrom<String> for ColumnName {
1086 type Error = String;
1087
1088 fn try_from(name: String) -> Result<Self, Self::Error> {
1089 if name.trim().is_empty() {
1090 return Err("a status_mapping option name cannot be blank".to_owned());
1091 }
1092 Ok(Self(name))
1093 }
1094}
1095
1096/// The two closed states this product can mean.
1097///
1098/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
1099/// work nor abandoned work, so nothing here ever writes it.
1100#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1101#[serde(rename_all = "kebab-case")]
1102pub enum ClosedState {
1103 /// `COMPLETED` — precisely done.
1104 Completed,
1105 /// `NOT_PLANNED` — precisely cancelled.
1106 NotPlanned,
1107}
1108
1109impl ClosedState {
1110 const fn reason(self) -> &'static str {
1111 match self {
1112 Self::Completed => "COMPLETED",
1113 Self::NotPlanned => "NOT_PLANNED",
1114 }
1115 }
1116}
1117
1118/// Configuration for one GitHub Projects v2 board.
1119#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1120#[serde(default, deny_unknown_fields)]
1121pub struct GitHubProjectsConfig {
1122 /// Login of the user or organization which owns the board.
1123 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
1124 /// The project number shown in the board's GitHub URL.
1125 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
1126 // llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] This doc is the field's schema description, which is what a person configuring the source reads, so it has to say when the field decides an issue's repository and when the item's own field does; the rule's one executable source is `GitHubProjectsSource::creation_target`, and `tests/plugin.rs` drives each case named here against the loopback board.
1127 /// `owner/name` of the repository this source creates an issue in when the item's own
1128 /// `repositories` field does not decide it.
1129 ///
1130 /// An item naming exactly one repository is created there; a task or a document naming
1131 /// none or several is created in its parent project's repository; and a project, or a
1132 /// task or document with no parent, naming none or several is created here. A board
1133 /// has no repository of its own and `createIssue` requires one, so a write without
1134 /// this is refused naming the field. Reads never need it.
1135 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
1136 // llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate]
1137 /// Environment variable containing a fine-grained token with Projects and Issues
1138 /// read/write plus Pull requests read-only access for every repository represented on
1139 /// the board.
1140 #[serde(default = "default_token_env")]
1141 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
1142 /// GraphQL endpoint. GitHub Enterprise installations may override it.
1143 #[serde(default = "default_endpoint")]
1144 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
1145 /// Per-instance mapping from a status category to where it lands on this board.
1146 ///
1147 /// A category this does not mention keeps its shipped default: `backlog` to
1148 /// "Backlog", `todo` to "Todo", `in-progress` to "In Progress", `done` to closed as
1149 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
1150 /// disabled.
1151 #[serde(default)]
1152 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
1153 /// How fast this source writes, and how long it waits out a rate-limit refusal.
1154 ///
1155 /// Every field keeps its shipped default when it is absent, and the defaults are
1156 /// GitHub's own published limits rather than taste. See [`Pacing`].
1157 #[serde(default)]
1158 pub pacing: PacingConfig,
1159}
1160
1161/// How fast this source writes, and how long it waits out a rate-limit refusal.
1162///
1163/// Configurable because a GitHub Enterprise installation sets its own limits and an
1164/// operator who has already been refused may want to go slower still — not because the
1165/// defaults are guesses.
1166#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1167#[serde(default, deny_unknown_fields)]
1168pub struct PacingConfig {
1169 /// Shortest interval between two content-creating mutations, in milliseconds.
1170 ///
1171 /// Zero sends them as fast as they are asked for, which is what a fixture server on
1172 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
1173 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1174 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
1175 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
1176 /// zero while there is a budget to spend, because a schedule of zero-length waits
1177 /// consumes none of it and so never ends.
1178 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
1179 /// Total time one call may spend waiting out rate limits, in milliseconds.
1180 ///
1181 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
1182 /// the bound is what makes this a wait rather than a hang.
1183 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1184}
1185
1186/// The largest any pacing setting may be, in milliseconds.
1187///
1188/// One hour. GitHub's own harshest published bound on content-generating requests works
1189/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
1190/// anything a real limit asks for, and past it the settings stop describing pacing at all:
1191/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
1192/// and an interval beyond it is a command that never sends its second mutation. It also
1193/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
1194/// what an `Instant` can hold on every platform.
1195pub const MAX_PACING_MS: u64 = 3_600_000;
1196
1197/// [`PacingConfig`] with every default resolved and every value checked, which is what the
1198/// source holds.
1199#[derive(Debug, Clone, Copy)]
1200struct Pacing {
1201 min_mutation_interval: Duration,
1202 retry_backoff: Duration,
1203 retry_budget: Duration,
1204}
1205
1206impl Pacing {
1207 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
1208 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
1209 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
1210 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
1211 message: format!(
1212 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
1213 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
1214 GitHub's own harshest published limit"
1215 ),
1216 }),
1217 Some(value) => Ok(Duration::from_millis(value)),
1218 None => Ok(Duration::from_millis(default)),
1219 };
1220 let retry_backoff = bounded(
1221 config.retry_backoff_ms,
1222 RETRY_BACKOFF_MS,
1223 "retry_backoff_ms",
1224 )?;
1225 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
1226 if retry_backoff.is_zero() && !retry_budget.is_zero() {
1227 return Err(SourceError::Config {
1228 message: format!(
1229 "pacing.retry_backoff_ms of source {instance} is 0 while \
1230 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
1231 none of that budget, so it would retry a refusal forever. Set a backoff of \
1232 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
1233 waiting at all",
1234 retry_budget.as_millis()
1235 ),
1236 });
1237 }
1238 Ok(Self {
1239 min_mutation_interval: bounded(
1240 config.min_mutation_interval_ms,
1241 MIN_MUTATION_INTERVAL_MS,
1242 "min_mutation_interval_ms",
1243 )?,
1244 retry_backoff,
1245 retry_budget,
1246 })
1247 }
1248}
1249
1250/// Factory for [`GitHubProjectsSource`].
1251#[derive(Debug, Clone, Copy, Default)]
1252pub struct Plugin;
1253
1254impl SourcePlugin for Plugin {
1255 fn kind(&self) -> &'static str {
1256 KIND
1257 }
1258 fn config_schema(&self) -> Schema {
1259 schema_for!(GitHubProjectsConfig)
1260 }
1261 fn build(
1262 &self,
1263 name: &SourceName,
1264 config: &Value,
1265 secrets: &dyn SecretResolver,
1266 ) -> Result<Box<dyn TaskSource>, SourceError> {
1267 self.build_recording_into(name, config, secrets, Arc::new(Accounting::new()))
1268 }
1269}
1270
1271impl Plugin {
1272 /// Build a source recording every request it sends into an accounting the caller holds.
1273 ///
1274 /// [`SourcePlugin::build`] is this with an accounting of its own, which is what the
1275 /// registry gets. This is for a caller that is also calling GitHub itself and wants one
1276 /// session total rather than two — see [`accounting`] and
1277 /// [`GitHubProjectsSource::recording_into`].
1278 ///
1279 /// # Errors
1280 ///
1281 /// Exactly [`SourcePlugin::build`]'s, with the same source name in front of each:
1282 /// [`SourceError::Config`] for configuration this plugin cannot use and
1283 /// [`SourceError::Auth`] for a credential it cannot find.
1284 pub fn build_recording_into(
1285 &self,
1286 name: &SourceName,
1287 config: &Value,
1288 secrets: &dyn SecretResolver,
1289 ledger: Arc<Accounting>,
1290 ) -> Result<Box<dyn TaskSource>, SourceError> {
1291 let config: GitHubProjectsConfig =
1292 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
1293 message: format!("source {name}: {e}"),
1294 })?;
1295 let source = GitHubProjectsSource::recording_into(name, config, secrets, ledger).map_err(
1296 |error| match error {
1297 SourceError::Config { message } => SourceError::Config {
1298 message: format!("source {name}: {message}"),
1299 },
1300 SourceError::Auth { message } => SourceError::Auth {
1301 message: format!("source {name}: {message}"),
1302 },
1303 other => other,
1304 },
1305 )?;
1306 Ok(Box::new(source))
1307 }
1308}
1309
1310/// Where a status category lands on this board, once configuration is resolved.
1311#[derive(Debug, Clone, PartialEq, Eq)]
1312enum StatusTarget {
1313 /// Not usable against this instance.
1314 Disabled,
1315 /// The board's `Status` option of this name.
1316 Column(ColumnName),
1317 /// A closed issue, with the reason that says which closed it means.
1318 Closed(ClosedState),
1319}
1320
1321/// Every status category, in the order the vocabulary declares them.
1322///
1323/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
1324/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
1325/// added to the shared vocabulary fails to compile until it is named there, and this
1326/// crate's suite reconciles this list against that enum's own derived schema, which is
1327/// generated from the variants rather than written beside them. The schema is what
1328/// catches a list left one short — a list checking only the positions it already holds
1329/// would pass while every mapping indexed by the new position panicked.
1330pub const CATEGORIES: [StatusCategory; 7] = [
1331 StatusCategory::Draft,
1332 StatusCategory::Backlog,
1333 StatusCategory::Todo,
1334 StatusCategory::InProgress,
1335 StatusCategory::Done,
1336 StatusCategory::Cancelled,
1337 StatusCategory::Unknown,
1338];
1339
1340/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
1341#[must_use]
1342pub const fn category_position(category: StatusCategory) -> usize {
1343 match category {
1344 StatusCategory::Draft => 0,
1345 StatusCategory::Backlog => 1,
1346 StatusCategory::Todo => 2,
1347 StatusCategory::InProgress => 3,
1348 StatusCategory::Done => 4,
1349 StatusCategory::Cancelled => 5,
1350 StatusCategory::Unknown => 6,
1351 }
1352}
1353
1354/// The spelling a status category is configured and reported under.
1355fn category_name(category: StatusCategory) -> &'static str {
1356 match category {
1357 StatusCategory::Draft => "draft",
1358 StatusCategory::Backlog => "backlog",
1359 StatusCategory::Todo => "todo",
1360 StatusCategory::InProgress => "in-progress",
1361 StatusCategory::Done => "done",
1362 StatusCategory::Cancelled => "cancelled",
1363 StatusCategory::Unknown => "unknown",
1364 }
1365}
1366
1367/// A shipped default's option name.
1368///
1369/// The literals below are this file's own and non-blank, and they are validated by the
1370/// one constructor a configured name goes through rather than beside it.
1371fn shipped_column(name: &'static str) -> ColumnName {
1372 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
1373}
1374
1375/// The shipped default for one category, before this instance's configuration.
1376fn shipped_default(category: StatusCategory) -> StatusTarget {
1377 match category {
1378 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
1379 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
1380 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
1381 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
1382 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
1383 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
1384 }
1385}
1386
1387/// This instance's complete category-to-target mapping, read in both directions.
1388///
1389/// One target per category, held at that category's own [`category_position`], so a
1390/// category missing from the mapping, named twice in it, or filed out of order is a
1391/// state this type cannot hold rather than one [`Self::target`] has to defend against.
1392#[derive(Debug, Clone)]
1393struct StatusMapping {
1394 targets: [StatusTarget; CATEGORIES.len()],
1395}
1396
1397impl StatusMapping {
1398 fn resolve(
1399 configured: BTreeMap<String, Option<StatusTargetConfig>>,
1400 instance: &SourceName,
1401 ) -> Result<Self, SourceError> {
1402 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
1403 for (key, value) in configured {
1404 let category = CATEGORIES
1405 .iter()
1406 .find(|category| category_name(**category) == key)
1407 .ok_or_else(|| SourceError::Config {
1408 message: format!(
1409 "status_mapping names {key:?}, which is not a status category of source \
1410 {instance}; the categories are {}",
1411 CATEGORIES
1412 .iter()
1413 .map(|category| category_name(*category))
1414 .collect::<Vec<_>>()
1415 .join(", ")
1416 ),
1417 })?;
1418 overrides.insert(category_name(*category), value);
1419 }
1420 // `CATEGORIES[position] == category` for every category — the crate's suite
1421 // asserts it — so mapping the list in order fills each category's own slot.
1422 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
1423 None => shipped_default(category),
1424 Some(None) => StatusTarget::Disabled,
1425 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
1426 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
1427 });
1428 let mapping = Self { targets };
1429 for (index, category) in CATEGORIES.into_iter().enumerate() {
1430 let StatusTarget::Column(option) = mapping.target(category) else {
1431 continue;
1432 };
1433 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
1434 matches!(mapping.target(**earlier), StatusTarget::Column(name)
1435 if name.as_str().eq_ignore_ascii_case(option.as_str()))
1436 }) {
1437 return Err(SourceError::Config {
1438 message: format!(
1439 "status_mapping of source {instance} sends both {} and {} to the board \
1440 option {:?}; one option cannot read back as two categories",
1441 category_name(*other),
1442 category_name(category),
1443 option.as_str()
1444 ),
1445 });
1446 }
1447 }
1448 Ok(mapping)
1449 }
1450
1451 fn target(&self, category: StatusCategory) -> &StatusTarget {
1452 &self.targets[category_position(category)]
1453 }
1454
1455 /// The category a board option name reports, or `None` when nothing maps to it.
1456 fn category_of(&self, option: &str) -> Option<StatusCategory> {
1457 CATEGORIES.into_iter().find(|category| {
1458 matches!(self.target(*category), StatusTarget::Column(name)
1459 if name.as_str().eq_ignore_ascii_case(option))
1460 })
1461 }
1462}
1463
1464// llmlint: ignore-block[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate] Every `createIssue` names one of these, and which one is the rule — a reader who reaches the type from `create_and_file_issue` gets the rule in one sentence here without the method's refusals, which stay on `creation_target`, the rule's one executable source; `tests/plugin.rs` drives every arm of it against the loopback board.
1465/// One repository this source can create an issue in, as `owner/name`.
1466///
1467/// Every `createIssue` this source sends names one of these: the item's own single
1468/// `repositories` entry, else its parent project issue's repository, else the configured
1469/// [`GitHubProjectsConfig::repository`]. [`GitHubProjectsSource::creation_target`] makes
1470/// that choice and says what it refuses before `createIssue`.
1471// llmlint: ignore-end[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate]
1472#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
1473struct RepositoryTarget {
1474 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1475 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1476}
1477
1478impl RepositoryTarget {
1479 fn parse(value: &str) -> Result<Self, SourceError> {
1480 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
1481 message: format!(
1482 "repository must be spelled owner/name; {value:?} names no repository"
1483 ),
1484 })?;
1485 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
1486 return Err(SourceError::Config {
1487 message: format!(
1488 "repository must be spelled owner/name with a GitHub login and one \
1489 repository name; {value:?} is not"
1490 ),
1491 });
1492 }
1493 Ok(Self {
1494 owner: owner.to_owned(),
1495 name: name.to_owned(),
1496 })
1497 }
1498
1499 /// The one host whose repositories this source creates issues in, spelled once: it is
1500 /// what [`Self::origin`] renders and what [`Self::from_origin`] accepts.
1501 const HOST: &str = "github.com";
1502
1503 fn origin(&self) -> String {
1504 format!("{}/{}/{}", Self::HOST, self.owner, self.name)
1505 }
1506
1507 /// The repository a normalized origin names, or why it is none this source can create
1508 /// an issue in: another host, or more or fewer than `owner/name` under this one.
1509 fn from_origin(origin: &Repository) -> Result<Self, String> {
1510 let not_here = || {
1511 format!(
1512 "{} is not a {}/owner/name repository",
1513 origin.as_str(),
1514 Self::HOST
1515 )
1516 };
1517 let (host, rest) = origin.as_str().split_once('/').ok_or_else(not_here)?;
1518 if host != Self::HOST {
1519 return Err(not_here());
1520 }
1521 Self::parse(rest).map_err(|_| not_here())
1522 }
1523
1524 fn slug(&self) -> String {
1525 format!("{}/{}", self.owner, self.name)
1526 }
1527}
1528
1529/// A source which reads GitHub afresh for every operation.
1530pub struct GitHubProjectsSource {
1531 /// This source's configured name, used both to tell a far end naming this source
1532 /// from one naming a system it knows nothing about, and to name the instance a
1533 /// status refusal is about.
1534 name: SourceName,
1535 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
1536 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
1537 repository: Option<RepositoryTarget>,
1538 endpoint: Url,
1539 token: SecretString,
1540 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1541 statuses: StatusMapping,
1542 client: Client,
1543 /// Every item this source has created since it was built, in the order it created
1544 /// them.
1545 ///
1546 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1547 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1548 /// a copy resolving a dependency on an item it had just created refused it as not
1549 /// found. A board read is completed from this — an item remembered here and absent from
1550 /// the read is added back, because the board really does hold it and only the read is
1551 /// behind.
1552 ///
1553 /// It is not a cache of a user's work: nothing is remembered that this process did not
1554 /// itself just write, it lives and dies with the process, and it is never consulted for
1555 /// an item this source did not create.
1556 created: Mutex<Vec<Resolved>>,
1557 /// How fast this source writes, and how long it waits out a refusal.
1558 pacing: Pacing,
1559 /// When the last content-creating mutation finished, or the moment the furthest-out
1560 /// reserved slot releases the next one, whichever is later — so the one after it can be
1561 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1562 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1563 /// what it is measured from.
1564 last_mutation: Mutex<Option<Instant>>,
1565 /// The board as this process last read it, for the length of one command.
1566 ///
1567 /// A copy of a project used to re-read the whole board, paged, before writing each of
1568 /// its items, which is by far the largest part of a copy's request count and none of
1569 /// its work. Nothing else changes this board while a command runs — this source's own
1570 /// writes are the only writer — so one read answers them all.
1571 ///
1572 /// It is not a store of a user's work and it is not the cache the no-persistence
1573 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1574 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1575 /// an item this command created and then depends on resolves whether or not GitHub's
1576 /// own eventually-consistent read has caught up. A write to an item already on the
1577 /// board updates the entry here too, so what this holds is the last read plus this
1578 /// process's own writes rather than a snapshot taken before them.
1579 board_cache: Mutex<Option<Board>>,
1580 /// Each destination repository's node id, resolved once per repository
1581 /// rather than per issue created.
1582 ///
1583 /// A repository's node id does not change, and re-reading it for every issue of a copy
1584 /// spent one request per item on an answer this source already had. It is a map rather
1585 /// than one entry because a copy files each item in the repository its own
1586 /// `repositories` field names, so a plan across five repositories asks GitHub five
1587 /// times and not once per item.
1588 repository_cache: Mutex<BTreeMap<RepositoryTarget, String>>,
1589 /// What every request this source sends is recorded into.
1590 ///
1591 /// Ordinary code path, not a mode: [`Self::send_once`] records into it at the one place
1592 /// a request leaves this crate, so nothing has to be switched on for a session to be
1593 /// counted. It is shared rather than owned so a caller accounting for a whole session —
1594 /// its own schema verification, board lookups, residue sweep and cleanup beside this
1595 /// source's reads and writes — adds up one accounting instead of two. See
1596 /// [`accounting`] for what a record carries and what a session's spend is and is not.
1597 ledger: Arc<Accounting>,
1598}
1599
1600impl GitHubProjectsSource {
1601 /// Validate configuration and capture the named credential without exposing it.
1602 ///
1603 /// # Errors
1604 ///
1605 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1606 /// [`SourceError::Auth`] when the named credential is missing or empty.
1607 pub fn new(
1608 name: &SourceName,
1609 config: GitHubProjectsConfig,
1610 secrets: &dyn SecretResolver,
1611 ) -> Result<Self, SourceError> {
1612 Self::recording_into(name, config, secrets, Arc::new(Accounting::new()))
1613 }
1614
1615 /// The same, recording every request it sends into an accounting the caller holds too.
1616 ///
1617 /// [`Self::new`] is this with an accounting of its own. A caller that is also making
1618 /// its own calls to GitHub — a lane verifying a schema, sweeping residue or cleaning
1619 /// up — passes the one it records those into, so the session total accounts for the
1620 /// whole session rather than for this source's share of it.
1621 ///
1622 /// # Errors
1623 ///
1624 /// Exactly [`Self::new`]'s: [`SourceError::Config`] for a configuration this instance
1625 /// cannot use and [`SourceError::Auth`] when the named credential is missing or empty.
1626 pub fn recording_into(
1627 name: &SourceName,
1628 config: GitHubProjectsConfig,
1629 secrets: &dyn SecretResolver,
1630 ledger: Arc<Accounting>,
1631 ) -> Result<Self, SourceError> {
1632 if !valid_github_owner(&config.owner) {
1633 return Err(SourceError::Config {
1634 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1635 });
1636 }
1637 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1638 return Err(SourceError::Config {
1639 message: format!("project_number must be between 1 and {}", i32::MAX),
1640 });
1641 }
1642 if !valid_environment_name(&config.token_env) {
1643 return Err(SourceError::Config {
1644 message: "token_env must be a valid environment-variable name".into(),
1645 });
1646 }
1647 let repository = config
1648 .repository
1649 .as_deref()
1650 .map(RepositoryTarget::parse)
1651 .transpose()?;
1652 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1653 message: format!("endpoint is not a valid URL: {e}"),
1654 })?;
1655 if endpoint.scheme() != "https"
1656 && !(endpoint.scheme() == "http"
1657 && endpoint
1658 .host_str()
1659 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1660 {
1661 return Err(SourceError::Config {
1662 message:
1663 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1664 .into(),
1665 });
1666 }
1667 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1668 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1669 })?;
1670 Ok(Self {
1671 name: name.clone(),
1672 owner: config.owner,
1673 project_number: config.project_number,
1674 repository,
1675 endpoint,
1676 token,
1677 credential_name: config.token_env,
1678 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1679 client: Client::builder()
1680 .user_agent("onetaskgraph")
1681 .build()
1682 .map_err(|e| SourceError::Config {
1683 message: format!("cannot build HTTP client: {e}"),
1684 })?,
1685 created: Mutex::new(Vec::new()),
1686 pacing: Pacing::resolve(config.pacing, name)?,
1687 last_mutation: Mutex::new(None),
1688 board_cache: Mutex::new(None),
1689 repository_cache: Mutex::new(BTreeMap::new()),
1690 ledger,
1691 })
1692 }
1693
1694 /// A snapshot of every request this source has sent, and what each cost.
1695 ///
1696 /// A value to hold and compare rather than a borrow of the accounting itself, so two
1697 /// of them can sit side by side. When this source was built with
1698 /// [`Self::recording_into`] the snapshot is the whole shared session, which is the
1699 /// point of building it that way.
1700 #[must_use]
1701 pub fn accounting(&self) -> accounting::Session {
1702 self.ledger.snapshot()
1703 }
1704
1705 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1706 /// rate limit rather than handing it straight back as an error.
1707 ///
1708 /// Retrying is safe for every document here, including the mutations, and the reason
1709 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1710 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1711 /// this replays has already taken effect. An outcome this source cannot know — the
1712 /// send failed, or the body could not be read, so the mutation may well have landed —
1713 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1714 /// attempt. A duplicate write would come from replaying one of those, and none is
1715 /// replayed.
1716 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1717 let doing = operation_description(query);
1718 let mut waited = Duration::ZERO;
1719 let mut waits = 0_u32;
1720 let mut backoff = self.pacing.retry_backoff;
1721 loop {
1722 if is_mutation(query) {
1723 let spacing = self.reserve_mutation_slot();
1724 if !spacing.is_zero() {
1725 tokio::time::sleep(spacing).await;
1726 }
1727 }
1728 let attempt = self.send_once(query, &variables).await;
1729 if is_mutation(query) {
1730 self.finish_mutation();
1731 }
1732 let limited = match attempt {
1733 Ok(data) => return Ok(data),
1734 Err(Attempt::Failed(error)) => return Err(error),
1735 Err(Attempt::Limited(limited)) => limited,
1736 };
1737 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1738 // move that extends a secondary limit, so a hint below the schedule's own next
1739 // wait is raised to it.
1740 let wait = match limited.hint {
1741 Some(hint) => Duration::from_secs(hint).max(backoff),
1742 None => backoff,
1743 };
1744 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1745 // A wait of nothing spends none of the budget, so it is exhaustion rather
1746 // than a retry. `Pacing::resolve` rules out every way of configuring one
1747 // except a budget of zero, where reporting the first refusal is the ask.
1748 if wait.is_zero() || wait > remaining {
1749 return Err(limited.exhausted(
1750 doing,
1751 waits,
1752 waited,
1753 wait,
1754 self.pacing.retry_budget,
1755 ));
1756 }
1757 tokio::time::sleep(wait).await;
1758 waited += wait;
1759 waits += 1;
1760 backoff = backoff.saturating_mul(2);
1761 }
1762 }
1763
1764 /// The next moment a content-creating mutation may leave this source, as a wait from
1765 /// now.
1766 ///
1767 /// The slot is reserved under the lock and the waiting happens outside it, so two
1768 /// callers take two slots rather than the same one — and no lock is held across an
1769 /// await.
1770 ///
1771 /// The moment it is spaced from is the previous mutation's *completion*, which
1772 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1773 /// own is the wrong thing to measure from.
1774 fn reserve_mutation_slot(&self) -> Duration {
1775 if self.pacing.min_mutation_interval.is_zero() {
1776 return Duration::ZERO;
1777 }
1778 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1779 // it would turn an earlier failure into a second one for no gain.
1780 let mut last = self
1781 .last_mutation
1782 .lock()
1783 .unwrap_or_else(std::sync::PoisonError::into_inner);
1784 let now = Instant::now();
1785 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1786 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1787 let at = last.map_or(now, |previous| {
1788 previous
1789 .checked_add(self.pacing.min_mutation_interval)
1790 .map_or(now, |earliest| earliest.max(now))
1791 });
1792 *last = Some(at);
1793 at.saturating_duration_since(now)
1794 }
1795
1796 /// Record that a content-creating mutation has finished, so the next one is spaced
1797 /// from here rather than from the moment this one was released.
1798 ///
1799 /// This source can only choose when a request *departs*; the limiter counts when it
1800 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1801 /// departure from the last therefore hands the limiter a gap of the interval less that
1802 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1803 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1804 /// slower machine while passing on a quick one.
1805 ///
1806 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1807 /// previous request had already arrived before its response came back, so its arrival
1808 /// is no later than this moment, and the next mutation is released at least the
1809 /// interval after this moment and arrives no earlier than it is released: the gap the
1810 /// limiter measures is therefore at least the interval, whatever transit costs and on
1811 /// whatever platform. The price is that a mutation's own round trip no longer counts
1812 /// towards its spacing, which makes this source slightly slower than the configured
1813 /// rate rather than slightly faster — the safe side of a limit that punishes being
1814 /// wrong by refusing reads for the next fifty minutes.
1815 ///
1816 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1817 /// and one that never left costs only a wait nobody needed.
1818 fn finish_mutation(&self) {
1819 if self.pacing.min_mutation_interval.is_zero() {
1820 return;
1821 }
1822 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1823 let mut last = self
1824 .last_mutation
1825 .lock()
1826 .unwrap_or_else(std::sync::PoisonError::into_inner);
1827 let now = Instant::now();
1828 // `max` rather than an assignment: a concurrent caller may already have reserved a
1829 // slot further out, and completing this request must never pull that slot back in.
1830 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1831 }
1832
1833 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1834 /// failure that waiting cannot help — and recorded, whichever of the three it was.
1835 ///
1836 /// This is the one place a request leaves this crate, which is why the accounting is
1837 /// here rather than at each of the callers: a read path added later is counted without
1838 /// anybody remembering to count it, and
1839 /// `the_session_report_counts_every_request_the_board_served_and_what_each_cost` fails
1840 /// when one is not.
1841 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1842 let Attempted {
1843 result,
1844 limits,
1845 reported_cost,
1846 } = self.attempt(query, variables).await;
1847 // No `otherwise` name: every document this source sends is one of its own, and the
1848 // inventory gate on `graphql::DOCUMENTS` is what keeps that true.
1849 let sending = accounting::Request::graphql(query, variables, None, reported_cost);
1850 let outcome = match &result {
1851 Ok(_) => accounting::Outcome::Answered,
1852 Err(Attempt::Limited(_)) => accounting::Outcome::RateLimited,
1853 Err(Attempt::Failed(_)) => accounting::Outcome::Refused,
1854 };
1855 self.ledger.record(sending.finished(outcome, limits));
1856 result
1857 }
1858
1859 /// The attempt itself, with what its response said about the rate limit alongside.
1860 ///
1861 /// The two are returned together rather than recorded here because every one of the
1862 /// early exits below is a different outcome, and a record written at each of them is a
1863 /// record one of them can be added without.
1864 async fn attempt(&self, query: &str, variables: &Value) -> Attempted {
1865 let mut limits = accounting::RateLimit::default();
1866 let mut reported_cost = None;
1867 let result = self
1868 .attempted(query, variables, &mut limits, &mut reported_cost)
1869 .await;
1870 Attempted {
1871 result,
1872 limits,
1873 reported_cost,
1874 }
1875 }
1876
1877 /// One HTTP attempt, filling in what its response said about the rate limit as it goes.
1878 async fn attempted(
1879 &self,
1880 query: &str,
1881 variables: &Value,
1882 limits: &mut accounting::RateLimit,
1883 reported_cost: &mut Option<u64>,
1884 ) -> Result<Value, Attempt> {
1885 let response = self
1886 .client
1887 .post(self.endpoint.clone())
1888 .bearer_auth(self.token.expose_secret())
1889 .json(&json!({"query": query, "variables": variables}))
1890 .send()
1891 .await
1892 .map_err(|e| {
1893 Attempt::Failed(SourceError::Unavailable {
1894 message: format!("GitHub GraphQL request failed: {e}"),
1895 })
1896 })?;
1897 let status = response.status();
1898 let header = |name: &str| whole_seconds(response.headers().get(name));
1899 *limits = accounting::RateLimit::read(|name| {
1900 response
1901 .headers()
1902 .get(name)
1903 .and_then(|value| value.to_str().ok())
1904 .map(str::to_owned)
1905 });
1906 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
1907 // that are not text at all — is "not known to be exhausted". This never makes a
1908 // response a refusal on its own: it says which limiter a refusal is attributed to
1909 // and where its hint comes from, so a value this cannot read costs a hint rather
1910 // than an answer.
1911 let exhausted = response
1912 .headers()
1913 .get("x-ratelimit-remaining")
1914 .and_then(|value| value.to_str().ok())
1915 == Some("0");
1916 // `retry-after` is what GitHub asks for when it asks; when it does not and the
1917 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
1918 // which is the same question answered as an absolute time. Nothing else here is a
1919 // hint, and a schedule is what answers a refusal that carries none.
1920 let hint = header("retry-after").or_else(|| {
1921 exhausted
1922 .then(|| header("x-ratelimit-reset"))
1923 .flatten()
1924 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
1925 });
1926 // Read before it is parsed, because the evidence which tells a secondary rate
1927 // limit from a rejected credential is in the body of a response whose status says
1928 // only "forbidden" — and a non-success response was never parsed at all.
1929 let body = response.text().await.map_err(|e| {
1930 Attempt::Failed(SourceError::Unavailable {
1931 message: format!("GitHub GraphQL response could not be read: {e}"),
1932 })
1933 })?;
1934 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
1935 return Err(Attempt::Limited(Limited { limiter, hint }));
1936 }
1937 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
1938 return Err(Attempt::Failed(SourceError::Auth {
1939 message: format!(
1940 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
1941 ),
1942 }));
1943 }
1944 if !status.is_success() {
1945 return Err(Attempt::Failed(SourceError::Unavailable {
1946 message: format!("GitHub GraphQL returned HTTP {status}"),
1947 }));
1948 }
1949 // GitHub reports what a call cost only when the document asked it to, and no
1950 // document this source sends does — so this is `None` here and carries the figure
1951 // for a caller whose own document selects `rateLimit { cost }`. What it must never
1952 // pick up is a `dryRun` probe's cost, which is some other document's.
1953 *reported_cost = serde_json::from_str::<Value>(&body)
1954 .ok()
1955 .as_ref()
1956 .and_then(|body| body.pointer("/data/rateLimit/cost"))
1957 .and_then(Value::as_u64);
1958 self.answer(&body).map_err(Attempt::Failed)
1959 }
1960
1961 /// What one successful HTTP response says, once its GraphQL errors are read.
1962 fn answer(&self, body: &str) -> Result<Value, SourceError> {
1963 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
1964 message: format!("GitHub returned invalid JSON: {e}"),
1965 })?;
1966 let errors = body
1967 .get("errors")
1968 .map(|value| {
1969 value.as_array().ok_or_else(|| SourceError::Malformed {
1970 message: "GitHub response errors is not an array".into(),
1971 })
1972 })
1973 .transpose()?;
1974 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
1975 let messages = errors
1976 .iter()
1977 .filter_map(|e| e.get("message").and_then(Value::as_str))
1978 .collect::<Vec<_>>()
1979 .join("; ");
1980 let message = if messages.is_empty() {
1981 "GitHub returned GraphQL errors".into()
1982 } else {
1983 messages
1984 };
1985 let normalized = message.to_ascii_lowercase();
1986 if normalized.contains("resource not accessible") || normalized.contains("scope") {
1987 return Err(SourceError::Auth {
1988 message: format!(
1989 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
1990 self.credential_name
1991 ),
1992 });
1993 }
1994 return Err(SourceError::Refused { message });
1995 }
1996 body.get("data")
1997 .filter(|data| data.is_object())
1998 .cloned()
1999 .ok_or_else(|| SourceError::Malformed {
2000 message: "GitHub response has no data object".into(),
2001 })
2002 }
2003
2004 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
2005 // GraphQL cannot independently page them inside the outer item page. This source page is
2006 // deliberately bounded at that published maximum; the live drift journey exercises it.
2007 async fn board_page(
2008 &self,
2009 items_after: Option<&str>,
2010 items_first: u32,
2011 ) -> Result<Value, SourceError> {
2012 let data = self
2013 .graphql(
2014 graphql::BOARD,
2015 json!({"owner":self.owner,"number":self.project_number,
2016 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
2017 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
2018 )
2019 .await?;
2020 data.pointer("/owner/projectV2")
2021 .filter(|v| !v.is_null())
2022 .cloned()
2023 .ok_or_else(|| SourceError::Refused {
2024 message: format!(
2025 "GitHub project {}/{} was not found or is not visible to the token",
2026 self.owner, self.project_number
2027 ),
2028 })
2029 }
2030
2031 /// The search that finds the issues of this board, narrowed by `also` when it is
2032 /// given.
2033 ///
2034 /// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
2035 /// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
2036 /// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
2037 /// here on purpose — GitHub accepts it and silently ignores it, so a project is told
2038 /// from a task by the `parent` field each issue carries rather than by the search.
2039 fn board_search(&self, also: Option<&str>) -> String {
2040 let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
2041 match also {
2042 Some(also) => format!("{scope} {also}"),
2043 None => scope,
2044 }
2045 }
2046
2047 /// One issue this source reached directly, as the board item a read of the board would
2048 /// have produced — or `None` when this board does not hold it.
2049 ///
2050 /// The board half of an issue rides along on `Issue.projectItems`, so the value handed
2051 /// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
2052 /// item's own id, that item's field values, and the issue as its content. One resolver
2053 /// for both routes is what makes an issue read through a search, through its own node
2054 /// id, or through its project's sub-issues report the same title, the same status, the
2055 /// same labels and the same qualified id.
2056 ///
2057 /// An issue with no entry for *this* board is not this source's to report, which is
2058 /// what keeps an id naming some other repository's issue from being answered as an item
2059 /// of this board. That answer is given about an **exhausted** connection and never
2060 /// about an unread page: the entry is looked for on the page in hand, and only if that
2061 /// page reports more of the connection, in [`Self::board_membership`]'s walk of the
2062 /// rest of it.
2063 async fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
2064 if optional_str(issue, "__typename")? != Some("Issue") {
2065 return Ok(None);
2066 }
2067 let memberships = issue
2068 .get("projectItems")
2069 .ok_or_else(|| SourceError::Malformed {
2070 message: "GitHub issue is missing projectItems".into(),
2071 })?;
2072 let nodes = memberships
2073 .get("nodes")
2074 .and_then(Value::as_array)
2075 .ok_or_else(|| SourceError::Malformed {
2076 message: "GitHub issue projectItems.nodes is not an array".into(),
2077 })?;
2078 let held = match self.board_entry(nodes) {
2079 Some(held) => held.clone(),
2080 None => {
2081 let info = memberships
2082 .get("pageInfo")
2083 .ok_or_else(|| SourceError::Malformed {
2084 message: "GitHub issue projectItems has no pageInfo".into(),
2085 })?;
2086 // The page held no entry for this board. Whether that means the issue is
2087 // not on it is a question about the rest of the connection, and only a
2088 // connection with no rest answers it here.
2089 if !required_bool(info, "hasNextPage")? {
2090 return Ok(None);
2091 }
2092 let cursor = required_str(info, "endCursor")?;
2093 validate_cursor_progress(None, cursor)?;
2094 let issue_id = required_str(issue, "id")?;
2095 match self.board_membership(issue_id, cursor).await? {
2096 Some(held) => held,
2097 None => return Ok(None),
2098 }
2099 }
2100 };
2101 let item = json!({
2102 "id": required_str(&held, "id")?,
2103 "fieldValues": held.get("fieldValues"),
2104 "content": issue,
2105 });
2106 self.resolve(&item)
2107 }
2108
2109 /// This board's own entry among one page of an issue's `Issue.projectItems`.
2110 ///
2111 /// One spelling of *which membership is this board's*, so the page a read carries and
2112 /// the pages [`Self::board_membership`] walks are searched by the same rule.
2113 fn board_entry<'a>(&self, nodes: &'a [Value]) -> Option<&'a Value> {
2114 nodes.iter().find(|node| {
2115 node.pointer("/project/number").and_then(Value::as_u64)
2116 == Some(u64::from(self.project_number))
2117 })
2118 }
2119
2120 /// The rest of one issue's board memberships, from `after`, for this board's entry.
2121 ///
2122 /// The recovery read: a page of memberships that holds no entry for this board says
2123 /// nothing about the memberships past it, so the connection is walked to exhaustion
2124 /// before an issue is reported as one this board does not hold. `Ok(None)` is that
2125 /// positive answer — the whole connection was read and no entry named this board —
2126 /// rather than a failure, and the walk is held to
2127 /// [`validate_cursor_progress`] like every other page walk here, so a source answering
2128 /// with a cursor that does not advance is refused instead of spun on.
2129 async fn board_membership(
2130 &self,
2131 issue: &str,
2132 after: &str,
2133 ) -> Result<Option<Value>, SourceError> {
2134 let mut after = after.to_owned();
2135 loop {
2136 let data = self
2137 .graphql(
2138 graphql::ISSUE_BOARD_ITEMS,
2139 json!({"id":issue,"first":MAX_PAGE_SIZE,"after":after,
2140 "nestedFirst":NESTED_PAGE_SIZE}),
2141 )
2142 .await?;
2143 let Some(connection) = data
2144 .pointer("/node/projectItems")
2145 .filter(|value| !value.is_null())
2146 else {
2147 // The id resolved to nothing, or to something with no memberships to walk —
2148 // which is the same answer as a connection holding no entry for this board.
2149 return Ok(None);
2150 };
2151 let nodes = connection
2152 .get("nodes")
2153 .and_then(Value::as_array)
2154 .ok_or_else(|| SourceError::Malformed {
2155 message: "GitHub issue projectItems.nodes is not an array".into(),
2156 })?;
2157 if let Some(held) = self.board_entry(nodes) {
2158 return Ok(Some(held.clone()));
2159 }
2160 let info = connection
2161 .get("pageInfo")
2162 .ok_or_else(|| SourceError::Malformed {
2163 message: "GitHub issue projectItems has no pageInfo".into(),
2164 })?;
2165 let next = required_bool(info, "hasNextPage")?
2166 .then(|| required_str(info, "endCursor"))
2167 .transpose()?;
2168 match next {
2169 Some(next) => {
2170 validate_cursor_progress(Some(&after), next)?;
2171 after = next.to_owned();
2172 }
2173 None => return Ok(None),
2174 }
2175 }
2176 }
2177
2178 /// One page of a board-scoped issue search, and where the next page resumes.
2179 async fn search_page(
2180 &self,
2181 search: &str,
2182 first: u32,
2183 after: Option<&str>,
2184 ) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
2185 let data = self
2186 .graphql(
2187 graphql::SEARCH_ISSUES,
2188 json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
2189 "after":after,"nestedFirst":NESTED_PAGE_SIZE,
2190 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2191 )
2192 .await?;
2193 let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
2194 message: "GitHub search response has no search connection".into(),
2195 })?;
2196 let mut found = Vec::new();
2197 for node in connection
2198 .get("nodes")
2199 .and_then(Value::as_array)
2200 .ok_or_else(|| SourceError::Malformed {
2201 message: "GitHub search nodes is not an array".into(),
2202 })?
2203 {
2204 if let Some(resolved) = self.resolve_issue(node).await? {
2205 found.push(resolved);
2206 }
2207 }
2208 let info = connection
2209 .get("pageInfo")
2210 .ok_or_else(|| SourceError::Malformed {
2211 message: "GitHub search connection has no pageInfo".into(),
2212 })?;
2213 let next = required_bool(info, "hasNextPage")?
2214 .then(|| required_str(info, "endCursor"))
2215 .transpose()?
2216 .map(str::to_owned);
2217 if let Some(next) = &next {
2218 validate_cursor_progress(after, next)?;
2219 }
2220 Ok((found, next))
2221 }
2222
2223 /// Every issue this board holds, walked to exhaustion, completed with what this run
2224 /// wrote.
2225 ///
2226 /// The completion is not an optimisation and it is not a cache: GitHub's issue search
2227 /// is an index and is eventually consistent, so an issue this run created seconds ago
2228 /// is routinely absent from it, and a project listed straight after being written would
2229 /// otherwise be missing from its own board. What is added back is only what this
2230 /// process itself wrote, out of [`Self::created`], which lives and dies with the
2231 /// process.
2232 async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
2233 let mut after: Option<String> = None;
2234 let mut found = Vec::new();
2235 let search = self.board_search(None);
2236 loop {
2237 let (page, next) = self
2238 .search_page(&search, MAX_PAGE_SIZE, after.as_deref())
2239 .await?;
2240 found.extend(page);
2241 match next {
2242 Some(next) => after = Some(next),
2243 None => break,
2244 }
2245 }
2246 self.completed_with_written(found, |_| true)
2247 }
2248
2249 /// `found`, with everything this run wrote that `keep` accepts and the read did not
2250 /// report.
2251 ///
2252 /// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
2253 /// at all: the search index is behind, and a node read of an item filed moments ago can
2254 /// be too.
2255 fn completed_with_written(
2256 &self,
2257 mut found: Vec<Resolved>,
2258 keep: impl Fn(&Resolved) -> bool,
2259 ) -> Result<Vec<Resolved>, SourceError> {
2260 for own in self.created()?.iter().filter(|own| keep(own)) {
2261 if !found.iter().any(|item| item.id == own.id) {
2262 found.push(own.clone());
2263 }
2264 }
2265 Ok(found)
2266 }
2267
2268 /// What resolving one node id reached.
2269 ///
2270 /// Three answers rather than an `Option`, because a board *draft* is none of the other
2271 /// two: it is not an issue, it has no node of its own this source can read the board
2272 /// half off, and its only home is the board's own item connection — so a read of one
2273 /// is completed from there rather than reported as nothing.
2274 async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
2275 let asked = self
2276 .graphql(
2277 graphql::ISSUE,
2278 json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
2279 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2280 )
2281 .await;
2282 let data = match asked {
2283 Ok(data) => data,
2284 // A string that is not a node id at all is not a failure to report: it is an id
2285 // this board does not hold, which is what every read of one already answers.
2286 Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
2287 Err(error) => return Err(error),
2288 };
2289 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
2290 return Ok(Reached::Nothing);
2291 };
2292 if optional_str(node, "__typename")? == Some("DraftIssue") {
2293 return Ok(Reached::Draft);
2294 }
2295 Ok(match self.resolve_issue(node).await? {
2296 Some(item) => Reached::Held(Box::new(item)),
2297 None => Reached::Nothing,
2298 })
2299 }
2300
2301 /// One item of this board by its own id, whatever kind it is.
2302 ///
2303 /// Resolved from the identifier alone: no search, board-wide or otherwise. What this
2304 /// run wrote is read first, because a node read of an item created moments ago can
2305 /// still be behind the board field values written onto it — see [`Self::created`].
2306 async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
2307 if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
2308 return Ok(Some(own.clone()));
2309 }
2310 match self.reach(id).await? {
2311 Reached::Held(item) => Ok(Some(*item)),
2312 Reached::Nothing => Ok(None),
2313 // The one read that still costs the board: a draft lives nowhere else.
2314 Reached::Draft => Ok(self
2315 .board()
2316 .await?
2317 .items
2318 .into_iter()
2319 .find(|item| item.id == *id)),
2320 }
2321 }
2322
2323 /// Everything filed under one issue of this board, walked to exhaustion — or `None`
2324 /// when that id names nothing here with a sub-issue relationship to walk.
2325 ///
2326 /// `None` and an empty answer are different: `None` is *this is not an issue of this
2327 /// GitHub*, which is what sends a project selector on to be read as a name, and an
2328 /// empty vector is a project that holds nothing.
2329 async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
2330 let mut after: Option<String> = None;
2331 let mut children = Vec::new();
2332 loop {
2333 let asked = self
2334 .graphql(
2335 graphql::SUB_ISSUES,
2336 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
2337 "nestedFirst":NESTED_PAGE_SIZE,
2338 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2339 )
2340 .await;
2341 let data = match asked {
2342 Ok(data) => data,
2343 // A string that is not a node id at all is not a failure to report: it is
2344 // the ordinary answer to a selector naming a project by its name.
2345 Err(error) if unresolvable_node(&error) => return Ok(None),
2346 Err(error) => return Err(error),
2347 };
2348 let Some(connection) = data
2349 .pointer("/node/subIssues")
2350 .filter(|value| !value.is_null())
2351 else {
2352 // No such node, or one with no sub-issue relationship — a board draft is
2353 // the one this board can really hold.
2354 return Ok(None);
2355 };
2356 for node in connection
2357 .get("nodes")
2358 .and_then(Value::as_array)
2359 .ok_or_else(|| SourceError::Malformed {
2360 message: "GitHub subIssues.nodes is not an array".into(),
2361 })?
2362 {
2363 if let Some(resolved) = self.resolve_issue(node).await? {
2364 children.push(resolved);
2365 }
2366 }
2367 let info = connection
2368 .get("pageInfo")
2369 .ok_or_else(|| SourceError::Malformed {
2370 message: "GitHub subIssues connection has no pageInfo".into(),
2371 })?;
2372 let next = required_bool(info, "hasNextPage")?
2373 .then(|| required_str(info, "endCursor"))
2374 .transpose()?;
2375 match next {
2376 Some(next) => {
2377 validate_cursor_progress(after.as_deref(), next)?;
2378 after = Some(next.to_owned());
2379 }
2380 None => return Ok(Some(children)),
2381 }
2382 }
2383 }
2384
2385 /// Which issue of this board a project *name* is, or `None` when none is.
2386 ///
2387 /// One bounded query which filters on that name at the server, rather than a walk of
2388 /// every issue the board holds. The name is compared again here: the qualifier narrows
2389 /// what GitHub sends, and this source decides what it names.
2390 async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
2391 let search = self.board_search(Some(&title_qualifier(name)));
2392 let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
2393 Ok(candidates
2394 .into_iter()
2395 .find(|item| {
2396 item.kind == BoardKind::Work(ItemKind::Project)
2397 && item.title.eq_ignore_ascii_case(name)
2398 })
2399 .map(|item| item.id))
2400 }
2401
2402 /// Everything filed under one project of this board: the sub-issues of the issue that
2403 /// project is.
2404 ///
2405 /// Tasks *and* documents, because a document filed under a project is a sub-issue of it
2406 /// too — the caller keeps the kind it asked for. Nothing about this grows as the board
2407 /// gains projects, or as another project gains tasks.
2408 ///
2409 /// A qualified id names the issue and is asked for its sub-issues directly: one
2410 /// request, no search of any kind. Only a selector GitHub cannot resolve that way is
2411 /// read as a project *name*, which costs the one bounded search
2412 /// [`Self::project_by_name`] makes.
2413 async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
2414 let (project, children) = match self.sub_issues(selector).await? {
2415 Some(children) => (selector.clone(), children),
2416 None => match self.project_by_name(&selector.0).await? {
2417 Some(project) => {
2418 let children = self.sub_issues(&project).await?.unwrap_or_default();
2419 (project, children)
2420 }
2421 None => return Ok(Vec::new()),
2422 },
2423 };
2424 self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
2425 }
2426
2427 /// Every item on the board, with the one board identity they all share.
2428 ///
2429 /// See [`Self::board_cache`]. The completion from `created` happens on every call
2430 /// rather than once, which is what the cache could otherwise have broken.
2431 async fn board(&self) -> Result<Board, SourceError> {
2432 let cached = self.board_cache()?.clone();
2433 let mut board = match cached {
2434 Some(board) => board,
2435 None => {
2436 let read = self.read_board().await?;
2437 *self.board_cache()? = Some(read.clone());
2438 read
2439 }
2440 };
2441 for own in self.created()?.iter() {
2442 if !board.items.iter().any(|item| item.id == own.id) {
2443 board.items.push(own.clone());
2444 }
2445 }
2446 Ok(board)
2447 }
2448
2449 /// This process's own view of the board, or the refusal a poisoned lock is.
2450 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
2451 self.board_cache
2452 .lock()
2453 .map_err(|_| SourceError::Unavailable {
2454 message: "this source's view of the board was left inconsistent by an earlier \
2455 failure; next: run the command again"
2456 .into(),
2457 })
2458 }
2459
2460 /// Bring this process's own view of the board up to an item it has just written.
2461 ///
2462 /// A created item goes to `created`, which is what completes a board read GitHub's own
2463 /// eventual consistency has left behind. An item that was already there is replaced
2464 /// where it sits, so a second write of it in the same command reads its real parent
2465 /// rather than the one it had before the first write.
2466 ///
2467 /// "Where it sits" is two places, and missing the first leaves a stale record that
2468 /// wins: an item this same run created is held in `created` and not in the cached
2469 /// board, and `board` completes the cached board *from* `created`, so replacing only
2470 /// the cached copy of such an item replaces nothing and the read still reports the
2471 /// title it was created with.
2472 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
2473 if created {
2474 self.created()?.push(item);
2475 return Ok(());
2476 }
2477 {
2478 let mut own = self.created()?;
2479 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
2480 *held = item;
2481 return Ok(());
2482 }
2483 }
2484 if let Some(board) = self.board_cache()?.as_mut()
2485 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
2486 {
2487 *held = item;
2488 }
2489 Ok(())
2490 }
2491
2492 /// Forget one item this process has just deleted, from both halves of its own view.
2493 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
2494 self.created()?.retain(|own| own.id != *id);
2495 if let Some(board) = self.board_cache()?.as_mut() {
2496 board.items.retain(|item| item.id != *id);
2497 }
2498 Ok(())
2499 }
2500
2501 /// Every page of the board, read from GitHub.
2502 async fn read_board(&self) -> Result<Board, SourceError> {
2503 let mut after: Option<String> = None;
2504 let mut items = Vec::new();
2505 let mut board;
2506 loop {
2507 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
2508 for item in page
2509 .pointer("/items/nodes")
2510 .and_then(Value::as_array)
2511 .ok_or_else(|| SourceError::Malformed {
2512 message: "GitHub project items.nodes is not an array".into(),
2513 })?
2514 {
2515 if let Some(resolved) = self.resolve(item)? {
2516 items.push(resolved);
2517 }
2518 }
2519 let info = page
2520 .pointer("/items/pageInfo")
2521 .ok_or_else(|| SourceError::Malformed {
2522 message: "GitHub project items have no pageInfo".into(),
2523 })?;
2524 let has_next = required_bool(info, "hasNextPage")?;
2525 let next = has_next
2526 .then(|| required_str(info, "endCursor"))
2527 .transpose()?;
2528 board = page.clone();
2529 match next {
2530 Some(next) => {
2531 validate_cursor_progress(after.as_deref(), next)?;
2532 after = Some(next.to_owned());
2533 }
2534 None => break,
2535 }
2536 }
2537 Ok(Board {
2538 id: required_str(&board, "id")?.to_owned(),
2539 fields: board.get("fields").cloned().unwrap_or(Value::Null),
2540 items,
2541 })
2542 }
2543
2544 /// The items this source has created, for completing a board read that is behind.
2545 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
2546 self.created.lock().map_err(|_| SourceError::Unavailable {
2547 message: "this source's record of what it created in this run was left \
2548 inconsistent by an earlier failure; next: run the command again"
2549 .into(),
2550 })
2551 }
2552
2553 /// One board item as this source reports it, or `None` for content it ignores.
2554 ///
2555 /// A pull request is neither a project nor a task — it is somebody's change, not a
2556 /// unit of plan — and an item whose content the token cannot see has nothing to
2557 /// report at all.
2558 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
2559 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
2560 message: "GitHub project item is missing content".into(),
2561 })?;
2562 if content.is_null() {
2563 return Ok(None);
2564 }
2565 let content_kind = match required_str(content, "__typename")? {
2566 "Issue" => ContentKind::Issue,
2567 "DraftIssue" => ContentKind::DraftIssue,
2568 _ => return Ok(None),
2569 };
2570 let field_values = item
2571 .get("fieldValues")
2572 .ok_or_else(|| SourceError::Malformed {
2573 message: "GitHub project item is missing fieldValues".into(),
2574 })?;
2575 complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
2576 let nodes = field_values
2577 .get("nodes")
2578 .and_then(Value::as_array)
2579 .ok_or_else(|| SourceError::Malformed {
2580 message: "GitHub project item fieldValues.nodes is not an array".into(),
2581 })?;
2582 if let Some(labels) = content.get("labels") {
2583 complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
2584 }
2585 let (body, slot) = metadata_body(optional_str(content, "body")?.map(str::to_owned))?;
2586 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
2587 .map(|id| NativeId(id.to_owned()));
2588 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
2589 // to read one from; it is a task, and never a project.
2590 let sub_issues = match content_kind {
2591 ContentKind::Issue => sub_issue_total(content)?,
2592 ContentKind::DraftIssue => 0,
2593 };
2594 let content_id = required_str(content, "id")?;
2595 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
2596 message: format!("GitHub issue {content_id}: {message}"),
2597 })?;
2598 let raw_title = required_str(content, "title")?;
2599 // The design prefix is read *first*, before either of the two rules that separate
2600 // a project from a task. A document is not work whatever sub-issues it has and
2601 // whatever marker it carries, and reading the prefix later would make a design
2602 // issue with none of either an empty project.
2603 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
2604 BoardKind::Document
2605 } else if parent.is_some() {
2606 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
2607 // under a project is that project's task even when it has sub-issues of its
2608 // own.
2609 BoardKind::Work(ItemKind::Task)
2610 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
2611 BoardKind::Work(ItemKind::Project)
2612 } else {
2613 BoardKind::Work(ItemKind::Task)
2614 };
2615 // The title a person wrote, which for a document is the one without the prefix —
2616 // the same way `content` above is the body without this source's metadata slot.
2617 let title = match kind {
2618 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
2619 BoardKind::Work(_) => raw_title.to_owned(),
2620 };
2621 let own_repository = content
2622 .pointer("/repository/nameWithOwner")
2623 .and_then(Value::as_str)
2624 .map(|origin| Repository::try_from(format!("{}/{origin}", RepositoryTarget::HOST)))
2625 .transpose()
2626 .map_err(|message| SourceError::Malformed { message })?;
2627 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
2628 Repository::from_metadata(&slot)
2629 .map_err(|message| SourceError::Malformed { message })?
2630 } else {
2631 own_repository.clone().into_iter().collect()
2632 };
2633 Ok(Some(Resolved {
2634 item_id: required_str(item, "id")?.to_owned(),
2635 id: NativeId(content_id.to_owned()),
2636 content_kind,
2637 kind,
2638 title,
2639 body: body.filter(|value| !value.is_empty()),
2640 status: self.status(item, content)?,
2641 labels: labels(content)?,
2642 parent,
2643 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
2644 url: optional_str(content, "url")?.map(str::to_owned),
2645 created_at: optional_time(content, "createdAt")?,
2646 updated_at: optional_time(content, "updatedAt")?,
2647 own_repository,
2648 repositories,
2649 slot,
2650 }))
2651 }
2652
2653 /// The status one board item reports.
2654 ///
2655 /// The closed state decides the category and the `Status` option decides the name, so
2656 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
2657 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
2658 /// duplicate is not finished work, and calling it done is a lie the next copy would
2659 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
2660 /// it is read permissively rather than refused — reads are faithful, and refusals
2661 /// belong on writes.
2662 fn status(&self, item: &Value, content: &Value) -> Result<Status, SourceError> {
2663 let nodes = item
2664 .pointer("/fieldValues/nodes")
2665 .and_then(Value::as_array)
2666 .expect("resolve validates fieldValues.nodes before mapping status");
2667 let option = nodes
2668 .iter()
2669 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
2670 .map(|value| required_str(value, "name"))
2671 .transpose()?;
2672 let state = optional_str(content, "state")?;
2673 if state == Some("CLOSED") {
2674 let category = match optional_str(content, "stateReason")? {
2675 None | Some("COMPLETED") => StatusCategory::Done,
2676 Some("NOT_PLANNED") => StatusCategory::Cancelled,
2677 Some(_) => StatusCategory::Unknown,
2678 };
2679 let fallback = match category {
2680 StatusCategory::Done => "Done",
2681 StatusCategory::Cancelled => "Cancelled",
2682 _ => "Closed",
2683 };
2684 return Ok(Status {
2685 category,
2686 name: option.unwrap_or(fallback).to_owned(),
2687 });
2688 }
2689 let name = option.unwrap_or("Open").to_owned();
2690 Ok(Status {
2691 category: self
2692 .statuses
2693 .category_of(&name)
2694 .unwrap_or(StatusCategory::Unknown),
2695 name,
2696 })
2697 }
2698
2699 /// The board Status option this write selects, or the refusal that says why not.
2700 ///
2701 /// For a column target the option is what the status *is*, so a board that has no such
2702 /// option is a refusal naming the status and the instance. For a closed target the
2703 /// issue's own state carries the category, and the option carries only the name a
2704 /// reader reports — so an option spelled the way this status is spelled is selected
2705 /// when the board has one, and nothing is refused when it does not.
2706 fn column_for(
2707 &self,
2708 board: &Board,
2709 status: &Status,
2710 target: &StatusTarget,
2711 ) -> Result<Option<(String, String)>, SourceError> {
2712 let (wanted, required) = match target {
2713 StatusTarget::Column(wanted) => (wanted.as_str(), true),
2714 StatusTarget::Closed(_) => (status.name.as_str(), false),
2715 StatusTarget::Disabled => return Ok(None),
2716 };
2717 let missing = |detail: &str| SourceError::Refused {
2718 message: format!(
2719 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
2720 category_name(status.category),
2721 self.name,
2722 category_name(status.category)
2723 ),
2724 };
2725 let Some(field) = Board::field(&board.fields, "Status")? else {
2726 return if required {
2727 Err(missing("this board has no Status field"))
2728 } else {
2729 Ok(None)
2730 };
2731 };
2732 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
2733 return if required {
2734 Err(missing(
2735 "this board's Status field is not a single-select field",
2736 ))
2737 } else {
2738 Ok(None)
2739 };
2740 }
2741 let option = field
2742 .get("options")
2743 .and_then(Value::as_array)
2744 .and_then(|options| {
2745 options.iter().find(|option| {
2746 option
2747 .get("name")
2748 .and_then(Value::as_str)
2749 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
2750 })
2751 });
2752 match option {
2753 None if required => Err(missing("this board does not have it")),
2754 None => Ok(None),
2755 Some(option) => Ok(Some((
2756 required_str(field, "id")?.to_owned(),
2757 required_str(option, "id")?.to_owned(),
2758 ))),
2759 }
2760 }
2761
2762 /// This instance's target for a category, refusing one it has disabled.
2763 ///
2764 /// Nothing here mutates the board's option set to make room for a status. GitHub
2765 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
2766 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
2767 /// field and every item's status.
2768 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
2769 let target = self.statuses.target(category).clone();
2770 if target != StatusTarget::Disabled {
2771 return Ok(target);
2772 }
2773 Err(SourceError::Refused {
2774 message: if category == StatusCategory::Draft {
2775 format!(
2776 "status draft is disabled for source {}: draft is incompatible with this \
2777 integration because GitHub draft issues cannot have sub-issues, and this \
2778 source stores a project's tasks as its issue's sub-issues",
2779 self.name
2780 )
2781 } else {
2782 format!(
2783 "status {} is disabled for source {}; set status_mapping.{} of this source \
2784 to a board Status option name or to a closed state",
2785 category_name(category),
2786 self.name,
2787 category_name(category)
2788 )
2789 },
2790 })
2791 }
2792
2793 async fn set_item_field(
2794 &self,
2795 board_id: &str,
2796 item_id: &str,
2797 field_id: &str,
2798 value: Value,
2799 ) -> Result<(), SourceError> {
2800 let data = self
2801 .graphql(
2802 graphql::UPDATE_FIELD,
2803 json!({"input":{
2804 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
2805 }}),
2806 )
2807 .await?;
2808 let returned = data
2809 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
2810 .ok_or_else(|| SourceError::Malformed {
2811 message: "GitHub field update returned no project item".into(),
2812 })?;
2813 if required_str(returned, "id")? != item_id {
2814 return Err(SourceError::Malformed {
2815 message: "GitHub field update returned the wrong project item".into(),
2816 });
2817 }
2818 Ok(())
2819 }
2820
2821 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
2822 let mut after: Option<String> = None;
2823 let mut ids = Vec::new();
2824 loop {
2825 let data = self
2826 .graphql(
2827 graphql::ISSUE_DEPENDENCIES,
2828 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
2829 )
2830 .await?;
2831 let connection =
2832 data.pointer("/node/blockedBy")
2833 .ok_or_else(|| SourceError::Malformed {
2834 message: "GitHub dependency response has no blockedBy connection".into(),
2835 })?;
2836 ids.extend(
2837 connection
2838 .get("nodes")
2839 .and_then(Value::as_array)
2840 .ok_or_else(|| SourceError::Malformed {
2841 message: "GitHub dependency response nodes is not an array".into(),
2842 })?
2843 .iter()
2844 .map(|value| required_str(value, "id").map(str::to_owned))
2845 .collect::<Result<Vec<_>, _>>()?,
2846 );
2847 let next = next_cursor(connection)?;
2848 if let Some(next) = &next {
2849 validate_cursor_progress(after.as_deref(), &next.0)?;
2850 }
2851 after = next.map(|cursor| cursor.0);
2852 if after.is_none() {
2853 return Ok(ids);
2854 }
2855 }
2856 }
2857
2858 async fn dependencies(
2859 &self,
2860 id: &NativeId,
2861 near_kind: ItemKind,
2862 direction: Direction,
2863 page: &PageRequest,
2864 ) -> Result<Page<DependencyEdge>, SourceError> {
2865 validate_page(page)?;
2866 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
2867 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
2868 let recorded = recorded_offset(cursor, direction)?;
2869 // Asked for even in the recorded phase, whose page reads nothing from the
2870 // connection: `__typename` is what says whether this item has a native
2871 // relationship at all, and that is what decides which far ends the reserved key is
2872 // allowed to hold.
2873 let data = self
2874 .graphql(
2875 graphql::ISSUE_DEPENDENCIES,
2876 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
2877 "after":if recorded.is_some() {None} else {cursor}}),
2878 )
2879 .await?;
2880 let node =
2881 data.get("node")
2882 .filter(|v| !v.is_null())
2883 .ok_or_else(|| SourceError::Refused {
2884 message: format!(
2885 "GitHub item {} was not found or does not support dependencies",
2886 id.0
2887 ),
2888 })?;
2889 let connection_name = match direction {
2890 Direction::DependsOn => "blockedBy",
2891 Direction::DependedOnBy => "blocking",
2892 };
2893 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
2894 // named natively and the reserved key may hold any far end. An issue's connections
2895 // hold issues, and this source reads them at the near item's own level.
2896 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
2897 if let Some(offset) = recorded {
2898 return Ok(recorded_page(
2899 self.recorded_edges(id, near_kind, direction, natively_names)
2900 .await?,
2901 offset,
2902 limit,
2903 ));
2904 }
2905 if natively_names.is_none() {
2906 return Ok(recorded_page(
2907 self.recorded_edges(id, near_kind, direction, natively_names)
2908 .await?,
2909 0,
2910 limit,
2911 ));
2912 }
2913 let connection = node
2914 .get(connection_name)
2915 .ok_or_else(|| SourceError::Malformed {
2916 message: "GitHub dependency response is missing its connection".into(),
2917 })?;
2918 let nodes = connection
2919 .get("nodes")
2920 .and_then(Value::as_array)
2921 .ok_or_else(|| SourceError::Malformed {
2922 message: "GitHub dependency response nodes is not an array".into(),
2923 })?;
2924 // `from` depends on `to`, always. GitHub spells the same relationship from either
2925 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
2926 // it — so the near item is `from` in one direction and `to` in the other.
2927 let items = nodes
2928 .iter()
2929 .map(|value| {
2930 let related = NativeId(required_str(value, "id")?.into());
2931 let related_kind = related_kind(value)?;
2932 let (from, to) = match direction {
2933 Direction::DependsOn => (
2934 DependencyEndpoint::from_native(id.clone(), near_kind),
2935 DependencyEndpoint::from_native(related, related_kind),
2936 ),
2937 Direction::DependedOnBy => (
2938 DependencyEndpoint::from_native(related, related_kind),
2939 DependencyEndpoint::from_native(id.clone(), near_kind),
2940 ),
2941 };
2942 Ok(DependencyEdge {
2943 from,
2944 to,
2945 kind: DependencyKind::Blocks,
2946 })
2947 })
2948 .collect::<Result<Vec<_>, SourceError>>()?;
2949 let mut next = next_cursor(connection)?;
2950 if let Some(next) = &next {
2951 validate_cursor_progress(cursor, &next.0)?;
2952 }
2953 if next.is_none()
2954 && !self
2955 .recorded_edges(id, near_kind, direction, natively_names)
2956 .await?
2957 .is_empty()
2958 {
2959 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
2960 }
2961 Ok(Page { items, next })
2962 }
2963
2964 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
2965 /// a far end in another source has to live: no GitHub issue relationship can name one.
2966 ///
2967 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
2968 /// source never writes one down.
2969 ///
2970 /// The metadata lives in the item's own body slot, so reading it costs one board scan.
2971 /// That is why it happens once the native connection is spent rather than on every
2972 /// page.
2973 async fn recorded_edges(
2974 &self,
2975 id: &NativeId,
2976 near_kind: ItemKind,
2977 direction: Direction,
2978 natively_names: Option<ItemKind>,
2979 ) -> Result<Vec<DependencyEdge>, SourceError> {
2980 if direction != Direction::DependsOn {
2981 return Ok(Vec::new());
2982 }
2983 let Some(item) = self
2984 .board()
2985 .await?
2986 .items
2987 .into_iter()
2988 .find(|item| item.id == *id)
2989 else {
2990 return Ok(Vec::new());
2991 };
2992 DependencyEdge::recorded(&item.slot, id, near_kind, &self.name, natively_names)
2993 .map_err(|message| SourceError::Malformed { message })
2994 }
2995
2996 fn configured_repository(&self) -> Result<&RepositoryTarget, SourceError> {
2997 self.repository
2998 .as_ref()
2999 .ok_or_else(|| SourceError::Refused {
3000 message: format!(
3001 "source {} has no repository configured, and a GitHub Projects board has no \
3002 repository of its own to create an issue in; set repository: owner/name on \
3003 this source",
3004 self.name
3005 ),
3006 })
3007 }
3008
3009 /// The repository one new issue is created in, under the rule [`RepositoryTarget`]
3010 /// states.
3011 ///
3012 /// The fallback is demanded first, whichever arm answers: a write without a configured
3013 /// repository is refused naming the field exactly as it was before the rule existed,
3014 /// so a source that could not write before cannot write now, rather than writing for
3015 /// the one item whose own field happens to decide it.
3016 ///
3017 /// Everything this refuses is refused before `createIssue`, so a refusal leaves no
3018 /// issue behind: an entry that is not a repository on [`RepositoryTarget::HOST`], an
3019 /// entry owned by someone other than the owner of the parent issue's repository —
3020 /// GitHub accepts a sub-issue from another repository of the same owner and from no
3021 /// other, so `addSubIssue` would refuse it after the issue existed — a parent the
3022 /// board does not hold, and a parent that is a draft, which GitHub gives no sub-issues,
3023 /// both of which `addSubIssue` would likewise refuse too late. Whether the entry exists
3024 /// and is visible to the token is checked where its node id is resolved, still before
3025 /// `createIssue`. The parent is read off `board`, which is completed from
3026 /// this process's own record, so a project created moments ago in this command answers
3027 /// though GitHub's board read has not caught up.
3028 fn creation_target(
3029 &self,
3030 board: &Board,
3031 incoming: &Incoming<'_>,
3032 ) -> Result<RepositoryTarget, SourceError> {
3033 let fallback = self.configured_repository()?;
3034 let what = |incoming: &Incoming<'_>| {
3035 format!(
3036 "{} {:?}",
3037 incoming.written.kind().describes(),
3038 incoming.title
3039 )
3040 };
3041 let parent = incoming
3042 .parent
3043 .map(|parent| {
3044 board
3045 .items
3046 .iter()
3047 .find(|item| item.id == *parent)
3048 .ok_or_else(|| SourceError::Refused {
3049 message: format!(
3050 "GitHub project issue {} was not found on the board of source {}, \
3051 so {} cannot be filed under it",
3052 parent.0,
3053 self.name,
3054 what(incoming)
3055 ),
3056 })
3057 })
3058 .transpose()?;
3059 let parents_repository = parent
3060 .map(|parent| {
3061 // A draft is on the board and so is found, but it has no repository to
3062 // place a task in and GitHub gives it no sub-issues, so `addSubIssue`
3063 // would refuse the task only once `createIssue` had made it.
3064 if parent.content_kind == ContentKind::DraftIssue {
3065 return Err(SourceError::Refused {
3066 message: format!(
3067 "GitHub project item {} on the board of source {} is a draft, \
3068 which cannot have sub-issues, so {} cannot be filed under it",
3069 parent.id.0,
3070 self.name,
3071 what(incoming)
3072 ),
3073 });
3074 }
3075 // An issue's repository is where a sub-issue is placed and whose owner it
3076 // is compared against, so a parent whose repository this source cannot
3077 // spell as `owner/name` — GitHub's login grammar is wider than this
3078 // source's floor — is one nothing can be filed under.
3079 parent
3080 .own_repository
3081 .as_ref()
3082 .and_then(|origin| RepositoryTarget::from_origin(origin).ok())
3083 .ok_or_else(|| SourceError::Malformed {
3084 message: format!(
3085 "GitHub project issue {} on the board of source {} is in {}, which \
3086 is not a {}/owner/name repository this source can place {} in",
3087 parent.id.0,
3088 self.name,
3089 parent
3090 .own_repository
3091 .as_ref()
3092 .map_or("no repository", Repository::as_str),
3093 RepositoryTarget::HOST,
3094 what(incoming)
3095 ),
3096 })
3097 })
3098 .transpose()?;
3099 match incoming.repositories {
3100 [named] => {
3101 let target =
3102 RepositoryTarget::from_origin(named).map_err(|_| SourceError::Refused {
3103 message: format!(
3104 "{} names repository {}, which is not a {}/owner/name repository \
3105 source {} can create an issue in; name one that is, or name none",
3106 what(incoming),
3107 named.as_str(),
3108 RepositoryTarget::HOST,
3109 self.name
3110 ),
3111 })?;
3112 if let Some(parents) = &parents_repository
3113 && parents.owner != target.owner
3114 {
3115 return Err(SourceError::Refused {
3116 message: format!(
3117 "{} names repository {}, owned by {}, but its project's issue is in \
3118 {}, owned by {}, and GitHub files a sub-issue only in a repository \
3119 of the same owner as its parent issue; name a repository of {}, or \
3120 name none",
3121 what(incoming),
3122 target.slug(),
3123 target.owner,
3124 parents.slug(),
3125 parents.owner,
3126 parents.owner
3127 ),
3128 });
3129 }
3130 Ok(target)
3131 }
3132 _ => Ok(parents_repository.unwrap_or_else(|| fallback.clone())),
3133 }
3134 }
3135
3136 /// The node id of the repository `incoming` is being created in, or the refusal naming
3137 /// the item and the repository the token cannot see.
3138 ///
3139 /// Resolved once per command per repository; see [`Self::repository_cache`].
3140 async fn repository_id(
3141 &self,
3142 repository: &RepositoryTarget,
3143 incoming: &Incoming<'_>,
3144 ) -> Result<String, SourceError> {
3145 if let Some(id) = self.repository_cache()?.get(repository).cloned() {
3146 return Ok(id);
3147 }
3148 let data = self
3149 .graphql(
3150 graphql::REPOSITORY,
3151 json!({"owner":repository.owner,"name":repository.name}),
3152 )
3153 .await?;
3154 let node = data
3155 .get("repository")
3156 .filter(|value| !value.is_null())
3157 .ok_or_else(|| SourceError::Refused {
3158 message: format!(
3159 "GitHub repository {} was not found or is not visible to the token, so {} \
3160 {:?} cannot be created in it",
3161 repository.slug(),
3162 incoming.written.kind().describes(),
3163 incoming.title
3164 ),
3165 })?;
3166 let id = required_str(node, "id")?.to_owned();
3167 self.repository_cache()?
3168 .insert(repository.clone(), id.clone());
3169 Ok(id)
3170 }
3171
3172 fn repository_cache(
3173 &self,
3174 ) -> Result<std::sync::MutexGuard<'_, BTreeMap<RepositoryTarget, String>>, SourceError> {
3175 self.repository_cache
3176 .lock()
3177 .map_err(|_| SourceError::Unavailable {
3178 message: "this source's record of the destination repository was left \
3179 inconsistent by an earlier failure; next: run the command again"
3180 .into(),
3181 })
3182 }
3183
3184 /// Create or update one board item, whichever kind it is.
3185 async fn write_item(
3186 &self,
3187 incoming: &Incoming<'_>,
3188 target: Option<&NativeId>,
3189 depends_on: &[DependencyEdge],
3190 ) -> Result<NativeId, SourceError> {
3191 // Refused before anything is read or written: a task or a project titled the way
3192 // this board spells a document would land as an issue this same source reads back
3193 // as a document, so the field this destination cannot carry is named rather than
3194 // written and silently reclassified.
3195 if let Written::Work(kind, _) = incoming.written
3196 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
3197 {
3198 return Err(SourceError::Refused {
3199 message: format!(
3200 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
3201 spells a document, so it would read back as one rather than as a {}; \
3202 retitle it, or copy it as a document",
3203 kind.marker(),
3204 self.name,
3205 kind.marker()
3206 ),
3207 });
3208 }
3209 let board = self.board().await?;
3210 let status_target = incoming
3211 .written
3212 .status()
3213 .map(|status| self.resolved_target(status.category))
3214 .transpose()?;
3215 let column = match (incoming.written.status(), status_target.as_ref()) {
3216 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
3217 _ => None,
3218 };
3219 let existing = target
3220 .map(|target| {
3221 board
3222 .items
3223 .iter()
3224 .find(|item| item.id == *target)
3225 .ok_or_else(|| SourceError::Refused {
3226 message: format!("GitHub destination item {} was not found", target.0),
3227 })
3228 })
3229 .transpose()?;
3230 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
3231 if content_kind == ContentKind::DraftIssue {
3232 if let (Some(StatusTarget::Closed(_)), Some(status)) =
3233 (status_target.as_ref(), incoming.written.status())
3234 {
3235 return Err(SourceError::Refused {
3236 message: format!(
3237 "status {} of source {} closes the item's issue, and GitHub draft items \
3238 have no open or closed state",
3239 category_name(status.category),
3240 self.name
3241 ),
3242 });
3243 }
3244 if incoming.parent.is_some() {
3245 return Err(SourceError::Refused {
3246 message: "GitHub draft items cannot be a project's sub-issue".into(),
3247 });
3248 }
3249 }
3250 match existing {
3251 Some(item) if content_kind == ContentKind::Issue => {
3252 if item.labels != incoming.labels {
3253 return Err(SourceError::Refused {
3254 message: "GitHub issue labels differ from the labels being written".into(),
3255 });
3256 }
3257 }
3258 _ => {
3259 if !incoming.labels.is_empty() {
3260 return Err(SourceError::Refused {
3261 message: "GitHub items created by this destination carry no labels".into(),
3262 });
3263 }
3264 }
3265 }
3266
3267 // An existing issue is never moved; a new one is created where the rule says. The
3268 // repository the issue really lives in is what the slot below is written against,
3269 // so a single entry that is where the issue is created travels as no key at all,
3270 // and the read side derives it back from the issue.
3271 let (own_repository, creation_target) = match existing {
3272 Some(item) => (item.own_repository.clone(), None),
3273 None => {
3274 let target = self.creation_target(&board, incoming)?;
3275 let origin = Repository::try_from(target.origin())
3276 .map_err(|message| SourceError::Config { message })?;
3277 (Some(origin), Some(target))
3278 }
3279 };
3280 let (native, fallback) = self
3281 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
3282 .await?;
3283 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
3284 let body = compose_body(incoming.content, &slot)?;
3285 // Read before anything is created, for the reason the field below is: a value
3286 // this destination cannot store has to refuse, and refusing after `createIssue`
3287 // would leave an issue behind that nothing asked for. The engine writes a
3288 // qualified id here; a caller handing this key anything else is told so rather
3289 // than having it silently stored as no origin at all.
3290 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
3291 let origin = match incoming.metadata.get(ORIGIN_KEY) {
3292 None => "",
3293 Some(Value::String(origin)) => origin.as_str(),
3294 Some(other) => {
3295 return Err(SourceError::Refused {
3296 message: format!(
3297 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
3298 is {other}"
3299 ),
3300 });
3301 }
3302 };
3303 // Resolved before anything is created: a board that cannot carry the copy origin
3304 // has to refuse the write, and refusing it after `createIssue` would leave an
3305 // issue behind that nothing asked for.
3306 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
3307 Some(field) => {
3308 if required_str(field, "__typename")? != "ProjectV2Field" {
3309 return Err(SourceError::Refused {
3310 message: format!(
3311 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
3312 ),
3313 });
3314 }
3315 Some(required_str(field, "id")?.to_owned())
3316 }
3317 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
3318 return Err(SourceError::Refused {
3319 message: format!(
3320 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
3321 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
3322 the board"
3323 ),
3324 });
3325 }
3326 None => None,
3327 };
3328
3329 let (content_id, item_id, url) = match existing {
3330 Some(item) => {
3331 self.update_existing(item, incoming, &body, status_target.as_ref())
3332 .await?;
3333 (item.id.clone(), item.item_id.clone(), item.url.clone())
3334 }
3335 None => {
3336 let target = creation_target
3337 .as_ref()
3338 .ok_or_else(|| SourceError::Malformed {
3339 message: "a new item was decided without a repository to create it in"
3340 .into(),
3341 })?;
3342 self.create_and_file_issue(&board, target, incoming, &body, status_target.as_ref())
3343 .await?
3344 }
3345 };
3346
3347 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
3348 // then each board field, the parent and the dependencies — and GitHub can fail at
3349 // any of them. Everything this source can refuse *before* the first of those is
3350 // already checked above, so what is left is GitHub itself failing part way. When it
3351 // does over an item this call created, the issue is taken back: a write that
3352 // refused must not leave an item behind that nobody asked for, and one that does
3353 // makes the retry create a second.
3354 let landed = self
3355 .finish_write(
3356 &board,
3357 incoming,
3358 &content_id,
3359 &item_id,
3360 content_kind,
3361 existing,
3362 origin_field.as_deref(),
3363 origin,
3364 column,
3365 &native,
3366 )
3367 .await;
3368 if let Err(error) = landed {
3369 if existing.is_none() {
3370 // Best effort, and the write's own failure is what the caller is told: a
3371 // refusal naming the tidy-up would hide why the write failed at all.
3372 let _ = self.delete_issue(&content_id).await;
3373 }
3374 return Err(error);
3375 }
3376
3377 // So the rest of this command reads what it just did rather than what the board
3378 // said before it. See `remember_written` for which half takes it.
3379 let remembered = Resolved {
3380 item_id,
3381 id: content_id.clone(),
3382 content_kind,
3383 kind: incoming.written.kind(),
3384 title: incoming.title.to_owned(),
3385 // The visible half of the body this write composed, split back off it the
3386 // way a read splits it — so what this record reports is what a read of the
3387 // same issue reports, rather than the person's text with the metadata slot
3388 // still on the end of it.
3389 body: metadata_body(body.clone())?.0,
3390 // A document has no status of its own; what it reads back as is whatever
3391 // the issue's own state says, which is what a re-read reports.
3392 status: incoming
3393 .written
3394 .status()
3395 .cloned()
3396 .unwrap_or_else(|| Status {
3397 category: StatusCategory::Unknown,
3398 name: "Open".to_owned(),
3399 }),
3400 labels: incoming.labels.to_vec(),
3401 parent: incoming.parent.cloned(),
3402 origin: (!origin.is_empty()).then(|| origin.to_owned()),
3403 // In the update path this is the item's own url, read off `existing` where the
3404 // tuple above was bound, so one expression serves both halves.
3405 url,
3406 created_at: existing.and_then(|item| item.created_at),
3407 updated_at: existing.and_then(|item| item.updated_at),
3408 own_repository,
3409 repositories: incoming.repositories.to_vec(),
3410 slot,
3411 };
3412 self.remember_written(remembered, existing.is_none())?;
3413 Ok(content_id)
3414 }
3415
3416 /// Everything a write does after the item exists: its board fields, its parent, and
3417 /// its dependencies.
3418 ///
3419 /// Split out of `write_item` so there is one place a failure past the point of no
3420 /// return is caught, rather than a tidy-up repeated at each `?` above.
3421 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
3422 // so there is one place a failure past the point of no return is caught, and its
3423 // arguments are exactly the values that tail already had in scope. Bundling them into a
3424 // struct would describe no concept — it would be "the arguments of this function" — and
3425 // would put the whole of `write_item`'s locals behind one more indirection.
3426 #[allow(clippy::too_many_arguments)]
3427 async fn finish_write(
3428 &self,
3429 board: &Board,
3430 incoming: &Incoming<'_>,
3431 content_id: &NativeId,
3432 item_id: &str,
3433 content_kind: ContentKind,
3434 existing: Option<&Resolved>,
3435 origin_field: Option<&str>,
3436 origin: &str,
3437 column: Option<(String, String)>,
3438 native: &[String],
3439 ) -> Result<(), SourceError> {
3440 if let Some(field_id) = origin_field {
3441 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
3442 .await?;
3443 }
3444
3445 if let Some((field_id, option_id)) = column {
3446 self.set_item_field(
3447 &board.id,
3448 item_id,
3449 &field_id,
3450 json!({"singleSelectOptionId":option_id}),
3451 )
3452 .await?;
3453 }
3454
3455 if content_kind == ContentKind::Issue {
3456 self.reparent(
3457 existing.and_then(|item| item.parent.clone()),
3458 content_id,
3459 incoming.parent,
3460 )
3461 .await?;
3462 // A document takes part in no dependency graph, so writing one neither reads
3463 // nor changes the issue's own `blockedBy` relationships. Reconciling them
3464 // against the empty list a document write carries would *delete* whatever
3465 // relationships a person had made on that issue, which is a write nobody
3466 // asked for.
3467 if incoming.written.kind() != BoardKind::Document {
3468 self.reconcile_blocked_by(content_id, native).await?;
3469 }
3470 }
3471 Ok(())
3472 }
3473
3474 /// Delete one issue, which takes its board item with it.
3475 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
3476 let data = self
3477 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3478 .await?;
3479 data.pointer("/deleteIssue/repository")
3480 .filter(|value| !value.is_null())
3481 .ok_or_else(|| SourceError::Malformed {
3482 message: "GitHub issue deletion returned no repository".into(),
3483 })?;
3484 self.forget(id)?;
3485 Ok(())
3486 }
3487
3488 /// Remove one item this copy created, so a copy that could not finish leaves the board
3489 /// as it found it.
3490 ///
3491 /// Deleting the issue takes its board item with it, so there is no second mutation to
3492 /// keep in step. An id the board does not hold is not an error: the item is already
3493 /// gone, which is the state this asks for.
3494 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
3495 let board = self.board().await?;
3496 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
3497 return Ok(());
3498 };
3499 if item.content_kind == ContentKind::DraftIssue {
3500 return Err(SourceError::Refused {
3501 message: format!(
3502 "GitHub item {} is a draft, and this source removes an item by deleting \
3503 its issue; next: remove it from the board by hand",
3504 id.0
3505 ),
3506 });
3507 }
3508 let data = self
3509 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3510 .await?;
3511 data.pointer("/deleteIssue/repository")
3512 .filter(|value| !value.is_null())
3513 .ok_or_else(|| SourceError::Malformed {
3514 message: "GitHub issue deletion returned no repository".into(),
3515 })?;
3516 self.forget(id)?;
3517 Ok(())
3518 }
3519
3520 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
3521 async fn partition_edges(
3522 &self,
3523 board: &Board,
3524 near_kind: BoardKind,
3525 near_content: ContentKind,
3526 depends_on: &[DependencyEdge],
3527 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
3528 let mut native = Vec::new();
3529 let mut fallback = Vec::new();
3530 for edge in depends_on {
3531 let same_source = edge
3532 .to
3533 .source()
3534 .is_none_or(|source| source == self.name.as_str());
3535 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
3536 // `DependencyEndpoint::source` both read it that way — and a native id may hold
3537 // colons of its own, so the far end is everything after that one separator.
3538 // Splitting at the last would truncate `work:urn:task:7` to `7`.
3539 let far_id = if edge.to.is_qualified() {
3540 edge.to
3541 .id()
3542 .split_once(':')
3543 .map_or(edge.to.id(), |(_, native)| native)
3544 } else {
3545 edge.to.id()
3546 };
3547 let far = if same_source {
3548 Some(
3549 board
3550 .items
3551 .iter()
3552 .find(|item| item.id.0 == far_id)
3553 .ok_or_else(|| SourceError::Refused {
3554 message: format!("GitHub dependency item {far_id} was not found"),
3555 })?,
3556 )
3557 } else {
3558 None
3559 };
3560 // The caller says which kind the far end is, and this board holds the far end
3561 // itself, so a disagreement is settled here rather than stored: recorded, the
3562 // wrong kind would read back as a cross-level edge that never existed; written
3563 // natively, it would name a relationship of a different level than the caller
3564 // asked for.
3565 //
3566 // A far end this board holds as a *document* fails the same comparison and is
3567 // refused by the same sentence: `ItemKind` has no document variant because
3568 // nothing may point at one, so no caller can name it correctly and the refusal
3569 // is the only honest answer.
3570 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
3571 return Err(SourceError::Refused {
3572 message: format!(
3573 "GitHub dependency item {far_id} is a {} of this board, and this item \
3574 names it as a {}; record the kind it is",
3575 disagreeing.kind.describes(),
3576 edge.to.kind.marker()
3577 ),
3578 });
3579 }
3580 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
3581 // however the far end is spelled — and one classified native here would be
3582 // written nowhere at all, because a draft's native reconciliation never runs.
3583 let native_here = near_content == ContentKind::Issue
3584 && far.is_some_and(|far| {
3585 far.content_kind == ContentKind::Issue
3586 && BoardKind::Work(edge.to.kind) == near_kind
3587 });
3588 if native_here {
3589 native.push(far_id.to_owned());
3590 } else {
3591 fallback.push(edge.clone());
3592 }
3593 }
3594 Ok((native, fallback))
3595 }
3596
3597 async fn update_existing(
3598 &self,
3599 item: &Resolved,
3600 incoming: &Incoming<'_>,
3601 body: &Option<String>,
3602 status_target: Option<&StatusTarget>,
3603 ) -> Result<(), SourceError> {
3604 let title = incoming.written_title();
3605 let (operation, input, pointer) = match item.content_kind {
3606 ContentKind::DraftIssue => (
3607 graphql::UPDATE_DRAFT,
3608 json!({"draftIssueId":item.id.0,"title":title,"body":body}),
3609 "/updateProjectV2DraftIssue/draftIssue",
3610 ),
3611 ContentKind::Issue => (
3612 graphql::UPDATE_ISSUE,
3613 json!({"id":item.id.0,"title":title,"body":body,
3614 "stateInput":state_input(status_target)}),
3615 "/updateIssue/issue",
3616 ),
3617 };
3618 let data = self.graphql(operation, json!({"input":input})).await?;
3619 let returned = data
3620 .pointer(pointer)
3621 .ok_or_else(|| SourceError::Malformed {
3622 message: "GitHub item update returned no item".into(),
3623 })?;
3624 if required_str(returned, "id")? != item.id.0 {
3625 return Err(SourceError::Malformed {
3626 message: "GitHub item update returned the wrong item".into(),
3627 });
3628 }
3629 Ok(())
3630 }
3631
3632 /// Creates one issue, files it on the board, and closes it when the status says so.
3633 ///
3634 /// Three calls rather than one: `createIssue` needs a repository and answers with an
3635 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
3636 /// closed status is a state of the issue rather than a field of the board item.
3637 /// Creates the issue, files it on the board, and reports what a read of it would say:
3638 /// its content id, its board item id, and the web address GitHub gave it.
3639 ///
3640 /// The address comes back here because this is the only place it is known before
3641 /// GitHub's own board read catches up — an item this run created answers the reads
3642 /// that follow it out of the record below, and one remembered without its address
3643 /// would report no location for the rest of the run.
3644 async fn create_and_file_issue(
3645 &self,
3646 board: &Board,
3647 repository: &RepositoryTarget,
3648 incoming: &Incoming<'_>,
3649 body: &Option<String>,
3650 status_target: Option<&StatusTarget>,
3651 ) -> Result<(NativeId, String, Option<String>), SourceError> {
3652 let repository_id = self.repository_id(repository, incoming).await?;
3653 let data = self
3654 .graphql(
3655 graphql::CREATE_ISSUE,
3656 json!({"input":{
3657 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
3658 }}),
3659 )
3660 .await?;
3661 let created = data
3662 .pointer("/createIssue/issue")
3663 .filter(|value| !value.is_null())
3664 .ok_or_else(|| SourceError::Malformed {
3665 message: "GitHub issue creation returned no issue".into(),
3666 })?;
3667 let content_id = NativeId(required_str(created, "id")?.to_owned());
3668 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
3669 // a response without it is not worth failing a landed write over — the item simply
3670 // reports no location until the board read catches up, which is what it did before.
3671 let url = optional_str(created, "url")?.map(str::to_owned);
3672 // The issue exists from here on, so a failure filing it on the board takes it
3673 // back: an issue in the repository that is on no board is an item nobody asked for
3674 // and nothing here would find again.
3675 let added = match self
3676 .graphql(
3677 graphql::ADD_TO_BOARD,
3678 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
3679 )
3680 .await
3681 {
3682 Ok(added) => added,
3683 Err(error) => {
3684 let _ = self.delete_issue(&content_id).await;
3685 return Err(error);
3686 }
3687 };
3688 let item = added
3689 .pointer("/addProjectV2ItemById/item")
3690 .filter(|value| !value.is_null())
3691 .ok_or_else(|| SourceError::Malformed {
3692 message: "GitHub board addition returned no project item".into(),
3693 })?;
3694 if let Some(StatusTarget::Closed(_)) = status_target {
3695 let closed = self
3696 .graphql(
3697 graphql::UPDATE_ISSUE,
3698 json!({"input":{"id":content_id.0,"stateInput":state_input(status_target)}}),
3699 )
3700 .await?;
3701 let returned =
3702 closed
3703 .pointer("/updateIssue/issue")
3704 .ok_or_else(|| SourceError::Malformed {
3705 message: "GitHub item update returned no item".into(),
3706 })?;
3707 if required_str(returned, "id")? != content_id.0 {
3708 return Err(SourceError::Malformed {
3709 message: "GitHub item update returned the wrong item".into(),
3710 });
3711 }
3712 }
3713 Ok((content_id, required_str(item, "id")?.to_owned(), url))
3714 }
3715
3716 /// Move one issue under the project it now belongs to, or out of the one it left.
3717 async fn reparent(
3718 &self,
3719 held: Option<NativeId>,
3720 child: &NativeId,
3721 wanted: Option<&NativeId>,
3722 ) -> Result<(), SourceError> {
3723 if held.as_ref() == wanted {
3724 return Ok(());
3725 }
3726 if let Some(held) = &held {
3727 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
3728 .await?;
3729 }
3730 if let Some(wanted) = wanted {
3731 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
3732 .await?;
3733 }
3734 Ok(())
3735 }
3736
3737 async fn sub_issue(
3738 &self,
3739 operation: &str,
3740 parent: &NativeId,
3741 child: &NativeId,
3742 root: &str,
3743 ) -> Result<(), SourceError> {
3744 let data = self
3745 .graphql(
3746 operation,
3747 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
3748 )
3749 .await?;
3750 let issue =
3751 data.pointer(&format!("/{root}/issue"))
3752 .ok_or_else(|| SourceError::Malformed {
3753 message: "GitHub sub-issue update returned no issue".into(),
3754 })?;
3755 let sub =
3756 data.pointer(&format!("/{root}/subIssue"))
3757 .ok_or_else(|| SourceError::Malformed {
3758 message: "GitHub sub-issue update returned no sub-issue".into(),
3759 })?;
3760 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
3761 return Err(SourceError::Malformed {
3762 message: "GitHub sub-issue update returned the wrong issues".into(),
3763 });
3764 }
3765 Ok(())
3766 }
3767
3768 async fn reconcile_blocked_by(
3769 &self,
3770 content_id: &NativeId,
3771 native: &[String],
3772 ) -> Result<(), SourceError> {
3773 let current = self.native_dependency_ids(content_id).await?;
3774 for (operation, far_id) in current
3775 .iter()
3776 .filter(|id| !native.contains(id))
3777 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
3778 .chain(
3779 native
3780 .iter()
3781 .filter(|id| !current.contains(id))
3782 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
3783 )
3784 {
3785 let data = self
3786 .graphql(
3787 operation,
3788 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
3789 )
3790 .await?;
3791 let root = if operation == graphql::ADD_BLOCKED_BY {
3792 "addBlockedBy"
3793 } else {
3794 "removeBlockedBy"
3795 };
3796 let issue =
3797 data.pointer(&format!("/{root}/issue"))
3798 .ok_or_else(|| SourceError::Malformed {
3799 message: "GitHub dependency update returned no issue".into(),
3800 })?;
3801 let blocker = data
3802 .pointer(&format!("/{root}/blockingIssue"))
3803 .ok_or_else(|| SourceError::Malformed {
3804 message: "GitHub dependency update returned no blocking issue".into(),
3805 })?;
3806 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
3807 {
3808 return Err(SourceError::Malformed {
3809 message: "GitHub dependency update returned the wrong issues".into(),
3810 });
3811 }
3812 }
3813 Ok(())
3814 }
3815}
3816
3817/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
3818enum Reached {
3819 /// An issue this board holds, resolved into everything this source reports about it.
3820 Held(Box<Resolved>),
3821 /// Nothing this board holds: no such node, or a node on some other board.
3822 Nothing,
3823 /// A board draft, which exists only inside the board's own item connection.
3824 Draft,
3825}
3826
3827/// What GitHub says when a string is not a node id it can resolve.
3828///
3829/// Matched because it is the ordinary answer to a project selector naming a project by its
3830/// *name*, and reporting that as a failure would make naming one impossible. It is read
3831/// off the refusal GitHub sent, never guessed from the shape of the string: this source
3832/// does not define the syntax of a GitHub node id and would be wrong about it.
3833const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
3834
3835/// Whether this refusal is GitHub saying the id names no node at all.
3836fn unresolvable_node(error: &SourceError) -> bool {
3837 matches!(error, SourceError::Refused { message }
3838 if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
3839}
3840
3841/// One project name, as a search qualifier which filters on it at the server.
3842///
3843/// Quoted so the whole title is one phrase rather than a bag of words, with the two
3844/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
3845/// the way it documents. A title matched here is still compared for equality afterwards:
3846/// the qualifier narrows what the server sends, and this source decides what it names.
3847fn title_qualifier(name: &str) -> String {
3848 let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
3849 format!("in:title \"{escaped}\"")
3850}
3851
3852/// The board, and every item on it this source reports.
3853#[derive(Clone)]
3854struct Board {
3855 id: String,
3856 fields: Value,
3857 items: Vec<Resolved>,
3858}
3859
3860impl Board {
3861 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
3862 complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
3863 let nodes = fields
3864 .get("nodes")
3865 .and_then(Value::as_array)
3866 .ok_or_else(|| SourceError::Malformed {
3867 message: "GitHub project fields.nodes is not an array".into(),
3868 })?;
3869 Ok(nodes
3870 .iter()
3871 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
3872 }
3873}
3874
3875/// One board item, resolved into everything this source reports about it.
3876#[derive(Clone)]
3877struct Resolved {
3878 item_id: String,
3879 id: NativeId,
3880 content_kind: ContentKind,
3881 kind: BoardKind,
3882 title: String,
3883 body: Option<String>,
3884 status: Status,
3885 labels: Vec<Label>,
3886 parent: Option<NativeId>,
3887 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
3888 origin: Option<String>,
3889 url: Option<String>,
3890 created_at: Option<DateTime<Utc>>,
3891 updated_at: Option<DateTime<Utc>>,
3892 own_repository: Option<Repository>,
3893 repositories: Vec<Repository>,
3894 slot: BTreeMap<String, Value>,
3895}
3896
3897impl Resolved {
3898 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
3899 /// in a field of its own, and none of the three keys that are only an encoding.
3900 fn metadata(&self) -> BTreeMap<String, Value> {
3901 let mut metadata = self.slot.clone();
3902 metadata.remove(Repository::METADATA_KEY);
3903 metadata.remove(DependencyEdge::RECORDED_KEY);
3904 metadata.remove(ItemKind::METADATA_KEY);
3905 if let Some(origin) = &self.origin {
3906 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
3907 }
3908 metadata
3909 }
3910
3911 /// Where this item is, as a link a reader can open.
3912 ///
3913 /// A board is a hosted place and every issue on it has a web address, so that address
3914 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
3915 /// of place it is, so a reader knows to open it rather than to read a file out. It
3916 /// does not replace or derive from `url`: the field goes on reporting exactly what it
3917 /// reported before, and this says what that address *is*.
3918 ///
3919 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
3920 /// rather than a third variant, which is the contract's "the source did not say". An
3921 /// issue this run created is not one of those: its address comes back from the
3922 /// creating mutation, so it is somewhere a reader can open from the moment it exists
3923 /// rather than from whenever the board read catches up.
3924 fn location(&self) -> Option<Location> {
3925 self.url.clone().map(Location::Url)
3926 }
3927
3928 fn task(&self) -> Task {
3929 Task {
3930 id: self.id.clone(),
3931 title: self.title.clone(),
3932 content: self.body.clone(),
3933 status: self.status.clone(),
3934 labels: self.labels.clone(),
3935 project: self.parent.clone(),
3936 url: self.url.clone(),
3937 location: self.location(),
3938 created_at: self.created_at,
3939 updated_at: self.updated_at,
3940 metadata: self.metadata(),
3941 repositories: self.repositories.clone(),
3942 }
3943 }
3944
3945 fn project(&self) -> Project {
3946 Project {
3947 id: self.id.clone(),
3948 title: self.title.clone(),
3949 content: self.body.clone(),
3950 status: self.status.clone(),
3951 labels: self.labels.clone(),
3952 url: self.url.clone(),
3953 location: self.location(),
3954 created_at: self.created_at,
3955 updated_at: self.updated_at,
3956 metadata: self.metadata(),
3957 repositories: self.repositories.clone(),
3958 }
3959 }
3960
3961 /// The same issue as a document: the project it is filed under, and no status and no
3962 /// dependencies, because a document is not work.
3963 fn document(&self) -> Document {
3964 Document {
3965 id: self.id.clone(),
3966 title: self.title.clone(),
3967 content: self.body.clone(),
3968 project: self.parent.clone(),
3969 labels: self.labels.clone(),
3970 url: self.url.clone(),
3971 location: self.location(),
3972 created_at: self.created_at,
3973 updated_at: self.updated_at,
3974 metadata: self.metadata(),
3975 repositories: self.repositories.clone(),
3976 }
3977 }
3978}
3979
3980/// What one write is, and the status that comes with being it.
3981///
3982/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
3983/// status and a task or a project always has one, so "a document carrying a status" and
3984/// "a task carrying none" are states a write cannot be in rather than states every use
3985/// site below has to defend against.
3986enum Written<'a> {
3987 /// A document, which is not work and so has no status at all.
3988 Document,
3989 /// A task or a project, and the status it is being written with.
3990 Work(ItemKind, &'a Status),
3991}
3992
3993impl Written<'_> {
3994 /// Which of the board's three kinds this write is.
3995 const fn kind(&self) -> BoardKind {
3996 match self {
3997 Self::Document => BoardKind::Document,
3998 Self::Work(kind, _) => BoardKind::Work(*kind),
3999 }
4000 }
4001
4002 /// The status this write carries. A document carries none, so a write of one says
4003 /// nothing about the issue's open or closed state and selects no board `Status`
4004 /// option.
4005 const fn status(&self) -> Option<&Status> {
4006 match self {
4007 Self::Document => None,
4008 Self::Work(_, status) => Some(status),
4009 }
4010 }
4011}
4012
4013/// The item being written, in the one shape all three write methods reach.
4014struct Incoming<'a> {
4015 written: Written<'a>,
4016 /// The title a person wrote. A document's goes onto the issue with
4017 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
4018 title: &'a str,
4019 content: Option<&'a str>,
4020 labels: &'a [Label],
4021 metadata: &'a BTreeMap<String, Value>,
4022 repositories: &'a [Repository],
4023 parent: Option<&'a NativeId>,
4024}
4025
4026impl Incoming<'_> {
4027 /// The title this write puts on the issue.
4028 fn written_title(&self) -> String {
4029 match self.written {
4030 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
4031 Written::Work(..) => self.title.to_owned(),
4032 }
4033 }
4034}
4035
4036#[derive(Clone, Copy, PartialEq, Eq)]
4037enum ContentKind {
4038 DraftIssue,
4039 Issue,
4040}
4041
4042/// What one board issue is: a document, or the work an [`ItemKind`] names.
4043///
4044/// A type of this source's own rather than an `ItemKind` with a third variant, because
4045/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
4046/// document — the contract keeps a document out of that enum deliberately. Holding the
4047/// board's three answers in one value is what makes every place that asks "which is this?"
4048/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
4049/// two thirds of the board.
4050#[derive(Clone, Copy, PartialEq, Eq)]
4051enum BoardKind {
4052 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
4053 Document,
4054 /// Every other issue, and every draft.
4055 Work(ItemKind),
4056}
4057
4058impl BoardKind {
4059 /// How a refusal names this kind to the person reading it.
4060 const fn describes(self) -> &'static str {
4061 match self {
4062 Self::Document => "document",
4063 Self::Work(kind) => kind.marker(),
4064 }
4065 }
4066}
4067
4068/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
4069///
4070/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
4071/// the shared cross-source journeys assert one answer to one question, so two sources
4072/// that disagree about what "carries the label bug" means fail them.
4073fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
4074 let holds = |name: &String| {
4075 labels
4076 .iter()
4077 .any(|label| label.name.eq_ignore_ascii_case(name))
4078 };
4079 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
4080 && filter.all_of.iter().all(holds)
4081 && !filter.none_of.iter().any(holds)
4082}
4083
4084/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
4085/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
4086fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
4087 statuses.is_empty() || statuses.contains(&category)
4088}
4089
4090/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
4091///
4092/// `content` is the item's own prose — the body with this source's trailing metadata
4093/// comment already taken off — so a search never matches an encoding the author of the
4094/// issue never wrote.
4095fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
4096 let terms = query.terms.to_lowercase();
4097 let in_title = title.to_lowercase().contains(&terms);
4098 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
4099 match query.fields {
4100 TextFields::Title => in_title,
4101 TextFields::Content => in_content,
4102 TextFields::TitleOrContent => in_title || in_content,
4103 }
4104}
4105
4106/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
4107///
4108/// The project predicate is passed separately because a read narrowed to one project has
4109/// already answered it by asking *that project* for its own items — and re-applying it
4110/// there would compare the caller's selector, which may be a project's **name**, against
4111/// the id of the project that name resolved to, and keep nothing. Every other read passes
4112/// `query.project` and applies it here, which is what keeps `projects` a predicate this
4113/// source really does apply.
4114fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
4115 labels_match(&task.labels, &query.labels)
4116 && status_matches(task.status.category, &query.statuses)
4117 && match project {
4118 ProjectFilter::Any => true,
4119 ProjectFilter::Orphans => task.project.is_none(),
4120 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
4121 }
4122 && query
4123 .text
4124 .as_ref()
4125 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
4126}
4127
4128fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
4129 labels_match(&project.labels, &query.labels)
4130 && status_matches(project.status.category, &query.statuses)
4131 && query
4132 .text
4133 .as_ref()
4134 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
4135}
4136
4137/// The same three predicates a task query carries, minus the status filter.
4138///
4139/// A document is not work, so it has no status for one to compare against and the query
4140/// type carries none. The project predicate is the same one — a design issue filed under a
4141/// project issue is in that project, and one filed under nothing is in none — so it is
4142/// spelled the same way here rather than answered differently.
4143fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
4144 labels_match(&document.labels, &query.labels)
4145 && match project {
4146 ProjectFilter::Any => true,
4147 ProjectFilter::Orphans => document.project.is_none(),
4148 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
4149 }
4150 && query
4151 .text
4152 .as_ref()
4153 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
4154}
4155
4156#[async_trait::async_trait]
4157impl TaskSource for GitHubProjectsSource {
4158 fn kind(&self) -> &'static str {
4159 KIND
4160 }
4161 fn capabilities(&self) -> Capabilities {
4162 Capabilities {
4163 projects: Support::Native,
4164 documents: Support::Native,
4165 orphan_tasks: Support::Native,
4166 filter_by_label: Support::Native,
4167 filter_by_status: Support::Native,
4168 search_title: Support::Native,
4169 search_content: Support::Native,
4170 task_dependencies: DependencySupport::BothDirections,
4171 project_dependencies: DependencySupport::BothDirections,
4172 max_page_size: MAX_PAGE_SIZE,
4173 }
4174 }
4175 async fn health(&self) -> Result<Health, SourceError> {
4176 let board = self.board_page(None, 1).await?;
4177 Ok(Health {
4178 reachable: true,
4179 detail: Some(format!(
4180 "reading GitHub project {}/{} ({})",
4181 self.owner,
4182 self.project_number,
4183 required_str(&board, "title")?
4184 )),
4185 })
4186 }
4187 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
4188 Ok(self
4189 .item_by_id(id)
4190 .await?
4191 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4192 .map(|item| item.task()))
4193 }
4194 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
4195 Ok(self
4196 .item_by_id(id)
4197 .await?
4198 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4199 .map(|item| item.project()))
4200 }
4201 async fn query_tasks(
4202 &self,
4203 query: &TaskQuery,
4204 page: &PageRequest,
4205 ) -> Result<Page<Task>, SourceError> {
4206 validate_page(page)?;
4207 // A read narrowed to one project asks that project for its own tasks, so nothing
4208 // about it costs what the rest of the board holds. Every other task read is a
4209 // question about the whole board and is answered by reading it.
4210 let (held, membership) = match &query.project {
4211 ProjectFilter::Is(project) => (
4212 self.project_children(project).await?,
4213 // Answered by where these items came from; see `task_matches`.
4214 &ProjectFilter::Any,
4215 ),
4216 ProjectFilter::Any | ProjectFilter::Orphans => {
4217 (self.board().await?.items, &query.project)
4218 }
4219 };
4220 // Filtered before paged: a page of a filtered result is a page of the survivors,
4221 // never the survivors of a page.
4222 let tasks = held
4223 .iter()
4224 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4225 .map(Resolved::task)
4226 .filter(|task| task_matches(task, query, membership))
4227 .collect();
4228 Ok(offset_page(
4229 tasks,
4230 numeric_cursor(page.cursor.as_ref())?,
4231 page.limit.min(MAX_PAGE_SIZE) as usize,
4232 ))
4233 }
4234 async fn query_projects(
4235 &self,
4236 query: &ProjectQuery,
4237 page: &PageRequest,
4238 ) -> Result<Page<Project>, SourceError> {
4239 validate_page(page)?;
4240 // The projects a board holds are found by an issue search scoped to that board,
4241 // never by walking the board's own item connection: what tells a project from a
4242 // task is the `parent` each issue carries, which costs nothing to read.
4243 let projects = self
4244 .board_issues()
4245 .await?
4246 .iter()
4247 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4248 .map(Resolved::project)
4249 .filter(|project| project_matches(project, query))
4250 .collect();
4251 Ok(offset_page(
4252 projects,
4253 numeric_cursor(page.cursor.as_ref())?,
4254 page.limit.min(MAX_PAGE_SIZE) as usize,
4255 ))
4256 }
4257 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
4258 Ok(self
4259 .item_by_id(id)
4260 .await?
4261 .filter(|item| item.kind == BoardKind::Document)
4262 .map(|item| item.document()))
4263 }
4264 async fn query_documents(
4265 &self,
4266 query: &DocumentQuery,
4267 page: &PageRequest,
4268 ) -> Result<Page<Document>, SourceError> {
4269 validate_page(page)?;
4270 // Narrowed to one project, this is the same sub-issue read a task list scoped to
4271 // that project makes — a document filed under a project is a sub-issue of it too,
4272 // and which of them come back is the kind this caller asked for.
4273 let (held, membership) = match &query.project {
4274 ProjectFilter::Is(project) => (
4275 self.project_children(project).await?,
4276 // Answered by where these items came from; see `task_matches`.
4277 &ProjectFilter::Any,
4278 ),
4279 ProjectFilter::Any | ProjectFilter::Orphans => {
4280 (self.board().await?.items, &query.project)
4281 }
4282 };
4283 // Filtered before paged, exactly as a task read is: a page of a filtered result is
4284 // a page of the survivors, never the survivors of a page.
4285 let documents = held
4286 .iter()
4287 .filter(|item| item.kind == BoardKind::Document)
4288 .map(Resolved::document)
4289 .filter(|document| document_matches(document, query, membership))
4290 .collect();
4291 Ok(offset_page(
4292 documents,
4293 numeric_cursor(page.cursor.as_ref())?,
4294 page.limit.min(MAX_PAGE_SIZE) as usize,
4295 ))
4296 }
4297 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
4298 validate_page(page)?;
4299 let offset = numeric_cursor(page.cursor.as_ref())?;
4300 let mut labels = self
4301 .board()
4302 .await?
4303 .items
4304 .into_iter()
4305 .flat_map(|item| item.labels)
4306 .fold(Vec::new(), |mut all, label| {
4307 if !all.iter().any(|x: &Label| x.id == label.id) {
4308 all.push(label);
4309 }
4310 all
4311 });
4312 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
4313 Ok(offset_page(
4314 labels,
4315 offset,
4316 page.limit.min(MAX_PAGE_SIZE) as usize,
4317 ))
4318 }
4319 async fn task_dependencies(
4320 &self,
4321 id: &NativeId,
4322 direction: Direction,
4323 page: &PageRequest,
4324 ) -> Result<Page<DependencyEdge>, SourceError> {
4325 self.dependencies(id, ItemKind::Task, direction, page).await
4326 }
4327 async fn project_dependencies(
4328 &self,
4329 id: &NativeId,
4330 direction: Direction,
4331 page: &PageRequest,
4332 ) -> Result<Page<DependencyEdge>, SourceError> {
4333 self.dependencies(id, ItemKind::Project, direction, page)
4334 .await
4335 }
4336
4337 fn writes(&self) -> WriteSupport {
4338 WriteSupport::Supported
4339 }
4340
4341 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
4342 self.write_item(
4343 &Incoming {
4344 written: Written::Work(ItemKind::Task, &write.item.status),
4345 title: &write.item.title,
4346 content: write.item.content.as_deref(),
4347 labels: &write.item.labels,
4348 metadata: &write.item.metadata,
4349 repositories: &write.item.repositories,
4350 parent: write.item.project.as_ref(),
4351 },
4352 write.target.as_ref(),
4353 &write.depends_on,
4354 )
4355 .await
4356 }
4357
4358 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
4359 self.write_item(
4360 &Incoming {
4361 written: Written::Work(ItemKind::Project, &write.item.status),
4362 title: &write.item.title,
4363 content: write.item.content.as_deref(),
4364 labels: &write.item.labels,
4365 metadata: &write.item.metadata,
4366 repositories: &write.item.repositories,
4367 parent: None,
4368 },
4369 write.target.as_ref(),
4370 &write.depends_on,
4371 )
4372 .await
4373 }
4374
4375 /// Create or update one document, which is one issue titled the way this board spells
4376 /// a document.
4377 ///
4378 /// Everything else is exactly a task write: caller metadata goes to the same canonical
4379 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
4380 /// or a field this board cannot carry is refused by name rather than dropped, a target
4381 /// naming an issue this board does not hold is refused rather than created, and an
4382 /// issue this call created is taken back when the rest of the write fails.
4383 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
4384 // A document takes part in no dependency graph, so there is no far end to write
4385 // natively and none to record: a caller naming one is told so rather than having it
4386 // stored under the reserved key, where a later read would report an edge the
4387 // contract says cannot exist.
4388 if !write.depends_on.is_empty() {
4389 return Err(SourceError::Refused {
4390 message: format!(
4391 "this write names {} dependencies for a document, and a document takes \
4392 part in no dependency graph; next: put the dependency on the task or \
4393 project the document is about",
4394 write.depends_on.len()
4395 ),
4396 });
4397 }
4398 self.write_item(
4399 &Incoming {
4400 written: Written::Document,
4401 title: &write.item.title,
4402 content: write.item.content.as_deref(),
4403 labels: &write.item.labels,
4404 metadata: &write.item.metadata,
4405 repositories: &write.item.repositories,
4406 parent: write.item.project.as_ref(),
4407 },
4408 write.target.as_ref(),
4409 &[],
4410 )
4411 .await
4412 }
4413
4414 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
4415 self.delete_item(id).await
4416 }
4417
4418 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
4419 self.delete_item(id).await
4420 }
4421
4422 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
4423 self.delete_item(id).await
4424 }
4425}
4426
4427/// Where the recorded tail of a dependency walk resumes; see
4428/// [`GitHubProjectsSource::recorded_edges`].
4429const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
4430
4431/// The board text field this source keeps a copy's origin in.
4432///
4433/// Named after the key it holds, and held to that name by the guard below rather than by
4434/// a reader noticing.
4435const ORIGIN_FIELD: &str = "onetaskgraph.origin";
4436
4437/// The metadata key that field holds.
4438///
4439/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
4440/// constructs or interprets the qualified id it carries. This source names it only to
4441/// route it — a short, typed value belongs in a typed field rather than in the body slot
4442/// a caller's own prose shares.
4443///
4444/// Restated rather than imported, because no plugin crate may depend on the engine. What
4445/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
4446/// target in `check`: it reads the engine's own literal and fails naming the file and the
4447/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
4448/// that creates a second item every run instead of finding the one it wrote — and that is
4449/// too late to learn it.
4450const ORIGIN_KEY: &str = "onetaskgraph.origin";
4451
4452/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
4453///
4454/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
4455/// is derived from the far end, never written down on the near item — so only a forward
4456/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
4457/// it did not come from, and it is told so rather than answered with an empty page that
4458/// reads as a walk which ended.
4459fn recorded_offset(
4460 cursor: Option<&str>,
4461 direction: Direction,
4462) -> Result<Option<usize>, SourceError> {
4463 cursor
4464 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
4465 .map(|offset| {
4466 if direction != Direction::DependsOn {
4467 return Err(SourceError::Config {
4468 message: format!(
4469 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
4470 reverse dependency read never issues; resume it in the direction \
4471 that reported it"
4472 ),
4473 });
4474 }
4475 offset.parse().map_err(|_| SourceError::Config {
4476 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
4477 })
4478 })
4479 .transpose()
4480}
4481
4482fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
4483 let mut page = offset_page(edges, offset, limit.max(1));
4484 page.next = page
4485 .next
4486 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
4487 page
4488}
4489
4490/// The kind of one issue reached through a dependency connection.
4491///
4492/// The same questions the board scan asks, over the fields the dependency document
4493/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
4494/// then anything with sub-issues or the marker is a project.
4495///
4496/// # Errors
4497///
4498/// A far end this board holds as a document is refused rather than reported. The two
4499/// answers that are not refusals would both be wrong: reporting it as a task names an id
4500/// no task read of this source can find, and reporting it as a project names one no
4501/// project read can. There is no third value to return — `ItemKind` has no document
4502/// variant, because nothing may point at a document — so the relationship itself is what
4503/// the person is told about.
4504fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
4505 let id = required_str(value, "id")?;
4506 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
4507 return Err(SourceError::Refused {
4508 message: format!(
4509 "GitHub issue {id} is a document of this board — its title begins \
4510 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
4511 on by one; next: remove that issue's blocking relationship on this board"
4512 ),
4513 });
4514 }
4515 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
4516 if parent.is_some() {
4517 return Ok(ItemKind::Task);
4518 }
4519 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
4520 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
4521 message: format!("GitHub issue {id}: {message}"),
4522 })?;
4523 let sub_issues = sub_issue_total(value)?;
4524 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
4525 ItemKind::Project
4526 } else {
4527 ItemKind::Task
4528 })
4529}
4530
4531/// The `IssueStateUpdateInput` one status target asks for.
4532///
4533/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
4534/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
4535/// a currently-closed issue: without that the item would read back `Unknown` and a copy
4536/// would report a change forever. A document has no status at all, and asks for neither.
4537fn state_input(target: Option<&StatusTarget>) -> Value {
4538 match target {
4539 Some(StatusTarget::Closed(reason)) => {
4540 json!({"value":"CLOSED","stateReason":reason.reason()})
4541 }
4542 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
4543 // A document has no status, so a write of one says nothing about the issue's open
4544 // or closed state rather than forcing it open: `stateInput` is what carries that
4545 // instruction, and an explicit null asks for no change to it.
4546 None => Value::Null,
4547 }
4548}
4549
4550/// The metadata one write stores in the item's body slot.
4551///
4552/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
4553/// rather than carried: the kind marker so an empty project stays readable, the
4554/// repository list only when it is not exactly the issue's own repository, and the far
4555/// ends no relationship here can name.
4556fn slot_metadata(
4557 incoming: &Incoming<'_>,
4558 own_repository: Option<&Repository>,
4559 fallback: &[DependencyEdge],
4560) -> BTreeMap<String, Value> {
4561 let mut metadata = incoming.metadata.clone();
4562 metadata.remove(ORIGIN_KEY);
4563 match incoming.written.kind() {
4564 BoardKind::Work(kind) => metadata.insert(
4565 ItemKind::METADATA_KEY.to_owned(),
4566 Value::String(kind.marker().to_owned()),
4567 ),
4568 // A document is told by its title, so it carries no kind marker: that key names
4569 // what a dependency endpoint points at, and nothing may point at a document.
4570 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
4571 };
4572 let derivable = own_repository
4573 .map(|own| incoming.repositories == [own.clone()])
4574 .unwrap_or(incoming.repositories.is_empty());
4575 if derivable {
4576 metadata.remove(Repository::METADATA_KEY);
4577 } else {
4578 metadata.insert(
4579 Repository::METADATA_KEY.to_owned(),
4580 Value::Array(
4581 incoming
4582 .repositories
4583 .iter()
4584 .map(|repository| Value::String(repository.as_str().to_owned()))
4585 .collect(),
4586 ),
4587 );
4588 }
4589 if fallback.is_empty() {
4590 metadata.remove(DependencyEdge::RECORDED_KEY);
4591 } else {
4592 metadata.insert(
4593 DependencyEdge::RECORDED_KEY.to_owned(),
4594 Value::Array(
4595 fallback
4596 .iter()
4597 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
4598 .collect(),
4599 ),
4600 );
4601 }
4602 metadata
4603}
4604
4605/// Every label one item carries, from its content's own connection and nowhere else.
4606///
4607/// There is no second place to read one from: no document this source sends selects the
4608/// board's built-in `Labels` field, because GitHub derives it from the content and a draft
4609/// cannot carry one at all. The module documentation records the three schema facts that
4610/// settle it.
4611fn labels(content: &Value) -> Result<Vec<Label>, SourceError> {
4612 optional_nodes(content.get("labels"), "content labels")?
4613 .into_iter()
4614 .flatten()
4615 .map(|v| {
4616 Ok(Label {
4617 id: NativeId(required_str(v, "id")?.to_owned()),
4618 name: required_str(v, "name")?.to_owned(),
4619 color: optional_str(v, "color")?.map(str::to_owned),
4620 })
4621 })
4622 .collect()
4623}
4624
4625fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
4626 let Some(node) = field_values
4627 .iter()
4628 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
4629 else {
4630 return Ok(None);
4631 };
4632 Ok(optional_str(node, "text")?.map(str::to_owned))
4633}
4634
4635fn valid_github_owner(owner: &str) -> bool {
4636 !owner.is_empty()
4637 && owner.len() <= 39
4638 && !owner.starts_with('-')
4639 && !owner.ends_with('-')
4640 && !owner.contains("--")
4641 && owner
4642 .bytes()
4643 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
4644}
4645
4646/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
4647/// neither of the two names a path segment already means.
4648fn valid_github_repository_name(name: &str) -> bool {
4649 !name.is_empty()
4650 && name.len() <= 100
4651 && name != "."
4652 && name != ".."
4653 && name
4654 .bytes()
4655 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
4656}
4657
4658fn valid_environment_name(name: &str) -> bool {
4659 let mut bytes = name.bytes();
4660 bytes
4661 .next()
4662 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
4663 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
4664}
4665
4666/// How many sub-issues one issue has.
4667///
4668/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
4669/// absent or non-integer one is a response this source cannot read — and reading it as
4670/// zero would classify a project as a task, which is exactly the mistake the marker
4671/// exists to keep from happening quietly.
4672fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
4673 let summary = issue
4674 .get("subIssuesSummary")
4675 .ok_or_else(|| SourceError::Malformed {
4676 message: "GitHub issue is missing subIssuesSummary".into(),
4677 })?;
4678 summary
4679 .get("total")
4680 .and_then(Value::as_u64)
4681 .ok_or_else(|| SourceError::Malformed {
4682 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
4683 })
4684}
4685
4686fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
4687 value
4688 .get(field)
4689 .and_then(Value::as_str)
4690 .ok_or_else(|| SourceError::Malformed {
4691 message: format!("GitHub response is missing string field {field}"),
4692 })
4693}
4694
4695/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
4696/// needs one — Linear spells them too, in its own description field.
4697///
4698/// Restated rather than shared, because a plugin crate depends on the contract crate and
4699/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
4700/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
4701/// source round-trips its own writes perfectly well under its own spelling.
4702const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
4703const METADATA_CLOSE: &str = "\n-->";
4704
4705/// The visible body and the metadata slot at the end of it.
4706///
4707/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
4708/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
4709/// own content and is left alone.
4710fn metadata_body(
4711 body: Option<String>,
4712) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
4713 let Some(body) = body else {
4714 return Ok((None, BTreeMap::new()));
4715 };
4716 let Some(start) = body.rfind(METADATA_OPEN) else {
4717 return Ok((Some(body), BTreeMap::new()));
4718 };
4719 let encoded_start = start + METADATA_OPEN.len();
4720 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
4721 return Err(SourceError::Malformed {
4722 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
4723 });
4724 };
4725 let encoded_end = encoded_start + relative_end;
4726 if !body[encoded_end + METADATA_CLOSE.len()..].trim().is_empty() {
4727 return Ok((Some(body), BTreeMap::new()));
4728 }
4729 let metadata = serde_json::from_str(&body[encoded_start..encoded_end]).map_err(|error| {
4730 SourceError::Malformed {
4731 message: format!(
4732 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
4733 ),
4734 }
4735 })?;
4736 let visible = body[..start].trim_end();
4737 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
4738}
4739
4740fn compose_body(
4741 content: Option<&str>,
4742 metadata: &BTreeMap<String, Value>,
4743) -> Result<Option<String>, SourceError> {
4744 let visible = content.unwrap_or_default();
4745 if metadata.is_empty() {
4746 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
4747 }
4748 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
4749 message: error.to_string(),
4750 })?;
4751 Ok(Some(if visible.is_empty() {
4752 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4753 } else {
4754 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4755 }))
4756}
4757
4758fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
4759 value
4760 .get(field)
4761 .and_then(Value::as_bool)
4762 .ok_or_else(|| SourceError::Malformed {
4763 message: format!("GitHub response is missing boolean field {field}"),
4764 })
4765}
4766fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
4767 match value.get(field) {
4768 None | Some(Value::Null) => Ok(None),
4769 Some(value) => value
4770 .as_str()
4771 .map(Some)
4772 .ok_or_else(|| SourceError::Malformed {
4773 message: format!("GitHub response field {field} is not a string or null"),
4774 }),
4775 }
4776}
4777fn optional_nodes<'a>(
4778 connection: Option<&'a Value>,
4779 name: &str,
4780) -> Result<Option<&'a Vec<Value>>, SourceError> {
4781 match connection {
4782 None | Some(Value::Null) => Ok(None),
4783 Some(value) => value
4784 .get("nodes")
4785 .and_then(Value::as_array)
4786 .map(Some)
4787 .ok_or_else(|| SourceError::Malformed {
4788 message: format!("GitHub {name}.nodes is not an array"),
4789 }),
4790 }
4791}
4792fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
4793 let page_info = connection
4794 .get("pageInfo")
4795 .ok_or_else(|| SourceError::Malformed {
4796 message: format!("GitHub {name} has no pageInfo"),
4797 })?;
4798 if required_bool(page_info, "hasNextPage")? {
4799 return Err(SourceError::Malformed {
4800 message: format!(
4801 "GitHub {name} exceeds the supported nested connection size of {size}"
4802 ),
4803 });
4804 }
4805 Ok(())
4806}
4807fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
4808 optional_str(value, field)?
4809 .map(|timestamp| {
4810 timestamp.parse().map_err(|error| SourceError::Malformed {
4811 message: format!("GitHub response field {field} is not a timestamp: {error}"),
4812 })
4813 })
4814 .transpose()
4815}
4816fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
4817 if page.limit == 0 {
4818 Err(SourceError::Config {
4819 message: "page limit must be at least 1".into(),
4820 })
4821 } else {
4822 Ok(())
4823 }
4824}
4825fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
4826 let page = connection
4827 .get("pageInfo")
4828 .filter(|value| value.is_object())
4829 .ok_or_else(|| SourceError::Malformed {
4830 message: "GitHub connection is missing pageInfo".into(),
4831 })?;
4832 if required_bool(page, "hasNextPage")? {
4833 let cursor = required_str(page, "endCursor")?;
4834 validate_cursor_progress(None, cursor)?;
4835 Ok(Some(Cursor(cursor.into())))
4836 } else {
4837 Ok(None)
4838 }
4839}
4840fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
4841 if next.is_empty() || previous == Some(next) {
4842 Err(SourceError::Malformed {
4843 message: "GitHub pagination cursor is empty or did not advance".into(),
4844 })
4845 } else {
4846 Ok(())
4847 }
4848}
4849fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
4850 cursor.map_or(Ok(0), |c| {
4851 c.0.parse().map_err(|_| SourceError::Config {
4852 message: "page cursor is invalid".into(),
4853 })
4854 })
4855}
4856fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
4857 if offset > items.len() {
4858 return Page::last(vec![]);
4859 }
4860 let tail = items.split_off(offset);
4861 let mut selected = tail;
4862 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
4863 selected.truncate(limit);
4864 Page {
4865 items: selected,
4866 next,
4867 }
4868}