onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so a write without [`GitHubProjectsConfig::repository`] is refused
11//! naming the field — but that repository is the *fallback*, not the home of every item.
12//!
13//! <!-- llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] The rule's one
14//! executable source is `GitHubProjectsSource::creation_target`; this is where a reader of
15//! the module meets it, and `tests/plugin.rs` drives every arm below against the loopback
16//! board and asserts on `createIssue`'s own `repositoryId`, so the prose cannot outlive a
17//! change to the rule. -->
18//! **Which repository an issue is created in is decided by the item's own `repositories`
19//! field, under one rule.** Exactly one entry names the repository the issue is created in:
20//! a task issue is where a person finds the work from the repository it changes, and one
21//! filed in a board's nominated repository is invisible from every other. Zero entries, or
22//! two or more, name none, so a task's or a document's issue is created in the repository
23//! its parent project's issue lives in — read from the board, or from this process's own
24//! record of a project it created earlier in the same command — and a project's issue, or
25//! a task or document written with no parent, is created in the configured `repository:`.
26//! What that rule refuses, it refuses before `createIssue`, so no issue is half-created. An
27//! existing issue is never moved: the update path leaves the issue where it is and records
28//! the list in the metadata slot when it differs, so the read side's derivation and the
29//! creation rule agree by construction.
30//! <!-- llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate] -->
31//!
32//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
33//! board has no document type and nothing but issues to hold one in, so the title is the
34//! discriminator and it is the whole of it. The title this source *reports* is the one a
35//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
36//! the body so `content` is what the person wrote — and writing a document puts the prefix
37//! back, so a round trip returns the title that went in.
38//!
39//! **Telling a document from a project from a task.** The design prefix is read **first**:
40//! a document is never a project and never a task, whatever sub-issues it has or does not
41//! have. Only then does the rest apply — a board issue is a project when *either* it has
42//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
43//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
44//! necessary: it is what makes an *empty* project — the state a project copy passes
45//! through between creating the project and filing its first task — readable as a
46//! project, while the sub-issue arm lets a person author a project on the board by hand
47//! with no knowledge of this product's metadata at all. Reading the prefix later than the
48//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
49//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
50//! task nor a document and are ignored.
51//!
52//! **Where an entity is, is a link.** Every project, task and document this source reports
53//! carries a [`Location::Url`] naming the issue's own web address — the same address the
54//! `url` field already reports, in the shape that says a reader can open it. That is the
55//! contrast the location contract exists for: a reader holding an entity from this source
56//! is handed something to link to and one holding an entity from a folder of Markdown is
57//! handed a path, and neither has to know which plugin answered. It does not replace or
58//! derive from `url`; that field goes on reporting what it always reported.
59//!
60//! **Where metadata lives.** Short typed things go to typed fields and native relations:
61//! status to the board's `Status` single-select and the issue's own state, the copy
62//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
63//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
64//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
65//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
66//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
67//! why neither can hold a caller's own prose.
68//!
69//! **Status.** `status_mapping` is per-instance configuration from a status category to
70//! `null`, a board `Status` option name, or a closed state of `completed` or
71//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
72//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
73//! and a mistake destroys every item's status. A status this board cannot represent is a
74//! refusal naming the status and the instance instead.
75//!
76//! `unknown` is disabled by default because this source cannot preserve an open-ended
77//! status word: it writes an existing board option or the issue's closed state and never
78//! creates an option. An operator may map `unknown` to one existing option, in which case
79//! every unknown word lands on that option and reads back as `unknown` under the option's
80//! name. Mapping it to a closed state is refused because GitHub reads that state back as
81//! `done` or `cancelled`, changing the category on every copy. This differs from
82//! `local-md`, which writes and reads the original word itself.
83//!
84//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
85//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
86//! finished tasks were only moved to a "Done" column would read 0% complete forever.
87//!
88//! # What this source declares, field by field
89//!
90//! One verdict per field of [`Capabilities`], and what `Native` means when this source
91//! says it. *Proven* means a shared journey drives it against the real
92//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
93//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
94//! [`capabilities`](TaskSource::capabilities) from parting.
95//!
96//! | Field | Verdict |
97//! | --- | --- |
98//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
99//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
100//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
101//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
102//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
103//! | `search_title` | **Supported and proven,** over `Issue.title`. |
104//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
105//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
106//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
107//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
108//!
109//! Nothing here is unsupported. `documents` is not a predicate — it says this source has
110//! documents, which it does — and the three facts behind the uniform `Native` on the
111//! predicates beside it are recorded below rather than re-derived, because a reader who
112//! takes `Native` to mean *the remote service filters* will read that uniformity as a
113//! lie.
114//!
115//! First, the plugin contract defines `Support::Native` as *the source applies this
116//! predicate itself*, and says nothing about where it applies it. What the declaration
117//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
118//! — so that the engine may push it down and apply nothing of its own.
119//!
120//! Second, this source can keep that promise for every predicate at no additional API
121//! cost, because whichever of the three reads below answers a query has already read every
122//! item that query could keep before it filters anything. Filtering those items is
123//! in-process work over data already in hand.
124//!
125//! Third, no predicate but `projects` could be pushed into the API even if that were
126//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
127//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
128//! label set, a status column or a substring of a body, and its title qualifier matches
129//! tokens where this source — and the local Markdown source beside it — match substrings,
130//! so pushing a search down would silently *narrow* the answer. What a project filter has
131//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
132//! the issue for them is both cheaper and exact. So there is one predicate this source
133//! applies by asking a narrower question, six it applies in process, and none it is unable
134//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
135//! source has already done, and declaring `projects` native while ignoring the filter
136//! (which this source once did) silently returns another project's tasks, because the
137//! engine trusts the declaration and applies nothing locally.
138//!
139//! # The three ways this source reaches an item, and what each costs
140//!
141//! A board read is charged for what its *nested* connections could return rather than for
142//! what was asked, so one whole-board read costs the same whether the question was about
143//! one project or about all of them. That is why a question about one project is never
144//! answered by reading the board:
145//!
146//! | The question | What is sent | What it costs |
147//! | --- | --- | --- |
148//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
149//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
150//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
151//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
152//! | which board item one issue is, past the page that came with it | [`graphql::ISSUE_BOARD_ITEMS`] — that issue's own `projectItems` | one issue's memberships |
153//!
154//! The board half of an issue — its board item's id, its `Status` option and this
155//! source's origin text field — rides along on `Issue.projectItems` in the first three, so
156//! an item reached any of those ways resolves through the same
157//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
158//! same status, the same labels and the same qualified id. That connection comes back a
159//! *page* at a time, at `BOARD_ITEMS_PAGE_SIZE`, so the entry for this board is looked for
160//! on the page in hand and — only if that page reports more of the connection — in the
161//! last row's read of that one issue's memberships, resumed from the page's own cursor and
162//! walked to exhaustion. An issue with no entry for *this* board is not this source's to
163//! report, which is what keeps an id naming another repository's issue from being answered
164//! as an item of this board; and because the page is where the search starts rather than
165//! where it ends, that answer is one about a connection read to exhaustion and never about
166//! an unread page. Nothing costs the extra read but an issue on more boards than a page
167//! holds: an issue this board really does not hold reports no next page, so its
168//! memberships are already exhausted where they arrived.
169//!
170//! **No document here selects the board's own `Labels` field, and nothing is lost by
171//! that.** An item's labels are read from its content alone, wherever that content is
172//! reached: the three documents above select `Issue.labels` on the fragment, and
173//! [`graphql::BOARD`] selects the same connection on the `... on Issue` arm of its
174//! `content`. A board's `Labels` field is not one anybody fills in: it is a built-in
175//! `ProjectV2FieldType`, it is absent from `ProjectV2CustomFieldType` so no project can
176//! create one, and `ProjectV2FieldValue` — the whole of what
177//! `updateProjectV2ItemFieldValue` accepts — offers no way to write one. So GitHub derives
178//! it from the content, for every content type it exists on, and there is nothing it can
179//! hold that the content does not already say: for an `Issue` it *is* that issue's own
180//! labels, so selecting it beside them unions a set with itself.
181//!
182//! **A draft loses nothing by that either**, which is the reasoning this paragraph once had
183//! backwards. `DraftIssue` exposes no `labels` field, and by the three schema facts above
184//! it cannot carry a board `Labels` value to be derived from one — so a draft has nothing
185//! to select *and nothing to lose*, and reports no labels at all. A `PullRequest` item is
186//! discarded by [`GitHubProjectsSource::resolve`] before labels are read. Both halves are
187//! held to that by tests in `tests/plugin.rs`: the four ways an item is reached report one
188//! label set, and that set is the fixture issue's own, by
189//! `an_item_reports_the_same_labels_title_status_and_id_however_it_is_reached`; and a board
190//! item whose content is a draft reports an empty set, by
191//! `a_board_item_whose_content_is_a_draft_reports_no_labels_at_all`. The absence of the
192//! selection is held over [`graphql::DOCUMENTS`] by
193//! `no_document_selects_the_boards_own_labels_field`.
194//!
195//! The last row is still the board's own item connection, and deliberately: a **draft**
196//! board item is not an issue, so no search and no node read can reach one, and the reads
197//! that have to answer for the whole board are the ones whose cost is the board's size
198//! anyway.
199//!
200//! **What a read may return is capped too, and that cap is on the document rather than on
201//! the board.** GitHub limits the number of nodes **one query may return** to
202//! [`NODE_COUNT_LIMIT`] and refuses a query above that before executing it: the answer is
203//! an error naming the connection the count crossed at, not a slow or a partial result.
204//! Every board this source reads is refused the same way, so no board is too big for these
205//! documents and none is small enough to save one that is over.
206//!
207//! The count is arithmetic over the document's own text: each connection contributes the
208//! `first:` it asks for, counts **multiply** down a nested path and **sum** across sibling
209//! paths. Those are [GitHub's published rules][node-limits] and this workspace does not
210//! restate them — `github-graphql-node-count` implements them, and
211//! [`worst_case_node_count`] under [`largest_page_sizes`] is where every node count here
212//! comes from. `every_document_this_source_sends_stays_under_githubs_node_limit`, in
213//! `tests/node_count.rs`, recomputes every document in [`graphql::DOCUMENTS`] from that
214//! same text on every run and fails naming any that reaches the limit — so a connection
215//! added to a shared fragment is caught there rather than by GitHub.
216//!
217//! What decides those counts is the page sizes: [`MAX_PAGE_SIZE`] on the outer page,
218//! `NESTED_PAGE_SIZE` on the connections hanging off one item, and
219//! `BOARD_ITEMS_PAGE_SIZE` on the page of an issue's board memberships a read carries.
220//! `$nestedFirst` is spent twice down one path of a board read, so that constant is
221//! effectively squared there, which is why it is the one the limit is most sensitive to.
222//! `BOARD_ITEMS_PAGE_SIZE` is small for a reason of its own, recorded beside it: what a
223//! page of memberships misses is recovered by one further read rather than refused, so it
224//! buys a bound every read pays for at the price of a request only a multi-board issue
225//! pays.
226//!
227//! **`nodeCount` and `cost` are two numbers against two limits, and both are computed
228//! offline here — per document, one document at a time.** `nodeCount` is the one above: the
229//! most nodes one query may return, checked per query and bounded by [`NODE_COUNT_LIMIT`].
230//! `cost` is rate-limit points, metered per hour across everything one credential does; it
231//! is what the two limiters [`Limiter`] tells apart meter, and a document under
232//! [`NODE_COUNT_LIMIT`] still says nothing about its price. [`worst_case_point_cost`] is
233//! that second number, and `tests/point_cost.rs` pins every document in
234//! [`graphql::DOCUMENTS`] at what it costs — there being no per-call point ceiling to hold
235//! one under, the pin itself is the check. The credentialed lane reconciles both figures
236//! against GitHub's own, off a probe it already sends.
237//!
238//! **What is pinned that way is a per-document price and never a session's.** The record in
239//! `session-cost.md` measures the two quantities a whole session can be counted in offline —
240//! **requests** and **worst-case nodes** — and neither is points. What one whole session
241//! consumes of the hourly point allowance is observable only from a credentialed run's own
242//! `x-ratelimit-*` headers, which is what [`accounting`] fills its per-budget figures from
243//! and what `tests/live.rs` prints at the end of every run.
244//!
245//! [node-limits]: https://docs.github.com/en/graphql/overview/rate-limits-and-node-limits-for-the-graphql-api
246//!
247//! **Where a read-after-write guarantee comes from, since a search index cannot supply
248//! one.** GitHub's issue search is eventually consistent and answers a write made moments
249//! ago with the value from before it. Resolving a node id is not, so a read by id and a
250//! project's own sub-issues are already current. What closes the gap for the search is
251//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
252//! what this process itself wrote, so an item created seconds ago is reported whether or
253//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
254//! and the record dies with the process.
255//!
256//! Filtering happens before paging, so a page of a filtered result is a page of the
257//! survivors rather than the survivors of a page. Label and text matching answer the same
258//! question the same way the local Markdown source's do, so one cross-source expectation
259//! holds for both.
260//!
261//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
262//! has one source, `capabilities`, and the note above is the reasoning behind it rather
263//! than a second copy of it: without the three facts recorded here a reader takes the
264//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
265//! crate's own capabilities test, which pins every field of it against a fully spelled-out
266//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
267//! fails to compile there rather than going unasserted. -->
268//! The fixture-server tests above run wherever this crate is selected; the credentialed
269//! lane runs in the same required check, beside them, and can fail it — it verifies the
270//! current schema, then drives every field of the table above against the real board. It builds its own fixture there — two projects, one task filed under each,
271//! one filed under neither, a label on one of the three and a closed status on another —
272//! because that shape is what tells an honoured predicate from an ignored one: a board
273//! holding a single project answers a project filter the same way whether or not this
274//! source applies it, which is exactly how the defect above went unseen.
275//!
276//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
277//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
278//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
279//! nominated is what keeps a credentialed write lane off a board and a repository nobody
280//! nominated; it never asks GitHub which project was updated most recently. Before it
281//! starts, the lane also clears any item titled — and any repository label named — the way
282//! it titles and names its own artifacts, which is self-healing after an interrupted run:
283//! a process killed between its writes and its cleanup leaves artifacts the next run
284//! removes.
285//!
286//! # What a session of requests costs, and where the report is
287//!
288//! This source records **every** request it sends into [`accounting::Accounting`], at
289//! `send_once` — the one place a request leaves this crate, which is why a read path added
290//! later is counted without anybody remembering to count it. That is the whole of what this
291//! crate adds to the arrangement; [`accounting`] is where what a record carries, how a
292//! session's spend is arrived at, and what it deliberately does not know are set out.
293//!
294//! What one whole session of the live journey costs, counted that way against this crate's
295//! loopback fixture board, is written down in `session-cost.md` beside this crate — with the
296//! reduction it came out of, and with what it does and does not say about rate-limit points.
297//!
298//! [`GitHubProjectsSource::accounting`] is the read: a snapshot to hold and compare, which
299//! [`accounting::Session::report`] renders the session report from. It is on the ordinary
300//! code path — no environment variable, no feature, no build configuration — because an
301//! instrument nobody switches on measures nothing, and
302//! [`Plugin::build_recording_into`] is how a caller making its own calls beside this
303//! source's counts the whole session rather than this source's share. The credentialed lane
304//! in `tests/live.rs` does exactly that, and prints the report at the end of every run,
305//! passed or failed.
306//!
307//! **A live session refuses to start unless the account can afford it.** Before it does any
308//! of the work it exists to do, the journey makes one request — `GET /rate_limit`, which
309//! GitHub documents as not counting against the REST rate limit and which answers both of
310//! its budgets at once — and starts only if, for each of them, what remains minus this
311//! session's estimated cost is still at least
312//! `onetaskgraph_live::RETAINED_BUFFER` — twenty per cent — of that budget's whole
313//! allowance. A session that cannot **declines**: it did not run, so it is
314//! neither a pass nor a failing assertion, and it says which budget was short, that budget's
315//! limit, what remained, the estimate, the buffer and when it resets — then stops, without
316//! waiting for the budget to come back. The estimate is derived offline from
317//! `tests/fixtures/session-cost.txt` and a cost model stated in `tests/journey/budget.rs`,
318//! which is also where the published rule that model rests on is cited; the accounting
319//! above records the gate's own read like any other request, and
320//! [`accounting::Session::report`] prints the estimate beside what the session really spent.
321//!
322//! **GitHub is the authority on both of its own numbers, and the credentialed lane goes and
323//! asks it.** Everything above computes `nodeCount` and `cost` offline from a document's own
324//! text, which is what lets it run on every platform and on a pull request from a fork with
325//! no credential — and that is what actually stops a regression merging. But an offline
326//! arithmetic can only ever agree with itself: if GitHub changes its rules, this workspace
327//! goes on computing the old answer and nothing notices. So `tests/live.rs` reconciles them.
328//! GitHub's schema exposes `rateLimit(dryRun: true)`, whose `nodeCount` is *"the maximum
329//! number of nodes this query may return"* and whose `cost` is what that document would
330//! spend, both for a document **without executing it**, and the lane asks it for every query
331//! document this source sends, under the largest bindings this source sends, and fails when
332//! GitHub's figure and [`worst_case_node_count`] or [`worst_case_point_cost`] disagree. A
333//! mutation is skipped, because `rateLimit` is a field of `Query` and cannot be asked about
334//! one; the offline pins still cover it. It records what those calls reported about the
335//! account's own allowance, because whether asking is free is a thing to observe rather than
336//! to assume. Two quantities, not one: [`NODE_COUNT_LIMIT`] bounds `nodeCount` per query,
337//! and `cost` is metered against an hourly allowance the accounting above reads off a
338//! credentialed run's own response headers.
339//!
340//! **GitHub has two rate limiters and this source is refused by both, so nothing here
341//! treats them as one thing.** The primary budget is the hourly allowance `gh api
342//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
343//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
344//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
345//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
346//! one part of.
347#![deny(missing_docs)]
348
349use std::collections::BTreeMap;
350use std::sync::{Arc, Mutex};
351use std::time::{Duration, Instant};
352
353use chrono::{DateTime, Utc};
354use onetaskgraph_plugin_api::{
355 Capabilities, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind, DependencySupport,
356 Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label, LabelFilter, Location,
357 Metering, NativeId, Page, PageRequest, Project, ProjectFilter, ProjectQuery, Repository,
358 SecretResolver, SourceError, SourceName, SourcePlugin, Status, StatusCategory, Support, Task,
359 TaskQuery, TaskSource, TextFields, TextQuery, WriteSupport,
360};
361use reqwest::{Client, StatusCode, Url};
362use schemars::{Schema, schema_for};
363use secrecy::{ExposeSecret, SecretString};
364use serde::Deserialize;
365use serde_json::{Value, json};
366
367pub mod accounting;
368
369use accounting::Accounting;
370
371/// The registry name for this plugin.
372pub const KIND: &str = "github-projects";
373/// GitHub's maximum connection page size.
374pub const MAX_PAGE_SIZE: u32 = 100;
375
376/// The most nodes any one document this source sends may be asked to return.
377///
378/// GitHub's own published per-query ceiling, taken from
379/// [`github_graphql_node_count::NODE_LIMIT`] rather than written out again here, so this
380/// workspace cannot hold a stale copy of somebody else's number. A query above it is
381/// **refused before it is executed**, whoever is asking and whatever board they are
382/// asking about — so this is a bound on the documents rather than a budget that runs out.
383///
384/// This is `nodeCount`, the maximum number of nodes *one query may return*. It is not
385/// `cost`, the rate-limit points a call spends against an hourly allowance shared by
386/// everything the credential does — two numbers against two limits, and this constant
387/// bounds only the first. The second is computed offline too, per document:
388/// [`worst_case_point_cost`], pinned for every document in [`graphql::DOCUMENTS`] by
389/// `tests/point_cost.rs`, and reconciled against GitHub's own `cost` by the credentialed
390/// lane. There is no constant like this one to hold a price under, because points are an
391/// hourly allowance rather than a per-call bound.
392///
393/// Neither is a session's price. What `session-cost.md` records of a whole session is its
394/// **requests** and its **worst-case nodes**; what a whole session spends in points is
395/// reported only by a credentialed run's own `x-ratelimit-*` headers, through
396/// [`accounting`]. The module section on the three ways this source reaches an item says how
397/// the count is arrived at, and which of the page sizes below decide it.
398pub const NODE_COUNT_LIMIT: u64 = github_graphql_node_count::NODE_LIMIT;
399
400/// Nested connection size for the connections that hang off one item.
401///
402/// It multiplies through every document that reaches an item under a page — the count
403/// rules multiply down a nested path — so it is the constant [`NODE_COUNT_LIMIT`] is most
404/// sensitive to. `tests/node_count.rs` is what holds the pair together: it recomputes
405/// every document under these constants and fails naming any that reaches the limit, so
406/// raising this is caught there rather than by GitHub.
407const NESTED_PAGE_SIZE: u32 = 50;
408/// How many of one issue's board memberships are read when an issue is reached directly.
409///
410/// An issue reached through a search or through its own node id carries its board half in
411/// `Issue.projectItems`, and only the entry for *this* board is read. This connection sits
412/// under a page of issues, so every point of it multiplies through the whole document and
413/// is paid for whether or not any issue is on a second board — which is why it is
414/// deliberately far smaller than [`NESTED_PAGE_SIZE`].
415///
416/// **Three, because what a page misses is now recovered rather than refused**, and the
417/// recovery is what the value is chosen against. An issue whose entry for this board sits
418/// past this page costs one further request — [`graphql::ISSUE_BOARD_ITEMS`], resumed from
419/// that page's own cursor — so the value trades a bound every read pays for a request only
420/// a multi-board issue pays. At one, a deployment whose issues commonly sit on two or more
421/// boards would pay that request *per issue*, which is order N against the one page per
422/// hundred issues a read costs today. At three it is only reached by an issue on four or
423/// more boards at once, which keeps the recovery path exceptional rather than routine for
424/// a plausible deployment.
425const BOARD_ITEMS_PAGE_SIZE: u32 = 3;
426
427pub use github_graphql_node_count::{NodeCountError, Variables};
428
429/// The largest value this source can bind to each page-size variable its documents name.
430///
431/// Every `first:` in [`graphql`] reads one of these three, and each is capped at the
432/// constant above it wherever a caller's own limit could reach it — `$first` at
433/// [`MAX_PAGE_SIZE`], `$nestedFirst` at `NESTED_PAGE_SIZE`, `$boardItems` at
434/// `BOARD_ITEMS_PAGE_SIZE`. So this is the worst case a caller can drive this source to,
435/// not one configuration of it, which is what makes a bound computed under it a bound on
436/// every read.
437pub fn largest_page_sizes() -> Variables {
438 Variables::from([
439 ("first".to_owned(), MAX_PAGE_SIZE),
440 ("nestedFirst".to_owned(), NESTED_PAGE_SIZE),
441 ("boardItems".to_owned(), BOARD_ITEMS_PAGE_SIZE),
442 ])
443}
444
445/// The most nodes `document` could be asked to return, by GitHub's published rules.
446///
447/// Computed offline from the document's own text under [`largest_page_sizes`] — no
448/// network, no credential and no schema — by
449/// [`github_graphql_node_count::node_count`], which is where the rules themselves live.
450/// A document at or above [`NODE_COUNT_LIMIT`] is one GitHub refuses before executing, so
451/// this is what a check holds every document in [`graphql::DOCUMENTS`] below.
452///
453/// # Errors
454///
455/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
456/// no single operation, or binds a page size this source does not name — each of which is
457/// a defect in the document rather than a number.
458pub fn worst_case_node_count(document: &str) -> Result<u64, NodeCountError> {
459 node_count(document, &largest_page_sizes())
460}
461
462/// The most rate-limit points one call of `document` could spend, by GitHub's published
463/// rules.
464///
465/// Computed offline from the document's own text under [`largest_page_sizes`] — no
466/// network, no credential and no schema — by
467/// [`github_graphql_node_count::point_cost`], which is where the rules themselves live.
468/// This is `cost`, metered **per hour** against the allowance one credential shares across
469/// everything it does; it is not `nodeCount`, which is [`worst_case_node_count`] and is
470/// bounded per query by [`NODE_COUNT_LIMIT`]. There is no per-call ceiling to hold this
471/// under, so what `tests/point_cost.rs` does with it is pin every document in
472/// [`graphql::DOCUMENTS`] at what it costs, and the credentialed lane reconciles those
473/// figures against GitHub's own reported `cost`.
474///
475/// # Errors
476///
477/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
478/// no single operation, or binds a page size this source does not name — each of which is
479/// a defect in the document rather than a number.
480pub fn worst_case_point_cost(document: &str) -> Result<u64, NodeCountError> {
481 github_graphql_node_count::point_cost(document, &largest_page_sizes())
482}
483
484/// The most nodes `document` could be asked to return under `variables`.
485///
486/// [`worst_case_node_count`] is this under [`largest_page_sizes`], and the accounting in
487/// [`accounting`] is this under the bindings one request really sent — one spelling of the
488/// calculation, so a bound checked offline and a cost recorded at run time cannot come to
489/// disagree. The rules themselves live in [`github_graphql_node_count::node_count`].
490///
491/// # Errors
492///
493/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
494/// no single operation, or binds a page size `variables` does not name.
495pub fn node_count(document: &str, variables: &Variables) -> Result<u64, NodeCountError> {
496 github_graphql_node_count::node_count(document, variables)
497}
498
499/// The issue-title prefix that makes a board issue a document.
500///
501/// A GitHub Projects board has no document type — it holds issues — so the discriminator
502/// is the title, and this is the whole of it: an issue whose title begins with these bytes
503/// is a document and every other issue is the task or project the sub-issue rule makes it.
504///
505/// It is spelled **once**, here, and read rather than restated everywhere else — including
506/// by the shared journeys, which take it from this constant so a board fixture cannot
507/// drift from what this source reads. `docs/metadata.md` records the two consequences that
508/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
509/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
510/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
511pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
512
513/// Exact GraphQL query documents issued by this plugin.
514///
515/// Keeping the production documents here lets the pinned-schema test validate the same
516/// bytes that are sent to GitHub, rather than a test-only copy which could drift
517/// independently. No document in this module writes the board itself, and none of them
518/// names `updateProjectV2Field`.
519pub mod graphql {
520 /// The board half of one item: the field values every document here reads it from.
521 ///
522 /// A macro for the same reason [`board_issue!`] below is one, a level further in. This
523 /// selection is needed by that fragment, by [`BOARD`] under the board's own `items`,
524 /// and by [`ISSUE_BOARD_ITEMS`] under a membership walk — and all three have to produce
525 /// *the same value*, because
526 /// [`GitHubProjectsSource::resolve`](super::GitHubProjectsSource) reads them through
527 /// one path. Three spellings of it is what would drift, so there is one.
528 ///
529 /// The `Status` option and this source's own origin text field are the whole of it. It
530 /// selects no `ProjectV2ItemFieldLabelValue`: GitHub derives that field from the item's
531 /// content, so it holds nothing the content's own `labels` do not already say, and it
532 /// would sit a label connection two page sizes deep.
533 macro_rules! board_item_values {
534 () => {
535 r#"fieldValues(first:$nestedFirst){nodes{
536 ... on ProjectV2ItemFieldSingleSelectValue{name field{
537 ... on ProjectV2SingleSelectField{id name options{id name}}
538 }}
539 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
540 }pageInfo{hasNextPage}}"#
541 };
542 }
543
544 /// Everything this source reads about one issue, wherever it reaches that issue.
545 ///
546 /// A macro rather than a constant so the three documents below can `concat!` it: one
547 /// spelling of these fields is what makes an issue read through the board-scoped
548 /// search, through its own node id, and through its project's sub-issue relationship
549 /// resolve to *the same* item, which is the whole of what
550 /// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
551 ///
552 /// `projectItems` is what carries the board half of an issue: the board item's own id
553 /// and the [`board_item_values!`] above — the `Status` option and this source's origin
554 /// text field — that a `ProjectV2.items` read used to carry. It is asked for on the
555 /// issue rather than on the board, which is what makes the cost of a read proportional
556 /// to what was asked for instead of to the board's size.
557 ///
558 /// It carries a *page* of that connection, at `BOARD_ITEMS_PAGE_SIZE`, and its
559 /// `endCursor` is what [`ISSUE_BOARD_ITEMS`] resumes from when this board's entry is
560 /// not on that page: a page here is where the search for the entry starts rather than
561 /// where it ends.
562 ///
563 /// It does **not** select the board's `Labels` field value, and that is the whole of
564 /// what keeps the three documents below under [`NODE_COUNT_LIMIT`](super::NODE_COUNT_LIMIT):
565 /// a label connection there sits under `fieldValues` under `projectItems` under a page
566 /// of issues, spending `$nestedFirst` twice down one path, and took
567 /// [`SEARCH_ISSUES`] and [`SUB_ISSUES`] to 2,556,100 nodes against a limit of 500,000.
568 /// No label is lost — this is a fragment `on Issue`, whose own `labels` are selected
569 /// above, and that connection is where every label this source reports comes from. No
570 /// document in this module selects the board field any longer, [`BOARD`] included; the
571 /// module documentation records why nothing it could have held is lost.
572 macro_rules! board_issue {
573 () => {
574 concat!(
575 r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
576 labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
577 projectItems(first:$boardItems){nodes{id project{id number}
578 "#,
579 board_item_values!(),
580 r#"}pageInfo{hasNextPage endCursor}}}"#
581 )
582 };
583 }
584
585 /// Every issue of one board, found by a search scoped to that board.
586 ///
587 /// This is how the projects a board holds are listed, and it selects no `items`
588 /// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
589 /// container walked page by page, so nothing nested inside a board item is paid for.
590 /// Which of the issues it returns is a project is then read off `parent` — GitHub
591 /// accepts `-has:parent` as a search qualifier and silently ignores it, so the
592 /// discriminator has to be applied to the field, which is a scalar on the issue and
593 /// costs nothing.
594 pub const SEARCH_ISSUES: &str = concat!(
595 r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
596 search(query:$search,type:$type,first:$first,after:$after){
597 pageInfo{hasNextPage endCursor}
598 nodes{__typename ...BoardIssue}
599 }
600 }"#,
601 board_issue!()
602 );
603
604 /// One issue by its own node id, which is what a qualified id names here.
605 ///
606 /// Strongly consistent, unlike the search above: GitHub's issue search is an index and
607 /// answers a write made moments ago with the value from before it, and resolving a node
608 /// id does not.
609 pub const ISSUE: &str = concat!(
610 r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
611 node(id:$id){__typename ...BoardIssue}
612 }"#,
613 board_issue!()
614 );
615
616 /// One project's tasks: the sub-issues of the issue that project is.
617 ///
618 /// The work this costs is the project's own size. Nothing about it grows as the board
619 /// gains projects, or as those projects gain tasks.
620 pub const SUB_ISSUES: &str = concat!(
621 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
622 node(id:$id){__typename
623 ... on Issue{subIssues(first:$first,after:$after){
624 pageInfo{hasNextPage endCursor}
625 nodes{__typename ...BoardIssue}
626 }}}
627 }"#,
628 board_issue!()
629 );
630
631 /// Reads the board's fields and one page of its items.
632 pub const BOARD: &str = concat!(
633 r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
634 owner:repositoryOwner(login:$owner){
635 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
636 }
637 } fragment Board on ProjectV2 { id title
638 fields(first:$nestedFirst){nodes{
639 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
640 ... on ProjectV2Field{__typename id name}
641 }pageInfo{hasNextPage}}
642 items(first:$first,after:$after){nodes{id "#,
643 board_item_values!(),
644 r#" content{
645 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
646 ... on PullRequest{__typename id}
647 ... on DraftIssue{__typename id title body createdAt updatedAt}
648 }} pageInfo{hasNextPage endCursor}}
649 }"#
650 );
651
652 /// One issue's board memberships alone, walked past the page a read of it carried.
653 ///
654 /// The recovery read behind [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource):
655 /// every document above carries a *page* of `Issue.projectItems`, and an issue on more
656 /// boards than that page holds may have this board's entry past its end. This asks that
657 /// one issue for its memberships and nothing else — the caller already holds the issue —
658 /// so an answer of "this board does not hold it" is only ever given about a connection
659 /// read to exhaustion.
660 ///
661 /// It selects the board item's id, its project number and the same
662 /// [`board_item_values!`] the fragment does, because what it produces is handed to the
663 /// very same resolver: an issue recovered this way reports the same title, the same
664 /// status, the same labels and the same qualified id as one whose entry was on the
665 /// page.
666 ///
667 /// `$first` rather than `$boardItems`: this document reads one issue, so nothing
668 /// multiplies through it and the membership connection can be walked at
669 /// [`MAX_PAGE_SIZE`](super::MAX_PAGE_SIZE) — which is what keeps the recovery to one
670 /// further request for any issue a person really keeps.
671 pub const ISSUE_BOARD_ITEMS: &str = concat!(
672 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!){
673 node(id:$id){
674 ... on Issue{projectItems(first:$first,after:$after){
675 nodes{id project{id number}
676 "#,
677 board_item_values!(),
678 r#"}
679 pageInfo{hasNextPage endCursor}}}
680 }
681 }"#
682 );
683 /// Resolves the configured repository's node id, which creating an issue requires.
684 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
685 /// Reads both dependency directions for one issue, with each far end's own kind — and
686 /// the issue's own body, which is where an edge to another source is recorded, so that
687 /// half of a dependency read needs no second read of the issue or of the board.
688 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
689 ... on Issue{body
690 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
691 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
692 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
693 /// Creates one issue in the configured repository.
694 pub const CREATE_ISSUE: &str =
695 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
696 /// Puts an existing issue on the configured board.
697 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
698 /// Updates an issue's visible fields and its open or closed state in one call.
699 pub const UPDATE_ISSUE: &str =
700 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
701 /// Updates an existing draft's user-visible fields.
702 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
703 /// Updates a text or single-select value on one project item.
704 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
705 /// Files one issue under another as a sub-issue, which is what project membership is.
706 pub const ADD_SUB_ISSUE: &str =
707 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
708 /// Takes one issue back out of its parent.
709 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
710 /// Adds GitHub's native issue blocked-by relationship.
711 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
712 /// Removes one native issue blocked-by relationship.
713 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
714 /// Deletes one issue, which takes its board item with it.
715 ///
716 /// The engine sends this in one situation only: undoing a copy that could not finish,
717 /// over the items that same copy created. Deleting the issue removes the board item
718 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
719 pub const DELETE_ISSUE: &str =
720 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
721
722 /// Every document above, with what this source is doing when it sends one.
723 ///
724 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
725 /// name the call that was refused, and a `match` with a catch-all arm would answer a
726 /// document added later with "talking to GitHub" and never say so.
727 ///
728 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
729 /// const` here that this list omits, so the two cannot part — which is the same guard
730 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
731 pub const DOCUMENTS: [(&str, &str); 17] = [
732 (SEARCH_ISSUES, "searching this board's issues"),
733 (ISSUE, "reading one issue"),
734 (
735 ISSUE_BOARD_ITEMS,
736 "reading one issue's board memberships past the page it came with",
737 ),
738 (SUB_ISSUES, "reading a project's tasks"),
739 (BOARD, "reading the board"),
740 (REPOSITORY, "reading the destination repository"),
741 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
742 (CREATE_ISSUE, "creating an issue"),
743 (ADD_TO_BOARD, "adding an issue to the board"),
744 (UPDATE_ISSUE, "updating an issue"),
745 (UPDATE_DRAFT, "updating a draft item"),
746 (UPDATE_FIELD, "writing a board field"),
747 (ADD_SUB_ISSUE, "filing an issue under its project"),
748 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
749 (ADD_BLOCKED_BY, "recording a dependency"),
750 (REMOVE_BLOCKED_BY, "removing a dependency"),
751 (DELETE_ISSUE, "deleting an issue"),
752 ];
753}
754
755/// Which of GitHub's two rate limiters refused a request.
756///
757/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
758/// secondary one — so an operator told the wrong one takes the wrong next step, which is
759/// the whole reason this is carried rather than collapsed into "rate limited".
760#[derive(Debug, Clone, Copy, PartialEq, Eq)]
761enum Limiter {
762 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
763 Primary,
764 /// The burst limiter over content-generating requests, which nothing reports.
765 Secondary,
766}
767
768/// The wordings GitHub answers a secondary rate limit with.
769///
770/// It sends them under a forbidden status, under a too-many-requests status, and inside
771/// the `errors` of a *successful* response, which is why the text is what this matches on
772/// rather than the status. `abuse detection` is the wording GitHub used before the
773/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
774/// what a burst of content creation is refused with.
775///
776/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
777/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
778/// when, and the drift gate reconciles the two lists both ways. Public for that gate
779/// alone — a caller has no use for it, and matching on a refusal is this source's job.
780pub const SECONDARY_WORDINGS: [&str; 5] = [
781 "secondary rate limit",
782 "temporarily blocked from content creation",
783 "abuse detection",
784 "submitted too quickly",
785 "exceeded a secondary",
786];
787
788/// The wordings GitHub answers an exhausted primary budget with.
789///
790/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
791/// than looked for in the response text. Pinned and gated exactly as
792/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
793pub const PRIMARY_WORDINGS: [&str; 3] = [
794 "api rate limit exceeded",
795 "rate limit exceeded",
796 "rate_limited",
797];
798
799/// What a response *says about itself*, which is the only place a refusal can be read.
800///
801/// Deliberately not the whole response body. A board is a place people write about their
802/// own work, and a task on it titled "the secondary rate limit" would, matched across the
803/// raw text, turn a perfectly good answer into a refusal this source then waited out and
804/// reported. So the item data is never read: what is read is GitHub's own REST-style
805/// `message` envelope, which is what a forbidden status carries, and the `message` and
806/// `type` of each GraphQL error, which is where a *successful* response says it.
807///
808/// A body that is not JSON at all has nothing structured to read, so only a failing
809/// response's own text is taken — a successful response that is not JSON is malformed
810/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
811fn refusal_wording(status: StatusCode, body: &str) -> String {
812 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
813 return if status.is_success() {
814 String::new()
815 } else {
816 body.to_owned()
817 };
818 };
819 let mut said: Vec<&str> = parsed
820 .get("message")
821 .and_then(Value::as_str)
822 .into_iter()
823 .collect();
824 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
825 for error in errors {
826 said.extend(
827 ["message", "type"]
828 .into_iter()
829 .filter_map(|key| error.get(key).and_then(Value::as_str)),
830 );
831 }
832 }
833 said.join("; ")
834}
835
836impl Limiter {
837 /// Which limiter refused this response, or `None` when none of them did.
838 ///
839 /// The wording is read first and the status only decides what carries none of it,
840 /// because GitHub answers a secondary limit with a forbidden status far more often
841 /// than with too-many-requests — while a forbidden status saying nothing about a limit
842 /// really is a credential this token lacks.
843 ///
844 /// A response is a refusal because of its status or its own wording. A spent budget
845 /// only ever explains one; it never turns an answer into a refusal.
846 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
847 let normalized = refusal_wording(status, body).to_ascii_lowercase();
848 if SECONDARY_WORDINGS
849 .iter()
850 .any(|wording| normalized.contains(wording))
851 {
852 return Some(Self::Secondary);
853 }
854 if status == StatusCode::TOO_MANY_REQUESTS {
855 return Some(Self::Primary);
856 }
857 // An exhausted budget *explains* a response that failed; it does not make one that
858 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
859 // request the budget allowed as well as on the ones it then refuses, so reading
860 // the header alone threw away a good answer — and, once refusals were retried,
861 // replayed a request that had already taken effect.
862 if !status.is_success() && budget_exhausted {
863 return Some(Self::Primary);
864 }
865 // A successful response saying it: GitHub reports a GraphQL rate limit in the
866 // `errors` of an HTTP 200, where nothing about the status says so at all.
867 if status.is_success()
868 && PRIMARY_WORDINGS
869 .iter()
870 .any(|wording| normalized.contains(wording))
871 {
872 return Some(Self::Primary);
873 }
874 None
875 }
876
877 /// What this limiter is called where an operator can look it up.
878 const fn name(self) -> &'static str {
879 match self {
880 Self::Primary => "GitHub's primary API rate limit",
881 Self::Secondary => "GitHub's secondary rate limit",
882 }
883 }
884
885 /// What the endpoint an operator would go and check says about this limiter.
886 const fn where_to_look(self) -> &'static str {
887 match self {
888 Self::Primary => {
889 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
890 comes back."
891 }
892 Self::Secondary => {
893 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
894 primary budget and does not report this one, so budget showing there says \
895 nothing about this refusal, and every further attempt extends it."
896 }
897 }
898 }
899
900 /// The next step this limiter actually calls for.
901 const fn what_to_do(self) -> &'static str {
902 match self {
903 Self::Primary => {
904 "wait for the reset `gh api rate_limit` reports, then run the command again."
905 }
906 Self::Secondary => {
907 "leave this board alone for a few minutes, then run the command again — or \
908 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
909 }
910 }
911 }
912}
913
914/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
915#[derive(Debug, Clone, Copy)]
916struct Limited {
917 limiter: Limiter,
918 hint: Option<u64>,
919}
920
921impl Limited {
922 /// What the caller is told once this source has waited as long as it may.
923 ///
924 /// Both limiters report as [`SourceError::RateLimited`], because that is what
925 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
926 /// about *which* limiter it was makes it a different kind of failure. What differs is
927 /// the operator's next step, and that is what the message carries — a secondary
928 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
929 /// budget looks fine, and then back to retry the very burst that was refused.
930 fn exhausted(
931 self,
932 doing: &str,
933 waits: u32,
934 waited: Duration,
935 needed: Duration,
936 budget: Duration,
937 ) -> SourceError {
938 SourceError::RateLimited {
939 retry_after_seconds: self.hint,
940 message: Some(format!(
941 "{} refused this source while {doing}; it waited {} out over {} and was refused \
942 again, and the next wait of {} would take it past the {} one call may spend \
943 waiting. {} next: {}",
944 self.limiter.name(),
945 plural(waits, "refusal"),
946 seconds(waited),
947 seconds(needed),
948 seconds(budget),
949 self.limiter.where_to_look(),
950 self.limiter.what_to_do(),
951 )),
952 }
953 }
954}
955
956/// One HTTP attempt's result, with what its response said about the rate limit.
957///
958/// The two travel together so the record and the outcome are written from the same place:
959/// what a response said about the budget is only readable while that response is in hand,
960/// and what the attempt *meant* is only decidable once its body has been read.
961struct Attempted {
962 result: Result<Value, Attempt>,
963 limits: accounting::RateLimit,
964 /// GitHub's own reported cost for this call, for a document that asked for it.
965 reported_cost: Option<u64>,
966}
967
968/// One attempt's outcome: an error to report, or a rate limit to wait out.
969enum Attempt {
970 Failed(SourceError),
971 Limited(Limited),
972}
973
974fn plural(count: u32, thing: &str) -> String {
975 if count == 1 {
976 format!("{count} {thing}")
977 } else {
978 format!("{count} {thing}s")
979 }
980}
981
982fn seconds(duration: Duration) -> String {
983 format!("{:.1}s", duration.as_secs_f64())
984}
985
986/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
987///
988/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
989/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
990/// header, and neither is what makes a response a refusal — so the whole cost of one this
991/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
992/// instead. Refusing the response over the header would turn a readable refusal into an
993/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
994fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
995 value
996 .and_then(|value| value.to_str().ok())
997 .and_then(|value| value.trim().parse::<u64>().ok())
998}
999
1000/// Every mutation this source sends creates content — an issue, a board item, a field of
1001/// one, a sub-issue link, a dependency — and no query in [`graphql::DOCUMENTS`] does, so
1002/// what the secondary limiter counts and what the keyword says are the same set. That is
1003/// what makes the keyword a sound test rather than a convenient one.
1004fn is_mutation(query: &str) -> bool {
1005 query.trim_start().starts_with("mutation")
1006}
1007
1008/// What this source was doing, for a diagnostic that has to say so.
1009///
1010/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
1011/// a document added without a description is caught by that list's own gate instead of
1012/// falling through to the vague arm below.
1013fn operation_description(query: &str) -> &'static str {
1014 graphql::DOCUMENTS
1015 .iter()
1016 .find(|(document, _)| *document == query)
1017 .map_or("talking to GitHub", |(_, doing)| *doing)
1018}
1019
1020/// GitHub's published ceiling on content-generating requests, per minute.
1021///
1022/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
1023/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
1024/// from it, so a pacing value checked only against itself cannot go stale here.
1025pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
1026/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
1027/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
1028/// governs; it is pinned beside its sibling so the gate would notice either one moving.
1029pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
1030/// Shortest interval between two content-creating mutations, in milliseconds.
1031///
1032/// GitHub documents two secondary limits on content-generating requests:
1033/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
1034/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
1035/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
1036/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
1037/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
1038/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
1039/// deliberately *not* what this paces at. An installation that wants the hourly bound
1040/// honoured for a long sequence of copies says so through
1041/// `pacing.min_mutation_interval_ms`.
1042pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
1043/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
1044///
1045/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
1046/// own advice for a secondary limit — wait, and wait longer each time — without spending
1047/// the first minute of a transient refusal doing nothing.
1048pub const RETRY_BACKOFF_MS: u64 = 1_000;
1049/// Total time one call may spend waiting out rate limits before it reports a failure.
1050///
1051/// Two minutes is long enough to ride out the refusals a paced copy still collects and
1052/// short enough that a command an operator is watching returns. The bound is what makes
1053/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
1054/// the limiter, not in a process nobody can tell from a wedged one.
1055pub const RETRY_BUDGET_MS: u64 = 120_000;
1056
1057fn default_token_env() -> String {
1058 "GH_PROJECTS_TOKEN".to_owned()
1059}
1060fn default_endpoint() -> String {
1061 "https://api.github.com/graphql".to_owned()
1062}
1063
1064/// Where one status category lands on this board.
1065///
1066/// `null` — an absent value — disables the category for this instance, and using a
1067/// disabled status is a refusal naming the status and the instance.
1068#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
1069#[serde(untagged)]
1070pub enum StatusTargetConfig {
1071 /// The name of a `Status` single-select option already on the board.
1072 Column(ColumnName),
1073 /// A closed issue state, whose reason is what tells done from cancelled.
1074 Closed {
1075 /// The `IssueClosedStateReason` to close with.
1076 closed: ClosedState,
1077 },
1078}
1079
1080/// The name of a `Status` single-select option on the board.
1081///
1082/// Validated on the way in rather than checked later, so a blank option name — which
1083/// nothing on a board can be — is a state this type cannot hold.
1084#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1085#[serde(try_from = "String")]
1086pub struct ColumnName(String);
1087
1088impl ColumnName {
1089 /// The option name, as the board spells it.
1090 fn as_str(&self) -> &str {
1091 &self.0
1092 }
1093}
1094
1095impl TryFrom<String> for ColumnName {
1096 type Error = String;
1097
1098 fn try_from(name: String) -> Result<Self, Self::Error> {
1099 if name.trim().is_empty() {
1100 return Err("a status_mapping option name cannot be blank".to_owned());
1101 }
1102 Ok(Self(name))
1103 }
1104}
1105
1106/// The two closed states this product can mean.
1107///
1108/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
1109/// work nor abandoned work, so nothing here ever writes it.
1110#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1111#[serde(rename_all = "kebab-case")]
1112pub enum ClosedState {
1113 /// `COMPLETED` — precisely done.
1114 Completed,
1115 /// `NOT_PLANNED` — precisely cancelled.
1116 NotPlanned,
1117}
1118
1119impl ClosedState {
1120 const fn reason(self) -> &'static str {
1121 match self {
1122 Self::Completed => "COMPLETED",
1123 Self::NotPlanned => "NOT_PLANNED",
1124 }
1125 }
1126}
1127
1128/// Configuration for one GitHub Projects v2 board.
1129#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1130#[serde(default, deny_unknown_fields)]
1131pub struct GitHubProjectsConfig {
1132 /// Login of the user or organization which owns the board.
1133 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
1134 /// The project number shown in the board's GitHub URL.
1135 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
1136 // llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] This doc is the field's schema description, which is what a person configuring the source reads, so it has to say when the field decides an issue's repository and when the item's own field does; the rule's one executable source is `GitHubProjectsSource::creation_target`, and `tests/plugin.rs` drives each case named here against the loopback board.
1137 /// `owner/name` of the repository this source creates an issue in when the item's own
1138 /// `repositories` field does not decide it.
1139 ///
1140 /// An item naming exactly one repository is created there; a task or a document naming
1141 /// none or several is created in its parent project's repository; and a project, or a
1142 /// task or document with no parent, naming none or several is created here. A board
1143 /// has no repository of its own and `createIssue` requires one, so a write without
1144 /// this is refused naming the field. Reads never need it.
1145 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
1146 // llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate]
1147 /// Environment variable containing a fine-grained token with Projects and Issues
1148 /// read/write plus Pull requests read-only access for every repository represented on
1149 /// the board.
1150 #[serde(default = "default_token_env")]
1151 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
1152 /// GraphQL endpoint. GitHub Enterprise installations may override it.
1153 #[serde(default = "default_endpoint")]
1154 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
1155 /// Per-instance mapping from a status category to where it lands on this board.
1156 ///
1157 /// A category this does not mention keeps its shipped default: `backlog` to
1158 /// "Backlog", `todo` to "Todo", `in-progress` to "In Progress", `done` to closed as
1159 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
1160 /// disabled. `unknown` may name one existing board option; every unknown word then
1161 /// lands on that option and reads back as `unknown` under its name. It cannot name a
1162 /// closed state because that reads back as `done` or `cancelled`. Unlike `local-md`,
1163 /// this source cannot keep each unknown word because it never creates board options.
1164 #[serde(default)]
1165 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
1166 /// How fast this source writes, and how long it waits out a rate-limit refusal.
1167 ///
1168 /// Every field keeps its shipped default when it is absent, and the defaults are
1169 /// GitHub's own published limits rather than taste. See [`Pacing`].
1170 #[serde(default)]
1171 pub pacing: PacingConfig,
1172}
1173
1174/// How fast this source writes, and how long it waits out a rate-limit refusal.
1175///
1176/// Configurable because a GitHub Enterprise installation sets its own limits and an
1177/// operator who has already been refused may want to go slower still — not because the
1178/// defaults are guesses.
1179#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1180#[serde(default, deny_unknown_fields)]
1181pub struct PacingConfig {
1182 /// Shortest interval between two content-creating mutations, in milliseconds.
1183 ///
1184 /// Zero sends them as fast as they are asked for, which is what a fixture server on
1185 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
1186 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1187 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
1188 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
1189 /// zero while there is a budget to spend, because a schedule of zero-length waits
1190 /// consumes none of it and so never ends.
1191 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
1192 /// Total time one call may spend waiting out rate limits, in milliseconds.
1193 ///
1194 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
1195 /// the bound is what makes this a wait rather than a hang.
1196 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1197}
1198
1199/// The largest any pacing setting may be, in milliseconds.
1200///
1201/// One hour. GitHub's own harshest published bound on content-generating requests works
1202/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
1203/// anything a real limit asks for, and past it the settings stop describing pacing at all:
1204/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
1205/// and an interval beyond it is a command that never sends its second mutation. It also
1206/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
1207/// what an `Instant` can hold on every platform.
1208pub const MAX_PACING_MS: u64 = 3_600_000;
1209
1210/// [`PacingConfig`] with every default resolved and every value checked, which is what the
1211/// source holds.
1212#[derive(Debug, Clone, Copy)]
1213struct Pacing {
1214 min_mutation_interval: Duration,
1215 retry_backoff: Duration,
1216 retry_budget: Duration,
1217}
1218
1219impl Pacing {
1220 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
1221 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
1222 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
1223 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
1224 message: format!(
1225 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
1226 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
1227 GitHub's own harshest published limit"
1228 ),
1229 }),
1230 Some(value) => Ok(Duration::from_millis(value)),
1231 None => Ok(Duration::from_millis(default)),
1232 };
1233 let retry_backoff = bounded(
1234 config.retry_backoff_ms,
1235 RETRY_BACKOFF_MS,
1236 "retry_backoff_ms",
1237 )?;
1238 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
1239 if retry_backoff.is_zero() && !retry_budget.is_zero() {
1240 return Err(SourceError::Config {
1241 message: format!(
1242 "pacing.retry_backoff_ms of source {instance} is 0 while \
1243 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
1244 none of that budget, so it would retry a refusal forever. Set a backoff of \
1245 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
1246 waiting at all",
1247 retry_budget.as_millis()
1248 ),
1249 });
1250 }
1251 Ok(Self {
1252 min_mutation_interval: bounded(
1253 config.min_mutation_interval_ms,
1254 MIN_MUTATION_INTERVAL_MS,
1255 "min_mutation_interval_ms",
1256 )?,
1257 retry_backoff,
1258 retry_budget,
1259 })
1260 }
1261}
1262
1263/// Factory for [`GitHubProjectsSource`].
1264#[derive(Debug, Clone, Copy, Default)]
1265pub struct Plugin;
1266
1267impl SourcePlugin for Plugin {
1268 fn kind(&self) -> &'static str {
1269 KIND
1270 }
1271 fn config_schema(&self) -> Schema {
1272 schema_for!(GitHubProjectsConfig)
1273 }
1274 fn build(
1275 &self,
1276 name: &SourceName,
1277 config: &Value,
1278 secrets: &dyn SecretResolver,
1279 ) -> Result<Box<dyn TaskSource>, SourceError> {
1280 self.build_recording_into(name, config, secrets, Arc::new(Accounting::new()))
1281 }
1282}
1283
1284impl Plugin {
1285 /// Build a source recording every request it sends into an accounting the caller holds.
1286 ///
1287 /// [`SourcePlugin::build`] is this with an accounting of its own, which is what the
1288 /// registry gets. This is for a caller that is also calling GitHub itself and wants one
1289 /// session total rather than two — see [`accounting`] and
1290 /// [`GitHubProjectsSource::recording_into`].
1291 ///
1292 /// # Errors
1293 ///
1294 /// Exactly [`SourcePlugin::build`]'s, with the same source name in front of each:
1295 /// [`SourceError::Config`] for configuration this plugin cannot use and
1296 /// [`SourceError::Auth`] for a credential it cannot find.
1297 pub fn build_recording_into(
1298 &self,
1299 name: &SourceName,
1300 config: &Value,
1301 secrets: &dyn SecretResolver,
1302 ledger: Arc<Accounting>,
1303 ) -> Result<Box<dyn TaskSource>, SourceError> {
1304 let config: GitHubProjectsConfig =
1305 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
1306 message: format!("source {name}: {e}"),
1307 })?;
1308 let source = GitHubProjectsSource::recording_into(name, config, secrets, ledger).map_err(
1309 |error| match error {
1310 SourceError::Config { message } => SourceError::Config {
1311 message: format!("source {name}: {message}"),
1312 },
1313 SourceError::Auth { message } => SourceError::Auth {
1314 message: format!("source {name}: {message}"),
1315 },
1316 other => other,
1317 },
1318 )?;
1319 Ok(Box::new(source))
1320 }
1321}
1322
1323/// Where a status category lands on this board, once configuration is resolved.
1324#[derive(Debug, Clone, PartialEq, Eq)]
1325enum StatusTarget {
1326 /// Not usable against this instance.
1327 Disabled,
1328 /// The board's `Status` option of this name.
1329 Column(ColumnName),
1330 /// A closed issue, with the reason that says which closed it means.
1331 Closed(ClosedState),
1332}
1333
1334/// Every status category, in the order the vocabulary declares them.
1335///
1336/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
1337/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
1338/// added to the shared vocabulary fails to compile until it is named there, and this
1339/// crate's suite reconciles this list against that enum's own derived schema, which is
1340/// generated from the variants rather than written beside them. The schema is what
1341/// catches a list left one short — a list checking only the positions it already holds
1342/// would pass while every mapping indexed by the new position panicked.
1343pub const CATEGORIES: [StatusCategory; 7] = [
1344 StatusCategory::Draft,
1345 StatusCategory::Backlog,
1346 StatusCategory::Todo,
1347 StatusCategory::InProgress,
1348 StatusCategory::Done,
1349 StatusCategory::Cancelled,
1350 StatusCategory::Unknown,
1351];
1352
1353/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
1354#[must_use]
1355pub const fn category_position(category: StatusCategory) -> usize {
1356 match category {
1357 StatusCategory::Draft => 0,
1358 StatusCategory::Backlog => 1,
1359 StatusCategory::Todo => 2,
1360 StatusCategory::InProgress => 3,
1361 StatusCategory::Done => 4,
1362 StatusCategory::Cancelled => 5,
1363 StatusCategory::Unknown => 6,
1364 }
1365}
1366
1367/// The spelling a status category is configured and reported under.
1368fn category_name(category: StatusCategory) -> &'static str {
1369 match category {
1370 StatusCategory::Draft => "draft",
1371 StatusCategory::Backlog => "backlog",
1372 StatusCategory::Todo => "todo",
1373 StatusCategory::InProgress => "in-progress",
1374 StatusCategory::Done => "done",
1375 StatusCategory::Cancelled => "cancelled",
1376 StatusCategory::Unknown => "unknown",
1377 }
1378}
1379
1380/// A shipped default's option name.
1381///
1382/// The literals below are this file's own and non-blank, and they are validated by the
1383/// one constructor a configured name goes through rather than beside it.
1384fn shipped_column(name: &'static str) -> ColumnName {
1385 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
1386}
1387
1388/// The shipped default for one category, before this instance's configuration.
1389fn shipped_default(category: StatusCategory) -> StatusTarget {
1390 match category {
1391 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
1392 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
1393 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
1394 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
1395 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
1396 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
1397 }
1398}
1399
1400/// This instance's complete category-to-target mapping, read in both directions.
1401///
1402/// One target per category, held at that category's own [`category_position`], so a
1403/// category missing from the mapping, named twice in it, or filed out of order is a
1404/// state this type cannot hold rather than one [`Self::target`] has to defend against.
1405#[derive(Debug, Clone)]
1406struct StatusMapping {
1407 targets: [StatusTarget; CATEGORIES.len()],
1408}
1409
1410impl StatusMapping {
1411 fn resolve(
1412 configured: BTreeMap<String, Option<StatusTargetConfig>>,
1413 instance: &SourceName,
1414 ) -> Result<Self, SourceError> {
1415 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
1416 for (key, value) in configured {
1417 let category = CATEGORIES
1418 .iter()
1419 .find(|category| category_name(**category) == key)
1420 .ok_or_else(|| SourceError::Config {
1421 message: format!(
1422 "status_mapping names {key:?}, which is not a status category of source \
1423 {instance}; the categories are {}",
1424 CATEGORIES
1425 .iter()
1426 .map(|category| category_name(*category))
1427 .collect::<Vec<_>>()
1428 .join(", ")
1429 ),
1430 })?;
1431 overrides.insert(category_name(*category), value);
1432 }
1433 if let Some(Some(StatusTargetConfig::Closed { closed })) =
1434 overrides.get(category_name(StatusCategory::Unknown))
1435 {
1436 let read_back = match closed {
1437 ClosedState::Completed => StatusCategory::Done,
1438 ClosedState::NotPlanned => StatusCategory::Cancelled,
1439 };
1440 return Err(SourceError::Config {
1441 message: format!(
1442 "status_mapping.unknown of source {instance} cannot target the closed state \
1443 {} because a copy reads that item back as {}; map unknown to one existing \
1444 board Status option instead",
1445 closed.reason().to_ascii_lowercase().replace('_', "-"),
1446 category_name(read_back)
1447 ),
1448 });
1449 }
1450 // `CATEGORIES[position] == category` for every category — the crate's suite
1451 // asserts it — so mapping the list in order fills each category's own slot.
1452 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
1453 None => shipped_default(category),
1454 Some(None) => StatusTarget::Disabled,
1455 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
1456 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
1457 });
1458 let mapping = Self { targets };
1459 for (index, category) in CATEGORIES.into_iter().enumerate() {
1460 let StatusTarget::Column(option) = mapping.target(category) else {
1461 continue;
1462 };
1463 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
1464 matches!(mapping.target(**earlier), StatusTarget::Column(name)
1465 if name.as_str().eq_ignore_ascii_case(option.as_str()))
1466 }) {
1467 return Err(SourceError::Config {
1468 message: format!(
1469 "status_mapping of source {instance} sends both {} and {} to the board \
1470 option {:?}; one option cannot read back as two categories",
1471 category_name(*other),
1472 category_name(category),
1473 option.as_str()
1474 ),
1475 });
1476 }
1477 }
1478 Ok(mapping)
1479 }
1480
1481 fn target(&self, category: StatusCategory) -> &StatusTarget {
1482 &self.targets[category_position(category)]
1483 }
1484
1485 /// The category a board option name reports, or `None` when nothing maps to it.
1486 fn category_of(&self, option: &str) -> Option<StatusCategory> {
1487 CATEGORIES.into_iter().find(|category| {
1488 matches!(self.target(*category), StatusTarget::Column(name)
1489 if name.as_str().eq_ignore_ascii_case(option))
1490 })
1491 }
1492}
1493
1494// llmlint: ignore-block[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate] Every `createIssue` names one of these, and which one is the rule — a reader who reaches the type from `create_and_file_issue` gets the rule in one sentence here without the method's refusals, which stay on `creation_target`, the rule's one executable source; `tests/plugin.rs` drives every arm of it against the loopback board.
1495/// One repository this source can create an issue in, as `owner/name`.
1496///
1497/// Every `createIssue` this source sends names one of these: the item's own single
1498/// `repositories` entry, else its parent project issue's repository, else the configured
1499/// [`GitHubProjectsConfig::repository`]. [`GitHubProjectsSource::creation_target`] makes
1500/// that choice and says what it refuses before `createIssue`.
1501// llmlint: ignore-end[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate]
1502#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
1503struct RepositoryTarget {
1504 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1505 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1506}
1507
1508impl RepositoryTarget {
1509 fn parse(value: &str) -> Result<Self, SourceError> {
1510 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
1511 message: format!(
1512 "repository must be spelled owner/name; {value:?} names no repository"
1513 ),
1514 })?;
1515 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
1516 return Err(SourceError::Config {
1517 message: format!(
1518 "repository must be spelled owner/name with a GitHub login and one \
1519 repository name; {value:?} is not"
1520 ),
1521 });
1522 }
1523 Ok(Self {
1524 owner: owner.to_owned(),
1525 name: name.to_owned(),
1526 })
1527 }
1528
1529 /// The one host whose repositories this source creates issues in, spelled once: it is
1530 /// what [`Self::origin`] renders and what [`Self::from_origin`] accepts.
1531 const HOST: &str = "github.com";
1532
1533 fn origin(&self) -> String {
1534 format!("{}/{}/{}", Self::HOST, self.owner, self.name)
1535 }
1536
1537 /// The repository a normalized origin names, or why it is none this source can create
1538 /// an issue in: another host, or more or fewer than `owner/name` under this one.
1539 fn from_origin(origin: &Repository) -> Result<Self, String> {
1540 let not_here = || {
1541 format!(
1542 "{} is not a {}/owner/name repository",
1543 origin.as_str(),
1544 Self::HOST
1545 )
1546 };
1547 let (host, rest) = origin.as_str().split_once('/').ok_or_else(not_here)?;
1548 if host != Self::HOST {
1549 return Err(not_here());
1550 }
1551 Self::parse(rest).map_err(|_| not_here())
1552 }
1553
1554 fn slug(&self) -> String {
1555 format!("{}/{}", self.owner, self.name)
1556 }
1557}
1558
1559/// A source which reads GitHub afresh for every operation.
1560pub struct GitHubProjectsSource {
1561 /// This source's configured name, used both to tell a far end naming this source
1562 /// from one naming a system it knows nothing about, and to name the instance a
1563 /// status refusal is about.
1564 name: SourceName,
1565 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
1566 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
1567 repository: Option<RepositoryTarget>,
1568 endpoint: Url,
1569 token: SecretString,
1570 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1571 statuses: StatusMapping,
1572 client: Client,
1573 /// Every item this source has created since it was built, in the order it created
1574 /// them.
1575 ///
1576 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1577 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1578 /// a copy resolving a dependency on an item it had just created refused it as not
1579 /// found. A board read is completed from this — an item remembered here and absent from
1580 /// the read is added back, because the board really does hold it and only the read is
1581 /// behind.
1582 ///
1583 /// It is not a cache of a user's work: nothing is remembered that this process did not
1584 /// itself just write, it lives and dies with the process, and it is never consulted for
1585 /// an item this source did not create.
1586 created: Mutex<Vec<Resolved>>,
1587 /// How fast this source writes, and how long it waits out a refusal.
1588 pacing: Pacing,
1589 /// When the last content-creating mutation finished, or the moment the furthest-out
1590 /// reserved slot releases the next one, whichever is later — so the one after it can be
1591 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1592 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1593 /// what it is measured from.
1594 last_mutation: Mutex<Option<Instant>>,
1595 /// The board as this process last read it, for the length of one command.
1596 ///
1597 /// A copy of a project used to re-read the whole board, paged, before writing each of
1598 /// its items, which is by far the largest part of a copy's request count and none of
1599 /// its work. Nothing else changes this board while a command runs — this source's own
1600 /// writes are the only writer — so one read answers them all.
1601 ///
1602 /// It is not a store of a user's work and it is not the cache the no-persistence
1603 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1604 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1605 /// an item this command created and then depends on resolves whether or not GitHub's
1606 /// own eventually-consistent read has caught up. A write to an item already on the
1607 /// board updates the entry here too, so what this holds is the last read plus this
1608 /// process's own writes rather than a snapshot taken before them.
1609 board_cache: Mutex<Option<Board>>,
1610 /// Each destination repository's node id, resolved once per repository
1611 /// rather than per issue created.
1612 ///
1613 /// A repository's node id does not change, and re-reading it for every issue of a copy
1614 /// spent one request per item on an answer this source already had. It is a map rather
1615 /// than one entry because a copy files each item in the repository its own
1616 /// `repositories` field names, so a plan across five repositories asks GitHub five
1617 /// times and not once per item.
1618 repository_cache: Mutex<BTreeMap<RepositoryTarget, String>>,
1619 /// What every request this source sends is recorded into.
1620 ///
1621 /// Ordinary code path, not a mode: [`Self::send_once`] records into it at the one place
1622 /// a request leaves this crate, so nothing has to be switched on for a session to be
1623 /// counted. It is shared rather than owned so a caller accounting for a whole session —
1624 /// its own schema verification, board lookups, residue sweep and cleanup beside this
1625 /// source's reads and writes — adds up one accounting instead of two. See
1626 /// [`accounting`] for what a record carries and what a session's spend is and is not.
1627 ledger: Arc<Accounting>,
1628}
1629
1630impl GitHubProjectsSource {
1631 /// Validate configuration and capture the named credential without exposing it.
1632 ///
1633 /// # Errors
1634 ///
1635 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1636 /// [`SourceError::Auth`] when the named credential is missing or empty.
1637 pub fn new(
1638 name: &SourceName,
1639 config: GitHubProjectsConfig,
1640 secrets: &dyn SecretResolver,
1641 ) -> Result<Self, SourceError> {
1642 Self::recording_into(name, config, secrets, Arc::new(Accounting::new()))
1643 }
1644
1645 /// The same, recording every request it sends into an accounting the caller holds too.
1646 ///
1647 /// [`Self::new`] is this with an accounting of its own. A caller that is also making
1648 /// its own calls to GitHub — a lane verifying a schema, sweeping residue or cleaning
1649 /// up — passes the one it records those into, so the session total accounts for the
1650 /// whole session rather than for this source's share of it.
1651 ///
1652 /// # Errors
1653 ///
1654 /// Exactly [`Self::new`]'s: [`SourceError::Config`] for a configuration this instance
1655 /// cannot use and [`SourceError::Auth`] when the named credential is missing or empty.
1656 pub fn recording_into(
1657 name: &SourceName,
1658 config: GitHubProjectsConfig,
1659 secrets: &dyn SecretResolver,
1660 ledger: Arc<Accounting>,
1661 ) -> Result<Self, SourceError> {
1662 if !valid_github_owner(&config.owner) {
1663 return Err(SourceError::Config {
1664 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1665 });
1666 }
1667 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1668 return Err(SourceError::Config {
1669 message: format!("project_number must be between 1 and {}", i32::MAX),
1670 });
1671 }
1672 if !valid_environment_name(&config.token_env) {
1673 return Err(SourceError::Config {
1674 message: "token_env must be a valid environment-variable name".into(),
1675 });
1676 }
1677 let repository = config
1678 .repository
1679 .as_deref()
1680 .map(RepositoryTarget::parse)
1681 .transpose()?;
1682 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1683 message: format!("endpoint is not a valid URL: {e}"),
1684 })?;
1685 if endpoint.scheme() != "https"
1686 && !(endpoint.scheme() == "http"
1687 && endpoint
1688 .host_str()
1689 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1690 {
1691 return Err(SourceError::Config {
1692 message:
1693 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1694 .into(),
1695 });
1696 }
1697 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1698 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1699 })?;
1700 Ok(Self {
1701 name: name.clone(),
1702 owner: config.owner,
1703 project_number: config.project_number,
1704 repository,
1705 endpoint,
1706 token,
1707 credential_name: config.token_env,
1708 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1709 client: Client::builder()
1710 .user_agent("onetaskgraph")
1711 .build()
1712 .map_err(|e| SourceError::Config {
1713 message: format!("cannot build HTTP client: {e}"),
1714 })?,
1715 created: Mutex::new(Vec::new()),
1716 pacing: Pacing::resolve(config.pacing, name)?,
1717 last_mutation: Mutex::new(None),
1718 board_cache: Mutex::new(None),
1719 repository_cache: Mutex::new(BTreeMap::new()),
1720 ledger,
1721 })
1722 }
1723
1724 /// A snapshot of every request this source has sent, and what each cost.
1725 ///
1726 /// A value to hold and compare rather than a borrow of the accounting itself, so two
1727 /// of them can sit side by side. When this source was built with
1728 /// [`Self::recording_into`] the snapshot is the whole shared session, which is the
1729 /// point of building it that way.
1730 #[must_use]
1731 pub fn accounting(&self) -> accounting::Session {
1732 self.ledger.snapshot()
1733 }
1734
1735 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1736 /// rate limit rather than handing it straight back as an error.
1737 ///
1738 /// Retrying is safe for every document here, including the mutations, and the reason
1739 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1740 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1741 /// this replays has already taken effect. An outcome this source cannot know — the
1742 /// send failed, or the body could not be read, so the mutation may well have landed —
1743 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1744 /// attempt. A duplicate write would come from replaying one of those, and none is
1745 /// replayed.
1746 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1747 let doing = operation_description(query);
1748 let mut waited = Duration::ZERO;
1749 let mut waits = 0_u32;
1750 let mut backoff = self.pacing.retry_backoff;
1751 loop {
1752 if is_mutation(query) {
1753 let spacing = self.reserve_mutation_slot();
1754 if !spacing.is_zero() {
1755 tokio::time::sleep(spacing).await;
1756 }
1757 }
1758 let attempt = self.send_once(query, &variables).await;
1759 if is_mutation(query) {
1760 self.finish_mutation();
1761 }
1762 let limited = match attempt {
1763 Ok(data) => return Ok(data),
1764 Err(Attempt::Failed(error)) => return Err(error),
1765 Err(Attempt::Limited(limited)) => limited,
1766 };
1767 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1768 // move that extends a secondary limit, so a hint below the schedule's own next
1769 // wait is raised to it.
1770 let wait = match limited.hint {
1771 Some(hint) => Duration::from_secs(hint).max(backoff),
1772 None => backoff,
1773 };
1774 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1775 // A wait of nothing spends none of the budget, so it is exhaustion rather
1776 // than a retry. `Pacing::resolve` rules out every way of configuring one
1777 // except a budget of zero, where reporting the first refusal is the ask.
1778 if wait.is_zero() || wait > remaining {
1779 return Err(limited.exhausted(
1780 doing,
1781 waits,
1782 waited,
1783 wait,
1784 self.pacing.retry_budget,
1785 ));
1786 }
1787 tokio::time::sleep(wait).await;
1788 waited += wait;
1789 waits += 1;
1790 backoff = backoff.saturating_mul(2);
1791 }
1792 }
1793
1794 /// The next moment a content-creating mutation may leave this source, as a wait from
1795 /// now.
1796 ///
1797 /// The slot is reserved under the lock and the waiting happens outside it, so two
1798 /// callers take two slots rather than the same one — and no lock is held across an
1799 /// await.
1800 ///
1801 /// The moment it is spaced from is the previous mutation's *completion*, which
1802 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1803 /// own is the wrong thing to measure from.
1804 fn reserve_mutation_slot(&self) -> Duration {
1805 if self.pacing.min_mutation_interval.is_zero() {
1806 return Duration::ZERO;
1807 }
1808 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1809 // it would turn an earlier failure into a second one for no gain.
1810 let mut last = self
1811 .last_mutation
1812 .lock()
1813 .unwrap_or_else(std::sync::PoisonError::into_inner);
1814 let now = Instant::now();
1815 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1816 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1817 let at = last.map_or(now, |previous| {
1818 previous
1819 .checked_add(self.pacing.min_mutation_interval)
1820 .map_or(now, |earliest| earliest.max(now))
1821 });
1822 *last = Some(at);
1823 at.saturating_duration_since(now)
1824 }
1825
1826 /// Record that a content-creating mutation has finished, so the next one is spaced
1827 /// from here rather than from the moment this one was released.
1828 ///
1829 /// This source can only choose when a request *departs*; the limiter counts when it
1830 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1831 /// departure from the last therefore hands the limiter a gap of the interval less that
1832 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1833 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1834 /// slower machine while passing on a quick one.
1835 ///
1836 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1837 /// previous request had already arrived before its response came back, so its arrival
1838 /// is no later than this moment, and the next mutation is released at least the
1839 /// interval after this moment and arrives no earlier than it is released: the gap the
1840 /// limiter measures is therefore at least the interval, whatever transit costs and on
1841 /// whatever platform. The price is that a mutation's own round trip no longer counts
1842 /// towards its spacing, which makes this source slightly slower than the configured
1843 /// rate rather than slightly faster — the safe side of a limit that punishes being
1844 /// wrong by refusing reads for the next fifty minutes.
1845 ///
1846 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1847 /// and one that never left costs only a wait nobody needed.
1848 fn finish_mutation(&self) {
1849 if self.pacing.min_mutation_interval.is_zero() {
1850 return;
1851 }
1852 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1853 let mut last = self
1854 .last_mutation
1855 .lock()
1856 .unwrap_or_else(std::sync::PoisonError::into_inner);
1857 let now = Instant::now();
1858 // `max` rather than an assignment: a concurrent caller may already have reserved a
1859 // slot further out, and completing this request must never pull that slot back in.
1860 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1861 }
1862
1863 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1864 /// failure that waiting cannot help — and recorded, whichever of the three it was.
1865 ///
1866 /// This is the one place a request leaves this crate, which is why the accounting is
1867 /// here rather than at each of the callers: a read path added later is counted without
1868 /// anybody remembering to count it, and
1869 /// `the_session_report_counts_every_request_the_board_served_and_what_each_cost` fails
1870 /// when one is not.
1871 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1872 let Attempted {
1873 result,
1874 limits,
1875 reported_cost,
1876 } = self.attempt(query, variables).await;
1877 // No `otherwise` name: every document this source sends is one of its own, and the
1878 // inventory gate on `graphql::DOCUMENTS` is what keeps that true.
1879 let sending = accounting::Request::graphql(query, variables, None, reported_cost);
1880 let outcome = match &result {
1881 Ok(_) => accounting::Outcome::Answered,
1882 Err(Attempt::Limited(_)) => accounting::Outcome::RateLimited,
1883 Err(Attempt::Failed(_)) => accounting::Outcome::Refused,
1884 };
1885 self.ledger.record(sending.finished(outcome, limits));
1886 result
1887 }
1888
1889 /// The attempt itself, with what its response said about the rate limit alongside.
1890 ///
1891 /// The two are returned together rather than recorded here because every one of the
1892 /// early exits below is a different outcome, and a record written at each of them is a
1893 /// record one of them can be added without.
1894 async fn attempt(&self, query: &str, variables: &Value) -> Attempted {
1895 let mut limits = accounting::RateLimit::default();
1896 let mut reported_cost = None;
1897 let result = self
1898 .attempted(query, variables, &mut limits, &mut reported_cost)
1899 .await;
1900 Attempted {
1901 result,
1902 limits,
1903 reported_cost,
1904 }
1905 }
1906
1907 /// One HTTP attempt, filling in what its response said about the rate limit as it goes.
1908 async fn attempted(
1909 &self,
1910 query: &str,
1911 variables: &Value,
1912 limits: &mut accounting::RateLimit,
1913 reported_cost: &mut Option<u64>,
1914 ) -> Result<Value, Attempt> {
1915 let response = self
1916 .client
1917 .post(self.endpoint.clone())
1918 .bearer_auth(self.token.expose_secret())
1919 .json(&json!({"query": query, "variables": variables}))
1920 .send()
1921 .await
1922 .map_err(|e| {
1923 Attempt::Failed(SourceError::Unavailable {
1924 message: format!("GitHub GraphQL request failed: {e}"),
1925 })
1926 })?;
1927 let status = response.status();
1928 let header = |name: &str| whole_seconds(response.headers().get(name));
1929 *limits = accounting::RateLimit::read(|name| {
1930 response
1931 .headers()
1932 .get(name)
1933 .and_then(|value| value.to_str().ok())
1934 .map(str::to_owned)
1935 });
1936 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
1937 // that are not text at all — is "not known to be exhausted". This never makes a
1938 // response a refusal on its own: it says which limiter a refusal is attributed to
1939 // and where its hint comes from, so a value this cannot read costs a hint rather
1940 // than an answer.
1941 let exhausted = response
1942 .headers()
1943 .get("x-ratelimit-remaining")
1944 .and_then(|value| value.to_str().ok())
1945 == Some("0");
1946 // `retry-after` is what GitHub asks for when it asks; when it does not and the
1947 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
1948 // which is the same question answered as an absolute time. Nothing else here is a
1949 // hint, and a schedule is what answers a refusal that carries none.
1950 let hint = header("retry-after").or_else(|| {
1951 exhausted
1952 .then(|| header("x-ratelimit-reset"))
1953 .flatten()
1954 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
1955 });
1956 // Read before it is parsed, because the evidence which tells a secondary rate
1957 // limit from a rejected credential is in the body of a response whose status says
1958 // only "forbidden" — and a non-success response was never parsed at all.
1959 let body = response.text().await.map_err(|e| {
1960 Attempt::Failed(SourceError::Unavailable {
1961 message: format!("GitHub GraphQL response could not be read: {e}"),
1962 })
1963 })?;
1964 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
1965 return Err(Attempt::Limited(Limited { limiter, hint }));
1966 }
1967 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
1968 return Err(Attempt::Failed(SourceError::Auth {
1969 message: format!(
1970 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
1971 ),
1972 }));
1973 }
1974 if !status.is_success() {
1975 return Err(Attempt::Failed(SourceError::Unavailable {
1976 message: format!("GitHub GraphQL returned HTTP {status}"),
1977 }));
1978 }
1979 // GitHub reports what a call cost only when the document asked it to, and no
1980 // document this source sends does — so this is `None` here and carries the figure
1981 // for a caller whose own document selects `rateLimit { cost }`. What it must never
1982 // pick up is a `dryRun` probe's cost, which is some other document's.
1983 *reported_cost = serde_json::from_str::<Value>(&body)
1984 .ok()
1985 .as_ref()
1986 .and_then(|body| body.pointer("/data/rateLimit/cost"))
1987 .and_then(Value::as_u64);
1988 self.answer(&body).map_err(Attempt::Failed)
1989 }
1990
1991 /// What one successful HTTP response says, once its GraphQL errors are read.
1992 fn answer(&self, body: &str) -> Result<Value, SourceError> {
1993 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
1994 message: format!("GitHub returned invalid JSON: {e}"),
1995 })?;
1996 let errors = body
1997 .get("errors")
1998 .map(|value| {
1999 value.as_array().ok_or_else(|| SourceError::Malformed {
2000 message: "GitHub response errors is not an array".into(),
2001 })
2002 })
2003 .transpose()?;
2004 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
2005 let messages = errors
2006 .iter()
2007 .filter_map(|e| e.get("message").and_then(Value::as_str))
2008 .collect::<Vec<_>>()
2009 .join("; ");
2010 let message = if messages.is_empty() {
2011 "GitHub returned GraphQL errors".into()
2012 } else {
2013 messages
2014 };
2015 let normalized = message.to_ascii_lowercase();
2016 if normalized.contains("resource not accessible") || normalized.contains("scope") {
2017 return Err(SourceError::Auth {
2018 message: format!(
2019 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
2020 self.credential_name
2021 ),
2022 });
2023 }
2024 return Err(SourceError::Refused { message });
2025 }
2026 body.get("data")
2027 .filter(|data| data.is_object())
2028 .cloned()
2029 .ok_or_else(|| SourceError::Malformed {
2030 message: "GitHub response has no data object".into(),
2031 })
2032 }
2033
2034 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
2035 // GraphQL cannot independently page them inside the outer item page. This source page is
2036 // deliberately bounded at that published maximum; the live drift journey exercises it.
2037 async fn board_page(
2038 &self,
2039 items_after: Option<&str>,
2040 items_first: u32,
2041 ) -> Result<Value, SourceError> {
2042 let data = self
2043 .graphql(
2044 graphql::BOARD,
2045 json!({"owner":self.owner,"number":self.project_number,
2046 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
2047 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
2048 )
2049 .await?;
2050 data.pointer("/owner/projectV2")
2051 .filter(|v| !v.is_null())
2052 .cloned()
2053 .ok_or_else(|| SourceError::Refused {
2054 message: format!(
2055 "GitHub project {}/{} was not found or is not visible to the token",
2056 self.owner, self.project_number
2057 ),
2058 })
2059 }
2060
2061 /// The search that finds the issues of this board, narrowed by `also` when it is
2062 /// given.
2063 ///
2064 /// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
2065 /// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
2066 /// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
2067 /// here on purpose — GitHub accepts it and silently ignores it, so a project is told
2068 /// from a task by the `parent` field each issue carries rather than by the search.
2069 fn board_search(&self, also: Option<&str>) -> String {
2070 let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
2071 match also {
2072 Some(also) => format!("{scope} {also}"),
2073 None => scope,
2074 }
2075 }
2076
2077 /// One issue this source reached directly, as the board item a read of the board would
2078 /// have produced — or `None` when this board does not hold it.
2079 ///
2080 /// The board half of an issue rides along on `Issue.projectItems`, so the value handed
2081 /// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
2082 /// item's own id, that item's field values, and the issue as its content. One resolver
2083 /// for both routes is what makes an issue read through a search, through its own node
2084 /// id, or through its project's sub-issues report the same title, the same status, the
2085 /// same labels and the same qualified id.
2086 ///
2087 /// An issue with no entry for *this* board is not this source's to report, which is
2088 /// what keeps an id naming some other repository's issue from being answered as an item
2089 /// of this board. That answer is given about an **exhausted** connection and never
2090 /// about an unread page: the entry is looked for on the page in hand, and only if that
2091 /// page reports more of the connection, in [`Self::board_membership`]'s walk of the
2092 /// rest of it.
2093 async fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
2094 if optional_str(issue, "__typename")? != Some("Issue") {
2095 return Ok(None);
2096 }
2097 let memberships = issue
2098 .get("projectItems")
2099 .ok_or_else(|| SourceError::Malformed {
2100 message: "GitHub issue is missing projectItems".into(),
2101 })?;
2102 let nodes = memberships
2103 .get("nodes")
2104 .and_then(Value::as_array)
2105 .ok_or_else(|| SourceError::Malformed {
2106 message: "GitHub issue projectItems.nodes is not an array".into(),
2107 })?;
2108 let held = match self.board_entry(nodes) {
2109 Some(held) => held.clone(),
2110 None => {
2111 let info = memberships
2112 .get("pageInfo")
2113 .ok_or_else(|| SourceError::Malformed {
2114 message: "GitHub issue projectItems has no pageInfo".into(),
2115 })?;
2116 // The page held no entry for this board. Whether that means the issue is
2117 // not on it is a question about the rest of the connection, and only a
2118 // connection with no rest answers it here.
2119 if !required_bool(info, "hasNextPage")? {
2120 return Ok(None);
2121 }
2122 let cursor = required_str(info, "endCursor")?;
2123 validate_cursor_progress(None, cursor)?;
2124 let issue_id = required_str(issue, "id")?;
2125 match self.board_membership(issue_id, cursor).await? {
2126 Some(held) => held,
2127 None => return Ok(None),
2128 }
2129 }
2130 };
2131 let item = json!({
2132 "id": required_str(&held, "id")?,
2133 "project": held.get("project"),
2134 "fieldValues": held.get("fieldValues"),
2135 "content": issue,
2136 });
2137 self.resolve(&item)
2138 }
2139
2140 /// This board's own entry among one page of an issue's `Issue.projectItems`.
2141 ///
2142 /// One spelling of *which membership is this board's*, so the page a read carries and
2143 /// the pages [`Self::board_membership`] walks are searched by the same rule.
2144 fn board_entry<'a>(&self, nodes: &'a [Value]) -> Option<&'a Value> {
2145 nodes.iter().find(|node| {
2146 node.pointer("/project/number").and_then(Value::as_u64)
2147 == Some(u64::from(self.project_number))
2148 })
2149 }
2150
2151 /// The rest of one issue's board memberships, from `after`, for this board's entry.
2152 ///
2153 /// The recovery read: a page of memberships that holds no entry for this board says
2154 /// nothing about the memberships past it, so the connection is walked to exhaustion
2155 /// before an issue is reported as one this board does not hold. `Ok(None)` is that
2156 /// positive answer — the whole connection was read and no entry named this board —
2157 /// rather than a failure, and the walk is held to
2158 /// [`validate_cursor_progress`] like every other page walk here, so a source answering
2159 /// with a cursor that does not advance is refused instead of spun on.
2160 async fn board_membership(
2161 &self,
2162 issue: &str,
2163 after: &str,
2164 ) -> Result<Option<Value>, SourceError> {
2165 let mut after = after.to_owned();
2166 loop {
2167 let data = self
2168 .graphql(
2169 graphql::ISSUE_BOARD_ITEMS,
2170 json!({"id":issue,"first":MAX_PAGE_SIZE,"after":after,
2171 "nestedFirst":NESTED_PAGE_SIZE}),
2172 )
2173 .await?;
2174 let Some(connection) = data
2175 .pointer("/node/projectItems")
2176 .filter(|value| !value.is_null())
2177 else {
2178 // The id resolved to nothing, or to something with no memberships to walk —
2179 // which is the same answer as a connection holding no entry for this board.
2180 return Ok(None);
2181 };
2182 let nodes = connection
2183 .get("nodes")
2184 .and_then(Value::as_array)
2185 .ok_or_else(|| SourceError::Malformed {
2186 message: "GitHub issue projectItems.nodes is not an array".into(),
2187 })?;
2188 if let Some(held) = self.board_entry(nodes) {
2189 return Ok(Some(held.clone()));
2190 }
2191 let info = connection
2192 .get("pageInfo")
2193 .ok_or_else(|| SourceError::Malformed {
2194 message: "GitHub issue projectItems has no pageInfo".into(),
2195 })?;
2196 let next = required_bool(info, "hasNextPage")?
2197 .then(|| required_str(info, "endCursor"))
2198 .transpose()?;
2199 match next {
2200 Some(next) => {
2201 validate_cursor_progress(Some(&after), next)?;
2202 after = next.to_owned();
2203 }
2204 None => return Ok(None),
2205 }
2206 }
2207 }
2208
2209 /// One page of a board-scoped issue search, and where the next page resumes.
2210 async fn search_page(
2211 &self,
2212 search: &str,
2213 first: u32,
2214 after: Option<&str>,
2215 ) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
2216 let data = self
2217 .graphql(
2218 graphql::SEARCH_ISSUES,
2219 json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
2220 "after":after,"nestedFirst":NESTED_PAGE_SIZE,
2221 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2222 )
2223 .await?;
2224 let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
2225 message: "GitHub search response has no search connection".into(),
2226 })?;
2227 let mut found = Vec::new();
2228 for node in connection
2229 .get("nodes")
2230 .and_then(Value::as_array)
2231 .ok_or_else(|| SourceError::Malformed {
2232 message: "GitHub search nodes is not an array".into(),
2233 })?
2234 {
2235 if let Some(resolved) = self.resolve_issue(node).await? {
2236 found.push(resolved);
2237 }
2238 }
2239 let info = connection
2240 .get("pageInfo")
2241 .ok_or_else(|| SourceError::Malformed {
2242 message: "GitHub search connection has no pageInfo".into(),
2243 })?;
2244 let next = required_bool(info, "hasNextPage")?
2245 .then(|| required_str(info, "endCursor"))
2246 .transpose()?
2247 .map(str::to_owned);
2248 if let Some(next) = &next {
2249 validate_cursor_progress(after, next)?;
2250 }
2251 Ok((found, next))
2252 }
2253
2254 /// Every issue this board holds, walked to exhaustion, completed with what this run
2255 /// wrote.
2256 ///
2257 /// The completion is not an optimisation and it is not a cache: GitHub's issue search
2258 /// is an index and is eventually consistent, so an issue this run created seconds ago
2259 /// is routinely absent from it, and a project listed straight after being written would
2260 /// otherwise be missing from its own board. What is added back is only what this
2261 /// process itself wrote, out of [`Self::created`], which lives and dies with the
2262 /// process.
2263 async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
2264 let mut after: Option<String> = None;
2265 let mut found = Vec::new();
2266 let search = self.board_search(None);
2267 loop {
2268 let (page, next) = self
2269 .search_page(&search, MAX_PAGE_SIZE, after.as_deref())
2270 .await?;
2271 found.extend(page);
2272 match next {
2273 Some(next) => after = Some(next),
2274 None => break,
2275 }
2276 }
2277 self.completed_with_written(found, |_| true)
2278 }
2279
2280 /// `found`, with everything this run wrote that `keep` accepts and the read did not
2281 /// report.
2282 ///
2283 /// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
2284 /// at all: the search index is behind, and a node read of an item filed moments ago can
2285 /// be too.
2286 fn completed_with_written(
2287 &self,
2288 mut found: Vec<Resolved>,
2289 keep: impl Fn(&Resolved) -> bool,
2290 ) -> Result<Vec<Resolved>, SourceError> {
2291 for own in self.created()?.iter().filter(|own| keep(own)) {
2292 if !found.iter().any(|item| item.id == own.id) {
2293 found.push(own.clone());
2294 }
2295 }
2296 Ok(found)
2297 }
2298
2299 /// What resolving one node id reached.
2300 ///
2301 /// Three answers rather than an `Option`, because a board *draft* is none of the other
2302 /// two: it is not an issue, it has no node of its own this source can read the board
2303 /// half off, and its only home is the board's own item connection — so a read of one
2304 /// is completed from there rather than reported as nothing.
2305 async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
2306 let asked = self
2307 .graphql(
2308 graphql::ISSUE,
2309 json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
2310 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2311 )
2312 .await;
2313 let data = match asked {
2314 Ok(data) => data,
2315 // A string that is not a node id at all is not a failure to report: it is an id
2316 // this board does not hold, which is what every read of one already answers.
2317 Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
2318 Err(error) => return Err(error),
2319 };
2320 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
2321 return Ok(Reached::Nothing);
2322 };
2323 if optional_str(node, "__typename")? == Some("DraftIssue") {
2324 return Ok(Reached::Draft);
2325 }
2326 Ok(match self.resolve_issue(node).await? {
2327 Some(item) => Reached::Held(Box::new(item)),
2328 None => Reached::Nothing,
2329 })
2330 }
2331
2332 /// One item of this board by its own id, whatever kind it is.
2333 ///
2334 /// Resolved from the identifier alone: no search, board-wide or otherwise. What this
2335 /// run wrote is read first, because a node read of an item created moments ago can
2336 /// still be behind the board field values written onto it — see [`Self::created`].
2337 async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
2338 if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
2339 return Ok(Some(own.clone()));
2340 }
2341 match self.reach(id).await? {
2342 Reached::Held(item) => Ok(Some(*item)),
2343 Reached::Nothing => Ok(None),
2344 // The one read that still costs the board: a draft lives nowhere else.
2345 Reached::Draft => Ok(self
2346 .board()
2347 .await?
2348 .items
2349 .into_iter()
2350 .find(|item| item.id == *id)),
2351 }
2352 }
2353
2354 /// Everything filed under one issue of this board, walked to exhaustion — or `None`
2355 /// when that id names nothing here with a sub-issue relationship to walk.
2356 ///
2357 /// `None` and an empty answer are different: `None` is *this is not an issue of this
2358 /// GitHub*, which is what sends a project selector on to be read as a name, and an
2359 /// empty vector is a project that holds nothing.
2360 async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
2361 let mut after: Option<String> = None;
2362 let mut children = Vec::new();
2363 loop {
2364 let asked = self
2365 .graphql(
2366 graphql::SUB_ISSUES,
2367 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
2368 "nestedFirst":NESTED_PAGE_SIZE,
2369 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2370 )
2371 .await;
2372 let data = match asked {
2373 Ok(data) => data,
2374 // A string that is not a node id at all is not a failure to report: it is
2375 // the ordinary answer to a selector naming a project by its name.
2376 Err(error) if unresolvable_node(&error) => return Ok(None),
2377 Err(error) => return Err(error),
2378 };
2379 let Some(connection) = data
2380 .pointer("/node/subIssues")
2381 .filter(|value| !value.is_null())
2382 else {
2383 // No such node, or one with no sub-issue relationship — a board draft is
2384 // the one this board can really hold.
2385 return Ok(None);
2386 };
2387 for node in connection
2388 .get("nodes")
2389 .and_then(Value::as_array)
2390 .ok_or_else(|| SourceError::Malformed {
2391 message: "GitHub subIssues.nodes is not an array".into(),
2392 })?
2393 {
2394 if let Some(resolved) = self.resolve_issue(node).await? {
2395 children.push(resolved);
2396 }
2397 }
2398 let info = connection
2399 .get("pageInfo")
2400 .ok_or_else(|| SourceError::Malformed {
2401 message: "GitHub subIssues connection has no pageInfo".into(),
2402 })?;
2403 let next = required_bool(info, "hasNextPage")?
2404 .then(|| required_str(info, "endCursor"))
2405 .transpose()?;
2406 match next {
2407 Some(next) => {
2408 validate_cursor_progress(after.as_deref(), next)?;
2409 after = Some(next.to_owned());
2410 }
2411 None => return Ok(Some(children)),
2412 }
2413 }
2414 }
2415
2416 /// Which issue of this board a project *name* is, or `None` when none is.
2417 ///
2418 /// One bounded query which filters on that name at the server, rather than a walk of
2419 /// every issue the board holds. The name is compared again here: the qualifier narrows
2420 /// what GitHub sends, and this source decides what it names.
2421 async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
2422 let search = self.board_search(Some(&title_qualifier(name)));
2423 let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
2424 Ok(candidates
2425 .into_iter()
2426 .find(|item| {
2427 item.kind == BoardKind::Work(ItemKind::Project)
2428 && item.title.eq_ignore_ascii_case(name)
2429 })
2430 .map(|item| item.id))
2431 }
2432
2433 /// Everything filed under one project of this board: the sub-issues of the issue that
2434 /// project is.
2435 ///
2436 /// Tasks *and* documents, because a document filed under a project is a sub-issue of it
2437 /// too — the caller keeps the kind it asked for. Nothing about this grows as the board
2438 /// gains projects, or as another project gains tasks.
2439 ///
2440 /// A qualified id names the issue and is asked for its sub-issues directly: one
2441 /// request, no search of any kind. Only a selector GitHub cannot resolve that way is
2442 /// read as a project *name*, which costs the one bounded search
2443 /// [`Self::project_by_name`] makes.
2444 async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
2445 let (project, children) = match self.sub_issues(selector).await? {
2446 Some(children) => (selector.clone(), children),
2447 None => match self.project_by_name(&selector.0).await? {
2448 Some(project) => {
2449 let children = self.sub_issues(&project).await?.unwrap_or_default();
2450 (project, children)
2451 }
2452 None => return Ok(Vec::new()),
2453 },
2454 };
2455 self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
2456 }
2457
2458 /// Every item on the board, with the one board identity they all share.
2459 ///
2460 /// See [`Self::board_cache`]. The completion from `created` happens on every call
2461 /// rather than once, which is what the cache could otherwise have broken.
2462 async fn board(&self) -> Result<Board, SourceError> {
2463 let cached = self.board_cache()?.clone();
2464 let mut board = match cached {
2465 Some(board) => board,
2466 None => {
2467 let read = self.read_board().await?;
2468 *self.board_cache()? = Some(read.clone());
2469 read
2470 }
2471 };
2472 for own in self.created()?.iter() {
2473 if !board.items.iter().any(|item| item.id == own.id) {
2474 board.items.push(own.clone());
2475 }
2476 }
2477 Ok(board)
2478 }
2479
2480 /// This process's own view of the board, or the refusal a poisoned lock is.
2481 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
2482 self.board_cache
2483 .lock()
2484 .map_err(|_| SourceError::Unavailable {
2485 message: "this source's view of the board was left inconsistent by an earlier \
2486 failure; next: run the command again"
2487 .into(),
2488 })
2489 }
2490
2491 /// Bring this process's own view of the board up to an item it has just written.
2492 ///
2493 /// A created item goes to `created`, which is what completes a board read GitHub's own
2494 /// eventual consistency has left behind. An item that was already there is replaced
2495 /// where it sits, so a second write of it in the same command reads its real parent
2496 /// rather than the one it had before the first write.
2497 ///
2498 /// "Where it sits" is two places, and missing the first leaves a stale record that
2499 /// wins: an item this same run created is held in `created` and not in the cached
2500 /// board, and `board` completes the cached board *from* `created`, so replacing only
2501 /// the cached copy of such an item replaces nothing and the read still reports the
2502 /// title it was created with.
2503 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
2504 if created {
2505 self.created()?.push(item);
2506 return Ok(());
2507 }
2508 {
2509 let mut own = self.created()?;
2510 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
2511 *held = item;
2512 return Ok(());
2513 }
2514 }
2515 if let Some(board) = self.board_cache()?.as_mut()
2516 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
2517 {
2518 *held = item;
2519 }
2520 Ok(())
2521 }
2522
2523 /// Forget one item this process has just deleted, from both halves of its own view.
2524 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
2525 self.created()?.retain(|own| own.id != *id);
2526 if let Some(board) = self.board_cache()?.as_mut() {
2527 board.items.retain(|item| item.id != *id);
2528 }
2529 Ok(())
2530 }
2531
2532 /// Every page of the board, read from GitHub.
2533 async fn read_board(&self) -> Result<Board, SourceError> {
2534 let mut after: Option<String> = None;
2535 let mut items = Vec::new();
2536 let mut board;
2537 loop {
2538 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
2539 for item in page
2540 .pointer("/items/nodes")
2541 .and_then(Value::as_array)
2542 .ok_or_else(|| SourceError::Malformed {
2543 message: "GitHub project items.nodes is not an array".into(),
2544 })?
2545 {
2546 if let Some(resolved) = self.resolve(item)? {
2547 items.push(resolved);
2548 }
2549 }
2550 let info = page
2551 .pointer("/items/pageInfo")
2552 .ok_or_else(|| SourceError::Malformed {
2553 message: "GitHub project items have no pageInfo".into(),
2554 })?;
2555 let has_next = required_bool(info, "hasNextPage")?;
2556 let next = has_next
2557 .then(|| required_str(info, "endCursor"))
2558 .transpose()?;
2559 board = page.clone();
2560 match next {
2561 Some(next) => {
2562 validate_cursor_progress(after.as_deref(), next)?;
2563 after = Some(next.to_owned());
2564 }
2565 None => break,
2566 }
2567 }
2568 Ok(Board {
2569 id: required_str(&board, "id")?.to_owned(),
2570 fields: board.get("fields").cloned().unwrap_or(Value::Null),
2571 items,
2572 })
2573 }
2574
2575 /// The items this source has created, for completing a board read that is behind.
2576 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
2577 self.created.lock().map_err(|_| SourceError::Unavailable {
2578 message: "this source's record of what it created in this run was left \
2579 inconsistent by an earlier failure; next: run the command again"
2580 .into(),
2581 })
2582 }
2583
2584 /// One board item as this source reports it, or `None` for content it ignores.
2585 ///
2586 /// A pull request is neither a project nor a task — it is somebody's change, not a
2587 /// unit of plan — and an item whose content the token cannot see has nothing to
2588 /// report at all.
2589 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
2590 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
2591 message: "GitHub project item is missing content".into(),
2592 })?;
2593 if content.is_null() {
2594 return Ok(None);
2595 }
2596 let content_kind = match required_str(content, "__typename")? {
2597 "Issue" => ContentKind::Issue,
2598 "DraftIssue" => ContentKind::DraftIssue,
2599 _ => return Ok(None),
2600 };
2601 let field_values = item
2602 .get("fieldValues")
2603 .ok_or_else(|| SourceError::Malformed {
2604 message: "GitHub project item is missing fieldValues".into(),
2605 })?;
2606 complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
2607 let nodes = field_values
2608 .get("nodes")
2609 .and_then(Value::as_array)
2610 .ok_or_else(|| SourceError::Malformed {
2611 message: "GitHub project item fieldValues.nodes is not an array".into(),
2612 })?;
2613 if let Some(labels) = content.get("labels") {
2614 complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
2615 }
2616 let (body, slot) = metadata_body(optional_str(content, "body")?.map(str::to_owned))?;
2617 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
2618 .map(|id| NativeId(id.to_owned()));
2619 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
2620 // to read one from; it is a task, and never a project.
2621 let sub_issues = match content_kind {
2622 ContentKind::Issue => sub_issue_total(content)?,
2623 ContentKind::DraftIssue => 0,
2624 };
2625 let content_id = required_str(content, "id")?;
2626 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
2627 message: format!("GitHub issue {content_id}: {message}"),
2628 })?;
2629 let raw_title = required_str(content, "title")?;
2630 // The design prefix is read *first*, before either of the two rules that separate
2631 // a project from a task. A document is not work whatever sub-issues it has and
2632 // whatever marker it carries, and reading the prefix later would make a design
2633 // issue with none of either an empty project.
2634 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
2635 BoardKind::Document
2636 } else if parent.is_some() {
2637 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
2638 // under a project is that project's task even when it has sub-issues of its
2639 // own.
2640 BoardKind::Work(ItemKind::Task)
2641 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
2642 BoardKind::Work(ItemKind::Project)
2643 } else {
2644 BoardKind::Work(ItemKind::Task)
2645 };
2646 // The title a person wrote, which for a document is the one without the prefix —
2647 // the same way `content` above is the body without this source's metadata slot.
2648 let title = match kind {
2649 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
2650 BoardKind::Work(_) => raw_title.to_owned(),
2651 };
2652 let own_repository = content
2653 .pointer("/repository/nameWithOwner")
2654 .and_then(Value::as_str)
2655 .map(|origin| Repository::try_from(format!("{}/{origin}", RepositoryTarget::HOST)))
2656 .transpose()
2657 .map_err(|message| SourceError::Malformed { message })?;
2658 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
2659 Repository::from_metadata(&slot)
2660 .map_err(|message| SourceError::Malformed { message })?
2661 } else {
2662 own_repository.clone().into_iter().collect()
2663 };
2664 Ok(Some(Resolved {
2665 item_id: required_str(item, "id")?.to_owned(),
2666 id: NativeId(content_id.to_owned()),
2667 content_kind,
2668 kind,
2669 title,
2670 body: body.filter(|value| !value.is_empty()),
2671 status: self.status(item, content)?,
2672 labels: labels(content)?,
2673 parent,
2674 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
2675 url: optional_str(content, "url")?.map(str::to_owned),
2676 created_at: optional_time(content, "createdAt")?,
2677 updated_at: optional_time(content, "updatedAt")?,
2678 own_repository,
2679 repositories,
2680 slot,
2681 // Present when the item was reached through its own issue, whose board entry
2682 // names the board; a read of the board's own items has the board already. An
2683 // empty id names nothing a field write could address, so it is read as absent and
2684 // the write goes back to reading the board.
2685 board_id: item
2686 .pointer("/project/id")
2687 .and_then(Value::as_str)
2688 .filter(|id| !id.is_empty())
2689 .map(str::to_owned),
2690 fields: field_definitions(nodes),
2691 }))
2692 }
2693
2694 /// The status one board item reports.
2695 ///
2696 /// The closed state decides the category and the `Status` option decides the name, so
2697 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
2698 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
2699 /// duplicate is not finished work, and calling it done is a lie the next copy would
2700 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
2701 /// it is read permissively rather than refused — reads are faithful, and refusals
2702 /// belong on writes.
2703 fn status(&self, item: &Value, content: &Value) -> Result<Status, SourceError> {
2704 let nodes = item
2705 .pointer("/fieldValues/nodes")
2706 .and_then(Value::as_array)
2707 .expect("resolve validates fieldValues.nodes before mapping status");
2708 let option = nodes
2709 .iter()
2710 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
2711 .map(|value| required_str(value, "name"))
2712 .transpose()?;
2713 let state = optional_str(content, "state")?;
2714 if state == Some("CLOSED") {
2715 let category = match optional_str(content, "stateReason")? {
2716 None | Some("COMPLETED") => StatusCategory::Done,
2717 Some("NOT_PLANNED") => StatusCategory::Cancelled,
2718 Some(_) => StatusCategory::Unknown,
2719 };
2720 let fallback = match category {
2721 StatusCategory::Done => "Done",
2722 StatusCategory::Cancelled => "Cancelled",
2723 _ => "Closed",
2724 };
2725 return Ok(Status {
2726 category,
2727 name: option.unwrap_or(fallback).to_owned(),
2728 });
2729 }
2730 let name = option.unwrap_or("Open").to_owned();
2731 Ok(Status {
2732 category: self
2733 .statuses
2734 .category_of(&name)
2735 .unwrap_or(StatusCategory::Unknown),
2736 name,
2737 })
2738 }
2739
2740 /// The board Status option this write selects, or the refusal that says why not.
2741 ///
2742 /// For a column target the option is what the status *is*, so a board that has no such
2743 /// option is a refusal naming the status and the instance. For a closed target the
2744 /// issue's own state carries the category, and the option carries only the name a
2745 /// reader reports — so an option spelled the way this status is spelled is selected
2746 /// when the board has one, and nothing is refused when it does not.
2747 fn column_for(
2748 &self,
2749 board: &Board,
2750 status: &Status,
2751 target: &StatusTarget,
2752 ) -> Result<Option<(String, String)>, SourceError> {
2753 let (wanted, required) = match target {
2754 StatusTarget::Column(wanted) => (wanted.as_str(), true),
2755 StatusTarget::Closed(_) => (status.name.as_str(), false),
2756 StatusTarget::Disabled => return Ok(None),
2757 };
2758 let missing = |detail: &str| SourceError::Refused {
2759 message: format!(
2760 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
2761 category_name(status.category),
2762 self.name,
2763 category_name(status.category)
2764 ),
2765 };
2766 let Some(field) = Board::field(&board.fields, "Status")? else {
2767 return if required {
2768 Err(missing("this board has no Status field"))
2769 } else {
2770 Ok(None)
2771 };
2772 };
2773 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
2774 return if required {
2775 Err(missing(
2776 "this board's Status field is not a single-select field",
2777 ))
2778 } else {
2779 Ok(None)
2780 };
2781 }
2782 let option = field
2783 .get("options")
2784 .and_then(Value::as_array)
2785 .and_then(|options| {
2786 options.iter().find(|option| {
2787 option
2788 .get("name")
2789 .and_then(Value::as_str)
2790 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
2791 })
2792 });
2793 match option {
2794 None if required => Err(missing("this board does not have it")),
2795 None => Ok(None),
2796 Some(option) => Ok(Some((
2797 required_str(field, "id")?.to_owned(),
2798 required_str(option, "id")?.to_owned(),
2799 ))),
2800 }
2801 }
2802
2803 /// This instance's target for a category, refusing one it has disabled.
2804 ///
2805 /// Nothing here mutates the board's option set to make room for a status. GitHub
2806 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
2807 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
2808 /// field and every item's status.
2809 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
2810 let target = self.statuses.target(category).clone();
2811 if target != StatusTarget::Disabled {
2812 return Ok(target);
2813 }
2814 Err(SourceError::Refused {
2815 message: if category == StatusCategory::Draft {
2816 format!(
2817 "status draft is disabled for source {}: draft is incompatible with this \
2818 integration because GitHub draft issues cannot have sub-issues, and this \
2819 source stores a project's tasks as its issue's sub-issues",
2820 self.name
2821 )
2822 } else if category == StatusCategory::Unknown {
2823 format!(
2824 "status {} is disabled for source {}; set status_mapping.{} of this source \
2825 to one board Status option name; every word classified unknown is written \
2826 to that one option",
2827 category_name(category),
2828 self.name,
2829 category_name(category)
2830 )
2831 } else {
2832 format!(
2833 "status {} is disabled for source {}; set status_mapping.{} of this source \
2834 to a board Status option name or to a closed state",
2835 category_name(category),
2836 self.name,
2837 category_name(category)
2838 )
2839 },
2840 })
2841 }
2842
2843 async fn set_item_field(
2844 &self,
2845 board_id: &str,
2846 item_id: &str,
2847 field_id: &str,
2848 value: Value,
2849 ) -> Result<(), SourceError> {
2850 let data = self
2851 .graphql(
2852 graphql::UPDATE_FIELD,
2853 json!({"input":{
2854 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
2855 }}),
2856 )
2857 .await?;
2858 let returned = data
2859 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
2860 .ok_or_else(|| SourceError::Malformed {
2861 message: "GitHub field update returned no project item".into(),
2862 })?;
2863 if required_str(returned, "id")? != item_id {
2864 return Err(SourceError::Malformed {
2865 message: "GitHub field update returned the wrong project item".into(),
2866 });
2867 }
2868 Ok(())
2869 }
2870
2871 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
2872 let mut after: Option<String> = None;
2873 let mut ids = Vec::new();
2874 loop {
2875 let data = self
2876 .graphql(
2877 graphql::ISSUE_DEPENDENCIES,
2878 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
2879 )
2880 .await?;
2881 let connection =
2882 data.pointer("/node/blockedBy")
2883 .ok_or_else(|| SourceError::Malformed {
2884 message: "GitHub dependency response has no blockedBy connection".into(),
2885 })?;
2886 ids.extend(
2887 connection
2888 .get("nodes")
2889 .and_then(Value::as_array)
2890 .ok_or_else(|| SourceError::Malformed {
2891 message: "GitHub dependency response nodes is not an array".into(),
2892 })?
2893 .iter()
2894 .map(|value| required_str(value, "id").map(str::to_owned))
2895 .collect::<Result<Vec<_>, _>>()?,
2896 );
2897 let next = next_cursor(connection)?;
2898 if let Some(next) = &next {
2899 validate_cursor_progress(after.as_deref(), &next.0)?;
2900 }
2901 after = next.map(|cursor| cursor.0);
2902 if after.is_none() {
2903 return Ok(ids);
2904 }
2905 }
2906 }
2907
2908 async fn dependencies(
2909 &self,
2910 id: &NativeId,
2911 near_kind: ItemKind,
2912 direction: Direction,
2913 page: &PageRequest,
2914 ) -> Result<Page<DependencyEdge>, SourceError> {
2915 validate_page(page)?;
2916 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
2917 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
2918 let recorded = recorded_offset(cursor, direction)?;
2919 // Asked for even in the recorded phase, whose page reads nothing from the
2920 // connection: `__typename` is what says whether this item has a native
2921 // relationship at all, and that is what decides which far ends the reserved key is
2922 // allowed to hold.
2923 let data = self
2924 .graphql(
2925 graphql::ISSUE_DEPENDENCIES,
2926 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
2927 "after":if recorded.is_some() {None} else {cursor}}),
2928 )
2929 .await?;
2930 let node =
2931 data.get("node")
2932 .filter(|v| !v.is_null())
2933 .ok_or_else(|| SourceError::Refused {
2934 message: format!(
2935 "GitHub item {} was not found or does not support dependencies",
2936 id.0
2937 ),
2938 })?;
2939 let connection_name = match direction {
2940 Direction::DependsOn => "blockedBy",
2941 Direction::DependedOnBy => "blocking",
2942 };
2943 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
2944 // named natively and the reserved key may hold any far end. An issue's connections
2945 // hold issues, and this source reads them at the near item's own level.
2946 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
2947 if let Some(offset) = recorded {
2948 return Ok(recorded_page(
2949 self.recorded_edges(id, near_kind, direction, natively_names, node)
2950 .await?,
2951 offset,
2952 limit,
2953 ));
2954 }
2955 if natively_names.is_none() {
2956 return Ok(recorded_page(
2957 self.recorded_edges(id, near_kind, direction, natively_names, node)
2958 .await?,
2959 0,
2960 limit,
2961 ));
2962 }
2963 let connection = node
2964 .get(connection_name)
2965 .ok_or_else(|| SourceError::Malformed {
2966 message: "GitHub dependency response is missing its connection".into(),
2967 })?;
2968 let nodes = connection
2969 .get("nodes")
2970 .and_then(Value::as_array)
2971 .ok_or_else(|| SourceError::Malformed {
2972 message: "GitHub dependency response nodes is not an array".into(),
2973 })?;
2974 // `from` depends on `to`, always. GitHub spells the same relationship from either
2975 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
2976 // it — so the near item is `from` in one direction and `to` in the other.
2977 let items = nodes
2978 .iter()
2979 .map(|value| {
2980 let related = NativeId(required_str(value, "id")?.into());
2981 let related_kind = related_kind(value)?;
2982 let (from, to) = match direction {
2983 Direction::DependsOn => (
2984 DependencyEndpoint::from_native(id.clone(), near_kind),
2985 DependencyEndpoint::from_native(related, related_kind),
2986 ),
2987 Direction::DependedOnBy => (
2988 DependencyEndpoint::from_native(related, related_kind),
2989 DependencyEndpoint::from_native(id.clone(), near_kind),
2990 ),
2991 };
2992 Ok(DependencyEdge {
2993 from,
2994 to,
2995 kind: DependencyKind::Blocks,
2996 })
2997 })
2998 .collect::<Result<Vec<_>, SourceError>>()?;
2999 let mut next = next_cursor(connection)?;
3000 if let Some(next) = &next {
3001 validate_cursor_progress(cursor, &next.0)?;
3002 }
3003 if next.is_none()
3004 && !self
3005 .recorded_edges(id, near_kind, direction, natively_names, node)
3006 .await?
3007 .is_empty()
3008 {
3009 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
3010 }
3011 Ok(Page { items, next })
3012 }
3013
3014 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
3015 /// a far end in another source has to live: no GitHub issue relationship can name one.
3016 ///
3017 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
3018 /// source never writes one down.
3019 ///
3020 /// The metadata lives in the item's own body slot, and `node` is the dependency read's
3021 /// own answer, which carries an issue's body — so an issue's recorded edges cost no
3022 /// request beyond the read already made, and reading the board for them would be a
3023 /// walk of every item for one field of one. A draft has no body in that answer, because
3024 /// a draft lives only inside the board, so a draft's are read off the board as before.
3025 async fn recorded_edges(
3026 &self,
3027 id: &NativeId,
3028 near_kind: ItemKind,
3029 direction: Direction,
3030 natively_names: Option<ItemKind>,
3031 node: &Value,
3032 ) -> Result<Vec<DependencyEdge>, SourceError> {
3033 if direction != Direction::DependsOn {
3034 return Ok(Vec::new());
3035 }
3036 let slot = match node.get("body") {
3037 Some(body) if natively_names.is_some() => {
3038 metadata_body(body.as_str().map(str::to_owned))?.1
3039 }
3040 _ => {
3041 let Some(item) = self
3042 .board()
3043 .await?
3044 .items
3045 .into_iter()
3046 .find(|item| item.id == *id)
3047 else {
3048 return Ok(Vec::new());
3049 };
3050 item.slot
3051 }
3052 };
3053 DependencyEdge::recorded(&slot, id, near_kind, &self.name, natively_names)
3054 .map_err(|message| SourceError::Malformed { message })
3055 }
3056
3057 fn configured_repository(&self) -> Result<&RepositoryTarget, SourceError> {
3058 self.repository
3059 .as_ref()
3060 .ok_or_else(|| SourceError::Refused {
3061 message: format!(
3062 "source {} has no repository configured, and a GitHub Projects board has no \
3063 repository of its own to create an issue in; set repository: owner/name on \
3064 this source",
3065 self.name
3066 ),
3067 })
3068 }
3069
3070 /// The repository one new issue is created in, under the rule [`RepositoryTarget`]
3071 /// states.
3072 ///
3073 /// The fallback is demanded first, whichever arm answers: a write without a configured
3074 /// repository is refused naming the field exactly as it was before the rule existed,
3075 /// so a source that could not write before cannot write now, rather than writing for
3076 /// the one item whose own field happens to decide it.
3077 ///
3078 /// Everything this refuses is refused before `createIssue`, so a refusal leaves no
3079 /// issue behind: an entry that is not a repository on [`RepositoryTarget::HOST`], an
3080 /// entry owned by someone other than the owner of the parent issue's repository —
3081 /// GitHub accepts a sub-issue from another repository of the same owner and from no
3082 /// other, so `addSubIssue` would refuse it after the issue existed — a parent the
3083 /// board does not hold, and a parent that is a draft, which GitHub gives no sub-issues,
3084 /// both of which `addSubIssue` would likewise refuse too late. Whether the entry exists
3085 /// and is visible to the token is checked where its node id is resolved, still before
3086 /// `createIssue`. The parent is read off `board`, which is completed from
3087 /// this process's own record, so a project created moments ago in this command answers
3088 /// though GitHub's board read has not caught up.
3089 fn creation_target(
3090 &self,
3091 board: &Board,
3092 incoming: &Incoming<'_>,
3093 ) -> Result<RepositoryTarget, SourceError> {
3094 let fallback = self.configured_repository()?;
3095 let what = |incoming: &Incoming<'_>| {
3096 format!(
3097 "{} {:?}",
3098 incoming.written.kind().describes(),
3099 incoming.title
3100 )
3101 };
3102 let parent = incoming
3103 .parent
3104 .map(|parent| {
3105 board
3106 .items
3107 .iter()
3108 .find(|item| item.id == *parent)
3109 .ok_or_else(|| SourceError::Refused {
3110 message: format!(
3111 "GitHub project issue {} was not found on the board of source {}, \
3112 so {} cannot be filed under it",
3113 parent.0,
3114 self.name,
3115 what(incoming)
3116 ),
3117 })
3118 })
3119 .transpose()?;
3120 let parents_repository = parent
3121 .map(|parent| {
3122 // A draft is on the board and so is found, but it has no repository to
3123 // place a task in and GitHub gives it no sub-issues, so `addSubIssue`
3124 // would refuse the task only once `createIssue` had made it.
3125 if parent.content_kind == ContentKind::DraftIssue {
3126 return Err(SourceError::Refused {
3127 message: format!(
3128 "GitHub project item {} on the board of source {} is a draft, \
3129 which cannot have sub-issues, so {} cannot be filed under it",
3130 parent.id.0,
3131 self.name,
3132 what(incoming)
3133 ),
3134 });
3135 }
3136 // An issue's repository is where a sub-issue is placed and whose owner it
3137 // is compared against, so a parent whose repository this source cannot
3138 // spell as `owner/name` — GitHub's login grammar is wider than this
3139 // source's floor — is one nothing can be filed under.
3140 parent
3141 .own_repository
3142 .as_ref()
3143 .and_then(|origin| RepositoryTarget::from_origin(origin).ok())
3144 .ok_or_else(|| SourceError::Malformed {
3145 message: format!(
3146 "GitHub project issue {} on the board of source {} is in {}, which \
3147 is not a {}/owner/name repository this source can place {} in",
3148 parent.id.0,
3149 self.name,
3150 parent
3151 .own_repository
3152 .as_ref()
3153 .map_or("no repository", Repository::as_str),
3154 RepositoryTarget::HOST,
3155 what(incoming)
3156 ),
3157 })
3158 })
3159 .transpose()?;
3160 match incoming.repositories {
3161 [named] => {
3162 let target =
3163 RepositoryTarget::from_origin(named).map_err(|_| SourceError::Refused {
3164 message: format!(
3165 "{} names repository {}, which is not a {}/owner/name repository \
3166 source {} can create an issue in; name one that is, or name none",
3167 what(incoming),
3168 named.as_str(),
3169 RepositoryTarget::HOST,
3170 self.name
3171 ),
3172 })?;
3173 if let Some(parents) = &parents_repository
3174 && parents.owner != target.owner
3175 {
3176 return Err(SourceError::Refused {
3177 message: format!(
3178 "{} names repository {}, owned by {}, but its project's issue is in \
3179 {}, owned by {}, and GitHub files a sub-issue only in a repository \
3180 of the same owner as its parent issue; name a repository of {}, or \
3181 name none",
3182 what(incoming),
3183 target.slug(),
3184 target.owner,
3185 parents.slug(),
3186 parents.owner,
3187 parents.owner
3188 ),
3189 });
3190 }
3191 Ok(target)
3192 }
3193 _ => Ok(parents_repository.unwrap_or_else(|| fallback.clone())),
3194 }
3195 }
3196
3197 /// The node id of the repository `incoming` is being created in, or the refusal naming
3198 /// the item and the repository the token cannot see.
3199 ///
3200 /// Resolved once per command per repository; see [`Self::repository_cache`].
3201 async fn repository_id(
3202 &self,
3203 repository: &RepositoryTarget,
3204 incoming: &Incoming<'_>,
3205 ) -> Result<String, SourceError> {
3206 if let Some(id) = self.repository_cache()?.get(repository).cloned() {
3207 return Ok(id);
3208 }
3209 let data = self
3210 .graphql(
3211 graphql::REPOSITORY,
3212 json!({"owner":repository.owner,"name":repository.name}),
3213 )
3214 .await?;
3215 let node = data
3216 .get("repository")
3217 .filter(|value| !value.is_null())
3218 .ok_or_else(|| SourceError::Refused {
3219 message: format!(
3220 "GitHub repository {} was not found or is not visible to the token, so {} \
3221 {:?} cannot be created in it",
3222 repository.slug(),
3223 incoming.written.kind().describes(),
3224 incoming.title
3225 ),
3226 })?;
3227 let id = required_str(node, "id")?.to_owned();
3228 self.repository_cache()?
3229 .insert(repository.clone(), id.clone());
3230 Ok(id)
3231 }
3232
3233 fn repository_cache(
3234 &self,
3235 ) -> Result<std::sync::MutexGuard<'_, BTreeMap<RepositoryTarget, String>>, SourceError> {
3236 self.repository_cache
3237 .lock()
3238 .map_err(|_| SourceError::Unavailable {
3239 message: "this source's record of the destination repository was left \
3240 inconsistent by an earlier failure; next: run the command again"
3241 .into(),
3242 })
3243 }
3244
3245 /// Create or update one board item, whichever kind it is.
3246 async fn write_item(
3247 &self,
3248 incoming: &Incoming<'_>,
3249 target: Option<&NativeId>,
3250 depends_on: &[DependencyEdge],
3251 ) -> Result<NativeId, SourceError> {
3252 // Refused before anything is read or written: a task or a project titled the way
3253 // this board spells a document would land as an issue this same source reads back
3254 // as a document, so the field this destination cannot carry is named rather than
3255 // written and silently reclassified.
3256 if let Written::Work(kind, _) = incoming.written
3257 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
3258 {
3259 return Err(SourceError::Refused {
3260 message: format!(
3261 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
3262 spells a document, so it would read back as one rather than as a {}; \
3263 retitle it, or copy it as a document",
3264 kind.marker(),
3265 self.name,
3266 kind.marker()
3267 ),
3268 });
3269 }
3270 let board = match target {
3271 Some(target) => match self.board_for_update(target, incoming, depends_on).await? {
3272 Some(board) => board,
3273 None => self.board().await?,
3274 },
3275 None => self.board().await?,
3276 };
3277 let status_target = incoming
3278 .written
3279 .status()
3280 .map(|status| self.resolved_target(status.category))
3281 .transpose()?;
3282 let column = match (incoming.written.status(), status_target.as_ref()) {
3283 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
3284 _ => None,
3285 };
3286 let existing = target
3287 .map(|target| {
3288 board
3289 .items
3290 .iter()
3291 .find(|item| item.id == *target)
3292 .ok_or_else(|| SourceError::Refused {
3293 message: format!("GitHub destination item {} was not found", target.0),
3294 })
3295 })
3296 .transpose()?;
3297 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
3298 if content_kind == ContentKind::DraftIssue {
3299 if let (Some(StatusTarget::Closed(_)), Some(status)) =
3300 (status_target.as_ref(), incoming.written.status())
3301 {
3302 return Err(SourceError::Refused {
3303 message: format!(
3304 "status {} of source {} closes the item's issue, and GitHub draft items \
3305 have no open or closed state",
3306 category_name(status.category),
3307 self.name
3308 ),
3309 });
3310 }
3311 if incoming.parent.is_some() {
3312 return Err(SourceError::Refused {
3313 message: "GitHub draft items cannot be a project's sub-issue".into(),
3314 });
3315 }
3316 }
3317 match existing {
3318 Some(item) if content_kind == ContentKind::Issue => {
3319 if item.labels != incoming.labels {
3320 return Err(SourceError::Refused {
3321 message: "GitHub issue labels differ from the labels being written".into(),
3322 });
3323 }
3324 }
3325 _ => {
3326 if !incoming.labels.is_empty() {
3327 return Err(SourceError::Refused {
3328 message: "GitHub items created by this destination carry no labels".into(),
3329 });
3330 }
3331 }
3332 }
3333
3334 // An existing issue is never moved; a new one is created where the rule says. The
3335 // repository the issue really lives in is what the slot below is written against,
3336 // so a single entry that is where the issue is created travels as no key at all,
3337 // and the read side derives it back from the issue.
3338 let (own_repository, creation_target) = match existing {
3339 Some(item) => (item.own_repository.clone(), None),
3340 None => {
3341 let target = self.creation_target(&board, incoming)?;
3342 let origin = Repository::try_from(target.origin())
3343 .map_err(|message| SourceError::Config { message })?;
3344 (Some(origin), Some(target))
3345 }
3346 };
3347 let (native, fallback) = self
3348 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
3349 .await?;
3350 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
3351 let body = compose_body(incoming.content, &slot)?;
3352 // Read before anything is created, for the reason the field below is: a value
3353 // this destination cannot store has to refuse, and refusing after `createIssue`
3354 // would leave an issue behind that nothing asked for. The engine writes a
3355 // qualified id here; a caller handing this key anything else is told so rather
3356 // than having it silently stored as no origin at all.
3357 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
3358 let origin = match incoming.metadata.get(ORIGIN_KEY) {
3359 None => "",
3360 Some(Value::String(origin)) => origin.as_str(),
3361 Some(other) => {
3362 return Err(SourceError::Refused {
3363 message: format!(
3364 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
3365 is {other}"
3366 ),
3367 });
3368 }
3369 };
3370 // Resolved before anything is created: a board that cannot carry the copy origin
3371 // has to refuse the write, and refusing it after `createIssue` would leave an
3372 // issue behind that nothing asked for.
3373 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
3374 Some(field) => {
3375 if required_str(field, "__typename")? != "ProjectV2Field" {
3376 return Err(SourceError::Refused {
3377 message: format!(
3378 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
3379 ),
3380 });
3381 }
3382 Some(required_str(field, "id")?.to_owned())
3383 }
3384 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
3385 return Err(SourceError::Refused {
3386 message: format!(
3387 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
3388 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
3389 the board"
3390 ),
3391 });
3392 }
3393 None => None,
3394 };
3395
3396 let (content_id, item_id, url) = match existing {
3397 Some(item) => {
3398 self.update_existing(item, incoming, &body, status_target.as_ref())
3399 .await?;
3400 (item.id.clone(), item.item_id.clone(), item.url.clone())
3401 }
3402 None => {
3403 let target = creation_target
3404 .as_ref()
3405 .ok_or_else(|| SourceError::Malformed {
3406 message: "a new item was decided without a repository to create it in"
3407 .into(),
3408 })?;
3409 self.create_and_file_issue(&board, target, incoming, &body, status_target.as_ref())
3410 .await?
3411 }
3412 };
3413
3414 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
3415 // then each board field, the parent and the dependencies — and GitHub can fail at
3416 // any of them. Everything this source can refuse *before* the first of those is
3417 // already checked above, so what is left is GitHub itself failing part way. When it
3418 // does over an item this call created, the issue is taken back: a write that
3419 // refused must not leave an item behind that nobody asked for, and one that does
3420 // makes the retry create a second.
3421 let landed = self
3422 .finish_write(
3423 &board,
3424 incoming,
3425 &content_id,
3426 &item_id,
3427 content_kind,
3428 existing,
3429 origin_field.as_deref(),
3430 origin,
3431 column,
3432 &native,
3433 )
3434 .await;
3435 if let Err(error) = landed {
3436 if existing.is_none() {
3437 // Best effort, and the write's own failure is what the caller is told: a
3438 // refusal naming the tidy-up would hide why the write failed at all.
3439 let _ = self.delete_issue(&content_id).await;
3440 }
3441 return Err(error);
3442 }
3443
3444 // So the rest of this command reads what it just did rather than what the board
3445 // said before it. See `remember_written` for which half takes it.
3446 let remembered = Resolved {
3447 item_id,
3448 id: content_id.clone(),
3449 content_kind,
3450 kind: incoming.written.kind(),
3451 title: incoming.title.to_owned(),
3452 // The visible half of the body this write composed, split back off it the
3453 // way a read splits it — so what this record reports is what a read of the
3454 // same issue reports, rather than the person's text with the metadata slot
3455 // still on the end of it.
3456 body: metadata_body(body.clone())?.0,
3457 // A document has no status of its own; what it reads back as is whatever
3458 // the issue's own state says, which is what a re-read reports.
3459 status: incoming
3460 .written
3461 .status()
3462 .cloned()
3463 .unwrap_or_else(|| Status {
3464 category: StatusCategory::Unknown,
3465 name: "Open".to_owned(),
3466 }),
3467 labels: incoming.labels.to_vec(),
3468 parent: incoming.parent.cloned(),
3469 origin: (!origin.is_empty()).then(|| origin.to_owned()),
3470 // In the update path this is the item's own url, read off `existing` where the
3471 // tuple above was bound, so one expression serves both halves.
3472 url,
3473 created_at: existing.and_then(|item| item.created_at),
3474 updated_at: existing.and_then(|item| item.updated_at),
3475 own_repository,
3476 repositories: incoming.repositories.to_vec(),
3477 slot,
3478 board_id: Some(board.id.clone()),
3479 fields: board
3480 .fields
3481 .get("nodes")
3482 .and_then(Value::as_array)
3483 .cloned()
3484 .unwrap_or_default(),
3485 };
3486 self.remember_written(remembered, existing.is_none())?;
3487 Ok(content_id)
3488 }
3489
3490 /// What an update of an item this board already holds needs of the board, read off that
3491 /// item rather than off the board — or `None` when the item cannot say enough, and the
3492 /// board has to be read after all.
3493 ///
3494 /// An update needs the item it updates, the board's own id for a field write, the
3495 /// definitions of the `Status` and origin fields it writes, and every same-source far end
3496 /// its edges name. The first three ride along on the item's own node read — its board
3497 /// entry names the board's id, and each field value on it carries the definition of the
3498 /// field it is a value of — and each far end is read by its own node id. So updating one
3499 /// item costs a read of that item rather than every page of the board, which is what a
3500 /// copy naming one member out of many is for.
3501 ///
3502 /// **Nothing the item does not say is guessed.** A field this item holds no value of may
3503 /// still be on the board, and a view that read it as absent would refuse a write the
3504 /// board can take, or skip a field write the board needs. So a board this command has
3505 /// already read is used as it is, and an item that does not name its board, holds no
3506 /// value of the origin field, or holds no `Status` value when the write carries a status,
3507 /// sends the write back to reading the board — exactly as every update read it before.
3508 async fn board_for_update(
3509 &self,
3510 target: &NativeId,
3511 incoming: &Incoming<'_>,
3512 depends_on: &[DependencyEdge],
3513 ) -> Result<Option<Board>, SourceError> {
3514 if self.board_cache()?.is_some() {
3515 return Ok(None);
3516 }
3517 let Some(existing) = self.item_by_id(target).await? else {
3518 return Ok(None);
3519 };
3520 let Some(board_id) = existing.board_id.clone() else {
3521 return Ok(None);
3522 };
3523 let defines = |name: &str| {
3524 existing
3525 .fields
3526 .iter()
3527 .any(|field| field.get("name").and_then(Value::as_str) == Some(name))
3528 };
3529 if !defines(ORIGIN_FIELD) || (incoming.written.status().is_some() && !defines("Status")) {
3530 return Ok(None);
3531 }
3532 let fields = json!({"nodes": existing.fields, "pageInfo": {"hasNextPage": false}});
3533 let mut items = vec![existing];
3534 for edge in depends_on {
3535 if !edge
3536 .to
3537 .source()
3538 .is_none_or(|source| source == self.name.as_str())
3539 {
3540 continue;
3541 }
3542 // The first colon, for the reason `partition_edges` gives.
3543 let far = NativeId(if edge.to.is_qualified() {
3544 edge.to
3545 .id()
3546 .split_once(':')
3547 .map_or(edge.to.id(), |(_, native)| native)
3548 .to_owned()
3549 } else {
3550 edge.to.id().to_owned()
3551 });
3552 if items.iter().any(|item| item.id == far) {
3553 continue;
3554 }
3555 match self.item_by_id(&far).await? {
3556 Some(item) => items.push(item),
3557 // Refused against the whole board, in the words that refusal has always had.
3558 None => return Ok(None),
3559 }
3560 }
3561 Ok(Some(Board {
3562 id: board_id,
3563 fields,
3564 items,
3565 }))
3566 }
3567
3568 /// Everything a write does after the item exists: its board fields, its parent, and
3569 /// its dependencies.
3570 ///
3571 /// Split out of `write_item` so there is one place a failure past the point of no
3572 /// return is caught, rather than a tidy-up repeated at each `?` above.
3573 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
3574 // so there is one place a failure past the point of no return is caught, and its
3575 // arguments are exactly the values that tail already had in scope. Bundling them into a
3576 // struct would describe no concept — it would be "the arguments of this function" — and
3577 // would put the whole of `write_item`'s locals behind one more indirection.
3578 #[allow(clippy::too_many_arguments)]
3579 async fn finish_write(
3580 &self,
3581 board: &Board,
3582 incoming: &Incoming<'_>,
3583 content_id: &NativeId,
3584 item_id: &str,
3585 content_kind: ContentKind,
3586 existing: Option<&Resolved>,
3587 origin_field: Option<&str>,
3588 origin: &str,
3589 column: Option<(String, String)>,
3590 native: &[String],
3591 ) -> Result<(), SourceError> {
3592 if let Some(field_id) = origin_field {
3593 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
3594 .await?;
3595 }
3596
3597 if let Some((field_id, option_id)) = column {
3598 self.set_item_field(
3599 &board.id,
3600 item_id,
3601 &field_id,
3602 json!({"singleSelectOptionId":option_id}),
3603 )
3604 .await?;
3605 }
3606
3607 if content_kind == ContentKind::Issue {
3608 self.reparent(
3609 existing.and_then(|item| item.parent.clone()),
3610 content_id,
3611 incoming.parent,
3612 )
3613 .await?;
3614 // A document takes part in no dependency graph, so writing one neither reads
3615 // nor changes the issue's own `blockedBy` relationships. Reconciling them
3616 // against the empty list a document write carries would *delete* whatever
3617 // relationships a person had made on that issue, which is a write nobody
3618 // asked for.
3619 if incoming.written.kind() != BoardKind::Document {
3620 self.reconcile_blocked_by(content_id, native).await?;
3621 }
3622 }
3623 Ok(())
3624 }
3625
3626 /// Delete one issue, which takes its board item with it.
3627 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
3628 let data = self
3629 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3630 .await?;
3631 data.pointer("/deleteIssue/repository")
3632 .filter(|value| !value.is_null())
3633 .ok_or_else(|| SourceError::Malformed {
3634 message: "GitHub issue deletion returned no repository".into(),
3635 })?;
3636 self.forget(id)?;
3637 Ok(())
3638 }
3639
3640 /// Remove one item this copy created, so a copy that could not finish leaves the board
3641 /// as it found it.
3642 ///
3643 /// Deleting the issue takes its board item with it, so there is no second mutation to
3644 /// keep in step. An id the board does not hold is not an error: the item is already
3645 /// gone, which is the state this asks for.
3646 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
3647 let board = self.board().await?;
3648 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
3649 return Ok(());
3650 };
3651 if item.content_kind == ContentKind::DraftIssue {
3652 return Err(SourceError::Refused {
3653 message: format!(
3654 "GitHub item {} is a draft, and this source removes an item by deleting \
3655 its issue; next: remove it from the board by hand",
3656 id.0
3657 ),
3658 });
3659 }
3660 let data = self
3661 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3662 .await?;
3663 data.pointer("/deleteIssue/repository")
3664 .filter(|value| !value.is_null())
3665 .ok_or_else(|| SourceError::Malformed {
3666 message: "GitHub issue deletion returned no repository".into(),
3667 })?;
3668 self.forget(id)?;
3669 Ok(())
3670 }
3671
3672 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
3673 async fn partition_edges(
3674 &self,
3675 board: &Board,
3676 near_kind: BoardKind,
3677 near_content: ContentKind,
3678 depends_on: &[DependencyEdge],
3679 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
3680 let mut native = Vec::new();
3681 let mut fallback = Vec::new();
3682 for edge in depends_on {
3683 let same_source = edge
3684 .to
3685 .source()
3686 .is_none_or(|source| source == self.name.as_str());
3687 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
3688 // `DependencyEndpoint::source` both read it that way — and a native id may hold
3689 // colons of its own, so the far end is everything after that one separator.
3690 // Splitting at the last would truncate `work:urn:task:7` to `7`.
3691 let far_id = if edge.to.is_qualified() {
3692 edge.to
3693 .id()
3694 .split_once(':')
3695 .map_or(edge.to.id(), |(_, native)| native)
3696 } else {
3697 edge.to.id()
3698 };
3699 let far = if same_source {
3700 Some(
3701 board
3702 .items
3703 .iter()
3704 .find(|item| item.id.0 == far_id)
3705 .ok_or_else(|| SourceError::Refused {
3706 message: format!("GitHub dependency item {far_id} was not found"),
3707 })?,
3708 )
3709 } else {
3710 None
3711 };
3712 // The caller says which kind the far end is, and this board holds the far end
3713 // itself, so a disagreement is settled here rather than stored: recorded, the
3714 // wrong kind would read back as a cross-level edge that never existed; written
3715 // natively, it would name a relationship of a different level than the caller
3716 // asked for.
3717 //
3718 // A far end this board holds as a *document* fails the same comparison and is
3719 // refused by the same sentence: `ItemKind` has no document variant because
3720 // nothing may point at one, so no caller can name it correctly and the refusal
3721 // is the only honest answer.
3722 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
3723 return Err(SourceError::Refused {
3724 message: format!(
3725 "GitHub dependency item {far_id} is a {} of this board, and this item \
3726 names it as a {}; record the kind it is",
3727 disagreeing.kind.describes(),
3728 edge.to.kind.marker()
3729 ),
3730 });
3731 }
3732 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
3733 // however the far end is spelled — and one classified native here would be
3734 // written nowhere at all, because a draft's native reconciliation never runs.
3735 let native_here = near_content == ContentKind::Issue
3736 && far.is_some_and(|far| {
3737 far.content_kind == ContentKind::Issue
3738 && BoardKind::Work(edge.to.kind) == near_kind
3739 });
3740 if native_here {
3741 native.push(far_id.to_owned());
3742 } else {
3743 fallback.push(edge.clone());
3744 }
3745 }
3746 Ok((native, fallback))
3747 }
3748
3749 async fn update_existing(
3750 &self,
3751 item: &Resolved,
3752 incoming: &Incoming<'_>,
3753 body: &Option<String>,
3754 status_target: Option<&StatusTarget>,
3755 ) -> Result<(), SourceError> {
3756 let title = incoming.written_title();
3757 let (operation, input, pointer) = match item.content_kind {
3758 ContentKind::DraftIssue => (
3759 graphql::UPDATE_DRAFT,
3760 json!({"draftIssueId":item.id.0,"title":title,"body":body}),
3761 "/updateProjectV2DraftIssue/draftIssue",
3762 ),
3763 ContentKind::Issue => (
3764 graphql::UPDATE_ISSUE,
3765 json!({"id":item.id.0,"title":title,"body":body,
3766 "stateInput":state_input(status_target)}),
3767 "/updateIssue/issue",
3768 ),
3769 };
3770 let data = self.graphql(operation, json!({"input":input})).await?;
3771 let returned = data
3772 .pointer(pointer)
3773 .ok_or_else(|| SourceError::Malformed {
3774 message: "GitHub item update returned no item".into(),
3775 })?;
3776 if required_str(returned, "id")? != item.id.0 {
3777 return Err(SourceError::Malformed {
3778 message: "GitHub item update returned the wrong item".into(),
3779 });
3780 }
3781 Ok(())
3782 }
3783
3784 /// Creates one issue, files it on the board, and closes it when the status says so.
3785 ///
3786 /// Three calls rather than one: `createIssue` needs a repository and answers with an
3787 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
3788 /// closed status is a state of the issue rather than a field of the board item.
3789 /// Creates the issue, files it on the board, and reports what a read of it would say:
3790 /// its content id, its board item id, and the web address GitHub gave it.
3791 ///
3792 /// The address comes back here because this is the only place it is known before
3793 /// GitHub's own board read catches up — an item this run created answers the reads
3794 /// that follow it out of the record below, and one remembered without its address
3795 /// would report no location for the rest of the run.
3796 async fn create_and_file_issue(
3797 &self,
3798 board: &Board,
3799 repository: &RepositoryTarget,
3800 incoming: &Incoming<'_>,
3801 body: &Option<String>,
3802 status_target: Option<&StatusTarget>,
3803 ) -> Result<(NativeId, String, Option<String>), SourceError> {
3804 let repository_id = self.repository_id(repository, incoming).await?;
3805 let data = self
3806 .graphql(
3807 graphql::CREATE_ISSUE,
3808 json!({"input":{
3809 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
3810 }}),
3811 )
3812 .await?;
3813 let created = data
3814 .pointer("/createIssue/issue")
3815 .filter(|value| !value.is_null())
3816 .ok_or_else(|| SourceError::Malformed {
3817 message: "GitHub issue creation returned no issue".into(),
3818 })?;
3819 let content_id = NativeId(required_str(created, "id")?.to_owned());
3820 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
3821 // a response without it is not worth failing a landed write over — the item simply
3822 // reports no location until the board read catches up, which is what it did before.
3823 let url = optional_str(created, "url")?.map(str::to_owned);
3824 // The issue exists from here on, so a failure filing it on the board takes it
3825 // back: an issue in the repository that is on no board is an item nobody asked for
3826 // and nothing here would find again.
3827 let added = match self
3828 .graphql(
3829 graphql::ADD_TO_BOARD,
3830 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
3831 )
3832 .await
3833 {
3834 Ok(added) => added,
3835 Err(error) => {
3836 let _ = self.delete_issue(&content_id).await;
3837 return Err(error);
3838 }
3839 };
3840 let item = added
3841 .pointer("/addProjectV2ItemById/item")
3842 .filter(|value| !value.is_null())
3843 .ok_or_else(|| SourceError::Malformed {
3844 message: "GitHub board addition returned no project item".into(),
3845 })?;
3846 if let Some(StatusTarget::Closed(_)) = status_target {
3847 let closed = self
3848 .graphql(
3849 graphql::UPDATE_ISSUE,
3850 json!({"input":{"id":content_id.0,"stateInput":state_input(status_target)}}),
3851 )
3852 .await?;
3853 let returned =
3854 closed
3855 .pointer("/updateIssue/issue")
3856 .ok_or_else(|| SourceError::Malformed {
3857 message: "GitHub item update returned no item".into(),
3858 })?;
3859 if required_str(returned, "id")? != content_id.0 {
3860 return Err(SourceError::Malformed {
3861 message: "GitHub item update returned the wrong item".into(),
3862 });
3863 }
3864 }
3865 Ok((content_id, required_str(item, "id")?.to_owned(), url))
3866 }
3867
3868 /// Move one issue under the project it now belongs to, or out of the one it left.
3869 async fn reparent(
3870 &self,
3871 held: Option<NativeId>,
3872 child: &NativeId,
3873 wanted: Option<&NativeId>,
3874 ) -> Result<(), SourceError> {
3875 if held.as_ref() == wanted {
3876 return Ok(());
3877 }
3878 if let Some(held) = &held {
3879 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
3880 .await?;
3881 }
3882 if let Some(wanted) = wanted {
3883 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
3884 .await?;
3885 }
3886 Ok(())
3887 }
3888
3889 async fn sub_issue(
3890 &self,
3891 operation: &str,
3892 parent: &NativeId,
3893 child: &NativeId,
3894 root: &str,
3895 ) -> Result<(), SourceError> {
3896 let data = self
3897 .graphql(
3898 operation,
3899 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
3900 )
3901 .await?;
3902 let issue =
3903 data.pointer(&format!("/{root}/issue"))
3904 .ok_or_else(|| SourceError::Malformed {
3905 message: "GitHub sub-issue update returned no issue".into(),
3906 })?;
3907 let sub =
3908 data.pointer(&format!("/{root}/subIssue"))
3909 .ok_or_else(|| SourceError::Malformed {
3910 message: "GitHub sub-issue update returned no sub-issue".into(),
3911 })?;
3912 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
3913 return Err(SourceError::Malformed {
3914 message: "GitHub sub-issue update returned the wrong issues".into(),
3915 });
3916 }
3917 Ok(())
3918 }
3919
3920 async fn reconcile_blocked_by(
3921 &self,
3922 content_id: &NativeId,
3923 native: &[String],
3924 ) -> Result<(), SourceError> {
3925 let current = self.native_dependency_ids(content_id).await?;
3926 for (operation, far_id) in current
3927 .iter()
3928 .filter(|id| !native.contains(id))
3929 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
3930 .chain(
3931 native
3932 .iter()
3933 .filter(|id| !current.contains(id))
3934 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
3935 )
3936 {
3937 let data = self
3938 .graphql(
3939 operation,
3940 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
3941 )
3942 .await?;
3943 let root = if operation == graphql::ADD_BLOCKED_BY {
3944 "addBlockedBy"
3945 } else {
3946 "removeBlockedBy"
3947 };
3948 let issue =
3949 data.pointer(&format!("/{root}/issue"))
3950 .ok_or_else(|| SourceError::Malformed {
3951 message: "GitHub dependency update returned no issue".into(),
3952 })?;
3953 let blocker = data
3954 .pointer(&format!("/{root}/blockingIssue"))
3955 .ok_or_else(|| SourceError::Malformed {
3956 message: "GitHub dependency update returned no blocking issue".into(),
3957 })?;
3958 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
3959 {
3960 return Err(SourceError::Malformed {
3961 message: "GitHub dependency update returned the wrong issues".into(),
3962 });
3963 }
3964 }
3965 Ok(())
3966 }
3967}
3968
3969/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
3970enum Reached {
3971 /// An issue this board holds, resolved into everything this source reports about it.
3972 Held(Box<Resolved>),
3973 /// Nothing this board holds: no such node, or a node on some other board.
3974 Nothing,
3975 /// A board draft, which exists only inside the board's own item connection.
3976 Draft,
3977}
3978
3979/// What GitHub says when a string is not a node id it can resolve.
3980///
3981/// Matched because it is the ordinary answer to a project selector naming a project by its
3982/// *name*, and reporting that as a failure would make naming one impossible. It is read
3983/// off the refusal GitHub sent, never guessed from the shape of the string: this source
3984/// does not define the syntax of a GitHub node id and would be wrong about it.
3985const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
3986
3987/// Whether this refusal is GitHub saying the id names no node at all.
3988fn unresolvable_node(error: &SourceError) -> bool {
3989 matches!(error, SourceError::Refused { message }
3990 if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
3991}
3992
3993/// One project name, as a search qualifier which filters on it at the server.
3994///
3995/// Quoted so the whole title is one phrase rather than a bag of words, with the two
3996/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
3997/// the way it documents. A title matched here is still compared for equality afterwards:
3998/// the qualifier narrows what the server sends, and this source decides what it names.
3999fn title_qualifier(name: &str) -> String {
4000 let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
4001 format!("in:title \"{escaped}\"")
4002}
4003
4004/// The board, and every item on it this source reports.
4005#[derive(Clone)]
4006struct Board {
4007 id: String,
4008 fields: Value,
4009 items: Vec<Resolved>,
4010}
4011
4012impl Board {
4013 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
4014 complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
4015 let nodes = fields
4016 .get("nodes")
4017 .and_then(Value::as_array)
4018 .ok_or_else(|| SourceError::Malformed {
4019 message: "GitHub project fields.nodes is not an array".into(),
4020 })?;
4021 Ok(nodes
4022 .iter()
4023 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
4024 }
4025}
4026
4027/// One board item, resolved into everything this source reports about it.
4028#[derive(Clone)]
4029struct Resolved {
4030 item_id: String,
4031 id: NativeId,
4032 content_kind: ContentKind,
4033 kind: BoardKind,
4034 title: String,
4035 body: Option<String>,
4036 status: Status,
4037 labels: Vec<Label>,
4038 parent: Option<NativeId>,
4039 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
4040 origin: Option<String>,
4041 url: Option<String>,
4042 created_at: Option<DateTime<Utc>>,
4043 updated_at: Option<DateTime<Utc>>,
4044 own_repository: Option<Repository>,
4045 repositories: Vec<Repository>,
4046 slot: BTreeMap<String, Value>,
4047 /// The node id of the board this item sits on, when the read that reached it said.
4048 board_id: Option<String>,
4049 /// The definition of every board field this item holds a value of, in the shape a read
4050 /// of the board's own `fields` gives one.
4051 ///
4052 /// Only the fields this item has a value in: a field it holds nothing of is not here,
4053 /// which says nothing about whether the board has it.
4054 fields: Vec<Value>,
4055}
4056
4057impl Resolved {
4058 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
4059 /// in a field of its own, and none of the three keys that are only an encoding.
4060 fn metadata(&self) -> BTreeMap<String, Value> {
4061 let mut metadata = self.slot.clone();
4062 metadata.remove(Repository::METADATA_KEY);
4063 metadata.remove(DependencyEdge::RECORDED_KEY);
4064 metadata.remove(ItemKind::METADATA_KEY);
4065 if let Some(origin) = &self.origin {
4066 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
4067 }
4068 metadata
4069 }
4070
4071 /// Where this item is, as a link a reader can open.
4072 ///
4073 /// A board is a hosted place and every issue on it has a web address, so that address
4074 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
4075 /// of place it is, so a reader knows to open it rather than to read a file out. It
4076 /// does not replace or derive from `url`: the field goes on reporting exactly what it
4077 /// reported before, and this says what that address *is*.
4078 ///
4079 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
4080 /// rather than a third variant, which is the contract's "the source did not say". An
4081 /// issue this run created is not one of those: its address comes back from the
4082 /// creating mutation, so it is somewhere a reader can open from the moment it exists
4083 /// rather than from whenever the board read catches up.
4084 fn location(&self) -> Option<Location> {
4085 self.url.clone().map(Location::Url)
4086 }
4087
4088 fn task(&self) -> Task {
4089 Task {
4090 id: self.id.clone(),
4091 title: self.title.clone(),
4092 content: self.body.clone(),
4093 status: self.status.clone(),
4094 labels: self.labels.clone(),
4095 project: self.parent.clone(),
4096 url: self.url.clone(),
4097 location: self.location(),
4098 created_at: self.created_at,
4099 updated_at: self.updated_at,
4100 metadata: self.metadata(),
4101 repositories: self.repositories.clone(),
4102 }
4103 }
4104
4105 fn project(&self) -> Project {
4106 Project {
4107 id: self.id.clone(),
4108 title: self.title.clone(),
4109 content: self.body.clone(),
4110 status: self.status.clone(),
4111 labels: self.labels.clone(),
4112 url: self.url.clone(),
4113 location: self.location(),
4114 created_at: self.created_at,
4115 updated_at: self.updated_at,
4116 metadata: self.metadata(),
4117 repositories: self.repositories.clone(),
4118 }
4119 }
4120
4121 /// The same issue as a document: the project it is filed under, and no status and no
4122 /// dependencies, because a document is not work.
4123 fn document(&self) -> Document {
4124 Document {
4125 id: self.id.clone(),
4126 title: self.title.clone(),
4127 content: self.body.clone(),
4128 project: self.parent.clone(),
4129 labels: self.labels.clone(),
4130 url: self.url.clone(),
4131 location: self.location(),
4132 created_at: self.created_at,
4133 updated_at: self.updated_at,
4134 metadata: self.metadata(),
4135 repositories: self.repositories.clone(),
4136 }
4137 }
4138}
4139
4140/// What one write is, and the status that comes with being it.
4141///
4142/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
4143/// status and a task or a project always has one, so "a document carrying a status" and
4144/// "a task carrying none" are states a write cannot be in rather than states every use
4145/// site below has to defend against.
4146enum Written<'a> {
4147 /// A document, which is not work and so has no status at all.
4148 Document,
4149 /// A task or a project, and the status it is being written with.
4150 Work(ItemKind, &'a Status),
4151}
4152
4153impl Written<'_> {
4154 /// Which of the board's three kinds this write is.
4155 const fn kind(&self) -> BoardKind {
4156 match self {
4157 Self::Document => BoardKind::Document,
4158 Self::Work(kind, _) => BoardKind::Work(*kind),
4159 }
4160 }
4161
4162 /// The status this write carries. A document carries none, so a write of one says
4163 /// nothing about the issue's open or closed state and selects no board `Status`
4164 /// option.
4165 const fn status(&self) -> Option<&Status> {
4166 match self {
4167 Self::Document => None,
4168 Self::Work(_, status) => Some(status),
4169 }
4170 }
4171}
4172
4173/// The item being written, in the one shape all three write methods reach.
4174struct Incoming<'a> {
4175 written: Written<'a>,
4176 /// The title a person wrote. A document's goes onto the issue with
4177 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
4178 title: &'a str,
4179 content: Option<&'a str>,
4180 labels: &'a [Label],
4181 metadata: &'a BTreeMap<String, Value>,
4182 repositories: &'a [Repository],
4183 parent: Option<&'a NativeId>,
4184}
4185
4186impl Incoming<'_> {
4187 /// The title this write puts on the issue.
4188 fn written_title(&self) -> String {
4189 match self.written {
4190 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
4191 Written::Work(..) => self.title.to_owned(),
4192 }
4193 }
4194}
4195
4196#[derive(Clone, Copy, PartialEq, Eq)]
4197enum ContentKind {
4198 DraftIssue,
4199 Issue,
4200}
4201
4202/// What one board issue is: a document, or the work an [`ItemKind`] names.
4203///
4204/// A type of this source's own rather than an `ItemKind` with a third variant, because
4205/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
4206/// document — the contract keeps a document out of that enum deliberately. Holding the
4207/// board's three answers in one value is what makes every place that asks "which is this?"
4208/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
4209/// two thirds of the board.
4210#[derive(Clone, Copy, PartialEq, Eq)]
4211enum BoardKind {
4212 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
4213 Document,
4214 /// Every other issue, and every draft.
4215 Work(ItemKind),
4216}
4217
4218impl BoardKind {
4219 /// How a refusal names this kind to the person reading it.
4220 const fn describes(self) -> &'static str {
4221 match self {
4222 Self::Document => "document",
4223 Self::Work(kind) => kind.marker(),
4224 }
4225 }
4226}
4227
4228/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
4229///
4230/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
4231/// the shared cross-source journeys assert one answer to one question, so two sources
4232/// that disagree about what "carries the label bug" means fail them.
4233fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
4234 let holds = |name: &String| {
4235 labels
4236 .iter()
4237 .any(|label| label.name.eq_ignore_ascii_case(name))
4238 };
4239 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
4240 && filter.all_of.iter().all(holds)
4241 && !filter.none_of.iter().any(holds)
4242}
4243
4244/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
4245/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
4246fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
4247 statuses.is_empty() || statuses.contains(&category)
4248}
4249
4250/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
4251///
4252/// `content` is the item's own prose — the body with this source's trailing metadata
4253/// comment already taken off — so a search never matches an encoding the author of the
4254/// issue never wrote.
4255fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
4256 let terms = query.terms.to_lowercase();
4257 let in_title = title.to_lowercase().contains(&terms);
4258 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
4259 match query.fields {
4260 TextFields::Title => in_title,
4261 TextFields::Content => in_content,
4262 TextFields::TitleOrContent => in_title || in_content,
4263 }
4264}
4265
4266/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
4267///
4268/// The project predicate is passed separately because a read narrowed to one project has
4269/// already answered it by asking *that project* for its own items — and re-applying it
4270/// there would compare the caller's selector, which may be a project's **name**, against
4271/// the id of the project that name resolved to, and keep nothing. Every other read passes
4272/// `query.project` and applies it here, which is what keeps `projects` a predicate this
4273/// source really does apply.
4274fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
4275 labels_match(&task.labels, &query.labels)
4276 && status_matches(task.status.category, &query.statuses)
4277 && match project {
4278 ProjectFilter::Any => true,
4279 ProjectFilter::Orphans => task.project.is_none(),
4280 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
4281 }
4282 && query
4283 .text
4284 .as_ref()
4285 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
4286}
4287
4288fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
4289 labels_match(&project.labels, &query.labels)
4290 && status_matches(project.status.category, &query.statuses)
4291 && query
4292 .text
4293 .as_ref()
4294 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
4295}
4296
4297/// The same three predicates a task query carries, minus the status filter.
4298///
4299/// A document is not work, so it has no status for one to compare against and the query
4300/// type carries none. The project predicate is the same one — a design issue filed under a
4301/// project issue is in that project, and one filed under nothing is in none — so it is
4302/// spelled the same way here rather than answered differently.
4303fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
4304 labels_match(&document.labels, &query.labels)
4305 && match project {
4306 ProjectFilter::Any => true,
4307 ProjectFilter::Orphans => document.project.is_none(),
4308 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
4309 }
4310 && query
4311 .text
4312 .as_ref()
4313 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
4314}
4315
4316#[async_trait::async_trait]
4317impl TaskSource for GitHubProjectsSource {
4318 fn kind(&self) -> &'static str {
4319 KIND
4320 }
4321 fn capabilities(&self) -> Capabilities {
4322 Capabilities {
4323 projects: Support::Native,
4324 documents: Support::Native,
4325 orphan_tasks: Support::Native,
4326 filter_by_label: Support::Native,
4327 filter_by_status: Support::Native,
4328 search_title: Support::Native,
4329 search_content: Support::Native,
4330 task_dependencies: DependencySupport::BothDirections,
4331 project_dependencies: DependencySupport::BothDirections,
4332 max_page_size: MAX_PAGE_SIZE,
4333 }
4334 }
4335 async fn health(&self) -> Result<Health, SourceError> {
4336 let board = self.board_page(None, 1).await?;
4337 Ok(Health {
4338 reachable: true,
4339 detail: Some(format!(
4340 "reading GitHub project {}/{} ({})",
4341 self.owner,
4342 self.project_number,
4343 required_str(&board, "title")?
4344 )),
4345 })
4346 }
4347 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
4348 Ok(self
4349 .item_by_id(id)
4350 .await?
4351 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4352 .map(|item| item.task()))
4353 }
4354 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
4355 Ok(self
4356 .item_by_id(id)
4357 .await?
4358 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4359 .map(|item| item.project()))
4360 }
4361 async fn query_tasks(
4362 &self,
4363 query: &TaskQuery,
4364 page: &PageRequest,
4365 ) -> Result<Page<Task>, SourceError> {
4366 validate_page(page)?;
4367 // A read narrowed to one project asks that project for its own tasks, so nothing
4368 // about it costs what the rest of the board holds. Every other task read is a
4369 // question about the whole board and is answered by reading it.
4370 let (held, membership) = match &query.project {
4371 ProjectFilter::Is(project) => (
4372 self.project_children(project).await?,
4373 // Answered by where these items came from; see `task_matches`.
4374 &ProjectFilter::Any,
4375 ),
4376 ProjectFilter::Any | ProjectFilter::Orphans => {
4377 (self.board().await?.items, &query.project)
4378 }
4379 };
4380 // Filtered before paged: a page of a filtered result is a page of the survivors,
4381 // never the survivors of a page.
4382 let tasks = held
4383 .iter()
4384 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4385 .map(Resolved::task)
4386 .filter(|task| task_matches(task, query, membership))
4387 .collect();
4388 Ok(offset_page(
4389 tasks,
4390 numeric_cursor(page.cursor.as_ref())?,
4391 page.limit.min(MAX_PAGE_SIZE) as usize,
4392 ))
4393 }
4394 async fn query_projects(
4395 &self,
4396 query: &ProjectQuery,
4397 page: &PageRequest,
4398 ) -> Result<Page<Project>, SourceError> {
4399 validate_page(page)?;
4400 // The projects a board holds are found by an issue search scoped to that board,
4401 // never by walking the board's own item connection: what tells a project from a
4402 // task is the `parent` each issue carries, which costs nothing to read.
4403 let projects = self
4404 .board_issues()
4405 .await?
4406 .iter()
4407 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4408 .map(Resolved::project)
4409 .filter(|project| project_matches(project, query))
4410 .collect();
4411 Ok(offset_page(
4412 projects,
4413 numeric_cursor(page.cursor.as_ref())?,
4414 page.limit.min(MAX_PAGE_SIZE) as usize,
4415 ))
4416 }
4417 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
4418 Ok(self
4419 .item_by_id(id)
4420 .await?
4421 .filter(|item| item.kind == BoardKind::Document)
4422 .map(|item| item.document()))
4423 }
4424 async fn query_documents(
4425 &self,
4426 query: &DocumentQuery,
4427 page: &PageRequest,
4428 ) -> Result<Page<Document>, SourceError> {
4429 validate_page(page)?;
4430 // Narrowed to one project, this is the same sub-issue read a task list scoped to
4431 // that project makes — a document filed under a project is a sub-issue of it too,
4432 // and which of them come back is the kind this caller asked for.
4433 let (held, membership) = match &query.project {
4434 ProjectFilter::Is(project) => (
4435 self.project_children(project).await?,
4436 // Answered by where these items came from; see `task_matches`.
4437 &ProjectFilter::Any,
4438 ),
4439 ProjectFilter::Any | ProjectFilter::Orphans => {
4440 (self.board().await?.items, &query.project)
4441 }
4442 };
4443 // Filtered before paged, exactly as a task read is: a page of a filtered result is
4444 // a page of the survivors, never the survivors of a page.
4445 let documents = held
4446 .iter()
4447 .filter(|item| item.kind == BoardKind::Document)
4448 .map(Resolved::document)
4449 .filter(|document| document_matches(document, query, membership))
4450 .collect();
4451 Ok(offset_page(
4452 documents,
4453 numeric_cursor(page.cursor.as_ref())?,
4454 page.limit.min(MAX_PAGE_SIZE) as usize,
4455 ))
4456 }
4457 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
4458 validate_page(page)?;
4459 let offset = numeric_cursor(page.cursor.as_ref())?;
4460 let mut labels = self
4461 .board()
4462 .await?
4463 .items
4464 .into_iter()
4465 .flat_map(|item| item.labels)
4466 .fold(Vec::new(), |mut all, label| {
4467 if !all.iter().any(|x: &Label| x.id == label.id) {
4468 all.push(label);
4469 }
4470 all
4471 });
4472 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
4473 Ok(offset_page(
4474 labels,
4475 offset,
4476 page.limit.min(MAX_PAGE_SIZE) as usize,
4477 ))
4478 }
4479 async fn task_dependencies(
4480 &self,
4481 id: &NativeId,
4482 direction: Direction,
4483 page: &PageRequest,
4484 ) -> Result<Page<DependencyEdge>, SourceError> {
4485 self.dependencies(id, ItemKind::Task, direction, page).await
4486 }
4487 async fn project_dependencies(
4488 &self,
4489 id: &NativeId,
4490 direction: Direction,
4491 page: &PageRequest,
4492 ) -> Result<Page<DependencyEdge>, SourceError> {
4493 self.dependencies(id, ItemKind::Project, direction, page)
4494 .await
4495 }
4496
4497 fn writes(&self) -> WriteSupport {
4498 WriteSupport::Supported
4499 }
4500
4501 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
4502 self.write_item(
4503 &Incoming {
4504 written: Written::Work(ItemKind::Task, &write.item.status),
4505 title: &write.item.title,
4506 content: write.item.content.as_deref(),
4507 labels: &write.item.labels,
4508 metadata: &write.item.metadata,
4509 repositories: &write.item.repositories,
4510 parent: write.item.project.as_ref(),
4511 },
4512 write.target.as_ref(),
4513 &write.depends_on,
4514 )
4515 .await
4516 }
4517
4518 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
4519 self.write_item(
4520 &Incoming {
4521 written: Written::Work(ItemKind::Project, &write.item.status),
4522 title: &write.item.title,
4523 content: write.item.content.as_deref(),
4524 labels: &write.item.labels,
4525 metadata: &write.item.metadata,
4526 repositories: &write.item.repositories,
4527 parent: None,
4528 },
4529 write.target.as_ref(),
4530 &write.depends_on,
4531 )
4532 .await
4533 }
4534
4535 /// Create or update one document, which is one issue titled the way this board spells
4536 /// a document.
4537 ///
4538 /// Everything else is exactly a task write: caller metadata goes to the same canonical
4539 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
4540 /// or a field this board cannot carry is refused by name rather than dropped, a target
4541 /// naming an issue this board does not hold is refused rather than created, and an
4542 /// issue this call created is taken back when the rest of the write fails.
4543 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
4544 // A document takes part in no dependency graph, so there is no far end to write
4545 // natively and none to record: a caller naming one is told so rather than having it
4546 // stored under the reserved key, where a later read would report an edge the
4547 // contract says cannot exist.
4548 if !write.depends_on.is_empty() {
4549 return Err(SourceError::Refused {
4550 message: format!(
4551 "this write names {} dependencies for a document, and a document takes \
4552 part in no dependency graph; next: put the dependency on the task or \
4553 project the document is about",
4554 write.depends_on.len()
4555 ),
4556 });
4557 }
4558 self.write_item(
4559 &Incoming {
4560 written: Written::Document,
4561 title: &write.item.title,
4562 content: write.item.content.as_deref(),
4563 labels: &write.item.labels,
4564 metadata: &write.item.metadata,
4565 repositories: &write.item.repositories,
4566 parent: write.item.project.as_ref(),
4567 },
4568 write.target.as_ref(),
4569 &[],
4570 )
4571 .await
4572 }
4573
4574 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
4575 self.delete_item(id).await
4576 }
4577
4578 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
4579 self.delete_item(id).await
4580 }
4581
4582 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
4583 self.delete_item(id).await
4584 }
4585
4586 /// Every request this source has recorded, and what each of GitHub's two budgets was
4587 /// attributed — read off the same accounting the session report is rendered from, so
4588 /// the two cannot count one request two ways.
4589 async fn metering(&self) -> Result<Option<Metering>, SourceError> {
4590 Ok(Some(self.ledger.snapshot().metering()))
4591 }
4592}
4593
4594/// Where the recorded tail of a dependency walk resumes; see
4595/// [`GitHubProjectsSource::recorded_edges`].
4596const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
4597
4598/// The board text field this source keeps a copy's origin in.
4599///
4600/// Named after the key it holds, and held to that name by the guard below rather than by
4601/// a reader noticing.
4602const ORIGIN_FIELD: &str = "onetaskgraph.origin";
4603
4604/// The metadata key that field holds.
4605///
4606/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
4607/// constructs or interprets the qualified id it carries. This source names it only to
4608/// route it — a short, typed value belongs in a typed field rather than in the body slot
4609/// a caller's own prose shares.
4610///
4611/// Restated rather than imported, because no plugin crate may depend on the engine. What
4612/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
4613/// target in `check`: it reads the engine's own literal and fails naming the file and the
4614/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
4615/// that creates a second item every run instead of finding the one it wrote — and that is
4616/// too late to learn it.
4617const ORIGIN_KEY: &str = "onetaskgraph.origin";
4618
4619/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
4620///
4621/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
4622/// is derived from the far end, never written down on the near item — so only a forward
4623/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
4624/// it did not come from, and it is told so rather than answered with an empty page that
4625/// reads as a walk which ended.
4626fn recorded_offset(
4627 cursor: Option<&str>,
4628 direction: Direction,
4629) -> Result<Option<usize>, SourceError> {
4630 cursor
4631 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
4632 .map(|offset| {
4633 if direction != Direction::DependsOn {
4634 return Err(SourceError::Config {
4635 message: format!(
4636 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
4637 reverse dependency read never issues; resume it in the direction \
4638 that reported it"
4639 ),
4640 });
4641 }
4642 offset.parse().map_err(|_| SourceError::Config {
4643 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
4644 })
4645 })
4646 .transpose()
4647}
4648
4649fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
4650 let mut page = offset_page(edges, offset, limit.max(1));
4651 page.next = page
4652 .next
4653 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
4654 page
4655}
4656
4657/// The kind of one issue reached through a dependency connection.
4658///
4659/// The same questions the board scan asks, over the fields the dependency document
4660/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
4661/// then anything with sub-issues or the marker is a project.
4662///
4663/// # Errors
4664///
4665/// A far end this board holds as a document is refused rather than reported. The two
4666/// answers that are not refusals would both be wrong: reporting it as a task names an id
4667/// no task read of this source can find, and reporting it as a project names one no
4668/// project read can. There is no third value to return — `ItemKind` has no document
4669/// variant, because nothing may point at a document — so the relationship itself is what
4670/// the person is told about.
4671fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
4672 let id = required_str(value, "id")?;
4673 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
4674 return Err(SourceError::Refused {
4675 message: format!(
4676 "GitHub issue {id} is a document of this board — its title begins \
4677 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
4678 on by one; next: remove that issue's blocking relationship on this board"
4679 ),
4680 });
4681 }
4682 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
4683 if parent.is_some() {
4684 return Ok(ItemKind::Task);
4685 }
4686 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
4687 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
4688 message: format!("GitHub issue {id}: {message}"),
4689 })?;
4690 let sub_issues = sub_issue_total(value)?;
4691 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
4692 ItemKind::Project
4693 } else {
4694 ItemKind::Task
4695 })
4696}
4697
4698/// The `IssueStateUpdateInput` one status target asks for.
4699///
4700/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
4701/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
4702/// a currently-closed issue: without that the item would read back `Unknown` and a copy
4703/// would report a change forever. A document has no status at all, and asks for neither.
4704fn state_input(target: Option<&StatusTarget>) -> Value {
4705 match target {
4706 Some(StatusTarget::Closed(reason)) => {
4707 json!({"value":"CLOSED","stateReason":reason.reason()})
4708 }
4709 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
4710 // A document has no status, so a write of one says nothing about the issue's open
4711 // or closed state rather than forcing it open: `stateInput` is what carries that
4712 // instruction, and an explicit null asks for no change to it.
4713 None => Value::Null,
4714 }
4715}
4716
4717/// The metadata one write stores in the item's body slot.
4718///
4719/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
4720/// rather than carried: the kind marker so an empty project stays readable, the
4721/// repository list only when it is not exactly the issue's own repository, and the far
4722/// ends no relationship here can name.
4723fn slot_metadata(
4724 incoming: &Incoming<'_>,
4725 own_repository: Option<&Repository>,
4726 fallback: &[DependencyEdge],
4727) -> BTreeMap<String, Value> {
4728 let mut metadata = incoming.metadata.clone();
4729 metadata.remove(ORIGIN_KEY);
4730 match incoming.written.kind() {
4731 BoardKind::Work(kind) => metadata.insert(
4732 ItemKind::METADATA_KEY.to_owned(),
4733 Value::String(kind.marker().to_owned()),
4734 ),
4735 // A document is told by its title, so it carries no kind marker: that key names
4736 // what a dependency endpoint points at, and nothing may point at a document.
4737 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
4738 };
4739 let derivable = own_repository
4740 .map(|own| incoming.repositories == [own.clone()])
4741 .unwrap_or(incoming.repositories.is_empty());
4742 if derivable {
4743 metadata.remove(Repository::METADATA_KEY);
4744 } else {
4745 metadata.insert(
4746 Repository::METADATA_KEY.to_owned(),
4747 Value::Array(
4748 incoming
4749 .repositories
4750 .iter()
4751 .map(|repository| Value::String(repository.as_str().to_owned()))
4752 .collect(),
4753 ),
4754 );
4755 }
4756 if fallback.is_empty() {
4757 metadata.remove(DependencyEdge::RECORDED_KEY);
4758 } else {
4759 metadata.insert(
4760 DependencyEdge::RECORDED_KEY.to_owned(),
4761 Value::Array(
4762 fallback
4763 .iter()
4764 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
4765 .collect(),
4766 ),
4767 );
4768 }
4769 metadata
4770}
4771
4772/// Every label one item carries, from its content's own connection and nowhere else.
4773///
4774/// There is no second place to read one from: no document this source sends selects the
4775/// board's built-in `Labels` field, because GitHub derives it from the content and a draft
4776/// cannot carry one at all. The module documentation records the three schema facts that
4777/// settle it.
4778fn labels(content: &Value) -> Result<Vec<Label>, SourceError> {
4779 optional_nodes(content.get("labels"), "content labels")?
4780 .into_iter()
4781 .flatten()
4782 .map(|v| {
4783 Ok(Label {
4784 id: NativeId(required_str(v, "id")?.to_owned()),
4785 name: required_str(v, "name")?.to_owned(),
4786 color: optional_str(v, "color")?.map(str::to_owned),
4787 })
4788 })
4789 .collect()
4790}
4791
4792/// The definition of each board field one item's values are values of, in the shape a read
4793/// of the board's own `fields` gives one.
4794///
4795/// A value names its field through a fragment on that field's own type, so the type is
4796/// known from which kind of value it is: a single-select value's field is a
4797/// `ProjectV2SingleSelectField`, options and all, and a text value's is a `ProjectV2Field`.
4798/// A value whose field carried no id, or an empty one, says nothing usable and is left out.
4799fn field_definitions(field_values: &[Value]) -> Vec<Value> {
4800 field_values
4801 .iter()
4802 .filter_map(|value| {
4803 let field = value.get("field")?.as_object()?;
4804 field.get("id")?.as_str().filter(|id| !id.is_empty())?;
4805 let typename = if value.get("text").is_some() {
4806 "ProjectV2Field"
4807 } else if value.get("name").is_some() {
4808 "ProjectV2SingleSelectField"
4809 } else {
4810 return None;
4811 };
4812 let mut defined = field.clone();
4813 defined.insert("__typename".to_owned(), json!(typename));
4814 Some(Value::Object(defined))
4815 })
4816 .collect()
4817}
4818
4819fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
4820 let Some(node) = field_values
4821 .iter()
4822 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
4823 else {
4824 return Ok(None);
4825 };
4826 Ok(optional_str(node, "text")?.map(str::to_owned))
4827}
4828
4829fn valid_github_owner(owner: &str) -> bool {
4830 !owner.is_empty()
4831 && owner.len() <= 39
4832 && !owner.starts_with('-')
4833 && !owner.ends_with('-')
4834 && !owner.contains("--")
4835 && owner
4836 .bytes()
4837 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
4838}
4839
4840/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
4841/// neither of the two names a path segment already means.
4842fn valid_github_repository_name(name: &str) -> bool {
4843 !name.is_empty()
4844 && name.len() <= 100
4845 && name != "."
4846 && name != ".."
4847 && name
4848 .bytes()
4849 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
4850}
4851
4852fn valid_environment_name(name: &str) -> bool {
4853 let mut bytes = name.bytes();
4854 bytes
4855 .next()
4856 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
4857 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
4858}
4859
4860/// How many sub-issues one issue has.
4861///
4862/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
4863/// absent or non-integer one is a response this source cannot read — and reading it as
4864/// zero would classify a project as a task, which is exactly the mistake the marker
4865/// exists to keep from happening quietly.
4866fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
4867 let summary = issue
4868 .get("subIssuesSummary")
4869 .ok_or_else(|| SourceError::Malformed {
4870 message: "GitHub issue is missing subIssuesSummary".into(),
4871 })?;
4872 summary
4873 .get("total")
4874 .and_then(Value::as_u64)
4875 .ok_or_else(|| SourceError::Malformed {
4876 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
4877 })
4878}
4879
4880fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
4881 value
4882 .get(field)
4883 .and_then(Value::as_str)
4884 .ok_or_else(|| SourceError::Malformed {
4885 message: format!("GitHub response is missing string field {field}"),
4886 })
4887}
4888
4889/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
4890/// needs one — Linear spells them too, in its own description field.
4891///
4892/// Restated rather than shared, because a plugin crate depends on the contract crate and
4893/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
4894/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
4895/// source round-trips its own writes perfectly well under its own spelling.
4896const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
4897const METADATA_CLOSE: &str = "\n-->";
4898
4899/// The visible body and the metadata slot at the end of it.
4900///
4901/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
4902/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
4903/// own content and is left alone.
4904fn metadata_body(
4905 body: Option<String>,
4906) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
4907 let Some(body) = body else {
4908 return Ok((None, BTreeMap::new()));
4909 };
4910 let Some(start) = body.rfind(METADATA_OPEN) else {
4911 return Ok((Some(body), BTreeMap::new()));
4912 };
4913 let encoded_start = start + METADATA_OPEN.len();
4914 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
4915 return Err(SourceError::Malformed {
4916 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
4917 });
4918 };
4919 let encoded_end = encoded_start + relative_end;
4920 if !body[encoded_end + METADATA_CLOSE.len()..].trim().is_empty() {
4921 return Ok((Some(body), BTreeMap::new()));
4922 }
4923 let metadata = serde_json::from_str(&body[encoded_start..encoded_end]).map_err(|error| {
4924 SourceError::Malformed {
4925 message: format!(
4926 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
4927 ),
4928 }
4929 })?;
4930 let visible = body[..start].trim_end();
4931 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
4932}
4933
4934fn compose_body(
4935 content: Option<&str>,
4936 metadata: &BTreeMap<String, Value>,
4937) -> Result<Option<String>, SourceError> {
4938 let visible = content.unwrap_or_default();
4939 if metadata.is_empty() {
4940 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
4941 }
4942 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
4943 message: error.to_string(),
4944 })?;
4945 Ok(Some(if visible.is_empty() {
4946 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4947 } else {
4948 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4949 }))
4950}
4951
4952fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
4953 value
4954 .get(field)
4955 .and_then(Value::as_bool)
4956 .ok_or_else(|| SourceError::Malformed {
4957 message: format!("GitHub response is missing boolean field {field}"),
4958 })
4959}
4960fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
4961 match value.get(field) {
4962 None | Some(Value::Null) => Ok(None),
4963 Some(value) => value
4964 .as_str()
4965 .map(Some)
4966 .ok_or_else(|| SourceError::Malformed {
4967 message: format!("GitHub response field {field} is not a string or null"),
4968 }),
4969 }
4970}
4971fn optional_nodes<'a>(
4972 connection: Option<&'a Value>,
4973 name: &str,
4974) -> Result<Option<&'a Vec<Value>>, SourceError> {
4975 match connection {
4976 None | Some(Value::Null) => Ok(None),
4977 Some(value) => value
4978 .get("nodes")
4979 .and_then(Value::as_array)
4980 .map(Some)
4981 .ok_or_else(|| SourceError::Malformed {
4982 message: format!("GitHub {name}.nodes is not an array"),
4983 }),
4984 }
4985}
4986fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
4987 let page_info = connection
4988 .get("pageInfo")
4989 .ok_or_else(|| SourceError::Malformed {
4990 message: format!("GitHub {name} has no pageInfo"),
4991 })?;
4992 if required_bool(page_info, "hasNextPage")? {
4993 return Err(SourceError::Malformed {
4994 message: format!(
4995 "GitHub {name} exceeds the supported nested connection size of {size}"
4996 ),
4997 });
4998 }
4999 Ok(())
5000}
5001fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
5002 optional_str(value, field)?
5003 .map(|timestamp| {
5004 timestamp.parse().map_err(|error| SourceError::Malformed {
5005 message: format!("GitHub response field {field} is not a timestamp: {error}"),
5006 })
5007 })
5008 .transpose()
5009}
5010fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
5011 if page.limit == 0 {
5012 Err(SourceError::Config {
5013 message: "page limit must be at least 1".into(),
5014 })
5015 } else {
5016 Ok(())
5017 }
5018}
5019fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
5020 let page = connection
5021 .get("pageInfo")
5022 .filter(|value| value.is_object())
5023 .ok_or_else(|| SourceError::Malformed {
5024 message: "GitHub connection is missing pageInfo".into(),
5025 })?;
5026 if required_bool(page, "hasNextPage")? {
5027 let cursor = required_str(page, "endCursor")?;
5028 validate_cursor_progress(None, cursor)?;
5029 Ok(Some(Cursor(cursor.into())))
5030 } else {
5031 Ok(None)
5032 }
5033}
5034fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
5035 if next.is_empty() || previous == Some(next) {
5036 Err(SourceError::Malformed {
5037 message: "GitHub pagination cursor is empty or did not advance".into(),
5038 })
5039 } else {
5040 Ok(())
5041 }
5042}
5043fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
5044 cursor.map_or(Ok(0), |c| {
5045 c.0.parse().map_err(|_| SourceError::Config {
5046 message: "page cursor is invalid".into(),
5047 })
5048 })
5049}
5050fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
5051 if offset > items.len() {
5052 return Page::last(vec![]);
5053 }
5054 let tail = items.split_off(offset);
5055 let mut selected = tail;
5056 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
5057 selected.truncate(limit);
5058 Page {
5059 items: selected,
5060 next,
5061 }
5062}