onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so [`GitHubProjectsConfig::repository`] names the one repository this
11//! source creates its project and task issues in; a write without it is refused naming
12//! the field.
13//!
14//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
15//! board has no document type and nothing but issues to hold one in, so the title is the
16//! discriminator and it is the whole of it. The title this source *reports* is the one a
17//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
18//! the body so `content` is what the person wrote — and writing a document puts the prefix
19//! back, so a round trip returns the title that went in.
20//!
21//! **Telling a document from a project from a task.** The design prefix is read **first**:
22//! a document is never a project and never a task, whatever sub-issues it has or does not
23//! have. Only then does the rest apply — a board issue is a project when *either* it has
24//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
25//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
26//! necessary: it is what makes an *empty* project — the state a project copy passes
27//! through between creating the project and filing its first task — readable as a
28//! project, while the sub-issue arm lets a person author a project on the board by hand
29//! with no knowledge of this product's metadata at all. Reading the prefix later than the
30//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
31//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
32//! task nor a document and are ignored.
33//!
34//! **Where an entity is, is a link.** Every project, task and document this source reports
35//! carries a [`Location::Url`] naming the issue's own web address — the same address the
36//! `url` field already reports, in the shape that says a reader can open it. That is the
37//! contrast the location contract exists for: a reader holding an entity from this source
38//! is handed something to link to and one holding an entity from a folder of Markdown is
39//! handed a path, and neither has to know which plugin answered. It does not replace or
40//! derive from `url`; that field goes on reporting what it always reported.
41//!
42//! **Where metadata lives.** Short typed things go to typed fields and native relations:
43//! status to the board's `Status` single-select and the issue's own state, the copy
44//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
45//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
46//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
47//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
48//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
49//! why neither can hold a caller's own prose.
50//!
51//! **Status.** `status_mapping` is per-instance configuration from a status category to
52//! `null`, a board `Status` option name, or a closed state of `completed` or
53//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
54//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
55//! and a mistake destroys every item's status. A status this board cannot represent is a
56//! refusal naming the status and the instance instead.
57//!
58//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
59//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
60//! finished tasks were only moved to a "Done" column would read 0% complete forever.
61//!
62//! # What this source declares, field by field
63//!
64//! One verdict per field of [`Capabilities`], and what `Native` means when this source
65//! says it. *Proven* means a shared journey drives it against the real
66//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
67//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
68//! [`capabilities`](TaskSource::capabilities) from parting.
69//!
70//! | Field | Verdict |
71//! | --- | --- |
72//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
73//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
74//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
75//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
76//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
77//! | `search_title` | **Supported and proven,** over `Issue.title`. |
78//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
79//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
80//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
81//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
82//!
83//! Nothing here is unsupported. `documents` is not a predicate — it says this source has
84//! documents, which it does — and the three facts behind the uniform `Native` on the
85//! predicates beside it are recorded below rather than re-derived, because a reader who
86//! takes `Native` to mean *the remote service filters* will read that uniformity as a
87//! lie.
88//!
89//! First, the plugin contract defines `Support::Native` as *the source applies this
90//! predicate itself*, and says nothing about where it applies it. What the declaration
91//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
92//! — so that the engine may push it down and apply nothing of its own.
93//!
94//! Second, this source can keep that promise for every predicate at no additional API
95//! cost, because whichever of the three reads below answers a query has already read every
96//! item that query could keep before it filters anything. Filtering those items is
97//! in-process work over data already in hand.
98//!
99//! Third, no predicate but `projects` could be pushed into the API even if that were
100//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
101//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
102//! label set, a status column or a substring of a body, and its title qualifier matches
103//! tokens where this source — and the local Markdown source beside it — match substrings,
104//! so pushing a search down would silently *narrow* the answer. What a project filter has
105//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
106//! the issue for them is both cheaper and exact. So there is one predicate this source
107//! applies by asking a narrower question, six it applies in process, and none it is unable
108//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
109//! source has already done, and declaring `projects` native while ignoring the filter
110//! (which this source once did) silently returns another project's tasks, because the
111//! engine trusts the declaration and applies nothing locally.
112//!
113//! # The three ways this source reaches an item, and what each costs
114//!
115//! A board read is charged for what its *nested* connections could return rather than for
116//! what was asked, so one whole-board read costs the same whether the question was about
117//! one project or about all of them. That is why a question about one project is never
118//! answered by reading the board:
119//!
120//! | The question | What is sent | What it costs |
121//! | --- | --- | --- |
122//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
123//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
124//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
125//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
126//! | which board item one issue is, past the page that came with it | [`graphql::ISSUE_BOARD_ITEMS`] — that issue's own `projectItems` | one issue's memberships |
127//!
128//! The board half of an issue — its board item's id, its `Status` option and this
129//! source's origin text field — rides along on `Issue.projectItems` in the first three, so
130//! an item reached any of those ways resolves through the same
131//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
132//! same status, the same labels and the same qualified id. That connection comes back a
133//! *page* at a time, at `BOARD_ITEMS_PAGE_SIZE`, so the entry for this board is looked for
134//! on the page in hand and — only if that page reports more of the connection — in the
135//! last row's read of that one issue's memberships, resumed from the page's own cursor and
136//! walked to exhaustion. An issue with no entry for *this* board is not this source's to
137//! report, which is what keeps an id naming another repository's issue from being answered
138//! as an item of this board; and because the page is where the search starts rather than
139//! where it ends, that answer is one about a connection read to exhaustion and never about
140//! an unread page. Nothing costs the extra read but an issue on more boards than a page
141//! holds: an issue this board really does not hold reports no next page, so its
142//! memberships are already exhausted where they arrived.
143//!
144//! **No document here selects the board's own `Labels` field, and nothing is lost by
145//! that.** An item's labels are read from its content alone, wherever that content is
146//! reached: the three documents above select `Issue.labels` on the fragment, and
147//! [`graphql::BOARD`] selects the same connection on the `... on Issue` arm of its
148//! `content`. A board's `Labels` field is not one anybody fills in: it is a built-in
149//! `ProjectV2FieldType`, it is absent from `ProjectV2CustomFieldType` so no project can
150//! create one, and `ProjectV2FieldValue` — the whole of what
151//! `updateProjectV2ItemFieldValue` accepts — offers no way to write one. So GitHub derives
152//! it from the content, for every content type it exists on, and there is nothing it can
153//! hold that the content does not already say: for an `Issue` it *is* that issue's own
154//! labels, so selecting it beside them unions a set with itself.
155//!
156//! **A draft loses nothing by that either**, which is the reasoning this paragraph once had
157//! backwards. `DraftIssue` exposes no `labels` field, and by the three schema facts above
158//! it cannot carry a board `Labels` value to be derived from one — so a draft has nothing
159//! to select *and nothing to lose*, and reports no labels at all. A `PullRequest` item is
160//! discarded by [`GitHubProjectsSource::resolve`] before labels are read. Both halves are
161//! held to that by tests in `tests/plugin.rs`: the four ways an item is reached report one
162//! label set, and that set is the fixture issue's own, by
163//! `an_item_reports_the_same_labels_title_status_and_id_however_it_is_reached`; and a board
164//! item whose content is a draft reports an empty set, by
165//! `a_board_item_whose_content_is_a_draft_reports_no_labels_at_all`. The absence of the
166//! selection is held over [`graphql::DOCUMENTS`] by
167//! `no_document_selects_the_boards_own_labels_field`.
168//!
169//! The last row is still the board's own item connection, and deliberately: a **draft**
170//! board item is not an issue, so no search and no node read can reach one, and the reads
171//! that have to answer for the whole board are the ones whose cost is the board's size
172//! anyway.
173//!
174//! **What a read may return is capped too, and that cap is on the document rather than on
175//! the board.** GitHub limits the number of nodes **one query may return** to
176//! [`NODE_COUNT_LIMIT`] and refuses a query above that before executing it: the answer is
177//! an error naming the connection the count crossed at, not a slow or a partial result.
178//! Every board this source reads is refused the same way, so no board is too big for these
179//! documents and none is small enough to save one that is over.
180//!
181//! The count is arithmetic over the document's own text: each connection contributes the
182//! `first:` it asks for, counts **multiply** down a nested path and **sum** across sibling
183//! paths. Those are [GitHub's published rules][node-limits] and this workspace does not
184//! restate them — `github-graphql-node-count` implements them, and
185//! [`worst_case_node_count`] under [`largest_page_sizes`] is where every node count here
186//! comes from. `every_document_this_source_sends_stays_under_githubs_node_limit`, in
187//! `tests/node_count.rs`, recomputes every document in [`graphql::DOCUMENTS`] from that
188//! same text on every run and fails naming any that reaches the limit — so a connection
189//! added to a shared fragment is caught there rather than by GitHub.
190//!
191//! What decides those counts is the page sizes: [`MAX_PAGE_SIZE`] on the outer page,
192//! `NESTED_PAGE_SIZE` on the connections hanging off one item, and
193//! `BOARD_ITEMS_PAGE_SIZE` on the page of an issue's board memberships a read carries.
194//! `$nestedFirst` is spent twice down one path of a board read, so that constant is
195//! effectively squared there, which is why it is the one the limit is most sensitive to.
196//! `BOARD_ITEMS_PAGE_SIZE` is small for a reason of its own, recorded beside it: what a
197//! page of memberships misses is recovered by one further read rather than refused, so it
198//! buys a bound every read pays for at the price of a request only a multi-board issue
199//! pays.
200//!
201//! **`nodeCount` and `cost` are two numbers against two limits, and none of this is about
202//! the second.** `nodeCount` is the one above: the most nodes one query may return,
203//! checked per query. `cost` is rate-limit points, metered per hour across everything one
204//! credential does; it is what the two limiters [`Limiter`] tells apart meter, and a
205//! document under [`NODE_COUNT_LIMIT`] says nothing about it.
206//!
207//! [node-limits]: https://docs.github.com/en/graphql/overview/rate-limits-and-node-limits-for-the-graphql-api
208//!
209//! **Where a read-after-write guarantee comes from, since a search index cannot supply
210//! one.** GitHub's issue search is eventually consistent and answers a write made moments
211//! ago with the value from before it. Resolving a node id is not, so a read by id and a
212//! project's own sub-issues are already current. What closes the gap for the search is
213//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
214//! what this process itself wrote, so an item created seconds ago is reported whether or
215//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
216//! and the record dies with the process.
217//!
218//! Filtering happens before paging, so a page of a filtered result is a page of the
219//! survivors rather than the survivors of a page. Label and text matching answer the same
220//! question the same way the local Markdown source's do, so one cross-source expectation
221//! holds for both.
222//!
223//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
224//! has one source, `capabilities`, and the note above is the reasoning behind it rather
225//! than a second copy of it: without the three facts recorded here a reader takes the
226//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
227//! crate's own capabilities test, which pins every field of it against a fully spelled-out
228//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
229//! fails to compile there rather than going unasserted. -->
230//! The fixture-server tests above run wherever this crate is selected; the credentialed
231//! lane runs in the same required check, beside them, and can fail it — it verifies the
232//! current schema, then drives every field of the table above against the real board. It builds its own fixture there — two projects, one task filed under each,
233//! one filed under neither, a label on one of the three and a closed status on another —
234//! because that shape is what tells an honoured predicate from an ignored one: a board
235//! holding a single project answers a project filter the same way whether or not this
236//! source applies it, which is exactly how the defect above went unseen.
237//!
238//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
239//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
240//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
241//! nominated is what keeps a credentialed write lane off a board and a repository nobody
242//! nominated; it never asks GitHub which project was updated most recently. Before it
243//! starts, the lane also clears any item titled — and any repository label named — the way
244//! it titles and names its own artifacts, which is self-healing after an interrupted run:
245//! a process killed between its writes and its cleanup leaves artifacts the next run
246//! removes.
247//!
248//! # What a session of requests costs, and where the report is
249//!
250//! This source records **every** request it sends into [`accounting::Accounting`], at
251//! `send_once` — the one place a request leaves this crate, which is why a read path added
252//! later is counted without anybody remembering to count it. That is the whole of what this
253//! crate adds to the arrangement; [`accounting`] is where what a record carries, how a
254//! session's spend is arrived at, and what it deliberately does not know are set out.
255//!
256//! What one whole session of the live journey costs, counted that way against this crate's
257//! loopback fixture board, is written down in `session-cost.md` beside this crate — with the
258//! reduction it came out of, and with what it does and does not say about rate-limit points.
259//!
260//! [`GitHubProjectsSource::accounting`] is the read: a snapshot to hold and compare, which
261//! [`accounting::Session::report`] renders the session report from. It is on the ordinary
262//! code path — no environment variable, no feature, no build configuration — because an
263//! instrument nobody switches on measures nothing, and
264//! [`Plugin::build_recording_into`] is how a caller making its own calls beside this
265//! source's counts the whole session rather than this source's share. The credentialed lane
266//! in `tests/live.rs` does exactly that, and prints the report at the end of every run,
267//! passed or failed.
268//!
269//! **A live session refuses to start unless the account can afford it.** Before it does any
270//! of the work it exists to do, the journey makes one request — `GET /rate_limit`, which
271//! GitHub documents as not counting against the REST rate limit and which answers both of
272//! its budgets at once — and starts only if, for each of them, what remains minus this
273//! session's estimated cost is still at least
274//! `onetaskgraph_live::RETAINED_BUFFER` — twenty per cent — of that budget's whole
275//! allowance. A session that cannot **declines**: it did not run, so it is
276//! neither a pass nor a failing assertion, and it says which budget was short, that budget's
277//! limit, what remained, the estimate, the buffer and when it resets — then stops, without
278//! waiting for the budget to come back. The estimate is derived offline from
279//! `tests/fixtures/session-cost.txt` and a cost model stated in `tests/journey/budget.rs`,
280//! which is also where the published rule that model rests on is cited; the accounting
281//! above records the gate's own read like any other request, and
282//! [`accounting::Session::report`] prints the estimate beside what the session really spent.
283//!
284//! **GitHub is the authority on node count, and the credentialed lane goes and asks it.**
285//! Everything above computes `nodeCount` offline from a document's own text, which is what
286//! lets it run on every platform and on a pull request from a fork with no credential — and
287//! that is what actually stops a regression merging. But an offline arithmetic can only
288//! ever agree with itself: if GitHub changes its rules, this workspace goes on computing
289//! the old answer and nothing notices. So `tests/live.rs` reconciles the two. GitHub's
290//! schema exposes `rateLimit(dryRun: true)`, whose `nodeCount` is *"the maximum number of
291//! nodes this query may return"* for a document **without executing it**, and the lane asks
292//! it for every query document this source sends, under the largest bindings this source
293//! sends, and fails when GitHub's number and [`worst_case_node_count`] disagree. It records
294//! what those calls reported about the account's own allowance, because whether asking is
295//! free is a thing to observe rather than to assume. Two quantities, not one:
296//! [`NODE_COUNT_LIMIT`] bounds `nodeCount`, and the accounting above measures `cost`.
297//!
298//! **GitHub has two rate limiters and this source is refused by both, so nothing here
299//! treats them as one thing.** The primary budget is the hourly allowance `gh api
300//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
301//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
302//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
303//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
304//! one part of.
305#![deny(missing_docs)]
306
307use std::collections::BTreeMap;
308use std::sync::{Arc, Mutex};
309use std::time::{Duration, Instant};
310
311use chrono::{DateTime, Utc};
312use onetaskgraph_plugin_api::{
313 Capabilities, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind, DependencySupport,
314 Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label, LabelFilter, Location,
315 NativeId, Page, PageRequest, Project, ProjectFilter, ProjectQuery, Repository, SecretResolver,
316 SourceError, SourceName, SourcePlugin, Status, StatusCategory, Support, Task, TaskQuery,
317 TaskSource, TextFields, TextQuery, WriteSupport,
318};
319use reqwest::{Client, StatusCode, Url};
320use schemars::{Schema, schema_for};
321use secrecy::{ExposeSecret, SecretString};
322use serde::Deserialize;
323use serde_json::{Value, json};
324
325pub mod accounting;
326
327use accounting::Accounting;
328
329/// The registry name for this plugin.
330pub const KIND: &str = "github-projects";
331/// GitHub's maximum connection page size.
332pub const MAX_PAGE_SIZE: u32 = 100;
333
334/// The most nodes any one document this source sends may be asked to return.
335///
336/// GitHub's own published per-query ceiling, taken from
337/// [`github_graphql_node_count::NODE_LIMIT`] rather than written out again here, so this
338/// workspace cannot hold a stale copy of somebody else's number. A query above it is
339/// **refused before it is executed**, whoever is asking and whatever board they are
340/// asking about — so this is a bound on the documents rather than a budget that runs out.
341///
342/// This is `nodeCount`, the maximum number of nodes *one query may return*. It is not
343/// `cost`, the rate-limit points a call spends against an hourly allowance shared by
344/// everything the credential does. Two numbers, two limits; nothing here is about the
345/// second. The module section on the three ways this source reaches an item says how the
346/// count is arrived at, and which of the page sizes below decide it.
347pub const NODE_COUNT_LIMIT: u64 = github_graphql_node_count::NODE_LIMIT;
348
349/// Nested connection size for the connections that hang off one item.
350///
351/// It multiplies through every document that reaches an item under a page — the count
352/// rules multiply down a nested path — so it is the constant [`NODE_COUNT_LIMIT`] is most
353/// sensitive to. `tests/node_count.rs` is what holds the pair together: it recomputes
354/// every document under these constants and fails naming any that reaches the limit, so
355/// raising this is caught there rather than by GitHub.
356const NESTED_PAGE_SIZE: u32 = 50;
357/// How many of one issue's board memberships are read when an issue is reached directly.
358///
359/// An issue reached through a search or through its own node id carries its board half in
360/// `Issue.projectItems`, and only the entry for *this* board is read. This connection sits
361/// under a page of issues, so every point of it multiplies through the whole document and
362/// is paid for whether or not any issue is on a second board — which is why it is
363/// deliberately far smaller than [`NESTED_PAGE_SIZE`].
364///
365/// **Three, because what a page misses is now recovered rather than refused**, and the
366/// recovery is what the value is chosen against. An issue whose entry for this board sits
367/// past this page costs one further request — [`graphql::ISSUE_BOARD_ITEMS`], resumed from
368/// that page's own cursor — so the value trades a bound every read pays for a request only
369/// a multi-board issue pays. At one, a deployment whose issues commonly sit on two or more
370/// boards would pay that request *per issue*, which is order N against the one page per
371/// hundred issues a read costs today. At three it is only reached by an issue on four or
372/// more boards at once, which keeps the recovery path exceptional rather than routine for
373/// a plausible deployment.
374const BOARD_ITEMS_PAGE_SIZE: u32 = 3;
375
376pub use github_graphql_node_count::{NodeCountError, Variables};
377
378/// The largest value this source can bind to each page-size variable its documents name.
379///
380/// Every `first:` in [`graphql`] reads one of these three, and each is capped at the
381/// constant above it wherever a caller's own limit could reach it — `$first` at
382/// [`MAX_PAGE_SIZE`], `$nestedFirst` at `NESTED_PAGE_SIZE`, `$boardItems` at
383/// `BOARD_ITEMS_PAGE_SIZE`. So this is the worst case a caller can drive this source to,
384/// not one configuration of it, which is what makes a bound computed under it a bound on
385/// every read.
386pub fn largest_page_sizes() -> Variables {
387 Variables::from([
388 ("first".to_owned(), MAX_PAGE_SIZE),
389 ("nestedFirst".to_owned(), NESTED_PAGE_SIZE),
390 ("boardItems".to_owned(), BOARD_ITEMS_PAGE_SIZE),
391 ])
392}
393
394/// The most nodes `document` could be asked to return, by GitHub's published rules.
395///
396/// Computed offline from the document's own text under [`largest_page_sizes`] — no
397/// network, no credential and no schema — by
398/// [`github_graphql_node_count::node_count`], which is where the rules themselves live.
399/// A document at or above [`NODE_COUNT_LIMIT`] is one GitHub refuses before executing, so
400/// this is what a check holds every document in [`graphql::DOCUMENTS`] below.
401///
402/// # Errors
403///
404/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
405/// no single operation, or binds a page size this source does not name — each of which is
406/// a defect in the document rather than a number.
407pub fn worst_case_node_count(document: &str) -> Result<u64, NodeCountError> {
408 node_count(document, &largest_page_sizes())
409}
410
411/// The most nodes `document` could be asked to return under `variables`.
412///
413/// [`worst_case_node_count`] is this under [`largest_page_sizes`], and the accounting in
414/// [`accounting`] is this under the bindings one request really sent — one spelling of the
415/// calculation, so a bound checked offline and a cost recorded at run time cannot come to
416/// disagree. The rules themselves live in [`github_graphql_node_count::node_count`].
417///
418/// # Errors
419///
420/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
421/// no single operation, or binds a page size `variables` does not name.
422pub fn node_count(document: &str, variables: &Variables) -> Result<u64, NodeCountError> {
423 github_graphql_node_count::node_count(document, variables)
424}
425
426/// The issue-title prefix that makes a board issue a document.
427///
428/// A GitHub Projects board has no document type — it holds issues — so the discriminator
429/// is the title, and this is the whole of it: an issue whose title begins with these bytes
430/// is a document and every other issue is the task or project the sub-issue rule makes it.
431///
432/// It is spelled **once**, here, and read rather than restated everywhere else — including
433/// by the shared journeys, which take it from this constant so a board fixture cannot
434/// drift from what this source reads. `docs/metadata.md` records the two consequences that
435/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
436/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
437/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
438pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
439
440/// Exact GraphQL query documents issued by this plugin.
441///
442/// Keeping the production documents here lets the pinned-schema test validate the same
443/// bytes that are sent to GitHub, rather than a test-only copy which could drift
444/// independently. No document in this module writes the board itself, and none of them
445/// names `updateProjectV2Field`.
446pub mod graphql {
447 /// The board half of one item: the field values every document here reads it from.
448 ///
449 /// A macro for the same reason [`board_issue!`] below is one, a level further in. This
450 /// selection is needed by that fragment, by [`BOARD`] under the board's own `items`,
451 /// and by [`ISSUE_BOARD_ITEMS`] under a membership walk — and all three have to produce
452 /// *the same value*, because
453 /// [`GitHubProjectsSource::resolve`](super::GitHubProjectsSource) reads them through
454 /// one path. Three spellings of it is what would drift, so there is one.
455 ///
456 /// The `Status` option and this source's own origin text field are the whole of it. It
457 /// selects no `ProjectV2ItemFieldLabelValue`: GitHub derives that field from the item's
458 /// content, so it holds nothing the content's own `labels` do not already say, and it
459 /// would sit a label connection two page sizes deep.
460 macro_rules! board_item_values {
461 () => {
462 r#"fieldValues(first:$nestedFirst){nodes{
463 ... on ProjectV2ItemFieldSingleSelectValue{name field{
464 ... on ProjectV2SingleSelectField{id name options{id name}}
465 }}
466 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
467 }pageInfo{hasNextPage}}"#
468 };
469 }
470
471 /// Everything this source reads about one issue, wherever it reaches that issue.
472 ///
473 /// A macro rather than a constant so the three documents below can `concat!` it: one
474 /// spelling of these fields is what makes an issue read through the board-scoped
475 /// search, through its own node id, and through its project's sub-issue relationship
476 /// resolve to *the same* item, which is the whole of what
477 /// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
478 ///
479 /// `projectItems` is what carries the board half of an issue: the board item's own id
480 /// and the [`board_item_values!`] above — the `Status` option and this source's origin
481 /// text field — that a `ProjectV2.items` read used to carry. It is asked for on the
482 /// issue rather than on the board, which is what makes the cost of a read proportional
483 /// to what was asked for instead of to the board's size.
484 ///
485 /// It carries a *page* of that connection, at `BOARD_ITEMS_PAGE_SIZE`, and its
486 /// `endCursor` is what [`ISSUE_BOARD_ITEMS`] resumes from when this board's entry is
487 /// not on that page: a page here is where the search for the entry starts rather than
488 /// where it ends.
489 ///
490 /// It does **not** select the board's `Labels` field value, and that is the whole of
491 /// what keeps the three documents below under [`NODE_COUNT_LIMIT`](super::NODE_COUNT_LIMIT):
492 /// a label connection there sits under `fieldValues` under `projectItems` under a page
493 /// of issues, spending `$nestedFirst` twice down one path, and took
494 /// [`SEARCH_ISSUES`] and [`SUB_ISSUES`] to 2,556,100 nodes against a limit of 500,000.
495 /// No label is lost — this is a fragment `on Issue`, whose own `labels` are selected
496 /// above, and that connection is where every label this source reports comes from. No
497 /// document in this module selects the board field any longer, [`BOARD`] included; the
498 /// module documentation records why nothing it could have held is lost.
499 macro_rules! board_issue {
500 () => {
501 concat!(
502 r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
503 labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
504 projectItems(first:$boardItems){nodes{id project{number}
505 "#,
506 board_item_values!(),
507 r#"}pageInfo{hasNextPage endCursor}}}"#
508 )
509 };
510 }
511
512 /// Every issue of one board, found by a search scoped to that board.
513 ///
514 /// This is how the projects a board holds are listed, and it selects no `items`
515 /// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
516 /// container walked page by page, so nothing nested inside a board item is paid for.
517 /// Which of the issues it returns is a project is then read off `parent` — GitHub
518 /// accepts `-has:parent` as a search qualifier and silently ignores it, so the
519 /// discriminator has to be applied to the field, which is a scalar on the issue and
520 /// costs nothing.
521 pub const SEARCH_ISSUES: &str = concat!(
522 r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
523 search(query:$search,type:$type,first:$first,after:$after){
524 pageInfo{hasNextPage endCursor}
525 nodes{__typename ...BoardIssue}
526 }
527 }"#,
528 board_issue!()
529 );
530
531 /// One issue by its own node id, which is what a qualified id names here.
532 ///
533 /// Strongly consistent, unlike the search above: GitHub's issue search is an index and
534 /// answers a write made moments ago with the value from before it, and resolving a node
535 /// id does not.
536 pub const ISSUE: &str = concat!(
537 r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
538 node(id:$id){__typename ...BoardIssue}
539 }"#,
540 board_issue!()
541 );
542
543 /// One project's tasks: the sub-issues of the issue that project is.
544 ///
545 /// The work this costs is the project's own size. Nothing about it grows as the board
546 /// gains projects, or as those projects gain tasks.
547 pub const SUB_ISSUES: &str = concat!(
548 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
549 node(id:$id){__typename
550 ... on Issue{subIssues(first:$first,after:$after){
551 pageInfo{hasNextPage endCursor}
552 nodes{__typename ...BoardIssue}
553 }}}
554 }"#,
555 board_issue!()
556 );
557
558 /// Reads the board's fields and one page of its items.
559 pub const BOARD: &str = concat!(
560 r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
561 owner:repositoryOwner(login:$owner){
562 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
563 }
564 } fragment Board on ProjectV2 { id title
565 fields(first:$nestedFirst){nodes{
566 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
567 ... on ProjectV2Field{__typename id name}
568 }pageInfo{hasNextPage}}
569 items(first:$first,after:$after){nodes{id "#,
570 board_item_values!(),
571 r#" content{
572 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
573 ... on PullRequest{__typename id}
574 ... on DraftIssue{__typename id title body createdAt updatedAt}
575 }} pageInfo{hasNextPage endCursor}}
576 }"#
577 );
578
579 /// One issue's board memberships alone, walked past the page a read of it carried.
580 ///
581 /// The recovery read behind [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource):
582 /// every document above carries a *page* of `Issue.projectItems`, and an issue on more
583 /// boards than that page holds may have this board's entry past its end. This asks that
584 /// one issue for its memberships and nothing else — the caller already holds the issue —
585 /// so an answer of "this board does not hold it" is only ever given about a connection
586 /// read to exhaustion.
587 ///
588 /// It selects the board item's id, its project number and the same
589 /// [`board_item_values!`] the fragment does, because what it produces is handed to the
590 /// very same resolver: an issue recovered this way reports the same title, the same
591 /// status, the same labels and the same qualified id as one whose entry was on the
592 /// page.
593 ///
594 /// `$first` rather than `$boardItems`: this document reads one issue, so nothing
595 /// multiplies through it and the membership connection can be walked at
596 /// [`MAX_PAGE_SIZE`](super::MAX_PAGE_SIZE) — which is what keeps the recovery to one
597 /// further request for any issue a person really keeps.
598 pub const ISSUE_BOARD_ITEMS: &str = concat!(
599 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!){
600 node(id:$id){
601 ... on Issue{projectItems(first:$first,after:$after){
602 nodes{id project{number}
603 "#,
604 board_item_values!(),
605 r#"}
606 pageInfo{hasNextPage endCursor}}}
607 }
608 }"#
609 );
610 /// Resolves the configured repository's node id, which creating an issue requires.
611 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
612 /// Reads both dependency directions for one issue, with each far end's own kind.
613 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
614 ... on Issue{
615 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
616 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
617 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
618 /// Creates one issue in the configured repository.
619 pub const CREATE_ISSUE: &str =
620 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
621 /// Puts an existing issue on the configured board.
622 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
623 /// Updates an issue's visible fields and its open or closed state in one call.
624 pub const UPDATE_ISSUE: &str =
625 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
626 /// Updates an existing draft's user-visible fields.
627 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
628 /// Updates a text or single-select value on one project item.
629 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
630 /// Files one issue under another as a sub-issue, which is what project membership is.
631 pub const ADD_SUB_ISSUE: &str =
632 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
633 /// Takes one issue back out of its parent.
634 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
635 /// Adds GitHub's native issue blocked-by relationship.
636 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
637 /// Removes one native issue blocked-by relationship.
638 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
639 /// Deletes one issue, which takes its board item with it.
640 ///
641 /// The engine sends this in one situation only: undoing a copy that could not finish,
642 /// over the items that same copy created. Deleting the issue removes the board item
643 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
644 pub const DELETE_ISSUE: &str =
645 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
646
647 /// Every document above, with what this source is doing when it sends one.
648 ///
649 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
650 /// name the call that was refused, and a `match` with a catch-all arm would answer a
651 /// document added later with "talking to GitHub" and never say so.
652 ///
653 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
654 /// const` here that this list omits, so the two cannot part — which is the same guard
655 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
656 pub const DOCUMENTS: [(&str, &str); 17] = [
657 (SEARCH_ISSUES, "searching this board's issues"),
658 (ISSUE, "reading one issue"),
659 (
660 ISSUE_BOARD_ITEMS,
661 "reading one issue's board memberships past the page it came with",
662 ),
663 (SUB_ISSUES, "reading a project's tasks"),
664 (BOARD, "reading the board"),
665 (REPOSITORY, "reading the destination repository"),
666 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
667 (CREATE_ISSUE, "creating an issue"),
668 (ADD_TO_BOARD, "adding an issue to the board"),
669 (UPDATE_ISSUE, "updating an issue"),
670 (UPDATE_DRAFT, "updating a draft item"),
671 (UPDATE_FIELD, "writing a board field"),
672 (ADD_SUB_ISSUE, "filing an issue under its project"),
673 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
674 (ADD_BLOCKED_BY, "recording a dependency"),
675 (REMOVE_BLOCKED_BY, "removing a dependency"),
676 (DELETE_ISSUE, "deleting an issue"),
677 ];
678}
679
680/// Which of GitHub's two rate limiters refused a request.
681///
682/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
683/// secondary one — so an operator told the wrong one takes the wrong next step, which is
684/// the whole reason this is carried rather than collapsed into "rate limited".
685#[derive(Debug, Clone, Copy, PartialEq, Eq)]
686enum Limiter {
687 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
688 Primary,
689 /// The burst limiter over content-generating requests, which nothing reports.
690 Secondary,
691}
692
693/// The wordings GitHub answers a secondary rate limit with.
694///
695/// It sends them under a forbidden status, under a too-many-requests status, and inside
696/// the `errors` of a *successful* response, which is why the text is what this matches on
697/// rather than the status. `abuse detection` is the wording GitHub used before the
698/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
699/// what a burst of content creation is refused with.
700///
701/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
702/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
703/// when, and the drift gate reconciles the two lists both ways. Public for that gate
704/// alone — a caller has no use for it, and matching on a refusal is this source's job.
705pub const SECONDARY_WORDINGS: [&str; 5] = [
706 "secondary rate limit",
707 "temporarily blocked from content creation",
708 "abuse detection",
709 "submitted too quickly",
710 "exceeded a secondary",
711];
712
713/// The wordings GitHub answers an exhausted primary budget with.
714///
715/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
716/// than looked for in the response text. Pinned and gated exactly as
717/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
718pub const PRIMARY_WORDINGS: [&str; 3] = [
719 "api rate limit exceeded",
720 "rate limit exceeded",
721 "rate_limited",
722];
723
724/// What a response *says about itself*, which is the only place a refusal can be read.
725///
726/// Deliberately not the whole response body. A board is a place people write about their
727/// own work, and a task on it titled "the secondary rate limit" would, matched across the
728/// raw text, turn a perfectly good answer into a refusal this source then waited out and
729/// reported. So the item data is never read: what is read is GitHub's own REST-style
730/// `message` envelope, which is what a forbidden status carries, and the `message` and
731/// `type` of each GraphQL error, which is where a *successful* response says it.
732///
733/// A body that is not JSON at all has nothing structured to read, so only a failing
734/// response's own text is taken — a successful response that is not JSON is malformed
735/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
736fn refusal_wording(status: StatusCode, body: &str) -> String {
737 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
738 return if status.is_success() {
739 String::new()
740 } else {
741 body.to_owned()
742 };
743 };
744 let mut said: Vec<&str> = parsed
745 .get("message")
746 .and_then(Value::as_str)
747 .into_iter()
748 .collect();
749 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
750 for error in errors {
751 said.extend(
752 ["message", "type"]
753 .into_iter()
754 .filter_map(|key| error.get(key).and_then(Value::as_str)),
755 );
756 }
757 }
758 said.join("; ")
759}
760
761impl Limiter {
762 /// Which limiter refused this response, or `None` when none of them did.
763 ///
764 /// The wording is read first and the status only decides what carries none of it,
765 /// because GitHub answers a secondary limit with a forbidden status far more often
766 /// than with too-many-requests — while a forbidden status saying nothing about a limit
767 /// really is a credential this token lacks.
768 ///
769 /// A response is a refusal because of its status or its own wording. A spent budget
770 /// only ever explains one; it never turns an answer into a refusal.
771 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
772 let normalized = refusal_wording(status, body).to_ascii_lowercase();
773 if SECONDARY_WORDINGS
774 .iter()
775 .any(|wording| normalized.contains(wording))
776 {
777 return Some(Self::Secondary);
778 }
779 if status == StatusCode::TOO_MANY_REQUESTS {
780 return Some(Self::Primary);
781 }
782 // An exhausted budget *explains* a response that failed; it does not make one that
783 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
784 // request the budget allowed as well as on the ones it then refuses, so reading
785 // the header alone threw away a good answer — and, once refusals were retried,
786 // replayed a request that had already taken effect.
787 if !status.is_success() && budget_exhausted {
788 return Some(Self::Primary);
789 }
790 // A successful response saying it: GitHub reports a GraphQL rate limit in the
791 // `errors` of an HTTP 200, where nothing about the status says so at all.
792 if status.is_success()
793 && PRIMARY_WORDINGS
794 .iter()
795 .any(|wording| normalized.contains(wording))
796 {
797 return Some(Self::Primary);
798 }
799 None
800 }
801
802 /// What this limiter is called where an operator can look it up.
803 const fn name(self) -> &'static str {
804 match self {
805 Self::Primary => "GitHub's primary API rate limit",
806 Self::Secondary => "GitHub's secondary rate limit",
807 }
808 }
809
810 /// What the endpoint an operator would go and check says about this limiter.
811 const fn where_to_look(self) -> &'static str {
812 match self {
813 Self::Primary => {
814 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
815 comes back."
816 }
817 Self::Secondary => {
818 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
819 primary budget and does not report this one, so budget showing there says \
820 nothing about this refusal, and every further attempt extends it."
821 }
822 }
823 }
824
825 /// The next step this limiter actually calls for.
826 const fn what_to_do(self) -> &'static str {
827 match self {
828 Self::Primary => {
829 "wait for the reset `gh api rate_limit` reports, then run the command again."
830 }
831 Self::Secondary => {
832 "leave this board alone for a few minutes, then run the command again — or \
833 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
834 }
835 }
836 }
837}
838
839/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
840#[derive(Debug, Clone, Copy)]
841struct Limited {
842 limiter: Limiter,
843 hint: Option<u64>,
844}
845
846impl Limited {
847 /// What the caller is told once this source has waited as long as it may.
848 ///
849 /// Both limiters report as [`SourceError::RateLimited`], because that is what
850 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
851 /// about *which* limiter it was makes it a different kind of failure. What differs is
852 /// the operator's next step, and that is what the message carries — a secondary
853 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
854 /// budget looks fine, and then back to retry the very burst that was refused.
855 fn exhausted(
856 self,
857 doing: &str,
858 waits: u32,
859 waited: Duration,
860 needed: Duration,
861 budget: Duration,
862 ) -> SourceError {
863 SourceError::RateLimited {
864 retry_after_seconds: self.hint,
865 message: Some(format!(
866 "{} refused this source while {doing}; it waited {} out over {} and was refused \
867 again, and the next wait of {} would take it past the {} one call may spend \
868 waiting. {} next: {}",
869 self.limiter.name(),
870 plural(waits, "refusal"),
871 seconds(waited),
872 seconds(needed),
873 seconds(budget),
874 self.limiter.where_to_look(),
875 self.limiter.what_to_do(),
876 )),
877 }
878 }
879}
880
881/// One HTTP attempt's result, with what its response said about the rate limit.
882///
883/// The two travel together so the record and the outcome are written from the same place:
884/// what a response said about the budget is only readable while that response is in hand,
885/// and what the attempt *meant* is only decidable once its body has been read.
886struct Attempted {
887 result: Result<Value, Attempt>,
888 limits: accounting::RateLimit,
889 /// GitHub's own reported cost for this call, for a document that asked for it.
890 reported_cost: Option<u64>,
891}
892
893/// One attempt's outcome: an error to report, or a rate limit to wait out.
894enum Attempt {
895 Failed(SourceError),
896 Limited(Limited),
897}
898
899fn plural(count: u32, thing: &str) -> String {
900 if count == 1 {
901 format!("{count} {thing}")
902 } else {
903 format!("{count} {thing}s")
904 }
905}
906
907fn seconds(duration: Duration) -> String {
908 format!("{:.1}s", duration.as_secs_f64())
909}
910
911/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
912///
913/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
914/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
915/// header, and neither is what makes a response a refusal — so the whole cost of one this
916/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
917/// instead. Refusing the response over the header would turn a readable refusal into an
918/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
919fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
920 value
921 .and_then(|value| value.to_str().ok())
922 .and_then(|value| value.trim().parse::<u64>().ok())
923}
924
925/// Every mutation this source sends creates content — an issue, a board item, a field of
926/// one, a sub-issue link, a dependency — and no query in [`graphql::DOCUMENTS`] does, so
927/// what the secondary limiter counts and what the keyword says are the same set. That is
928/// what makes the keyword a sound test rather than a convenient one.
929fn is_mutation(query: &str) -> bool {
930 query.trim_start().starts_with("mutation")
931}
932
933/// What this source was doing, for a diagnostic that has to say so.
934///
935/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
936/// a document added without a description is caught by that list's own gate instead of
937/// falling through to the vague arm below.
938fn operation_description(query: &str) -> &'static str {
939 graphql::DOCUMENTS
940 .iter()
941 .find(|(document, _)| *document == query)
942 .map_or("talking to GitHub", |(_, doing)| *doing)
943}
944
945/// GitHub's published ceiling on content-generating requests, per minute.
946///
947/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
948/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
949/// from it, so a pacing value checked only against itself cannot go stale here.
950pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
951/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
952/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
953/// governs; it is pinned beside its sibling so the gate would notice either one moving.
954pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
955/// Shortest interval between two content-creating mutations, in milliseconds.
956///
957/// GitHub documents two secondary limits on content-generating requests:
958/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
959/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
960/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
961/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
962/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
963/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
964/// deliberately *not* what this paces at. An installation that wants the hourly bound
965/// honoured for a long sequence of copies says so through
966/// `pacing.min_mutation_interval_ms`.
967pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
968/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
969///
970/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
971/// own advice for a secondary limit — wait, and wait longer each time — without spending
972/// the first minute of a transient refusal doing nothing.
973pub const RETRY_BACKOFF_MS: u64 = 1_000;
974/// Total time one call may spend waiting out rate limits before it reports a failure.
975///
976/// Two minutes is long enough to ride out the refusals a paced copy still collects and
977/// short enough that a command an operator is watching returns. The bound is what makes
978/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
979/// the limiter, not in a process nobody can tell from a wedged one.
980pub const RETRY_BUDGET_MS: u64 = 120_000;
981
982fn default_token_env() -> String {
983 "GH_PROJECTS_TOKEN".to_owned()
984}
985fn default_endpoint() -> String {
986 "https://api.github.com/graphql".to_owned()
987}
988
989/// Where one status category lands on this board.
990///
991/// `null` — an absent value — disables the category for this instance, and using a
992/// disabled status is a refusal naming the status and the instance.
993#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
994#[serde(untagged)]
995pub enum StatusTargetConfig {
996 /// The name of a `Status` single-select option already on the board.
997 Column(ColumnName),
998 /// A closed issue state, whose reason is what tells done from cancelled.
999 Closed {
1000 /// The `IssueClosedStateReason` to close with.
1001 closed: ClosedState,
1002 },
1003}
1004
1005/// The name of a `Status` single-select option on the board.
1006///
1007/// Validated on the way in rather than checked later, so a blank option name — which
1008/// nothing on a board can be — is a state this type cannot hold.
1009#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1010#[serde(try_from = "String")]
1011pub struct ColumnName(String);
1012
1013impl ColumnName {
1014 /// The option name, as the board spells it.
1015 fn as_str(&self) -> &str {
1016 &self.0
1017 }
1018}
1019
1020impl TryFrom<String> for ColumnName {
1021 type Error = String;
1022
1023 fn try_from(name: String) -> Result<Self, Self::Error> {
1024 if name.trim().is_empty() {
1025 return Err("a status_mapping option name cannot be blank".to_owned());
1026 }
1027 Ok(Self(name))
1028 }
1029}
1030
1031/// The two closed states this product can mean.
1032///
1033/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
1034/// work nor abandoned work, so nothing here ever writes it.
1035#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1036#[serde(rename_all = "kebab-case")]
1037pub enum ClosedState {
1038 /// `COMPLETED` — precisely done.
1039 Completed,
1040 /// `NOT_PLANNED` — precisely cancelled.
1041 NotPlanned,
1042}
1043
1044impl ClosedState {
1045 const fn reason(self) -> &'static str {
1046 match self {
1047 Self::Completed => "COMPLETED",
1048 Self::NotPlanned => "NOT_PLANNED",
1049 }
1050 }
1051}
1052
1053/// Configuration for one GitHub Projects v2 board.
1054#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1055#[serde(default, deny_unknown_fields)]
1056pub struct GitHubProjectsConfig {
1057 /// Login of the user or organization which owns the board.
1058 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
1059 /// The project number shown in the board's GitHub URL.
1060 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
1061 /// `owner/name` of the one repository this source creates its issues in.
1062 ///
1063 /// A board has no repository of its own and `createIssue` requires one, so a write
1064 /// without this is refused naming the field. Reads never need it.
1065 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
1066 /// Environment variable containing a fine-grained token with Projects and Issues
1067 /// read/write plus Pull requests read-only access for every repository represented on
1068 /// the board.
1069 #[serde(default = "default_token_env")]
1070 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
1071 /// GraphQL endpoint. GitHub Enterprise installations may override it.
1072 #[serde(default = "default_endpoint")]
1073 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
1074 /// Per-instance mapping from a status category to where it lands on this board.
1075 ///
1076 /// A category this does not mention keeps its shipped default: `backlog` to
1077 /// "Backlog", `todo` to "Todo", `in-progress` to "In Progress", `done` to closed as
1078 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
1079 /// disabled.
1080 #[serde(default)]
1081 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
1082 /// How fast this source writes, and how long it waits out a rate-limit refusal.
1083 ///
1084 /// Every field keeps its shipped default when it is absent, and the defaults are
1085 /// GitHub's own published limits rather than taste. See [`Pacing`].
1086 #[serde(default)]
1087 pub pacing: PacingConfig,
1088}
1089
1090/// How fast this source writes, and how long it waits out a rate-limit refusal.
1091///
1092/// Configurable because a GitHub Enterprise installation sets its own limits and an
1093/// operator who has already been refused may want to go slower still — not because the
1094/// defaults are guesses.
1095#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1096#[serde(default, deny_unknown_fields)]
1097pub struct PacingConfig {
1098 /// Shortest interval between two content-creating mutations, in milliseconds.
1099 ///
1100 /// Zero sends them as fast as they are asked for, which is what a fixture server on
1101 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
1102 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1103 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
1104 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
1105 /// zero while there is a budget to spend, because a schedule of zero-length waits
1106 /// consumes none of it and so never ends.
1107 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
1108 /// Total time one call may spend waiting out rate limits, in milliseconds.
1109 ///
1110 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
1111 /// the bound is what makes this a wait rather than a hang.
1112 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1113}
1114
1115/// The largest any pacing setting may be, in milliseconds.
1116///
1117/// One hour. GitHub's own harshest published bound on content-generating requests works
1118/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
1119/// anything a real limit asks for, and past it the settings stop describing pacing at all:
1120/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
1121/// and an interval beyond it is a command that never sends its second mutation. It also
1122/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
1123/// what an `Instant` can hold on every platform.
1124pub const MAX_PACING_MS: u64 = 3_600_000;
1125
1126/// [`PacingConfig`] with every default resolved and every value checked, which is what the
1127/// source holds.
1128#[derive(Debug, Clone, Copy)]
1129struct Pacing {
1130 min_mutation_interval: Duration,
1131 retry_backoff: Duration,
1132 retry_budget: Duration,
1133}
1134
1135impl Pacing {
1136 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
1137 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
1138 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
1139 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
1140 message: format!(
1141 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
1142 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
1143 GitHub's own harshest published limit"
1144 ),
1145 }),
1146 Some(value) => Ok(Duration::from_millis(value)),
1147 None => Ok(Duration::from_millis(default)),
1148 };
1149 let retry_backoff = bounded(
1150 config.retry_backoff_ms,
1151 RETRY_BACKOFF_MS,
1152 "retry_backoff_ms",
1153 )?;
1154 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
1155 if retry_backoff.is_zero() && !retry_budget.is_zero() {
1156 return Err(SourceError::Config {
1157 message: format!(
1158 "pacing.retry_backoff_ms of source {instance} is 0 while \
1159 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
1160 none of that budget, so it would retry a refusal forever. Set a backoff of \
1161 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
1162 waiting at all",
1163 retry_budget.as_millis()
1164 ),
1165 });
1166 }
1167 Ok(Self {
1168 min_mutation_interval: bounded(
1169 config.min_mutation_interval_ms,
1170 MIN_MUTATION_INTERVAL_MS,
1171 "min_mutation_interval_ms",
1172 )?,
1173 retry_backoff,
1174 retry_budget,
1175 })
1176 }
1177}
1178
1179/// Factory for [`GitHubProjectsSource`].
1180#[derive(Debug, Clone, Copy, Default)]
1181pub struct Plugin;
1182
1183impl SourcePlugin for Plugin {
1184 fn kind(&self) -> &'static str {
1185 KIND
1186 }
1187 fn config_schema(&self) -> Schema {
1188 schema_for!(GitHubProjectsConfig)
1189 }
1190 fn build(
1191 &self,
1192 name: &SourceName,
1193 config: &Value,
1194 secrets: &dyn SecretResolver,
1195 ) -> Result<Box<dyn TaskSource>, SourceError> {
1196 self.build_recording_into(name, config, secrets, Arc::new(Accounting::new()))
1197 }
1198}
1199
1200impl Plugin {
1201 /// Build a source recording every request it sends into an accounting the caller holds.
1202 ///
1203 /// [`SourcePlugin::build`] is this with an accounting of its own, which is what the
1204 /// registry gets. This is for a caller that is also calling GitHub itself and wants one
1205 /// session total rather than two — see [`accounting`] and
1206 /// [`GitHubProjectsSource::recording_into`].
1207 ///
1208 /// # Errors
1209 ///
1210 /// Exactly [`SourcePlugin::build`]'s, with the same source name in front of each:
1211 /// [`SourceError::Config`] for configuration this plugin cannot use and
1212 /// [`SourceError::Auth`] for a credential it cannot find.
1213 pub fn build_recording_into(
1214 &self,
1215 name: &SourceName,
1216 config: &Value,
1217 secrets: &dyn SecretResolver,
1218 ledger: Arc<Accounting>,
1219 ) -> Result<Box<dyn TaskSource>, SourceError> {
1220 let config: GitHubProjectsConfig =
1221 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
1222 message: format!("source {name}: {e}"),
1223 })?;
1224 let source = GitHubProjectsSource::recording_into(name, config, secrets, ledger).map_err(
1225 |error| match error {
1226 SourceError::Config { message } => SourceError::Config {
1227 message: format!("source {name}: {message}"),
1228 },
1229 SourceError::Auth { message } => SourceError::Auth {
1230 message: format!("source {name}: {message}"),
1231 },
1232 other => other,
1233 },
1234 )?;
1235 Ok(Box::new(source))
1236 }
1237}
1238
1239/// Where a status category lands on this board, once configuration is resolved.
1240#[derive(Debug, Clone, PartialEq, Eq)]
1241enum StatusTarget {
1242 /// Not usable against this instance.
1243 Disabled,
1244 /// The board's `Status` option of this name.
1245 Column(ColumnName),
1246 /// A closed issue, with the reason that says which closed it means.
1247 Closed(ClosedState),
1248}
1249
1250/// Every status category, in the order the vocabulary declares them.
1251///
1252/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
1253/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
1254/// added to the shared vocabulary fails to compile until it is named there, and this
1255/// crate's suite reconciles this list against that enum's own derived schema, which is
1256/// generated from the variants rather than written beside them. The schema is what
1257/// catches a list left one short — a list checking only the positions it already holds
1258/// would pass while every mapping indexed by the new position panicked.
1259pub const CATEGORIES: [StatusCategory; 7] = [
1260 StatusCategory::Draft,
1261 StatusCategory::Backlog,
1262 StatusCategory::Todo,
1263 StatusCategory::InProgress,
1264 StatusCategory::Done,
1265 StatusCategory::Cancelled,
1266 StatusCategory::Unknown,
1267];
1268
1269/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
1270#[must_use]
1271pub const fn category_position(category: StatusCategory) -> usize {
1272 match category {
1273 StatusCategory::Draft => 0,
1274 StatusCategory::Backlog => 1,
1275 StatusCategory::Todo => 2,
1276 StatusCategory::InProgress => 3,
1277 StatusCategory::Done => 4,
1278 StatusCategory::Cancelled => 5,
1279 StatusCategory::Unknown => 6,
1280 }
1281}
1282
1283/// The spelling a status category is configured and reported under.
1284fn category_name(category: StatusCategory) -> &'static str {
1285 match category {
1286 StatusCategory::Draft => "draft",
1287 StatusCategory::Backlog => "backlog",
1288 StatusCategory::Todo => "todo",
1289 StatusCategory::InProgress => "in-progress",
1290 StatusCategory::Done => "done",
1291 StatusCategory::Cancelled => "cancelled",
1292 StatusCategory::Unknown => "unknown",
1293 }
1294}
1295
1296/// A shipped default's option name.
1297///
1298/// The literals below are this file's own and non-blank, and they are validated by the
1299/// one constructor a configured name goes through rather than beside it.
1300fn shipped_column(name: &'static str) -> ColumnName {
1301 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
1302}
1303
1304/// The shipped default for one category, before this instance's configuration.
1305fn shipped_default(category: StatusCategory) -> StatusTarget {
1306 match category {
1307 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
1308 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
1309 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
1310 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
1311 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
1312 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
1313 }
1314}
1315
1316/// This instance's complete category-to-target mapping, read in both directions.
1317///
1318/// One target per category, held at that category's own [`category_position`], so a
1319/// category missing from the mapping, named twice in it, or filed out of order is a
1320/// state this type cannot hold rather than one [`Self::target`] has to defend against.
1321#[derive(Debug, Clone)]
1322struct StatusMapping {
1323 targets: [StatusTarget; CATEGORIES.len()],
1324}
1325
1326impl StatusMapping {
1327 fn resolve(
1328 configured: BTreeMap<String, Option<StatusTargetConfig>>,
1329 instance: &SourceName,
1330 ) -> Result<Self, SourceError> {
1331 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
1332 for (key, value) in configured {
1333 let category = CATEGORIES
1334 .iter()
1335 .find(|category| category_name(**category) == key)
1336 .ok_or_else(|| SourceError::Config {
1337 message: format!(
1338 "status_mapping names {key:?}, which is not a status category of source \
1339 {instance}; the categories are {}",
1340 CATEGORIES
1341 .iter()
1342 .map(|category| category_name(*category))
1343 .collect::<Vec<_>>()
1344 .join(", ")
1345 ),
1346 })?;
1347 overrides.insert(category_name(*category), value);
1348 }
1349 // `CATEGORIES[position] == category` for every category — the crate's suite
1350 // asserts it — so mapping the list in order fills each category's own slot.
1351 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
1352 None => shipped_default(category),
1353 Some(None) => StatusTarget::Disabled,
1354 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
1355 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
1356 });
1357 let mapping = Self { targets };
1358 for (index, category) in CATEGORIES.into_iter().enumerate() {
1359 let StatusTarget::Column(option) = mapping.target(category) else {
1360 continue;
1361 };
1362 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
1363 matches!(mapping.target(**earlier), StatusTarget::Column(name)
1364 if name.as_str().eq_ignore_ascii_case(option.as_str()))
1365 }) {
1366 return Err(SourceError::Config {
1367 message: format!(
1368 "status_mapping of source {instance} sends both {} and {} to the board \
1369 option {:?}; one option cannot read back as two categories",
1370 category_name(*other),
1371 category_name(category),
1372 option.as_str()
1373 ),
1374 });
1375 }
1376 }
1377 Ok(mapping)
1378 }
1379
1380 fn target(&self, category: StatusCategory) -> &StatusTarget {
1381 &self.targets[category_position(category)]
1382 }
1383
1384 /// The category a board option name reports, or `None` when nothing maps to it.
1385 fn category_of(&self, option: &str) -> Option<StatusCategory> {
1386 CATEGORIES.into_iter().find(|category| {
1387 matches!(self.target(*category), StatusTarget::Column(name)
1388 if name.as_str().eq_ignore_ascii_case(option))
1389 })
1390 }
1391}
1392
1393/// The one repository this source creates issues in.
1394#[derive(Debug, Clone)]
1395struct RepositoryTarget {
1396 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1397 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1398}
1399
1400impl RepositoryTarget {
1401 fn parse(value: &str) -> Result<Self, SourceError> {
1402 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
1403 message: format!(
1404 "repository must be spelled owner/name; {value:?} names no repository"
1405 ),
1406 })?;
1407 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
1408 return Err(SourceError::Config {
1409 message: format!(
1410 "repository must be spelled owner/name with a GitHub login and one \
1411 repository name; {value:?} is not"
1412 ),
1413 });
1414 }
1415 Ok(Self {
1416 owner: owner.to_owned(),
1417 name: name.to_owned(),
1418 })
1419 }
1420
1421 fn origin(&self) -> String {
1422 format!("github.com/{}/{}", self.owner, self.name)
1423 }
1424}
1425
1426/// A source which reads GitHub afresh for every operation.
1427pub struct GitHubProjectsSource {
1428 /// This source's configured name, used both to tell a far end naming this source
1429 /// from one naming a system it knows nothing about, and to name the instance a
1430 /// status refusal is about.
1431 name: SourceName,
1432 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
1433 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
1434 repository: Option<RepositoryTarget>,
1435 endpoint: Url,
1436 token: SecretString,
1437 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1438 statuses: StatusMapping,
1439 client: Client,
1440 /// Every item this source has created since it was built, in the order it created
1441 /// them.
1442 ///
1443 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1444 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1445 /// a copy resolving a dependency on an item it had just created refused it as not
1446 /// found. A board read is completed from this — an item remembered here and absent from
1447 /// the read is added back, because the board really does hold it and only the read is
1448 /// behind.
1449 ///
1450 /// It is not a cache of a user's work: nothing is remembered that this process did not
1451 /// itself just write, it lives and dies with the process, and it is never consulted for
1452 /// an item this source did not create.
1453 created: Mutex<Vec<Resolved>>,
1454 /// How fast this source writes, and how long it waits out a refusal.
1455 pacing: Pacing,
1456 /// When the last content-creating mutation finished, or the moment the furthest-out
1457 /// reserved slot releases the next one, whichever is later — so the one after it can be
1458 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1459 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1460 /// what it is measured from.
1461 last_mutation: Mutex<Option<Instant>>,
1462 /// The board as this process last read it, for the length of one command.
1463 ///
1464 /// A copy of a project used to re-read the whole board, paged, before writing each of
1465 /// its items, which is by far the largest part of a copy's request count and none of
1466 /// its work. Nothing else changes this board while a command runs — this source's own
1467 /// writes are the only writer — so one read answers them all.
1468 ///
1469 /// It is not a store of a user's work and it is not the cache the no-persistence
1470 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1471 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1472 /// an item this command created and then depends on resolves whether or not GitHub's
1473 /// own eventually-consistent read has caught up. A write to an item already on the
1474 /// board updates the entry here too, so what this holds is the last read plus this
1475 /// process's own writes rather than a snapshot taken before them.
1476 board_cache: Mutex<Option<Board>>,
1477 /// The destination repository's node id, resolved once rather than per issue created.
1478 ///
1479 /// A repository's node id does not change, and re-reading it for every issue of a copy
1480 /// spent one request per item on an answer this source already had.
1481 repository_cache: Mutex<Option<String>>,
1482 /// What every request this source sends is recorded into.
1483 ///
1484 /// Ordinary code path, not a mode: [`Self::send_once`] records into it at the one place
1485 /// a request leaves this crate, so nothing has to be switched on for a session to be
1486 /// counted. It is shared rather than owned so a caller accounting for a whole session —
1487 /// its own schema verification, board lookups, residue sweep and cleanup beside this
1488 /// source's reads and writes — adds up one accounting instead of two. See
1489 /// [`accounting`] for what a record carries and what a session's spend is and is not.
1490 ledger: Arc<Accounting>,
1491}
1492
1493impl GitHubProjectsSource {
1494 /// Validate configuration and capture the named credential without exposing it.
1495 ///
1496 /// # Errors
1497 ///
1498 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1499 /// [`SourceError::Auth`] when the named credential is missing or empty.
1500 pub fn new(
1501 name: &SourceName,
1502 config: GitHubProjectsConfig,
1503 secrets: &dyn SecretResolver,
1504 ) -> Result<Self, SourceError> {
1505 Self::recording_into(name, config, secrets, Arc::new(Accounting::new()))
1506 }
1507
1508 /// The same, recording every request it sends into an accounting the caller holds too.
1509 ///
1510 /// [`Self::new`] is this with an accounting of its own. A caller that is also making
1511 /// its own calls to GitHub — a lane verifying a schema, sweeping residue or cleaning
1512 /// up — passes the one it records those into, so the session total accounts for the
1513 /// whole session rather than for this source's share of it.
1514 ///
1515 /// # Errors
1516 ///
1517 /// Exactly [`Self::new`]'s: [`SourceError::Config`] for a configuration this instance
1518 /// cannot use and [`SourceError::Auth`] when the named credential is missing or empty.
1519 pub fn recording_into(
1520 name: &SourceName,
1521 config: GitHubProjectsConfig,
1522 secrets: &dyn SecretResolver,
1523 ledger: Arc<Accounting>,
1524 ) -> Result<Self, SourceError> {
1525 if !valid_github_owner(&config.owner) {
1526 return Err(SourceError::Config {
1527 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1528 });
1529 }
1530 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1531 return Err(SourceError::Config {
1532 message: format!("project_number must be between 1 and {}", i32::MAX),
1533 });
1534 }
1535 if !valid_environment_name(&config.token_env) {
1536 return Err(SourceError::Config {
1537 message: "token_env must be a valid environment-variable name".into(),
1538 });
1539 }
1540 let repository = config
1541 .repository
1542 .as_deref()
1543 .map(RepositoryTarget::parse)
1544 .transpose()?;
1545 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1546 message: format!("endpoint is not a valid URL: {e}"),
1547 })?;
1548 if endpoint.scheme() != "https"
1549 && !(endpoint.scheme() == "http"
1550 && endpoint
1551 .host_str()
1552 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1553 {
1554 return Err(SourceError::Config {
1555 message:
1556 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1557 .into(),
1558 });
1559 }
1560 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1561 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1562 })?;
1563 Ok(Self {
1564 name: name.clone(),
1565 owner: config.owner,
1566 project_number: config.project_number,
1567 repository,
1568 endpoint,
1569 token,
1570 credential_name: config.token_env,
1571 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1572 client: Client::builder()
1573 .user_agent("onetaskgraph")
1574 .build()
1575 .map_err(|e| SourceError::Config {
1576 message: format!("cannot build HTTP client: {e}"),
1577 })?,
1578 created: Mutex::new(Vec::new()),
1579 pacing: Pacing::resolve(config.pacing, name)?,
1580 last_mutation: Mutex::new(None),
1581 board_cache: Mutex::new(None),
1582 repository_cache: Mutex::new(None),
1583 ledger,
1584 })
1585 }
1586
1587 /// A snapshot of every request this source has sent, and what each cost.
1588 ///
1589 /// A value to hold and compare rather than a borrow of the accounting itself, so two
1590 /// of them can sit side by side. When this source was built with
1591 /// [`Self::recording_into`] the snapshot is the whole shared session, which is the
1592 /// point of building it that way.
1593 #[must_use]
1594 pub fn accounting(&self) -> accounting::Session {
1595 self.ledger.snapshot()
1596 }
1597
1598 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1599 /// rate limit rather than handing it straight back as an error.
1600 ///
1601 /// Retrying is safe for every document here, including the mutations, and the reason
1602 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1603 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1604 /// this replays has already taken effect. An outcome this source cannot know — the
1605 /// send failed, or the body could not be read, so the mutation may well have landed —
1606 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1607 /// attempt. A duplicate write would come from replaying one of those, and none is
1608 /// replayed.
1609 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1610 let doing = operation_description(query);
1611 let mut waited = Duration::ZERO;
1612 let mut waits = 0_u32;
1613 let mut backoff = self.pacing.retry_backoff;
1614 loop {
1615 if is_mutation(query) {
1616 let spacing = self.reserve_mutation_slot();
1617 if !spacing.is_zero() {
1618 tokio::time::sleep(spacing).await;
1619 }
1620 }
1621 let attempt = self.send_once(query, &variables).await;
1622 if is_mutation(query) {
1623 self.finish_mutation();
1624 }
1625 let limited = match attempt {
1626 Ok(data) => return Ok(data),
1627 Err(Attempt::Failed(error)) => return Err(error),
1628 Err(Attempt::Limited(limited)) => limited,
1629 };
1630 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1631 // move that extends a secondary limit, so a hint below the schedule's own next
1632 // wait is raised to it.
1633 let wait = match limited.hint {
1634 Some(hint) => Duration::from_secs(hint).max(backoff),
1635 None => backoff,
1636 };
1637 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1638 // A wait of nothing spends none of the budget, so it is exhaustion rather
1639 // than a retry. `Pacing::resolve` rules out every way of configuring one
1640 // except a budget of zero, where reporting the first refusal is the ask.
1641 if wait.is_zero() || wait > remaining {
1642 return Err(limited.exhausted(
1643 doing,
1644 waits,
1645 waited,
1646 wait,
1647 self.pacing.retry_budget,
1648 ));
1649 }
1650 tokio::time::sleep(wait).await;
1651 waited += wait;
1652 waits += 1;
1653 backoff = backoff.saturating_mul(2);
1654 }
1655 }
1656
1657 /// The next moment a content-creating mutation may leave this source, as a wait from
1658 /// now.
1659 ///
1660 /// The slot is reserved under the lock and the waiting happens outside it, so two
1661 /// callers take two slots rather than the same one — and no lock is held across an
1662 /// await.
1663 ///
1664 /// The moment it is spaced from is the previous mutation's *completion*, which
1665 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1666 /// own is the wrong thing to measure from.
1667 fn reserve_mutation_slot(&self) -> Duration {
1668 if self.pacing.min_mutation_interval.is_zero() {
1669 return Duration::ZERO;
1670 }
1671 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1672 // it would turn an earlier failure into a second one for no gain.
1673 let mut last = self
1674 .last_mutation
1675 .lock()
1676 .unwrap_or_else(std::sync::PoisonError::into_inner);
1677 let now = Instant::now();
1678 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1679 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1680 let at = last.map_or(now, |previous| {
1681 previous
1682 .checked_add(self.pacing.min_mutation_interval)
1683 .map_or(now, |earliest| earliest.max(now))
1684 });
1685 *last = Some(at);
1686 at.saturating_duration_since(now)
1687 }
1688
1689 /// Record that a content-creating mutation has finished, so the next one is spaced
1690 /// from here rather than from the moment this one was released.
1691 ///
1692 /// This source can only choose when a request *departs*; the limiter counts when it
1693 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1694 /// departure from the last therefore hands the limiter a gap of the interval less that
1695 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1696 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1697 /// slower machine while passing on a quick one.
1698 ///
1699 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1700 /// previous request had already arrived before its response came back, so its arrival
1701 /// is no later than this moment, and the next mutation is released at least the
1702 /// interval after this moment and arrives no earlier than it is released: the gap the
1703 /// limiter measures is therefore at least the interval, whatever transit costs and on
1704 /// whatever platform. The price is that a mutation's own round trip no longer counts
1705 /// towards its spacing, which makes this source slightly slower than the configured
1706 /// rate rather than slightly faster — the safe side of a limit that punishes being
1707 /// wrong by refusing reads for the next fifty minutes.
1708 ///
1709 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1710 /// and one that never left costs only a wait nobody needed.
1711 fn finish_mutation(&self) {
1712 if self.pacing.min_mutation_interval.is_zero() {
1713 return;
1714 }
1715 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1716 let mut last = self
1717 .last_mutation
1718 .lock()
1719 .unwrap_or_else(std::sync::PoisonError::into_inner);
1720 let now = Instant::now();
1721 // `max` rather than an assignment: a concurrent caller may already have reserved a
1722 // slot further out, and completing this request must never pull that slot back in.
1723 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1724 }
1725
1726 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1727 /// failure that waiting cannot help — and recorded, whichever of the three it was.
1728 ///
1729 /// This is the one place a request leaves this crate, which is why the accounting is
1730 /// here rather than at each of the callers: a read path added later is counted without
1731 /// anybody remembering to count it, and
1732 /// `the_session_report_counts_every_request_the_board_served_and_what_each_cost` fails
1733 /// when one is not.
1734 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1735 let Attempted {
1736 result,
1737 limits,
1738 reported_cost,
1739 } = self.attempt(query, variables).await;
1740 // No `otherwise` name: every document this source sends is one of its own, and the
1741 // inventory gate on `graphql::DOCUMENTS` is what keeps that true.
1742 let sending = accounting::Request::graphql(query, variables, None, reported_cost);
1743 let outcome = match &result {
1744 Ok(_) => accounting::Outcome::Answered,
1745 Err(Attempt::Limited(_)) => accounting::Outcome::RateLimited,
1746 Err(Attempt::Failed(_)) => accounting::Outcome::Refused,
1747 };
1748 self.ledger.record(sending.finished(outcome, limits));
1749 result
1750 }
1751
1752 /// The attempt itself, with what its response said about the rate limit alongside.
1753 ///
1754 /// The two are returned together rather than recorded here because every one of the
1755 /// early exits below is a different outcome, and a record written at each of them is a
1756 /// record one of them can be added without.
1757 async fn attempt(&self, query: &str, variables: &Value) -> Attempted {
1758 let mut limits = accounting::RateLimit::default();
1759 let mut reported_cost = None;
1760 let result = self
1761 .attempted(query, variables, &mut limits, &mut reported_cost)
1762 .await;
1763 Attempted {
1764 result,
1765 limits,
1766 reported_cost,
1767 }
1768 }
1769
1770 /// One HTTP attempt, filling in what its response said about the rate limit as it goes.
1771 async fn attempted(
1772 &self,
1773 query: &str,
1774 variables: &Value,
1775 limits: &mut accounting::RateLimit,
1776 reported_cost: &mut Option<u64>,
1777 ) -> Result<Value, Attempt> {
1778 let response = self
1779 .client
1780 .post(self.endpoint.clone())
1781 .bearer_auth(self.token.expose_secret())
1782 .json(&json!({"query": query, "variables": variables}))
1783 .send()
1784 .await
1785 .map_err(|e| {
1786 Attempt::Failed(SourceError::Unavailable {
1787 message: format!("GitHub GraphQL request failed: {e}"),
1788 })
1789 })?;
1790 let status = response.status();
1791 let header = |name: &str| whole_seconds(response.headers().get(name));
1792 *limits = accounting::RateLimit::read(|name| {
1793 response
1794 .headers()
1795 .get(name)
1796 .and_then(|value| value.to_str().ok())
1797 .map(str::to_owned)
1798 });
1799 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
1800 // that are not text at all — is "not known to be exhausted". This never makes a
1801 // response a refusal on its own: it says which limiter a refusal is attributed to
1802 // and where its hint comes from, so a value this cannot read costs a hint rather
1803 // than an answer.
1804 let exhausted = response
1805 .headers()
1806 .get("x-ratelimit-remaining")
1807 .and_then(|value| value.to_str().ok())
1808 == Some("0");
1809 // `retry-after` is what GitHub asks for when it asks; when it does not and the
1810 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
1811 // which is the same question answered as an absolute time. Nothing else here is a
1812 // hint, and a schedule is what answers a refusal that carries none.
1813 let hint = header("retry-after").or_else(|| {
1814 exhausted
1815 .then(|| header("x-ratelimit-reset"))
1816 .flatten()
1817 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
1818 });
1819 // Read before it is parsed, because the evidence which tells a secondary rate
1820 // limit from a rejected credential is in the body of a response whose status says
1821 // only "forbidden" — and a non-success response was never parsed at all.
1822 let body = response.text().await.map_err(|e| {
1823 Attempt::Failed(SourceError::Unavailable {
1824 message: format!("GitHub GraphQL response could not be read: {e}"),
1825 })
1826 })?;
1827 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
1828 return Err(Attempt::Limited(Limited { limiter, hint }));
1829 }
1830 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
1831 return Err(Attempt::Failed(SourceError::Auth {
1832 message: format!(
1833 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
1834 ),
1835 }));
1836 }
1837 if !status.is_success() {
1838 return Err(Attempt::Failed(SourceError::Unavailable {
1839 message: format!("GitHub GraphQL returned HTTP {status}"),
1840 }));
1841 }
1842 // GitHub reports what a call cost only when the document asked it to, and no
1843 // document this source sends does — so this is `None` here and carries the figure
1844 // for a caller whose own document selects `rateLimit { cost }`. What it must never
1845 // pick up is a `dryRun` probe's cost, which is some other document's.
1846 *reported_cost = serde_json::from_str::<Value>(&body)
1847 .ok()
1848 .as_ref()
1849 .and_then(|body| body.pointer("/data/rateLimit/cost"))
1850 .and_then(Value::as_u64);
1851 self.answer(&body).map_err(Attempt::Failed)
1852 }
1853
1854 /// What one successful HTTP response says, once its GraphQL errors are read.
1855 fn answer(&self, body: &str) -> Result<Value, SourceError> {
1856 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
1857 message: format!("GitHub returned invalid JSON: {e}"),
1858 })?;
1859 let errors = body
1860 .get("errors")
1861 .map(|value| {
1862 value.as_array().ok_or_else(|| SourceError::Malformed {
1863 message: "GitHub response errors is not an array".into(),
1864 })
1865 })
1866 .transpose()?;
1867 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
1868 let messages = errors
1869 .iter()
1870 .filter_map(|e| e.get("message").and_then(Value::as_str))
1871 .collect::<Vec<_>>()
1872 .join("; ");
1873 let message = if messages.is_empty() {
1874 "GitHub returned GraphQL errors".into()
1875 } else {
1876 messages
1877 };
1878 let normalized = message.to_ascii_lowercase();
1879 if normalized.contains("resource not accessible") || normalized.contains("scope") {
1880 return Err(SourceError::Auth {
1881 message: format!(
1882 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
1883 self.credential_name
1884 ),
1885 });
1886 }
1887 return Err(SourceError::Refused { message });
1888 }
1889 body.get("data")
1890 .filter(|data| data.is_object())
1891 .cloned()
1892 .ok_or_else(|| SourceError::Malformed {
1893 message: "GitHub response has no data object".into(),
1894 })
1895 }
1896
1897 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
1898 // GraphQL cannot independently page them inside the outer item page. This source page is
1899 // deliberately bounded at that published maximum; the live drift journey exercises it.
1900 async fn board_page(
1901 &self,
1902 items_after: Option<&str>,
1903 items_first: u32,
1904 ) -> Result<Value, SourceError> {
1905 let data = self
1906 .graphql(
1907 graphql::BOARD,
1908 json!({"owner":self.owner,"number":self.project_number,
1909 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
1910 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
1911 )
1912 .await?;
1913 data.pointer("/owner/projectV2")
1914 .filter(|v| !v.is_null())
1915 .cloned()
1916 .ok_or_else(|| SourceError::Refused {
1917 message: format!(
1918 "GitHub project {}/{} was not found or is not visible to the token",
1919 self.owner, self.project_number
1920 ),
1921 })
1922 }
1923
1924 /// The search that finds the issues of this board, narrowed by `also` when it is
1925 /// given.
1926 ///
1927 /// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
1928 /// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
1929 /// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
1930 /// here on purpose — GitHub accepts it and silently ignores it, so a project is told
1931 /// from a task by the `parent` field each issue carries rather than by the search.
1932 fn board_search(&self, also: Option<&str>) -> String {
1933 let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
1934 match also {
1935 Some(also) => format!("{scope} {also}"),
1936 None => scope,
1937 }
1938 }
1939
1940 /// One issue this source reached directly, as the board item a read of the board would
1941 /// have produced — or `None` when this board does not hold it.
1942 ///
1943 /// The board half of an issue rides along on `Issue.projectItems`, so the value handed
1944 /// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
1945 /// item's own id, that item's field values, and the issue as its content. One resolver
1946 /// for both routes is what makes an issue read through a search, through its own node
1947 /// id, or through its project's sub-issues report the same title, the same status, the
1948 /// same labels and the same qualified id.
1949 ///
1950 /// An issue with no entry for *this* board is not this source's to report, which is
1951 /// what keeps an id naming some other repository's issue from being answered as an item
1952 /// of this board. That answer is given about an **exhausted** connection and never
1953 /// about an unread page: the entry is looked for on the page in hand, and only if that
1954 /// page reports more of the connection, in [`Self::board_membership`]'s walk of the
1955 /// rest of it.
1956 async fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
1957 if optional_str(issue, "__typename")? != Some("Issue") {
1958 return Ok(None);
1959 }
1960 let memberships = issue
1961 .get("projectItems")
1962 .ok_or_else(|| SourceError::Malformed {
1963 message: "GitHub issue is missing projectItems".into(),
1964 })?;
1965 let nodes = memberships
1966 .get("nodes")
1967 .and_then(Value::as_array)
1968 .ok_or_else(|| SourceError::Malformed {
1969 message: "GitHub issue projectItems.nodes is not an array".into(),
1970 })?;
1971 let held = match self.board_entry(nodes) {
1972 Some(held) => held.clone(),
1973 None => {
1974 let info = memberships
1975 .get("pageInfo")
1976 .ok_or_else(|| SourceError::Malformed {
1977 message: "GitHub issue projectItems has no pageInfo".into(),
1978 })?;
1979 // The page held no entry for this board. Whether that means the issue is
1980 // not on it is a question about the rest of the connection, and only a
1981 // connection with no rest answers it here.
1982 if !required_bool(info, "hasNextPage")? {
1983 return Ok(None);
1984 }
1985 let cursor = required_str(info, "endCursor")?;
1986 validate_cursor_progress(None, cursor)?;
1987 let issue_id = required_str(issue, "id")?;
1988 match self.board_membership(issue_id, cursor).await? {
1989 Some(held) => held,
1990 None => return Ok(None),
1991 }
1992 }
1993 };
1994 let item = json!({
1995 "id": required_str(&held, "id")?,
1996 "fieldValues": held.get("fieldValues"),
1997 "content": issue,
1998 });
1999 self.resolve(&item)
2000 }
2001
2002 /// This board's own entry among one page of an issue's `Issue.projectItems`.
2003 ///
2004 /// One spelling of *which membership is this board's*, so the page a read carries and
2005 /// the pages [`Self::board_membership`] walks are searched by the same rule.
2006 fn board_entry<'a>(&self, nodes: &'a [Value]) -> Option<&'a Value> {
2007 nodes.iter().find(|node| {
2008 node.pointer("/project/number").and_then(Value::as_u64)
2009 == Some(u64::from(self.project_number))
2010 })
2011 }
2012
2013 /// The rest of one issue's board memberships, from `after`, for this board's entry.
2014 ///
2015 /// The recovery read: a page of memberships that holds no entry for this board says
2016 /// nothing about the memberships past it, so the connection is walked to exhaustion
2017 /// before an issue is reported as one this board does not hold. `Ok(None)` is that
2018 /// positive answer — the whole connection was read and no entry named this board —
2019 /// rather than a failure, and the walk is held to
2020 /// [`validate_cursor_progress`] like every other page walk here, so a source answering
2021 /// with a cursor that does not advance is refused instead of spun on.
2022 async fn board_membership(
2023 &self,
2024 issue: &str,
2025 after: &str,
2026 ) -> Result<Option<Value>, SourceError> {
2027 let mut after = after.to_owned();
2028 loop {
2029 let data = self
2030 .graphql(
2031 graphql::ISSUE_BOARD_ITEMS,
2032 json!({"id":issue,"first":MAX_PAGE_SIZE,"after":after,
2033 "nestedFirst":NESTED_PAGE_SIZE}),
2034 )
2035 .await?;
2036 let Some(connection) = data
2037 .pointer("/node/projectItems")
2038 .filter(|value| !value.is_null())
2039 else {
2040 // The id resolved to nothing, or to something with no memberships to walk —
2041 // which is the same answer as a connection holding no entry for this board.
2042 return Ok(None);
2043 };
2044 let nodes = connection
2045 .get("nodes")
2046 .and_then(Value::as_array)
2047 .ok_or_else(|| SourceError::Malformed {
2048 message: "GitHub issue projectItems.nodes is not an array".into(),
2049 })?;
2050 if let Some(held) = self.board_entry(nodes) {
2051 return Ok(Some(held.clone()));
2052 }
2053 let info = connection
2054 .get("pageInfo")
2055 .ok_or_else(|| SourceError::Malformed {
2056 message: "GitHub issue projectItems has no pageInfo".into(),
2057 })?;
2058 let next = required_bool(info, "hasNextPage")?
2059 .then(|| required_str(info, "endCursor"))
2060 .transpose()?;
2061 match next {
2062 Some(next) => {
2063 validate_cursor_progress(Some(&after), next)?;
2064 after = next.to_owned();
2065 }
2066 None => return Ok(None),
2067 }
2068 }
2069 }
2070
2071 /// One page of a board-scoped issue search, and where the next page resumes.
2072 async fn search_page(
2073 &self,
2074 search: &str,
2075 first: u32,
2076 after: Option<&str>,
2077 ) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
2078 let data = self
2079 .graphql(
2080 graphql::SEARCH_ISSUES,
2081 json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
2082 "after":after,"nestedFirst":NESTED_PAGE_SIZE,
2083 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2084 )
2085 .await?;
2086 let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
2087 message: "GitHub search response has no search connection".into(),
2088 })?;
2089 let mut found = Vec::new();
2090 for node in connection
2091 .get("nodes")
2092 .and_then(Value::as_array)
2093 .ok_or_else(|| SourceError::Malformed {
2094 message: "GitHub search nodes is not an array".into(),
2095 })?
2096 {
2097 if let Some(resolved) = self.resolve_issue(node).await? {
2098 found.push(resolved);
2099 }
2100 }
2101 let info = connection
2102 .get("pageInfo")
2103 .ok_or_else(|| SourceError::Malformed {
2104 message: "GitHub search connection has no pageInfo".into(),
2105 })?;
2106 let next = required_bool(info, "hasNextPage")?
2107 .then(|| required_str(info, "endCursor"))
2108 .transpose()?
2109 .map(str::to_owned);
2110 if let Some(next) = &next {
2111 validate_cursor_progress(after, next)?;
2112 }
2113 Ok((found, next))
2114 }
2115
2116 /// Every issue this board holds, walked to exhaustion, completed with what this run
2117 /// wrote.
2118 ///
2119 /// The completion is not an optimisation and it is not a cache: GitHub's issue search
2120 /// is an index and is eventually consistent, so an issue this run created seconds ago
2121 /// is routinely absent from it, and a project listed straight after being written would
2122 /// otherwise be missing from its own board. What is added back is only what this
2123 /// process itself wrote, out of [`Self::created`], which lives and dies with the
2124 /// process.
2125 async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
2126 let mut after: Option<String> = None;
2127 let mut found = Vec::new();
2128 let search = self.board_search(None);
2129 loop {
2130 let (page, next) = self
2131 .search_page(&search, MAX_PAGE_SIZE, after.as_deref())
2132 .await?;
2133 found.extend(page);
2134 match next {
2135 Some(next) => after = Some(next),
2136 None => break,
2137 }
2138 }
2139 self.completed_with_written(found, |_| true)
2140 }
2141
2142 /// `found`, with everything this run wrote that `keep` accepts and the read did not
2143 /// report.
2144 ///
2145 /// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
2146 /// at all: the search index is behind, and a node read of an item filed moments ago can
2147 /// be too.
2148 fn completed_with_written(
2149 &self,
2150 mut found: Vec<Resolved>,
2151 keep: impl Fn(&Resolved) -> bool,
2152 ) -> Result<Vec<Resolved>, SourceError> {
2153 for own in self.created()?.iter().filter(|own| keep(own)) {
2154 if !found.iter().any(|item| item.id == own.id) {
2155 found.push(own.clone());
2156 }
2157 }
2158 Ok(found)
2159 }
2160
2161 /// What resolving one node id reached.
2162 ///
2163 /// Three answers rather than an `Option`, because a board *draft* is none of the other
2164 /// two: it is not an issue, it has no node of its own this source can read the board
2165 /// half off, and its only home is the board's own item connection — so a read of one
2166 /// is completed from there rather than reported as nothing.
2167 async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
2168 let asked = self
2169 .graphql(
2170 graphql::ISSUE,
2171 json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
2172 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2173 )
2174 .await;
2175 let data = match asked {
2176 Ok(data) => data,
2177 // A string that is not a node id at all is not a failure to report: it is an id
2178 // this board does not hold, which is what every read of one already answers.
2179 Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
2180 Err(error) => return Err(error),
2181 };
2182 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
2183 return Ok(Reached::Nothing);
2184 };
2185 if optional_str(node, "__typename")? == Some("DraftIssue") {
2186 return Ok(Reached::Draft);
2187 }
2188 Ok(match self.resolve_issue(node).await? {
2189 Some(item) => Reached::Held(Box::new(item)),
2190 None => Reached::Nothing,
2191 })
2192 }
2193
2194 /// One item of this board by its own id, whatever kind it is.
2195 ///
2196 /// Resolved from the identifier alone: no search, board-wide or otherwise. What this
2197 /// run wrote is read first, because a node read of an item created moments ago can
2198 /// still be behind the board field values written onto it — see [`Self::created`].
2199 async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
2200 if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
2201 return Ok(Some(own.clone()));
2202 }
2203 match self.reach(id).await? {
2204 Reached::Held(item) => Ok(Some(*item)),
2205 Reached::Nothing => Ok(None),
2206 // The one read that still costs the board: a draft lives nowhere else.
2207 Reached::Draft => Ok(self
2208 .board()
2209 .await?
2210 .items
2211 .into_iter()
2212 .find(|item| item.id == *id)),
2213 }
2214 }
2215
2216 /// Everything filed under one issue of this board, walked to exhaustion — or `None`
2217 /// when that id names nothing here with a sub-issue relationship to walk.
2218 ///
2219 /// `None` and an empty answer are different: `None` is *this is not an issue of this
2220 /// GitHub*, which is what sends a project selector on to be read as a name, and an
2221 /// empty vector is a project that holds nothing.
2222 async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
2223 let mut after: Option<String> = None;
2224 let mut children = Vec::new();
2225 loop {
2226 let asked = self
2227 .graphql(
2228 graphql::SUB_ISSUES,
2229 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
2230 "nestedFirst":NESTED_PAGE_SIZE,
2231 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2232 )
2233 .await;
2234 let data = match asked {
2235 Ok(data) => data,
2236 // A string that is not a node id at all is not a failure to report: it is
2237 // the ordinary answer to a selector naming a project by its name.
2238 Err(error) if unresolvable_node(&error) => return Ok(None),
2239 Err(error) => return Err(error),
2240 };
2241 let Some(connection) = data
2242 .pointer("/node/subIssues")
2243 .filter(|value| !value.is_null())
2244 else {
2245 // No such node, or one with no sub-issue relationship — a board draft is
2246 // the one this board can really hold.
2247 return Ok(None);
2248 };
2249 for node in connection
2250 .get("nodes")
2251 .and_then(Value::as_array)
2252 .ok_or_else(|| SourceError::Malformed {
2253 message: "GitHub subIssues.nodes is not an array".into(),
2254 })?
2255 {
2256 if let Some(resolved) = self.resolve_issue(node).await? {
2257 children.push(resolved);
2258 }
2259 }
2260 let info = connection
2261 .get("pageInfo")
2262 .ok_or_else(|| SourceError::Malformed {
2263 message: "GitHub subIssues connection has no pageInfo".into(),
2264 })?;
2265 let next = required_bool(info, "hasNextPage")?
2266 .then(|| required_str(info, "endCursor"))
2267 .transpose()?;
2268 match next {
2269 Some(next) => {
2270 validate_cursor_progress(after.as_deref(), next)?;
2271 after = Some(next.to_owned());
2272 }
2273 None => return Ok(Some(children)),
2274 }
2275 }
2276 }
2277
2278 /// Which issue of this board a project *name* is, or `None` when none is.
2279 ///
2280 /// One bounded query which filters on that name at the server, rather than a walk of
2281 /// every issue the board holds. The name is compared again here: the qualifier narrows
2282 /// what GitHub sends, and this source decides what it names.
2283 async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
2284 let search = self.board_search(Some(&title_qualifier(name)));
2285 let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
2286 Ok(candidates
2287 .into_iter()
2288 .find(|item| {
2289 item.kind == BoardKind::Work(ItemKind::Project)
2290 && item.title.eq_ignore_ascii_case(name)
2291 })
2292 .map(|item| item.id))
2293 }
2294
2295 /// Everything filed under one project of this board: the sub-issues of the issue that
2296 /// project is.
2297 ///
2298 /// Tasks *and* documents, because a document filed under a project is a sub-issue of it
2299 /// too — the caller keeps the kind it asked for. Nothing about this grows as the board
2300 /// gains projects, or as another project gains tasks.
2301 ///
2302 /// A qualified id names the issue and is asked for its sub-issues directly: one
2303 /// request, no search of any kind. Only a selector GitHub cannot resolve that way is
2304 /// read as a project *name*, which costs the one bounded search
2305 /// [`Self::project_by_name`] makes.
2306 async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
2307 let (project, children) = match self.sub_issues(selector).await? {
2308 Some(children) => (selector.clone(), children),
2309 None => match self.project_by_name(&selector.0).await? {
2310 Some(project) => {
2311 let children = self.sub_issues(&project).await?.unwrap_or_default();
2312 (project, children)
2313 }
2314 None => return Ok(Vec::new()),
2315 },
2316 };
2317 self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
2318 }
2319
2320 /// Every item on the board, with the one board identity they all share.
2321 ///
2322 /// See [`Self::board_cache`]. The completion from `created` happens on every call
2323 /// rather than once, which is what the cache could otherwise have broken.
2324 async fn board(&self) -> Result<Board, SourceError> {
2325 let cached = self.board_cache()?.clone();
2326 let mut board = match cached {
2327 Some(board) => board,
2328 None => {
2329 let read = self.read_board().await?;
2330 *self.board_cache()? = Some(read.clone());
2331 read
2332 }
2333 };
2334 for own in self.created()?.iter() {
2335 if !board.items.iter().any(|item| item.id == own.id) {
2336 board.items.push(own.clone());
2337 }
2338 }
2339 Ok(board)
2340 }
2341
2342 /// This process's own view of the board, or the refusal a poisoned lock is.
2343 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
2344 self.board_cache
2345 .lock()
2346 .map_err(|_| SourceError::Unavailable {
2347 message: "this source's view of the board was left inconsistent by an earlier \
2348 failure; next: run the command again"
2349 .into(),
2350 })
2351 }
2352
2353 /// Bring this process's own view of the board up to an item it has just written.
2354 ///
2355 /// A created item goes to `created`, which is what completes a board read GitHub's own
2356 /// eventual consistency has left behind. An item that was already there is replaced
2357 /// where it sits, so a second write of it in the same command reads its real parent
2358 /// rather than the one it had before the first write.
2359 ///
2360 /// "Where it sits" is two places, and missing the first leaves a stale record that
2361 /// wins: an item this same run created is held in `created` and not in the cached
2362 /// board, and `board` completes the cached board *from* `created`, so replacing only
2363 /// the cached copy of such an item replaces nothing and the read still reports the
2364 /// title it was created with.
2365 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
2366 if created {
2367 self.created()?.push(item);
2368 return Ok(());
2369 }
2370 {
2371 let mut own = self.created()?;
2372 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
2373 *held = item;
2374 return Ok(());
2375 }
2376 }
2377 if let Some(board) = self.board_cache()?.as_mut()
2378 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
2379 {
2380 *held = item;
2381 }
2382 Ok(())
2383 }
2384
2385 /// Forget one item this process has just deleted, from both halves of its own view.
2386 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
2387 self.created()?.retain(|own| own.id != *id);
2388 if let Some(board) = self.board_cache()?.as_mut() {
2389 board.items.retain(|item| item.id != *id);
2390 }
2391 Ok(())
2392 }
2393
2394 /// Every page of the board, read from GitHub.
2395 async fn read_board(&self) -> Result<Board, SourceError> {
2396 let mut after: Option<String> = None;
2397 let mut items = Vec::new();
2398 let mut board;
2399 loop {
2400 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
2401 for item in page
2402 .pointer("/items/nodes")
2403 .and_then(Value::as_array)
2404 .ok_or_else(|| SourceError::Malformed {
2405 message: "GitHub project items.nodes is not an array".into(),
2406 })?
2407 {
2408 if let Some(resolved) = self.resolve(item)? {
2409 items.push(resolved);
2410 }
2411 }
2412 let info = page
2413 .pointer("/items/pageInfo")
2414 .ok_or_else(|| SourceError::Malformed {
2415 message: "GitHub project items have no pageInfo".into(),
2416 })?;
2417 let has_next = required_bool(info, "hasNextPage")?;
2418 let next = has_next
2419 .then(|| required_str(info, "endCursor"))
2420 .transpose()?;
2421 board = page.clone();
2422 match next {
2423 Some(next) => {
2424 validate_cursor_progress(after.as_deref(), next)?;
2425 after = Some(next.to_owned());
2426 }
2427 None => break,
2428 }
2429 }
2430 Ok(Board {
2431 id: required_str(&board, "id")?.to_owned(),
2432 fields: board.get("fields").cloned().unwrap_or(Value::Null),
2433 items,
2434 })
2435 }
2436
2437 /// The items this source has created, for completing a board read that is behind.
2438 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
2439 self.created.lock().map_err(|_| SourceError::Unavailable {
2440 message: "this source's record of what it created in this run was left \
2441 inconsistent by an earlier failure; next: run the command again"
2442 .into(),
2443 })
2444 }
2445
2446 /// One board item as this source reports it, or `None` for content it ignores.
2447 ///
2448 /// A pull request is neither a project nor a task — it is somebody's change, not a
2449 /// unit of plan — and an item whose content the token cannot see has nothing to
2450 /// report at all.
2451 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
2452 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
2453 message: "GitHub project item is missing content".into(),
2454 })?;
2455 if content.is_null() {
2456 return Ok(None);
2457 }
2458 let content_kind = match required_str(content, "__typename")? {
2459 "Issue" => ContentKind::Issue,
2460 "DraftIssue" => ContentKind::DraftIssue,
2461 _ => return Ok(None),
2462 };
2463 let field_values = item
2464 .get("fieldValues")
2465 .ok_or_else(|| SourceError::Malformed {
2466 message: "GitHub project item is missing fieldValues".into(),
2467 })?;
2468 complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
2469 let nodes = field_values
2470 .get("nodes")
2471 .and_then(Value::as_array)
2472 .ok_or_else(|| SourceError::Malformed {
2473 message: "GitHub project item fieldValues.nodes is not an array".into(),
2474 })?;
2475 if let Some(labels) = content.get("labels") {
2476 complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
2477 }
2478 let (body, slot) = metadata_body(optional_str(content, "body")?.map(str::to_owned))?;
2479 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
2480 .map(|id| NativeId(id.to_owned()));
2481 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
2482 // to read one from; it is a task, and never a project.
2483 let sub_issues = match content_kind {
2484 ContentKind::Issue => sub_issue_total(content)?,
2485 ContentKind::DraftIssue => 0,
2486 };
2487 let content_id = required_str(content, "id")?;
2488 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
2489 message: format!("GitHub issue {content_id}: {message}"),
2490 })?;
2491 let raw_title = required_str(content, "title")?;
2492 // The design prefix is read *first*, before either of the two rules that separate
2493 // a project from a task. A document is not work whatever sub-issues it has and
2494 // whatever marker it carries, and reading the prefix later would make a design
2495 // issue with none of either an empty project.
2496 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
2497 BoardKind::Document
2498 } else if parent.is_some() {
2499 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
2500 // under a project is that project's task even when it has sub-issues of its
2501 // own.
2502 BoardKind::Work(ItemKind::Task)
2503 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
2504 BoardKind::Work(ItemKind::Project)
2505 } else {
2506 BoardKind::Work(ItemKind::Task)
2507 };
2508 // The title a person wrote, which for a document is the one without the prefix —
2509 // the same way `content` above is the body without this source's metadata slot.
2510 let title = match kind {
2511 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
2512 BoardKind::Work(_) => raw_title.to_owned(),
2513 };
2514 let own_repository = content
2515 .pointer("/repository/nameWithOwner")
2516 .and_then(Value::as_str)
2517 .map(|origin| Repository::try_from(format!("github.com/{origin}")))
2518 .transpose()
2519 .map_err(|message| SourceError::Malformed { message })?;
2520 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
2521 Repository::from_metadata(&slot)
2522 .map_err(|message| SourceError::Malformed { message })?
2523 } else {
2524 own_repository.clone().into_iter().collect()
2525 };
2526 Ok(Some(Resolved {
2527 item_id: required_str(item, "id")?.to_owned(),
2528 id: NativeId(content_id.to_owned()),
2529 content_kind,
2530 kind,
2531 title,
2532 body: body.filter(|value| !value.is_empty()),
2533 status: self.status(item, content)?,
2534 labels: labels(content)?,
2535 parent,
2536 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
2537 url: optional_str(content, "url")?.map(str::to_owned),
2538 created_at: optional_time(content, "createdAt")?,
2539 updated_at: optional_time(content, "updatedAt")?,
2540 own_repository,
2541 repositories,
2542 slot,
2543 }))
2544 }
2545
2546 /// The status one board item reports.
2547 ///
2548 /// The closed state decides the category and the `Status` option decides the name, so
2549 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
2550 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
2551 /// duplicate is not finished work, and calling it done is a lie the next copy would
2552 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
2553 /// it is read permissively rather than refused — reads are faithful, and refusals
2554 /// belong on writes.
2555 fn status(&self, item: &Value, content: &Value) -> Result<Status, SourceError> {
2556 let nodes = item
2557 .pointer("/fieldValues/nodes")
2558 .and_then(Value::as_array)
2559 .expect("resolve validates fieldValues.nodes before mapping status");
2560 let option = nodes
2561 .iter()
2562 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
2563 .map(|value| required_str(value, "name"))
2564 .transpose()?;
2565 let state = optional_str(content, "state")?;
2566 if state == Some("CLOSED") {
2567 let category = match optional_str(content, "stateReason")? {
2568 None | Some("COMPLETED") => StatusCategory::Done,
2569 Some("NOT_PLANNED") => StatusCategory::Cancelled,
2570 Some(_) => StatusCategory::Unknown,
2571 };
2572 let fallback = match category {
2573 StatusCategory::Done => "Done",
2574 StatusCategory::Cancelled => "Cancelled",
2575 _ => "Closed",
2576 };
2577 return Ok(Status {
2578 category,
2579 name: option.unwrap_or(fallback).to_owned(),
2580 });
2581 }
2582 let name = option.unwrap_or("Open").to_owned();
2583 Ok(Status {
2584 category: self
2585 .statuses
2586 .category_of(&name)
2587 .unwrap_or(StatusCategory::Unknown),
2588 name,
2589 })
2590 }
2591
2592 /// The board Status option this write selects, or the refusal that says why not.
2593 ///
2594 /// For a column target the option is what the status *is*, so a board that has no such
2595 /// option is a refusal naming the status and the instance. For a closed target the
2596 /// issue's own state carries the category, and the option carries only the name a
2597 /// reader reports — so an option spelled the way this status is spelled is selected
2598 /// when the board has one, and nothing is refused when it does not.
2599 fn column_for(
2600 &self,
2601 board: &Board,
2602 status: &Status,
2603 target: &StatusTarget,
2604 ) -> Result<Option<(String, String)>, SourceError> {
2605 let (wanted, required) = match target {
2606 StatusTarget::Column(wanted) => (wanted.as_str(), true),
2607 StatusTarget::Closed(_) => (status.name.as_str(), false),
2608 StatusTarget::Disabled => return Ok(None),
2609 };
2610 let missing = |detail: &str| SourceError::Refused {
2611 message: format!(
2612 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
2613 category_name(status.category),
2614 self.name,
2615 category_name(status.category)
2616 ),
2617 };
2618 let Some(field) = Board::field(&board.fields, "Status")? else {
2619 return if required {
2620 Err(missing("this board has no Status field"))
2621 } else {
2622 Ok(None)
2623 };
2624 };
2625 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
2626 return if required {
2627 Err(missing(
2628 "this board's Status field is not a single-select field",
2629 ))
2630 } else {
2631 Ok(None)
2632 };
2633 }
2634 let option = field
2635 .get("options")
2636 .and_then(Value::as_array)
2637 .and_then(|options| {
2638 options.iter().find(|option| {
2639 option
2640 .get("name")
2641 .and_then(Value::as_str)
2642 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
2643 })
2644 });
2645 match option {
2646 None if required => Err(missing("this board does not have it")),
2647 None => Ok(None),
2648 Some(option) => Ok(Some((
2649 required_str(field, "id")?.to_owned(),
2650 required_str(option, "id")?.to_owned(),
2651 ))),
2652 }
2653 }
2654
2655 /// This instance's target for a category, refusing one it has disabled.
2656 ///
2657 /// Nothing here mutates the board's option set to make room for a status. GitHub
2658 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
2659 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
2660 /// field and every item's status.
2661 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
2662 let target = self.statuses.target(category).clone();
2663 if target != StatusTarget::Disabled {
2664 return Ok(target);
2665 }
2666 Err(SourceError::Refused {
2667 message: if category == StatusCategory::Draft {
2668 format!(
2669 "status draft is disabled for source {}: draft is incompatible with this \
2670 integration because GitHub draft issues cannot have sub-issues, and this \
2671 source stores a project's tasks as its issue's sub-issues",
2672 self.name
2673 )
2674 } else {
2675 format!(
2676 "status {} is disabled for source {}; set status_mapping.{} of this source \
2677 to a board Status option name or to a closed state",
2678 category_name(category),
2679 self.name,
2680 category_name(category)
2681 )
2682 },
2683 })
2684 }
2685
2686 async fn set_item_field(
2687 &self,
2688 board_id: &str,
2689 item_id: &str,
2690 field_id: &str,
2691 value: Value,
2692 ) -> Result<(), SourceError> {
2693 let data = self
2694 .graphql(
2695 graphql::UPDATE_FIELD,
2696 json!({"input":{
2697 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
2698 }}),
2699 )
2700 .await?;
2701 let returned = data
2702 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
2703 .ok_or_else(|| SourceError::Malformed {
2704 message: "GitHub field update returned no project item".into(),
2705 })?;
2706 if required_str(returned, "id")? != item_id {
2707 return Err(SourceError::Malformed {
2708 message: "GitHub field update returned the wrong project item".into(),
2709 });
2710 }
2711 Ok(())
2712 }
2713
2714 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
2715 let mut after: Option<String> = None;
2716 let mut ids = Vec::new();
2717 loop {
2718 let data = self
2719 .graphql(
2720 graphql::ISSUE_DEPENDENCIES,
2721 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
2722 )
2723 .await?;
2724 let connection =
2725 data.pointer("/node/blockedBy")
2726 .ok_or_else(|| SourceError::Malformed {
2727 message: "GitHub dependency response has no blockedBy connection".into(),
2728 })?;
2729 ids.extend(
2730 connection
2731 .get("nodes")
2732 .and_then(Value::as_array)
2733 .ok_or_else(|| SourceError::Malformed {
2734 message: "GitHub dependency response nodes is not an array".into(),
2735 })?
2736 .iter()
2737 .map(|value| required_str(value, "id").map(str::to_owned))
2738 .collect::<Result<Vec<_>, _>>()?,
2739 );
2740 let next = next_cursor(connection)?;
2741 if let Some(next) = &next {
2742 validate_cursor_progress(after.as_deref(), &next.0)?;
2743 }
2744 after = next.map(|cursor| cursor.0);
2745 if after.is_none() {
2746 return Ok(ids);
2747 }
2748 }
2749 }
2750
2751 async fn dependencies(
2752 &self,
2753 id: &NativeId,
2754 near_kind: ItemKind,
2755 direction: Direction,
2756 page: &PageRequest,
2757 ) -> Result<Page<DependencyEdge>, SourceError> {
2758 validate_page(page)?;
2759 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
2760 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
2761 let recorded = recorded_offset(cursor, direction)?;
2762 // Asked for even in the recorded phase, whose page reads nothing from the
2763 // connection: `__typename` is what says whether this item has a native
2764 // relationship at all, and that is what decides which far ends the reserved key is
2765 // allowed to hold.
2766 let data = self
2767 .graphql(
2768 graphql::ISSUE_DEPENDENCIES,
2769 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
2770 "after":if recorded.is_some() {None} else {cursor}}),
2771 )
2772 .await?;
2773 let node =
2774 data.get("node")
2775 .filter(|v| !v.is_null())
2776 .ok_or_else(|| SourceError::Refused {
2777 message: format!(
2778 "GitHub item {} was not found or does not support dependencies",
2779 id.0
2780 ),
2781 })?;
2782 let connection_name = match direction {
2783 Direction::DependsOn => "blockedBy",
2784 Direction::DependedOnBy => "blocking",
2785 };
2786 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
2787 // named natively and the reserved key may hold any far end. An issue's connections
2788 // hold issues, and this source reads them at the near item's own level.
2789 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
2790 if let Some(offset) = recorded {
2791 return Ok(recorded_page(
2792 self.recorded_edges(id, near_kind, direction, natively_names)
2793 .await?,
2794 offset,
2795 limit,
2796 ));
2797 }
2798 if natively_names.is_none() {
2799 return Ok(recorded_page(
2800 self.recorded_edges(id, near_kind, direction, natively_names)
2801 .await?,
2802 0,
2803 limit,
2804 ));
2805 }
2806 let connection = node
2807 .get(connection_name)
2808 .ok_or_else(|| SourceError::Malformed {
2809 message: "GitHub dependency response is missing its connection".into(),
2810 })?;
2811 let nodes = connection
2812 .get("nodes")
2813 .and_then(Value::as_array)
2814 .ok_or_else(|| SourceError::Malformed {
2815 message: "GitHub dependency response nodes is not an array".into(),
2816 })?;
2817 // `from` depends on `to`, always. GitHub spells the same relationship from either
2818 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
2819 // it — so the near item is `from` in one direction and `to` in the other.
2820 let items = nodes
2821 .iter()
2822 .map(|value| {
2823 let related = NativeId(required_str(value, "id")?.into());
2824 let related_kind = related_kind(value)?;
2825 let (from, to) = match direction {
2826 Direction::DependsOn => (
2827 DependencyEndpoint::from_native(id.clone(), near_kind),
2828 DependencyEndpoint::from_native(related, related_kind),
2829 ),
2830 Direction::DependedOnBy => (
2831 DependencyEndpoint::from_native(related, related_kind),
2832 DependencyEndpoint::from_native(id.clone(), near_kind),
2833 ),
2834 };
2835 Ok(DependencyEdge {
2836 from,
2837 to,
2838 kind: DependencyKind::Blocks,
2839 })
2840 })
2841 .collect::<Result<Vec<_>, SourceError>>()?;
2842 let mut next = next_cursor(connection)?;
2843 if let Some(next) = &next {
2844 validate_cursor_progress(cursor, &next.0)?;
2845 }
2846 if next.is_none()
2847 && !self
2848 .recorded_edges(id, near_kind, direction, natively_names)
2849 .await?
2850 .is_empty()
2851 {
2852 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
2853 }
2854 Ok(Page { items, next })
2855 }
2856
2857 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
2858 /// a far end in another source has to live: no GitHub issue relationship can name one.
2859 ///
2860 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
2861 /// source never writes one down.
2862 ///
2863 /// The metadata lives in the item's own body slot, so reading it costs one board scan.
2864 /// That is why it happens once the native connection is spent rather than on every
2865 /// page.
2866 async fn recorded_edges(
2867 &self,
2868 id: &NativeId,
2869 near_kind: ItemKind,
2870 direction: Direction,
2871 natively_names: Option<ItemKind>,
2872 ) -> Result<Vec<DependencyEdge>, SourceError> {
2873 if direction != Direction::DependsOn {
2874 return Ok(Vec::new());
2875 }
2876 let Some(item) = self
2877 .board()
2878 .await?
2879 .items
2880 .into_iter()
2881 .find(|item| item.id == *id)
2882 else {
2883 return Ok(Vec::new());
2884 };
2885 DependencyEdge::recorded(&item.slot, id, near_kind, &self.name, natively_names)
2886 .map_err(|message| SourceError::Malformed { message })
2887 }
2888
2889 /// The configured repository's node id, or the refusal naming the field it needs.
2890 ///
2891 /// Resolved once per command; see [`Self::repository_cache`].
2892 async fn repository_id(&self) -> Result<String, SourceError> {
2893 if let Some(id) = self.repository_cache()?.clone() {
2894 return Ok(id);
2895 }
2896 let repository = self
2897 .repository
2898 .as_ref()
2899 .ok_or_else(|| SourceError::Refused {
2900 message: format!(
2901 "source {} has no repository configured, and a GitHub Projects board has no \
2902 repository of its own to create an issue in; set repository: owner/name on \
2903 this source",
2904 self.name
2905 ),
2906 })?;
2907 let data = self
2908 .graphql(
2909 graphql::REPOSITORY,
2910 json!({"owner":repository.owner,"name":repository.name}),
2911 )
2912 .await?;
2913 let node = data
2914 .get("repository")
2915 .filter(|value| !value.is_null())
2916 .ok_or_else(|| SourceError::Refused {
2917 message: format!(
2918 "GitHub repository {}/{} was not found or is not visible to the token",
2919 repository.owner, repository.name
2920 ),
2921 })?;
2922 let id = required_str(node, "id")?.to_owned();
2923 *self.repository_cache()? = Some(id.clone());
2924 Ok(id)
2925 }
2926
2927 /// This process's own record of the destination repository's node id.
2928 fn repository_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<String>>, SourceError> {
2929 self.repository_cache
2930 .lock()
2931 .map_err(|_| SourceError::Unavailable {
2932 message: "this source's record of the destination repository was left \
2933 inconsistent by an earlier failure; next: run the command again"
2934 .into(),
2935 })
2936 }
2937
2938 /// Create or update one board item, whichever kind it is.
2939 async fn write_item(
2940 &self,
2941 incoming: &Incoming<'_>,
2942 target: Option<&NativeId>,
2943 depends_on: &[DependencyEdge],
2944 ) -> Result<NativeId, SourceError> {
2945 // Refused before anything is read or written: a task or a project titled the way
2946 // this board spells a document would land as an issue this same source reads back
2947 // as a document, so the field this destination cannot carry is named rather than
2948 // written and silently reclassified.
2949 if let Written::Work(kind, _) = incoming.written
2950 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
2951 {
2952 return Err(SourceError::Refused {
2953 message: format!(
2954 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
2955 spells a document, so it would read back as one rather than as a {}; \
2956 retitle it, or copy it as a document",
2957 kind.marker(),
2958 self.name,
2959 kind.marker()
2960 ),
2961 });
2962 }
2963 let board = self.board().await?;
2964 let status_target = incoming
2965 .written
2966 .status()
2967 .map(|status| self.resolved_target(status.category))
2968 .transpose()?;
2969 let column = match (incoming.written.status(), status_target.as_ref()) {
2970 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
2971 _ => None,
2972 };
2973 let existing = target
2974 .map(|target| {
2975 board
2976 .items
2977 .iter()
2978 .find(|item| item.id == *target)
2979 .ok_or_else(|| SourceError::Refused {
2980 message: format!("GitHub destination item {} was not found", target.0),
2981 })
2982 })
2983 .transpose()?;
2984 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
2985 if content_kind == ContentKind::DraftIssue {
2986 if let (Some(StatusTarget::Closed(_)), Some(status)) =
2987 (status_target.as_ref(), incoming.written.status())
2988 {
2989 return Err(SourceError::Refused {
2990 message: format!(
2991 "status {} of source {} closes the item's issue, and GitHub draft items \
2992 have no open or closed state",
2993 category_name(status.category),
2994 self.name
2995 ),
2996 });
2997 }
2998 if incoming.parent.is_some() {
2999 return Err(SourceError::Refused {
3000 message: "GitHub draft items cannot be a project's sub-issue".into(),
3001 });
3002 }
3003 }
3004 match existing {
3005 Some(item) if content_kind == ContentKind::Issue => {
3006 if item.labels != incoming.labels {
3007 return Err(SourceError::Refused {
3008 message: "GitHub issue labels differ from the labels being written".into(),
3009 });
3010 }
3011 }
3012 _ => {
3013 if !incoming.labels.is_empty() {
3014 return Err(SourceError::Refused {
3015 message: "GitHub items created by this destination carry no labels".into(),
3016 });
3017 }
3018 }
3019 }
3020
3021 let own_repository = match existing {
3022 Some(item) => item.own_repository.clone(),
3023 None => self
3024 .repository
3025 .as_ref()
3026 .map(|repository| Repository::try_from(repository.origin()))
3027 .transpose()
3028 .map_err(|message| SourceError::Config { message })?,
3029 };
3030 let (native, fallback) = self
3031 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
3032 .await?;
3033 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
3034 let body = compose_body(incoming.content, &slot)?;
3035 // Read before anything is created, for the reason the field below is: a value
3036 // this destination cannot store has to refuse, and refusing after `createIssue`
3037 // would leave an issue behind that nothing asked for. The engine writes a
3038 // qualified id here; a caller handing this key anything else is told so rather
3039 // than having it silently stored as no origin at all.
3040 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
3041 let origin = match incoming.metadata.get(ORIGIN_KEY) {
3042 None => "",
3043 Some(Value::String(origin)) => origin.as_str(),
3044 Some(other) => {
3045 return Err(SourceError::Refused {
3046 message: format!(
3047 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
3048 is {other}"
3049 ),
3050 });
3051 }
3052 };
3053 // Resolved before anything is created: a board that cannot carry the copy origin
3054 // has to refuse the write, and refusing it after `createIssue` would leave an
3055 // issue behind that nothing asked for.
3056 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
3057 Some(field) => {
3058 if required_str(field, "__typename")? != "ProjectV2Field" {
3059 return Err(SourceError::Refused {
3060 message: format!(
3061 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
3062 ),
3063 });
3064 }
3065 Some(required_str(field, "id")?.to_owned())
3066 }
3067 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
3068 return Err(SourceError::Refused {
3069 message: format!(
3070 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
3071 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
3072 the board"
3073 ),
3074 });
3075 }
3076 None => None,
3077 };
3078
3079 let (content_id, item_id, url) = match existing {
3080 Some(item) => {
3081 self.update_existing(item, incoming, &body, status_target.as_ref())
3082 .await?;
3083 (item.id.clone(), item.item_id.clone(), item.url.clone())
3084 }
3085 None => {
3086 self.create_and_file_issue(&board, incoming, &body, status_target.as_ref())
3087 .await?
3088 }
3089 };
3090
3091 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
3092 // then each board field, the parent and the dependencies — and GitHub can fail at
3093 // any of them. Everything this source can refuse *before* the first of those is
3094 // already checked above, so what is left is GitHub itself failing part way. When it
3095 // does over an item this call created, the issue is taken back: a write that
3096 // refused must not leave an item behind that nobody asked for, and one that does
3097 // makes the retry create a second.
3098 let landed = self
3099 .finish_write(
3100 &board,
3101 incoming,
3102 &content_id,
3103 &item_id,
3104 content_kind,
3105 existing,
3106 origin_field.as_deref(),
3107 origin,
3108 column,
3109 &native,
3110 )
3111 .await;
3112 if let Err(error) = landed {
3113 if existing.is_none() {
3114 // Best effort, and the write's own failure is what the caller is told: a
3115 // refusal naming the tidy-up would hide why the write failed at all.
3116 let _ = self.delete_issue(&content_id).await;
3117 }
3118 return Err(error);
3119 }
3120
3121 // So the rest of this command reads what it just did rather than what the board
3122 // said before it. See `remember_written` for which half takes it.
3123 let remembered = Resolved {
3124 item_id,
3125 id: content_id.clone(),
3126 content_kind,
3127 kind: incoming.written.kind(),
3128 title: incoming.title.to_owned(),
3129 // The visible half of the body this write composed, split back off it the
3130 // way a read splits it — so what this record reports is what a read of the
3131 // same issue reports, rather than the person's text with the metadata slot
3132 // still on the end of it.
3133 body: metadata_body(body.clone())?.0,
3134 // A document has no status of its own; what it reads back as is whatever
3135 // the issue's own state says, which is what a re-read reports.
3136 status: incoming
3137 .written
3138 .status()
3139 .cloned()
3140 .unwrap_or_else(|| Status {
3141 category: StatusCategory::Unknown,
3142 name: "Open".to_owned(),
3143 }),
3144 labels: incoming.labels.to_vec(),
3145 parent: incoming.parent.cloned(),
3146 origin: (!origin.is_empty()).then(|| origin.to_owned()),
3147 // In the update path this is the item's own url, read off `existing` where the
3148 // tuple above was bound, so one expression serves both halves.
3149 url,
3150 created_at: existing.and_then(|item| item.created_at),
3151 updated_at: existing.and_then(|item| item.updated_at),
3152 own_repository,
3153 repositories: incoming.repositories.to_vec(),
3154 slot,
3155 };
3156 self.remember_written(remembered, existing.is_none())?;
3157 Ok(content_id)
3158 }
3159
3160 /// Everything a write does after the item exists: its board fields, its parent, and
3161 /// its dependencies.
3162 ///
3163 /// Split out of `write_item` so there is one place a failure past the point of no
3164 /// return is caught, rather than a tidy-up repeated at each `?` above.
3165 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
3166 // so there is one place a failure past the point of no return is caught, and its
3167 // arguments are exactly the values that tail already had in scope. Bundling them into a
3168 // struct would describe no concept — it would be "the arguments of this function" — and
3169 // would put the whole of `write_item`'s locals behind one more indirection.
3170 #[allow(clippy::too_many_arguments)]
3171 async fn finish_write(
3172 &self,
3173 board: &Board,
3174 incoming: &Incoming<'_>,
3175 content_id: &NativeId,
3176 item_id: &str,
3177 content_kind: ContentKind,
3178 existing: Option<&Resolved>,
3179 origin_field: Option<&str>,
3180 origin: &str,
3181 column: Option<(String, String)>,
3182 native: &[String],
3183 ) -> Result<(), SourceError> {
3184 if let Some(field_id) = origin_field {
3185 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
3186 .await?;
3187 }
3188
3189 if let Some((field_id, option_id)) = column {
3190 self.set_item_field(
3191 &board.id,
3192 item_id,
3193 &field_id,
3194 json!({"singleSelectOptionId":option_id}),
3195 )
3196 .await?;
3197 }
3198
3199 if content_kind == ContentKind::Issue {
3200 self.reparent(
3201 existing.and_then(|item| item.parent.clone()),
3202 content_id,
3203 incoming.parent,
3204 )
3205 .await?;
3206 // A document takes part in no dependency graph, so writing one neither reads
3207 // nor changes the issue's own `blockedBy` relationships. Reconciling them
3208 // against the empty list a document write carries would *delete* whatever
3209 // relationships a person had made on that issue, which is a write nobody
3210 // asked for.
3211 if incoming.written.kind() != BoardKind::Document {
3212 self.reconcile_blocked_by(content_id, native).await?;
3213 }
3214 }
3215 Ok(())
3216 }
3217
3218 /// Delete one issue, which takes its board item with it.
3219 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
3220 let data = self
3221 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3222 .await?;
3223 data.pointer("/deleteIssue/repository")
3224 .filter(|value| !value.is_null())
3225 .ok_or_else(|| SourceError::Malformed {
3226 message: "GitHub issue deletion returned no repository".into(),
3227 })?;
3228 self.forget(id)?;
3229 Ok(())
3230 }
3231
3232 /// Remove one item this copy created, so a copy that could not finish leaves the board
3233 /// as it found it.
3234 ///
3235 /// Deleting the issue takes its board item with it, so there is no second mutation to
3236 /// keep in step. An id the board does not hold is not an error: the item is already
3237 /// gone, which is the state this asks for.
3238 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
3239 let board = self.board().await?;
3240 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
3241 return Ok(());
3242 };
3243 if item.content_kind == ContentKind::DraftIssue {
3244 return Err(SourceError::Refused {
3245 message: format!(
3246 "GitHub item {} is a draft, and this source removes an item by deleting \
3247 its issue; next: remove it from the board by hand",
3248 id.0
3249 ),
3250 });
3251 }
3252 let data = self
3253 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3254 .await?;
3255 data.pointer("/deleteIssue/repository")
3256 .filter(|value| !value.is_null())
3257 .ok_or_else(|| SourceError::Malformed {
3258 message: "GitHub issue deletion returned no repository".into(),
3259 })?;
3260 self.forget(id)?;
3261 Ok(())
3262 }
3263
3264 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
3265 async fn partition_edges(
3266 &self,
3267 board: &Board,
3268 near_kind: BoardKind,
3269 near_content: ContentKind,
3270 depends_on: &[DependencyEdge],
3271 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
3272 let mut native = Vec::new();
3273 let mut fallback = Vec::new();
3274 for edge in depends_on {
3275 let same_source = edge
3276 .to
3277 .source()
3278 .is_none_or(|source| source == self.name.as_str());
3279 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
3280 // `DependencyEndpoint::source` both read it that way — and a native id may hold
3281 // colons of its own, so the far end is everything after that one separator.
3282 // Splitting at the last would truncate `work:urn:task:7` to `7`.
3283 let far_id = if edge.to.is_qualified() {
3284 edge.to
3285 .id()
3286 .split_once(':')
3287 .map_or(edge.to.id(), |(_, native)| native)
3288 } else {
3289 edge.to.id()
3290 };
3291 let far = if same_source {
3292 Some(
3293 board
3294 .items
3295 .iter()
3296 .find(|item| item.id.0 == far_id)
3297 .ok_or_else(|| SourceError::Refused {
3298 message: format!("GitHub dependency item {far_id} was not found"),
3299 })?,
3300 )
3301 } else {
3302 None
3303 };
3304 // The caller says which kind the far end is, and this board holds the far end
3305 // itself, so a disagreement is settled here rather than stored: recorded, the
3306 // wrong kind would read back as a cross-level edge that never existed; written
3307 // natively, it would name a relationship of a different level than the caller
3308 // asked for.
3309 //
3310 // A far end this board holds as a *document* fails the same comparison and is
3311 // refused by the same sentence: `ItemKind` has no document variant because
3312 // nothing may point at one, so no caller can name it correctly and the refusal
3313 // is the only honest answer.
3314 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
3315 return Err(SourceError::Refused {
3316 message: format!(
3317 "GitHub dependency item {far_id} is a {} of this board, and this item \
3318 names it as a {}; record the kind it is",
3319 disagreeing.kind.describes(),
3320 edge.to.kind.marker()
3321 ),
3322 });
3323 }
3324 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
3325 // however the far end is spelled — and one classified native here would be
3326 // written nowhere at all, because a draft's native reconciliation never runs.
3327 let native_here = near_content == ContentKind::Issue
3328 && far.is_some_and(|far| {
3329 far.content_kind == ContentKind::Issue
3330 && BoardKind::Work(edge.to.kind) == near_kind
3331 });
3332 if native_here {
3333 native.push(far_id.to_owned());
3334 } else {
3335 fallback.push(edge.clone());
3336 }
3337 }
3338 Ok((native, fallback))
3339 }
3340
3341 async fn update_existing(
3342 &self,
3343 item: &Resolved,
3344 incoming: &Incoming<'_>,
3345 body: &Option<String>,
3346 status_target: Option<&StatusTarget>,
3347 ) -> Result<(), SourceError> {
3348 let title = incoming.written_title();
3349 let (operation, input, pointer) = match item.content_kind {
3350 ContentKind::DraftIssue => (
3351 graphql::UPDATE_DRAFT,
3352 json!({"draftIssueId":item.id.0,"title":title,"body":body}),
3353 "/updateProjectV2DraftIssue/draftIssue",
3354 ),
3355 ContentKind::Issue => (
3356 graphql::UPDATE_ISSUE,
3357 json!({"id":item.id.0,"title":title,"body":body,
3358 "stateInput":state_input(status_target)}),
3359 "/updateIssue/issue",
3360 ),
3361 };
3362 let data = self.graphql(operation, json!({"input":input})).await?;
3363 let returned = data
3364 .pointer(pointer)
3365 .ok_or_else(|| SourceError::Malformed {
3366 message: "GitHub item update returned no item".into(),
3367 })?;
3368 if required_str(returned, "id")? != item.id.0 {
3369 return Err(SourceError::Malformed {
3370 message: "GitHub item update returned the wrong item".into(),
3371 });
3372 }
3373 Ok(())
3374 }
3375
3376 /// Creates one issue, files it on the board, and closes it when the status says so.
3377 ///
3378 /// Three calls rather than one: `createIssue` needs a repository and answers with an
3379 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
3380 /// closed status is a state of the issue rather than a field of the board item.
3381 /// Creates the issue, files it on the board, and reports what a read of it would say:
3382 /// its content id, its board item id, and the web address GitHub gave it.
3383 ///
3384 /// The address comes back here because this is the only place it is known before
3385 /// GitHub's own board read catches up — an item this run created answers the reads
3386 /// that follow it out of the record below, and one remembered without its address
3387 /// would report no location for the rest of the run.
3388 async fn create_and_file_issue(
3389 &self,
3390 board: &Board,
3391 incoming: &Incoming<'_>,
3392 body: &Option<String>,
3393 status_target: Option<&StatusTarget>,
3394 ) -> Result<(NativeId, String, Option<String>), SourceError> {
3395 let repository_id = self.repository_id().await?;
3396 let data = self
3397 .graphql(
3398 graphql::CREATE_ISSUE,
3399 json!({"input":{
3400 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
3401 }}),
3402 )
3403 .await?;
3404 let created = data
3405 .pointer("/createIssue/issue")
3406 .filter(|value| !value.is_null())
3407 .ok_or_else(|| SourceError::Malformed {
3408 message: "GitHub issue creation returned no issue".into(),
3409 })?;
3410 let content_id = NativeId(required_str(created, "id")?.to_owned());
3411 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
3412 // a response without it is not worth failing a landed write over — the item simply
3413 // reports no location until the board read catches up, which is what it did before.
3414 let url = optional_str(created, "url")?.map(str::to_owned);
3415 // The issue exists from here on, so a failure filing it on the board takes it
3416 // back: an issue in the repository that is on no board is an item nobody asked for
3417 // and nothing here would find again.
3418 let added = match self
3419 .graphql(
3420 graphql::ADD_TO_BOARD,
3421 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
3422 )
3423 .await
3424 {
3425 Ok(added) => added,
3426 Err(error) => {
3427 let _ = self.delete_issue(&content_id).await;
3428 return Err(error);
3429 }
3430 };
3431 let item = added
3432 .pointer("/addProjectV2ItemById/item")
3433 .filter(|value| !value.is_null())
3434 .ok_or_else(|| SourceError::Malformed {
3435 message: "GitHub board addition returned no project item".into(),
3436 })?;
3437 if let Some(StatusTarget::Closed(_)) = status_target {
3438 let closed = self
3439 .graphql(
3440 graphql::UPDATE_ISSUE,
3441 json!({"input":{"id":content_id.0,"stateInput":state_input(status_target)}}),
3442 )
3443 .await?;
3444 let returned =
3445 closed
3446 .pointer("/updateIssue/issue")
3447 .ok_or_else(|| SourceError::Malformed {
3448 message: "GitHub item update returned no item".into(),
3449 })?;
3450 if required_str(returned, "id")? != content_id.0 {
3451 return Err(SourceError::Malformed {
3452 message: "GitHub item update returned the wrong item".into(),
3453 });
3454 }
3455 }
3456 Ok((content_id, required_str(item, "id")?.to_owned(), url))
3457 }
3458
3459 /// Move one issue under the project it now belongs to, or out of the one it left.
3460 async fn reparent(
3461 &self,
3462 held: Option<NativeId>,
3463 child: &NativeId,
3464 wanted: Option<&NativeId>,
3465 ) -> Result<(), SourceError> {
3466 if held.as_ref() == wanted {
3467 return Ok(());
3468 }
3469 if let Some(held) = &held {
3470 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
3471 .await?;
3472 }
3473 if let Some(wanted) = wanted {
3474 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
3475 .await?;
3476 }
3477 Ok(())
3478 }
3479
3480 async fn sub_issue(
3481 &self,
3482 operation: &str,
3483 parent: &NativeId,
3484 child: &NativeId,
3485 root: &str,
3486 ) -> Result<(), SourceError> {
3487 let data = self
3488 .graphql(
3489 operation,
3490 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
3491 )
3492 .await?;
3493 let issue =
3494 data.pointer(&format!("/{root}/issue"))
3495 .ok_or_else(|| SourceError::Malformed {
3496 message: "GitHub sub-issue update returned no issue".into(),
3497 })?;
3498 let sub =
3499 data.pointer(&format!("/{root}/subIssue"))
3500 .ok_or_else(|| SourceError::Malformed {
3501 message: "GitHub sub-issue update returned no sub-issue".into(),
3502 })?;
3503 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
3504 return Err(SourceError::Malformed {
3505 message: "GitHub sub-issue update returned the wrong issues".into(),
3506 });
3507 }
3508 Ok(())
3509 }
3510
3511 async fn reconcile_blocked_by(
3512 &self,
3513 content_id: &NativeId,
3514 native: &[String],
3515 ) -> Result<(), SourceError> {
3516 let current = self.native_dependency_ids(content_id).await?;
3517 for (operation, far_id) in current
3518 .iter()
3519 .filter(|id| !native.contains(id))
3520 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
3521 .chain(
3522 native
3523 .iter()
3524 .filter(|id| !current.contains(id))
3525 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
3526 )
3527 {
3528 let data = self
3529 .graphql(
3530 operation,
3531 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
3532 )
3533 .await?;
3534 let root = if operation == graphql::ADD_BLOCKED_BY {
3535 "addBlockedBy"
3536 } else {
3537 "removeBlockedBy"
3538 };
3539 let issue =
3540 data.pointer(&format!("/{root}/issue"))
3541 .ok_or_else(|| SourceError::Malformed {
3542 message: "GitHub dependency update returned no issue".into(),
3543 })?;
3544 let blocker = data
3545 .pointer(&format!("/{root}/blockingIssue"))
3546 .ok_or_else(|| SourceError::Malformed {
3547 message: "GitHub dependency update returned no blocking issue".into(),
3548 })?;
3549 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
3550 {
3551 return Err(SourceError::Malformed {
3552 message: "GitHub dependency update returned the wrong issues".into(),
3553 });
3554 }
3555 }
3556 Ok(())
3557 }
3558}
3559
3560/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
3561enum Reached {
3562 /// An issue this board holds, resolved into everything this source reports about it.
3563 Held(Box<Resolved>),
3564 /// Nothing this board holds: no such node, or a node on some other board.
3565 Nothing,
3566 /// A board draft, which exists only inside the board's own item connection.
3567 Draft,
3568}
3569
3570/// What GitHub says when a string is not a node id it can resolve.
3571///
3572/// Matched because it is the ordinary answer to a project selector naming a project by its
3573/// *name*, and reporting that as a failure would make naming one impossible. It is read
3574/// off the refusal GitHub sent, never guessed from the shape of the string: this source
3575/// does not define the syntax of a GitHub node id and would be wrong about it.
3576const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
3577
3578/// Whether this refusal is GitHub saying the id names no node at all.
3579fn unresolvable_node(error: &SourceError) -> bool {
3580 matches!(error, SourceError::Refused { message }
3581 if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
3582}
3583
3584/// One project name, as a search qualifier which filters on it at the server.
3585///
3586/// Quoted so the whole title is one phrase rather than a bag of words, with the two
3587/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
3588/// the way it documents. A title matched here is still compared for equality afterwards:
3589/// the qualifier narrows what the server sends, and this source decides what it names.
3590fn title_qualifier(name: &str) -> String {
3591 let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
3592 format!("in:title \"{escaped}\"")
3593}
3594
3595/// The board, and every item on it this source reports.
3596#[derive(Clone)]
3597struct Board {
3598 id: String,
3599 fields: Value,
3600 items: Vec<Resolved>,
3601}
3602
3603impl Board {
3604 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
3605 complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
3606 let nodes = fields
3607 .get("nodes")
3608 .and_then(Value::as_array)
3609 .ok_or_else(|| SourceError::Malformed {
3610 message: "GitHub project fields.nodes is not an array".into(),
3611 })?;
3612 Ok(nodes
3613 .iter()
3614 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
3615 }
3616}
3617
3618/// One board item, resolved into everything this source reports about it.
3619#[derive(Clone)]
3620struct Resolved {
3621 item_id: String,
3622 id: NativeId,
3623 content_kind: ContentKind,
3624 kind: BoardKind,
3625 title: String,
3626 body: Option<String>,
3627 status: Status,
3628 labels: Vec<Label>,
3629 parent: Option<NativeId>,
3630 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
3631 origin: Option<String>,
3632 url: Option<String>,
3633 created_at: Option<DateTime<Utc>>,
3634 updated_at: Option<DateTime<Utc>>,
3635 own_repository: Option<Repository>,
3636 repositories: Vec<Repository>,
3637 slot: BTreeMap<String, Value>,
3638}
3639
3640impl Resolved {
3641 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
3642 /// in a field of its own, and none of the three keys that are only an encoding.
3643 fn metadata(&self) -> BTreeMap<String, Value> {
3644 let mut metadata = self.slot.clone();
3645 metadata.remove(Repository::METADATA_KEY);
3646 metadata.remove(DependencyEdge::RECORDED_KEY);
3647 metadata.remove(ItemKind::METADATA_KEY);
3648 if let Some(origin) = &self.origin {
3649 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
3650 }
3651 metadata
3652 }
3653
3654 /// Where this item is, as a link a reader can open.
3655 ///
3656 /// A board is a hosted place and every issue on it has a web address, so that address
3657 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
3658 /// of place it is, so a reader knows to open it rather than to read a file out. It
3659 /// does not replace or derive from `url`: the field goes on reporting exactly what it
3660 /// reported before, and this says what that address *is*.
3661 ///
3662 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
3663 /// rather than a third variant, which is the contract's "the source did not say". An
3664 /// issue this run created is not one of those: its address comes back from the
3665 /// creating mutation, so it is somewhere a reader can open from the moment it exists
3666 /// rather than from whenever the board read catches up.
3667 fn location(&self) -> Option<Location> {
3668 self.url.clone().map(Location::Url)
3669 }
3670
3671 fn task(&self) -> Task {
3672 Task {
3673 id: self.id.clone(),
3674 title: self.title.clone(),
3675 content: self.body.clone(),
3676 status: self.status.clone(),
3677 labels: self.labels.clone(),
3678 project: self.parent.clone(),
3679 url: self.url.clone(),
3680 location: self.location(),
3681 created_at: self.created_at,
3682 updated_at: self.updated_at,
3683 metadata: self.metadata(),
3684 repositories: self.repositories.clone(),
3685 }
3686 }
3687
3688 fn project(&self) -> Project {
3689 Project {
3690 id: self.id.clone(),
3691 title: self.title.clone(),
3692 content: self.body.clone(),
3693 status: self.status.clone(),
3694 labels: self.labels.clone(),
3695 url: self.url.clone(),
3696 location: self.location(),
3697 created_at: self.created_at,
3698 updated_at: self.updated_at,
3699 metadata: self.metadata(),
3700 repositories: self.repositories.clone(),
3701 }
3702 }
3703
3704 /// The same issue as a document: the project it is filed under, and no status and no
3705 /// dependencies, because a document is not work.
3706 fn document(&self) -> Document {
3707 Document {
3708 id: self.id.clone(),
3709 title: self.title.clone(),
3710 content: self.body.clone(),
3711 project: self.parent.clone(),
3712 labels: self.labels.clone(),
3713 url: self.url.clone(),
3714 location: self.location(),
3715 created_at: self.created_at,
3716 updated_at: self.updated_at,
3717 metadata: self.metadata(),
3718 repositories: self.repositories.clone(),
3719 }
3720 }
3721}
3722
3723/// What one write is, and the status that comes with being it.
3724///
3725/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
3726/// status and a task or a project always has one, so "a document carrying a status" and
3727/// "a task carrying none" are states a write cannot be in rather than states every use
3728/// site below has to defend against.
3729enum Written<'a> {
3730 /// A document, which is not work and so has no status at all.
3731 Document,
3732 /// A task or a project, and the status it is being written with.
3733 Work(ItemKind, &'a Status),
3734}
3735
3736impl Written<'_> {
3737 /// Which of the board's three kinds this write is.
3738 const fn kind(&self) -> BoardKind {
3739 match self {
3740 Self::Document => BoardKind::Document,
3741 Self::Work(kind, _) => BoardKind::Work(*kind),
3742 }
3743 }
3744
3745 /// The status this write carries. A document carries none, so a write of one says
3746 /// nothing about the issue's open or closed state and selects no board `Status`
3747 /// option.
3748 const fn status(&self) -> Option<&Status> {
3749 match self {
3750 Self::Document => None,
3751 Self::Work(_, status) => Some(status),
3752 }
3753 }
3754}
3755
3756/// The item being written, in the one shape all three write methods reach.
3757struct Incoming<'a> {
3758 written: Written<'a>,
3759 /// The title a person wrote. A document's goes onto the issue with
3760 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
3761 title: &'a str,
3762 content: Option<&'a str>,
3763 labels: &'a [Label],
3764 metadata: &'a BTreeMap<String, Value>,
3765 repositories: &'a [Repository],
3766 parent: Option<&'a NativeId>,
3767}
3768
3769impl Incoming<'_> {
3770 /// The title this write puts on the issue.
3771 fn written_title(&self) -> String {
3772 match self.written {
3773 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
3774 Written::Work(..) => self.title.to_owned(),
3775 }
3776 }
3777}
3778
3779#[derive(Clone, Copy, PartialEq, Eq)]
3780enum ContentKind {
3781 DraftIssue,
3782 Issue,
3783}
3784
3785/// What one board issue is: a document, or the work an [`ItemKind`] names.
3786///
3787/// A type of this source's own rather than an `ItemKind` with a third variant, because
3788/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
3789/// document — the contract keeps a document out of that enum deliberately. Holding the
3790/// board's three answers in one value is what makes every place that asks "which is this?"
3791/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
3792/// two thirds of the board.
3793#[derive(Clone, Copy, PartialEq, Eq)]
3794enum BoardKind {
3795 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
3796 Document,
3797 /// Every other issue, and every draft.
3798 Work(ItemKind),
3799}
3800
3801impl BoardKind {
3802 /// How a refusal names this kind to the person reading it.
3803 const fn describes(self) -> &'static str {
3804 match self {
3805 Self::Document => "document",
3806 Self::Work(kind) => kind.marker(),
3807 }
3808 }
3809}
3810
3811/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
3812///
3813/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
3814/// the shared cross-source journeys assert one answer to one question, so two sources
3815/// that disagree about what "carries the label bug" means fail them.
3816fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
3817 let holds = |name: &String| {
3818 labels
3819 .iter()
3820 .any(|label| label.name.eq_ignore_ascii_case(name))
3821 };
3822 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
3823 && filter.all_of.iter().all(holds)
3824 && !filter.none_of.iter().any(holds)
3825}
3826
3827/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
3828/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
3829fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
3830 statuses.is_empty() || statuses.contains(&category)
3831}
3832
3833/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
3834///
3835/// `content` is the item's own prose — the body with this source's trailing metadata
3836/// comment already taken off — so a search never matches an encoding the author of the
3837/// issue never wrote.
3838fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
3839 let terms = query.terms.to_lowercase();
3840 let in_title = title.to_lowercase().contains(&terms);
3841 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
3842 match query.fields {
3843 TextFields::Title => in_title,
3844 TextFields::Content => in_content,
3845 TextFields::TitleOrContent => in_title || in_content,
3846 }
3847}
3848
3849/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
3850///
3851/// The project predicate is passed separately because a read narrowed to one project has
3852/// already answered it by asking *that project* for its own items — and re-applying it
3853/// there would compare the caller's selector, which may be a project's **name**, against
3854/// the id of the project that name resolved to, and keep nothing. Every other read passes
3855/// `query.project` and applies it here, which is what keeps `projects` a predicate this
3856/// source really does apply.
3857fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
3858 labels_match(&task.labels, &query.labels)
3859 && status_matches(task.status.category, &query.statuses)
3860 && match project {
3861 ProjectFilter::Any => true,
3862 ProjectFilter::Orphans => task.project.is_none(),
3863 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
3864 }
3865 && query
3866 .text
3867 .as_ref()
3868 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
3869}
3870
3871fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
3872 labels_match(&project.labels, &query.labels)
3873 && status_matches(project.status.category, &query.statuses)
3874 && query
3875 .text
3876 .as_ref()
3877 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
3878}
3879
3880/// The same three predicates a task query carries, minus the status filter.
3881///
3882/// A document is not work, so it has no status for one to compare against and the query
3883/// type carries none. The project predicate is the same one — a design issue filed under a
3884/// project issue is in that project, and one filed under nothing is in none — so it is
3885/// spelled the same way here rather than answered differently.
3886fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
3887 labels_match(&document.labels, &query.labels)
3888 && match project {
3889 ProjectFilter::Any => true,
3890 ProjectFilter::Orphans => document.project.is_none(),
3891 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
3892 }
3893 && query
3894 .text
3895 .as_ref()
3896 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
3897}
3898
3899#[async_trait::async_trait]
3900impl TaskSource for GitHubProjectsSource {
3901 fn kind(&self) -> &'static str {
3902 KIND
3903 }
3904 fn capabilities(&self) -> Capabilities {
3905 Capabilities {
3906 projects: Support::Native,
3907 documents: Support::Native,
3908 orphan_tasks: Support::Native,
3909 filter_by_label: Support::Native,
3910 filter_by_status: Support::Native,
3911 search_title: Support::Native,
3912 search_content: Support::Native,
3913 task_dependencies: DependencySupport::BothDirections,
3914 project_dependencies: DependencySupport::BothDirections,
3915 max_page_size: MAX_PAGE_SIZE,
3916 }
3917 }
3918 async fn health(&self) -> Result<Health, SourceError> {
3919 let board = self.board_page(None, 1).await?;
3920 Ok(Health {
3921 reachable: true,
3922 detail: Some(format!(
3923 "reading GitHub project {}/{} ({})",
3924 self.owner,
3925 self.project_number,
3926 required_str(&board, "title")?
3927 )),
3928 })
3929 }
3930 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
3931 Ok(self
3932 .item_by_id(id)
3933 .await?
3934 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3935 .map(|item| item.task()))
3936 }
3937 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
3938 Ok(self
3939 .item_by_id(id)
3940 .await?
3941 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
3942 .map(|item| item.project()))
3943 }
3944 async fn query_tasks(
3945 &self,
3946 query: &TaskQuery,
3947 page: &PageRequest,
3948 ) -> Result<Page<Task>, SourceError> {
3949 validate_page(page)?;
3950 // A read narrowed to one project asks that project for its own tasks, so nothing
3951 // about it costs what the rest of the board holds. Every other task read is a
3952 // question about the whole board and is answered by reading it.
3953 let (held, membership) = match &query.project {
3954 ProjectFilter::Is(project) => (
3955 self.project_children(project).await?,
3956 // Answered by where these items came from; see `task_matches`.
3957 &ProjectFilter::Any,
3958 ),
3959 ProjectFilter::Any | ProjectFilter::Orphans => {
3960 (self.board().await?.items, &query.project)
3961 }
3962 };
3963 // Filtered before paged: a page of a filtered result is a page of the survivors,
3964 // never the survivors of a page.
3965 let tasks = held
3966 .iter()
3967 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3968 .map(Resolved::task)
3969 .filter(|task| task_matches(task, query, membership))
3970 .collect();
3971 Ok(offset_page(
3972 tasks,
3973 numeric_cursor(page.cursor.as_ref())?,
3974 page.limit.min(MAX_PAGE_SIZE) as usize,
3975 ))
3976 }
3977 async fn query_projects(
3978 &self,
3979 query: &ProjectQuery,
3980 page: &PageRequest,
3981 ) -> Result<Page<Project>, SourceError> {
3982 validate_page(page)?;
3983 // The projects a board holds are found by an issue search scoped to that board,
3984 // never by walking the board's own item connection: what tells a project from a
3985 // task is the `parent` each issue carries, which costs nothing to read.
3986 let projects = self
3987 .board_issues()
3988 .await?
3989 .iter()
3990 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
3991 .map(Resolved::project)
3992 .filter(|project| project_matches(project, query))
3993 .collect();
3994 Ok(offset_page(
3995 projects,
3996 numeric_cursor(page.cursor.as_ref())?,
3997 page.limit.min(MAX_PAGE_SIZE) as usize,
3998 ))
3999 }
4000 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
4001 Ok(self
4002 .item_by_id(id)
4003 .await?
4004 .filter(|item| item.kind == BoardKind::Document)
4005 .map(|item| item.document()))
4006 }
4007 async fn query_documents(
4008 &self,
4009 query: &DocumentQuery,
4010 page: &PageRequest,
4011 ) -> Result<Page<Document>, SourceError> {
4012 validate_page(page)?;
4013 // Narrowed to one project, this is the same sub-issue read a task list scoped to
4014 // that project makes — a document filed under a project is a sub-issue of it too,
4015 // and which of them come back is the kind this caller asked for.
4016 let (held, membership) = match &query.project {
4017 ProjectFilter::Is(project) => (
4018 self.project_children(project).await?,
4019 // Answered by where these items came from; see `task_matches`.
4020 &ProjectFilter::Any,
4021 ),
4022 ProjectFilter::Any | ProjectFilter::Orphans => {
4023 (self.board().await?.items, &query.project)
4024 }
4025 };
4026 // Filtered before paged, exactly as a task read is: a page of a filtered result is
4027 // a page of the survivors, never the survivors of a page.
4028 let documents = held
4029 .iter()
4030 .filter(|item| item.kind == BoardKind::Document)
4031 .map(Resolved::document)
4032 .filter(|document| document_matches(document, query, membership))
4033 .collect();
4034 Ok(offset_page(
4035 documents,
4036 numeric_cursor(page.cursor.as_ref())?,
4037 page.limit.min(MAX_PAGE_SIZE) as usize,
4038 ))
4039 }
4040 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
4041 validate_page(page)?;
4042 let offset = numeric_cursor(page.cursor.as_ref())?;
4043 let mut labels = self
4044 .board()
4045 .await?
4046 .items
4047 .into_iter()
4048 .flat_map(|item| item.labels)
4049 .fold(Vec::new(), |mut all, label| {
4050 if !all.iter().any(|x: &Label| x.id == label.id) {
4051 all.push(label);
4052 }
4053 all
4054 });
4055 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
4056 Ok(offset_page(
4057 labels,
4058 offset,
4059 page.limit.min(MAX_PAGE_SIZE) as usize,
4060 ))
4061 }
4062 async fn task_dependencies(
4063 &self,
4064 id: &NativeId,
4065 direction: Direction,
4066 page: &PageRequest,
4067 ) -> Result<Page<DependencyEdge>, SourceError> {
4068 self.dependencies(id, ItemKind::Task, direction, page).await
4069 }
4070 async fn project_dependencies(
4071 &self,
4072 id: &NativeId,
4073 direction: Direction,
4074 page: &PageRequest,
4075 ) -> Result<Page<DependencyEdge>, SourceError> {
4076 self.dependencies(id, ItemKind::Project, direction, page)
4077 .await
4078 }
4079
4080 fn writes(&self) -> WriteSupport {
4081 WriteSupport::Supported
4082 }
4083
4084 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
4085 self.write_item(
4086 &Incoming {
4087 written: Written::Work(ItemKind::Task, &write.item.status),
4088 title: &write.item.title,
4089 content: write.item.content.as_deref(),
4090 labels: &write.item.labels,
4091 metadata: &write.item.metadata,
4092 repositories: &write.item.repositories,
4093 parent: write.item.project.as_ref(),
4094 },
4095 write.target.as_ref(),
4096 &write.depends_on,
4097 )
4098 .await
4099 }
4100
4101 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
4102 self.write_item(
4103 &Incoming {
4104 written: Written::Work(ItemKind::Project, &write.item.status),
4105 title: &write.item.title,
4106 content: write.item.content.as_deref(),
4107 labels: &write.item.labels,
4108 metadata: &write.item.metadata,
4109 repositories: &write.item.repositories,
4110 parent: None,
4111 },
4112 write.target.as_ref(),
4113 &write.depends_on,
4114 )
4115 .await
4116 }
4117
4118 /// Create or update one document, which is one issue titled the way this board spells
4119 /// a document.
4120 ///
4121 /// Everything else is exactly a task write: caller metadata goes to the same canonical
4122 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
4123 /// or a field this board cannot carry is refused by name rather than dropped, a target
4124 /// naming an issue this board does not hold is refused rather than created, and an
4125 /// issue this call created is taken back when the rest of the write fails.
4126 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
4127 // A document takes part in no dependency graph, so there is no far end to write
4128 // natively and none to record: a caller naming one is told so rather than having it
4129 // stored under the reserved key, where a later read would report an edge the
4130 // contract says cannot exist.
4131 if !write.depends_on.is_empty() {
4132 return Err(SourceError::Refused {
4133 message: format!(
4134 "this write names {} dependencies for a document, and a document takes \
4135 part in no dependency graph; next: put the dependency on the task or \
4136 project the document is about",
4137 write.depends_on.len()
4138 ),
4139 });
4140 }
4141 self.write_item(
4142 &Incoming {
4143 written: Written::Document,
4144 title: &write.item.title,
4145 content: write.item.content.as_deref(),
4146 labels: &write.item.labels,
4147 metadata: &write.item.metadata,
4148 repositories: &write.item.repositories,
4149 parent: write.item.project.as_ref(),
4150 },
4151 write.target.as_ref(),
4152 &[],
4153 )
4154 .await
4155 }
4156
4157 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
4158 self.delete_item(id).await
4159 }
4160
4161 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
4162 self.delete_item(id).await
4163 }
4164
4165 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
4166 self.delete_item(id).await
4167 }
4168}
4169
4170/// Where the recorded tail of a dependency walk resumes; see
4171/// [`GitHubProjectsSource::recorded_edges`].
4172const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
4173
4174/// The board text field this source keeps a copy's origin in.
4175///
4176/// Named after the key it holds, and held to that name by the guard below rather than by
4177/// a reader noticing.
4178const ORIGIN_FIELD: &str = "onetaskgraph.origin";
4179
4180/// The metadata key that field holds.
4181///
4182/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
4183/// constructs or interprets the qualified id it carries. This source names it only to
4184/// route it — a short, typed value belongs in a typed field rather than in the body slot
4185/// a caller's own prose shares.
4186///
4187/// Restated rather than imported, because no plugin crate may depend on the engine. What
4188/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
4189/// target in `check`: it reads the engine's own literal and fails naming the file and the
4190/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
4191/// that creates a second item every run instead of finding the one it wrote — and that is
4192/// too late to learn it.
4193const ORIGIN_KEY: &str = "onetaskgraph.origin";
4194
4195/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
4196///
4197/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
4198/// is derived from the far end, never written down on the near item — so only a forward
4199/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
4200/// it did not come from, and it is told so rather than answered with an empty page that
4201/// reads as a walk which ended.
4202fn recorded_offset(
4203 cursor: Option<&str>,
4204 direction: Direction,
4205) -> Result<Option<usize>, SourceError> {
4206 cursor
4207 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
4208 .map(|offset| {
4209 if direction != Direction::DependsOn {
4210 return Err(SourceError::Config {
4211 message: format!(
4212 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
4213 reverse dependency read never issues; resume it in the direction \
4214 that reported it"
4215 ),
4216 });
4217 }
4218 offset.parse().map_err(|_| SourceError::Config {
4219 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
4220 })
4221 })
4222 .transpose()
4223}
4224
4225fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
4226 let mut page = offset_page(edges, offset, limit.max(1));
4227 page.next = page
4228 .next
4229 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
4230 page
4231}
4232
4233/// The kind of one issue reached through a dependency connection.
4234///
4235/// The same questions the board scan asks, over the fields the dependency document
4236/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
4237/// then anything with sub-issues or the marker is a project.
4238///
4239/// # Errors
4240///
4241/// A far end this board holds as a document is refused rather than reported. The two
4242/// answers that are not refusals would both be wrong: reporting it as a task names an id
4243/// no task read of this source can find, and reporting it as a project names one no
4244/// project read can. There is no third value to return — `ItemKind` has no document
4245/// variant, because nothing may point at a document — so the relationship itself is what
4246/// the person is told about.
4247fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
4248 let id = required_str(value, "id")?;
4249 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
4250 return Err(SourceError::Refused {
4251 message: format!(
4252 "GitHub issue {id} is a document of this board — its title begins \
4253 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
4254 on by one; next: remove that issue's blocking relationship on this board"
4255 ),
4256 });
4257 }
4258 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
4259 if parent.is_some() {
4260 return Ok(ItemKind::Task);
4261 }
4262 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
4263 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
4264 message: format!("GitHub issue {id}: {message}"),
4265 })?;
4266 let sub_issues = sub_issue_total(value)?;
4267 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
4268 ItemKind::Project
4269 } else {
4270 ItemKind::Task
4271 })
4272}
4273
4274/// The `IssueStateUpdateInput` one status target asks for.
4275///
4276/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
4277/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
4278/// a currently-closed issue: without that the item would read back `Unknown` and a copy
4279/// would report a change forever. A document has no status at all, and asks for neither.
4280fn state_input(target: Option<&StatusTarget>) -> Value {
4281 match target {
4282 Some(StatusTarget::Closed(reason)) => {
4283 json!({"value":"CLOSED","stateReason":reason.reason()})
4284 }
4285 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
4286 // A document has no status, so a write of one says nothing about the issue's open
4287 // or closed state rather than forcing it open: `stateInput` is what carries that
4288 // instruction, and an explicit null asks for no change to it.
4289 None => Value::Null,
4290 }
4291}
4292
4293/// The metadata one write stores in the item's body slot.
4294///
4295/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
4296/// rather than carried: the kind marker so an empty project stays readable, the
4297/// repository list only when it is not exactly the issue's own repository, and the far
4298/// ends no relationship here can name.
4299fn slot_metadata(
4300 incoming: &Incoming<'_>,
4301 own_repository: Option<&Repository>,
4302 fallback: &[DependencyEdge],
4303) -> BTreeMap<String, Value> {
4304 let mut metadata = incoming.metadata.clone();
4305 metadata.remove(ORIGIN_KEY);
4306 match incoming.written.kind() {
4307 BoardKind::Work(kind) => metadata.insert(
4308 ItemKind::METADATA_KEY.to_owned(),
4309 Value::String(kind.marker().to_owned()),
4310 ),
4311 // A document is told by its title, so it carries no kind marker: that key names
4312 // what a dependency endpoint points at, and nothing may point at a document.
4313 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
4314 };
4315 let derivable = own_repository
4316 .map(|own| incoming.repositories == [own.clone()])
4317 .unwrap_or(incoming.repositories.is_empty());
4318 if derivable {
4319 metadata.remove(Repository::METADATA_KEY);
4320 } else {
4321 metadata.insert(
4322 Repository::METADATA_KEY.to_owned(),
4323 Value::Array(
4324 incoming
4325 .repositories
4326 .iter()
4327 .map(|repository| Value::String(repository.as_str().to_owned()))
4328 .collect(),
4329 ),
4330 );
4331 }
4332 if fallback.is_empty() {
4333 metadata.remove(DependencyEdge::RECORDED_KEY);
4334 } else {
4335 metadata.insert(
4336 DependencyEdge::RECORDED_KEY.to_owned(),
4337 Value::Array(
4338 fallback
4339 .iter()
4340 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
4341 .collect(),
4342 ),
4343 );
4344 }
4345 metadata
4346}
4347
4348/// Every label one item carries, from its content's own connection and nowhere else.
4349///
4350/// There is no second place to read one from: no document this source sends selects the
4351/// board's built-in `Labels` field, because GitHub derives it from the content and a draft
4352/// cannot carry one at all. The module documentation records the three schema facts that
4353/// settle it.
4354fn labels(content: &Value) -> Result<Vec<Label>, SourceError> {
4355 optional_nodes(content.get("labels"), "content labels")?
4356 .into_iter()
4357 .flatten()
4358 .map(|v| {
4359 Ok(Label {
4360 id: NativeId(required_str(v, "id")?.to_owned()),
4361 name: required_str(v, "name")?.to_owned(),
4362 color: optional_str(v, "color")?.map(str::to_owned),
4363 })
4364 })
4365 .collect()
4366}
4367
4368fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
4369 let Some(node) = field_values
4370 .iter()
4371 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
4372 else {
4373 return Ok(None);
4374 };
4375 Ok(optional_str(node, "text")?.map(str::to_owned))
4376}
4377
4378fn valid_github_owner(owner: &str) -> bool {
4379 !owner.is_empty()
4380 && owner.len() <= 39
4381 && !owner.starts_with('-')
4382 && !owner.ends_with('-')
4383 && !owner.contains("--")
4384 && owner
4385 .bytes()
4386 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
4387}
4388
4389/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
4390/// neither of the two names a path segment already means.
4391fn valid_github_repository_name(name: &str) -> bool {
4392 !name.is_empty()
4393 && name.len() <= 100
4394 && name != "."
4395 && name != ".."
4396 && name
4397 .bytes()
4398 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
4399}
4400
4401fn valid_environment_name(name: &str) -> bool {
4402 let mut bytes = name.bytes();
4403 bytes
4404 .next()
4405 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
4406 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
4407}
4408
4409/// How many sub-issues one issue has.
4410///
4411/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
4412/// absent or non-integer one is a response this source cannot read — and reading it as
4413/// zero would classify a project as a task, which is exactly the mistake the marker
4414/// exists to keep from happening quietly.
4415fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
4416 let summary = issue
4417 .get("subIssuesSummary")
4418 .ok_or_else(|| SourceError::Malformed {
4419 message: "GitHub issue is missing subIssuesSummary".into(),
4420 })?;
4421 summary
4422 .get("total")
4423 .and_then(Value::as_u64)
4424 .ok_or_else(|| SourceError::Malformed {
4425 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
4426 })
4427}
4428
4429fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
4430 value
4431 .get(field)
4432 .and_then(Value::as_str)
4433 .ok_or_else(|| SourceError::Malformed {
4434 message: format!("GitHub response is missing string field {field}"),
4435 })
4436}
4437
4438/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
4439/// needs one — Linear spells them too, in its own description field.
4440///
4441/// Restated rather than shared, because a plugin crate depends on the contract crate and
4442/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
4443/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
4444/// source round-trips its own writes perfectly well under its own spelling.
4445const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
4446const METADATA_CLOSE: &str = "\n-->";
4447
4448/// The visible body and the metadata slot at the end of it.
4449///
4450/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
4451/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
4452/// own content and is left alone.
4453fn metadata_body(
4454 body: Option<String>,
4455) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
4456 let Some(body) = body else {
4457 return Ok((None, BTreeMap::new()));
4458 };
4459 let Some(start) = body.rfind(METADATA_OPEN) else {
4460 return Ok((Some(body), BTreeMap::new()));
4461 };
4462 let encoded_start = start + METADATA_OPEN.len();
4463 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
4464 return Err(SourceError::Malformed {
4465 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
4466 });
4467 };
4468 let encoded_end = encoded_start + relative_end;
4469 if !body[encoded_end + METADATA_CLOSE.len()..].trim().is_empty() {
4470 return Ok((Some(body), BTreeMap::new()));
4471 }
4472 let metadata = serde_json::from_str(&body[encoded_start..encoded_end]).map_err(|error| {
4473 SourceError::Malformed {
4474 message: format!(
4475 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
4476 ),
4477 }
4478 })?;
4479 let visible = body[..start].trim_end();
4480 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
4481}
4482
4483fn compose_body(
4484 content: Option<&str>,
4485 metadata: &BTreeMap<String, Value>,
4486) -> Result<Option<String>, SourceError> {
4487 let visible = content.unwrap_or_default();
4488 if metadata.is_empty() {
4489 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
4490 }
4491 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
4492 message: error.to_string(),
4493 })?;
4494 Ok(Some(if visible.is_empty() {
4495 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4496 } else {
4497 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4498 }))
4499}
4500
4501fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
4502 value
4503 .get(field)
4504 .and_then(Value::as_bool)
4505 .ok_or_else(|| SourceError::Malformed {
4506 message: format!("GitHub response is missing boolean field {field}"),
4507 })
4508}
4509fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
4510 match value.get(field) {
4511 None | Some(Value::Null) => Ok(None),
4512 Some(value) => value
4513 .as_str()
4514 .map(Some)
4515 .ok_or_else(|| SourceError::Malformed {
4516 message: format!("GitHub response field {field} is not a string or null"),
4517 }),
4518 }
4519}
4520fn optional_nodes<'a>(
4521 connection: Option<&'a Value>,
4522 name: &str,
4523) -> Result<Option<&'a Vec<Value>>, SourceError> {
4524 match connection {
4525 None | Some(Value::Null) => Ok(None),
4526 Some(value) => value
4527 .get("nodes")
4528 .and_then(Value::as_array)
4529 .map(Some)
4530 .ok_or_else(|| SourceError::Malformed {
4531 message: format!("GitHub {name}.nodes is not an array"),
4532 }),
4533 }
4534}
4535fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
4536 let page_info = connection
4537 .get("pageInfo")
4538 .ok_or_else(|| SourceError::Malformed {
4539 message: format!("GitHub {name} has no pageInfo"),
4540 })?;
4541 if required_bool(page_info, "hasNextPage")? {
4542 return Err(SourceError::Malformed {
4543 message: format!(
4544 "GitHub {name} exceeds the supported nested connection size of {size}"
4545 ),
4546 });
4547 }
4548 Ok(())
4549}
4550fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
4551 optional_str(value, field)?
4552 .map(|timestamp| {
4553 timestamp.parse().map_err(|error| SourceError::Malformed {
4554 message: format!("GitHub response field {field} is not a timestamp: {error}"),
4555 })
4556 })
4557 .transpose()
4558}
4559fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
4560 if page.limit == 0 {
4561 Err(SourceError::Config {
4562 message: "page limit must be at least 1".into(),
4563 })
4564 } else {
4565 Ok(())
4566 }
4567}
4568fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
4569 let page = connection
4570 .get("pageInfo")
4571 .filter(|value| value.is_object())
4572 .ok_or_else(|| SourceError::Malformed {
4573 message: "GitHub connection is missing pageInfo".into(),
4574 })?;
4575 if required_bool(page, "hasNextPage")? {
4576 let cursor = required_str(page, "endCursor")?;
4577 validate_cursor_progress(None, cursor)?;
4578 Ok(Some(Cursor(cursor.into())))
4579 } else {
4580 Ok(None)
4581 }
4582}
4583fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
4584 if next.is_empty() || previous == Some(next) {
4585 Err(SourceError::Malformed {
4586 message: "GitHub pagination cursor is empty or did not advance".into(),
4587 })
4588 } else {
4589 Ok(())
4590 }
4591}
4592fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
4593 cursor.map_or(Ok(0), |c| {
4594 c.0.parse().map_err(|_| SourceError::Config {
4595 message: "page cursor is invalid".into(),
4596 })
4597 })
4598}
4599fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
4600 if offset > items.len() {
4601 return Page::last(vec![]);
4602 }
4603 let tail = items.split_off(offset);
4604 let mut selected = tail;
4605 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
4606 selected.truncate(limit);
4607 Page {
4608 items: selected,
4609 next,
4610 }
4611}