onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so a write without [`GitHubProjectsConfig::repository`] is refused
11//! naming the field — but that repository is the *fallback*, not the home of every item.
12//!
13//! <!-- llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] The rule's one
14//! executable source is `GitHubProjectsSource::creation_target`; this is where a reader of
15//! the module meets it, and `tests/plugin.rs` drives every arm below against the loopback
16//! board and asserts on `createIssue`'s own `repositoryId`, so the prose cannot outlive a
17//! change to the rule. -->
18//! **Which repository an issue is created in is decided by the item's own `repositories`
19//! field, under one rule.** Exactly one entry names the repository the issue is created in:
20//! a task issue is where a person finds the work from the repository it changes, and one
21//! filed in a board's nominated repository is invisible from every other. Zero entries, or
22//! two or more, name none, so a task's or a document's issue is created in the repository
23//! its parent project's issue lives in — read from the board, or from this process's own
24//! record of a project it created earlier in the same command — and a project's issue, or
25//! a task or document written with no parent, is created in the configured `repository:`.
26//! What that rule refuses, it refuses before `createIssue`, so no issue is half-created. An
27//! existing issue is never moved: the update path leaves the issue where it is and records
28//! the list in the metadata slot when it differs, so the read side's derivation and the
29//! creation rule agree by construction.
30//! <!-- llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate] -->
31//!
32//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
33//! board has no document type and nothing but issues to hold one in, so the title is the
34//! discriminator and it is the whole of it. The title this source *reports* is the one a
35//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
36//! the body so `content` is what the person wrote — and writing a document puts the prefix
37//! back, so a round trip returns the title that went in.
38//!
39//! **Telling a document from a project from a task.** The design prefix is read **first**:
40//! a document is never a project and never a task, whatever sub-issues it has or does not
41//! have. Only then does the rest apply — a board issue is a project when *either* it has
42//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
43//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
44//! necessary: it is what makes an *empty* project — the state a project copy passes
45//! through between creating the project and filing its first task — readable as a
46//! project, while the sub-issue arm lets a person author a project on the board by hand
47//! with no knowledge of this product's metadata at all. Reading the prefix later than the
48//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
49//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
50//! task nor a document and are ignored.
51//!
52//! **A task's comments are its issue's comments.** They are read off `Issue.comments` and
53//! written with `addComment`, `updateIssueComment` and `deleteIssueComment`, and a comment's
54//! id is GitHub's own node id for the `IssueComment`. Two things GitHub decides are refused
55//! rather than papered over: a board **draft** is not an issue and has no comments at all, so
56//! a comment call on one is refused rather than answered with an empty page; and GitHub signs
57//! every comment as the account the token belongs to, so a comment handed an author of its
58//! own is refused rather than posted under another name. GitHub's comment mutations take the
59//! comment's id and nothing else, so an edit or a delete first reads which issue that comment
60//! is on, and a comment on some other issue is one this task does not have.
61//!
62//! **Where an entity is, is a link.** Every project, task and document this source reports
63//! carries a [`Location::Url`] naming the issue's own web address — the same address the
64//! `url` field already reports, in the shape that says a reader can open it. That is the
65//! contrast the location contract exists for: a reader holding an entity from this source
66//! is handed something to link to and one holding an entity from a folder of Markdown is
67//! handed a path, and neither has to know which plugin answered. It does not replace or
68//! derive from `url`; that field goes on reporting what it always reported.
69//!
70//! **Where metadata lives.** Short typed things go to typed fields and native relations:
71//! status to the board's `Status` single-select and the issue's own state, the copy
72//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
73//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
74//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
75//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
76//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
77//! why neither can hold a caller's own prose.
78//!
79//! **Status.** `status_mapping` is per-instance configuration from a status category to
80//! `null`, a board `Status` option name, or a closed state of `completed` or
81//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
82//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
83//! and a mistake destroys every item's status. A status this board cannot represent is a
84//! refusal naming the status and the instance instead.
85//!
86//! `unknown` is disabled by default because this source cannot preserve an open-ended
87//! status word: it writes an existing board option or the issue's closed state and never
88//! creates an option. An operator may map `unknown` to one existing option, in which case
89//! every unknown word lands on that option and reads back as `unknown` under the option's
90//! name. Mapping it to a closed state is refused because GitHub reads that state back as
91//! `done` or `cancelled`, changing the category on every copy. This differs from
92//! `local-md`, which writes and reads the original word itself.
93//!
94//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
95//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
96//! finished tasks were only moved to a "Done" column would read 0% complete forever.
97//!
98//! # What this source declares, field by field
99//!
100//! One verdict per field of [`Capabilities`], and what `Native` means when this source
101//! says it. *Proven* means a shared journey drives it against the real
102//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
103//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
104//! [`capabilities`](TaskSource::capabilities) from parting.
105//!
106//! | Field | Verdict |
107//! | --- | --- |
108//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
109//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
110//! | `comments` | **Supported and proven,** over the task issue's own comment connection, oldest first and paged by GitHub's own cursor; added, edited and removed through GitHub's comment mutations, paced as every other mutation is. A draft item has no comments on GitHub and is refused, and so is an author, because GitHub records the signed-in account as every comment's author. |
111//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
112//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
113//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
114//! | `search_title` | **Supported and proven,** over `Issue.title`. |
115//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
116//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
117//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
118//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
119//!
120//! Nothing here is unsupported. `documents` and `comments` are not predicates — they say this
121//! source has documents and that its tasks have comments, both of which hold — and the three
122//! facts behind the uniform `Native` on the
123//! predicates beside it are recorded below rather than re-derived, because a reader who
124//! takes `Native` to mean *the remote service filters* will read that uniformity as a
125//! lie.
126//!
127//! First, the plugin contract defines `Support::Native` as *the source applies this
128//! predicate itself*, and says nothing about where it applies it. What the declaration
129//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
130//! — so that the engine may push it down and apply nothing of its own.
131//!
132//! Second, this source can keep that promise for every predicate at no additional API
133//! cost, because whichever of the three reads below answers a query has already read every
134//! item that query could keep before it filters anything. Filtering those items is
135//! in-process work over data already in hand.
136//!
137//! Third, no predicate but `projects` could be pushed into the API even if that were
138//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
139//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
140//! label set, a status column or a substring of a body, and its title qualifier matches
141//! tokens where this source — and the local Markdown source beside it — match substrings,
142//! so pushing a search down would silently *narrow* the answer. What a project filter has
143//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
144//! the issue for them is both cheaper and exact. So there is one predicate this source
145//! applies by asking a narrower question, six it applies in process, and none it is unable
146//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
147//! source has already done, and declaring `projects` native while ignoring the filter
148//! (which this source once did) silently returns another project's tasks, because the
149//! engine trusts the declaration and applies nothing locally.
150//!
151//! # The three ways this source reaches an item, and what each costs
152//!
153//! A board read is charged for what its *nested* connections could return rather than for
154//! what was asked, so one whole-board read costs the same whether the question was about
155//! one project or about all of them. That is why a question about one project is never
156//! answered by reading the board:
157//!
158//! | The question | What is sent | What it costs |
159//! | --- | --- | --- |
160//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
161//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
162//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
163//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
164//! | which board item one issue is, past the page that came with it | [`graphql::ISSUE_BOARD_ITEMS`] — that issue's own `projectItems` | one issue's memberships |
165//!
166//! The board half of an issue — its board item's id, its `Status` option and this
167//! source's origin text field — rides along on `Issue.projectItems` in the first three, so
168//! an item reached any of those ways resolves through the same
169//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
170//! same status, the same labels and the same qualified id. That connection comes back a
171//! *page* at a time, at `BOARD_ITEMS_PAGE_SIZE`, so the entry for this board is looked for
172//! on the page in hand and — only if that page reports more of the connection — in the
173//! last row's read of that one issue's memberships, resumed from the page's own cursor and
174//! walked to exhaustion. An issue with no entry for *this* board is not this source's to
175//! report, which is what keeps an id naming another repository's issue from being answered
176//! as an item of this board; and because the page is where the search starts rather than
177//! where it ends, that answer is one about a connection read to exhaustion and never about
178//! an unread page. Nothing costs the extra read but an issue on more boards than a page
179//! holds: an issue this board really does not hold reports no next page, so its
180//! memberships are already exhausted where they arrived.
181//!
182//! **No document here selects the board's own `Labels` field, and nothing is lost by
183//! that.** An item's labels are read from its content alone, wherever that content is
184//! reached: the three documents above select `Issue.labels` on the fragment, and
185//! [`graphql::BOARD`] selects the same connection on the `... on Issue` arm of its
186//! `content`. A board's `Labels` field is not one anybody fills in: it is a built-in
187//! `ProjectV2FieldType`, it is absent from `ProjectV2CustomFieldType` so no project can
188//! create one, and `ProjectV2FieldValue` — the whole of what
189//! `updateProjectV2ItemFieldValue` accepts — offers no way to write one. So GitHub derives
190//! it from the content, for every content type it exists on, and there is nothing it can
191//! hold that the content does not already say: for an `Issue` it *is* that issue's own
192//! labels, so selecting it beside them unions a set with itself.
193//!
194//! **A draft loses nothing by that either**, which is the reasoning this paragraph once had
195//! backwards. `DraftIssue` exposes no `labels` field, and by the three schema facts above
196//! it cannot carry a board `Labels` value to be derived from one — so a draft has nothing
197//! to select *and nothing to lose*, and reports no labels at all. A `PullRequest` item is
198//! discarded by [`GitHubProjectsSource::resolve`] before labels are read. Both halves are
199//! held to that by tests in `tests/plugin.rs`: the four ways an item is reached report one
200//! label set, and that set is the fixture issue's own, by
201//! `an_item_reports_the_same_labels_title_status_and_id_however_it_is_reached`; and a board
202//! item whose content is a draft reports an empty set, by
203//! `a_board_item_whose_content_is_a_draft_reports_no_labels_at_all`. The absence of the
204//! selection is held over [`graphql::DOCUMENTS`] by
205//! `no_document_selects_the_boards_own_labels_field`.
206//!
207//! The last row is still the board's own item connection, and deliberately: a **draft**
208//! board item is not an issue, so no search and no node read can reach one, and the reads
209//! that have to answer for the whole board are the ones whose cost is the board's size
210//! anyway.
211//!
212//! **What a read may return is capped too, and that cap is on the document rather than on
213//! the board.** GitHub limits the number of nodes **one query may return** to
214//! [`NODE_COUNT_LIMIT`] and refuses a query above that before executing it: the answer is
215//! an error naming the connection the count crossed at, not a slow or a partial result.
216//! Every board this source reads is refused the same way, so no board is too big for these
217//! documents and none is small enough to save one that is over.
218//!
219//! The count is arithmetic over the document's own text: each connection contributes the
220//! `first:` it asks for, counts **multiply** down a nested path and **sum** across sibling
221//! paths. Those are [GitHub's published rules][node-limits] and this workspace does not
222//! restate them — `github-graphql-node-count` implements them, and
223//! [`worst_case_node_count`] under [`largest_page_sizes`] is where every node count here
224//! comes from. `every_document_this_source_sends_stays_under_githubs_node_limit`, in
225//! `tests/node_count.rs`, recomputes every document in [`graphql::DOCUMENTS`] from that
226//! same text on every run and fails naming any that reaches the limit — so a connection
227//! added to a shared fragment is caught there rather than by GitHub.
228//!
229//! What decides those counts is the page sizes: [`MAX_PAGE_SIZE`] on the outer page,
230//! `NESTED_PAGE_SIZE` on the connections hanging off one item, and
231//! `BOARD_ITEMS_PAGE_SIZE` on the page of an issue's board memberships a read carries.
232//! `$nestedFirst` is spent twice down one path of a board read, so that constant is
233//! effectively squared there, which is why it is the one the limit is most sensitive to.
234//! `BOARD_ITEMS_PAGE_SIZE` is small for a reason of its own, recorded beside it: what a
235//! page of memberships misses is recovered by one further read rather than refused, so it
236//! buys a bound every read pays for at the price of a request only a multi-board issue
237//! pays.
238//!
239//! **`nodeCount` and `cost` are two numbers against two limits, and both are computed
240//! offline here — per document, one document at a time.** `nodeCount` is the one above: the
241//! most nodes one query may return, checked per query and bounded by [`NODE_COUNT_LIMIT`].
242//! `cost` is rate-limit points, metered per hour across everything one credential does; it
243//! is what the two limiters [`Limiter`] tells apart meter, and a document under
244//! [`NODE_COUNT_LIMIT`] still says nothing about its price. [`worst_case_point_cost`] is
245//! that second number, and `tests/point_cost.rs` pins every document in
246//! [`graphql::DOCUMENTS`] at what it costs — there being no per-call point ceiling to hold
247//! one under, the pin itself is the check. The credentialed lane reconciles both figures
248//! against GitHub's own, off a probe it already sends.
249//!
250//! **What is pinned that way is a per-document price and never a session's.** The record in
251//! `session-cost.md` measures the two quantities a whole session can be counted in offline —
252//! **requests** and **worst-case nodes** — and neither is points. What one whole session
253//! consumes of the hourly point allowance is observable only from a credentialed run's own
254//! `x-ratelimit-*` headers, which is what [`accounting`] fills its per-budget figures from
255//! and what `tests/live.rs` prints at the end of every run.
256//!
257//! [node-limits]: https://docs.github.com/en/graphql/overview/rate-limits-and-node-limits-for-the-graphql-api
258//!
259//! **Where a read-after-write guarantee comes from, since a search index cannot supply
260//! one.** GitHub's issue search is eventually consistent and answers a write made moments
261//! ago with the value from before it. Resolving a node id is not, so a read by id and a
262//! project's own sub-issues are already current. What closes the gap for the search is
263//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
264//! what this process itself wrote, so an item created seconds ago is reported whether or
265//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
266//! and the record dies with the process.
267//!
268//! Filtering happens before paging, so a page of a filtered result is a page of the
269//! survivors rather than the survivors of a page. Label and text matching answer the same
270//! question the same way the local Markdown source's do, so one cross-source expectation
271//! holds for both.
272//!
273//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
274//! has one source, `capabilities`, and the note above is the reasoning behind it rather
275//! than a second copy of it: without the three facts recorded here a reader takes the
276//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
277//! crate's own capabilities test, which pins every field of it against a fully spelled-out
278//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
279//! fails to compile there rather than going unasserted. -->
280//! The fixture-server tests above run wherever this crate is selected; the credentialed
281//! lane runs in the same required check, beside them, and can fail it — it verifies the
282//! current schema, then drives every field of the table above against the real board. It builds its own fixture there — two projects, one task filed under each,
283//! one filed under neither, a label on one of the three and a closed status on another —
284//! because that shape is what tells an honoured predicate from an ignored one: a board
285//! holding a single project answers a project filter the same way whether or not this
286//! source applies it, which is exactly how the defect above went unseen.
287//!
288//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
289//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
290//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
291//! nominated is what keeps a credentialed write lane off a board and a repository nobody
292//! nominated; it never asks GitHub which project was updated most recently. Before it
293//! starts, the lane also clears any item titled — and any repository label named — the way
294//! it titles and names its own artifacts, which is self-healing after an interrupted run:
295//! a process killed between its writes and its cleanup leaves artifacts the next run
296//! removes.
297//!
298//! # What a session of requests costs, and where the report is
299//!
300//! This source records **every** request it sends into [`accounting::Accounting`], at
301//! `send_once` — the one place a request leaves this crate, which is why a read path added
302//! later is counted without anybody remembering to count it. That is the whole of what this
303//! crate adds to the arrangement; [`accounting`] is where what a record carries, how a
304//! session's spend is arrived at, and what it deliberately does not know are set out.
305//!
306//! What one whole session of the live journey costs, counted that way against this crate's
307//! loopback fixture board, is written down in `session-cost.md` beside this crate — with the
308//! reduction it came out of, and with what it does and does not say about rate-limit points.
309//!
310//! [`GitHubProjectsSource::accounting`] is the read: a snapshot to hold and compare, which
311//! [`accounting::Session::report`] renders the session report from. It is on the ordinary
312//! code path — no environment variable, no feature, no build configuration — because an
313//! instrument nobody switches on measures nothing, and
314//! [`Plugin::build_recording_into`] is how a caller making its own calls beside this
315//! source's counts the whole session rather than this source's share. The credentialed lane
316//! in `tests/live.rs` does exactly that, and prints the report at the end of every run,
317//! passed or failed.
318//!
319//! **A live session refuses to start unless the account can afford it.** Before it does any
320//! of the work it exists to do, the journey makes one request — `GET /rate_limit`, which
321//! GitHub documents as not counting against the REST rate limit and which answers both of
322//! its budgets at once — and starts only if, for each of them, what remains minus this
323//! session's estimated cost is still at least
324//! `onetaskgraph_live::RETAINED_BUFFER` — twenty per cent — of that budget's whole
325//! allowance. A session that cannot **declines**: it did not run, so it is
326//! neither a pass nor a failing assertion, and it says which budget was short, that budget's
327//! limit, what remained, the estimate, the buffer and when it resets — then stops, without
328//! waiting for the budget to come back. The estimate is derived offline from
329//! `tests/fixtures/session-cost.txt` and a cost model stated in `tests/journey/budget.rs`,
330//! which is also where the published rule that model rests on is cited; the accounting
331//! above records the gate's own read like any other request, and
332//! [`accounting::Session::report`] prints the estimate beside what the session really spent.
333//!
334//! **GitHub is the authority on both of its own numbers, and the credentialed lane goes and
335//! asks it.** Everything above computes `nodeCount` and `cost` offline from a document's own
336//! text, which is what lets it run on every platform and on a pull request from a fork with
337//! no credential — and that is what actually stops a regression merging. But an offline
338//! arithmetic can only ever agree with itself: if GitHub changes its rules, this workspace
339//! goes on computing the old answer and nothing notices. So `tests/live.rs` reconciles them.
340//! GitHub's schema exposes `rateLimit(dryRun: true)`, whose `nodeCount` is *"the maximum
341//! number of nodes this query may return"* and whose `cost` is what that document would
342//! spend, both for a document **without executing it**, and the lane asks it for every query
343//! document this source sends, under the largest bindings this source sends, and fails when
344//! GitHub's figure and [`worst_case_node_count`] or [`worst_case_point_cost`] disagree. A
345//! mutation is skipped, because `rateLimit` is a field of `Query` and cannot be asked about
346//! one; the offline pins still cover it. It records what those calls reported about the
347//! account's own allowance, because whether asking is free is a thing to observe rather than
348//! to assume. Two quantities, not one: [`NODE_COUNT_LIMIT`] bounds `nodeCount` per query,
349//! and `cost` is metered against an hourly allowance the accounting above reads off a
350//! credentialed run's own response headers.
351//!
352//! **GitHub has two rate limiters and this source is refused by both, so nothing here
353//! treats them as one thing.** The primary budget is the hourly allowance `gh api
354//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
355//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
356//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
357//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
358//! one part of.
359#![deny(missing_docs)]
360
361use std::collections::BTreeMap;
362use std::sync::{Arc, Mutex};
363use std::time::{Duration, Instant};
364
365use chrono::{DateTime, Utc};
366use onetaskgraph_plugin_api::{
367 Capabilities, Comment, CommentBody, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind,
368 DependencySupport, Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label,
369 LabelFilter, Location, Metering, NativeId, NewComment, Page, PageRequest, Project,
370 ProjectFilter, ProjectQuery, Repository, SecretResolver, SourceError, SourceName, SourcePlugin,
371 Status, StatusCategory, Support, Task, TaskQuery, TaskRef, TaskSource, TextFields, TextQuery,
372 WriteSupport,
373};
374use reqwest::{Client, StatusCode, Url};
375use schemars::{Schema, schema_for};
376use secrecy::{ExposeSecret, SecretString};
377use serde::Deserialize;
378use serde_json::{Value, json};
379
380pub mod accounting;
381
382use accounting::Accounting;
383
384/// The registry name for this plugin.
385pub const KIND: &str = "github-projects";
386/// GitHub's maximum connection page size.
387pub const MAX_PAGE_SIZE: u32 = 100;
388
389/// The most nodes any one document this source sends may be asked to return.
390///
391/// GitHub's own published per-query ceiling, taken from
392/// [`github_graphql_node_count::NODE_LIMIT`] rather than written out again here, so this
393/// workspace cannot hold a stale copy of somebody else's number. A query above it is
394/// **refused before it is executed**, whoever is asking and whatever board they are
395/// asking about — so this is a bound on the documents rather than a budget that runs out.
396///
397/// This is `nodeCount`, the maximum number of nodes *one query may return*. It is not
398/// `cost`, the rate-limit points a call spends against an hourly allowance shared by
399/// everything the credential does — two numbers against two limits, and this constant
400/// bounds only the first. The second is computed offline too, per document:
401/// [`worst_case_point_cost`], pinned for every document in [`graphql::DOCUMENTS`] by
402/// `tests/point_cost.rs`, and reconciled against GitHub's own `cost` by the credentialed
403/// lane. There is no constant like this one to hold a price under, because points are an
404/// hourly allowance rather than a per-call bound.
405///
406/// Neither is a session's price. What `session-cost.md` records of a whole session is its
407/// **requests** and its **worst-case nodes**; what a whole session spends in points is
408/// reported only by a credentialed run's own `x-ratelimit-*` headers, through
409/// [`accounting`]. The module section on the three ways this source reaches an item says how
410/// the count is arrived at, and which of the page sizes below decide it.
411pub const NODE_COUNT_LIMIT: u64 = github_graphql_node_count::NODE_LIMIT;
412
413/// Nested connection size for the connections that hang off one item.
414///
415/// It multiplies through every document that reaches an item under a page — the count
416/// rules multiply down a nested path — so it is the constant [`NODE_COUNT_LIMIT`] is most
417/// sensitive to. `tests/node_count.rs` is what holds the pair together: it recomputes
418/// every document under these constants and fails naming any that reaches the limit, so
419/// raising this is caught there rather than by GitHub.
420const NESTED_PAGE_SIZE: u32 = 50;
421/// How many of one issue's board memberships are read when an issue is reached directly.
422///
423/// An issue reached through a search or through its own node id carries its board half in
424/// `Issue.projectItems`, and only the entry for *this* board is read. This connection sits
425/// under a page of issues, so every point of it multiplies through the whole document and
426/// is paid for whether or not any issue is on a second board — which is why it is
427/// deliberately far smaller than [`NESTED_PAGE_SIZE`].
428///
429/// **Three, because what a page misses is now recovered rather than refused**, and the
430/// recovery is what the value is chosen against. An issue whose entry for this board sits
431/// past this page costs one further request — [`graphql::ISSUE_BOARD_ITEMS`], resumed from
432/// that page's own cursor — so the value trades a bound every read pays for a request only
433/// a multi-board issue pays. At one, a deployment whose issues commonly sit on two or more
434/// boards would pay that request *per issue*, which is order N against the one page per
435/// hundred issues a read costs today. At three it is only reached by an issue on four or
436/// more boards at once, which keeps the recovery path exceptional rather than routine for
437/// a plausible deployment.
438const BOARD_ITEMS_PAGE_SIZE: u32 = 3;
439
440pub use github_graphql_node_count::{NodeCountError, Variables};
441
442/// The largest value this source can bind to each page-size variable its documents name.
443///
444/// Every `first:` in [`graphql`] reads one of these three, and each is capped at the
445/// constant above it wherever a caller's own limit could reach it — `$first` at
446/// [`MAX_PAGE_SIZE`], `$nestedFirst` at `NESTED_PAGE_SIZE`, `$boardItems` at
447/// `BOARD_ITEMS_PAGE_SIZE`. So this is the worst case a caller can drive this source to,
448/// not one configuration of it, which is what makes a bound computed under it a bound on
449/// every read.
450pub fn largest_page_sizes() -> Variables {
451 Variables::from([
452 ("first".to_owned(), MAX_PAGE_SIZE),
453 ("nestedFirst".to_owned(), NESTED_PAGE_SIZE),
454 ("boardItems".to_owned(), BOARD_ITEMS_PAGE_SIZE),
455 ])
456}
457
458/// The most nodes `document` could be asked to return, by GitHub's published rules.
459///
460/// Computed offline from the document's own text under [`largest_page_sizes`] — no
461/// network, no credential and no schema — by
462/// [`github_graphql_node_count::node_count`], which is where the rules themselves live.
463/// A document at or above [`NODE_COUNT_LIMIT`] is one GitHub refuses before executing, so
464/// this is what a check holds every document in [`graphql::DOCUMENTS`] below.
465///
466/// # Errors
467///
468/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
469/// no single operation, or binds a page size this source does not name — each of which is
470/// a defect in the document rather than a number.
471pub fn worst_case_node_count(document: &str) -> Result<u64, NodeCountError> {
472 node_count(document, &largest_page_sizes())
473}
474
475/// The most rate-limit points one call of `document` could spend, by GitHub's published
476/// rules.
477///
478/// Computed offline from the document's own text under [`largest_page_sizes`] — no
479/// network, no credential and no schema — by
480/// [`github_graphql_node_count::point_cost`], which is where the rules themselves live.
481/// This is `cost`, metered **per hour** against the allowance one credential shares across
482/// everything it does; it is not `nodeCount`, which is [`worst_case_node_count`] and is
483/// bounded per query by [`NODE_COUNT_LIMIT`]. There is no per-call ceiling to hold this
484/// under, so what `tests/point_cost.rs` does with it is pin every document in
485/// [`graphql::DOCUMENTS`] at what it costs, and the credentialed lane reconciles those
486/// figures against GitHub's own reported `cost`.
487///
488/// # Errors
489///
490/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
491/// no single operation, or binds a page size this source does not name — each of which is
492/// a defect in the document rather than a number.
493pub fn worst_case_point_cost(document: &str) -> Result<u64, NodeCountError> {
494 github_graphql_node_count::point_cost(document, &largest_page_sizes())
495}
496
497/// The most nodes `document` could be asked to return under `variables`.
498///
499/// [`worst_case_node_count`] is this under [`largest_page_sizes`], and the accounting in
500/// [`accounting`] is this under the bindings one request really sent — one spelling of the
501/// calculation, so a bound checked offline and a cost recorded at run time cannot come to
502/// disagree. The rules themselves live in [`github_graphql_node_count::node_count`].
503///
504/// # Errors
505///
506/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
507/// no single operation, or binds a page size `variables` does not name.
508pub fn node_count(document: &str, variables: &Variables) -> Result<u64, NodeCountError> {
509 github_graphql_node_count::node_count(document, variables)
510}
511
512/// The issue-title prefix that makes a board issue a document.
513///
514/// A GitHub Projects board has no document type — it holds issues — so the discriminator
515/// is the title, and this is the whole of it: an issue whose title begins with these bytes
516/// is a document and every other issue is the task or project the sub-issue rule makes it.
517///
518/// It is spelled **once**, here, and read rather than restated everywhere else — including
519/// by the shared journeys, which take it from this constant so a board fixture cannot
520/// drift from what this source reads. `docs/metadata.md` records the two consequences that
521/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
522/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
523/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
524pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
525
526/// Exact GraphQL query documents issued by this plugin.
527///
528/// Keeping the production documents here lets the pinned-schema test validate the same
529/// bytes that are sent to GitHub, rather than a test-only copy which could drift
530/// independently. No document in this module writes the board itself, and none of them
531/// names `updateProjectV2Field`.
532pub mod graphql {
533 /// The board half of one item: the field values every document here reads it from.
534 ///
535 /// A macro for the same reason [`board_issue!`] below is one, a level further in. This
536 /// selection is needed by that fragment, by [`BOARD`] under the board's own `items`,
537 /// and by [`ISSUE_BOARD_ITEMS`] under a membership walk — and all three have to produce
538 /// *the same value*, because
539 /// [`GitHubProjectsSource::resolve`](super::GitHubProjectsSource) reads them through
540 /// one path. Three spellings of it is what would drift, so there is one.
541 ///
542 /// The `Status` option and this source's own origin text field are the whole of it. It
543 /// selects no `ProjectV2ItemFieldLabelValue`: GitHub derives that field from the item's
544 /// content, so it holds nothing the content's own `labels` do not already say, and it
545 /// would sit a label connection two page sizes deep.
546 macro_rules! board_item_values {
547 () => {
548 r#"fieldValues(first:$nestedFirst){nodes{
549 ... on ProjectV2ItemFieldSingleSelectValue{name field{
550 ... on ProjectV2SingleSelectField{id name options{id name}}
551 }}
552 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
553 }pageInfo{hasNextPage}}"#
554 };
555 }
556
557 /// Everything this source reads about one issue, wherever it reaches that issue.
558 ///
559 /// A macro rather than a constant so the three documents below can `concat!` it: one
560 /// spelling of these fields is what makes an issue read through the board-scoped
561 /// search, through its own node id, and through its project's sub-issue relationship
562 /// resolve to *the same* item, which is the whole of what
563 /// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
564 ///
565 /// `projectItems` is what carries the board half of an issue: the board item's own id
566 /// and the [`board_item_values!`] above — the `Status` option and this source's origin
567 /// text field — that a `ProjectV2.items` read used to carry. It is asked for on the
568 /// issue rather than on the board, which is what makes the cost of a read proportional
569 /// to what was asked for instead of to the board's size.
570 ///
571 /// It carries a *page* of that connection, at `BOARD_ITEMS_PAGE_SIZE`, and its
572 /// `endCursor` is what [`ISSUE_BOARD_ITEMS`] resumes from when this board's entry is
573 /// not on that page: a page here is where the search for the entry starts rather than
574 /// where it ends.
575 ///
576 /// It does **not** select the board's `Labels` field value, and that is the whole of
577 /// what keeps the three documents below under [`NODE_COUNT_LIMIT`](super::NODE_COUNT_LIMIT):
578 /// a label connection there sits under `fieldValues` under `projectItems` under a page
579 /// of issues, spending `$nestedFirst` twice down one path, and took
580 /// [`SEARCH_ISSUES`] and [`SUB_ISSUES`] to 2,556,100 nodes against a limit of 500,000.
581 /// No label is lost — this is a fragment `on Issue`, whose own `labels` are selected
582 /// above, and that connection is where every label this source reports comes from. No
583 /// document in this module selects the board field any longer, [`BOARD`] included; the
584 /// module documentation records why nothing it could have held is lost.
585 macro_rules! board_issue {
586 () => {
587 concat!(
588 r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
589 labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
590 projectItems(first:$boardItems){nodes{id project{id number}
591 "#,
592 board_item_values!(),
593 r#"}pageInfo{hasNextPage endCursor}}}"#
594 )
595 };
596 }
597
598 /// Every issue of one board, found by a search scoped to that board.
599 ///
600 /// This is how the projects a board holds are listed, and it selects no `items`
601 /// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
602 /// container walked page by page, so nothing nested inside a board item is paid for.
603 /// Which of the issues it returns is a project is then read off `parent` — GitHub
604 /// accepts `-has:parent` as a search qualifier and silently ignores it, so the
605 /// discriminator has to be applied to the field, which is a scalar on the issue and
606 /// costs nothing.
607 pub const SEARCH_ISSUES: &str = concat!(
608 r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
609 search(query:$search,type:$type,first:$first,after:$after){
610 pageInfo{hasNextPage endCursor}
611 nodes{__typename ...BoardIssue}
612 }
613 }"#,
614 board_issue!()
615 );
616
617 /// One issue by its own node id, which is what a qualified id names here.
618 ///
619 /// Strongly consistent, unlike the search above: GitHub's issue search is an index and
620 /// answers a write made moments ago with the value from before it, and resolving a node
621 /// id does not.
622 pub const ISSUE: &str = concat!(
623 r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
624 node(id:$id){__typename ...BoardIssue}
625 }"#,
626 board_issue!()
627 );
628
629 /// One project's tasks: the sub-issues of the issue that project is.
630 ///
631 /// The work this costs is the project's own size. Nothing about it grows as the board
632 /// gains projects, or as those projects gain tasks.
633 pub const SUB_ISSUES: &str = concat!(
634 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
635 node(id:$id){__typename
636 ... on Issue{subIssues(first:$first,after:$after){
637 pageInfo{hasNextPage endCursor}
638 nodes{__typename ...BoardIssue}
639 }}}
640 }"#,
641 board_issue!()
642 );
643
644 /// Reads the board's fields and one page of its items.
645 pub const BOARD: &str = concat!(
646 r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
647 owner:repositoryOwner(login:$owner){
648 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
649 }
650 } fragment Board on ProjectV2 { id title
651 fields(first:$nestedFirst){nodes{
652 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
653 ... on ProjectV2Field{__typename id name}
654 }pageInfo{hasNextPage}}
655 items(first:$first,after:$after){nodes{id "#,
656 board_item_values!(),
657 r#" content{
658 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
659 ... on PullRequest{__typename id}
660 ... on DraftIssue{__typename id title body createdAt updatedAt}
661 }} pageInfo{hasNextPage endCursor}}
662 }"#
663 );
664
665 /// One issue's board memberships alone, walked past the page a read of it carried.
666 ///
667 /// The recovery read behind [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource):
668 /// every document above carries a *page* of `Issue.projectItems`, and an issue on more
669 /// boards than that page holds may have this board's entry past its end. This asks that
670 /// one issue for its memberships and nothing else — the caller already holds the issue —
671 /// so an answer of "this board does not hold it" is only ever given about a connection
672 /// read to exhaustion.
673 ///
674 /// It selects the board item's id, its project number and the same
675 /// [`board_item_values!`] the fragment does, because what it produces is handed to the
676 /// very same resolver: an issue recovered this way reports the same title, the same
677 /// status, the same labels and the same qualified id as one whose entry was on the
678 /// page.
679 ///
680 /// `$first` rather than `$boardItems`: this document reads one issue, so nothing
681 /// multiplies through it and the membership connection can be walked at
682 /// [`MAX_PAGE_SIZE`](super::MAX_PAGE_SIZE) — which is what keeps the recovery to one
683 /// further request for any issue a person really keeps.
684 pub const ISSUE_BOARD_ITEMS: &str = concat!(
685 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!){
686 node(id:$id){
687 ... on Issue{projectItems(first:$first,after:$after){
688 nodes{id project{id number}
689 "#,
690 board_item_values!(),
691 r#"}
692 pageInfo{hasNextPage endCursor}}}
693 }
694 }"#
695 );
696 /// Resolves the configured repository's node id, which creating an issue requires.
697 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
698 /// Reads both dependency directions for one issue, with each far end's own kind — and
699 /// the issue's own body, which is where an edge to another source is recorded, so that
700 /// half of a dependency read needs no second read of the issue or of the board.
701 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
702 ... on Issue{body
703 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
704 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
705 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
706 /// Creates one issue in the configured repository.
707 pub const CREATE_ISSUE: &str =
708 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
709 /// Puts an existing issue on the configured board.
710 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
711 /// Updates an issue's visible fields and its open or closed state in one call.
712 pub const UPDATE_ISSUE: &str =
713 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
714 /// Updates an existing draft's user-visible fields.
715 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
716 /// Updates a text or single-select value on one project item.
717 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
718 /// Files one issue under another as a sub-issue, which is what project membership is.
719 pub const ADD_SUB_ISSUE: &str =
720 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
721 /// Takes one issue back out of its parent.
722 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
723 /// Adds GitHub's native issue blocked-by relationship.
724 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
725 /// Removes one native issue blocked-by relationship.
726 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
727 /// Deletes one issue, which takes its board item with it.
728 ///
729 /// The engine sends this in one situation only: undoing a copy that could not finish,
730 /// over the items that same copy created. Deleting the issue removes the board item
731 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
732 pub const DELETE_ISSUE: &str =
733 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
734
735 /// Everything this source reads about one issue comment, wherever it reaches one.
736 ///
737 /// A macro for the reason [`board_issue!`] is one: a comment listed, a comment just added
738 /// and a comment just edited are handed to one mapper, so they are selected by one
739 /// spelling. `author` is `Actor`, which GitHub answers `null` for an account that no
740 /// longer exists, and `login` is the one member every kind of actor carries.
741 macro_rules! issue_comment {
742 () => {
743 "id author{login} createdAt updatedAt body url"
744 };
745 }
746
747 /// One task's comments: a page of its issue's own `comments` connection.
748 ///
749 /// **No `orderBy`, and that is what makes the page oldest first.** GitHub's only
750 /// `IssueCommentOrder` field is `UPDATED_AT`, which would move a comment to the end of the
751 /// list every time somebody edited it; left unordered the connection answers in the order
752 /// the comments were written, which is the order GitHub documents for the same collection
753 /// over REST — ascending id. Nothing multiplies through it, so `$first` is the whole of its
754 /// node count and the caller's own page size is pushed straight down.
755 pub const ISSUE_COMMENTS: &str = concat!(
756 r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename ... on Issue{comments(first:$first,after:$after){nodes{"#,
757 issue_comment!(),
758 r#"}pageInfo{hasNextPage endCursor}}}}}"#
759 );
760 /// Which issue one comment is on, read before that comment is edited or removed.
761 ///
762 /// GitHub's comment mutations take the comment's id and nothing else, so without this a
763 /// comment id given against the wrong task would change a comment on another issue.
764 pub const COMMENT_ISSUE: &str =
765 r#"query($id:ID!){node(id:$id){__typename ... on IssueComment{id issue{id}}}}"#;
766 /// Adds one comment to an issue, signed as the account the token belongs to.
767 pub const ADD_COMMENT: &str = concat!(
768 r#"mutation($input:AddCommentInput!){addComment(input:$input){subject{id} commentEdge{node{"#,
769 issue_comment!(),
770 r#"}}}}"#
771 );
772 /// Replaces the body of one issue comment.
773 pub const UPDATE_COMMENT: &str = concat!(
774 r#"mutation($input:UpdateIssueCommentInput!){updateIssueComment(input:$input){issueComment{"#,
775 issue_comment!(),
776 r#"}}}"#
777 );
778 /// Removes one issue comment. Its payload carries nothing about the comment it removed.
779 pub const DELETE_COMMENT: &str = r#"mutation($input:DeleteIssueCommentInput!){deleteIssueComment(input:$input){clientMutationId}}"#;
780
781 /// Every document above, with what this source is doing when it sends one.
782 ///
783 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
784 /// name the call that was refused, and a `match` with a catch-all arm would answer a
785 /// document added later with "talking to GitHub" and never say so.
786 ///
787 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
788 /// const` here that this list omits, so the two cannot part — which is the same guard
789 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
790 pub const DOCUMENTS: [(&str, &str); 22] = [
791 (SEARCH_ISSUES, "searching this board's issues"),
792 (ISSUE, "reading one issue"),
793 (
794 ISSUE_BOARD_ITEMS,
795 "reading one issue's board memberships past the page it came with",
796 ),
797 (SUB_ISSUES, "reading a project's tasks"),
798 (BOARD, "reading the board"),
799 (REPOSITORY, "reading the destination repository"),
800 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
801 (CREATE_ISSUE, "creating an issue"),
802 (ADD_TO_BOARD, "adding an issue to the board"),
803 (UPDATE_ISSUE, "updating an issue"),
804 (UPDATE_DRAFT, "updating a draft item"),
805 (UPDATE_FIELD, "writing a board field"),
806 (ADD_SUB_ISSUE, "filing an issue under its project"),
807 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
808 (ADD_BLOCKED_BY, "recording a dependency"),
809 (REMOVE_BLOCKED_BY, "removing a dependency"),
810 (DELETE_ISSUE, "deleting an issue"),
811 (ISSUE_COMMENTS, "reading a task's comments"),
812 (COMMENT_ISSUE, "reading which issue a comment is on"),
813 (ADD_COMMENT, "adding a comment"),
814 (UPDATE_COMMENT, "editing a comment"),
815 (DELETE_COMMENT, "deleting a comment"),
816 ];
817}
818
819/// Which of GitHub's two rate limiters refused a request.
820///
821/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
822/// secondary one — so an operator told the wrong one takes the wrong next step, which is
823/// the whole reason this is carried rather than collapsed into "rate limited".
824#[derive(Debug, Clone, Copy, PartialEq, Eq)]
825enum Limiter {
826 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
827 Primary,
828 /// The burst limiter over content-generating requests, which nothing reports.
829 Secondary,
830}
831
832/// The wordings GitHub answers a secondary rate limit with.
833///
834/// It sends them under a forbidden status, under a too-many-requests status, and inside
835/// the `errors` of a *successful* response, which is why the text is what this matches on
836/// rather than the status. `abuse detection` is the wording GitHub used before the
837/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
838/// what a burst of content creation is refused with.
839///
840/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
841/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
842/// when, and the drift gate reconciles the two lists both ways. Public for that gate
843/// alone — a caller has no use for it, and matching on a refusal is this source's job.
844pub const SECONDARY_WORDINGS: [&str; 5] = [
845 "secondary rate limit",
846 "temporarily blocked from content creation",
847 "abuse detection",
848 "submitted too quickly",
849 "exceeded a secondary",
850];
851
852/// The wordings GitHub answers an exhausted primary budget with.
853///
854/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
855/// than looked for in the response text. Pinned and gated exactly as
856/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
857pub const PRIMARY_WORDINGS: [&str; 3] = [
858 "api rate limit exceeded",
859 "rate limit exceeded",
860 "rate_limited",
861];
862
863/// What a response *says about itself*, which is the only place a refusal can be read.
864///
865/// Deliberately not the whole response body. A board is a place people write about their
866/// own work, and a task on it titled "the secondary rate limit" would, matched across the
867/// raw text, turn a perfectly good answer into a refusal this source then waited out and
868/// reported. So the item data is never read: what is read is GitHub's own REST-style
869/// `message` envelope, which is what a forbidden status carries, and the `message` and
870/// `type` of each GraphQL error, which is where a *successful* response says it.
871///
872/// A body that is not JSON at all has nothing structured to read, so only a failing
873/// response's own text is taken — a successful response that is not JSON is malformed
874/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
875fn refusal_wording(status: StatusCode, body: &str) -> String {
876 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
877 return if status.is_success() {
878 String::new()
879 } else {
880 body.to_owned()
881 };
882 };
883 let mut said: Vec<&str> = parsed
884 .get("message")
885 .and_then(Value::as_str)
886 .into_iter()
887 .collect();
888 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
889 for error in errors {
890 said.extend(
891 ["message", "type"]
892 .into_iter()
893 .filter_map(|key| error.get(key).and_then(Value::as_str)),
894 );
895 }
896 }
897 said.join("; ")
898}
899
900impl Limiter {
901 /// Which limiter refused this response, or `None` when none of them did.
902 ///
903 /// The wording is read first and the status only decides what carries none of it,
904 /// because GitHub answers a secondary limit with a forbidden status far more often
905 /// than with too-many-requests — while a forbidden status saying nothing about a limit
906 /// really is a credential this token lacks.
907 ///
908 /// A response is a refusal because of its status or its own wording. A spent budget
909 /// only ever explains one; it never turns an answer into a refusal.
910 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
911 let normalized = refusal_wording(status, body).to_ascii_lowercase();
912 if SECONDARY_WORDINGS
913 .iter()
914 .any(|wording| normalized.contains(wording))
915 {
916 return Some(Self::Secondary);
917 }
918 if status == StatusCode::TOO_MANY_REQUESTS {
919 return Some(Self::Primary);
920 }
921 // An exhausted budget *explains* a response that failed; it does not make one that
922 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
923 // request the budget allowed as well as on the ones it then refuses, so reading
924 // the header alone threw away a good answer — and, once refusals were retried,
925 // replayed a request that had already taken effect.
926 if !status.is_success() && budget_exhausted {
927 return Some(Self::Primary);
928 }
929 // A successful response saying it: GitHub reports a GraphQL rate limit in the
930 // `errors` of an HTTP 200, where nothing about the status says so at all.
931 if status.is_success()
932 && PRIMARY_WORDINGS
933 .iter()
934 .any(|wording| normalized.contains(wording))
935 {
936 return Some(Self::Primary);
937 }
938 None
939 }
940
941 /// What this limiter is called where an operator can look it up.
942 const fn name(self) -> &'static str {
943 match self {
944 Self::Primary => "GitHub's primary API rate limit",
945 Self::Secondary => "GitHub's secondary rate limit",
946 }
947 }
948
949 /// What the endpoint an operator would go and check says about this limiter.
950 const fn where_to_look(self) -> &'static str {
951 match self {
952 Self::Primary => {
953 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
954 comes back."
955 }
956 Self::Secondary => {
957 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
958 primary budget and does not report this one, so budget showing there says \
959 nothing about this refusal, and every further attempt extends it."
960 }
961 }
962 }
963
964 /// The next step this limiter actually calls for.
965 const fn what_to_do(self) -> &'static str {
966 match self {
967 Self::Primary => {
968 "wait for the reset `gh api rate_limit` reports, then run the command again."
969 }
970 Self::Secondary => {
971 "leave this board alone for a few minutes, then run the command again — or \
972 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
973 }
974 }
975 }
976}
977
978/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
979#[derive(Debug, Clone, Copy)]
980struct Limited {
981 limiter: Limiter,
982 hint: Option<u64>,
983}
984
985impl Limited {
986 /// What the caller is told once this source has waited as long as it may.
987 ///
988 /// Both limiters report as [`SourceError::RateLimited`], because that is what
989 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
990 /// about *which* limiter it was makes it a different kind of failure. What differs is
991 /// the operator's next step, and that is what the message carries — a secondary
992 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
993 /// budget looks fine, and then back to retry the very burst that was refused.
994 fn exhausted(
995 self,
996 doing: &str,
997 waits: u32,
998 waited: Duration,
999 needed: Duration,
1000 budget: Duration,
1001 ) -> SourceError {
1002 SourceError::RateLimited {
1003 retry_after_seconds: self.hint,
1004 message: Some(format!(
1005 "{} refused this source while {doing}; it waited {} out over {} and was refused \
1006 again, and the next wait of {} would take it past the {} one call may spend \
1007 waiting. {} next: {}",
1008 self.limiter.name(),
1009 plural(waits, "refusal"),
1010 seconds(waited),
1011 seconds(needed),
1012 seconds(budget),
1013 self.limiter.where_to_look(),
1014 self.limiter.what_to_do(),
1015 )),
1016 }
1017 }
1018}
1019
1020/// One HTTP attempt's result, with what its response said about the rate limit.
1021///
1022/// The two travel together so the record and the outcome are written from the same place:
1023/// what a response said about the budget is only readable while that response is in hand,
1024/// and what the attempt *meant* is only decidable once its body has been read.
1025struct Attempted {
1026 result: Result<Value, Attempt>,
1027 limits: accounting::RateLimit,
1028 /// GitHub's own reported cost for this call, for a document that asked for it.
1029 reported_cost: Option<u64>,
1030}
1031
1032/// One attempt's outcome: an error to report, or a rate limit to wait out.
1033enum Attempt {
1034 Failed(SourceError),
1035 Limited(Limited),
1036}
1037
1038fn plural(count: u32, thing: &str) -> String {
1039 if count == 1 {
1040 format!("{count} {thing}")
1041 } else {
1042 format!("{count} {thing}s")
1043 }
1044}
1045
1046fn seconds(duration: Duration) -> String {
1047 format!("{:.1}s", duration.as_secs_f64())
1048}
1049
1050/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
1051///
1052/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
1053/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
1054/// header, and neither is what makes a response a refusal — so the whole cost of one this
1055/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
1056/// instead. Refusing the response over the header would turn a readable refusal into an
1057/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
1058fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
1059 value
1060 .and_then(|value| value.to_str().ok())
1061 .and_then(|value| value.trim().parse::<u64>().ok())
1062}
1063
1064/// Every mutation this source sends creates content — an issue, a board item, a field of
1065/// one, a sub-issue link, a dependency, a comment — or edits or removes content of that
1066/// kind, and no query in [`graphql::DOCUMENTS`] does, so what the secondary limiter counts
1067/// and what the keyword says are the same set. That is what makes the keyword a sound test
1068/// rather than a convenient one: pacing an edit or a removal the limiter might not have
1069/// counted costs a wait, and not pacing one it did count costs the next fifty minutes.
1070fn is_mutation(query: &str) -> bool {
1071 query.trim_start().starts_with("mutation")
1072}
1073
1074/// What this source was doing, for a diagnostic that has to say so.
1075///
1076/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
1077/// a document added without a description is caught by that list's own gate instead of
1078/// falling through to the vague arm below.
1079fn operation_description(query: &str) -> &'static str {
1080 graphql::DOCUMENTS
1081 .iter()
1082 .find(|(document, _)| *document == query)
1083 .map_or("talking to GitHub", |(_, doing)| *doing)
1084}
1085
1086/// GitHub's published ceiling on content-generating requests, per minute.
1087///
1088/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
1089/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
1090/// from it, so a pacing value checked only against itself cannot go stale here.
1091pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
1092/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
1093/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
1094/// governs; it is pinned beside its sibling so the gate would notice either one moving.
1095pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
1096/// Shortest interval between two content-creating mutations, in milliseconds.
1097///
1098/// GitHub documents two secondary limits on content-generating requests:
1099/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
1100/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
1101/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
1102/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
1103/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
1104/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
1105/// deliberately *not* what this paces at. An installation that wants the hourly bound
1106/// honoured for a long sequence of copies says so through
1107/// `pacing.min_mutation_interval_ms`.
1108pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
1109/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
1110///
1111/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
1112/// own advice for a secondary limit — wait, and wait longer each time — without spending
1113/// the first minute of a transient refusal doing nothing.
1114pub const RETRY_BACKOFF_MS: u64 = 1_000;
1115/// Total time one call may spend waiting out rate limits before it reports a failure.
1116///
1117/// Two minutes is long enough to ride out the refusals a paced copy still collects and
1118/// short enough that a command an operator is watching returns. The bound is what makes
1119/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
1120/// the limiter, not in a process nobody can tell from a wedged one.
1121pub const RETRY_BUDGET_MS: u64 = 120_000;
1122
1123fn default_token_env() -> String {
1124 "GH_PROJECTS_TOKEN".to_owned()
1125}
1126fn default_endpoint() -> String {
1127 "https://api.github.com/graphql".to_owned()
1128}
1129
1130/// Where one status category lands on this board.
1131///
1132/// `null` — an absent value — disables the category for this instance, and using a
1133/// disabled status is a refusal naming the status and the instance.
1134#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
1135#[serde(untagged)]
1136pub enum StatusTargetConfig {
1137 /// The name of a `Status` single-select option already on the board.
1138 Column(ColumnName),
1139 /// A closed issue state, whose reason is what tells done from cancelled.
1140 Closed {
1141 /// The `IssueClosedStateReason` to close with.
1142 closed: ClosedState,
1143 },
1144}
1145
1146/// The name of a `Status` single-select option on the board.
1147///
1148/// Validated on the way in rather than checked later, so a blank option name — which
1149/// nothing on a board can be — is a state this type cannot hold.
1150#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1151#[serde(try_from = "String")]
1152pub struct ColumnName(String);
1153
1154impl ColumnName {
1155 /// The option name, as the board spells it.
1156 fn as_str(&self) -> &str {
1157 &self.0
1158 }
1159}
1160
1161impl TryFrom<String> for ColumnName {
1162 type Error = String;
1163
1164 fn try_from(name: String) -> Result<Self, Self::Error> {
1165 if name.trim().is_empty() {
1166 return Err("a status_mapping option name cannot be blank".to_owned());
1167 }
1168 Ok(Self(name))
1169 }
1170}
1171
1172/// The two closed states this product can mean.
1173///
1174/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
1175/// work nor abandoned work, so nothing here ever writes it.
1176#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1177#[serde(rename_all = "kebab-case")]
1178pub enum ClosedState {
1179 /// `COMPLETED` — precisely done.
1180 Completed,
1181 /// `NOT_PLANNED` — precisely cancelled.
1182 NotPlanned,
1183}
1184
1185impl ClosedState {
1186 const fn reason(self) -> &'static str {
1187 match self {
1188 Self::Completed => "COMPLETED",
1189 Self::NotPlanned => "NOT_PLANNED",
1190 }
1191 }
1192}
1193
1194/// Configuration for one GitHub Projects v2 board.
1195#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1196#[serde(default, deny_unknown_fields)]
1197pub struct GitHubProjectsConfig {
1198 /// Login of the user or organization which owns the board.
1199 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
1200 /// The project number shown in the board's GitHub URL.
1201 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
1202 // llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] This doc is the field's schema description, which is what a person configuring the source reads, so it has to say when the field decides an issue's repository and when the item's own field does; the rule's one executable source is `GitHubProjectsSource::creation_target`, and `tests/plugin.rs` drives each case named here against the loopback board.
1203 /// `owner/name` of the repository this source creates an issue in when the item's own
1204 /// `repositories` field does not decide it.
1205 ///
1206 /// An item naming exactly one repository is created there; a task or a document naming
1207 /// none or several is created in its parent project's repository; and a project, or a
1208 /// task or document with no parent, naming none or several is created here. A board
1209 /// has no repository of its own and `createIssue` requires one, so a write without
1210 /// this is refused naming the field. Reads never need it.
1211 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
1212 // llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate]
1213 /// Environment variable containing a fine-grained token with Projects and Issues
1214 /// read/write plus Pull requests read-only access for every repository represented on
1215 /// the board.
1216 #[serde(default = "default_token_env")]
1217 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
1218 /// GraphQL endpoint. GitHub Enterprise installations may override it.
1219 #[serde(default = "default_endpoint")]
1220 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
1221 /// Per-instance mapping from a status category to where it lands on this board.
1222 ///
1223 /// A category this does not mention keeps its shipped default: `backlog` to
1224 /// "Backlog", `todo` to "Todo", `queued` to "Queued", `in-progress` to "In Progress",
1225 /// `done` to closed as
1226 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
1227 /// disabled. `unknown` may name one existing board option; every unknown word then
1228 /// lands on that option and reads back as `unknown` under its name. It cannot name a
1229 /// closed state because that reads back as `done` or `cancelled`. Unlike `local-md`,
1230 /// this source cannot keep each unknown word because it never creates board options.
1231 #[serde(default)]
1232 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
1233 /// How fast this source writes, and how long it waits out a rate-limit refusal.
1234 ///
1235 /// Every field keeps its shipped default when it is absent, and the defaults are
1236 /// GitHub's own published limits rather than taste. See [`Pacing`].
1237 #[serde(default)]
1238 pub pacing: PacingConfig,
1239}
1240
1241/// How fast this source writes, and how long it waits out a rate-limit refusal.
1242///
1243/// Configurable because a GitHub Enterprise installation sets its own limits and an
1244/// operator who has already been refused may want to go slower still — not because the
1245/// defaults are guesses.
1246#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1247#[serde(default, deny_unknown_fields)]
1248pub struct PacingConfig {
1249 /// Shortest interval between two content-creating mutations, in milliseconds.
1250 ///
1251 /// Zero sends them as fast as they are asked for, which is what a fixture server on
1252 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
1253 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1254 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
1255 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
1256 /// zero while there is a budget to spend, because a schedule of zero-length waits
1257 /// consumes none of it and so never ends.
1258 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
1259 /// Total time one call may spend waiting out rate limits, in milliseconds.
1260 ///
1261 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
1262 /// the bound is what makes this a wait rather than a hang.
1263 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1264}
1265
1266/// The largest any pacing setting may be, in milliseconds.
1267///
1268/// One hour. GitHub's own harshest published bound on content-generating requests works
1269/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
1270/// anything a real limit asks for, and past it the settings stop describing pacing at all:
1271/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
1272/// and an interval beyond it is a command that never sends its second mutation. It also
1273/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
1274/// what an `Instant` can hold on every platform.
1275pub const MAX_PACING_MS: u64 = 3_600_000;
1276
1277/// [`PacingConfig`] with every default resolved and every value checked, which is what the
1278/// source holds.
1279#[derive(Debug, Clone, Copy)]
1280struct Pacing {
1281 min_mutation_interval: Duration,
1282 retry_backoff: Duration,
1283 retry_budget: Duration,
1284}
1285
1286impl Pacing {
1287 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
1288 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
1289 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
1290 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
1291 message: format!(
1292 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
1293 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
1294 GitHub's own harshest published limit"
1295 ),
1296 }),
1297 Some(value) => Ok(Duration::from_millis(value)),
1298 None => Ok(Duration::from_millis(default)),
1299 };
1300 let retry_backoff = bounded(
1301 config.retry_backoff_ms,
1302 RETRY_BACKOFF_MS,
1303 "retry_backoff_ms",
1304 )?;
1305 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
1306 if retry_backoff.is_zero() && !retry_budget.is_zero() {
1307 return Err(SourceError::Config {
1308 message: format!(
1309 "pacing.retry_backoff_ms of source {instance} is 0 while \
1310 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
1311 none of that budget, so it would retry a refusal forever. Set a backoff of \
1312 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
1313 waiting at all",
1314 retry_budget.as_millis()
1315 ),
1316 });
1317 }
1318 Ok(Self {
1319 min_mutation_interval: bounded(
1320 config.min_mutation_interval_ms,
1321 MIN_MUTATION_INTERVAL_MS,
1322 "min_mutation_interval_ms",
1323 )?,
1324 retry_backoff,
1325 retry_budget,
1326 })
1327 }
1328}
1329
1330/// Factory for [`GitHubProjectsSource`].
1331#[derive(Debug, Clone, Copy, Default)]
1332pub struct Plugin;
1333
1334impl SourcePlugin for Plugin {
1335 fn kind(&self) -> &'static str {
1336 KIND
1337 }
1338 fn config_schema(&self) -> Schema {
1339 schema_for!(GitHubProjectsConfig)
1340 }
1341 fn build(
1342 &self,
1343 name: &SourceName,
1344 config: &Value,
1345 secrets: &dyn SecretResolver,
1346 ) -> Result<Box<dyn TaskSource>, SourceError> {
1347 self.build_recording_into(name, config, secrets, Arc::new(Accounting::new()))
1348 }
1349}
1350
1351impl Plugin {
1352 /// Build a source recording every request it sends into an accounting the caller holds.
1353 ///
1354 /// [`SourcePlugin::build`] is this with an accounting of its own, which is what the
1355 /// registry gets. This is for a caller that is also calling GitHub itself and wants one
1356 /// session total rather than two — see [`accounting`] and
1357 /// [`GitHubProjectsSource::recording_into`].
1358 ///
1359 /// # Errors
1360 ///
1361 /// Exactly [`SourcePlugin::build`]'s, with the same source name in front of each:
1362 /// [`SourceError::Config`] for configuration this plugin cannot use and
1363 /// [`SourceError::Auth`] for a credential it cannot find.
1364 pub fn build_recording_into(
1365 &self,
1366 name: &SourceName,
1367 config: &Value,
1368 secrets: &dyn SecretResolver,
1369 ledger: Arc<Accounting>,
1370 ) -> Result<Box<dyn TaskSource>, SourceError> {
1371 let config: GitHubProjectsConfig =
1372 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
1373 message: format!("source {name}: {e}"),
1374 })?;
1375 let source = GitHubProjectsSource::recording_into(name, config, secrets, ledger).map_err(
1376 |error| match error {
1377 SourceError::Config { message } => SourceError::Config {
1378 message: format!("source {name}: {message}"),
1379 },
1380 SourceError::Auth { message } => SourceError::Auth {
1381 message: format!("source {name}: {message}"),
1382 },
1383 other => other,
1384 },
1385 )?;
1386 Ok(Box::new(source))
1387 }
1388}
1389
1390/// Where a status category lands on this board, once configuration is resolved.
1391#[derive(Debug, Clone, PartialEq, Eq)]
1392enum StatusTarget {
1393 /// Not usable against this instance.
1394 Disabled,
1395 /// The board's `Status` option of this name.
1396 Column(ColumnName),
1397 /// A closed issue, with the reason that says which closed it means.
1398 Closed(ClosedState),
1399}
1400
1401/// Every status category, in the order the vocabulary declares them.
1402///
1403/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
1404/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
1405/// added to the shared vocabulary fails to compile until it is named there, and this
1406/// crate's suite reconciles this list against that enum's own derived schema, which is
1407/// generated from the variants rather than written beside them. The schema is what
1408/// catches a list left one short — a list checking only the positions it already holds
1409/// would pass while every mapping indexed by the new position panicked.
1410pub const CATEGORIES: [StatusCategory; 8] = [
1411 StatusCategory::Draft,
1412 StatusCategory::Backlog,
1413 StatusCategory::Todo,
1414 StatusCategory::Queued,
1415 StatusCategory::InProgress,
1416 StatusCategory::Done,
1417 StatusCategory::Cancelled,
1418 StatusCategory::Unknown,
1419];
1420
1421/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
1422#[must_use]
1423pub const fn category_position(category: StatusCategory) -> usize {
1424 match category {
1425 StatusCategory::Draft => 0,
1426 StatusCategory::Backlog => 1,
1427 StatusCategory::Todo => 2,
1428 StatusCategory::Queued => 3,
1429 StatusCategory::InProgress => 4,
1430 StatusCategory::Done => 5,
1431 StatusCategory::Cancelled => 6,
1432 StatusCategory::Unknown => 7,
1433 }
1434}
1435
1436/// The spelling a status category is configured and reported under.
1437fn category_name(category: StatusCategory) -> &'static str {
1438 match category {
1439 StatusCategory::Draft => "draft",
1440 StatusCategory::Backlog => "backlog",
1441 StatusCategory::Todo => "todo",
1442 StatusCategory::Queued => "queued",
1443 StatusCategory::InProgress => "in-progress",
1444 StatusCategory::Done => "done",
1445 StatusCategory::Cancelled => "cancelled",
1446 StatusCategory::Unknown => "unknown",
1447 }
1448}
1449
1450/// A shipped default's option name.
1451///
1452/// The literals below are this file's own and non-blank, and they are validated by the
1453/// one constructor a configured name goes through rather than beside it.
1454fn shipped_column(name: &'static str) -> ColumnName {
1455 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
1456}
1457
1458/// The shipped default for one category, before this instance's configuration.
1459fn shipped_default(category: StatusCategory) -> StatusTarget {
1460 match category {
1461 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
1462 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
1463 StatusCategory::Queued => StatusTarget::Column(shipped_column("Queued")),
1464 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
1465 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
1466 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
1467 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
1468 }
1469}
1470
1471/// This instance's complete category-to-target mapping, read in both directions.
1472///
1473/// One target per category, held at that category's own [`category_position`], so a
1474/// category missing from the mapping, named twice in it, or filed out of order is a
1475/// state this type cannot hold rather than one [`Self::target`] has to defend against.
1476#[derive(Debug, Clone)]
1477struct StatusMapping {
1478 targets: [StatusTarget; CATEGORIES.len()],
1479}
1480
1481impl StatusMapping {
1482 fn resolve(
1483 configured: BTreeMap<String, Option<StatusTargetConfig>>,
1484 instance: &SourceName,
1485 ) -> Result<Self, SourceError> {
1486 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
1487 for (key, value) in configured {
1488 let category = CATEGORIES
1489 .iter()
1490 .find(|category| category_name(**category) == key)
1491 .ok_or_else(|| SourceError::Config {
1492 message: format!(
1493 "status_mapping names {key:?}, which is not a status category of source \
1494 {instance}; the categories are {}",
1495 CATEGORIES
1496 .iter()
1497 .map(|category| category_name(*category))
1498 .collect::<Vec<_>>()
1499 .join(", ")
1500 ),
1501 })?;
1502 overrides.insert(category_name(*category), value);
1503 }
1504 if let Some(Some(StatusTargetConfig::Closed { closed })) =
1505 overrides.get(category_name(StatusCategory::Unknown))
1506 {
1507 let read_back = match closed {
1508 ClosedState::Completed => StatusCategory::Done,
1509 ClosedState::NotPlanned => StatusCategory::Cancelled,
1510 };
1511 return Err(SourceError::Config {
1512 message: format!(
1513 "status_mapping.unknown of source {instance} cannot target the closed state \
1514 {} because a copy reads that item back as {}; map unknown to one existing \
1515 board Status option instead",
1516 closed.reason().to_ascii_lowercase().replace('_', "-"),
1517 category_name(read_back)
1518 ),
1519 });
1520 }
1521 // `CATEGORIES[position] == category` for every category — the crate's suite
1522 // asserts it — so mapping the list in order fills each category's own slot.
1523 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
1524 None => shipped_default(category),
1525 Some(None) => StatusTarget::Disabled,
1526 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
1527 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
1528 });
1529 let mapping = Self { targets };
1530 for (index, category) in CATEGORIES.into_iter().enumerate() {
1531 let StatusTarget::Column(option) = mapping.target(category) else {
1532 continue;
1533 };
1534 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
1535 matches!(mapping.target(**earlier), StatusTarget::Column(name)
1536 if name.as_str().eq_ignore_ascii_case(option.as_str()))
1537 }) {
1538 return Err(SourceError::Config {
1539 message: format!(
1540 "status_mapping of source {instance} sends both {} and {} to the board \
1541 option {:?}; one option cannot read back as two categories",
1542 category_name(*other),
1543 category_name(category),
1544 option.as_str()
1545 ),
1546 });
1547 }
1548 }
1549 Ok(mapping)
1550 }
1551
1552 fn target(&self, category: StatusCategory) -> &StatusTarget {
1553 &self.targets[category_position(category)]
1554 }
1555
1556 /// The category a board option name reports, or `None` when nothing maps to it.
1557 fn category_of(&self, option: &str) -> Option<StatusCategory> {
1558 CATEGORIES.into_iter().find(|category| {
1559 matches!(self.target(*category), StatusTarget::Column(name)
1560 if name.as_str().eq_ignore_ascii_case(option))
1561 })
1562 }
1563
1564 /// The status an item reports, from the three things a read of it says: its board
1565 /// `Status` option, whether its issue is closed, and the reason it was closed with.
1566 ///
1567 /// The closed state decides the category and the `Status` option decides the name, so
1568 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
1569 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
1570 /// duplicate is not finished work, and calling it done is a lie the next copy would
1571 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
1572 /// it is read permissively rather than refused — reads are faithful, and refusals
1573 /// belong on writes.
1574 ///
1575 /// One function of those three rather than of a response, so a narrow status write can
1576 /// answer what a re-read would report by applying it to the state it has just written.
1577 fn status(&self, option: Option<&str>, closed: bool, reason: Option<&str>) -> Status {
1578 if closed {
1579 let category = match reason {
1580 None | Some("COMPLETED") => StatusCategory::Done,
1581 Some("NOT_PLANNED") => StatusCategory::Cancelled,
1582 Some(_) => StatusCategory::Unknown,
1583 };
1584 let fallback = match category {
1585 StatusCategory::Done => "Done",
1586 StatusCategory::Cancelled => "Cancelled",
1587 _ => "Closed",
1588 };
1589 return Status {
1590 category,
1591 name: option.unwrap_or(fallback).to_owned(),
1592 };
1593 }
1594 let name = option.unwrap_or("Open").to_owned();
1595 Status {
1596 category: self.category_of(&name).unwrap_or(StatusCategory::Unknown),
1597 name,
1598 }
1599 }
1600}
1601
1602// llmlint: ignore-block[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate] Every `createIssue` names one of these, and which one is the rule — a reader who reaches the type from `create_and_file_issue` gets the rule in one sentence here without the method's refusals, which stay on `creation_target`, the rule's one executable source; `tests/plugin.rs` drives every arm of it against the loopback board.
1603/// One repository this source can create an issue in, as `owner/name`.
1604///
1605/// Every `createIssue` this source sends names one of these: the item's own single
1606/// `repositories` entry, else its parent project issue's repository, else the configured
1607/// [`GitHubProjectsConfig::repository`]. [`GitHubProjectsSource::creation_target`] makes
1608/// that choice and says what it refuses before `createIssue`.
1609// llmlint: ignore-end[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate]
1610#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
1611struct RepositoryTarget {
1612 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1613 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1614}
1615
1616impl RepositoryTarget {
1617 fn parse(value: &str) -> Result<Self, SourceError> {
1618 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
1619 message: format!(
1620 "repository must be spelled owner/name; {value:?} names no repository"
1621 ),
1622 })?;
1623 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
1624 return Err(SourceError::Config {
1625 message: format!(
1626 "repository must be spelled owner/name with a GitHub login and one \
1627 repository name; {value:?} is not"
1628 ),
1629 });
1630 }
1631 Ok(Self {
1632 owner: owner.to_owned(),
1633 name: name.to_owned(),
1634 })
1635 }
1636
1637 /// The one host whose repositories this source creates issues in, spelled once: it is
1638 /// what [`Self::origin`] renders and what [`Self::from_origin`] accepts.
1639 const HOST: &str = "github.com";
1640
1641 fn origin(&self) -> String {
1642 format!("{}/{}/{}", Self::HOST, self.owner, self.name)
1643 }
1644
1645 /// The repository a normalized origin names, or why it is none this source can create
1646 /// an issue in: another host, or more or fewer than `owner/name` under this one.
1647 fn from_origin(origin: &Repository) -> Result<Self, String> {
1648 let not_here = || {
1649 format!(
1650 "{} is not a {}/owner/name repository",
1651 origin.as_str(),
1652 Self::HOST
1653 )
1654 };
1655 let (host, rest) = origin.as_str().split_once('/').ok_or_else(not_here)?;
1656 if host != Self::HOST {
1657 return Err(not_here());
1658 }
1659 Self::parse(rest).map_err(|_| not_here())
1660 }
1661
1662 fn slug(&self) -> String {
1663 format!("{}/{}", self.owner, self.name)
1664 }
1665}
1666
1667/// A source which reads GitHub afresh for every operation.
1668pub struct GitHubProjectsSource {
1669 /// This source's configured name, used both to tell a far end naming this source
1670 /// from one naming a system it knows nothing about, and to name the instance a
1671 /// status refusal is about.
1672 name: SourceName,
1673 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
1674 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
1675 repository: Option<RepositoryTarget>,
1676 endpoint: Url,
1677 token: SecretString,
1678 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1679 statuses: StatusMapping,
1680 client: Client,
1681 /// Every item this source has created since it was built, in the order it created
1682 /// them.
1683 ///
1684 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1685 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1686 /// a copy resolving a dependency on an item it had just created refused it as not
1687 /// found. A board read is completed from this — an item remembered here and absent from
1688 /// the read is added back, because the board really does hold it and only the read is
1689 /// behind.
1690 ///
1691 /// It is not a cache of a user's work: nothing is remembered that this process did not
1692 /// itself just write, it lives and dies with the process, and it is never consulted for
1693 /// an item this source did not create.
1694 created: Mutex<Vec<Resolved>>,
1695 /// How fast this source writes, and how long it waits out a refusal.
1696 pacing: Pacing,
1697 /// When the last content-creating mutation finished, or the moment the furthest-out
1698 /// reserved slot releases the next one, whichever is later — so the one after it can be
1699 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1700 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1701 /// what it is measured from.
1702 last_mutation: Mutex<Option<Instant>>,
1703 /// The board as this process last read it, for the length of one command.
1704 ///
1705 /// A copy of a project used to re-read the whole board, paged, before writing each of
1706 /// its items, which is by far the largest part of a copy's request count and none of
1707 /// its work. Nothing else changes this board while a command runs — this source's own
1708 /// writes are the only writer — so one read answers them all.
1709 ///
1710 /// It is not a store of a user's work and it is not the cache the no-persistence
1711 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1712 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1713 /// an item this command created and then depends on resolves whether or not GitHub's
1714 /// own eventually-consistent read has caught up. A write to an item already on the
1715 /// board updates the entry here too, so what this holds is the last read plus this
1716 /// process's own writes rather than a snapshot taken before them.
1717 board_cache: Mutex<Option<Board>>,
1718 /// Each destination repository's node id, resolved once per repository
1719 /// rather than per issue created.
1720 ///
1721 /// A repository's node id does not change, and re-reading it for every issue of a copy
1722 /// spent one request per item on an answer this source already had. It is a map rather
1723 /// than one entry because a copy files each item in the repository its own
1724 /// `repositories` field names, so a plan across five repositories asks GitHub five
1725 /// times and not once per item.
1726 repository_cache: Mutex<BTreeMap<RepositoryTarget, String>>,
1727 /// What every request this source sends is recorded into.
1728 ///
1729 /// Ordinary code path, not a mode: [`Self::send_once`] records into it at the one place
1730 /// a request leaves this crate, so nothing has to be switched on for a session to be
1731 /// counted. It is shared rather than owned so a caller accounting for a whole session —
1732 /// its own schema verification, board lookups, residue sweep and cleanup beside this
1733 /// source's reads and writes — adds up one accounting instead of two. See
1734 /// [`accounting`] for what a record carries and what a session's spend is and is not.
1735 ledger: Arc<Accounting>,
1736}
1737
1738impl GitHubProjectsSource {
1739 /// Validate configuration and capture the named credential without exposing it.
1740 ///
1741 /// # Errors
1742 ///
1743 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1744 /// [`SourceError::Auth`] when the named credential is missing or empty.
1745 pub fn new(
1746 name: &SourceName,
1747 config: GitHubProjectsConfig,
1748 secrets: &dyn SecretResolver,
1749 ) -> Result<Self, SourceError> {
1750 Self::recording_into(name, config, secrets, Arc::new(Accounting::new()))
1751 }
1752
1753 /// The same, recording every request it sends into an accounting the caller holds too.
1754 ///
1755 /// [`Self::new`] is this with an accounting of its own. A caller that is also making
1756 /// its own calls to GitHub — a lane verifying a schema, sweeping residue or cleaning
1757 /// up — passes the one it records those into, so the session total accounts for the
1758 /// whole session rather than for this source's share of it.
1759 ///
1760 /// # Errors
1761 ///
1762 /// Exactly [`Self::new`]'s: [`SourceError::Config`] for a configuration this instance
1763 /// cannot use and [`SourceError::Auth`] when the named credential is missing or empty.
1764 pub fn recording_into(
1765 name: &SourceName,
1766 config: GitHubProjectsConfig,
1767 secrets: &dyn SecretResolver,
1768 ledger: Arc<Accounting>,
1769 ) -> Result<Self, SourceError> {
1770 if !valid_github_owner(&config.owner) {
1771 return Err(SourceError::Config {
1772 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1773 });
1774 }
1775 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1776 return Err(SourceError::Config {
1777 message: format!("project_number must be between 1 and {}", i32::MAX),
1778 });
1779 }
1780 if !valid_environment_name(&config.token_env) {
1781 return Err(SourceError::Config {
1782 message: "token_env must be a valid environment-variable name".into(),
1783 });
1784 }
1785 let repository = config
1786 .repository
1787 .as_deref()
1788 .map(RepositoryTarget::parse)
1789 .transpose()?;
1790 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1791 message: format!("endpoint is not a valid URL: {e}"),
1792 })?;
1793 if endpoint.scheme() != "https"
1794 && !(endpoint.scheme() == "http"
1795 && endpoint
1796 .host_str()
1797 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1798 {
1799 return Err(SourceError::Config {
1800 message:
1801 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1802 .into(),
1803 });
1804 }
1805 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1806 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1807 })?;
1808 Ok(Self {
1809 name: name.clone(),
1810 owner: config.owner,
1811 project_number: config.project_number,
1812 repository,
1813 endpoint,
1814 token,
1815 credential_name: config.token_env,
1816 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1817 client: Client::builder()
1818 .user_agent("onetaskgraph")
1819 .build()
1820 .map_err(|e| SourceError::Config {
1821 message: format!("cannot build HTTP client: {e}"),
1822 })?,
1823 created: Mutex::new(Vec::new()),
1824 pacing: Pacing::resolve(config.pacing, name)?,
1825 last_mutation: Mutex::new(None),
1826 board_cache: Mutex::new(None),
1827 repository_cache: Mutex::new(BTreeMap::new()),
1828 ledger,
1829 })
1830 }
1831
1832 /// A snapshot of every request this source has sent, and what each cost.
1833 ///
1834 /// A value to hold and compare rather than a borrow of the accounting itself, so two
1835 /// of them can sit side by side. When this source was built with
1836 /// [`Self::recording_into`] the snapshot is the whole shared session, which is the
1837 /// point of building it that way.
1838 #[must_use]
1839 pub fn accounting(&self) -> accounting::Session {
1840 self.ledger.snapshot()
1841 }
1842
1843 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1844 /// rate limit rather than handing it straight back as an error.
1845 ///
1846 /// Retrying is safe for every document here, including the mutations, and the reason
1847 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1848 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1849 /// this replays has already taken effect. An outcome this source cannot know — the
1850 /// send failed, or the body could not be read, so the mutation may well have landed —
1851 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1852 /// attempt. A duplicate write would come from replaying one of those, and none is
1853 /// replayed.
1854 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1855 let doing = operation_description(query);
1856 let mut waited = Duration::ZERO;
1857 let mut waits = 0_u32;
1858 let mut backoff = self.pacing.retry_backoff;
1859 loop {
1860 if is_mutation(query) {
1861 let spacing = self.reserve_mutation_slot();
1862 if !spacing.is_zero() {
1863 tokio::time::sleep(spacing).await;
1864 }
1865 }
1866 let attempt = self.send_once(query, &variables).await;
1867 if is_mutation(query) {
1868 self.finish_mutation();
1869 }
1870 let limited = match attempt {
1871 Ok(data) => return Ok(data),
1872 Err(Attempt::Failed(error)) => return Err(error),
1873 Err(Attempt::Limited(limited)) => limited,
1874 };
1875 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1876 // move that extends a secondary limit, so a hint below the schedule's own next
1877 // wait is raised to it.
1878 let wait = match limited.hint {
1879 Some(hint) => Duration::from_secs(hint).max(backoff),
1880 None => backoff,
1881 };
1882 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1883 // A wait of nothing spends none of the budget, so it is exhaustion rather
1884 // than a retry. `Pacing::resolve` rules out every way of configuring one
1885 // except a budget of zero, where reporting the first refusal is the ask.
1886 if wait.is_zero() || wait > remaining {
1887 return Err(limited.exhausted(
1888 doing,
1889 waits,
1890 waited,
1891 wait,
1892 self.pacing.retry_budget,
1893 ));
1894 }
1895 tokio::time::sleep(wait).await;
1896 waited += wait;
1897 waits += 1;
1898 backoff = backoff.saturating_mul(2);
1899 }
1900 }
1901
1902 /// The next moment a content-creating mutation may leave this source, as a wait from
1903 /// now.
1904 ///
1905 /// The slot is reserved under the lock and the waiting happens outside it, so two
1906 /// callers take two slots rather than the same one — and no lock is held across an
1907 /// await.
1908 ///
1909 /// The moment it is spaced from is the previous mutation's *completion*, which
1910 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1911 /// own is the wrong thing to measure from.
1912 fn reserve_mutation_slot(&self) -> Duration {
1913 if self.pacing.min_mutation_interval.is_zero() {
1914 return Duration::ZERO;
1915 }
1916 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1917 // it would turn an earlier failure into a second one for no gain.
1918 let mut last = self
1919 .last_mutation
1920 .lock()
1921 .unwrap_or_else(std::sync::PoisonError::into_inner);
1922 let now = Instant::now();
1923 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1924 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1925 let at = last.map_or(now, |previous| {
1926 previous
1927 .checked_add(self.pacing.min_mutation_interval)
1928 .map_or(now, |earliest| earliest.max(now))
1929 });
1930 *last = Some(at);
1931 at.saturating_duration_since(now)
1932 }
1933
1934 /// Record that a content-creating mutation has finished, so the next one is spaced
1935 /// from here rather than from the moment this one was released.
1936 ///
1937 /// This source can only choose when a request *departs*; the limiter counts when it
1938 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1939 /// departure from the last therefore hands the limiter a gap of the interval less that
1940 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1941 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1942 /// slower machine while passing on a quick one.
1943 ///
1944 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1945 /// previous request had already arrived before its response came back, so its arrival
1946 /// is no later than this moment, and the next mutation is released at least the
1947 /// interval after this moment and arrives no earlier than it is released: the gap the
1948 /// limiter measures is therefore at least the interval, whatever transit costs and on
1949 /// whatever platform. The price is that a mutation's own round trip no longer counts
1950 /// towards its spacing, which makes this source slightly slower than the configured
1951 /// rate rather than slightly faster — the safe side of a limit that punishes being
1952 /// wrong by refusing reads for the next fifty minutes.
1953 ///
1954 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1955 /// and one that never left costs only a wait nobody needed.
1956 fn finish_mutation(&self) {
1957 if self.pacing.min_mutation_interval.is_zero() {
1958 return;
1959 }
1960 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1961 let mut last = self
1962 .last_mutation
1963 .lock()
1964 .unwrap_or_else(std::sync::PoisonError::into_inner);
1965 let now = Instant::now();
1966 // `max` rather than an assignment: a concurrent caller may already have reserved a
1967 // slot further out, and completing this request must never pull that slot back in.
1968 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1969 }
1970
1971 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1972 /// failure that waiting cannot help — and recorded, whichever of the three it was.
1973 ///
1974 /// This is the one place a request leaves this crate, which is why the accounting is
1975 /// here rather than at each of the callers: a read path added later is counted without
1976 /// anybody remembering to count it, and
1977 /// `the_session_report_counts_every_request_the_board_served_and_what_each_cost` fails
1978 /// when one is not.
1979 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1980 let Attempted {
1981 result,
1982 limits,
1983 reported_cost,
1984 } = self.attempt(query, variables).await;
1985 // No `otherwise` name: every document this source sends is one of its own, and the
1986 // inventory gate on `graphql::DOCUMENTS` is what keeps that true.
1987 let sending = accounting::Request::graphql(query, variables, None, reported_cost);
1988 let outcome = match &result {
1989 Ok(_) => accounting::Outcome::Answered,
1990 Err(Attempt::Limited(_)) => accounting::Outcome::RateLimited,
1991 Err(Attempt::Failed(_)) => accounting::Outcome::Refused,
1992 };
1993 self.ledger.record(sending.finished(outcome, limits));
1994 result
1995 }
1996
1997 /// The attempt itself, with what its response said about the rate limit alongside.
1998 ///
1999 /// The two are returned together rather than recorded here because every one of the
2000 /// early exits below is a different outcome, and a record written at each of them is a
2001 /// record one of them can be added without.
2002 async fn attempt(&self, query: &str, variables: &Value) -> Attempted {
2003 let mut limits = accounting::RateLimit::default();
2004 let mut reported_cost = None;
2005 let result = self
2006 .attempted(query, variables, &mut limits, &mut reported_cost)
2007 .await;
2008 Attempted {
2009 result,
2010 limits,
2011 reported_cost,
2012 }
2013 }
2014
2015 /// One HTTP attempt, filling in what its response said about the rate limit as it goes.
2016 async fn attempted(
2017 &self,
2018 query: &str,
2019 variables: &Value,
2020 limits: &mut accounting::RateLimit,
2021 reported_cost: &mut Option<u64>,
2022 ) -> Result<Value, Attempt> {
2023 let response = self
2024 .client
2025 .post(self.endpoint.clone())
2026 .bearer_auth(self.token.expose_secret())
2027 .json(&json!({"query": query, "variables": variables}))
2028 .send()
2029 .await
2030 .map_err(|e| {
2031 Attempt::Failed(SourceError::Unavailable {
2032 message: format!("GitHub GraphQL request failed: {e}"),
2033 })
2034 })?;
2035 let status = response.status();
2036 let header = |name: &str| whole_seconds(response.headers().get(name));
2037 *limits = accounting::RateLimit::read(|name| {
2038 response
2039 .headers()
2040 .get(name)
2041 .and_then(|value| value.to_str().ok())
2042 .map(str::to_owned)
2043 });
2044 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
2045 // that are not text at all — is "not known to be exhausted". This never makes a
2046 // response a refusal on its own: it says which limiter a refusal is attributed to
2047 // and where its hint comes from, so a value this cannot read costs a hint rather
2048 // than an answer.
2049 let exhausted = response
2050 .headers()
2051 .get("x-ratelimit-remaining")
2052 .and_then(|value| value.to_str().ok())
2053 == Some("0");
2054 // `retry-after` is what GitHub asks for when it asks; when it does not and the
2055 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
2056 // which is the same question answered as an absolute time. Nothing else here is a
2057 // hint, and a schedule is what answers a refusal that carries none.
2058 let hint = header("retry-after").or_else(|| {
2059 exhausted
2060 .then(|| header("x-ratelimit-reset"))
2061 .flatten()
2062 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
2063 });
2064 // Read before it is parsed, because the evidence which tells a secondary rate
2065 // limit from a rejected credential is in the body of a response whose status says
2066 // only "forbidden" — and a non-success response was never parsed at all.
2067 let body = response.text().await.map_err(|e| {
2068 Attempt::Failed(SourceError::Unavailable {
2069 message: format!("GitHub GraphQL response could not be read: {e}"),
2070 })
2071 })?;
2072 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
2073 return Err(Attempt::Limited(Limited { limiter, hint }));
2074 }
2075 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
2076 return Err(Attempt::Failed(SourceError::Auth {
2077 message: format!(
2078 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
2079 ),
2080 }));
2081 }
2082 if !status.is_success() {
2083 return Err(Attempt::Failed(SourceError::Unavailable {
2084 message: format!("GitHub GraphQL returned HTTP {status}"),
2085 }));
2086 }
2087 // GitHub reports what a call cost only when the document asked it to, and no
2088 // document this source sends does — so this is `None` here and carries the figure
2089 // for a caller whose own document selects `rateLimit { cost }`. What it must never
2090 // pick up is a `dryRun` probe's cost, which is some other document's.
2091 *reported_cost = serde_json::from_str::<Value>(&body)
2092 .ok()
2093 .as_ref()
2094 .and_then(|body| body.pointer("/data/rateLimit/cost"))
2095 .and_then(Value::as_u64);
2096 self.answer(&body).map_err(Attempt::Failed)
2097 }
2098
2099 /// What one successful HTTP response says, once its GraphQL errors are read.
2100 fn answer(&self, body: &str) -> Result<Value, SourceError> {
2101 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
2102 message: format!("GitHub returned invalid JSON: {e}"),
2103 })?;
2104 let errors = body
2105 .get("errors")
2106 .map(|value| {
2107 value.as_array().ok_or_else(|| SourceError::Malformed {
2108 message: "GitHub response errors is not an array".into(),
2109 })
2110 })
2111 .transpose()?;
2112 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
2113 let messages = errors
2114 .iter()
2115 .filter_map(|e| e.get("message").and_then(Value::as_str))
2116 .collect::<Vec<_>>()
2117 .join("; ");
2118 let message = if messages.is_empty() {
2119 "GitHub returned GraphQL errors".into()
2120 } else {
2121 messages
2122 };
2123 let normalized = message.to_ascii_lowercase();
2124 if normalized.contains("resource not accessible") || normalized.contains("scope") {
2125 return Err(SourceError::Auth {
2126 message: format!(
2127 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
2128 self.credential_name
2129 ),
2130 });
2131 }
2132 return Err(SourceError::Refused { message });
2133 }
2134 body.get("data")
2135 .filter(|data| data.is_object())
2136 .cloned()
2137 .ok_or_else(|| SourceError::Malformed {
2138 message: "GitHub response has no data object".into(),
2139 })
2140 }
2141
2142 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
2143 // GraphQL cannot independently page them inside the outer item page. This source page is
2144 // deliberately bounded at that published maximum; the live drift journey exercises it.
2145 async fn board_page(
2146 &self,
2147 items_after: Option<&str>,
2148 items_first: u32,
2149 ) -> Result<Value, SourceError> {
2150 let data = self
2151 .graphql(
2152 graphql::BOARD,
2153 json!({"owner":self.owner,"number":self.project_number,
2154 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
2155 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
2156 )
2157 .await?;
2158 data.pointer("/owner/projectV2")
2159 .filter(|v| !v.is_null())
2160 .cloned()
2161 .ok_or_else(|| SourceError::Refused {
2162 message: format!(
2163 "GitHub project {}/{} was not found or is not visible to the token",
2164 self.owner, self.project_number
2165 ),
2166 })
2167 }
2168
2169 /// The search that finds the issues of this board, narrowed by `also` when it is
2170 /// given.
2171 ///
2172 /// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
2173 /// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
2174 /// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
2175 /// here on purpose — GitHub accepts it and silently ignores it, so a project is told
2176 /// from a task by the `parent` field each issue carries rather than by the search.
2177 fn board_search(&self, also: Option<&str>) -> String {
2178 let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
2179 match also {
2180 Some(also) => format!("{scope} {also}"),
2181 None => scope,
2182 }
2183 }
2184
2185 /// One issue this source reached directly, as the board item a read of the board would
2186 /// have produced — or `None` when this board does not hold it.
2187 ///
2188 /// The board half of an issue rides along on `Issue.projectItems`, so the value handed
2189 /// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
2190 /// item's own id, that item's field values, and the issue as its content. One resolver
2191 /// for both routes is what makes an issue read through a search, through its own node
2192 /// id, or through its project's sub-issues report the same title, the same status, the
2193 /// same labels and the same qualified id.
2194 ///
2195 /// An issue with no entry for *this* board is not this source's to report, which is
2196 /// what keeps an id naming some other repository's issue from being answered as an item
2197 /// of this board. That answer is given about an **exhausted** connection and never
2198 /// about an unread page: the entry is looked for on the page in hand, and only if that
2199 /// page reports more of the connection, in [`Self::board_membership`]'s walk of the
2200 /// rest of it.
2201 async fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
2202 if optional_str(issue, "__typename")? != Some("Issue") {
2203 return Ok(None);
2204 }
2205 let memberships = issue
2206 .get("projectItems")
2207 .ok_or_else(|| SourceError::Malformed {
2208 message: "GitHub issue is missing projectItems".into(),
2209 })?;
2210 let nodes = memberships
2211 .get("nodes")
2212 .and_then(Value::as_array)
2213 .ok_or_else(|| SourceError::Malformed {
2214 message: "GitHub issue projectItems.nodes is not an array".into(),
2215 })?;
2216 let held = match self.board_entry(nodes) {
2217 Some(held) => held.clone(),
2218 None => {
2219 let info = memberships
2220 .get("pageInfo")
2221 .ok_or_else(|| SourceError::Malformed {
2222 message: "GitHub issue projectItems has no pageInfo".into(),
2223 })?;
2224 // The page held no entry for this board. Whether that means the issue is
2225 // not on it is a question about the rest of the connection, and only a
2226 // connection with no rest answers it here.
2227 if !required_bool(info, "hasNextPage")? {
2228 return Ok(None);
2229 }
2230 let cursor = required_str(info, "endCursor")?;
2231 validate_cursor_progress(None, cursor)?;
2232 let issue_id = required_str(issue, "id")?;
2233 match self.board_membership(issue_id, cursor).await? {
2234 Some(held) => held,
2235 None => return Ok(None),
2236 }
2237 }
2238 };
2239 let item = json!({
2240 "id": required_str(&held, "id")?,
2241 "project": held.get("project"),
2242 "fieldValues": held.get("fieldValues"),
2243 "content": issue,
2244 });
2245 self.resolve(&item)
2246 }
2247
2248 /// This board's own entry among one page of an issue's `Issue.projectItems`.
2249 ///
2250 /// One spelling of *which membership is this board's*, so the page a read carries and
2251 /// the pages [`Self::board_membership`] walks are searched by the same rule.
2252 fn board_entry<'a>(&self, nodes: &'a [Value]) -> Option<&'a Value> {
2253 nodes.iter().find(|node| {
2254 node.pointer("/project/number").and_then(Value::as_u64)
2255 == Some(u64::from(self.project_number))
2256 })
2257 }
2258
2259 /// The rest of one issue's board memberships, from `after`, for this board's entry.
2260 ///
2261 /// The recovery read: a page of memberships that holds no entry for this board says
2262 /// nothing about the memberships past it, so the connection is walked to exhaustion
2263 /// before an issue is reported as one this board does not hold. `Ok(None)` is that
2264 /// positive answer — the whole connection was read and no entry named this board —
2265 /// rather than a failure, and the walk is held to
2266 /// [`validate_cursor_progress`] like every other page walk here, so a source answering
2267 /// with a cursor that does not advance is refused instead of spun on.
2268 async fn board_membership(
2269 &self,
2270 issue: &str,
2271 after: &str,
2272 ) -> Result<Option<Value>, SourceError> {
2273 let mut after = after.to_owned();
2274 loop {
2275 let data = self
2276 .graphql(
2277 graphql::ISSUE_BOARD_ITEMS,
2278 json!({"id":issue,"first":MAX_PAGE_SIZE,"after":after,
2279 "nestedFirst":NESTED_PAGE_SIZE}),
2280 )
2281 .await?;
2282 let Some(connection) = data
2283 .pointer("/node/projectItems")
2284 .filter(|value| !value.is_null())
2285 else {
2286 // The id resolved to nothing, or to something with no memberships to walk —
2287 // which is the same answer as a connection holding no entry for this board.
2288 return Ok(None);
2289 };
2290 let nodes = connection
2291 .get("nodes")
2292 .and_then(Value::as_array)
2293 .ok_or_else(|| SourceError::Malformed {
2294 message: "GitHub issue projectItems.nodes is not an array".into(),
2295 })?;
2296 if let Some(held) = self.board_entry(nodes) {
2297 return Ok(Some(held.clone()));
2298 }
2299 let info = connection
2300 .get("pageInfo")
2301 .ok_or_else(|| SourceError::Malformed {
2302 message: "GitHub issue projectItems has no pageInfo".into(),
2303 })?;
2304 let next = required_bool(info, "hasNextPage")?
2305 .then(|| required_str(info, "endCursor"))
2306 .transpose()?;
2307 match next {
2308 Some(next) => {
2309 validate_cursor_progress(Some(&after), next)?;
2310 after = next.to_owned();
2311 }
2312 None => return Ok(None),
2313 }
2314 }
2315 }
2316
2317 /// One page of a board-scoped issue search, and where the next page resumes.
2318 async fn search_page(
2319 &self,
2320 search: &str,
2321 first: u32,
2322 after: Option<&str>,
2323 ) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
2324 let data = self
2325 .graphql(
2326 graphql::SEARCH_ISSUES,
2327 json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
2328 "after":after,"nestedFirst":NESTED_PAGE_SIZE,
2329 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2330 )
2331 .await?;
2332 let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
2333 message: "GitHub search response has no search connection".into(),
2334 })?;
2335 let mut found = Vec::new();
2336 for node in connection
2337 .get("nodes")
2338 .and_then(Value::as_array)
2339 .ok_or_else(|| SourceError::Malformed {
2340 message: "GitHub search nodes is not an array".into(),
2341 })?
2342 {
2343 if let Some(resolved) = self.resolve_issue(node).await? {
2344 found.push(resolved);
2345 }
2346 }
2347 let info = connection
2348 .get("pageInfo")
2349 .ok_or_else(|| SourceError::Malformed {
2350 message: "GitHub search connection has no pageInfo".into(),
2351 })?;
2352 let next = required_bool(info, "hasNextPage")?
2353 .then(|| required_str(info, "endCursor"))
2354 .transpose()?
2355 .map(str::to_owned);
2356 if let Some(next) = &next {
2357 validate_cursor_progress(after, next)?;
2358 }
2359 Ok((found, next))
2360 }
2361
2362 /// Every issue this board holds, walked to exhaustion, completed with what this run
2363 /// wrote.
2364 ///
2365 /// The completion is not an optimisation and it is not a cache: GitHub's issue search
2366 /// is an index and is eventually consistent, so an issue this run created seconds ago
2367 /// is routinely absent from it, and a project listed straight after being written would
2368 /// otherwise be missing from its own board. What is added back is only what this
2369 /// process itself wrote, out of [`Self::created`], which lives and dies with the
2370 /// process.
2371 async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
2372 let mut after: Option<String> = None;
2373 let mut found = Vec::new();
2374 let search = self.board_search(None);
2375 loop {
2376 let (page, next) = self
2377 .search_page(&search, MAX_PAGE_SIZE, after.as_deref())
2378 .await?;
2379 found.extend(page);
2380 match next {
2381 Some(next) => after = Some(next),
2382 None => break,
2383 }
2384 }
2385 self.completed_with_written(found, |_| true)
2386 }
2387
2388 /// `found`, with everything this run wrote that `keep` accepts and the read did not
2389 /// report.
2390 ///
2391 /// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
2392 /// at all: the search index is behind, and a node read of an item filed moments ago can
2393 /// be too.
2394 fn completed_with_written(
2395 &self,
2396 mut found: Vec<Resolved>,
2397 keep: impl Fn(&Resolved) -> bool,
2398 ) -> Result<Vec<Resolved>, SourceError> {
2399 for own in self.created()?.iter().filter(|own| keep(own)) {
2400 if !found.iter().any(|item| item.id == own.id) {
2401 found.push(own.clone());
2402 }
2403 }
2404 Ok(found)
2405 }
2406
2407 /// What resolving one node id reached.
2408 ///
2409 /// Three answers rather than an `Option`, because a board *draft* is none of the other
2410 /// two: it is not an issue, it has no node of its own this source can read the board
2411 /// half off, and its only home is the board's own item connection — so a read of one
2412 /// is completed from there rather than reported as nothing.
2413 async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
2414 let asked = self
2415 .graphql(
2416 graphql::ISSUE,
2417 json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
2418 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2419 )
2420 .await;
2421 let data = match asked {
2422 Ok(data) => data,
2423 // A string that is not a node id at all is not a failure to report: it is an id
2424 // this board does not hold, which is what every read of one already answers.
2425 Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
2426 Err(error) => return Err(error),
2427 };
2428 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
2429 return Ok(Reached::Nothing);
2430 };
2431 if optional_str(node, "__typename")? == Some("DraftIssue") {
2432 return Ok(Reached::Draft);
2433 }
2434 Ok(match self.resolve_issue(node).await? {
2435 Some(item) => Reached::Held(Box::new(item)),
2436 None => Reached::Nothing,
2437 })
2438 }
2439
2440 /// One item of this board by its own id, whatever kind it is.
2441 ///
2442 /// Resolved from the identifier alone: no search, board-wide or otherwise. What this
2443 /// run wrote is read first, because a node read of an item created moments ago can
2444 /// still be behind the board field values written onto it — see [`Self::created`].
2445 async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
2446 if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
2447 return Ok(Some(own.clone()));
2448 }
2449 match self.reach(id).await? {
2450 Reached::Held(item) => Ok(Some(*item)),
2451 Reached::Nothing => Ok(None),
2452 // The one read that still costs the board: a draft lives nowhere else.
2453 Reached::Draft => Ok(self
2454 .board()
2455 .await?
2456 .items
2457 .into_iter()
2458 .find(|item| item.id == *id)),
2459 }
2460 }
2461
2462 /// Everything filed under one issue of this board, walked to exhaustion — or `None`
2463 /// when that id names nothing here with a sub-issue relationship to walk.
2464 ///
2465 /// `None` and an empty answer are different: `None` is *this is not an issue of this
2466 /// GitHub*, which is what sends a project selector on to be read as a name, and an
2467 /// empty vector is a project that holds nothing.
2468 async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
2469 let mut after: Option<String> = None;
2470 let mut children = Vec::new();
2471 loop {
2472 let asked = self
2473 .graphql(
2474 graphql::SUB_ISSUES,
2475 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
2476 "nestedFirst":NESTED_PAGE_SIZE,
2477 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2478 )
2479 .await;
2480 let data = match asked {
2481 Ok(data) => data,
2482 // A string that is not a node id at all is not a failure to report: it is
2483 // the ordinary answer to a selector naming a project by its name.
2484 Err(error) if unresolvable_node(&error) => return Ok(None),
2485 Err(error) => return Err(error),
2486 };
2487 let Some(connection) = data
2488 .pointer("/node/subIssues")
2489 .filter(|value| !value.is_null())
2490 else {
2491 // No such node, or one with no sub-issue relationship — a board draft is
2492 // the one this board can really hold.
2493 return Ok(None);
2494 };
2495 for node in connection
2496 .get("nodes")
2497 .and_then(Value::as_array)
2498 .ok_or_else(|| SourceError::Malformed {
2499 message: "GitHub subIssues.nodes is not an array".into(),
2500 })?
2501 {
2502 if let Some(resolved) = self.resolve_issue(node).await? {
2503 children.push(resolved);
2504 }
2505 }
2506 let info = connection
2507 .get("pageInfo")
2508 .ok_or_else(|| SourceError::Malformed {
2509 message: "GitHub subIssues connection has no pageInfo".into(),
2510 })?;
2511 let next = required_bool(info, "hasNextPage")?
2512 .then(|| required_str(info, "endCursor"))
2513 .transpose()?;
2514 match next {
2515 Some(next) => {
2516 validate_cursor_progress(after.as_deref(), next)?;
2517 after = Some(next.to_owned());
2518 }
2519 None => return Ok(Some(children)),
2520 }
2521 }
2522 }
2523
2524 /// Which issue of this board a project *name* is, or `None` when none is.
2525 ///
2526 /// One bounded query which filters on that name at the server, rather than a walk of
2527 /// every issue the board holds. The name is compared again here: the qualifier narrows
2528 /// what GitHub sends, and this source decides what it names.
2529 async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
2530 let search = self.board_search(Some(&title_qualifier(name)));
2531 let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
2532 Ok(candidates
2533 .into_iter()
2534 .find(|item| {
2535 item.kind == BoardKind::Work(ItemKind::Project)
2536 && item.title.eq_ignore_ascii_case(name)
2537 })
2538 .map(|item| item.id))
2539 }
2540
2541 /// Everything filed under one project of this board: the sub-issues of the issue that
2542 /// project is.
2543 ///
2544 /// Tasks *and* documents, because a document filed under a project is a sub-issue of it
2545 /// too — the caller keeps the kind it asked for. Nothing about this grows as the board
2546 /// gains projects, or as another project gains tasks.
2547 ///
2548 /// A qualified id names the issue and is asked for its sub-issues directly: one
2549 /// request, no search of any kind. Only a selector GitHub cannot resolve that way is
2550 /// read as a project *name*, which costs the one bounded search
2551 /// [`Self::project_by_name`] makes.
2552 async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
2553 let (project, children) = match self.sub_issues(selector).await? {
2554 Some(children) => (selector.clone(), children),
2555 None => match self.project_by_name(&selector.0).await? {
2556 Some(project) => {
2557 let children = self.sub_issues(&project).await?.unwrap_or_default();
2558 (project, children)
2559 }
2560 None => return Ok(Vec::new()),
2561 },
2562 };
2563 self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
2564 }
2565
2566 /// Every item on the board, with the one board identity they all share.
2567 ///
2568 /// See [`Self::board_cache`]. The completion from `created` happens on every call
2569 /// rather than once, which is what the cache could otherwise have broken.
2570 async fn board(&self) -> Result<Board, SourceError> {
2571 let cached = self.board_cache()?.clone();
2572 let mut board = match cached {
2573 Some(board) => board,
2574 None => {
2575 let read = self.read_board().await?;
2576 *self.board_cache()? = Some(read.clone());
2577 read
2578 }
2579 };
2580 for own in self.created()?.iter() {
2581 if !board.items.iter().any(|item| item.id == own.id) {
2582 board.items.push(own.clone());
2583 }
2584 }
2585 Ok(board)
2586 }
2587
2588 /// This process's own view of the board, or the refusal a poisoned lock is.
2589 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
2590 self.board_cache
2591 .lock()
2592 .map_err(|_| SourceError::Unavailable {
2593 message: "this source's view of the board was left inconsistent by an earlier \
2594 failure; next: run the command again"
2595 .into(),
2596 })
2597 }
2598
2599 /// Bring this process's own view of the board up to an item it has just written.
2600 ///
2601 /// A created item goes to `created`, which is what completes a board read GitHub's own
2602 /// eventual consistency has left behind. An item that was already there is replaced
2603 /// where it sits, so a second write of it in the same command reads its real parent
2604 /// rather than the one it had before the first write.
2605 ///
2606 /// "Where it sits" is two places, and missing the first leaves a stale record that
2607 /// wins: an item this same run created is held in `created` and not in the cached
2608 /// board, and `board` completes the cached board *from* `created`, so replacing only
2609 /// the cached copy of such an item replaces nothing and the read still reports the
2610 /// title it was created with.
2611 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
2612 if created {
2613 self.created()?.push(item);
2614 return Ok(());
2615 }
2616 {
2617 let mut own = self.created()?;
2618 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
2619 *held = item;
2620 return Ok(());
2621 }
2622 }
2623 if let Some(board) = self.board_cache()?.as_mut()
2624 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
2625 {
2626 *held = item;
2627 }
2628 Ok(())
2629 }
2630
2631 /// Forget one item this process has just deleted, from both halves of its own view.
2632 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
2633 self.created()?.retain(|own| own.id != *id);
2634 if let Some(board) = self.board_cache()?.as_mut() {
2635 board.items.retain(|item| item.id != *id);
2636 }
2637 Ok(())
2638 }
2639
2640 /// Every page of the board, read from GitHub.
2641 async fn read_board(&self) -> Result<Board, SourceError> {
2642 let mut after: Option<String> = None;
2643 let mut items = Vec::new();
2644 let mut board;
2645 loop {
2646 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
2647 for item in page
2648 .pointer("/items/nodes")
2649 .and_then(Value::as_array)
2650 .ok_or_else(|| SourceError::Malformed {
2651 message: "GitHub project items.nodes is not an array".into(),
2652 })?
2653 {
2654 if let Some(resolved) = self.resolve(item)? {
2655 items.push(resolved);
2656 }
2657 }
2658 let info = page
2659 .pointer("/items/pageInfo")
2660 .ok_or_else(|| SourceError::Malformed {
2661 message: "GitHub project items have no pageInfo".into(),
2662 })?;
2663 let has_next = required_bool(info, "hasNextPage")?;
2664 let next = has_next
2665 .then(|| required_str(info, "endCursor"))
2666 .transpose()?;
2667 board = page.clone();
2668 match next {
2669 Some(next) => {
2670 validate_cursor_progress(after.as_deref(), next)?;
2671 after = Some(next.to_owned());
2672 }
2673 None => break,
2674 }
2675 }
2676 Ok(Board {
2677 id: required_str(&board, "id")?.to_owned(),
2678 fields: board.get("fields").cloned().unwrap_or(Value::Null),
2679 items,
2680 })
2681 }
2682
2683 /// The items this source has created, for completing a board read that is behind.
2684 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
2685 self.created.lock().map_err(|_| SourceError::Unavailable {
2686 message: "this source's record of what it created in this run was left \
2687 inconsistent by an earlier failure; next: run the command again"
2688 .into(),
2689 })
2690 }
2691
2692 /// One board item as this source reports it, or `None` for content it ignores.
2693 ///
2694 /// A pull request is neither a project nor a task — it is somebody's change, not a
2695 /// unit of plan — and an item whose content the token cannot see has nothing to
2696 /// report at all.
2697 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
2698 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
2699 message: "GitHub project item is missing content".into(),
2700 })?;
2701 if content.is_null() {
2702 return Ok(None);
2703 }
2704 let content_kind = match required_str(content, "__typename")? {
2705 "Issue" => ContentKind::Issue,
2706 "DraftIssue" => ContentKind::DraftIssue,
2707 _ => return Ok(None),
2708 };
2709 let field_values = item
2710 .get("fieldValues")
2711 .ok_or_else(|| SourceError::Malformed {
2712 message: "GitHub project item is missing fieldValues".into(),
2713 })?;
2714 complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
2715 let nodes = field_values
2716 .get("nodes")
2717 .and_then(Value::as_array)
2718 .ok_or_else(|| SourceError::Malformed {
2719 message: "GitHub project item fieldValues.nodes is not an array".into(),
2720 })?;
2721 if let Some(labels) = content.get("labels") {
2722 complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
2723 }
2724 let raw_body = optional_str(content, "body")?.map(str::to_owned);
2725 let (body, slot) = metadata_body(raw_body.clone())?;
2726 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
2727 .map(|id| NativeId(id.to_owned()));
2728 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
2729 // to read one from; it is a task, and never a project.
2730 let sub_issues = match content_kind {
2731 ContentKind::Issue => sub_issue_total(content)?,
2732 ContentKind::DraftIssue => 0,
2733 };
2734 let content_id = required_str(content, "id")?;
2735 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
2736 message: format!("GitHub issue {content_id}: {message}"),
2737 })?;
2738 let raw_title = required_str(content, "title")?;
2739 // The design prefix is read *first*, before either of the two rules that separate
2740 // a project from a task. A document is not work whatever sub-issues it has and
2741 // whatever marker it carries, and reading the prefix later would make a design
2742 // issue with none of either an empty project.
2743 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
2744 BoardKind::Document
2745 } else if parent.is_some() {
2746 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
2747 // under a project is that project's task even when it has sub-issues of its
2748 // own.
2749 BoardKind::Work(ItemKind::Task)
2750 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
2751 BoardKind::Work(ItemKind::Project)
2752 } else {
2753 BoardKind::Work(ItemKind::Task)
2754 };
2755 // The title a person wrote, which for a document is the one without the prefix —
2756 // the same way `content` above is the body without this source's metadata slot.
2757 let title = match kind {
2758 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
2759 BoardKind::Work(_) => raw_title.to_owned(),
2760 };
2761 let own_repository = content
2762 .pointer("/repository/nameWithOwner")
2763 .and_then(Value::as_str)
2764 .map(|origin| Repository::try_from(format!("{}/{origin}", RepositoryTarget::HOST)))
2765 .transpose()
2766 .map_err(|message| SourceError::Malformed { message })?;
2767 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
2768 Repository::from_metadata(&slot)
2769 .map_err(|message| SourceError::Malformed { message })?
2770 } else {
2771 own_repository.clone().into_iter().collect()
2772 };
2773 let id = NativeId(content_id.to_owned());
2774 // Read only for a task, because only a task has either list: a project or a
2775 // document holding one of these keys holds nothing this source reports, and the
2776 // keys are left out of its caller-visible metadata all the same.
2777 let (delivers, delivered_by) = if kind == BoardKind::Work(ItemKind::Task) {
2778 let listed = |key: &str| {
2779 TaskRef::from_value(key, &id, Some(&self.name), slot.get(key))
2780 .map_err(|message| SourceError::Malformed { message })
2781 };
2782 (
2783 listed(TaskRef::DELIVERS_KEY)?,
2784 listed(TaskRef::DELIVERED_BY_KEY)?,
2785 )
2786 } else {
2787 (Vec::new(), Vec::new())
2788 };
2789 let (option, closed, reason) = Self::status_parts(nodes, content)?;
2790 Ok(Some(Resolved {
2791 item_id: required_str(item, "id")?.to_owned(),
2792 id,
2793 content_kind,
2794 kind,
2795 title,
2796 body: body.filter(|value| !value.is_empty()),
2797 raw_body,
2798 status: self.statuses.status(option, closed, reason),
2799 option: option.map(str::to_owned),
2800 closed,
2801 delivers,
2802 delivered_by,
2803 labels: labels(content)?,
2804 parent,
2805 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
2806 url: optional_str(content, "url")?.map(str::to_owned),
2807 created_at: optional_time(content, "createdAt")?,
2808 updated_at: optional_time(content, "updatedAt")?,
2809 own_repository,
2810 repositories,
2811 slot,
2812 // Present when the item was reached through its own issue, whose board entry
2813 // names the board; a read of the board's own items has the board already. An
2814 // empty id names nothing a field write could address, so it is read as absent and
2815 // the write goes back to reading the board.
2816 board_id: item
2817 .pointer("/project/id")
2818 .and_then(Value::as_str)
2819 .filter(|id| !id.is_empty())
2820 .map(str::to_owned),
2821 fields: field_definitions(nodes),
2822 }))
2823 }
2824
2825 /// What one board item's status is read from: its `Status` option, whether its issue
2826 /// is closed, and the reason it was closed with. [`StatusMapping::status`] turns the
2827 /// three into the status it reports.
2828 fn status_parts<'a>(
2829 field_values: &'a [Value],
2830 content: &'a Value,
2831 ) -> Result<(Option<&'a str>, bool, Option<&'a str>), SourceError> {
2832 let option = field_values
2833 .iter()
2834 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
2835 .map(|value| required_str(value, "name"))
2836 .transpose()?;
2837 let closed = optional_str(content, "state")? == Some("CLOSED");
2838 Ok((option, closed, optional_str(content, "stateReason")?))
2839 }
2840
2841 /// The board Status option this write selects, or the refusal that says why not.
2842 ///
2843 /// For a column target the option is what the status *is*, so a board that has no such
2844 /// option is a refusal naming the status and the instance. For a closed target the
2845 /// issue's own state carries the category, and the option carries only the name a
2846 /// reader reports — so an option spelled the way this status is spelled is selected
2847 /// when the board has one, and nothing is refused when it does not.
2848 ///
2849 /// Answers the field's id, the option's id, and the option's name as the board spells
2850 /// it — which is the name a read of the item reports once it sits there.
2851 fn column_for(
2852 &self,
2853 board: &Board,
2854 status: &Status,
2855 target: &StatusTarget,
2856 ) -> Result<Option<(String, String, String)>, SourceError> {
2857 let (wanted, required) = match target {
2858 StatusTarget::Column(wanted) => (wanted.as_str(), true),
2859 StatusTarget::Closed(_) => (status.name.as_str(), false),
2860 StatusTarget::Disabled => return Ok(None),
2861 };
2862 let missing = |detail: &str| SourceError::Refused {
2863 message: format!(
2864 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
2865 category_name(status.category),
2866 self.name,
2867 category_name(status.category)
2868 ),
2869 };
2870 let Some(field) = Board::field(&board.fields, "Status")? else {
2871 return if required {
2872 Err(missing("this board has no Status field"))
2873 } else {
2874 Ok(None)
2875 };
2876 };
2877 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
2878 return if required {
2879 Err(missing(
2880 "this board's Status field is not a single-select field",
2881 ))
2882 } else {
2883 Ok(None)
2884 };
2885 }
2886 let option = field
2887 .get("options")
2888 .and_then(Value::as_array)
2889 .and_then(|options| {
2890 options.iter().find(|option| {
2891 option
2892 .get("name")
2893 .and_then(Value::as_str)
2894 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
2895 })
2896 });
2897 match option {
2898 None if required => Err(missing("this board does not have it")),
2899 None => Ok(None),
2900 Some(option) => Ok(Some((
2901 required_str(field, "id")?.to_owned(),
2902 required_str(option, "id")?.to_owned(),
2903 required_str(option, "name")?.to_owned(),
2904 ))),
2905 }
2906 }
2907
2908 /// The refusal a status that closes an issue is answered with over a board draft.
2909 fn closes_a_draft(&self, category: StatusCategory) -> SourceError {
2910 SourceError::Refused {
2911 message: format!(
2912 "status {} of source {} closes the item's issue, and GitHub draft items have \
2913 no open or closed state",
2914 category_name(category),
2915 self.name
2916 ),
2917 }
2918 }
2919
2920 /// What a status write to one item needs of the board: the board's id and the
2921 /// definition of its `Status` field, read off the item when the item says both.
2922 ///
2923 /// The same reasoning as [`Self::board_for_update`]: a node read of the item names its
2924 /// board, and its `Status` value carries that field's definition, options and all. An
2925 /// item that does not say — no board id, or no `Status` value to read the field off —
2926 /// sends this back to reading the board, as does a board this command has already read.
2927 async fn status_board(&self, item: &Resolved) -> Result<Board, SourceError> {
2928 let defines_status = item
2929 .fields
2930 .iter()
2931 .any(|field| field.get("name").and_then(Value::as_str) == Some("Status"));
2932 if self.board_cache()?.is_none()
2933 && defines_status
2934 && let Some(board_id) = &item.board_id
2935 {
2936 return Ok(Board {
2937 id: board_id.clone(),
2938 fields: json!({"nodes": item.fields, "pageInfo": {"hasNextPage": false}}),
2939 items: Vec::new(),
2940 });
2941 }
2942 self.board().await
2943 }
2944
2945 /// Set one task's status and nothing else; see [`TaskSource::set_task_status`].
2946 async fn set_status(
2947 &self,
2948 id: &NativeId,
2949 category: StatusCategory,
2950 ) -> Result<Option<Status>, SourceError> {
2951 // Refused before anything is read, in the words a write of the same status is.
2952 let target = self.resolved_target(category)?;
2953 let Some(mut item) = self
2954 .item_by_id(id)
2955 .await?
2956 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
2957 else {
2958 return Ok(None);
2959 };
2960 if let StatusTarget::Closed(reason) = &target {
2961 // The issue's own state carries a closed category, so the board option is left
2962 // exactly where it is and goes on naming the status.
2963 if item.content_kind == ContentKind::DraftIssue {
2964 return Err(self.closes_a_draft(category));
2965 }
2966 self.update_content(
2967 ContentKind::Issue,
2968 &item.id,
2969 json!({"stateInput": state_input(Some(&target))}),
2970 )
2971 .await?;
2972 item.closed = true;
2973 item.status = self
2974 .statuses
2975 .status(item.option.as_deref(), true, Some(reason.reason()));
2976 } else {
2977 let board = self.status_board(&item).await?;
2978 let wanted = Status {
2979 category,
2980 name: category_name(category).to_owned(),
2981 };
2982 let (field, option, name) =
2983 self.column_for(&board, &wanted, &target)?.ok_or_else(|| {
2984 SourceError::Malformed {
2985 message: format!(
2986 "status {} of source {} names no board Status option",
2987 category_name(category),
2988 self.name
2989 ),
2990 }
2991 })?;
2992 // An option is what an open item's status is, so a closed issue is reopened
2993 // first — sitting closed in the column, it would read back as closed. A draft has
2994 // no state to reopen.
2995 if item.content_kind == ContentKind::Issue && item.closed {
2996 self.update_content(
2997 ContentKind::Issue,
2998 &item.id,
2999 json!({"stateInput": state_input(Some(&target))}),
3000 )
3001 .await?;
3002 item.closed = false;
3003 }
3004 self.set_item_field(
3005 &board.id,
3006 &item.item_id,
3007 &field,
3008 json!({"singleSelectOptionId": option}),
3009 )
3010 .await?;
3011 item.status = self.statuses.status(Some(&name), false, None);
3012 item.option = Some(name);
3013 }
3014 let status = item.status.clone();
3015 self.remember_written(item, false)?;
3016 Ok(Some(status))
3017 }
3018
3019 /// Replace one task's `delivered_by` and nothing else; see
3020 /// [`TaskSource::set_delivered_by`].
3021 ///
3022 /// One update of the body, which differs from the body GitHub holds only inside the
3023 /// metadata slot — see [`with_slot`]. A body that would not change is not sent at all.
3024 async fn replace_delivered_by(
3025 &self,
3026 id: &NativeId,
3027 delivered_by: &[TaskRef],
3028 ) -> Result<Option<()>, SourceError> {
3029 let entries = TaskRef::listed(
3030 TaskRef::DELIVERED_BY_KEY,
3031 id,
3032 Some(&self.name),
3033 delivered_by.to_vec(),
3034 )
3035 .map_err(|message| SourceError::Refused { message })?;
3036 let Some(mut item) = self
3037 .item_by_id(id)
3038 .await?
3039 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3040 else {
3041 return Ok(None);
3042 };
3043 let held = item.raw_body.clone().unwrap_or_default();
3044 let mut slot = item.slot.clone();
3045 set_task_list(&mut slot, TaskRef::DELIVERED_BY_KEY, &entries);
3046 let body = with_slot(&held, &slot)?;
3047 if body != held {
3048 self.update_content(item.content_kind, &item.id, json!({"body": body}))
3049 .await?;
3050 }
3051 let (visible, slot) = metadata_body(Some(body.clone()))?;
3052 item.body = visible.filter(|value| !value.is_empty());
3053 item.raw_body = Some(body);
3054 item.slot = slot;
3055 item.delivered_by = entries;
3056 self.remember_written(item, false)?;
3057 Ok(Some(()))
3058 }
3059
3060 /// This instance's target for a category, refusing one it has disabled.
3061 ///
3062 /// Nothing here mutates the board's option set to make room for a status. GitHub
3063 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
3064 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
3065 /// field and every item's status.
3066 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
3067 let target = self.statuses.target(category).clone();
3068 if target != StatusTarget::Disabled {
3069 return Ok(target);
3070 }
3071 Err(SourceError::Refused {
3072 message: if category == StatusCategory::Draft {
3073 format!(
3074 "status draft is disabled for source {}: draft is incompatible with this \
3075 integration because GitHub draft issues cannot have sub-issues, and this \
3076 source stores a project's tasks as its issue's sub-issues",
3077 self.name
3078 )
3079 } else if category == StatusCategory::Unknown {
3080 format!(
3081 "status {} is disabled for source {}; set status_mapping.{} of this source \
3082 to one board Status option name; every word classified unknown is written \
3083 to that one option",
3084 category_name(category),
3085 self.name,
3086 category_name(category)
3087 )
3088 } else {
3089 format!(
3090 "status {} is disabled for source {}; set status_mapping.{} of this source \
3091 to a board Status option name or to a closed state",
3092 category_name(category),
3093 self.name,
3094 category_name(category)
3095 )
3096 },
3097 })
3098 }
3099
3100 async fn set_item_field(
3101 &self,
3102 board_id: &str,
3103 item_id: &str,
3104 field_id: &str,
3105 value: Value,
3106 ) -> Result<(), SourceError> {
3107 let data = self
3108 .graphql(
3109 graphql::UPDATE_FIELD,
3110 json!({"input":{
3111 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
3112 }}),
3113 )
3114 .await?;
3115 let returned = data
3116 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
3117 .ok_or_else(|| SourceError::Malformed {
3118 message: "GitHub field update returned no project item".into(),
3119 })?;
3120 if required_str(returned, "id")? != item_id {
3121 return Err(SourceError::Malformed {
3122 message: "GitHub field update returned the wrong project item".into(),
3123 });
3124 }
3125 Ok(())
3126 }
3127
3128 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
3129 let mut after: Option<String> = None;
3130 let mut ids = Vec::new();
3131 loop {
3132 let data = self
3133 .graphql(
3134 graphql::ISSUE_DEPENDENCIES,
3135 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
3136 )
3137 .await?;
3138 let connection =
3139 data.pointer("/node/blockedBy")
3140 .ok_or_else(|| SourceError::Malformed {
3141 message: "GitHub dependency response has no blockedBy connection".into(),
3142 })?;
3143 ids.extend(
3144 connection
3145 .get("nodes")
3146 .and_then(Value::as_array)
3147 .ok_or_else(|| SourceError::Malformed {
3148 message: "GitHub dependency response nodes is not an array".into(),
3149 })?
3150 .iter()
3151 .map(|value| required_str(value, "id").map(str::to_owned))
3152 .collect::<Result<Vec<_>, _>>()?,
3153 );
3154 let next = next_cursor(connection)?;
3155 if let Some(next) = &next {
3156 validate_cursor_progress(after.as_deref(), &next.0)?;
3157 }
3158 after = next.map(|cursor| cursor.0);
3159 if after.is_none() {
3160 return Ok(ids);
3161 }
3162 }
3163 }
3164
3165 async fn dependencies(
3166 &self,
3167 id: &NativeId,
3168 near_kind: ItemKind,
3169 direction: Direction,
3170 page: &PageRequest,
3171 ) -> Result<Page<DependencyEdge>, SourceError> {
3172 validate_page(page)?;
3173 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
3174 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
3175 let recorded = recorded_offset(cursor, direction)?;
3176 // Asked for even in the recorded phase, whose page reads nothing from the
3177 // connection: `__typename` is what says whether this item has a native
3178 // relationship at all, and that is what decides which far ends the reserved key is
3179 // allowed to hold.
3180 let data = self
3181 .graphql(
3182 graphql::ISSUE_DEPENDENCIES,
3183 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
3184 "after":if recorded.is_some() {None} else {cursor}}),
3185 )
3186 .await?;
3187 let node =
3188 data.get("node")
3189 .filter(|v| !v.is_null())
3190 .ok_or_else(|| SourceError::Refused {
3191 message: format!(
3192 "GitHub item {} was not found or does not support dependencies",
3193 id.0
3194 ),
3195 })?;
3196 let connection_name = match direction {
3197 Direction::DependsOn => "blockedBy",
3198 Direction::DependedOnBy => "blocking",
3199 };
3200 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
3201 // named natively and the reserved key may hold any far end. An issue's connections
3202 // hold issues, and this source reads them at the near item's own level.
3203 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
3204 if let Some(offset) = recorded {
3205 return Ok(recorded_page(
3206 self.recorded_edges(id, near_kind, direction, natively_names, node)
3207 .await?,
3208 offset,
3209 limit,
3210 ));
3211 }
3212 if natively_names.is_none() {
3213 return Ok(recorded_page(
3214 self.recorded_edges(id, near_kind, direction, natively_names, node)
3215 .await?,
3216 0,
3217 limit,
3218 ));
3219 }
3220 let connection = node
3221 .get(connection_name)
3222 .ok_or_else(|| SourceError::Malformed {
3223 message: "GitHub dependency response is missing its connection".into(),
3224 })?;
3225 let nodes = connection
3226 .get("nodes")
3227 .and_then(Value::as_array)
3228 .ok_or_else(|| SourceError::Malformed {
3229 message: "GitHub dependency response nodes is not an array".into(),
3230 })?;
3231 // `from` depends on `to`, always. GitHub spells the same relationship from either
3232 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
3233 // it — so the near item is `from` in one direction and `to` in the other.
3234 let items = nodes
3235 .iter()
3236 .map(|value| {
3237 let related = NativeId(required_str(value, "id")?.into());
3238 let related_kind = related_kind(value)?;
3239 let (from, to) = match direction {
3240 Direction::DependsOn => (
3241 DependencyEndpoint::from_native(id.clone(), near_kind),
3242 DependencyEndpoint::from_native(related, related_kind),
3243 ),
3244 Direction::DependedOnBy => (
3245 DependencyEndpoint::from_native(related, related_kind),
3246 DependencyEndpoint::from_native(id.clone(), near_kind),
3247 ),
3248 };
3249 Ok(DependencyEdge {
3250 from,
3251 to,
3252 kind: DependencyKind::Blocks,
3253 })
3254 })
3255 .collect::<Result<Vec<_>, SourceError>>()?;
3256 let mut next = next_cursor(connection)?;
3257 if let Some(next) = &next {
3258 validate_cursor_progress(cursor, &next.0)?;
3259 }
3260 if next.is_none()
3261 && !self
3262 .recorded_edges(id, near_kind, direction, natively_names, node)
3263 .await?
3264 .is_empty()
3265 {
3266 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
3267 }
3268 Ok(Page { items, next })
3269 }
3270
3271 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
3272 /// a far end in another source has to live: no GitHub issue relationship can name one.
3273 ///
3274 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
3275 /// source never writes one down.
3276 ///
3277 /// The metadata lives in the item's own body slot, and `node` is the dependency read's
3278 /// own answer, which carries an issue's body — so an issue's recorded edges cost no
3279 /// request beyond the read already made, and reading the board for them would be a
3280 /// walk of every item for one field of one. A draft has no body in that answer, because
3281 /// a draft lives only inside the board, so a draft's are read off the board as before.
3282 async fn recorded_edges(
3283 &self,
3284 id: &NativeId,
3285 near_kind: ItemKind,
3286 direction: Direction,
3287 natively_names: Option<ItemKind>,
3288 node: &Value,
3289 ) -> Result<Vec<DependencyEdge>, SourceError> {
3290 if direction != Direction::DependsOn {
3291 return Ok(Vec::new());
3292 }
3293 let slot = match node.get("body") {
3294 Some(body) if natively_names.is_some() => {
3295 metadata_body(body.as_str().map(str::to_owned))?.1
3296 }
3297 _ => {
3298 let Some(item) = self
3299 .board()
3300 .await?
3301 .items
3302 .into_iter()
3303 .find(|item| item.id == *id)
3304 else {
3305 return Ok(Vec::new());
3306 };
3307 item.slot
3308 }
3309 };
3310 DependencyEdge::recorded(&slot, id, near_kind, &self.name, natively_names)
3311 .map_err(|message| SourceError::Malformed { message })
3312 }
3313
3314 fn configured_repository(&self) -> Result<&RepositoryTarget, SourceError> {
3315 self.repository
3316 .as_ref()
3317 .ok_or_else(|| SourceError::Refused {
3318 message: format!(
3319 "source {} has no repository configured, and a GitHub Projects board has no \
3320 repository of its own to create an issue in; set repository: owner/name on \
3321 this source",
3322 self.name
3323 ),
3324 })
3325 }
3326
3327 /// The repository one new issue is created in, under the rule [`RepositoryTarget`]
3328 /// states.
3329 ///
3330 /// The fallback is demanded first, whichever arm answers: a write without a configured
3331 /// repository is refused naming the field exactly as it was before the rule existed,
3332 /// so a source that could not write before cannot write now, rather than writing for
3333 /// the one item whose own field happens to decide it.
3334 ///
3335 /// Everything this refuses is refused before `createIssue`, so a refusal leaves no
3336 /// issue behind: an entry that is not a repository on [`RepositoryTarget::HOST`], an
3337 /// entry owned by someone other than the owner of the parent issue's repository —
3338 /// GitHub accepts a sub-issue from another repository of the same owner and from no
3339 /// other, so `addSubIssue` would refuse it after the issue existed — a parent the
3340 /// board does not hold, and a parent that is a draft, which GitHub gives no sub-issues,
3341 /// both of which `addSubIssue` would likewise refuse too late. Whether the entry exists
3342 /// and is visible to the token is checked where its node id is resolved, still before
3343 /// `createIssue`. The parent is read off `board`, which is completed from
3344 /// this process's own record, so a project created moments ago in this command answers
3345 /// though GitHub's board read has not caught up.
3346 fn creation_target(
3347 &self,
3348 board: &Board,
3349 incoming: &Incoming<'_>,
3350 ) -> Result<RepositoryTarget, SourceError> {
3351 let fallback = self.configured_repository()?;
3352 let what = |incoming: &Incoming<'_>| {
3353 format!(
3354 "{} {:?}",
3355 incoming.written.kind().describes(),
3356 incoming.title
3357 )
3358 };
3359 let parent = incoming
3360 .parent
3361 .map(|parent| {
3362 board
3363 .items
3364 .iter()
3365 .find(|item| item.id == *parent)
3366 .ok_or_else(|| SourceError::Refused {
3367 message: format!(
3368 "GitHub project issue {} was not found on the board of source {}, \
3369 so {} cannot be filed under it",
3370 parent.0,
3371 self.name,
3372 what(incoming)
3373 ),
3374 })
3375 })
3376 .transpose()?;
3377 let parents_repository = parent
3378 .map(|parent| {
3379 // A draft is on the board and so is found, but it has no repository to
3380 // place a task in and GitHub gives it no sub-issues, so `addSubIssue`
3381 // would refuse the task only once `createIssue` had made it.
3382 if parent.content_kind == ContentKind::DraftIssue {
3383 return Err(SourceError::Refused {
3384 message: format!(
3385 "GitHub project item {} on the board of source {} is a draft, \
3386 which cannot have sub-issues, so {} cannot be filed under it",
3387 parent.id.0,
3388 self.name,
3389 what(incoming)
3390 ),
3391 });
3392 }
3393 // An issue's repository is where a sub-issue is placed and whose owner it
3394 // is compared against, so a parent whose repository this source cannot
3395 // spell as `owner/name` — GitHub's login grammar is wider than this
3396 // source's floor — is one nothing can be filed under.
3397 parent
3398 .own_repository
3399 .as_ref()
3400 .and_then(|origin| RepositoryTarget::from_origin(origin).ok())
3401 .ok_or_else(|| SourceError::Malformed {
3402 message: format!(
3403 "GitHub project issue {} on the board of source {} is in {}, which \
3404 is not a {}/owner/name repository this source can place {} in",
3405 parent.id.0,
3406 self.name,
3407 parent
3408 .own_repository
3409 .as_ref()
3410 .map_or("no repository", Repository::as_str),
3411 RepositoryTarget::HOST,
3412 what(incoming)
3413 ),
3414 })
3415 })
3416 .transpose()?;
3417 match incoming.repositories {
3418 [named] => {
3419 let target =
3420 RepositoryTarget::from_origin(named).map_err(|_| SourceError::Refused {
3421 message: format!(
3422 "{} names repository {}, which is not a {}/owner/name repository \
3423 source {} can create an issue in; name one that is, or name none",
3424 what(incoming),
3425 named.as_str(),
3426 RepositoryTarget::HOST,
3427 self.name
3428 ),
3429 })?;
3430 if let Some(parents) = &parents_repository
3431 && parents.owner != target.owner
3432 {
3433 return Err(SourceError::Refused {
3434 message: format!(
3435 "{} names repository {}, owned by {}, but its project's issue is in \
3436 {}, owned by {}, and GitHub files a sub-issue only in a repository \
3437 of the same owner as its parent issue; name a repository of {}, or \
3438 name none",
3439 what(incoming),
3440 target.slug(),
3441 target.owner,
3442 parents.slug(),
3443 parents.owner,
3444 parents.owner
3445 ),
3446 });
3447 }
3448 Ok(target)
3449 }
3450 _ => Ok(parents_repository.unwrap_or_else(|| fallback.clone())),
3451 }
3452 }
3453
3454 /// The node id of the repository `incoming` is being created in, or the refusal naming
3455 /// the item and the repository the token cannot see.
3456 ///
3457 /// Resolved once per command per repository; see [`Self::repository_cache`].
3458 async fn repository_id(
3459 &self,
3460 repository: &RepositoryTarget,
3461 incoming: &Incoming<'_>,
3462 ) -> Result<String, SourceError> {
3463 if let Some(id) = self.repository_cache()?.get(repository).cloned() {
3464 return Ok(id);
3465 }
3466 let data = self
3467 .graphql(
3468 graphql::REPOSITORY,
3469 json!({"owner":repository.owner,"name":repository.name}),
3470 )
3471 .await?;
3472 let node = data
3473 .get("repository")
3474 .filter(|value| !value.is_null())
3475 .ok_or_else(|| SourceError::Refused {
3476 message: format!(
3477 "GitHub repository {} was not found or is not visible to the token, so {} \
3478 {:?} cannot be created in it",
3479 repository.slug(),
3480 incoming.written.kind().describes(),
3481 incoming.title
3482 ),
3483 })?;
3484 let id = required_str(node, "id")?.to_owned();
3485 self.repository_cache()?
3486 .insert(repository.clone(), id.clone());
3487 Ok(id)
3488 }
3489
3490 fn repository_cache(
3491 &self,
3492 ) -> Result<std::sync::MutexGuard<'_, BTreeMap<RepositoryTarget, String>>, SourceError> {
3493 self.repository_cache
3494 .lock()
3495 .map_err(|_| SourceError::Unavailable {
3496 message: "this source's record of the destination repository was left \
3497 inconsistent by an earlier failure; next: run the command again"
3498 .into(),
3499 })
3500 }
3501
3502 /// Create or update one board item, whichever kind it is.
3503 async fn write_item(
3504 &self,
3505 incoming: &Incoming<'_>,
3506 target: Option<&NativeId>,
3507 depends_on: &[DependencyEdge],
3508 ) -> Result<NativeId, SourceError> {
3509 // Refused before anything is read or written: a task or a project titled the way
3510 // this board spells a document would land as an issue this same source reads back
3511 // as a document, so the field this destination cannot carry is named rather than
3512 // written and silently reclassified.
3513 if let Written::Work(kind, _) = incoming.written
3514 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
3515 {
3516 return Err(SourceError::Refused {
3517 message: format!(
3518 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
3519 spells a document, so it would read back as one rather than as a {}; \
3520 retitle it, or copy it as a document",
3521 kind.marker(),
3522 self.name,
3523 kind.marker()
3524 ),
3525 });
3526 }
3527 let board = match target {
3528 Some(target) => match self.board_for_update(target, incoming, depends_on).await? {
3529 Some(board) => board,
3530 None => self.board().await?,
3531 },
3532 None => self.board().await?,
3533 };
3534 let status_target = incoming
3535 .written
3536 .status()
3537 .map(|status| self.resolved_target(status.category))
3538 .transpose()?;
3539 let column = match (incoming.written.status(), status_target.as_ref()) {
3540 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
3541 _ => None,
3542 };
3543 let existing = target
3544 .map(|target| {
3545 board
3546 .items
3547 .iter()
3548 .find(|item| item.id == *target)
3549 .ok_or_else(|| SourceError::Refused {
3550 message: format!("GitHub destination item {} was not found", target.0),
3551 })
3552 })
3553 .transpose()?;
3554 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
3555 if content_kind == ContentKind::DraftIssue {
3556 if let (Some(StatusTarget::Closed(_)), Some(status)) =
3557 (status_target.as_ref(), incoming.written.status())
3558 {
3559 return Err(self.closes_a_draft(status.category));
3560 }
3561 if incoming.parent.is_some() {
3562 return Err(SourceError::Refused {
3563 message: "GitHub draft items cannot be a project's sub-issue".into(),
3564 });
3565 }
3566 }
3567 match existing {
3568 Some(item) if content_kind == ContentKind::Issue => {
3569 if item.labels != incoming.labels {
3570 return Err(SourceError::Refused {
3571 message: "GitHub issue labels differ from the labels being written".into(),
3572 });
3573 }
3574 }
3575 _ => {
3576 if !incoming.labels.is_empty() {
3577 return Err(SourceError::Refused {
3578 message: "GitHub items created by this destination carry no labels".into(),
3579 });
3580 }
3581 }
3582 }
3583
3584 // An existing issue is never moved; a new one is created where the rule says. The
3585 // repository the issue really lives in is what the slot below is written against,
3586 // so a single entry that is where the issue is created travels as no key at all,
3587 // and the read side derives it back from the issue.
3588 let (own_repository, creation_target) = match existing {
3589 Some(item) => (item.own_repository.clone(), None),
3590 None => {
3591 let target = self.creation_target(&board, incoming)?;
3592 let origin = Repository::try_from(target.origin())
3593 .map_err(|message| SourceError::Config { message })?;
3594 (Some(origin), Some(target))
3595 }
3596 };
3597 let (native, fallback) = self
3598 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
3599 .await?;
3600 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
3601 let body = compose_body(incoming.content, &slot)?;
3602 // Read before anything is created, for the reason the field below is: a value
3603 // this destination cannot store has to refuse, and refusing after `createIssue`
3604 // would leave an issue behind that nothing asked for. The engine writes a
3605 // qualified id here; a caller handing this key anything else is told so rather
3606 // than having it silently stored as no origin at all.
3607 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
3608 let origin = match incoming.metadata.get(ORIGIN_KEY) {
3609 None => "",
3610 Some(Value::String(origin)) => origin.as_str(),
3611 Some(other) => {
3612 return Err(SourceError::Refused {
3613 message: format!(
3614 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
3615 is {other}"
3616 ),
3617 });
3618 }
3619 };
3620 // Resolved before anything is created: a board that cannot carry the copy origin
3621 // has to refuse the write, and refusing it after `createIssue` would leave an
3622 // issue behind that nothing asked for.
3623 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
3624 Some(field) => {
3625 if required_str(field, "__typename")? != "ProjectV2Field" {
3626 return Err(SourceError::Refused {
3627 message: format!(
3628 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
3629 ),
3630 });
3631 }
3632 Some(required_str(field, "id")?.to_owned())
3633 }
3634 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
3635 return Err(SourceError::Refused {
3636 message: format!(
3637 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
3638 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
3639 the board"
3640 ),
3641 });
3642 }
3643 None => None,
3644 };
3645
3646 let (content_id, item_id, url) = match existing {
3647 Some(item) => {
3648 self.update_existing(item, incoming, &body, status_target.as_ref())
3649 .await?;
3650 (item.id.clone(), item.item_id.clone(), item.url.clone())
3651 }
3652 None => {
3653 let target = creation_target
3654 .as_ref()
3655 .ok_or_else(|| SourceError::Malformed {
3656 message: "a new item was decided without a repository to create it in"
3657 .into(),
3658 })?;
3659 self.create_and_file_issue(&board, target, incoming, &body, status_target.as_ref())
3660 .await?
3661 }
3662 };
3663
3664 let written_option = column.as_ref().map(|(_, _, name)| name.clone());
3665 let column = column.map(|(field, option, _)| (field, option));
3666 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
3667 // then each board field, the parent and the dependencies — and GitHub can fail at
3668 // any of them. Everything this source can refuse *before* the first of those is
3669 // already checked above, so what is left is GitHub itself failing part way. When it
3670 // does over an item this call created, the issue is taken back: a write that
3671 // refused must not leave an item behind that nobody asked for, and one that does
3672 // makes the retry create a second.
3673 let landed = self
3674 .finish_write(
3675 &board,
3676 incoming,
3677 &content_id,
3678 &item_id,
3679 content_kind,
3680 existing,
3681 origin_field.as_deref(),
3682 origin,
3683 column,
3684 &native,
3685 )
3686 .await;
3687 if let Err(error) = landed {
3688 if existing.is_none() {
3689 // Best effort, and the write's own failure is what the caller is told: a
3690 // refusal naming the tidy-up would hide why the write failed at all.
3691 let _ = self.delete_issue(&content_id).await;
3692 }
3693 return Err(error);
3694 }
3695
3696 // So the rest of this command reads what it just did rather than what the board
3697 // said before it. See `remember_written` for which half takes it.
3698 let remembered = Resolved {
3699 item_id,
3700 id: content_id.clone(),
3701 content_kind,
3702 kind: incoming.written.kind(),
3703 title: incoming.title.to_owned(),
3704 // The visible half of the body this write composed, split back off it the
3705 // way a read splits it — so what this record reports is what a read of the
3706 // same issue reports, rather than the person's text with the metadata slot
3707 // still on the end of it.
3708 body: metadata_body(body.clone())?.0,
3709 raw_body: body.clone(),
3710 // A document has no status of its own; what it reads back as is whatever
3711 // the issue's own state says, which is what a re-read reports.
3712 status: incoming
3713 .written
3714 .status()
3715 .cloned()
3716 .unwrap_or_else(|| Status {
3717 category: StatusCategory::Unknown,
3718 name: "Open".to_owned(),
3719 }),
3720 option: written_option.or_else(|| existing.and_then(|item| item.option.clone())),
3721 // What `state_input` asked for: closed for a closed target, open for any other
3722 // status, and the issue's own state left as it was by a document write.
3723 closed: content_kind == ContentKind::Issue
3724 && match status_target.as_ref() {
3725 Some(StatusTarget::Closed(_)) => true,
3726 Some(_) => false,
3727 None => existing.is_some_and(|item| item.closed),
3728 },
3729 delivers: incoming.delivers.to_vec(),
3730 delivered_by: incoming.delivered_by.to_vec(),
3731 labels: incoming.labels.to_vec(),
3732 parent: incoming.parent.cloned(),
3733 origin: (!origin.is_empty()).then(|| origin.to_owned()),
3734 // In the update path this is the item's own url, read off `existing` where the
3735 // tuple above was bound, so one expression serves both halves.
3736 url,
3737 created_at: existing.and_then(|item| item.created_at),
3738 updated_at: existing.and_then(|item| item.updated_at),
3739 own_repository,
3740 repositories: incoming.repositories.to_vec(),
3741 slot,
3742 board_id: Some(board.id.clone()),
3743 fields: board
3744 .fields
3745 .get("nodes")
3746 .and_then(Value::as_array)
3747 .cloned()
3748 .unwrap_or_default(),
3749 };
3750 self.remember_written(remembered, existing.is_none())?;
3751 Ok(content_id)
3752 }
3753
3754 /// What an update of an item this board already holds needs of the board, read off that
3755 /// item rather than off the board — or `None` when the item cannot say enough, and the
3756 /// board has to be read after all.
3757 ///
3758 /// An update needs the item it updates, the board's own id for a field write, the
3759 /// definitions of the `Status` and origin fields it writes, and every same-source far end
3760 /// its edges name. The first three ride along on the item's own node read — its board
3761 /// entry names the board's id, and each field value on it carries the definition of the
3762 /// field it is a value of — and each far end is read by its own node id. So updating one
3763 /// item costs a read of that item rather than every page of the board, which is what a
3764 /// copy naming one member out of many is for.
3765 ///
3766 /// **Nothing the item does not say is guessed.** A field this item holds no value of may
3767 /// still be on the board, and a view that read it as absent would refuse a write the
3768 /// board can take, or skip a field write the board needs. So a board this command has
3769 /// already read is used as it is, and an item that does not name its board, holds no
3770 /// value of the origin field, or holds no `Status` value when the write carries a status,
3771 /// sends the write back to reading the board — exactly as every update read it before.
3772 async fn board_for_update(
3773 &self,
3774 target: &NativeId,
3775 incoming: &Incoming<'_>,
3776 depends_on: &[DependencyEdge],
3777 ) -> Result<Option<Board>, SourceError> {
3778 if self.board_cache()?.is_some() {
3779 return Ok(None);
3780 }
3781 let Some(existing) = self.item_by_id(target).await? else {
3782 return Ok(None);
3783 };
3784 let Some(board_id) = existing.board_id.clone() else {
3785 return Ok(None);
3786 };
3787 let defines = |name: &str| {
3788 existing
3789 .fields
3790 .iter()
3791 .any(|field| field.get("name").and_then(Value::as_str) == Some(name))
3792 };
3793 if !defines(ORIGIN_FIELD) || (incoming.written.status().is_some() && !defines("Status")) {
3794 return Ok(None);
3795 }
3796 let fields = json!({"nodes": existing.fields, "pageInfo": {"hasNextPage": false}});
3797 let mut items = vec![existing];
3798 for edge in depends_on {
3799 if !edge
3800 .to
3801 .source()
3802 .is_none_or(|source| source == self.name.as_str())
3803 {
3804 continue;
3805 }
3806 // The first colon, for the reason `partition_edges` gives.
3807 let far = NativeId(if edge.to.is_qualified() {
3808 edge.to
3809 .id()
3810 .split_once(':')
3811 .map_or(edge.to.id(), |(_, native)| native)
3812 .to_owned()
3813 } else {
3814 edge.to.id().to_owned()
3815 });
3816 if items.iter().any(|item| item.id == far) {
3817 continue;
3818 }
3819 match self.item_by_id(&far).await? {
3820 Some(item) => items.push(item),
3821 // Refused against the whole board, in the words that refusal has always had.
3822 None => return Ok(None),
3823 }
3824 }
3825 Ok(Some(Board {
3826 id: board_id,
3827 fields,
3828 items,
3829 }))
3830 }
3831
3832 /// Everything a write does after the item exists: its board fields, its parent, and
3833 /// its dependencies.
3834 ///
3835 /// Split out of `write_item` so there is one place a failure past the point of no
3836 /// return is caught, rather than a tidy-up repeated at each `?` above.
3837 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
3838 // so there is one place a failure past the point of no return is caught, and its
3839 // arguments are exactly the values that tail already had in scope. Bundling them into a
3840 // struct would describe no concept — it would be "the arguments of this function" — and
3841 // would put the whole of `write_item`'s locals behind one more indirection.
3842 #[allow(clippy::too_many_arguments)]
3843 async fn finish_write(
3844 &self,
3845 board: &Board,
3846 incoming: &Incoming<'_>,
3847 content_id: &NativeId,
3848 item_id: &str,
3849 content_kind: ContentKind,
3850 existing: Option<&Resolved>,
3851 origin_field: Option<&str>,
3852 origin: &str,
3853 column: Option<(String, String)>,
3854 native: &[String],
3855 ) -> Result<(), SourceError> {
3856 if let Some(field_id) = origin_field {
3857 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
3858 .await?;
3859 }
3860
3861 if let Some((field_id, option_id)) = column {
3862 self.set_item_field(
3863 &board.id,
3864 item_id,
3865 &field_id,
3866 json!({"singleSelectOptionId":option_id}),
3867 )
3868 .await?;
3869 }
3870
3871 if content_kind == ContentKind::Issue {
3872 self.reparent(
3873 existing.and_then(|item| item.parent.clone()),
3874 content_id,
3875 incoming.parent,
3876 )
3877 .await?;
3878 // A document takes part in no dependency graph, so writing one neither reads
3879 // nor changes the issue's own `blockedBy` relationships. Reconciling them
3880 // against the empty list a document write carries would *delete* whatever
3881 // relationships a person had made on that issue, which is a write nobody
3882 // asked for.
3883 if incoming.written.kind() != BoardKind::Document {
3884 self.reconcile_blocked_by(content_id, native).await?;
3885 }
3886 }
3887 Ok(())
3888 }
3889
3890 /// Delete one issue, which takes its board item with it.
3891 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
3892 let data = self
3893 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3894 .await?;
3895 data.pointer("/deleteIssue/repository")
3896 .filter(|value| !value.is_null())
3897 .ok_or_else(|| SourceError::Malformed {
3898 message: "GitHub issue deletion returned no repository".into(),
3899 })?;
3900 self.forget(id)?;
3901 Ok(())
3902 }
3903
3904 /// Remove one item this copy created, so a copy that could not finish leaves the board
3905 /// as it found it.
3906 ///
3907 /// Deleting the issue takes its board item with it, so there is no second mutation to
3908 /// keep in step. An id the board does not hold is not an error: the item is already
3909 /// gone, which is the state this asks for.
3910 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
3911 let board = self.board().await?;
3912 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
3913 return Ok(());
3914 };
3915 if item.content_kind == ContentKind::DraftIssue {
3916 return Err(SourceError::Refused {
3917 message: format!(
3918 "GitHub item {} is a draft, and this source removes an item by deleting \
3919 its issue; next: remove it from the board by hand",
3920 id.0
3921 ),
3922 });
3923 }
3924 let data = self
3925 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3926 .await?;
3927 data.pointer("/deleteIssue/repository")
3928 .filter(|value| !value.is_null())
3929 .ok_or_else(|| SourceError::Malformed {
3930 message: "GitHub issue deletion returned no repository".into(),
3931 })?;
3932 self.forget(id)?;
3933 Ok(())
3934 }
3935
3936 /// The issue a comment call on `task` is about, or `None` when this board holds no such
3937 /// task.
3938 ///
3939 /// Resolved exactly as [`TaskSource::get_task`] resolves it, so the comment verbs and a
3940 /// read of the task cannot disagree about which ids name one: a project or a document of
3941 /// this board is not a task here either.
3942 ///
3943 /// A **draft** is a task with nowhere to keep a comment, because GitHub keeps comments on
3944 /// issues and a draft is not one. It is refused rather than answered with an empty page,
3945 /// which would read as a task nobody has commented on yet.
3946 async fn commented_issue(&self, task: &NativeId) -> Result<Option<NativeId>, SourceError> {
3947 let Some(item) = self
3948 .item_by_id(task)
3949 .await?
3950 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3951 else {
3952 return Ok(None);
3953 };
3954 if item.content_kind == ContentKind::DraftIssue {
3955 return Err(SourceError::Refused {
3956 message: format!(
3957 "task {} of source {} is a draft item on the board, and GitHub keeps \
3958 comments on issues alone, so a draft has none to read or write; next: \
3959 convert the draft to an issue on the board, then comment on the issue it \
3960 becomes",
3961 task.0, self.name
3962 ),
3963 });
3964 }
3965 Ok(Some(item.id))
3966 }
3967
3968 /// Whether the comment `comment` is one of `issue`'s own.
3969 ///
3970 /// Read before an edit or a removal is sent, because GitHub's comment mutations take the
3971 /// comment's id and nothing else: a comment id given against the wrong task would
3972 /// otherwise change a comment on some other issue entirely. An id that names nothing, or
3973 /// names something that is not an issue comment, is a comment this task does not have —
3974 /// which is what GitHub refusing to resolve it means too.
3975 async fn comment_is_on(
3976 &self,
3977 issue: &NativeId,
3978 comment: &NativeId,
3979 ) -> Result<bool, SourceError> {
3980 let asked = self
3981 .graphql(graphql::COMMENT_ISSUE, json!({"id":comment.0}))
3982 .await;
3983 let data = match asked {
3984 Ok(data) => data,
3985 Err(error) if unresolvable_node(&error) => return Ok(false),
3986 Err(error) => return Err(error),
3987 };
3988 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
3989 return Ok(false);
3990 };
3991 if optional_str(node, "__typename")? != Some("IssueComment") {
3992 return Ok(false);
3993 }
3994 let on = node.get("issue").ok_or_else(|| SourceError::Malformed {
3995 message: format!("GitHub issue comment {} names no issue", comment.0),
3996 })?;
3997 Ok(required_str(on, "id")? == issue.0)
3998 }
3999
4000 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
4001 async fn partition_edges(
4002 &self,
4003 board: &Board,
4004 near_kind: BoardKind,
4005 near_content: ContentKind,
4006 depends_on: &[DependencyEdge],
4007 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
4008 let mut native = Vec::new();
4009 let mut fallback = Vec::new();
4010 for edge in depends_on {
4011 let same_source = edge
4012 .to
4013 .source()
4014 .is_none_or(|source| source == self.name.as_str());
4015 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
4016 // `DependencyEndpoint::source` both read it that way — and a native id may hold
4017 // colons of its own, so the far end is everything after that one separator.
4018 // Splitting at the last would truncate `work:urn:task:7` to `7`.
4019 let far_id = if edge.to.is_qualified() {
4020 edge.to
4021 .id()
4022 .split_once(':')
4023 .map_or(edge.to.id(), |(_, native)| native)
4024 } else {
4025 edge.to.id()
4026 };
4027 let far = if same_source {
4028 Some(
4029 board
4030 .items
4031 .iter()
4032 .find(|item| item.id.0 == far_id)
4033 .ok_or_else(|| SourceError::Refused {
4034 message: format!("GitHub dependency item {far_id} was not found"),
4035 })?,
4036 )
4037 } else {
4038 None
4039 };
4040 // The caller says which kind the far end is, and this board holds the far end
4041 // itself, so a disagreement is settled here rather than stored: recorded, the
4042 // wrong kind would read back as a cross-level edge that never existed; written
4043 // natively, it would name a relationship of a different level than the caller
4044 // asked for.
4045 //
4046 // A far end this board holds as a *document* fails the same comparison and is
4047 // refused by the same sentence: `ItemKind` has no document variant because
4048 // nothing may point at one, so no caller can name it correctly and the refusal
4049 // is the only honest answer.
4050 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
4051 return Err(SourceError::Refused {
4052 message: format!(
4053 "GitHub dependency item {far_id} is a {} of this board, and this item \
4054 names it as a {}; record the kind it is",
4055 disagreeing.kind.describes(),
4056 edge.to.kind.marker()
4057 ),
4058 });
4059 }
4060 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
4061 // however the far end is spelled — and one classified native here would be
4062 // written nowhere at all, because a draft's native reconciliation never runs.
4063 let native_here = near_content == ContentKind::Issue
4064 && far.is_some_and(|far| {
4065 far.content_kind == ContentKind::Issue
4066 && BoardKind::Work(edge.to.kind) == near_kind
4067 });
4068 if native_here {
4069 native.push(far_id.to_owned());
4070 } else {
4071 fallback.push(edge.clone());
4072 }
4073 }
4074 Ok((native, fallback))
4075 }
4076
4077 async fn update_existing(
4078 &self,
4079 item: &Resolved,
4080 incoming: &Incoming<'_>,
4081 body: &Option<String>,
4082 status_target: Option<&StatusTarget>,
4083 ) -> Result<(), SourceError> {
4084 let title = incoming.written_title();
4085 let fields = match item.content_kind {
4086 ContentKind::DraftIssue => json!({"title":title,"body":body}),
4087 ContentKind::Issue => json!({"title":title,"body":body,
4088 "stateInput":state_input(status_target)}),
4089 };
4090 self.update_content(item.content_kind, &item.id, fields)
4091 .await
4092 }
4093
4094 /// Update one board item's content with exactly `fields` beside its id, through the
4095 /// mutation its kind takes: `updateIssue` for an issue, `updateProjectV2DraftIssue` for
4096 /// a draft.
4097 ///
4098 /// Every input field either mutation leaves out is a field GitHub leaves as it is, which
4099 /// is what lets a narrow write carry the one thing it changes and nothing else.
4100 async fn update_content(
4101 &self,
4102 kind: ContentKind,
4103 id: &NativeId,
4104 fields: Value,
4105 ) -> Result<(), SourceError> {
4106 let (operation, id_key, pointer) = match kind {
4107 ContentKind::DraftIssue => (
4108 graphql::UPDATE_DRAFT,
4109 "draftIssueId",
4110 "/updateProjectV2DraftIssue/draftIssue",
4111 ),
4112 ContentKind::Issue => (graphql::UPDATE_ISSUE, "id", "/updateIssue/issue"),
4113 };
4114 let mut input = fields;
4115 input[id_key] = json!(id.0);
4116 let data = self.graphql(operation, json!({"input":input})).await?;
4117 let returned = data
4118 .pointer(pointer)
4119 .ok_or_else(|| SourceError::Malformed {
4120 message: "GitHub item update returned no item".into(),
4121 })?;
4122 if required_str(returned, "id")? != id.0 {
4123 return Err(SourceError::Malformed {
4124 message: "GitHub item update returned the wrong item".into(),
4125 });
4126 }
4127 Ok(())
4128 }
4129
4130 /// Creates one issue, files it on the board, and closes it when the status says so.
4131 ///
4132 /// Three calls rather than one: `createIssue` needs a repository and answers with an
4133 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
4134 /// closed status is a state of the issue rather than a field of the board item.
4135 /// Creates the issue, files it on the board, and reports what a read of it would say:
4136 /// its content id, its board item id, and the web address GitHub gave it.
4137 ///
4138 /// The address comes back here because this is the only place it is known before
4139 /// GitHub's own board read catches up — an item this run created answers the reads
4140 /// that follow it out of the record below, and one remembered without its address
4141 /// would report no location for the rest of the run.
4142 async fn create_and_file_issue(
4143 &self,
4144 board: &Board,
4145 repository: &RepositoryTarget,
4146 incoming: &Incoming<'_>,
4147 body: &Option<String>,
4148 status_target: Option<&StatusTarget>,
4149 ) -> Result<(NativeId, String, Option<String>), SourceError> {
4150 let repository_id = self.repository_id(repository, incoming).await?;
4151 let data = self
4152 .graphql(
4153 graphql::CREATE_ISSUE,
4154 json!({"input":{
4155 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
4156 }}),
4157 )
4158 .await?;
4159 let created = data
4160 .pointer("/createIssue/issue")
4161 .filter(|value| !value.is_null())
4162 .ok_or_else(|| SourceError::Malformed {
4163 message: "GitHub issue creation returned no issue".into(),
4164 })?;
4165 let content_id = NativeId(required_str(created, "id")?.to_owned());
4166 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
4167 // a response without it is not worth failing a landed write over — the item simply
4168 // reports no location until the board read catches up, which is what it did before.
4169 let url = optional_str(created, "url")?.map(str::to_owned);
4170 // The issue exists from here on, so a failure filing it on the board takes it
4171 // back: an issue in the repository that is on no board is an item nobody asked for
4172 // and nothing here would find again.
4173 let added = match self
4174 .graphql(
4175 graphql::ADD_TO_BOARD,
4176 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
4177 )
4178 .await
4179 {
4180 Ok(added) => added,
4181 Err(error) => {
4182 let _ = self.delete_issue(&content_id).await;
4183 return Err(error);
4184 }
4185 };
4186 let item = added
4187 .pointer("/addProjectV2ItemById/item")
4188 .filter(|value| !value.is_null())
4189 .ok_or_else(|| SourceError::Malformed {
4190 message: "GitHub board addition returned no project item".into(),
4191 })?;
4192 if let Some(StatusTarget::Closed(_)) = status_target {
4193 self.update_content(
4194 ContentKind::Issue,
4195 &content_id,
4196 json!({"stateInput":state_input(status_target)}),
4197 )
4198 .await?;
4199 }
4200 Ok((content_id, required_str(item, "id")?.to_owned(), url))
4201 }
4202
4203 /// Move one issue under the project it now belongs to, or out of the one it left.
4204 async fn reparent(
4205 &self,
4206 held: Option<NativeId>,
4207 child: &NativeId,
4208 wanted: Option<&NativeId>,
4209 ) -> Result<(), SourceError> {
4210 if held.as_ref() == wanted {
4211 return Ok(());
4212 }
4213 if let Some(held) = &held {
4214 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
4215 .await?;
4216 }
4217 if let Some(wanted) = wanted {
4218 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
4219 .await?;
4220 }
4221 Ok(())
4222 }
4223
4224 async fn sub_issue(
4225 &self,
4226 operation: &str,
4227 parent: &NativeId,
4228 child: &NativeId,
4229 root: &str,
4230 ) -> Result<(), SourceError> {
4231 let data = self
4232 .graphql(
4233 operation,
4234 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
4235 )
4236 .await?;
4237 let issue =
4238 data.pointer(&format!("/{root}/issue"))
4239 .ok_or_else(|| SourceError::Malformed {
4240 message: "GitHub sub-issue update returned no issue".into(),
4241 })?;
4242 let sub =
4243 data.pointer(&format!("/{root}/subIssue"))
4244 .ok_or_else(|| SourceError::Malformed {
4245 message: "GitHub sub-issue update returned no sub-issue".into(),
4246 })?;
4247 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
4248 return Err(SourceError::Malformed {
4249 message: "GitHub sub-issue update returned the wrong issues".into(),
4250 });
4251 }
4252 Ok(())
4253 }
4254
4255 async fn reconcile_blocked_by(
4256 &self,
4257 content_id: &NativeId,
4258 native: &[String],
4259 ) -> Result<(), SourceError> {
4260 let current = self.native_dependency_ids(content_id).await?;
4261 for (operation, far_id) in current
4262 .iter()
4263 .filter(|id| !native.contains(id))
4264 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
4265 .chain(
4266 native
4267 .iter()
4268 .filter(|id| !current.contains(id))
4269 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
4270 )
4271 {
4272 let data = self
4273 .graphql(
4274 operation,
4275 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
4276 )
4277 .await?;
4278 let root = if operation == graphql::ADD_BLOCKED_BY {
4279 "addBlockedBy"
4280 } else {
4281 "removeBlockedBy"
4282 };
4283 let issue =
4284 data.pointer(&format!("/{root}/issue"))
4285 .ok_or_else(|| SourceError::Malformed {
4286 message: "GitHub dependency update returned no issue".into(),
4287 })?;
4288 let blocker = data
4289 .pointer(&format!("/{root}/blockingIssue"))
4290 .ok_or_else(|| SourceError::Malformed {
4291 message: "GitHub dependency update returned no blocking issue".into(),
4292 })?;
4293 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
4294 {
4295 return Err(SourceError::Malformed {
4296 message: "GitHub dependency update returned the wrong issues".into(),
4297 });
4298 }
4299 }
4300 Ok(())
4301 }
4302}
4303
4304/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
4305enum Reached {
4306 /// An issue this board holds, resolved into everything this source reports about it.
4307 Held(Box<Resolved>),
4308 /// Nothing this board holds: no such node, or a node on some other board.
4309 Nothing,
4310 /// A board draft, which exists only inside the board's own item connection.
4311 Draft,
4312}
4313
4314/// What GitHub says when a string is not a node id it can resolve.
4315///
4316/// Matched because it is the ordinary answer to a project selector naming a project by its
4317/// *name*, and reporting that as a failure would make naming one impossible. It is read
4318/// off the refusal GitHub sent, never guessed from the shape of the string: this source
4319/// does not define the syntax of a GitHub node id and would be wrong about it.
4320const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
4321
4322/// Whether this refusal is GitHub saying the id names no node at all.
4323fn unresolvable_node(error: &SourceError) -> bool {
4324 matches!(error, SourceError::Refused { message }
4325 if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
4326}
4327
4328/// One project name, as a search qualifier which filters on it at the server.
4329///
4330/// Quoted so the whole title is one phrase rather than a bag of words, with the two
4331/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
4332/// the way it documents. A title matched here is still compared for equality afterwards:
4333/// the qualifier narrows what the server sends, and this source decides what it names.
4334fn title_qualifier(name: &str) -> String {
4335 let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
4336 format!("in:title \"{escaped}\"")
4337}
4338
4339/// The board, and every item on it this source reports.
4340#[derive(Clone)]
4341struct Board {
4342 id: String,
4343 fields: Value,
4344 items: Vec<Resolved>,
4345}
4346
4347impl Board {
4348 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
4349 complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
4350 let nodes = fields
4351 .get("nodes")
4352 .and_then(Value::as_array)
4353 .ok_or_else(|| SourceError::Malformed {
4354 message: "GitHub project fields.nodes is not an array".into(),
4355 })?;
4356 Ok(nodes
4357 .iter()
4358 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
4359 }
4360}
4361
4362/// One board item, resolved into everything this source reports about it.
4363#[derive(Clone)]
4364struct Resolved {
4365 item_id: String,
4366 id: NativeId,
4367 content_kind: ContentKind,
4368 kind: BoardKind,
4369 title: String,
4370 body: Option<String>,
4371 /// The body exactly as GitHub holds it, metadata slot and all, which is what a write
4372 /// that changes the slot alone has to keep byte for byte outside it.
4373 raw_body: Option<String>,
4374 status: Status,
4375 /// The name of the board `Status` option this item sits in, as the board spells it.
4376 option: Option<String>,
4377 /// Whether this item's issue is closed. A draft has no such state and is never closed.
4378 closed: bool,
4379 /// The tasks this one delivers, read out of its slot. Empty for anything not a task.
4380 delivers: Vec<TaskRef>,
4381 /// Every task that delivers this one, read out of its slot. Empty for anything not a
4382 /// task.
4383 delivered_by: Vec<TaskRef>,
4384 labels: Vec<Label>,
4385 parent: Option<NativeId>,
4386 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
4387 origin: Option<String>,
4388 url: Option<String>,
4389 created_at: Option<DateTime<Utc>>,
4390 updated_at: Option<DateTime<Utc>>,
4391 own_repository: Option<Repository>,
4392 repositories: Vec<Repository>,
4393 slot: BTreeMap<String, Value>,
4394 /// The node id of the board this item sits on, when the read that reached it said.
4395 board_id: Option<String>,
4396 /// The definition of every board field this item holds a value of, in the shape a read
4397 /// of the board's own `fields` gives one.
4398 ///
4399 /// Only the fields this item has a value in: a field it holds nothing of is not here,
4400 /// which says nothing about whether the board has it.
4401 fields: Vec<Value>,
4402}
4403
4404impl Resolved {
4405 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
4406 /// in a field of its own, and none of the five keys that are only an encoding.
4407 ///
4408 /// The two delivery keys are left out for every kind, not only for a task: they are
4409 /// the encoding of [`Task::delivers`] and [`Task::delivered_by`], and a project or a
4410 /// document carrying one holds nothing a caller's own metadata could mean by it.
4411 fn metadata(&self) -> BTreeMap<String, Value> {
4412 let mut metadata = self.slot.clone();
4413 metadata.remove(Repository::METADATA_KEY);
4414 metadata.remove(DependencyEdge::RECORDED_KEY);
4415 metadata.remove(ItemKind::METADATA_KEY);
4416 metadata.remove(TaskRef::DELIVERS_KEY);
4417 metadata.remove(TaskRef::DELIVERED_BY_KEY);
4418 if let Some(origin) = &self.origin {
4419 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
4420 }
4421 metadata
4422 }
4423
4424 /// Where this item is, as a link a reader can open.
4425 ///
4426 /// A board is a hosted place and every issue on it has a web address, so that address
4427 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
4428 /// of place it is, so a reader knows to open it rather than to read a file out. It
4429 /// does not replace or derive from `url`: the field goes on reporting exactly what it
4430 /// reported before, and this says what that address *is*.
4431 ///
4432 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
4433 /// rather than a third variant, which is the contract's "the source did not say". An
4434 /// issue this run created is not one of those: its address comes back from the
4435 /// creating mutation, so it is somewhere a reader can open from the moment it exists
4436 /// rather than from whenever the board read catches up.
4437 fn location(&self) -> Option<Location> {
4438 self.url.clone().map(Location::Url)
4439 }
4440
4441 fn task(&self) -> Task {
4442 Task {
4443 id: self.id.clone(),
4444 title: self.title.clone(),
4445 content: self.body.clone(),
4446 status: self.status.clone(),
4447 labels: self.labels.clone(),
4448 project: self.parent.clone(),
4449 url: self.url.clone(),
4450 location: self.location(),
4451 created_at: self.created_at,
4452 updated_at: self.updated_at,
4453 metadata: self.metadata(),
4454 repositories: self.repositories.clone(),
4455 delivers: self.delivers.clone(),
4456 delivered_by: self.delivered_by.clone(),
4457 }
4458 }
4459
4460 fn project(&self) -> Project {
4461 Project {
4462 id: self.id.clone(),
4463 title: self.title.clone(),
4464 content: self.body.clone(),
4465 status: self.status.clone(),
4466 labels: self.labels.clone(),
4467 url: self.url.clone(),
4468 location: self.location(),
4469 created_at: self.created_at,
4470 updated_at: self.updated_at,
4471 metadata: self.metadata(),
4472 repositories: self.repositories.clone(),
4473 }
4474 }
4475
4476 /// The same issue as a document: the project it is filed under, and no status and no
4477 /// dependencies, because a document is not work.
4478 fn document(&self) -> Document {
4479 Document {
4480 id: self.id.clone(),
4481 title: self.title.clone(),
4482 content: self.body.clone(),
4483 project: self.parent.clone(),
4484 labels: self.labels.clone(),
4485 url: self.url.clone(),
4486 location: self.location(),
4487 created_at: self.created_at,
4488 updated_at: self.updated_at,
4489 metadata: self.metadata(),
4490 repositories: self.repositories.clone(),
4491 }
4492 }
4493}
4494
4495/// What one write is, and the status that comes with being it.
4496///
4497/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
4498/// status and a task or a project always has one, so "a document carrying a status" and
4499/// "a task carrying none" are states a write cannot be in rather than states every use
4500/// site below has to defend against.
4501enum Written<'a> {
4502 /// A document, which is not work and so has no status at all.
4503 Document,
4504 /// A task or a project, and the status it is being written with.
4505 Work(ItemKind, &'a Status),
4506}
4507
4508impl Written<'_> {
4509 /// Which of the board's three kinds this write is.
4510 const fn kind(&self) -> BoardKind {
4511 match self {
4512 Self::Document => BoardKind::Document,
4513 Self::Work(kind, _) => BoardKind::Work(*kind),
4514 }
4515 }
4516
4517 /// The status this write carries. A document carries none, so a write of one says
4518 /// nothing about the issue's open or closed state and selects no board `Status`
4519 /// option.
4520 const fn status(&self) -> Option<&Status> {
4521 match self {
4522 Self::Document => None,
4523 Self::Work(_, status) => Some(status),
4524 }
4525 }
4526}
4527
4528/// The item being written, in the one shape all three write methods reach.
4529struct Incoming<'a> {
4530 written: Written<'a>,
4531 /// The title a person wrote. A document's goes onto the issue with
4532 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
4533 title: &'a str,
4534 content: Option<&'a str>,
4535 labels: &'a [Label],
4536 metadata: &'a BTreeMap<String, Value>,
4537 repositories: &'a [Repository],
4538 parent: Option<&'a NativeId>,
4539 /// [`Task::delivers`], already checked. Empty for a project or a document, which is
4540 /// what keeps either key out of their slot.
4541 delivers: &'a [TaskRef],
4542 /// [`Task::delivered_by`], already checked. Empty for a project or a document.
4543 delivered_by: &'a [TaskRef],
4544}
4545
4546impl Incoming<'_> {
4547 /// The title this write puts on the issue.
4548 fn written_title(&self) -> String {
4549 match self.written {
4550 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
4551 Written::Work(..) => self.title.to_owned(),
4552 }
4553 }
4554}
4555
4556#[derive(Clone, Copy, PartialEq, Eq)]
4557enum ContentKind {
4558 DraftIssue,
4559 Issue,
4560}
4561
4562/// What one board issue is: a document, or the work an [`ItemKind`] names.
4563///
4564/// A type of this source's own rather than an `ItemKind` with a third variant, because
4565/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
4566/// document — the contract keeps a document out of that enum deliberately. Holding the
4567/// board's three answers in one value is what makes every place that asks "which is this?"
4568/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
4569/// two thirds of the board.
4570#[derive(Clone, Copy, PartialEq, Eq)]
4571enum BoardKind {
4572 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
4573 Document,
4574 /// Every other issue, and every draft.
4575 Work(ItemKind),
4576}
4577
4578impl BoardKind {
4579 /// How a refusal names this kind to the person reading it.
4580 const fn describes(self) -> &'static str {
4581 match self {
4582 Self::Document => "document",
4583 Self::Work(kind) => kind.marker(),
4584 }
4585 }
4586}
4587
4588/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
4589///
4590/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
4591/// the shared cross-source journeys assert one answer to one question, so two sources
4592/// that disagree about what "carries the label bug" means fail them.
4593fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
4594 let holds = |name: &String| {
4595 labels
4596 .iter()
4597 .any(|label| label.name.eq_ignore_ascii_case(name))
4598 };
4599 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
4600 && filter.all_of.iter().all(holds)
4601 && !filter.none_of.iter().any(holds)
4602}
4603
4604/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
4605/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
4606fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
4607 statuses.is_empty() || statuses.contains(&category)
4608}
4609
4610/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
4611///
4612/// `content` is the item's own prose — the body with this source's trailing metadata
4613/// comment already taken off — so a search never matches an encoding the author of the
4614/// issue never wrote.
4615fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
4616 let terms = query.terms.to_lowercase();
4617 let in_title = title.to_lowercase().contains(&terms);
4618 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
4619 match query.fields {
4620 TextFields::Title => in_title,
4621 TextFields::Content => in_content,
4622 TextFields::TitleOrContent => in_title || in_content,
4623 }
4624}
4625
4626/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
4627///
4628/// The project predicate is passed separately because a read narrowed to one project has
4629/// already answered it by asking *that project* for its own items — and re-applying it
4630/// there would compare the caller's selector, which may be a project's **name**, against
4631/// the id of the project that name resolved to, and keep nothing. Every other read passes
4632/// `query.project` and applies it here, which is what keeps `projects` a predicate this
4633/// source really does apply.
4634fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
4635 labels_match(&task.labels, &query.labels)
4636 && status_matches(task.status.category, &query.statuses)
4637 && match project {
4638 ProjectFilter::Any => true,
4639 ProjectFilter::Orphans => task.project.is_none(),
4640 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
4641 }
4642 && query
4643 .text
4644 .as_ref()
4645 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
4646}
4647
4648fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
4649 labels_match(&project.labels, &query.labels)
4650 && status_matches(project.status.category, &query.statuses)
4651 && query
4652 .text
4653 .as_ref()
4654 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
4655}
4656
4657/// The same three predicates a task query carries, minus the status filter.
4658///
4659/// A document is not work, so it has no status for one to compare against and the query
4660/// type carries none. The project predicate is the same one — a design issue filed under a
4661/// project issue is in that project, and one filed under nothing is in none — so it is
4662/// spelled the same way here rather than answered differently.
4663fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
4664 labels_match(&document.labels, &query.labels)
4665 && match project {
4666 ProjectFilter::Any => true,
4667 ProjectFilter::Orphans => document.project.is_none(),
4668 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
4669 }
4670 && query
4671 .text
4672 .as_ref()
4673 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
4674}
4675
4676#[async_trait::async_trait]
4677impl TaskSource for GitHubProjectsSource {
4678 fn kind(&self) -> &'static str {
4679 KIND
4680 }
4681 fn capabilities(&self) -> Capabilities {
4682 Capabilities {
4683 projects: Support::Native,
4684 documents: Support::Native,
4685 comments: Support::Native,
4686 orphan_tasks: Support::Native,
4687 filter_by_label: Support::Native,
4688 filter_by_status: Support::Native,
4689 search_title: Support::Native,
4690 search_content: Support::Native,
4691 task_dependencies: DependencySupport::BothDirections,
4692 project_dependencies: DependencySupport::BothDirections,
4693 max_page_size: MAX_PAGE_SIZE,
4694 }
4695 }
4696 async fn health(&self) -> Result<Health, SourceError> {
4697 let board = self.board_page(None, 1).await?;
4698 Ok(Health {
4699 reachable: true,
4700 detail: Some(format!(
4701 "reading GitHub project {}/{} ({})",
4702 self.owner,
4703 self.project_number,
4704 required_str(&board, "title")?
4705 )),
4706 })
4707 }
4708 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
4709 Ok(self
4710 .item_by_id(id)
4711 .await?
4712 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4713 .map(|item| item.task()))
4714 }
4715 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
4716 Ok(self
4717 .item_by_id(id)
4718 .await?
4719 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4720 .map(|item| item.project()))
4721 }
4722 async fn query_tasks(
4723 &self,
4724 query: &TaskQuery,
4725 page: &PageRequest,
4726 ) -> Result<Page<Task>, SourceError> {
4727 validate_page(page)?;
4728 // A read narrowed to one project asks that project for its own tasks, so nothing
4729 // about it costs what the rest of the board holds. Every other task read is a
4730 // question about the whole board and is answered by reading it.
4731 let (held, membership) = match &query.project {
4732 ProjectFilter::Is(project) => (
4733 self.project_children(project).await?,
4734 // Answered by where these items came from; see `task_matches`.
4735 &ProjectFilter::Any,
4736 ),
4737 ProjectFilter::Any | ProjectFilter::Orphans => {
4738 (self.board().await?.items, &query.project)
4739 }
4740 };
4741 // Filtered before paged: a page of a filtered result is a page of the survivors,
4742 // never the survivors of a page.
4743 let tasks = held
4744 .iter()
4745 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4746 .map(Resolved::task)
4747 .filter(|task| task_matches(task, query, membership))
4748 .collect();
4749 Ok(offset_page(
4750 tasks,
4751 numeric_cursor(page.cursor.as_ref())?,
4752 page.limit.min(MAX_PAGE_SIZE) as usize,
4753 ))
4754 }
4755 async fn query_projects(
4756 &self,
4757 query: &ProjectQuery,
4758 page: &PageRequest,
4759 ) -> Result<Page<Project>, SourceError> {
4760 validate_page(page)?;
4761 // The projects a board holds are found by an issue search scoped to that board,
4762 // never by walking the board's own item connection: what tells a project from a
4763 // task is the `parent` each issue carries, which costs nothing to read.
4764 let projects = self
4765 .board_issues()
4766 .await?
4767 .iter()
4768 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4769 .map(Resolved::project)
4770 .filter(|project| project_matches(project, query))
4771 .collect();
4772 Ok(offset_page(
4773 projects,
4774 numeric_cursor(page.cursor.as_ref())?,
4775 page.limit.min(MAX_PAGE_SIZE) as usize,
4776 ))
4777 }
4778 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
4779 Ok(self
4780 .item_by_id(id)
4781 .await?
4782 .filter(|item| item.kind == BoardKind::Document)
4783 .map(|item| item.document()))
4784 }
4785 async fn query_documents(
4786 &self,
4787 query: &DocumentQuery,
4788 page: &PageRequest,
4789 ) -> Result<Page<Document>, SourceError> {
4790 validate_page(page)?;
4791 // Narrowed to one project, this is the same sub-issue read a task list scoped to
4792 // that project makes — a document filed under a project is a sub-issue of it too,
4793 // and which of them come back is the kind this caller asked for.
4794 let (held, membership) = match &query.project {
4795 ProjectFilter::Is(project) => (
4796 self.project_children(project).await?,
4797 // Answered by where these items came from; see `task_matches`.
4798 &ProjectFilter::Any,
4799 ),
4800 ProjectFilter::Any | ProjectFilter::Orphans => {
4801 (self.board().await?.items, &query.project)
4802 }
4803 };
4804 // Filtered before paged, exactly as a task read is: a page of a filtered result is
4805 // a page of the survivors, never the survivors of a page.
4806 let documents = held
4807 .iter()
4808 .filter(|item| item.kind == BoardKind::Document)
4809 .map(Resolved::document)
4810 .filter(|document| document_matches(document, query, membership))
4811 .collect();
4812 Ok(offset_page(
4813 documents,
4814 numeric_cursor(page.cursor.as_ref())?,
4815 page.limit.min(MAX_PAGE_SIZE) as usize,
4816 ))
4817 }
4818 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
4819 validate_page(page)?;
4820 let offset = numeric_cursor(page.cursor.as_ref())?;
4821 let mut labels = self
4822 .board()
4823 .await?
4824 .items
4825 .into_iter()
4826 .flat_map(|item| item.labels)
4827 .fold(Vec::new(), |mut all, label| {
4828 if !all.iter().any(|x: &Label| x.id == label.id) {
4829 all.push(label);
4830 }
4831 all
4832 });
4833 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
4834 Ok(offset_page(
4835 labels,
4836 offset,
4837 page.limit.min(MAX_PAGE_SIZE) as usize,
4838 ))
4839 }
4840 async fn task_dependencies(
4841 &self,
4842 id: &NativeId,
4843 direction: Direction,
4844 page: &PageRequest,
4845 ) -> Result<Page<DependencyEdge>, SourceError> {
4846 self.dependencies(id, ItemKind::Task, direction, page).await
4847 }
4848 async fn project_dependencies(
4849 &self,
4850 id: &NativeId,
4851 direction: Direction,
4852 page: &PageRequest,
4853 ) -> Result<Page<DependencyEdge>, SourceError> {
4854 self.dependencies(id, ItemKind::Project, direction, page)
4855 .await
4856 }
4857
4858 fn writes(&self) -> WriteSupport {
4859 WriteSupport::Supported
4860 }
4861
4862 /// Create or update one task.
4863 ///
4864 /// Its `delivers` and `delivered_by` are checked before anything is read or written —
4865 /// neither may name the task itself or name one task twice — and land in the body's
4866 /// metadata slot under their reserved keys, in place of any caller metadata of those
4867 /// names.
4868 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
4869 let near = write.target.as_ref().unwrap_or(&write.item.id);
4870 for (key, entries) in [
4871 (TaskRef::DELIVERS_KEY, &write.item.delivers),
4872 (TaskRef::DELIVERED_BY_KEY, &write.item.delivered_by),
4873 ] {
4874 TaskRef::listed(key, near, Some(&self.name), entries.clone())
4875 .map_err(|message| SourceError::Refused { message })?;
4876 }
4877 self.write_item(
4878 &Incoming {
4879 written: Written::Work(ItemKind::Task, &write.item.status),
4880 title: &write.item.title,
4881 content: write.item.content.as_deref(),
4882 labels: &write.item.labels,
4883 metadata: &write.item.metadata,
4884 repositories: &write.item.repositories,
4885 parent: write.item.project.as_ref(),
4886 delivers: &write.item.delivers,
4887 delivered_by: &write.item.delivered_by,
4888 },
4889 write.target.as_ref(),
4890 &write.depends_on,
4891 )
4892 .await
4893 }
4894
4895 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
4896 self.write_item(
4897 &Incoming {
4898 written: Written::Work(ItemKind::Project, &write.item.status),
4899 title: &write.item.title,
4900 content: write.item.content.as_deref(),
4901 labels: &write.item.labels,
4902 metadata: &write.item.metadata,
4903 repositories: &write.item.repositories,
4904 parent: None,
4905 delivers: &[],
4906 delivered_by: &[],
4907 },
4908 write.target.as_ref(),
4909 &write.depends_on,
4910 )
4911 .await
4912 }
4913
4914 /// Create or update one document, which is one issue titled the way this board spells
4915 /// a document.
4916 ///
4917 /// Everything else is exactly a task write: caller metadata goes to the same canonical
4918 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
4919 /// or a field this board cannot carry is refused by name rather than dropped, a target
4920 /// naming an issue this board does not hold is refused rather than created, and an
4921 /// issue this call created is taken back when the rest of the write fails.
4922 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
4923 // A document takes part in no dependency graph, so there is no far end to write
4924 // natively and none to record: a caller naming one is told so rather than having it
4925 // stored under the reserved key, where a later read would report an edge the
4926 // contract says cannot exist.
4927 if !write.depends_on.is_empty() {
4928 return Err(SourceError::Refused {
4929 message: format!(
4930 "this write names {} dependencies for a document, and a document takes \
4931 part in no dependency graph; next: put the dependency on the task or \
4932 project the document is about",
4933 write.depends_on.len()
4934 ),
4935 });
4936 }
4937 self.write_item(
4938 &Incoming {
4939 written: Written::Document,
4940 title: &write.item.title,
4941 content: write.item.content.as_deref(),
4942 labels: &write.item.labels,
4943 metadata: &write.item.metadata,
4944 repositories: &write.item.repositories,
4945 parent: write.item.project.as_ref(),
4946 delivers: &[],
4947 delivered_by: &[],
4948 },
4949 write.target.as_ref(),
4950 &[],
4951 )
4952 .await
4953 }
4954
4955 /// Set one task's status alone.
4956 ///
4957 /// A column target reopens a closed issue with an `updateIssue` carrying only its
4958 /// `stateInput`, then selects the board option with `updateProjectV2ItemFieldValue`; a
4959 /// closed target sends only that `updateIssue`, with the mapping's reason, and leaves
4960 /// the option where it is. No request carries a title, a body or a label. The status
4961 /// answered is what [`StatusMapping::status`] reads off the state just written, which is
4962 /// what a re-read reports.
4963 async fn set_task_status(
4964 &self,
4965 id: &NativeId,
4966 category: StatusCategory,
4967 ) -> Result<Option<Status>, SourceError> {
4968 self.set_status(id, category).await
4969 }
4970
4971 /// Replace one task's `delivered_by` with a single body update that changes the
4972 /// metadata slot and nothing outside it.
4973 async fn set_delivered_by(
4974 &self,
4975 id: &NativeId,
4976 delivered_by: &[TaskRef],
4977 ) -> Result<Option<()>, SourceError> {
4978 self.replace_delivered_by(id, delivered_by).await
4979 }
4980
4981 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
4982 self.delete_item(id).await
4983 }
4984
4985 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
4986 self.delete_item(id).await
4987 }
4988
4989 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
4990 self.delete_item(id).await
4991 }
4992
4993 /// One page of the task issue's own comments, walked by GitHub's own cursor.
4994 ///
4995 /// Nothing here filters, so nothing has to be read ahead of the page: the caller's limit is
4996 /// the page GitHub is asked for and GitHub's `endCursor` is the cursor handed back.
4997 async fn task_comments(
4998 &self,
4999 task: &NativeId,
5000 page: &PageRequest,
5001 ) -> Result<Option<Page<Comment>>, SourceError> {
5002 validate_page(page)?;
5003 let Some(issue) = self.commented_issue(task).await? else {
5004 return Ok(None);
5005 };
5006 let after = page.cursor.as_ref().map(|cursor| cursor.0.as_str());
5007 let data = self
5008 .graphql(
5009 graphql::ISSUE_COMMENTS,
5010 json!({"id":issue.0,"first":page.limit.min(MAX_PAGE_SIZE),"after":after}),
5011 )
5012 .await?;
5013 // The issue was there a moment ago; one removed since is no longer a task here.
5014 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
5015 return Ok(None);
5016 };
5017 let connection = node
5018 .get("comments")
5019 .filter(|value| !value.is_null())
5020 .ok_or_else(|| SourceError::Malformed {
5021 message: format!(
5022 "GitHub issue {} answered with no comments connection",
5023 issue.0
5024 ),
5025 })?;
5026 let items = optional_nodes(Some(connection), "issue comments")?
5027 .into_iter()
5028 .flatten()
5029 .map(comment_from)
5030 .collect::<Result<Vec<_>, _>>()?;
5031 let next = next_cursor(connection)?;
5032 if let Some(next) = &next {
5033 validate_cursor_progress(after, &next.0)?;
5034 }
5035 Ok(Some(Page { items, next }))
5036 }
5037
5038 /// Add one comment to the task's issue, as the account the token belongs to.
5039 ///
5040 /// The author is refused before anything is sent — not even the task is read — because
5041 /// no answer GitHub could give would make posting under another name than the one asked
5042 /// for the right outcome.
5043 async fn add_comment(
5044 &self,
5045 task: &NativeId,
5046 comment: &NewComment,
5047 ) -> Result<Option<Comment>, SourceError> {
5048 if let Some(author) = &comment.author {
5049 return Err(SourceError::Refused {
5050 message: format!(
5051 "source {} cannot post a comment as {author:?}: GitHub records the account \
5052 the token signs in as the author of every comment; next: leave --author \
5053 out, and the comment is posted as that account",
5054 self.name
5055 ),
5056 });
5057 }
5058 let Some(issue) = self.commented_issue(task).await? else {
5059 return Ok(None);
5060 };
5061 let data = self
5062 .graphql(
5063 graphql::ADD_COMMENT,
5064 json!({"input":{"subjectId":issue.0,"body":comment.body.as_str()}}),
5065 )
5066 .await?;
5067 let subject = data
5068 .pointer("/addComment/subject")
5069 .filter(|value| !value.is_null())
5070 .ok_or_else(|| SourceError::Malformed {
5071 message: "GitHub comment addition returned no subject".into(),
5072 })?;
5073 if required_str(subject, "id")? != issue.0 {
5074 return Err(SourceError::Malformed {
5075 message: "GitHub comment addition answered about another issue".into(),
5076 });
5077 }
5078 let added = data
5079 .pointer("/addComment/commentEdge/node")
5080 .filter(|value| !value.is_null())
5081 .ok_or_else(|| SourceError::Malformed {
5082 message: "GitHub comment addition returned no comment".into(),
5083 })?;
5084 comment_from(added).map(Some)
5085 }
5086
5087 async fn edit_comment(
5088 &self,
5089 task: &NativeId,
5090 comment: &NativeId,
5091 body: &CommentBody,
5092 ) -> Result<Option<Comment>, SourceError> {
5093 let Some(issue) = self.commented_issue(task).await? else {
5094 return Ok(None);
5095 };
5096 if !self.comment_is_on(&issue, comment).await? {
5097 return Ok(None);
5098 }
5099 let data = self
5100 .graphql(
5101 graphql::UPDATE_COMMENT,
5102 json!({"input":{"id":comment.0,"body":body.as_str()}}),
5103 )
5104 .await?;
5105 let edited = data
5106 .pointer("/updateIssueComment/issueComment")
5107 .filter(|value| !value.is_null())
5108 .ok_or_else(|| SourceError::Malformed {
5109 message: "GitHub comment update returned no comment".into(),
5110 })?;
5111 let edited = comment_from(edited)?;
5112 if edited.id != *comment {
5113 return Err(SourceError::Malformed {
5114 message: "GitHub comment update returned the wrong comment".into(),
5115 });
5116 }
5117 Ok(Some(edited))
5118 }
5119
5120 async fn delete_comment(
5121 &self,
5122 task: &NativeId,
5123 comment: &NativeId,
5124 ) -> Result<Option<NativeId>, SourceError> {
5125 let Some(issue) = self.commented_issue(task).await? else {
5126 return Ok(None);
5127 };
5128 if !self.comment_is_on(&issue, comment).await? {
5129 return Ok(None);
5130 }
5131 let data = self
5132 .graphql(graphql::DELETE_COMMENT, json!({"input":{"id":comment.0}}))
5133 .await?;
5134 // The payload says nothing about the comment it removed, so what is checked is that
5135 // GitHub answered the mutation at all rather than leaving it unanswered.
5136 data.get("deleteIssueComment")
5137 .filter(|value| !value.is_null())
5138 .ok_or_else(|| SourceError::Malformed {
5139 message: "GitHub comment deletion returned no payload".into(),
5140 })?;
5141 Ok(Some(comment.clone()))
5142 }
5143
5144 /// Every request this source has recorded, and what each of GitHub's two budgets was
5145 /// attributed — read off the same accounting the session report is rendered from, so
5146 /// the two cannot count one request two ways.
5147 async fn metering(&self) -> Result<Option<Metering>, SourceError> {
5148 Ok(Some(self.ledger.snapshot().metering()))
5149 }
5150}
5151
5152/// One issue comment as the contract carries it.
5153///
5154/// `author` is absent both when GitHub answers `null` for an account that no longer exists
5155/// and when it answers an actor with no login, because either way the source did not say who
5156/// wrote it — which is what an absent author means, rather than an author called nothing.
5157fn comment_from(value: &Value) -> Result<Comment, SourceError> {
5158 Ok(Comment {
5159 id: NativeId(required_str(value, "id")?.to_owned()),
5160 author: optional_str(value.get("author").unwrap_or(&Value::Null), "login")?
5161 .map(str::to_owned),
5162 created_at: optional_time(value, "createdAt")?,
5163 updated_at: optional_time(value, "updatedAt")?,
5164 body: required_str(value, "body")?.to_owned(),
5165 url: optional_str(value, "url")?.map(str::to_owned),
5166 })
5167}
5168
5169/// Where the recorded tail of a dependency walk resumes; see
5170/// [`GitHubProjectsSource::recorded_edges`].
5171const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
5172
5173/// The board text field this source keeps a copy's origin in.
5174///
5175/// Named after the key it holds, and held to that name by the guard below rather than by
5176/// a reader noticing.
5177const ORIGIN_FIELD: &str = "onetaskgraph.origin";
5178
5179/// The metadata key that field holds.
5180///
5181/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
5182/// constructs or interprets the qualified id it carries. This source names it only to
5183/// route it — a short, typed value belongs in a typed field rather than in the body slot
5184/// a caller's own prose shares.
5185///
5186/// Restated rather than imported, because no plugin crate may depend on the engine. What
5187/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
5188/// target in `check`: it reads the engine's own literal and fails naming the file and the
5189/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
5190/// that creates a second item every run instead of finding the one it wrote — and that is
5191/// too late to learn it.
5192const ORIGIN_KEY: &str = "onetaskgraph.origin";
5193
5194/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
5195///
5196/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
5197/// is derived from the far end, never written down on the near item — so only a forward
5198/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
5199/// it did not come from, and it is told so rather than answered with an empty page that
5200/// reads as a walk which ended.
5201fn recorded_offset(
5202 cursor: Option<&str>,
5203 direction: Direction,
5204) -> Result<Option<usize>, SourceError> {
5205 cursor
5206 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
5207 .map(|offset| {
5208 if direction != Direction::DependsOn {
5209 return Err(SourceError::Config {
5210 message: format!(
5211 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
5212 reverse dependency read never issues; resume it in the direction \
5213 that reported it"
5214 ),
5215 });
5216 }
5217 offset.parse().map_err(|_| SourceError::Config {
5218 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
5219 })
5220 })
5221 .transpose()
5222}
5223
5224fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
5225 let mut page = offset_page(edges, offset, limit.max(1));
5226 page.next = page
5227 .next
5228 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
5229 page
5230}
5231
5232/// The kind of one issue reached through a dependency connection.
5233///
5234/// The same questions the board scan asks, over the fields the dependency document
5235/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
5236/// then anything with sub-issues or the marker is a project.
5237///
5238/// # Errors
5239///
5240/// A far end this board holds as a document is refused rather than reported. The two
5241/// answers that are not refusals would both be wrong: reporting it as a task names an id
5242/// no task read of this source can find, and reporting it as a project names one no
5243/// project read can. There is no third value to return — `ItemKind` has no document
5244/// variant, because nothing may point at a document — so the relationship itself is what
5245/// the person is told about.
5246fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
5247 let id = required_str(value, "id")?;
5248 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
5249 return Err(SourceError::Refused {
5250 message: format!(
5251 "GitHub issue {id} is a document of this board — its title begins \
5252 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
5253 on by one; next: remove that issue's blocking relationship on this board"
5254 ),
5255 });
5256 }
5257 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
5258 if parent.is_some() {
5259 return Ok(ItemKind::Task);
5260 }
5261 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
5262 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
5263 message: format!("GitHub issue {id}: {message}"),
5264 })?;
5265 let sub_issues = sub_issue_total(value)?;
5266 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
5267 ItemKind::Project
5268 } else {
5269 ItemKind::Task
5270 })
5271}
5272
5273/// The `IssueStateUpdateInput` one status target asks for.
5274///
5275/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
5276/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
5277/// a currently-closed issue: without that the item would read back `Unknown` and a copy
5278/// would report a change forever. A document has no status at all, and asks for neither.
5279fn state_input(target: Option<&StatusTarget>) -> Value {
5280 match target {
5281 Some(StatusTarget::Closed(reason)) => {
5282 json!({"value":"CLOSED","stateReason":reason.reason()})
5283 }
5284 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
5285 // A document has no status, so a write of one says nothing about the issue's open
5286 // or closed state rather than forcing it open: `stateInput` is what carries that
5287 // instruction, and an explicit null asks for no change to it.
5288 None => Value::Null,
5289 }
5290}
5291
5292/// The metadata one write stores in the item's body slot.
5293///
5294/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
5295/// rather than carried: the kind marker so an empty project stays readable, the
5296/// repository list only when it is not exactly the issue's own repository, and the far
5297/// ends no relationship here can name.
5298fn slot_metadata(
5299 incoming: &Incoming<'_>,
5300 own_repository: Option<&Repository>,
5301 fallback: &[DependencyEdge],
5302) -> BTreeMap<String, Value> {
5303 let mut metadata = incoming.metadata.clone();
5304 metadata.remove(ORIGIN_KEY);
5305 match incoming.written.kind() {
5306 BoardKind::Work(kind) => metadata.insert(
5307 ItemKind::METADATA_KEY.to_owned(),
5308 Value::String(kind.marker().to_owned()),
5309 ),
5310 // A document is told by its title, so it carries no kind marker: that key names
5311 // what a dependency endpoint points at, and nothing may point at a document.
5312 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
5313 };
5314 let derivable = own_repository
5315 .map(|own| incoming.repositories == [own.clone()])
5316 .unwrap_or(incoming.repositories.is_empty());
5317 if derivable {
5318 metadata.remove(Repository::METADATA_KEY);
5319 } else {
5320 metadata.insert(
5321 Repository::METADATA_KEY.to_owned(),
5322 Value::Array(
5323 incoming
5324 .repositories
5325 .iter()
5326 .map(|repository| Value::String(repository.as_str().to_owned()))
5327 .collect(),
5328 ),
5329 );
5330 }
5331 // The typed lists are what land, whatever the caller's own metadata held under their
5332 // keys: a key of either name travelling beside the field would otherwise be a second
5333 // answer to the same question, and the field is the one the contract names.
5334 for (key, entries) in [
5335 (TaskRef::DELIVERS_KEY, incoming.delivers),
5336 (TaskRef::DELIVERED_BY_KEY, incoming.delivered_by),
5337 ] {
5338 set_task_list(&mut metadata, key, entries);
5339 }
5340 if fallback.is_empty() {
5341 metadata.remove(DependencyEdge::RECORDED_KEY);
5342 } else {
5343 metadata.insert(
5344 DependencyEdge::RECORDED_KEY.to_owned(),
5345 Value::Array(
5346 fallback
5347 .iter()
5348 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
5349 .collect(),
5350 ),
5351 );
5352 }
5353 metadata
5354}
5355
5356/// Every label one item carries, from its content's own connection and nowhere else.
5357///
5358/// There is no second place to read one from: no document this source sends selects the
5359/// board's built-in `Labels` field, because GitHub derives it from the content and a draft
5360/// cannot carry one at all. The module documentation records the three schema facts that
5361/// settle it.
5362fn labels(content: &Value) -> Result<Vec<Label>, SourceError> {
5363 optional_nodes(content.get("labels"), "content labels")?
5364 .into_iter()
5365 .flatten()
5366 .map(|v| {
5367 Ok(Label {
5368 id: NativeId(required_str(v, "id")?.to_owned()),
5369 name: required_str(v, "name")?.to_owned(),
5370 color: optional_str(v, "color")?.map(str::to_owned),
5371 })
5372 })
5373 .collect()
5374}
5375
5376/// The definition of each board field one item's values are values of, in the shape a read
5377/// of the board's own `fields` gives one.
5378///
5379/// A value names its field through a fragment on that field's own type, so the type is
5380/// known from which kind of value it is: a single-select value's field is a
5381/// `ProjectV2SingleSelectField`, options and all, and a text value's is a `ProjectV2Field`.
5382/// A value whose field carried no id, or an empty one, says nothing usable and is left out.
5383fn field_definitions(field_values: &[Value]) -> Vec<Value> {
5384 field_values
5385 .iter()
5386 .filter_map(|value| {
5387 let field = value.get("field")?.as_object()?;
5388 field.get("id")?.as_str().filter(|id| !id.is_empty())?;
5389 let typename = if value.get("text").is_some() {
5390 "ProjectV2Field"
5391 } else if value.get("name").is_some() {
5392 "ProjectV2SingleSelectField"
5393 } else {
5394 return None;
5395 };
5396 let mut defined = field.clone();
5397 defined.insert("__typename".to_owned(), json!(typename));
5398 Some(Value::Object(defined))
5399 })
5400 .collect()
5401}
5402
5403fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
5404 let Some(node) = field_values
5405 .iter()
5406 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
5407 else {
5408 return Ok(None);
5409 };
5410 Ok(optional_str(node, "text")?.map(str::to_owned))
5411}
5412
5413fn valid_github_owner(owner: &str) -> bool {
5414 !owner.is_empty()
5415 && owner.len() <= 39
5416 && !owner.starts_with('-')
5417 && !owner.ends_with('-')
5418 && !owner.contains("--")
5419 && owner
5420 .bytes()
5421 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
5422}
5423
5424/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
5425/// neither of the two names a path segment already means.
5426fn valid_github_repository_name(name: &str) -> bool {
5427 !name.is_empty()
5428 && name.len() <= 100
5429 && name != "."
5430 && name != ".."
5431 && name
5432 .bytes()
5433 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
5434}
5435
5436fn valid_environment_name(name: &str) -> bool {
5437 let mut bytes = name.bytes();
5438 bytes
5439 .next()
5440 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
5441 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
5442}
5443
5444/// How many sub-issues one issue has.
5445///
5446/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
5447/// absent or non-integer one is a response this source cannot read — and reading it as
5448/// zero would classify a project as a task, which is exactly the mistake the marker
5449/// exists to keep from happening quietly.
5450fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
5451 let summary = issue
5452 .get("subIssuesSummary")
5453 .ok_or_else(|| SourceError::Malformed {
5454 message: "GitHub issue is missing subIssuesSummary".into(),
5455 })?;
5456 summary
5457 .get("total")
5458 .and_then(Value::as_u64)
5459 .ok_or_else(|| SourceError::Malformed {
5460 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
5461 })
5462}
5463
5464fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
5465 value
5466 .get(field)
5467 .and_then(Value::as_str)
5468 .ok_or_else(|| SourceError::Malformed {
5469 message: format!("GitHub response is missing string field {field}"),
5470 })
5471}
5472
5473/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
5474/// needs one — Linear spells them too, in its own description field.
5475///
5476/// Restated rather than shared, because a plugin crate depends on the contract crate and
5477/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
5478/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
5479/// source round-trips its own writes perfectly well under its own spelling.
5480const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
5481const METADATA_CLOSE: &str = "\n-->";
5482
5483/// The visible body and the metadata slot at the end of it.
5484///
5485/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
5486/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
5487/// own content and is left alone.
5488fn metadata_body(
5489 body: Option<String>,
5490) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
5491 let Some(body) = body else {
5492 return Ok((None, BTreeMap::new()));
5493 };
5494 let Some(slot) = slot_span(&body)? else {
5495 return Ok((Some(body), BTreeMap::new()));
5496 };
5497 let metadata =
5498 serde_json::from_str(&body[slot.encoded_start..slot.encoded_end]).map_err(|error| {
5499 SourceError::Malformed {
5500 message: format!(
5501 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
5502 ),
5503 }
5504 })?;
5505 let visible = body[..slot.start].trim_end();
5506 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
5507}
5508
5509/// Where the metadata slot sits in one body, as byte offsets into it.
5510struct SlotSpan {
5511 /// Where [`METADATA_OPEN`] begins.
5512 start: usize,
5513 /// Where the encoded JSON begins, just past [`METADATA_OPEN`].
5514 encoded_start: usize,
5515 /// Where the encoded JSON ends, at the start of [`METADATA_CLOSE`].
5516 encoded_end: usize,
5517 /// Just past [`METADATA_CLOSE`].
5518 end: usize,
5519}
5520
5521/// The slot at the very end of `body`, or `None` when it has none.
5522///
5523/// The one reading of *where the slot is*, shared by [`metadata_body`], which reads it, and
5524/// [`with_slot`], which rewrites it — so the two cannot disagree about which comment is the
5525/// slot.
5526fn slot_span(body: &str) -> Result<Option<SlotSpan>, SourceError> {
5527 let Some(start) = body.rfind(METADATA_OPEN) else {
5528 return Ok(None);
5529 };
5530 let encoded_start = start + METADATA_OPEN.len();
5531 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
5532 return Err(SourceError::Malformed {
5533 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
5534 });
5535 };
5536 let encoded_end = encoded_start + relative_end;
5537 let end = encoded_end + METADATA_CLOSE.len();
5538 if !body[end..].trim().is_empty() {
5539 return Ok(None);
5540 }
5541 Ok(Some(SlotSpan {
5542 start,
5543 encoded_start,
5544 encoded_end,
5545 end,
5546 }))
5547}
5548
5549/// `body` with its metadata slot holding exactly `metadata`, and every byte outside the
5550/// slot as it was.
5551///
5552/// A slot that is there has its JSON replaced in place; one that becomes empty is removed
5553/// together with the one `"\n\n"` separating it from the prose before it. A body with no
5554/// slot gains one the way [`compose_body`] writes it — after a `"\n\n"`, or alone in an empty
5555/// body — and a body with no slot that is given no metadata is returned as it is.
5556fn with_slot(body: &str, metadata: &BTreeMap<String, Value>) -> Result<String, SourceError> {
5557 let encoded = if metadata.is_empty() {
5558 None
5559 } else {
5560 Some(
5561 serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
5562 message: error.to_string(),
5563 })?,
5564 )
5565 };
5566 Ok(match (slot_span(body)?, encoded) {
5567 (Some(slot), Some(encoded)) => format!(
5568 "{}{encoded}{}",
5569 &body[..slot.encoded_start],
5570 &body[slot.encoded_end..]
5571 ),
5572 (Some(slot), None) => {
5573 let before = &body[..slot.start];
5574 format!(
5575 "{}{}",
5576 before.strip_suffix("\n\n").unwrap_or(before),
5577 &body[slot.end..]
5578 )
5579 }
5580 (None, None) => body.to_owned(),
5581 (None, Some(encoded)) if body.is_empty() => {
5582 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
5583 }
5584 (None, Some(encoded)) => format!("{body}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}"),
5585 })
5586}
5587
5588/// Hold `entries` under `key` in one slot's metadata, or no such key when there are none.
5589fn set_task_list(metadata: &mut BTreeMap<String, Value>, key: &str, entries: &[TaskRef]) {
5590 if entries.is_empty() {
5591 metadata.remove(key);
5592 } else {
5593 metadata.insert(
5594 key.to_owned(),
5595 Value::Array(
5596 entries
5597 .iter()
5598 .map(|entry| Value::String(entry.as_str().to_owned()))
5599 .collect(),
5600 ),
5601 );
5602 }
5603}
5604
5605fn compose_body(
5606 content: Option<&str>,
5607 metadata: &BTreeMap<String, Value>,
5608) -> Result<Option<String>, SourceError> {
5609 let visible = content.unwrap_or_default();
5610 if metadata.is_empty() {
5611 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
5612 }
5613 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
5614 message: error.to_string(),
5615 })?;
5616 Ok(Some(if visible.is_empty() {
5617 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
5618 } else {
5619 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
5620 }))
5621}
5622
5623fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
5624 value
5625 .get(field)
5626 .and_then(Value::as_bool)
5627 .ok_or_else(|| SourceError::Malformed {
5628 message: format!("GitHub response is missing boolean field {field}"),
5629 })
5630}
5631fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
5632 match value.get(field) {
5633 None | Some(Value::Null) => Ok(None),
5634 Some(value) => value
5635 .as_str()
5636 .map(Some)
5637 .ok_or_else(|| SourceError::Malformed {
5638 message: format!("GitHub response field {field} is not a string or null"),
5639 }),
5640 }
5641}
5642fn optional_nodes<'a>(
5643 connection: Option<&'a Value>,
5644 name: &str,
5645) -> Result<Option<&'a Vec<Value>>, SourceError> {
5646 match connection {
5647 None | Some(Value::Null) => Ok(None),
5648 Some(value) => value
5649 .get("nodes")
5650 .and_then(Value::as_array)
5651 .map(Some)
5652 .ok_or_else(|| SourceError::Malformed {
5653 message: format!("GitHub {name}.nodes is not an array"),
5654 }),
5655 }
5656}
5657fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
5658 let page_info = connection
5659 .get("pageInfo")
5660 .ok_or_else(|| SourceError::Malformed {
5661 message: format!("GitHub {name} has no pageInfo"),
5662 })?;
5663 if required_bool(page_info, "hasNextPage")? {
5664 return Err(SourceError::Malformed {
5665 message: format!(
5666 "GitHub {name} exceeds the supported nested connection size of {size}"
5667 ),
5668 });
5669 }
5670 Ok(())
5671}
5672fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
5673 optional_str(value, field)?
5674 .map(|timestamp| {
5675 timestamp.parse().map_err(|error| SourceError::Malformed {
5676 message: format!("GitHub response field {field} is not a timestamp: {error}"),
5677 })
5678 })
5679 .transpose()
5680}
5681fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
5682 if page.limit == 0 {
5683 Err(SourceError::Config {
5684 message: "page limit must be at least 1".into(),
5685 })
5686 } else {
5687 Ok(())
5688 }
5689}
5690fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
5691 let page = connection
5692 .get("pageInfo")
5693 .filter(|value| value.is_object())
5694 .ok_or_else(|| SourceError::Malformed {
5695 message: "GitHub connection is missing pageInfo".into(),
5696 })?;
5697 if required_bool(page, "hasNextPage")? {
5698 let cursor = required_str(page, "endCursor")?;
5699 validate_cursor_progress(None, cursor)?;
5700 Ok(Some(Cursor(cursor.into())))
5701 } else {
5702 Ok(None)
5703 }
5704}
5705fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
5706 if next.is_empty() || previous == Some(next) {
5707 Err(SourceError::Malformed {
5708 message: "GitHub pagination cursor is empty or did not advance".into(),
5709 })
5710 } else {
5711 Ok(())
5712 }
5713}
5714fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
5715 cursor.map_or(Ok(0), |c| {
5716 c.0.parse().map_err(|_| SourceError::Config {
5717 message: "page cursor is invalid".into(),
5718 })
5719 })
5720}
5721fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
5722 if offset > items.len() {
5723 return Page::last(vec![]);
5724 }
5725 let tail = items.split_off(offset);
5726 let mut selected = tail;
5727 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
5728 selected.truncate(limit);
5729 Page {
5730 items: selected,
5731 next,
5732 }
5733}