onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so a write without [`GitHubProjectsConfig::repository`] is refused
11//! naming the field — but that repository is the *fallback*, not the home of every item.
12//!
13//! <!-- llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] The rule's one
14//! executable source is `GitHubProjectsSource::creation_target`; this is where a reader of
15//! the module meets it, and `tests/plugin.rs` drives every arm below against the loopback
16//! board and asserts on `createIssue`'s own `repositoryId`, so the prose cannot outlive a
17//! change to the rule. -->
18//! **Which repository an issue is created in is decided by the item's own `repositories`
19//! field, under one rule.** Exactly one entry names the repository the issue is created in:
20//! a task issue is where a person finds the work from the repository it changes, and one
21//! filed in a board's nominated repository is invisible from every other. Zero entries, or
22//! two or more, name none, so a task's or a document's issue is created in the repository
23//! its parent project's issue lives in — read from the board, or from this process's own
24//! record of a project it created earlier in the same command — and a project's issue, or
25//! a task or document written with no parent, is created in the configured `repository:`.
26//! What that rule refuses, it refuses before `createIssue`, so no issue is half-created. An
27//! existing issue is never moved: the update path leaves the issue where it is and records
28//! the list in the metadata slot when it differs, so the read side's derivation and the
29//! creation rule agree by construction.
30//! <!-- llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate] -->
31//!
32//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
33//! board has no document type and nothing but issues to hold one in, so the title is the
34//! discriminator and it is the whole of it. The title this source *reports* is the one a
35//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
36//! the body so `content` is what the person wrote — and writing a document puts the prefix
37//! back, so a round trip returns the title that went in.
38//!
39//! **Telling a document from a project from a task.** The design prefix is read **first**:
40//! a document is never a project and never a task, whatever sub-issues it has or does not
41//! have. Only then does the rest apply — a board issue is a project when *either* it has
42//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
43//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
44//! necessary: it is what makes an *empty* project — the state a project copy passes
45//! through between creating the project and filing its first task — readable as a
46//! project, while the sub-issue arm lets a person author a project on the board by hand
47//! with no knowledge of this product's metadata at all. Reading the prefix later than the
48//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
49//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
50//! task nor a document and are ignored.
51//!
52//! **A task's comments are its issue's comments.** They are read off `Issue.comments` and
53//! written with `addComment`, `updateIssueComment` and `deleteIssueComment`, and a comment's
54//! id is GitHub's own node id for the `IssueComment`. Two things GitHub decides are refused
55//! rather than papered over: a board **draft** is not an issue and has no comments at all, so
56//! a comment call on one is refused rather than answered with an empty page; and GitHub signs
57//! every comment as the account the token belongs to, so a comment handed an author of its
58//! own is refused rather than posted under another name. GitHub's comment mutations take the
59//! comment's id and nothing else, so an edit or a delete first reads which issue that comment
60//! is on, and a comment on some other issue is one this task does not have.
61//!
62//! **Where an entity is, is a link.** Every project, task and document this source reports
63//! carries a [`Location::Url`] naming the issue's own web address — the same address the
64//! `url` field already reports, in the shape that says a reader can open it. That is the
65//! contrast the location contract exists for: a reader holding an entity from this source
66//! is handed something to link to and one holding an entity from a folder of Markdown is
67//! handed a path, and neither has to know which plugin answered. It does not replace or
68//! derive from `url`; that field goes on reporting what it always reported.
69//!
70//! **Where metadata lives.** Short typed things go to typed fields and native relations:
71//! status to the board's `Status` single-select and the issue's own state, the copy
72//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
73//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
74//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
75//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
76//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
77//! why neither can hold a caller's own prose.
78//!
79//! **Status.** `status_mapping` is per-instance configuration from a status category to
80//! `null`, a board `Status` option name, or a closed state of `completed` or
81//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
82//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
83//! and a mistake destroys every item's status. A status this board cannot represent is a
84//! refusal naming the status and the instance instead.
85//!
86//! `unknown` is disabled by default because this source cannot preserve an open-ended
87//! status word: it writes an existing board option or the issue's closed state and never
88//! creates an option. An operator may map `unknown` to one existing option, in which case
89//! every unknown word lands on that option and reads back as `unknown` under the option's
90//! name. Mapping it to a closed state is refused because GitHub reads that state back as
91//! `done` or `cancelled`, changing the category on every copy. This differs from
92//! `local-md`, which writes and reads the original word itself.
93//!
94//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
95//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
96//! finished tasks were only moved to a "Done" column would read 0% complete forever.
97//!
98//! # What this source declares, field by field
99//!
100//! One verdict per field of [`Capabilities`], and what `Native` means when this source
101//! says it. *Proven* means a shared journey drives it against the real
102//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
103//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
104//! [`capabilities`](TaskSource::capabilities) from parting.
105//!
106//! | Field | Verdict |
107//! | --- | --- |
108//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
109//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
110//! | `comments` | **Supported and proven,** over the task issue's own comment connection, oldest first and paged by GitHub's own cursor; added, edited and removed through GitHub's comment mutations, paced as every other mutation is. A draft item has no comments on GitHub and is refused, and so is an author, because GitHub records the signed-in account as every comment's author. |
111//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
112//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
113//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
114//! | `search_title` | **Supported and proven,** over `Issue.title`. |
115//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
116//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
117//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
118//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
119//!
120//! Nothing here is unsupported. `documents` and `comments` are not predicates — they say this
121//! source has documents and that its tasks have comments, both of which hold — and the three
122//! facts behind the uniform `Native` on the
123//! predicates beside it are recorded below rather than re-derived, because a reader who
124//! takes `Native` to mean *the remote service filters* will read that uniformity as a
125//! lie.
126//!
127//! First, the plugin contract defines `Support::Native` as *the source applies this
128//! predicate itself*, and says nothing about where it applies it. What the declaration
129//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
130//! — so that the engine may push it down and apply nothing of its own.
131//!
132//! Second, this source can keep that promise for every predicate at no additional API
133//! cost, because whichever of the three reads below answers a query has already read every
134//! item that query could keep before it filters anything. Filtering those items is
135//! in-process work over data already in hand.
136//!
137//! Third, no predicate but `projects` could be pushed into the API even if that were
138//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
139//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
140//! label set, a status column or a substring of a body, and its title qualifier matches
141//! tokens where this source — and the local Markdown source beside it — match substrings,
142//! so pushing a search down would silently *narrow* the answer. What a project filter has
143//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
144//! the issue for them is both cheaper and exact. So there is one predicate this source
145//! applies by asking a narrower question, six it applies in process, and none it is unable
146//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
147//! source has already done, and declaring `projects` native while ignoring the filter
148//! (which this source once did) silently returns another project's tasks, because the
149//! engine trusts the declaration and applies nothing locally.
150//!
151//! # The three ways this source reaches an item, and what each costs
152//!
153//! A board read is charged for what its *nested* connections could return rather than for
154//! what was asked, so one whole-board read costs the same whether the question was about
155//! one project or about all of them. That is why a question about one project is never
156//! answered by reading the board:
157//!
158//! | The question | What is sent | What it costs |
159//! | --- | --- | --- |
160//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
161//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
162//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
163//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
164//! | which board item one issue is, past the page that came with it | [`graphql::ISSUE_BOARD_ITEMS`] — that issue's own `projectItems` | one issue's memberships |
165//!
166//! The board half of an issue — its board item's id, its `Status` option and this
167//! source's origin text field — rides along on `Issue.projectItems` in the first three, so
168//! an item reached any of those ways resolves through the same
169//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
170//! same status, the same labels and the same qualified id. That connection comes back a
171//! *page* at a time, at `BOARD_ITEMS_PAGE_SIZE`, so the entry for this board is looked for
172//! on the page in hand and — only if that page reports more of the connection — in the
173//! last row's read of that one issue's memberships, resumed from the page's own cursor and
174//! walked to exhaustion. An issue with no entry for *this* board is not this source's to
175//! report, which is what keeps an id naming another repository's issue from being answered
176//! as an item of this board; and because the page is where the search starts rather than
177//! where it ends, that answer is one about a connection read to exhaustion and never about
178//! an unread page. Nothing costs the extra read but an issue on more boards than a page
179//! holds: an issue this board really does not hold reports no next page, so its
180//! memberships are already exhausted where they arrived.
181//!
182//! **No document here selects the board's own `Labels` field, and nothing is lost by
183//! that.** An item's labels are read from its content alone, wherever that content is
184//! reached: the three documents above select `Issue.labels` on the fragment, and
185//! [`graphql::BOARD`] selects the same connection on the `... on Issue` arm of its
186//! `content`. A board's `Labels` field is not one anybody fills in: it is a built-in
187//! `ProjectV2FieldType`, it is absent from `ProjectV2CustomFieldType` so no project can
188//! create one, and `ProjectV2FieldValue` — the whole of what
189//! `updateProjectV2ItemFieldValue` accepts — offers no way to write one. So GitHub derives
190//! it from the content, for every content type it exists on, and there is nothing it can
191//! hold that the content does not already say: for an `Issue` it *is* that issue's own
192//! labels, so selecting it beside them unions a set with itself.
193//!
194//! **A draft loses nothing by that either**, which is the reasoning this paragraph once had
195//! backwards. `DraftIssue` exposes no `labels` field, and by the three schema facts above
196//! it cannot carry a board `Labels` value to be derived from one — so a draft has nothing
197//! to select *and nothing to lose*, and reports no labels at all. A `PullRequest` item is
198//! discarded by [`GitHubProjectsSource::resolve`] before labels are read. Both halves are
199//! held to that by tests in `tests/plugin.rs`: the four ways an item is reached report one
200//! label set, and that set is the fixture issue's own, by
201//! `an_item_reports_the_same_labels_title_status_and_id_however_it_is_reached`; and a board
202//! item whose content is a draft reports an empty set, by
203//! `a_board_item_whose_content_is_a_draft_reports_no_labels_at_all`. The absence of the
204//! selection is held over [`graphql::DOCUMENTS`] by
205//! `no_document_selects_the_boards_own_labels_field`.
206//!
207//! The last row is still the board's own item connection, and deliberately: a **draft**
208//! board item is not an issue, so no search and no node read can reach one, and the reads
209//! that have to answer for the whole board are the ones whose cost is the board's size
210//! anyway.
211//!
212//! **What a read may return is capped too, and that cap is on the document rather than on
213//! the board.** GitHub limits the number of nodes **one query may return** to
214//! [`NODE_COUNT_LIMIT`] and refuses a query above that before executing it: the answer is
215//! an error naming the connection the count crossed at, not a slow or a partial result.
216//! Every board this source reads is refused the same way, so no board is too big for these
217//! documents and none is small enough to save one that is over.
218//!
219//! The count is arithmetic over the document's own text: each connection contributes the
220//! `first:` it asks for, counts **multiply** down a nested path and **sum** across sibling
221//! paths. Those are [GitHub's published rules][node-limits] and this workspace does not
222//! restate them — `github-graphql-node-count` implements them, and
223//! [`worst_case_node_count`] under [`largest_page_sizes`] is where every node count here
224//! comes from. `every_document_this_source_sends_stays_under_githubs_node_limit`, in
225//! `tests/node_count.rs`, recomputes every document in [`graphql::DOCUMENTS`] from that
226//! same text on every run and fails naming any that reaches the limit — so a connection
227//! added to a shared fragment is caught there rather than by GitHub.
228//!
229//! What decides those counts is the page sizes: [`MAX_PAGE_SIZE`] on the outer page,
230//! `NESTED_PAGE_SIZE` on the connections hanging off one item, and
231//! `BOARD_ITEMS_PAGE_SIZE` on the page of an issue's board memberships a read carries.
232//! `$nestedFirst` is spent twice down one path of a board read, so that constant is
233//! effectively squared there, which is why it is the one the limit is most sensitive to.
234//! `BOARD_ITEMS_PAGE_SIZE` is small for a reason of its own, recorded beside it: what a
235//! page of memberships misses is recovered by one further read rather than refused, so it
236//! buys a bound every read pays for at the price of a request only a multi-board issue
237//! pays.
238//!
239//! **`nodeCount` and `cost` are two numbers against two limits, and both are computed
240//! offline here — per document, one document at a time.** `nodeCount` is the one above: the
241//! most nodes one query may return, checked per query and bounded by [`NODE_COUNT_LIMIT`].
242//! `cost` is rate-limit points, metered per hour across everything one credential does; it
243//! is what the two limiters [`Limiter`] tells apart meter, and a document under
244//! [`NODE_COUNT_LIMIT`] still says nothing about its price. [`worst_case_point_cost`] is
245//! that second number, and `tests/point_cost.rs` pins every document in
246//! [`graphql::DOCUMENTS`] at what it costs — there being no per-call point ceiling to hold
247//! one under, the pin itself is the check. The credentialed lane reconciles both figures
248//! against GitHub's own, off a probe it already sends.
249//!
250//! **What is pinned that way is a per-document price and never a session's.** The record in
251//! `session-cost.md` measures the two quantities a whole session can be counted in offline —
252//! **requests** and **worst-case nodes** — and neither is points. What one whole session
253//! consumes of the hourly point allowance is observable only from a credentialed run's own
254//! `x-ratelimit-*` headers, which is what [`accounting`] fills its per-budget figures from
255//! and what `tests/live.rs` prints at the end of every run.
256//!
257//! [node-limits]: https://docs.github.com/en/graphql/overview/rate-limits-and-node-limits-for-the-graphql-api
258//!
259//! **Where a read-after-write guarantee comes from, since a search index cannot supply
260//! one.** GitHub's issue search is eventually consistent and answers a write made moments
261//! ago with the value from before it. Resolving a node id is not, so a read by id and a
262//! project's own sub-issues are already current. What closes the gap for the search is
263//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
264//! what this process itself wrote, so an item created seconds ago is reported whether or
265//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
266//! and the record dies with the process.
267//!
268//! Filtering happens before paging, so a page of a filtered result is a page of the
269//! survivors rather than the survivors of a page. Label and text matching answer the same
270//! question the same way the local Markdown source's do, so one cross-source expectation
271//! holds for both.
272//!
273//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
274//! has one source, `capabilities`, and the note above is the reasoning behind it rather
275//! than a second copy of it: without the three facts recorded here a reader takes the
276//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
277//! crate's own capabilities test, which pins every field of it against a fully spelled-out
278//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
279//! fails to compile there rather than going unasserted. -->
280//! The fixture-server tests above run wherever this crate is selected; the credentialed
281//! lane runs in the same required check, beside them, and can fail it — it verifies the
282//! current schema, then drives every field of the table above against the real board. It builds its own fixture there — two projects, one task filed under each,
283//! one filed under neither, a label on one of the three and a closed status on another —
284//! because that shape is what tells an honoured predicate from an ignored one: a board
285//! holding a single project answers a project filter the same way whether or not this
286//! source applies it, which is exactly how the defect above went unseen.
287//!
288//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
289//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
290//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
291//! nominated is what keeps a credentialed write lane off a board and a repository nobody
292//! nominated; it never asks GitHub which project was updated most recently. Before it
293//! starts, the lane also clears any item titled — and any repository label named — the way
294//! it titles and names its own artifacts, which is self-healing after an interrupted run:
295//! a process killed between its writes and its cleanup leaves artifacts the next run
296//! removes.
297//!
298//! # What a session of requests costs, and where the report is
299//!
300//! This source records **every** request it sends into [`accounting::Accounting`], at
301//! `send_once` — the one place a request leaves this crate, which is why a read path added
302//! later is counted without anybody remembering to count it. That is the whole of what this
303//! crate adds to the arrangement; [`accounting`] is where what a record carries, how a
304//! session's spend is arrived at, and what it deliberately does not know are set out.
305//!
306//! What one whole session of the live journey costs, counted that way against this crate's
307//! loopback fixture board, is written down in `session-cost.md` beside this crate — with the
308//! reduction it came out of, and with what it does and does not say about rate-limit points.
309//!
310//! [`GitHubProjectsSource::accounting`] is the read: a snapshot to hold and compare, which
311//! [`accounting::Session::report`] renders the session report from. It is on the ordinary
312//! code path — no environment variable, no feature, no build configuration — because an
313//! instrument nobody switches on measures nothing, and
314//! [`Plugin::build_recording_into`] is how a caller making its own calls beside this
315//! source's counts the whole session rather than this source's share. The credentialed lane
316//! in `tests/live.rs` does exactly that, and prints the report at the end of every run,
317//! passed or failed.
318//!
319//! **A live session refuses to start unless the account can afford it.** Before it does any
320//! of the work it exists to do, the journey makes one request — `GET /rate_limit`, which
321//! GitHub documents as not counting against the REST rate limit and which answers both of
322//! its budgets at once — and starts only if, for each of them, what remains minus this
323//! session's estimated cost is still at least
324//! `onetaskgraph_live::RETAINED_BUFFER` — twenty per cent — of that budget's whole
325//! allowance. A session that cannot **declines**: it did not run, so it is
326//! neither a pass nor a failing assertion, and it says which budget was short, that budget's
327//! limit, what remained, the estimate, the buffer and when it resets — then stops, without
328//! waiting for the budget to come back. The estimate is derived offline from
329//! `tests/fixtures/session-cost.txt` and a cost model stated in `tests/journey/budget.rs`,
330//! which is also where the published rule that model rests on is cited; the accounting
331//! above records the gate's own read like any other request, and
332//! [`accounting::Session::report`] prints the estimate beside what the session really spent.
333//!
334//! **GitHub is the authority on both of its own numbers, and the credentialed lane goes and
335//! asks it.** Everything above computes `nodeCount` and `cost` offline from a document's own
336//! text, which is what lets it run on every platform and on a pull request from a fork with
337//! no credential — and that is what actually stops a regression merging. But an offline
338//! arithmetic can only ever agree with itself: if GitHub changes its rules, this workspace
339//! goes on computing the old answer and nothing notices. So `tests/live.rs` reconciles them.
340//! GitHub's schema exposes `rateLimit(dryRun: true)`, whose `nodeCount` is *"the maximum
341//! number of nodes this query may return"* and whose `cost` is what that document would
342//! spend, both for a document **without executing it**, and the lane asks it for every query
343//! document this source sends, under the largest bindings this source sends, and fails when
344//! GitHub's figure and [`worst_case_node_count`] or [`worst_case_point_cost`] disagree. A
345//! mutation is skipped, because `rateLimit` is a field of `Query` and cannot be asked about
346//! one; the offline pins still cover it. It records what those calls reported about the
347//! account's own allowance, because whether asking is free is a thing to observe rather than
348//! to assume. Two quantities, not one: [`NODE_COUNT_LIMIT`] bounds `nodeCount` per query,
349//! and `cost` is metered against an hourly allowance the accounting above reads off a
350//! credentialed run's own response headers.
351//!
352//! **GitHub has two rate limiters and this source is refused by both, so nothing here
353//! treats them as one thing.** The primary budget is the hourly allowance `gh api
354//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
355//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
356//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
357//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
358//! one part of.
359#![deny(missing_docs)]
360
361use std::collections::BTreeMap;
362use std::sync::{Arc, Mutex};
363use std::time::{Duration, Instant};
364
365use chrono::{DateTime, Utc};
366use onetaskgraph_plugin_api::{
367 Capabilities, Comment, CommentBody, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind,
368 DependencySupport, Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label,
369 LabelFilter, Location, Metering, NativeId, NewComment, Page, PageRequest, Project,
370 ProjectFilter, ProjectQuery, Repository, SecretResolver, SourceError, SourceName, SourcePlugin,
371 Status, StatusCategory, Support, Task, TaskQuery, TaskSource, TextFields, TextQuery,
372 WriteSupport,
373};
374use reqwest::{Client, StatusCode, Url};
375use schemars::{Schema, schema_for};
376use secrecy::{ExposeSecret, SecretString};
377use serde::Deserialize;
378use serde_json::{Value, json};
379
380pub mod accounting;
381
382use accounting::Accounting;
383
384/// The registry name for this plugin.
385pub const KIND: &str = "github-projects";
386/// GitHub's maximum connection page size.
387pub const MAX_PAGE_SIZE: u32 = 100;
388
389/// The most nodes any one document this source sends may be asked to return.
390///
391/// GitHub's own published per-query ceiling, taken from
392/// [`github_graphql_node_count::NODE_LIMIT`] rather than written out again here, so this
393/// workspace cannot hold a stale copy of somebody else's number. A query above it is
394/// **refused before it is executed**, whoever is asking and whatever board they are
395/// asking about — so this is a bound on the documents rather than a budget that runs out.
396///
397/// This is `nodeCount`, the maximum number of nodes *one query may return*. It is not
398/// `cost`, the rate-limit points a call spends against an hourly allowance shared by
399/// everything the credential does — two numbers against two limits, and this constant
400/// bounds only the first. The second is computed offline too, per document:
401/// [`worst_case_point_cost`], pinned for every document in [`graphql::DOCUMENTS`] by
402/// `tests/point_cost.rs`, and reconciled against GitHub's own `cost` by the credentialed
403/// lane. There is no constant like this one to hold a price under, because points are an
404/// hourly allowance rather than a per-call bound.
405///
406/// Neither is a session's price. What `session-cost.md` records of a whole session is its
407/// **requests** and its **worst-case nodes**; what a whole session spends in points is
408/// reported only by a credentialed run's own `x-ratelimit-*` headers, through
409/// [`accounting`]. The module section on the three ways this source reaches an item says how
410/// the count is arrived at, and which of the page sizes below decide it.
411pub const NODE_COUNT_LIMIT: u64 = github_graphql_node_count::NODE_LIMIT;
412
413/// Nested connection size for the connections that hang off one item.
414///
415/// It multiplies through every document that reaches an item under a page — the count
416/// rules multiply down a nested path — so it is the constant [`NODE_COUNT_LIMIT`] is most
417/// sensitive to. `tests/node_count.rs` is what holds the pair together: it recomputes
418/// every document under these constants and fails naming any that reaches the limit, so
419/// raising this is caught there rather than by GitHub.
420const NESTED_PAGE_SIZE: u32 = 50;
421/// How many of one issue's board memberships are read when an issue is reached directly.
422///
423/// An issue reached through a search or through its own node id carries its board half in
424/// `Issue.projectItems`, and only the entry for *this* board is read. This connection sits
425/// under a page of issues, so every point of it multiplies through the whole document and
426/// is paid for whether or not any issue is on a second board — which is why it is
427/// deliberately far smaller than [`NESTED_PAGE_SIZE`].
428///
429/// **Three, because what a page misses is now recovered rather than refused**, and the
430/// recovery is what the value is chosen against. An issue whose entry for this board sits
431/// past this page costs one further request — [`graphql::ISSUE_BOARD_ITEMS`], resumed from
432/// that page's own cursor — so the value trades a bound every read pays for a request only
433/// a multi-board issue pays. At one, a deployment whose issues commonly sit on two or more
434/// boards would pay that request *per issue*, which is order N against the one page per
435/// hundred issues a read costs today. At three it is only reached by an issue on four or
436/// more boards at once, which keeps the recovery path exceptional rather than routine for
437/// a plausible deployment.
438const BOARD_ITEMS_PAGE_SIZE: u32 = 3;
439
440pub use github_graphql_node_count::{NodeCountError, Variables};
441
442/// The largest value this source can bind to each page-size variable its documents name.
443///
444/// Every `first:` in [`graphql`] reads one of these three, and each is capped at the
445/// constant above it wherever a caller's own limit could reach it — `$first` at
446/// [`MAX_PAGE_SIZE`], `$nestedFirst` at `NESTED_PAGE_SIZE`, `$boardItems` at
447/// `BOARD_ITEMS_PAGE_SIZE`. So this is the worst case a caller can drive this source to,
448/// not one configuration of it, which is what makes a bound computed under it a bound on
449/// every read.
450pub fn largest_page_sizes() -> Variables {
451 Variables::from([
452 ("first".to_owned(), MAX_PAGE_SIZE),
453 ("nestedFirst".to_owned(), NESTED_PAGE_SIZE),
454 ("boardItems".to_owned(), BOARD_ITEMS_PAGE_SIZE),
455 ])
456}
457
458/// The most nodes `document` could be asked to return, by GitHub's published rules.
459///
460/// Computed offline from the document's own text under [`largest_page_sizes`] — no
461/// network, no credential and no schema — by
462/// [`github_graphql_node_count::node_count`], which is where the rules themselves live.
463/// A document at or above [`NODE_COUNT_LIMIT`] is one GitHub refuses before executing, so
464/// this is what a check holds every document in [`graphql::DOCUMENTS`] below.
465///
466/// # Errors
467///
468/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
469/// no single operation, or binds a page size this source does not name — each of which is
470/// a defect in the document rather than a number.
471pub fn worst_case_node_count(document: &str) -> Result<u64, NodeCountError> {
472 node_count(document, &largest_page_sizes())
473}
474
475/// The most rate-limit points one call of `document` could spend, by GitHub's published
476/// rules.
477///
478/// Computed offline from the document's own text under [`largest_page_sizes`] — no
479/// network, no credential and no schema — by
480/// [`github_graphql_node_count::point_cost`], which is where the rules themselves live.
481/// This is `cost`, metered **per hour** against the allowance one credential shares across
482/// everything it does; it is not `nodeCount`, which is [`worst_case_node_count`] and is
483/// bounded per query by [`NODE_COUNT_LIMIT`]. There is no per-call ceiling to hold this
484/// under, so what `tests/point_cost.rs` does with it is pin every document in
485/// [`graphql::DOCUMENTS`] at what it costs, and the credentialed lane reconciles those
486/// figures against GitHub's own reported `cost`.
487///
488/// # Errors
489///
490/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
491/// no single operation, or binds a page size this source does not name — each of which is
492/// a defect in the document rather than a number.
493pub fn worst_case_point_cost(document: &str) -> Result<u64, NodeCountError> {
494 github_graphql_node_count::point_cost(document, &largest_page_sizes())
495}
496
497/// The most nodes `document` could be asked to return under `variables`.
498///
499/// [`worst_case_node_count`] is this under [`largest_page_sizes`], and the accounting in
500/// [`accounting`] is this under the bindings one request really sent — one spelling of the
501/// calculation, so a bound checked offline and a cost recorded at run time cannot come to
502/// disagree. The rules themselves live in [`github_graphql_node_count::node_count`].
503///
504/// # Errors
505///
506/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
507/// no single operation, or binds a page size `variables` does not name.
508pub fn node_count(document: &str, variables: &Variables) -> Result<u64, NodeCountError> {
509 github_graphql_node_count::node_count(document, variables)
510}
511
512/// The issue-title prefix that makes a board issue a document.
513///
514/// A GitHub Projects board has no document type — it holds issues — so the discriminator
515/// is the title, and this is the whole of it: an issue whose title begins with these bytes
516/// is a document and every other issue is the task or project the sub-issue rule makes it.
517///
518/// It is spelled **once**, here, and read rather than restated everywhere else — including
519/// by the shared journeys, which take it from this constant so a board fixture cannot
520/// drift from what this source reads. `docs/metadata.md` records the two consequences that
521/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
522/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
523/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
524pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
525
526/// Exact GraphQL query documents issued by this plugin.
527///
528/// Keeping the production documents here lets the pinned-schema test validate the same
529/// bytes that are sent to GitHub, rather than a test-only copy which could drift
530/// independently. No document in this module writes the board itself, and none of them
531/// names `updateProjectV2Field`.
532pub mod graphql {
533 /// The board half of one item: the field values every document here reads it from.
534 ///
535 /// A macro for the same reason [`board_issue!`] below is one, a level further in. This
536 /// selection is needed by that fragment, by [`BOARD`] under the board's own `items`,
537 /// and by [`ISSUE_BOARD_ITEMS`] under a membership walk — and all three have to produce
538 /// *the same value*, because
539 /// [`GitHubProjectsSource::resolve`](super::GitHubProjectsSource) reads them through
540 /// one path. Three spellings of it is what would drift, so there is one.
541 ///
542 /// The `Status` option and this source's own origin text field are the whole of it. It
543 /// selects no `ProjectV2ItemFieldLabelValue`: GitHub derives that field from the item's
544 /// content, so it holds nothing the content's own `labels` do not already say, and it
545 /// would sit a label connection two page sizes deep.
546 macro_rules! board_item_values {
547 () => {
548 r#"fieldValues(first:$nestedFirst){nodes{
549 ... on ProjectV2ItemFieldSingleSelectValue{name field{
550 ... on ProjectV2SingleSelectField{id name options{id name}}
551 }}
552 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
553 }pageInfo{hasNextPage}}"#
554 };
555 }
556
557 /// Everything this source reads about one issue, wherever it reaches that issue.
558 ///
559 /// A macro rather than a constant so the three documents below can `concat!` it: one
560 /// spelling of these fields is what makes an issue read through the board-scoped
561 /// search, through its own node id, and through its project's sub-issue relationship
562 /// resolve to *the same* item, which is the whole of what
563 /// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
564 ///
565 /// `projectItems` is what carries the board half of an issue: the board item's own id
566 /// and the [`board_item_values!`] above — the `Status` option and this source's origin
567 /// text field — that a `ProjectV2.items` read used to carry. It is asked for on the
568 /// issue rather than on the board, which is what makes the cost of a read proportional
569 /// to what was asked for instead of to the board's size.
570 ///
571 /// It carries a *page* of that connection, at `BOARD_ITEMS_PAGE_SIZE`, and its
572 /// `endCursor` is what [`ISSUE_BOARD_ITEMS`] resumes from when this board's entry is
573 /// not on that page: a page here is where the search for the entry starts rather than
574 /// where it ends.
575 ///
576 /// It does **not** select the board's `Labels` field value, and that is the whole of
577 /// what keeps the three documents below under [`NODE_COUNT_LIMIT`](super::NODE_COUNT_LIMIT):
578 /// a label connection there sits under `fieldValues` under `projectItems` under a page
579 /// of issues, spending `$nestedFirst` twice down one path, and took
580 /// [`SEARCH_ISSUES`] and [`SUB_ISSUES`] to 2,556,100 nodes against a limit of 500,000.
581 /// No label is lost — this is a fragment `on Issue`, whose own `labels` are selected
582 /// above, and that connection is where every label this source reports comes from. No
583 /// document in this module selects the board field any longer, [`BOARD`] included; the
584 /// module documentation records why nothing it could have held is lost.
585 macro_rules! board_issue {
586 () => {
587 concat!(
588 r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
589 labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
590 projectItems(first:$boardItems){nodes{id project{id number}
591 "#,
592 board_item_values!(),
593 r#"}pageInfo{hasNextPage endCursor}}}"#
594 )
595 };
596 }
597
598 /// Every issue of one board, found by a search scoped to that board.
599 ///
600 /// This is how the projects a board holds are listed, and it selects no `items`
601 /// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
602 /// container walked page by page, so nothing nested inside a board item is paid for.
603 /// Which of the issues it returns is a project is then read off `parent` — GitHub
604 /// accepts `-has:parent` as a search qualifier and silently ignores it, so the
605 /// discriminator has to be applied to the field, which is a scalar on the issue and
606 /// costs nothing.
607 pub const SEARCH_ISSUES: &str = concat!(
608 r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
609 search(query:$search,type:$type,first:$first,after:$after){
610 pageInfo{hasNextPage endCursor}
611 nodes{__typename ...BoardIssue}
612 }
613 }"#,
614 board_issue!()
615 );
616
617 /// One issue by its own node id, which is what a qualified id names here.
618 ///
619 /// Strongly consistent, unlike the search above: GitHub's issue search is an index and
620 /// answers a write made moments ago with the value from before it, and resolving a node
621 /// id does not.
622 pub const ISSUE: &str = concat!(
623 r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
624 node(id:$id){__typename ...BoardIssue}
625 }"#,
626 board_issue!()
627 );
628
629 /// One project's tasks: the sub-issues of the issue that project is.
630 ///
631 /// The work this costs is the project's own size. Nothing about it grows as the board
632 /// gains projects, or as those projects gain tasks.
633 pub const SUB_ISSUES: &str = concat!(
634 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
635 node(id:$id){__typename
636 ... on Issue{subIssues(first:$first,after:$after){
637 pageInfo{hasNextPage endCursor}
638 nodes{__typename ...BoardIssue}
639 }}}
640 }"#,
641 board_issue!()
642 );
643
644 /// Reads the board's fields and one page of its items.
645 pub const BOARD: &str = concat!(
646 r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
647 owner:repositoryOwner(login:$owner){
648 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
649 }
650 } fragment Board on ProjectV2 { id title
651 fields(first:$nestedFirst){nodes{
652 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
653 ... on ProjectV2Field{__typename id name}
654 }pageInfo{hasNextPage}}
655 items(first:$first,after:$after){nodes{id "#,
656 board_item_values!(),
657 r#" content{
658 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
659 ... on PullRequest{__typename id}
660 ... on DraftIssue{__typename id title body createdAt updatedAt}
661 }} pageInfo{hasNextPage endCursor}}
662 }"#
663 );
664
665 /// One issue's board memberships alone, walked past the page a read of it carried.
666 ///
667 /// The recovery read behind [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource):
668 /// every document above carries a *page* of `Issue.projectItems`, and an issue on more
669 /// boards than that page holds may have this board's entry past its end. This asks that
670 /// one issue for its memberships and nothing else — the caller already holds the issue —
671 /// so an answer of "this board does not hold it" is only ever given about a connection
672 /// read to exhaustion.
673 ///
674 /// It selects the board item's id, its project number and the same
675 /// [`board_item_values!`] the fragment does, because what it produces is handed to the
676 /// very same resolver: an issue recovered this way reports the same title, the same
677 /// status, the same labels and the same qualified id as one whose entry was on the
678 /// page.
679 ///
680 /// `$first` rather than `$boardItems`: this document reads one issue, so nothing
681 /// multiplies through it and the membership connection can be walked at
682 /// [`MAX_PAGE_SIZE`](super::MAX_PAGE_SIZE) — which is what keeps the recovery to one
683 /// further request for any issue a person really keeps.
684 pub const ISSUE_BOARD_ITEMS: &str = concat!(
685 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!){
686 node(id:$id){
687 ... on Issue{projectItems(first:$first,after:$after){
688 nodes{id project{id number}
689 "#,
690 board_item_values!(),
691 r#"}
692 pageInfo{hasNextPage endCursor}}}
693 }
694 }"#
695 );
696 /// Resolves the configured repository's node id, which creating an issue requires.
697 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
698 /// Reads both dependency directions for one issue, with each far end's own kind — and
699 /// the issue's own body, which is where an edge to another source is recorded, so that
700 /// half of a dependency read needs no second read of the issue or of the board.
701 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
702 ... on Issue{body
703 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
704 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
705 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
706 /// Creates one issue in the configured repository.
707 pub const CREATE_ISSUE: &str =
708 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
709 /// Puts an existing issue on the configured board.
710 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
711 /// Updates an issue's visible fields and its open or closed state in one call.
712 pub const UPDATE_ISSUE: &str =
713 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
714 /// Updates an existing draft's user-visible fields.
715 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
716 /// Updates a text or single-select value on one project item.
717 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
718 /// Files one issue under another as a sub-issue, which is what project membership is.
719 pub const ADD_SUB_ISSUE: &str =
720 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
721 /// Takes one issue back out of its parent.
722 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
723 /// Adds GitHub's native issue blocked-by relationship.
724 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
725 /// Removes one native issue blocked-by relationship.
726 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
727 /// Deletes one issue, which takes its board item with it.
728 ///
729 /// The engine sends this in one situation only: undoing a copy that could not finish,
730 /// over the items that same copy created. Deleting the issue removes the board item
731 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
732 pub const DELETE_ISSUE: &str =
733 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
734
735 /// Everything this source reads about one issue comment, wherever it reaches one.
736 ///
737 /// A macro for the reason [`board_issue!`] is one: a comment listed, a comment just added
738 /// and a comment just edited are handed to one mapper, so they are selected by one
739 /// spelling. `author` is `Actor`, which GitHub answers `null` for an account that no
740 /// longer exists, and `login` is the one member every kind of actor carries.
741 macro_rules! issue_comment {
742 () => {
743 "id author{login} createdAt updatedAt body url"
744 };
745 }
746
747 /// One task's comments: a page of its issue's own `comments` connection.
748 ///
749 /// **No `orderBy`, and that is what makes the page oldest first.** GitHub's only
750 /// `IssueCommentOrder` field is `UPDATED_AT`, which would move a comment to the end of the
751 /// list every time somebody edited it; left unordered the connection answers in the order
752 /// the comments were written, which is the order GitHub documents for the same collection
753 /// over REST — ascending id. Nothing multiplies through it, so `$first` is the whole of its
754 /// node count and the caller's own page size is pushed straight down.
755 pub const ISSUE_COMMENTS: &str = concat!(
756 r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename ... on Issue{comments(first:$first,after:$after){nodes{"#,
757 issue_comment!(),
758 r#"}pageInfo{hasNextPage endCursor}}}}}"#
759 );
760 /// Which issue one comment is on, read before that comment is edited or removed.
761 ///
762 /// GitHub's comment mutations take the comment's id and nothing else, so without this a
763 /// comment id given against the wrong task would change a comment on another issue.
764 pub const COMMENT_ISSUE: &str =
765 r#"query($id:ID!){node(id:$id){__typename ... on IssueComment{id issue{id}}}}"#;
766 /// Adds one comment to an issue, signed as the account the token belongs to.
767 pub const ADD_COMMENT: &str = concat!(
768 r#"mutation($input:AddCommentInput!){addComment(input:$input){subject{id} commentEdge{node{"#,
769 issue_comment!(),
770 r#"}}}}"#
771 );
772 /// Replaces the body of one issue comment.
773 pub const UPDATE_COMMENT: &str = concat!(
774 r#"mutation($input:UpdateIssueCommentInput!){updateIssueComment(input:$input){issueComment{"#,
775 issue_comment!(),
776 r#"}}}"#
777 );
778 /// Removes one issue comment. Its payload carries nothing about the comment it removed.
779 pub const DELETE_COMMENT: &str = r#"mutation($input:DeleteIssueCommentInput!){deleteIssueComment(input:$input){clientMutationId}}"#;
780
781 /// Every document above, with what this source is doing when it sends one.
782 ///
783 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
784 /// name the call that was refused, and a `match` with a catch-all arm would answer a
785 /// document added later with "talking to GitHub" and never say so.
786 ///
787 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
788 /// const` here that this list omits, so the two cannot part — which is the same guard
789 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
790 pub const DOCUMENTS: [(&str, &str); 22] = [
791 (SEARCH_ISSUES, "searching this board's issues"),
792 (ISSUE, "reading one issue"),
793 (
794 ISSUE_BOARD_ITEMS,
795 "reading one issue's board memberships past the page it came with",
796 ),
797 (SUB_ISSUES, "reading a project's tasks"),
798 (BOARD, "reading the board"),
799 (REPOSITORY, "reading the destination repository"),
800 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
801 (CREATE_ISSUE, "creating an issue"),
802 (ADD_TO_BOARD, "adding an issue to the board"),
803 (UPDATE_ISSUE, "updating an issue"),
804 (UPDATE_DRAFT, "updating a draft item"),
805 (UPDATE_FIELD, "writing a board field"),
806 (ADD_SUB_ISSUE, "filing an issue under its project"),
807 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
808 (ADD_BLOCKED_BY, "recording a dependency"),
809 (REMOVE_BLOCKED_BY, "removing a dependency"),
810 (DELETE_ISSUE, "deleting an issue"),
811 (ISSUE_COMMENTS, "reading a task's comments"),
812 (COMMENT_ISSUE, "reading which issue a comment is on"),
813 (ADD_COMMENT, "adding a comment"),
814 (UPDATE_COMMENT, "editing a comment"),
815 (DELETE_COMMENT, "deleting a comment"),
816 ];
817}
818
819/// Which of GitHub's two rate limiters refused a request.
820///
821/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
822/// secondary one — so an operator told the wrong one takes the wrong next step, which is
823/// the whole reason this is carried rather than collapsed into "rate limited".
824#[derive(Debug, Clone, Copy, PartialEq, Eq)]
825enum Limiter {
826 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
827 Primary,
828 /// The burst limiter over content-generating requests, which nothing reports.
829 Secondary,
830}
831
832/// The wordings GitHub answers a secondary rate limit with.
833///
834/// It sends them under a forbidden status, under a too-many-requests status, and inside
835/// the `errors` of a *successful* response, which is why the text is what this matches on
836/// rather than the status. `abuse detection` is the wording GitHub used before the
837/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
838/// what a burst of content creation is refused with.
839///
840/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
841/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
842/// when, and the drift gate reconciles the two lists both ways. Public for that gate
843/// alone — a caller has no use for it, and matching on a refusal is this source's job.
844pub const SECONDARY_WORDINGS: [&str; 5] = [
845 "secondary rate limit",
846 "temporarily blocked from content creation",
847 "abuse detection",
848 "submitted too quickly",
849 "exceeded a secondary",
850];
851
852/// The wordings GitHub answers an exhausted primary budget with.
853///
854/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
855/// than looked for in the response text. Pinned and gated exactly as
856/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
857pub const PRIMARY_WORDINGS: [&str; 3] = [
858 "api rate limit exceeded",
859 "rate limit exceeded",
860 "rate_limited",
861];
862
863/// What a response *says about itself*, which is the only place a refusal can be read.
864///
865/// Deliberately not the whole response body. A board is a place people write about their
866/// own work, and a task on it titled "the secondary rate limit" would, matched across the
867/// raw text, turn a perfectly good answer into a refusal this source then waited out and
868/// reported. So the item data is never read: what is read is GitHub's own REST-style
869/// `message` envelope, which is what a forbidden status carries, and the `message` and
870/// `type` of each GraphQL error, which is where a *successful* response says it.
871///
872/// A body that is not JSON at all has nothing structured to read, so only a failing
873/// response's own text is taken — a successful response that is not JSON is malformed
874/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
875fn refusal_wording(status: StatusCode, body: &str) -> String {
876 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
877 return if status.is_success() {
878 String::new()
879 } else {
880 body.to_owned()
881 };
882 };
883 let mut said: Vec<&str> = parsed
884 .get("message")
885 .and_then(Value::as_str)
886 .into_iter()
887 .collect();
888 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
889 for error in errors {
890 said.extend(
891 ["message", "type"]
892 .into_iter()
893 .filter_map(|key| error.get(key).and_then(Value::as_str)),
894 );
895 }
896 }
897 said.join("; ")
898}
899
900impl Limiter {
901 /// Which limiter refused this response, or `None` when none of them did.
902 ///
903 /// The wording is read first and the status only decides what carries none of it,
904 /// because GitHub answers a secondary limit with a forbidden status far more often
905 /// than with too-many-requests — while a forbidden status saying nothing about a limit
906 /// really is a credential this token lacks.
907 ///
908 /// A response is a refusal because of its status or its own wording. A spent budget
909 /// only ever explains one; it never turns an answer into a refusal.
910 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
911 let normalized = refusal_wording(status, body).to_ascii_lowercase();
912 if SECONDARY_WORDINGS
913 .iter()
914 .any(|wording| normalized.contains(wording))
915 {
916 return Some(Self::Secondary);
917 }
918 if status == StatusCode::TOO_MANY_REQUESTS {
919 return Some(Self::Primary);
920 }
921 // An exhausted budget *explains* a response that failed; it does not make one that
922 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
923 // request the budget allowed as well as on the ones it then refuses, so reading
924 // the header alone threw away a good answer — and, once refusals were retried,
925 // replayed a request that had already taken effect.
926 if !status.is_success() && budget_exhausted {
927 return Some(Self::Primary);
928 }
929 // A successful response saying it: GitHub reports a GraphQL rate limit in the
930 // `errors` of an HTTP 200, where nothing about the status says so at all.
931 if status.is_success()
932 && PRIMARY_WORDINGS
933 .iter()
934 .any(|wording| normalized.contains(wording))
935 {
936 return Some(Self::Primary);
937 }
938 None
939 }
940
941 /// What this limiter is called where an operator can look it up.
942 const fn name(self) -> &'static str {
943 match self {
944 Self::Primary => "GitHub's primary API rate limit",
945 Self::Secondary => "GitHub's secondary rate limit",
946 }
947 }
948
949 /// What the endpoint an operator would go and check says about this limiter.
950 const fn where_to_look(self) -> &'static str {
951 match self {
952 Self::Primary => {
953 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
954 comes back."
955 }
956 Self::Secondary => {
957 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
958 primary budget and does not report this one, so budget showing there says \
959 nothing about this refusal, and every further attempt extends it."
960 }
961 }
962 }
963
964 /// The next step this limiter actually calls for.
965 const fn what_to_do(self) -> &'static str {
966 match self {
967 Self::Primary => {
968 "wait for the reset `gh api rate_limit` reports, then run the command again."
969 }
970 Self::Secondary => {
971 "leave this board alone for a few minutes, then run the command again — or \
972 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
973 }
974 }
975 }
976}
977
978/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
979#[derive(Debug, Clone, Copy)]
980struct Limited {
981 limiter: Limiter,
982 hint: Option<u64>,
983}
984
985impl Limited {
986 /// What the caller is told once this source has waited as long as it may.
987 ///
988 /// Both limiters report as [`SourceError::RateLimited`], because that is what
989 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
990 /// about *which* limiter it was makes it a different kind of failure. What differs is
991 /// the operator's next step, and that is what the message carries — a secondary
992 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
993 /// budget looks fine, and then back to retry the very burst that was refused.
994 fn exhausted(
995 self,
996 doing: &str,
997 waits: u32,
998 waited: Duration,
999 needed: Duration,
1000 budget: Duration,
1001 ) -> SourceError {
1002 SourceError::RateLimited {
1003 retry_after_seconds: self.hint,
1004 message: Some(format!(
1005 "{} refused this source while {doing}; it waited {} out over {} and was refused \
1006 again, and the next wait of {} would take it past the {} one call may spend \
1007 waiting. {} next: {}",
1008 self.limiter.name(),
1009 plural(waits, "refusal"),
1010 seconds(waited),
1011 seconds(needed),
1012 seconds(budget),
1013 self.limiter.where_to_look(),
1014 self.limiter.what_to_do(),
1015 )),
1016 }
1017 }
1018}
1019
1020/// One HTTP attempt's result, with what its response said about the rate limit.
1021///
1022/// The two travel together so the record and the outcome are written from the same place:
1023/// what a response said about the budget is only readable while that response is in hand,
1024/// and what the attempt *meant* is only decidable once its body has been read.
1025struct Attempted {
1026 result: Result<Value, Attempt>,
1027 limits: accounting::RateLimit,
1028 /// GitHub's own reported cost for this call, for a document that asked for it.
1029 reported_cost: Option<u64>,
1030}
1031
1032/// One attempt's outcome: an error to report, or a rate limit to wait out.
1033enum Attempt {
1034 Failed(SourceError),
1035 Limited(Limited),
1036}
1037
1038fn plural(count: u32, thing: &str) -> String {
1039 if count == 1 {
1040 format!("{count} {thing}")
1041 } else {
1042 format!("{count} {thing}s")
1043 }
1044}
1045
1046fn seconds(duration: Duration) -> String {
1047 format!("{:.1}s", duration.as_secs_f64())
1048}
1049
1050/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
1051///
1052/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
1053/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
1054/// header, and neither is what makes a response a refusal — so the whole cost of one this
1055/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
1056/// instead. Refusing the response over the header would turn a readable refusal into an
1057/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
1058fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
1059 value
1060 .and_then(|value| value.to_str().ok())
1061 .and_then(|value| value.trim().parse::<u64>().ok())
1062}
1063
1064/// Every mutation this source sends creates content — an issue, a board item, a field of
1065/// one, a sub-issue link, a dependency, a comment — or edits or removes content of that
1066/// kind, and no query in [`graphql::DOCUMENTS`] does, so what the secondary limiter counts
1067/// and what the keyword says are the same set. That is what makes the keyword a sound test
1068/// rather than a convenient one: pacing an edit or a removal the limiter might not have
1069/// counted costs a wait, and not pacing one it did count costs the next fifty minutes.
1070fn is_mutation(query: &str) -> bool {
1071 query.trim_start().starts_with("mutation")
1072}
1073
1074/// What this source was doing, for a diagnostic that has to say so.
1075///
1076/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
1077/// a document added without a description is caught by that list's own gate instead of
1078/// falling through to the vague arm below.
1079fn operation_description(query: &str) -> &'static str {
1080 graphql::DOCUMENTS
1081 .iter()
1082 .find(|(document, _)| *document == query)
1083 .map_or("talking to GitHub", |(_, doing)| *doing)
1084}
1085
1086/// GitHub's published ceiling on content-generating requests, per minute.
1087///
1088/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
1089/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
1090/// from it, so a pacing value checked only against itself cannot go stale here.
1091pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
1092/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
1093/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
1094/// governs; it is pinned beside its sibling so the gate would notice either one moving.
1095pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
1096/// Shortest interval between two content-creating mutations, in milliseconds.
1097///
1098/// GitHub documents two secondary limits on content-generating requests:
1099/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
1100/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
1101/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
1102/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
1103/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
1104/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
1105/// deliberately *not* what this paces at. An installation that wants the hourly bound
1106/// honoured for a long sequence of copies says so through
1107/// `pacing.min_mutation_interval_ms`.
1108pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
1109/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
1110///
1111/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
1112/// own advice for a secondary limit — wait, and wait longer each time — without spending
1113/// the first minute of a transient refusal doing nothing.
1114pub const RETRY_BACKOFF_MS: u64 = 1_000;
1115/// Total time one call may spend waiting out rate limits before it reports a failure.
1116///
1117/// Two minutes is long enough to ride out the refusals a paced copy still collects and
1118/// short enough that a command an operator is watching returns. The bound is what makes
1119/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
1120/// the limiter, not in a process nobody can tell from a wedged one.
1121pub const RETRY_BUDGET_MS: u64 = 120_000;
1122
1123fn default_token_env() -> String {
1124 "GH_PROJECTS_TOKEN".to_owned()
1125}
1126fn default_endpoint() -> String {
1127 "https://api.github.com/graphql".to_owned()
1128}
1129
1130/// Where one status category lands on this board.
1131///
1132/// `null` — an absent value — disables the category for this instance, and using a
1133/// disabled status is a refusal naming the status and the instance.
1134#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
1135#[serde(untagged)]
1136pub enum StatusTargetConfig {
1137 /// The name of a `Status` single-select option already on the board.
1138 Column(ColumnName),
1139 /// A closed issue state, whose reason is what tells done from cancelled.
1140 Closed {
1141 /// The `IssueClosedStateReason` to close with.
1142 closed: ClosedState,
1143 },
1144}
1145
1146/// The name of a `Status` single-select option on the board.
1147///
1148/// Validated on the way in rather than checked later, so a blank option name — which
1149/// nothing on a board can be — is a state this type cannot hold.
1150#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1151#[serde(try_from = "String")]
1152pub struct ColumnName(String);
1153
1154impl ColumnName {
1155 /// The option name, as the board spells it.
1156 fn as_str(&self) -> &str {
1157 &self.0
1158 }
1159}
1160
1161impl TryFrom<String> for ColumnName {
1162 type Error = String;
1163
1164 fn try_from(name: String) -> Result<Self, Self::Error> {
1165 if name.trim().is_empty() {
1166 return Err("a status_mapping option name cannot be blank".to_owned());
1167 }
1168 Ok(Self(name))
1169 }
1170}
1171
1172/// The two closed states this product can mean.
1173///
1174/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
1175/// work nor abandoned work, so nothing here ever writes it.
1176#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
1177#[serde(rename_all = "kebab-case")]
1178pub enum ClosedState {
1179 /// `COMPLETED` — precisely done.
1180 Completed,
1181 /// `NOT_PLANNED` — precisely cancelled.
1182 NotPlanned,
1183}
1184
1185impl ClosedState {
1186 const fn reason(self) -> &'static str {
1187 match self {
1188 Self::Completed => "COMPLETED",
1189 Self::NotPlanned => "NOT_PLANNED",
1190 }
1191 }
1192}
1193
1194/// Configuration for one GitHub Projects v2 board.
1195#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1196#[serde(default, deny_unknown_fields)]
1197pub struct GitHubProjectsConfig {
1198 /// Login of the user or organization which owns the board.
1199 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
1200 /// The project number shown in the board's GitHub URL.
1201 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
1202 // llmlint: ignore-block[contracts_have_one_source_or_a_drift_gate] This doc is the field's schema description, which is what a person configuring the source reads, so it has to say when the field decides an issue's repository and when the item's own field does; the rule's one executable source is `GitHubProjectsSource::creation_target`, and `tests/plugin.rs` drives each case named here against the loopback board.
1203 /// `owner/name` of the repository this source creates an issue in when the item's own
1204 /// `repositories` field does not decide it.
1205 ///
1206 /// An item naming exactly one repository is created there; a task or a document naming
1207 /// none or several is created in its parent project's repository; and a project, or a
1208 /// task or document with no parent, naming none or several is created here. A board
1209 /// has no repository of its own and `createIssue` requires one, so a write without
1210 /// this is refused naming the field. Reads never need it.
1211 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
1212 // llmlint: ignore-end[contracts_have_one_source_or_a_drift_gate]
1213 /// Environment variable containing a fine-grained token with Projects and Issues
1214 /// read/write plus Pull requests read-only access for every repository represented on
1215 /// the board.
1216 #[serde(default = "default_token_env")]
1217 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
1218 /// GraphQL endpoint. GitHub Enterprise installations may override it.
1219 #[serde(default = "default_endpoint")]
1220 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
1221 /// Per-instance mapping from a status category to where it lands on this board.
1222 ///
1223 /// A category this does not mention keeps its shipped default: `backlog` to
1224 /// "Backlog", `todo` to "Todo", `in-progress` to "In Progress", `done` to closed as
1225 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
1226 /// disabled. `unknown` may name one existing board option; every unknown word then
1227 /// lands on that option and reads back as `unknown` under its name. It cannot name a
1228 /// closed state because that reads back as `done` or `cancelled`. Unlike `local-md`,
1229 /// this source cannot keep each unknown word because it never creates board options.
1230 #[serde(default)]
1231 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
1232 /// How fast this source writes, and how long it waits out a rate-limit refusal.
1233 ///
1234 /// Every field keeps its shipped default when it is absent, and the defaults are
1235 /// GitHub's own published limits rather than taste. See [`Pacing`].
1236 #[serde(default)]
1237 pub pacing: PacingConfig,
1238}
1239
1240/// How fast this source writes, and how long it waits out a rate-limit refusal.
1241///
1242/// Configurable because a GitHub Enterprise installation sets its own limits and an
1243/// operator who has already been refused may want to go slower still — not because the
1244/// defaults are guesses.
1245#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
1246#[serde(default, deny_unknown_fields)]
1247pub struct PacingConfig {
1248 /// Shortest interval between two content-creating mutations, in milliseconds.
1249 ///
1250 /// Zero sends them as fast as they are asked for, which is what a fixture server on
1251 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
1252 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1253 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
1254 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
1255 /// zero while there is a budget to spend, because a schedule of zero-length waits
1256 /// consumes none of it and so never ends.
1257 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
1258 /// Total time one call may spend waiting out rate limits, in milliseconds.
1259 ///
1260 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
1261 /// the bound is what makes this a wait rather than a hang.
1262 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
1263}
1264
1265/// The largest any pacing setting may be, in milliseconds.
1266///
1267/// One hour. GitHub's own harshest published bound on content-generating requests works
1268/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
1269/// anything a real limit asks for, and past it the settings stop describing pacing at all:
1270/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
1271/// and an interval beyond it is a command that never sends its second mutation. It also
1272/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
1273/// what an `Instant` can hold on every platform.
1274pub const MAX_PACING_MS: u64 = 3_600_000;
1275
1276/// [`PacingConfig`] with every default resolved and every value checked, which is what the
1277/// source holds.
1278#[derive(Debug, Clone, Copy)]
1279struct Pacing {
1280 min_mutation_interval: Duration,
1281 retry_backoff: Duration,
1282 retry_budget: Duration,
1283}
1284
1285impl Pacing {
1286 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
1287 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
1288 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
1289 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
1290 message: format!(
1291 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
1292 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
1293 GitHub's own harshest published limit"
1294 ),
1295 }),
1296 Some(value) => Ok(Duration::from_millis(value)),
1297 None => Ok(Duration::from_millis(default)),
1298 };
1299 let retry_backoff = bounded(
1300 config.retry_backoff_ms,
1301 RETRY_BACKOFF_MS,
1302 "retry_backoff_ms",
1303 )?;
1304 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
1305 if retry_backoff.is_zero() && !retry_budget.is_zero() {
1306 return Err(SourceError::Config {
1307 message: format!(
1308 "pacing.retry_backoff_ms of source {instance} is 0 while \
1309 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
1310 none of that budget, so it would retry a refusal forever. Set a backoff of \
1311 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
1312 waiting at all",
1313 retry_budget.as_millis()
1314 ),
1315 });
1316 }
1317 Ok(Self {
1318 min_mutation_interval: bounded(
1319 config.min_mutation_interval_ms,
1320 MIN_MUTATION_INTERVAL_MS,
1321 "min_mutation_interval_ms",
1322 )?,
1323 retry_backoff,
1324 retry_budget,
1325 })
1326 }
1327}
1328
1329/// Factory for [`GitHubProjectsSource`].
1330#[derive(Debug, Clone, Copy, Default)]
1331pub struct Plugin;
1332
1333impl SourcePlugin for Plugin {
1334 fn kind(&self) -> &'static str {
1335 KIND
1336 }
1337 fn config_schema(&self) -> Schema {
1338 schema_for!(GitHubProjectsConfig)
1339 }
1340 fn build(
1341 &self,
1342 name: &SourceName,
1343 config: &Value,
1344 secrets: &dyn SecretResolver,
1345 ) -> Result<Box<dyn TaskSource>, SourceError> {
1346 self.build_recording_into(name, config, secrets, Arc::new(Accounting::new()))
1347 }
1348}
1349
1350impl Plugin {
1351 /// Build a source recording every request it sends into an accounting the caller holds.
1352 ///
1353 /// [`SourcePlugin::build`] is this with an accounting of its own, which is what the
1354 /// registry gets. This is for a caller that is also calling GitHub itself and wants one
1355 /// session total rather than two — see [`accounting`] and
1356 /// [`GitHubProjectsSource::recording_into`].
1357 ///
1358 /// # Errors
1359 ///
1360 /// Exactly [`SourcePlugin::build`]'s, with the same source name in front of each:
1361 /// [`SourceError::Config`] for configuration this plugin cannot use and
1362 /// [`SourceError::Auth`] for a credential it cannot find.
1363 pub fn build_recording_into(
1364 &self,
1365 name: &SourceName,
1366 config: &Value,
1367 secrets: &dyn SecretResolver,
1368 ledger: Arc<Accounting>,
1369 ) -> Result<Box<dyn TaskSource>, SourceError> {
1370 let config: GitHubProjectsConfig =
1371 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
1372 message: format!("source {name}: {e}"),
1373 })?;
1374 let source = GitHubProjectsSource::recording_into(name, config, secrets, ledger).map_err(
1375 |error| match error {
1376 SourceError::Config { message } => SourceError::Config {
1377 message: format!("source {name}: {message}"),
1378 },
1379 SourceError::Auth { message } => SourceError::Auth {
1380 message: format!("source {name}: {message}"),
1381 },
1382 other => other,
1383 },
1384 )?;
1385 Ok(Box::new(source))
1386 }
1387}
1388
1389/// Where a status category lands on this board, once configuration is resolved.
1390#[derive(Debug, Clone, PartialEq, Eq)]
1391enum StatusTarget {
1392 /// Not usable against this instance.
1393 Disabled,
1394 /// The board's `Status` option of this name.
1395 Column(ColumnName),
1396 /// A closed issue, with the reason that says which closed it means.
1397 Closed(ClosedState),
1398}
1399
1400/// Every status category, in the order the vocabulary declares them.
1401///
1402/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
1403/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
1404/// added to the shared vocabulary fails to compile until it is named there, and this
1405/// crate's suite reconciles this list against that enum's own derived schema, which is
1406/// generated from the variants rather than written beside them. The schema is what
1407/// catches a list left one short — a list checking only the positions it already holds
1408/// would pass while every mapping indexed by the new position panicked.
1409pub const CATEGORIES: [StatusCategory; 7] = [
1410 StatusCategory::Draft,
1411 StatusCategory::Backlog,
1412 StatusCategory::Todo,
1413 StatusCategory::InProgress,
1414 StatusCategory::Done,
1415 StatusCategory::Cancelled,
1416 StatusCategory::Unknown,
1417];
1418
1419/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
1420#[must_use]
1421pub const fn category_position(category: StatusCategory) -> usize {
1422 match category {
1423 StatusCategory::Draft => 0,
1424 StatusCategory::Backlog => 1,
1425 StatusCategory::Todo => 2,
1426 StatusCategory::InProgress => 3,
1427 StatusCategory::Done => 4,
1428 StatusCategory::Cancelled => 5,
1429 StatusCategory::Unknown => 6,
1430 }
1431}
1432
1433/// The spelling a status category is configured and reported under.
1434fn category_name(category: StatusCategory) -> &'static str {
1435 match category {
1436 StatusCategory::Draft => "draft",
1437 StatusCategory::Backlog => "backlog",
1438 StatusCategory::Todo => "todo",
1439 StatusCategory::InProgress => "in-progress",
1440 StatusCategory::Done => "done",
1441 StatusCategory::Cancelled => "cancelled",
1442 StatusCategory::Unknown => "unknown",
1443 }
1444}
1445
1446/// A shipped default's option name.
1447///
1448/// The literals below are this file's own and non-blank, and they are validated by the
1449/// one constructor a configured name goes through rather than beside it.
1450fn shipped_column(name: &'static str) -> ColumnName {
1451 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
1452}
1453
1454/// The shipped default for one category, before this instance's configuration.
1455fn shipped_default(category: StatusCategory) -> StatusTarget {
1456 match category {
1457 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
1458 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
1459 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
1460 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
1461 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
1462 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
1463 }
1464}
1465
1466/// This instance's complete category-to-target mapping, read in both directions.
1467///
1468/// One target per category, held at that category's own [`category_position`], so a
1469/// category missing from the mapping, named twice in it, or filed out of order is a
1470/// state this type cannot hold rather than one [`Self::target`] has to defend against.
1471#[derive(Debug, Clone)]
1472struct StatusMapping {
1473 targets: [StatusTarget; CATEGORIES.len()],
1474}
1475
1476impl StatusMapping {
1477 fn resolve(
1478 configured: BTreeMap<String, Option<StatusTargetConfig>>,
1479 instance: &SourceName,
1480 ) -> Result<Self, SourceError> {
1481 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
1482 for (key, value) in configured {
1483 let category = CATEGORIES
1484 .iter()
1485 .find(|category| category_name(**category) == key)
1486 .ok_or_else(|| SourceError::Config {
1487 message: format!(
1488 "status_mapping names {key:?}, which is not a status category of source \
1489 {instance}; the categories are {}",
1490 CATEGORIES
1491 .iter()
1492 .map(|category| category_name(*category))
1493 .collect::<Vec<_>>()
1494 .join(", ")
1495 ),
1496 })?;
1497 overrides.insert(category_name(*category), value);
1498 }
1499 if let Some(Some(StatusTargetConfig::Closed { closed })) =
1500 overrides.get(category_name(StatusCategory::Unknown))
1501 {
1502 let read_back = match closed {
1503 ClosedState::Completed => StatusCategory::Done,
1504 ClosedState::NotPlanned => StatusCategory::Cancelled,
1505 };
1506 return Err(SourceError::Config {
1507 message: format!(
1508 "status_mapping.unknown of source {instance} cannot target the closed state \
1509 {} because a copy reads that item back as {}; map unknown to one existing \
1510 board Status option instead",
1511 closed.reason().to_ascii_lowercase().replace('_', "-"),
1512 category_name(read_back)
1513 ),
1514 });
1515 }
1516 // `CATEGORIES[position] == category` for every category — the crate's suite
1517 // asserts it — so mapping the list in order fills each category's own slot.
1518 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
1519 None => shipped_default(category),
1520 Some(None) => StatusTarget::Disabled,
1521 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
1522 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
1523 });
1524 let mapping = Self { targets };
1525 for (index, category) in CATEGORIES.into_iter().enumerate() {
1526 let StatusTarget::Column(option) = mapping.target(category) else {
1527 continue;
1528 };
1529 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
1530 matches!(mapping.target(**earlier), StatusTarget::Column(name)
1531 if name.as_str().eq_ignore_ascii_case(option.as_str()))
1532 }) {
1533 return Err(SourceError::Config {
1534 message: format!(
1535 "status_mapping of source {instance} sends both {} and {} to the board \
1536 option {:?}; one option cannot read back as two categories",
1537 category_name(*other),
1538 category_name(category),
1539 option.as_str()
1540 ),
1541 });
1542 }
1543 }
1544 Ok(mapping)
1545 }
1546
1547 fn target(&self, category: StatusCategory) -> &StatusTarget {
1548 &self.targets[category_position(category)]
1549 }
1550
1551 /// The category a board option name reports, or `None` when nothing maps to it.
1552 fn category_of(&self, option: &str) -> Option<StatusCategory> {
1553 CATEGORIES.into_iter().find(|category| {
1554 matches!(self.target(*category), StatusTarget::Column(name)
1555 if name.as_str().eq_ignore_ascii_case(option))
1556 })
1557 }
1558}
1559
1560// llmlint: ignore-block[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate] Every `createIssue` names one of these, and which one is the rule — a reader who reaches the type from `create_and_file_issue` gets the rule in one sentence here without the method's refusals, which stay on `creation_target`, the rule's one executable source; `tests/plugin.rs` drives every arm of it against the loopback board.
1561/// One repository this source can create an issue in, as `owner/name`.
1562///
1563/// Every `createIssue` this source sends names one of these: the item's own single
1564/// `repositories` entry, else its parent project issue's repository, else the configured
1565/// [`GitHubProjectsConfig::repository`]. [`GitHubProjectsSource::creation_target`] makes
1566/// that choice and says what it refuses before `createIssue`.
1567// llmlint: ignore-end[comments_earn_their_place, contracts_have_one_source_or_a_drift_gate]
1568#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
1569struct RepositoryTarget {
1570 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1571 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1572}
1573
1574impl RepositoryTarget {
1575 fn parse(value: &str) -> Result<Self, SourceError> {
1576 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
1577 message: format!(
1578 "repository must be spelled owner/name; {value:?} names no repository"
1579 ),
1580 })?;
1581 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
1582 return Err(SourceError::Config {
1583 message: format!(
1584 "repository must be spelled owner/name with a GitHub login and one \
1585 repository name; {value:?} is not"
1586 ),
1587 });
1588 }
1589 Ok(Self {
1590 owner: owner.to_owned(),
1591 name: name.to_owned(),
1592 })
1593 }
1594
1595 /// The one host whose repositories this source creates issues in, spelled once: it is
1596 /// what [`Self::origin`] renders and what [`Self::from_origin`] accepts.
1597 const HOST: &str = "github.com";
1598
1599 fn origin(&self) -> String {
1600 format!("{}/{}/{}", Self::HOST, self.owner, self.name)
1601 }
1602
1603 /// The repository a normalized origin names, or why it is none this source can create
1604 /// an issue in: another host, or more or fewer than `owner/name` under this one.
1605 fn from_origin(origin: &Repository) -> Result<Self, String> {
1606 let not_here = || {
1607 format!(
1608 "{} is not a {}/owner/name repository",
1609 origin.as_str(),
1610 Self::HOST
1611 )
1612 };
1613 let (host, rest) = origin.as_str().split_once('/').ok_or_else(not_here)?;
1614 if host != Self::HOST {
1615 return Err(not_here());
1616 }
1617 Self::parse(rest).map_err(|_| not_here())
1618 }
1619
1620 fn slug(&self) -> String {
1621 format!("{}/{}", self.owner, self.name)
1622 }
1623}
1624
1625/// A source which reads GitHub afresh for every operation.
1626pub struct GitHubProjectsSource {
1627 /// This source's configured name, used both to tell a far end naming this source
1628 /// from one naming a system it knows nothing about, and to name the instance a
1629 /// status refusal is about.
1630 name: SourceName,
1631 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
1632 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
1633 repository: Option<RepositoryTarget>,
1634 endpoint: Url,
1635 token: SecretString,
1636 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1637 statuses: StatusMapping,
1638 client: Client,
1639 /// Every item this source has created since it was built, in the order it created
1640 /// them.
1641 ///
1642 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1643 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1644 /// a copy resolving a dependency on an item it had just created refused it as not
1645 /// found. A board read is completed from this — an item remembered here and absent from
1646 /// the read is added back, because the board really does hold it and only the read is
1647 /// behind.
1648 ///
1649 /// It is not a cache of a user's work: nothing is remembered that this process did not
1650 /// itself just write, it lives and dies with the process, and it is never consulted for
1651 /// an item this source did not create.
1652 created: Mutex<Vec<Resolved>>,
1653 /// How fast this source writes, and how long it waits out a refusal.
1654 pacing: Pacing,
1655 /// When the last content-creating mutation finished, or the moment the furthest-out
1656 /// reserved slot releases the next one, whichever is later — so the one after it can be
1657 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1658 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1659 /// what it is measured from.
1660 last_mutation: Mutex<Option<Instant>>,
1661 /// The board as this process last read it, for the length of one command.
1662 ///
1663 /// A copy of a project used to re-read the whole board, paged, before writing each of
1664 /// its items, which is by far the largest part of a copy's request count and none of
1665 /// its work. Nothing else changes this board while a command runs — this source's own
1666 /// writes are the only writer — so one read answers them all.
1667 ///
1668 /// It is not a store of a user's work and it is not the cache the no-persistence
1669 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1670 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1671 /// an item this command created and then depends on resolves whether or not GitHub's
1672 /// own eventually-consistent read has caught up. A write to an item already on the
1673 /// board updates the entry here too, so what this holds is the last read plus this
1674 /// process's own writes rather than a snapshot taken before them.
1675 board_cache: Mutex<Option<Board>>,
1676 /// Each destination repository's node id, resolved once per repository
1677 /// rather than per issue created.
1678 ///
1679 /// A repository's node id does not change, and re-reading it for every issue of a copy
1680 /// spent one request per item on an answer this source already had. It is a map rather
1681 /// than one entry because a copy files each item in the repository its own
1682 /// `repositories` field names, so a plan across five repositories asks GitHub five
1683 /// times and not once per item.
1684 repository_cache: Mutex<BTreeMap<RepositoryTarget, String>>,
1685 /// What every request this source sends is recorded into.
1686 ///
1687 /// Ordinary code path, not a mode: [`Self::send_once`] records into it at the one place
1688 /// a request leaves this crate, so nothing has to be switched on for a session to be
1689 /// counted. It is shared rather than owned so a caller accounting for a whole session —
1690 /// its own schema verification, board lookups, residue sweep and cleanup beside this
1691 /// source's reads and writes — adds up one accounting instead of two. See
1692 /// [`accounting`] for what a record carries and what a session's spend is and is not.
1693 ledger: Arc<Accounting>,
1694}
1695
1696impl GitHubProjectsSource {
1697 /// Validate configuration and capture the named credential without exposing it.
1698 ///
1699 /// # Errors
1700 ///
1701 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1702 /// [`SourceError::Auth`] when the named credential is missing or empty.
1703 pub fn new(
1704 name: &SourceName,
1705 config: GitHubProjectsConfig,
1706 secrets: &dyn SecretResolver,
1707 ) -> Result<Self, SourceError> {
1708 Self::recording_into(name, config, secrets, Arc::new(Accounting::new()))
1709 }
1710
1711 /// The same, recording every request it sends into an accounting the caller holds too.
1712 ///
1713 /// [`Self::new`] is this with an accounting of its own. A caller that is also making
1714 /// its own calls to GitHub — a lane verifying a schema, sweeping residue or cleaning
1715 /// up — passes the one it records those into, so the session total accounts for the
1716 /// whole session rather than for this source's share of it.
1717 ///
1718 /// # Errors
1719 ///
1720 /// Exactly [`Self::new`]'s: [`SourceError::Config`] for a configuration this instance
1721 /// cannot use and [`SourceError::Auth`] when the named credential is missing or empty.
1722 pub fn recording_into(
1723 name: &SourceName,
1724 config: GitHubProjectsConfig,
1725 secrets: &dyn SecretResolver,
1726 ledger: Arc<Accounting>,
1727 ) -> Result<Self, SourceError> {
1728 if !valid_github_owner(&config.owner) {
1729 return Err(SourceError::Config {
1730 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1731 });
1732 }
1733 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1734 return Err(SourceError::Config {
1735 message: format!("project_number must be between 1 and {}", i32::MAX),
1736 });
1737 }
1738 if !valid_environment_name(&config.token_env) {
1739 return Err(SourceError::Config {
1740 message: "token_env must be a valid environment-variable name".into(),
1741 });
1742 }
1743 let repository = config
1744 .repository
1745 .as_deref()
1746 .map(RepositoryTarget::parse)
1747 .transpose()?;
1748 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1749 message: format!("endpoint is not a valid URL: {e}"),
1750 })?;
1751 if endpoint.scheme() != "https"
1752 && !(endpoint.scheme() == "http"
1753 && endpoint
1754 .host_str()
1755 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1756 {
1757 return Err(SourceError::Config {
1758 message:
1759 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1760 .into(),
1761 });
1762 }
1763 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1764 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1765 })?;
1766 Ok(Self {
1767 name: name.clone(),
1768 owner: config.owner,
1769 project_number: config.project_number,
1770 repository,
1771 endpoint,
1772 token,
1773 credential_name: config.token_env,
1774 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1775 client: Client::builder()
1776 .user_agent("onetaskgraph")
1777 .build()
1778 .map_err(|e| SourceError::Config {
1779 message: format!("cannot build HTTP client: {e}"),
1780 })?,
1781 created: Mutex::new(Vec::new()),
1782 pacing: Pacing::resolve(config.pacing, name)?,
1783 last_mutation: Mutex::new(None),
1784 board_cache: Mutex::new(None),
1785 repository_cache: Mutex::new(BTreeMap::new()),
1786 ledger,
1787 })
1788 }
1789
1790 /// A snapshot of every request this source has sent, and what each cost.
1791 ///
1792 /// A value to hold and compare rather than a borrow of the accounting itself, so two
1793 /// of them can sit side by side. When this source was built with
1794 /// [`Self::recording_into`] the snapshot is the whole shared session, which is the
1795 /// point of building it that way.
1796 #[must_use]
1797 pub fn accounting(&self) -> accounting::Session {
1798 self.ledger.snapshot()
1799 }
1800
1801 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1802 /// rate limit rather than handing it straight back as an error.
1803 ///
1804 /// Retrying is safe for every document here, including the mutations, and the reason
1805 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1806 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1807 /// this replays has already taken effect. An outcome this source cannot know — the
1808 /// send failed, or the body could not be read, so the mutation may well have landed —
1809 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1810 /// attempt. A duplicate write would come from replaying one of those, and none is
1811 /// replayed.
1812 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1813 let doing = operation_description(query);
1814 let mut waited = Duration::ZERO;
1815 let mut waits = 0_u32;
1816 let mut backoff = self.pacing.retry_backoff;
1817 loop {
1818 if is_mutation(query) {
1819 let spacing = self.reserve_mutation_slot();
1820 if !spacing.is_zero() {
1821 tokio::time::sleep(spacing).await;
1822 }
1823 }
1824 let attempt = self.send_once(query, &variables).await;
1825 if is_mutation(query) {
1826 self.finish_mutation();
1827 }
1828 let limited = match attempt {
1829 Ok(data) => return Ok(data),
1830 Err(Attempt::Failed(error)) => return Err(error),
1831 Err(Attempt::Limited(limited)) => limited,
1832 };
1833 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1834 // move that extends a secondary limit, so a hint below the schedule's own next
1835 // wait is raised to it.
1836 let wait = match limited.hint {
1837 Some(hint) => Duration::from_secs(hint).max(backoff),
1838 None => backoff,
1839 };
1840 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1841 // A wait of nothing spends none of the budget, so it is exhaustion rather
1842 // than a retry. `Pacing::resolve` rules out every way of configuring one
1843 // except a budget of zero, where reporting the first refusal is the ask.
1844 if wait.is_zero() || wait > remaining {
1845 return Err(limited.exhausted(
1846 doing,
1847 waits,
1848 waited,
1849 wait,
1850 self.pacing.retry_budget,
1851 ));
1852 }
1853 tokio::time::sleep(wait).await;
1854 waited += wait;
1855 waits += 1;
1856 backoff = backoff.saturating_mul(2);
1857 }
1858 }
1859
1860 /// The next moment a content-creating mutation may leave this source, as a wait from
1861 /// now.
1862 ///
1863 /// The slot is reserved under the lock and the waiting happens outside it, so two
1864 /// callers take two slots rather than the same one — and no lock is held across an
1865 /// await.
1866 ///
1867 /// The moment it is spaced from is the previous mutation's *completion*, which
1868 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1869 /// own is the wrong thing to measure from.
1870 fn reserve_mutation_slot(&self) -> Duration {
1871 if self.pacing.min_mutation_interval.is_zero() {
1872 return Duration::ZERO;
1873 }
1874 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1875 // it would turn an earlier failure into a second one for no gain.
1876 let mut last = self
1877 .last_mutation
1878 .lock()
1879 .unwrap_or_else(std::sync::PoisonError::into_inner);
1880 let now = Instant::now();
1881 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1882 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1883 let at = last.map_or(now, |previous| {
1884 previous
1885 .checked_add(self.pacing.min_mutation_interval)
1886 .map_or(now, |earliest| earliest.max(now))
1887 });
1888 *last = Some(at);
1889 at.saturating_duration_since(now)
1890 }
1891
1892 /// Record that a content-creating mutation has finished, so the next one is spaced
1893 /// from here rather than from the moment this one was released.
1894 ///
1895 /// This source can only choose when a request *departs*; the limiter counts when it
1896 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1897 /// departure from the last therefore hands the limiter a gap of the interval less that
1898 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1899 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1900 /// slower machine while passing on a quick one.
1901 ///
1902 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1903 /// previous request had already arrived before its response came back, so its arrival
1904 /// is no later than this moment, and the next mutation is released at least the
1905 /// interval after this moment and arrives no earlier than it is released: the gap the
1906 /// limiter measures is therefore at least the interval, whatever transit costs and on
1907 /// whatever platform. The price is that a mutation's own round trip no longer counts
1908 /// towards its spacing, which makes this source slightly slower than the configured
1909 /// rate rather than slightly faster — the safe side of a limit that punishes being
1910 /// wrong by refusing reads for the next fifty minutes.
1911 ///
1912 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1913 /// and one that never left costs only a wait nobody needed.
1914 fn finish_mutation(&self) {
1915 if self.pacing.min_mutation_interval.is_zero() {
1916 return;
1917 }
1918 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1919 let mut last = self
1920 .last_mutation
1921 .lock()
1922 .unwrap_or_else(std::sync::PoisonError::into_inner);
1923 let now = Instant::now();
1924 // `max` rather than an assignment: a concurrent caller may already have reserved a
1925 // slot further out, and completing this request must never pull that slot back in.
1926 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1927 }
1928
1929 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1930 /// failure that waiting cannot help — and recorded, whichever of the three it was.
1931 ///
1932 /// This is the one place a request leaves this crate, which is why the accounting is
1933 /// here rather than at each of the callers: a read path added later is counted without
1934 /// anybody remembering to count it, and
1935 /// `the_session_report_counts_every_request_the_board_served_and_what_each_cost` fails
1936 /// when one is not.
1937 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1938 let Attempted {
1939 result,
1940 limits,
1941 reported_cost,
1942 } = self.attempt(query, variables).await;
1943 // No `otherwise` name: every document this source sends is one of its own, and the
1944 // inventory gate on `graphql::DOCUMENTS` is what keeps that true.
1945 let sending = accounting::Request::graphql(query, variables, None, reported_cost);
1946 let outcome = match &result {
1947 Ok(_) => accounting::Outcome::Answered,
1948 Err(Attempt::Limited(_)) => accounting::Outcome::RateLimited,
1949 Err(Attempt::Failed(_)) => accounting::Outcome::Refused,
1950 };
1951 self.ledger.record(sending.finished(outcome, limits));
1952 result
1953 }
1954
1955 /// The attempt itself, with what its response said about the rate limit alongside.
1956 ///
1957 /// The two are returned together rather than recorded here because every one of the
1958 /// early exits below is a different outcome, and a record written at each of them is a
1959 /// record one of them can be added without.
1960 async fn attempt(&self, query: &str, variables: &Value) -> Attempted {
1961 let mut limits = accounting::RateLimit::default();
1962 let mut reported_cost = None;
1963 let result = self
1964 .attempted(query, variables, &mut limits, &mut reported_cost)
1965 .await;
1966 Attempted {
1967 result,
1968 limits,
1969 reported_cost,
1970 }
1971 }
1972
1973 /// One HTTP attempt, filling in what its response said about the rate limit as it goes.
1974 async fn attempted(
1975 &self,
1976 query: &str,
1977 variables: &Value,
1978 limits: &mut accounting::RateLimit,
1979 reported_cost: &mut Option<u64>,
1980 ) -> Result<Value, Attempt> {
1981 let response = self
1982 .client
1983 .post(self.endpoint.clone())
1984 .bearer_auth(self.token.expose_secret())
1985 .json(&json!({"query": query, "variables": variables}))
1986 .send()
1987 .await
1988 .map_err(|e| {
1989 Attempt::Failed(SourceError::Unavailable {
1990 message: format!("GitHub GraphQL request failed: {e}"),
1991 })
1992 })?;
1993 let status = response.status();
1994 let header = |name: &str| whole_seconds(response.headers().get(name));
1995 *limits = accounting::RateLimit::read(|name| {
1996 response
1997 .headers()
1998 .get(name)
1999 .and_then(|value| value.to_str().ok())
2000 .map(str::to_owned)
2001 });
2002 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
2003 // that are not text at all — is "not known to be exhausted". This never makes a
2004 // response a refusal on its own: it says which limiter a refusal is attributed to
2005 // and where its hint comes from, so a value this cannot read costs a hint rather
2006 // than an answer.
2007 let exhausted = response
2008 .headers()
2009 .get("x-ratelimit-remaining")
2010 .and_then(|value| value.to_str().ok())
2011 == Some("0");
2012 // `retry-after` is what GitHub asks for when it asks; when it does not and the
2013 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
2014 // which is the same question answered as an absolute time. Nothing else here is a
2015 // hint, and a schedule is what answers a refusal that carries none.
2016 let hint = header("retry-after").or_else(|| {
2017 exhausted
2018 .then(|| header("x-ratelimit-reset"))
2019 .flatten()
2020 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
2021 });
2022 // Read before it is parsed, because the evidence which tells a secondary rate
2023 // limit from a rejected credential is in the body of a response whose status says
2024 // only "forbidden" — and a non-success response was never parsed at all.
2025 let body = response.text().await.map_err(|e| {
2026 Attempt::Failed(SourceError::Unavailable {
2027 message: format!("GitHub GraphQL response could not be read: {e}"),
2028 })
2029 })?;
2030 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
2031 return Err(Attempt::Limited(Limited { limiter, hint }));
2032 }
2033 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
2034 return Err(Attempt::Failed(SourceError::Auth {
2035 message: format!(
2036 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
2037 ),
2038 }));
2039 }
2040 if !status.is_success() {
2041 return Err(Attempt::Failed(SourceError::Unavailable {
2042 message: format!("GitHub GraphQL returned HTTP {status}"),
2043 }));
2044 }
2045 // GitHub reports what a call cost only when the document asked it to, and no
2046 // document this source sends does — so this is `None` here and carries the figure
2047 // for a caller whose own document selects `rateLimit { cost }`. What it must never
2048 // pick up is a `dryRun` probe's cost, which is some other document's.
2049 *reported_cost = serde_json::from_str::<Value>(&body)
2050 .ok()
2051 .as_ref()
2052 .and_then(|body| body.pointer("/data/rateLimit/cost"))
2053 .and_then(Value::as_u64);
2054 self.answer(&body).map_err(Attempt::Failed)
2055 }
2056
2057 /// What one successful HTTP response says, once its GraphQL errors are read.
2058 fn answer(&self, body: &str) -> Result<Value, SourceError> {
2059 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
2060 message: format!("GitHub returned invalid JSON: {e}"),
2061 })?;
2062 let errors = body
2063 .get("errors")
2064 .map(|value| {
2065 value.as_array().ok_or_else(|| SourceError::Malformed {
2066 message: "GitHub response errors is not an array".into(),
2067 })
2068 })
2069 .transpose()?;
2070 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
2071 let messages = errors
2072 .iter()
2073 .filter_map(|e| e.get("message").and_then(Value::as_str))
2074 .collect::<Vec<_>>()
2075 .join("; ");
2076 let message = if messages.is_empty() {
2077 "GitHub returned GraphQL errors".into()
2078 } else {
2079 messages
2080 };
2081 let normalized = message.to_ascii_lowercase();
2082 if normalized.contains("resource not accessible") || normalized.contains("scope") {
2083 return Err(SourceError::Auth {
2084 message: format!(
2085 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
2086 self.credential_name
2087 ),
2088 });
2089 }
2090 return Err(SourceError::Refused { message });
2091 }
2092 body.get("data")
2093 .filter(|data| data.is_object())
2094 .cloned()
2095 .ok_or_else(|| SourceError::Malformed {
2096 message: "GitHub response has no data object".into(),
2097 })
2098 }
2099
2100 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
2101 // GraphQL cannot independently page them inside the outer item page. This source page is
2102 // deliberately bounded at that published maximum; the live drift journey exercises it.
2103 async fn board_page(
2104 &self,
2105 items_after: Option<&str>,
2106 items_first: u32,
2107 ) -> Result<Value, SourceError> {
2108 let data = self
2109 .graphql(
2110 graphql::BOARD,
2111 json!({"owner":self.owner,"number":self.project_number,
2112 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
2113 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
2114 )
2115 .await?;
2116 data.pointer("/owner/projectV2")
2117 .filter(|v| !v.is_null())
2118 .cloned()
2119 .ok_or_else(|| SourceError::Refused {
2120 message: format!(
2121 "GitHub project {}/{} was not found or is not visible to the token",
2122 self.owner, self.project_number
2123 ),
2124 })
2125 }
2126
2127 /// The search that finds the issues of this board, narrowed by `also` when it is
2128 /// given.
2129 ///
2130 /// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
2131 /// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
2132 /// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
2133 /// here on purpose — GitHub accepts it and silently ignores it, so a project is told
2134 /// from a task by the `parent` field each issue carries rather than by the search.
2135 fn board_search(&self, also: Option<&str>) -> String {
2136 let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
2137 match also {
2138 Some(also) => format!("{scope} {also}"),
2139 None => scope,
2140 }
2141 }
2142
2143 /// One issue this source reached directly, as the board item a read of the board would
2144 /// have produced — or `None` when this board does not hold it.
2145 ///
2146 /// The board half of an issue rides along on `Issue.projectItems`, so the value handed
2147 /// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
2148 /// item's own id, that item's field values, and the issue as its content. One resolver
2149 /// for both routes is what makes an issue read through a search, through its own node
2150 /// id, or through its project's sub-issues report the same title, the same status, the
2151 /// same labels and the same qualified id.
2152 ///
2153 /// An issue with no entry for *this* board is not this source's to report, which is
2154 /// what keeps an id naming some other repository's issue from being answered as an item
2155 /// of this board. That answer is given about an **exhausted** connection and never
2156 /// about an unread page: the entry is looked for on the page in hand, and only if that
2157 /// page reports more of the connection, in [`Self::board_membership`]'s walk of the
2158 /// rest of it.
2159 async fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
2160 if optional_str(issue, "__typename")? != Some("Issue") {
2161 return Ok(None);
2162 }
2163 let memberships = issue
2164 .get("projectItems")
2165 .ok_or_else(|| SourceError::Malformed {
2166 message: "GitHub issue is missing projectItems".into(),
2167 })?;
2168 let nodes = memberships
2169 .get("nodes")
2170 .and_then(Value::as_array)
2171 .ok_or_else(|| SourceError::Malformed {
2172 message: "GitHub issue projectItems.nodes is not an array".into(),
2173 })?;
2174 let held = match self.board_entry(nodes) {
2175 Some(held) => held.clone(),
2176 None => {
2177 let info = memberships
2178 .get("pageInfo")
2179 .ok_or_else(|| SourceError::Malformed {
2180 message: "GitHub issue projectItems has no pageInfo".into(),
2181 })?;
2182 // The page held no entry for this board. Whether that means the issue is
2183 // not on it is a question about the rest of the connection, and only a
2184 // connection with no rest answers it here.
2185 if !required_bool(info, "hasNextPage")? {
2186 return Ok(None);
2187 }
2188 let cursor = required_str(info, "endCursor")?;
2189 validate_cursor_progress(None, cursor)?;
2190 let issue_id = required_str(issue, "id")?;
2191 match self.board_membership(issue_id, cursor).await? {
2192 Some(held) => held,
2193 None => return Ok(None),
2194 }
2195 }
2196 };
2197 let item = json!({
2198 "id": required_str(&held, "id")?,
2199 "project": held.get("project"),
2200 "fieldValues": held.get("fieldValues"),
2201 "content": issue,
2202 });
2203 self.resolve(&item)
2204 }
2205
2206 /// This board's own entry among one page of an issue's `Issue.projectItems`.
2207 ///
2208 /// One spelling of *which membership is this board's*, so the page a read carries and
2209 /// the pages [`Self::board_membership`] walks are searched by the same rule.
2210 fn board_entry<'a>(&self, nodes: &'a [Value]) -> Option<&'a Value> {
2211 nodes.iter().find(|node| {
2212 node.pointer("/project/number").and_then(Value::as_u64)
2213 == Some(u64::from(self.project_number))
2214 })
2215 }
2216
2217 /// The rest of one issue's board memberships, from `after`, for this board's entry.
2218 ///
2219 /// The recovery read: a page of memberships that holds no entry for this board says
2220 /// nothing about the memberships past it, so the connection is walked to exhaustion
2221 /// before an issue is reported as one this board does not hold. `Ok(None)` is that
2222 /// positive answer — the whole connection was read and no entry named this board —
2223 /// rather than a failure, and the walk is held to
2224 /// [`validate_cursor_progress`] like every other page walk here, so a source answering
2225 /// with a cursor that does not advance is refused instead of spun on.
2226 async fn board_membership(
2227 &self,
2228 issue: &str,
2229 after: &str,
2230 ) -> Result<Option<Value>, SourceError> {
2231 let mut after = after.to_owned();
2232 loop {
2233 let data = self
2234 .graphql(
2235 graphql::ISSUE_BOARD_ITEMS,
2236 json!({"id":issue,"first":MAX_PAGE_SIZE,"after":after,
2237 "nestedFirst":NESTED_PAGE_SIZE}),
2238 )
2239 .await?;
2240 let Some(connection) = data
2241 .pointer("/node/projectItems")
2242 .filter(|value| !value.is_null())
2243 else {
2244 // The id resolved to nothing, or to something with no memberships to walk —
2245 // which is the same answer as a connection holding no entry for this board.
2246 return Ok(None);
2247 };
2248 let nodes = connection
2249 .get("nodes")
2250 .and_then(Value::as_array)
2251 .ok_or_else(|| SourceError::Malformed {
2252 message: "GitHub issue projectItems.nodes is not an array".into(),
2253 })?;
2254 if let Some(held) = self.board_entry(nodes) {
2255 return Ok(Some(held.clone()));
2256 }
2257 let info = connection
2258 .get("pageInfo")
2259 .ok_or_else(|| SourceError::Malformed {
2260 message: "GitHub issue projectItems has no pageInfo".into(),
2261 })?;
2262 let next = required_bool(info, "hasNextPage")?
2263 .then(|| required_str(info, "endCursor"))
2264 .transpose()?;
2265 match next {
2266 Some(next) => {
2267 validate_cursor_progress(Some(&after), next)?;
2268 after = next.to_owned();
2269 }
2270 None => return Ok(None),
2271 }
2272 }
2273 }
2274
2275 /// One page of a board-scoped issue search, and where the next page resumes.
2276 async fn search_page(
2277 &self,
2278 search: &str,
2279 first: u32,
2280 after: Option<&str>,
2281 ) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
2282 let data = self
2283 .graphql(
2284 graphql::SEARCH_ISSUES,
2285 json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
2286 "after":after,"nestedFirst":NESTED_PAGE_SIZE,
2287 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2288 )
2289 .await?;
2290 let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
2291 message: "GitHub search response has no search connection".into(),
2292 })?;
2293 let mut found = Vec::new();
2294 for node in connection
2295 .get("nodes")
2296 .and_then(Value::as_array)
2297 .ok_or_else(|| SourceError::Malformed {
2298 message: "GitHub search nodes is not an array".into(),
2299 })?
2300 {
2301 if let Some(resolved) = self.resolve_issue(node).await? {
2302 found.push(resolved);
2303 }
2304 }
2305 let info = connection
2306 .get("pageInfo")
2307 .ok_or_else(|| SourceError::Malformed {
2308 message: "GitHub search connection has no pageInfo".into(),
2309 })?;
2310 let next = required_bool(info, "hasNextPage")?
2311 .then(|| required_str(info, "endCursor"))
2312 .transpose()?
2313 .map(str::to_owned);
2314 if let Some(next) = &next {
2315 validate_cursor_progress(after, next)?;
2316 }
2317 Ok((found, next))
2318 }
2319
2320 /// Every issue this board holds, walked to exhaustion, completed with what this run
2321 /// wrote.
2322 ///
2323 /// The completion is not an optimisation and it is not a cache: GitHub's issue search
2324 /// is an index and is eventually consistent, so an issue this run created seconds ago
2325 /// is routinely absent from it, and a project listed straight after being written would
2326 /// otherwise be missing from its own board. What is added back is only what this
2327 /// process itself wrote, out of [`Self::created`], which lives and dies with the
2328 /// process.
2329 async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
2330 let mut after: Option<String> = None;
2331 let mut found = Vec::new();
2332 let search = self.board_search(None);
2333 loop {
2334 let (page, next) = self
2335 .search_page(&search, MAX_PAGE_SIZE, after.as_deref())
2336 .await?;
2337 found.extend(page);
2338 match next {
2339 Some(next) => after = Some(next),
2340 None => break,
2341 }
2342 }
2343 self.completed_with_written(found, |_| true)
2344 }
2345
2346 /// `found`, with everything this run wrote that `keep` accepts and the read did not
2347 /// report.
2348 ///
2349 /// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
2350 /// at all: the search index is behind, and a node read of an item filed moments ago can
2351 /// be too.
2352 fn completed_with_written(
2353 &self,
2354 mut found: Vec<Resolved>,
2355 keep: impl Fn(&Resolved) -> bool,
2356 ) -> Result<Vec<Resolved>, SourceError> {
2357 for own in self.created()?.iter().filter(|own| keep(own)) {
2358 if !found.iter().any(|item| item.id == own.id) {
2359 found.push(own.clone());
2360 }
2361 }
2362 Ok(found)
2363 }
2364
2365 /// What resolving one node id reached.
2366 ///
2367 /// Three answers rather than an `Option`, because a board *draft* is none of the other
2368 /// two: it is not an issue, it has no node of its own this source can read the board
2369 /// half off, and its only home is the board's own item connection — so a read of one
2370 /// is completed from there rather than reported as nothing.
2371 async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
2372 let asked = self
2373 .graphql(
2374 graphql::ISSUE,
2375 json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
2376 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2377 )
2378 .await;
2379 let data = match asked {
2380 Ok(data) => data,
2381 // A string that is not a node id at all is not a failure to report: it is an id
2382 // this board does not hold, which is what every read of one already answers.
2383 Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
2384 Err(error) => return Err(error),
2385 };
2386 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
2387 return Ok(Reached::Nothing);
2388 };
2389 if optional_str(node, "__typename")? == Some("DraftIssue") {
2390 return Ok(Reached::Draft);
2391 }
2392 Ok(match self.resolve_issue(node).await? {
2393 Some(item) => Reached::Held(Box::new(item)),
2394 None => Reached::Nothing,
2395 })
2396 }
2397
2398 /// One item of this board by its own id, whatever kind it is.
2399 ///
2400 /// Resolved from the identifier alone: no search, board-wide or otherwise. What this
2401 /// run wrote is read first, because a node read of an item created moments ago can
2402 /// still be behind the board field values written onto it — see [`Self::created`].
2403 async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
2404 if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
2405 return Ok(Some(own.clone()));
2406 }
2407 match self.reach(id).await? {
2408 Reached::Held(item) => Ok(Some(*item)),
2409 Reached::Nothing => Ok(None),
2410 // The one read that still costs the board: a draft lives nowhere else.
2411 Reached::Draft => Ok(self
2412 .board()
2413 .await?
2414 .items
2415 .into_iter()
2416 .find(|item| item.id == *id)),
2417 }
2418 }
2419
2420 /// Everything filed under one issue of this board, walked to exhaustion — or `None`
2421 /// when that id names nothing here with a sub-issue relationship to walk.
2422 ///
2423 /// `None` and an empty answer are different: `None` is *this is not an issue of this
2424 /// GitHub*, which is what sends a project selector on to be read as a name, and an
2425 /// empty vector is a project that holds nothing.
2426 async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
2427 let mut after: Option<String> = None;
2428 let mut children = Vec::new();
2429 loop {
2430 let asked = self
2431 .graphql(
2432 graphql::SUB_ISSUES,
2433 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
2434 "nestedFirst":NESTED_PAGE_SIZE,
2435 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
2436 )
2437 .await;
2438 let data = match asked {
2439 Ok(data) => data,
2440 // A string that is not a node id at all is not a failure to report: it is
2441 // the ordinary answer to a selector naming a project by its name.
2442 Err(error) if unresolvable_node(&error) => return Ok(None),
2443 Err(error) => return Err(error),
2444 };
2445 let Some(connection) = data
2446 .pointer("/node/subIssues")
2447 .filter(|value| !value.is_null())
2448 else {
2449 // No such node, or one with no sub-issue relationship — a board draft is
2450 // the one this board can really hold.
2451 return Ok(None);
2452 };
2453 for node in connection
2454 .get("nodes")
2455 .and_then(Value::as_array)
2456 .ok_or_else(|| SourceError::Malformed {
2457 message: "GitHub subIssues.nodes is not an array".into(),
2458 })?
2459 {
2460 if let Some(resolved) = self.resolve_issue(node).await? {
2461 children.push(resolved);
2462 }
2463 }
2464 let info = connection
2465 .get("pageInfo")
2466 .ok_or_else(|| SourceError::Malformed {
2467 message: "GitHub subIssues connection has no pageInfo".into(),
2468 })?;
2469 let next = required_bool(info, "hasNextPage")?
2470 .then(|| required_str(info, "endCursor"))
2471 .transpose()?;
2472 match next {
2473 Some(next) => {
2474 validate_cursor_progress(after.as_deref(), next)?;
2475 after = Some(next.to_owned());
2476 }
2477 None => return Ok(Some(children)),
2478 }
2479 }
2480 }
2481
2482 /// Which issue of this board a project *name* is, or `None` when none is.
2483 ///
2484 /// One bounded query which filters on that name at the server, rather than a walk of
2485 /// every issue the board holds. The name is compared again here: the qualifier narrows
2486 /// what GitHub sends, and this source decides what it names.
2487 async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
2488 let search = self.board_search(Some(&title_qualifier(name)));
2489 let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
2490 Ok(candidates
2491 .into_iter()
2492 .find(|item| {
2493 item.kind == BoardKind::Work(ItemKind::Project)
2494 && item.title.eq_ignore_ascii_case(name)
2495 })
2496 .map(|item| item.id))
2497 }
2498
2499 /// Everything filed under one project of this board: the sub-issues of the issue that
2500 /// project is.
2501 ///
2502 /// Tasks *and* documents, because a document filed under a project is a sub-issue of it
2503 /// too — the caller keeps the kind it asked for. Nothing about this grows as the board
2504 /// gains projects, or as another project gains tasks.
2505 ///
2506 /// A qualified id names the issue and is asked for its sub-issues directly: one
2507 /// request, no search of any kind. Only a selector GitHub cannot resolve that way is
2508 /// read as a project *name*, which costs the one bounded search
2509 /// [`Self::project_by_name`] makes.
2510 async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
2511 let (project, children) = match self.sub_issues(selector).await? {
2512 Some(children) => (selector.clone(), children),
2513 None => match self.project_by_name(&selector.0).await? {
2514 Some(project) => {
2515 let children = self.sub_issues(&project).await?.unwrap_or_default();
2516 (project, children)
2517 }
2518 None => return Ok(Vec::new()),
2519 },
2520 };
2521 self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
2522 }
2523
2524 /// Every item on the board, with the one board identity they all share.
2525 ///
2526 /// See [`Self::board_cache`]. The completion from `created` happens on every call
2527 /// rather than once, which is what the cache could otherwise have broken.
2528 async fn board(&self) -> Result<Board, SourceError> {
2529 let cached = self.board_cache()?.clone();
2530 let mut board = match cached {
2531 Some(board) => board,
2532 None => {
2533 let read = self.read_board().await?;
2534 *self.board_cache()? = Some(read.clone());
2535 read
2536 }
2537 };
2538 for own in self.created()?.iter() {
2539 if !board.items.iter().any(|item| item.id == own.id) {
2540 board.items.push(own.clone());
2541 }
2542 }
2543 Ok(board)
2544 }
2545
2546 /// This process's own view of the board, or the refusal a poisoned lock is.
2547 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
2548 self.board_cache
2549 .lock()
2550 .map_err(|_| SourceError::Unavailable {
2551 message: "this source's view of the board was left inconsistent by an earlier \
2552 failure; next: run the command again"
2553 .into(),
2554 })
2555 }
2556
2557 /// Bring this process's own view of the board up to an item it has just written.
2558 ///
2559 /// A created item goes to `created`, which is what completes a board read GitHub's own
2560 /// eventual consistency has left behind. An item that was already there is replaced
2561 /// where it sits, so a second write of it in the same command reads its real parent
2562 /// rather than the one it had before the first write.
2563 ///
2564 /// "Where it sits" is two places, and missing the first leaves a stale record that
2565 /// wins: an item this same run created is held in `created` and not in the cached
2566 /// board, and `board` completes the cached board *from* `created`, so replacing only
2567 /// the cached copy of such an item replaces nothing and the read still reports the
2568 /// title it was created with.
2569 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
2570 if created {
2571 self.created()?.push(item);
2572 return Ok(());
2573 }
2574 {
2575 let mut own = self.created()?;
2576 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
2577 *held = item;
2578 return Ok(());
2579 }
2580 }
2581 if let Some(board) = self.board_cache()?.as_mut()
2582 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
2583 {
2584 *held = item;
2585 }
2586 Ok(())
2587 }
2588
2589 /// Forget one item this process has just deleted, from both halves of its own view.
2590 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
2591 self.created()?.retain(|own| own.id != *id);
2592 if let Some(board) = self.board_cache()?.as_mut() {
2593 board.items.retain(|item| item.id != *id);
2594 }
2595 Ok(())
2596 }
2597
2598 /// Every page of the board, read from GitHub.
2599 async fn read_board(&self) -> Result<Board, SourceError> {
2600 let mut after: Option<String> = None;
2601 let mut items = Vec::new();
2602 let mut board;
2603 loop {
2604 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
2605 for item in page
2606 .pointer("/items/nodes")
2607 .and_then(Value::as_array)
2608 .ok_or_else(|| SourceError::Malformed {
2609 message: "GitHub project items.nodes is not an array".into(),
2610 })?
2611 {
2612 if let Some(resolved) = self.resolve(item)? {
2613 items.push(resolved);
2614 }
2615 }
2616 let info = page
2617 .pointer("/items/pageInfo")
2618 .ok_or_else(|| SourceError::Malformed {
2619 message: "GitHub project items have no pageInfo".into(),
2620 })?;
2621 let has_next = required_bool(info, "hasNextPage")?;
2622 let next = has_next
2623 .then(|| required_str(info, "endCursor"))
2624 .transpose()?;
2625 board = page.clone();
2626 match next {
2627 Some(next) => {
2628 validate_cursor_progress(after.as_deref(), next)?;
2629 after = Some(next.to_owned());
2630 }
2631 None => break,
2632 }
2633 }
2634 Ok(Board {
2635 id: required_str(&board, "id")?.to_owned(),
2636 fields: board.get("fields").cloned().unwrap_or(Value::Null),
2637 items,
2638 })
2639 }
2640
2641 /// The items this source has created, for completing a board read that is behind.
2642 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
2643 self.created.lock().map_err(|_| SourceError::Unavailable {
2644 message: "this source's record of what it created in this run was left \
2645 inconsistent by an earlier failure; next: run the command again"
2646 .into(),
2647 })
2648 }
2649
2650 /// One board item as this source reports it, or `None` for content it ignores.
2651 ///
2652 /// A pull request is neither a project nor a task — it is somebody's change, not a
2653 /// unit of plan — and an item whose content the token cannot see has nothing to
2654 /// report at all.
2655 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
2656 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
2657 message: "GitHub project item is missing content".into(),
2658 })?;
2659 if content.is_null() {
2660 return Ok(None);
2661 }
2662 let content_kind = match required_str(content, "__typename")? {
2663 "Issue" => ContentKind::Issue,
2664 "DraftIssue" => ContentKind::DraftIssue,
2665 _ => return Ok(None),
2666 };
2667 let field_values = item
2668 .get("fieldValues")
2669 .ok_or_else(|| SourceError::Malformed {
2670 message: "GitHub project item is missing fieldValues".into(),
2671 })?;
2672 complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
2673 let nodes = field_values
2674 .get("nodes")
2675 .and_then(Value::as_array)
2676 .ok_or_else(|| SourceError::Malformed {
2677 message: "GitHub project item fieldValues.nodes is not an array".into(),
2678 })?;
2679 if let Some(labels) = content.get("labels") {
2680 complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
2681 }
2682 let (body, slot) = metadata_body(optional_str(content, "body")?.map(str::to_owned))?;
2683 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
2684 .map(|id| NativeId(id.to_owned()));
2685 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
2686 // to read one from; it is a task, and never a project.
2687 let sub_issues = match content_kind {
2688 ContentKind::Issue => sub_issue_total(content)?,
2689 ContentKind::DraftIssue => 0,
2690 };
2691 let content_id = required_str(content, "id")?;
2692 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
2693 message: format!("GitHub issue {content_id}: {message}"),
2694 })?;
2695 let raw_title = required_str(content, "title")?;
2696 // The design prefix is read *first*, before either of the two rules that separate
2697 // a project from a task. A document is not work whatever sub-issues it has and
2698 // whatever marker it carries, and reading the prefix later would make a design
2699 // issue with none of either an empty project.
2700 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
2701 BoardKind::Document
2702 } else if parent.is_some() {
2703 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
2704 // under a project is that project's task even when it has sub-issues of its
2705 // own.
2706 BoardKind::Work(ItemKind::Task)
2707 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
2708 BoardKind::Work(ItemKind::Project)
2709 } else {
2710 BoardKind::Work(ItemKind::Task)
2711 };
2712 // The title a person wrote, which for a document is the one without the prefix —
2713 // the same way `content` above is the body without this source's metadata slot.
2714 let title = match kind {
2715 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
2716 BoardKind::Work(_) => raw_title.to_owned(),
2717 };
2718 let own_repository = content
2719 .pointer("/repository/nameWithOwner")
2720 .and_then(Value::as_str)
2721 .map(|origin| Repository::try_from(format!("{}/{origin}", RepositoryTarget::HOST)))
2722 .transpose()
2723 .map_err(|message| SourceError::Malformed { message })?;
2724 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
2725 Repository::from_metadata(&slot)
2726 .map_err(|message| SourceError::Malformed { message })?
2727 } else {
2728 own_repository.clone().into_iter().collect()
2729 };
2730 Ok(Some(Resolved {
2731 item_id: required_str(item, "id")?.to_owned(),
2732 id: NativeId(content_id.to_owned()),
2733 content_kind,
2734 kind,
2735 title,
2736 body: body.filter(|value| !value.is_empty()),
2737 status: self.status(item, content)?,
2738 labels: labels(content)?,
2739 parent,
2740 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
2741 url: optional_str(content, "url")?.map(str::to_owned),
2742 created_at: optional_time(content, "createdAt")?,
2743 updated_at: optional_time(content, "updatedAt")?,
2744 own_repository,
2745 repositories,
2746 slot,
2747 // Present when the item was reached through its own issue, whose board entry
2748 // names the board; a read of the board's own items has the board already. An
2749 // empty id names nothing a field write could address, so it is read as absent and
2750 // the write goes back to reading the board.
2751 board_id: item
2752 .pointer("/project/id")
2753 .and_then(Value::as_str)
2754 .filter(|id| !id.is_empty())
2755 .map(str::to_owned),
2756 fields: field_definitions(nodes),
2757 }))
2758 }
2759
2760 /// The status one board item reports.
2761 ///
2762 /// The closed state decides the category and the `Status` option decides the name, so
2763 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
2764 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
2765 /// duplicate is not finished work, and calling it done is a lie the next copy would
2766 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
2767 /// it is read permissively rather than refused — reads are faithful, and refusals
2768 /// belong on writes.
2769 fn status(&self, item: &Value, content: &Value) -> Result<Status, SourceError> {
2770 let nodes = item
2771 .pointer("/fieldValues/nodes")
2772 .and_then(Value::as_array)
2773 .expect("resolve validates fieldValues.nodes before mapping status");
2774 let option = nodes
2775 .iter()
2776 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
2777 .map(|value| required_str(value, "name"))
2778 .transpose()?;
2779 let state = optional_str(content, "state")?;
2780 if state == Some("CLOSED") {
2781 let category = match optional_str(content, "stateReason")? {
2782 None | Some("COMPLETED") => StatusCategory::Done,
2783 Some("NOT_PLANNED") => StatusCategory::Cancelled,
2784 Some(_) => StatusCategory::Unknown,
2785 };
2786 let fallback = match category {
2787 StatusCategory::Done => "Done",
2788 StatusCategory::Cancelled => "Cancelled",
2789 _ => "Closed",
2790 };
2791 return Ok(Status {
2792 category,
2793 name: option.unwrap_or(fallback).to_owned(),
2794 });
2795 }
2796 let name = option.unwrap_or("Open").to_owned();
2797 Ok(Status {
2798 category: self
2799 .statuses
2800 .category_of(&name)
2801 .unwrap_or(StatusCategory::Unknown),
2802 name,
2803 })
2804 }
2805
2806 /// The board Status option this write selects, or the refusal that says why not.
2807 ///
2808 /// For a column target the option is what the status *is*, so a board that has no such
2809 /// option is a refusal naming the status and the instance. For a closed target the
2810 /// issue's own state carries the category, and the option carries only the name a
2811 /// reader reports — so an option spelled the way this status is spelled is selected
2812 /// when the board has one, and nothing is refused when it does not.
2813 fn column_for(
2814 &self,
2815 board: &Board,
2816 status: &Status,
2817 target: &StatusTarget,
2818 ) -> Result<Option<(String, String)>, SourceError> {
2819 let (wanted, required) = match target {
2820 StatusTarget::Column(wanted) => (wanted.as_str(), true),
2821 StatusTarget::Closed(_) => (status.name.as_str(), false),
2822 StatusTarget::Disabled => return Ok(None),
2823 };
2824 let missing = |detail: &str| SourceError::Refused {
2825 message: format!(
2826 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
2827 category_name(status.category),
2828 self.name,
2829 category_name(status.category)
2830 ),
2831 };
2832 let Some(field) = Board::field(&board.fields, "Status")? else {
2833 return if required {
2834 Err(missing("this board has no Status field"))
2835 } else {
2836 Ok(None)
2837 };
2838 };
2839 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
2840 return if required {
2841 Err(missing(
2842 "this board's Status field is not a single-select field",
2843 ))
2844 } else {
2845 Ok(None)
2846 };
2847 }
2848 let option = field
2849 .get("options")
2850 .and_then(Value::as_array)
2851 .and_then(|options| {
2852 options.iter().find(|option| {
2853 option
2854 .get("name")
2855 .and_then(Value::as_str)
2856 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
2857 })
2858 });
2859 match option {
2860 None if required => Err(missing("this board does not have it")),
2861 None => Ok(None),
2862 Some(option) => Ok(Some((
2863 required_str(field, "id")?.to_owned(),
2864 required_str(option, "id")?.to_owned(),
2865 ))),
2866 }
2867 }
2868
2869 /// This instance's target for a category, refusing one it has disabled.
2870 ///
2871 /// Nothing here mutates the board's option set to make room for a status. GitHub
2872 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
2873 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
2874 /// field and every item's status.
2875 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
2876 let target = self.statuses.target(category).clone();
2877 if target != StatusTarget::Disabled {
2878 return Ok(target);
2879 }
2880 Err(SourceError::Refused {
2881 message: if category == StatusCategory::Draft {
2882 format!(
2883 "status draft is disabled for source {}: draft is incompatible with this \
2884 integration because GitHub draft issues cannot have sub-issues, and this \
2885 source stores a project's tasks as its issue's sub-issues",
2886 self.name
2887 )
2888 } else if category == StatusCategory::Unknown {
2889 format!(
2890 "status {} is disabled for source {}; set status_mapping.{} of this source \
2891 to one board Status option name; every word classified unknown is written \
2892 to that one option",
2893 category_name(category),
2894 self.name,
2895 category_name(category)
2896 )
2897 } else {
2898 format!(
2899 "status {} is disabled for source {}; set status_mapping.{} of this source \
2900 to a board Status option name or to a closed state",
2901 category_name(category),
2902 self.name,
2903 category_name(category)
2904 )
2905 },
2906 })
2907 }
2908
2909 async fn set_item_field(
2910 &self,
2911 board_id: &str,
2912 item_id: &str,
2913 field_id: &str,
2914 value: Value,
2915 ) -> Result<(), SourceError> {
2916 let data = self
2917 .graphql(
2918 graphql::UPDATE_FIELD,
2919 json!({"input":{
2920 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
2921 }}),
2922 )
2923 .await?;
2924 let returned = data
2925 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
2926 .ok_or_else(|| SourceError::Malformed {
2927 message: "GitHub field update returned no project item".into(),
2928 })?;
2929 if required_str(returned, "id")? != item_id {
2930 return Err(SourceError::Malformed {
2931 message: "GitHub field update returned the wrong project item".into(),
2932 });
2933 }
2934 Ok(())
2935 }
2936
2937 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
2938 let mut after: Option<String> = None;
2939 let mut ids = Vec::new();
2940 loop {
2941 let data = self
2942 .graphql(
2943 graphql::ISSUE_DEPENDENCIES,
2944 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
2945 )
2946 .await?;
2947 let connection =
2948 data.pointer("/node/blockedBy")
2949 .ok_or_else(|| SourceError::Malformed {
2950 message: "GitHub dependency response has no blockedBy connection".into(),
2951 })?;
2952 ids.extend(
2953 connection
2954 .get("nodes")
2955 .and_then(Value::as_array)
2956 .ok_or_else(|| SourceError::Malformed {
2957 message: "GitHub dependency response nodes is not an array".into(),
2958 })?
2959 .iter()
2960 .map(|value| required_str(value, "id").map(str::to_owned))
2961 .collect::<Result<Vec<_>, _>>()?,
2962 );
2963 let next = next_cursor(connection)?;
2964 if let Some(next) = &next {
2965 validate_cursor_progress(after.as_deref(), &next.0)?;
2966 }
2967 after = next.map(|cursor| cursor.0);
2968 if after.is_none() {
2969 return Ok(ids);
2970 }
2971 }
2972 }
2973
2974 async fn dependencies(
2975 &self,
2976 id: &NativeId,
2977 near_kind: ItemKind,
2978 direction: Direction,
2979 page: &PageRequest,
2980 ) -> Result<Page<DependencyEdge>, SourceError> {
2981 validate_page(page)?;
2982 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
2983 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
2984 let recorded = recorded_offset(cursor, direction)?;
2985 // Asked for even in the recorded phase, whose page reads nothing from the
2986 // connection: `__typename` is what says whether this item has a native
2987 // relationship at all, and that is what decides which far ends the reserved key is
2988 // allowed to hold.
2989 let data = self
2990 .graphql(
2991 graphql::ISSUE_DEPENDENCIES,
2992 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
2993 "after":if recorded.is_some() {None} else {cursor}}),
2994 )
2995 .await?;
2996 let node =
2997 data.get("node")
2998 .filter(|v| !v.is_null())
2999 .ok_or_else(|| SourceError::Refused {
3000 message: format!(
3001 "GitHub item {} was not found or does not support dependencies",
3002 id.0
3003 ),
3004 })?;
3005 let connection_name = match direction {
3006 Direction::DependsOn => "blockedBy",
3007 Direction::DependedOnBy => "blocking",
3008 };
3009 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
3010 // named natively and the reserved key may hold any far end. An issue's connections
3011 // hold issues, and this source reads them at the near item's own level.
3012 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
3013 if let Some(offset) = recorded {
3014 return Ok(recorded_page(
3015 self.recorded_edges(id, near_kind, direction, natively_names, node)
3016 .await?,
3017 offset,
3018 limit,
3019 ));
3020 }
3021 if natively_names.is_none() {
3022 return Ok(recorded_page(
3023 self.recorded_edges(id, near_kind, direction, natively_names, node)
3024 .await?,
3025 0,
3026 limit,
3027 ));
3028 }
3029 let connection = node
3030 .get(connection_name)
3031 .ok_or_else(|| SourceError::Malformed {
3032 message: "GitHub dependency response is missing its connection".into(),
3033 })?;
3034 let nodes = connection
3035 .get("nodes")
3036 .and_then(Value::as_array)
3037 .ok_or_else(|| SourceError::Malformed {
3038 message: "GitHub dependency response nodes is not an array".into(),
3039 })?;
3040 // `from` depends on `to`, always. GitHub spells the same relationship from either
3041 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
3042 // it — so the near item is `from` in one direction and `to` in the other.
3043 let items = nodes
3044 .iter()
3045 .map(|value| {
3046 let related = NativeId(required_str(value, "id")?.into());
3047 let related_kind = related_kind(value)?;
3048 let (from, to) = match direction {
3049 Direction::DependsOn => (
3050 DependencyEndpoint::from_native(id.clone(), near_kind),
3051 DependencyEndpoint::from_native(related, related_kind),
3052 ),
3053 Direction::DependedOnBy => (
3054 DependencyEndpoint::from_native(related, related_kind),
3055 DependencyEndpoint::from_native(id.clone(), near_kind),
3056 ),
3057 };
3058 Ok(DependencyEdge {
3059 from,
3060 to,
3061 kind: DependencyKind::Blocks,
3062 })
3063 })
3064 .collect::<Result<Vec<_>, SourceError>>()?;
3065 let mut next = next_cursor(connection)?;
3066 if let Some(next) = &next {
3067 validate_cursor_progress(cursor, &next.0)?;
3068 }
3069 if next.is_none()
3070 && !self
3071 .recorded_edges(id, near_kind, direction, natively_names, node)
3072 .await?
3073 .is_empty()
3074 {
3075 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
3076 }
3077 Ok(Page { items, next })
3078 }
3079
3080 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
3081 /// a far end in another source has to live: no GitHub issue relationship can name one.
3082 ///
3083 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
3084 /// source never writes one down.
3085 ///
3086 /// The metadata lives in the item's own body slot, and `node` is the dependency read's
3087 /// own answer, which carries an issue's body — so an issue's recorded edges cost no
3088 /// request beyond the read already made, and reading the board for them would be a
3089 /// walk of every item for one field of one. A draft has no body in that answer, because
3090 /// a draft lives only inside the board, so a draft's are read off the board as before.
3091 async fn recorded_edges(
3092 &self,
3093 id: &NativeId,
3094 near_kind: ItemKind,
3095 direction: Direction,
3096 natively_names: Option<ItemKind>,
3097 node: &Value,
3098 ) -> Result<Vec<DependencyEdge>, SourceError> {
3099 if direction != Direction::DependsOn {
3100 return Ok(Vec::new());
3101 }
3102 let slot = match node.get("body") {
3103 Some(body) if natively_names.is_some() => {
3104 metadata_body(body.as_str().map(str::to_owned))?.1
3105 }
3106 _ => {
3107 let Some(item) = self
3108 .board()
3109 .await?
3110 .items
3111 .into_iter()
3112 .find(|item| item.id == *id)
3113 else {
3114 return Ok(Vec::new());
3115 };
3116 item.slot
3117 }
3118 };
3119 DependencyEdge::recorded(&slot, id, near_kind, &self.name, natively_names)
3120 .map_err(|message| SourceError::Malformed { message })
3121 }
3122
3123 fn configured_repository(&self) -> Result<&RepositoryTarget, SourceError> {
3124 self.repository
3125 .as_ref()
3126 .ok_or_else(|| SourceError::Refused {
3127 message: format!(
3128 "source {} has no repository configured, and a GitHub Projects board has no \
3129 repository of its own to create an issue in; set repository: owner/name on \
3130 this source",
3131 self.name
3132 ),
3133 })
3134 }
3135
3136 /// The repository one new issue is created in, under the rule [`RepositoryTarget`]
3137 /// states.
3138 ///
3139 /// The fallback is demanded first, whichever arm answers: a write without a configured
3140 /// repository is refused naming the field exactly as it was before the rule existed,
3141 /// so a source that could not write before cannot write now, rather than writing for
3142 /// the one item whose own field happens to decide it.
3143 ///
3144 /// Everything this refuses is refused before `createIssue`, so a refusal leaves no
3145 /// issue behind: an entry that is not a repository on [`RepositoryTarget::HOST`], an
3146 /// entry owned by someone other than the owner of the parent issue's repository —
3147 /// GitHub accepts a sub-issue from another repository of the same owner and from no
3148 /// other, so `addSubIssue` would refuse it after the issue existed — a parent the
3149 /// board does not hold, and a parent that is a draft, which GitHub gives no sub-issues,
3150 /// both of which `addSubIssue` would likewise refuse too late. Whether the entry exists
3151 /// and is visible to the token is checked where its node id is resolved, still before
3152 /// `createIssue`. The parent is read off `board`, which is completed from
3153 /// this process's own record, so a project created moments ago in this command answers
3154 /// though GitHub's board read has not caught up.
3155 fn creation_target(
3156 &self,
3157 board: &Board,
3158 incoming: &Incoming<'_>,
3159 ) -> Result<RepositoryTarget, SourceError> {
3160 let fallback = self.configured_repository()?;
3161 let what = |incoming: &Incoming<'_>| {
3162 format!(
3163 "{} {:?}",
3164 incoming.written.kind().describes(),
3165 incoming.title
3166 )
3167 };
3168 let parent = incoming
3169 .parent
3170 .map(|parent| {
3171 board
3172 .items
3173 .iter()
3174 .find(|item| item.id == *parent)
3175 .ok_or_else(|| SourceError::Refused {
3176 message: format!(
3177 "GitHub project issue {} was not found on the board of source {}, \
3178 so {} cannot be filed under it",
3179 parent.0,
3180 self.name,
3181 what(incoming)
3182 ),
3183 })
3184 })
3185 .transpose()?;
3186 let parents_repository = parent
3187 .map(|parent| {
3188 // A draft is on the board and so is found, but it has no repository to
3189 // place a task in and GitHub gives it no sub-issues, so `addSubIssue`
3190 // would refuse the task only once `createIssue` had made it.
3191 if parent.content_kind == ContentKind::DraftIssue {
3192 return Err(SourceError::Refused {
3193 message: format!(
3194 "GitHub project item {} on the board of source {} is a draft, \
3195 which cannot have sub-issues, so {} cannot be filed under it",
3196 parent.id.0,
3197 self.name,
3198 what(incoming)
3199 ),
3200 });
3201 }
3202 // An issue's repository is where a sub-issue is placed and whose owner it
3203 // is compared against, so a parent whose repository this source cannot
3204 // spell as `owner/name` — GitHub's login grammar is wider than this
3205 // source's floor — is one nothing can be filed under.
3206 parent
3207 .own_repository
3208 .as_ref()
3209 .and_then(|origin| RepositoryTarget::from_origin(origin).ok())
3210 .ok_or_else(|| SourceError::Malformed {
3211 message: format!(
3212 "GitHub project issue {} on the board of source {} is in {}, which \
3213 is not a {}/owner/name repository this source can place {} in",
3214 parent.id.0,
3215 self.name,
3216 parent
3217 .own_repository
3218 .as_ref()
3219 .map_or("no repository", Repository::as_str),
3220 RepositoryTarget::HOST,
3221 what(incoming)
3222 ),
3223 })
3224 })
3225 .transpose()?;
3226 match incoming.repositories {
3227 [named] => {
3228 let target =
3229 RepositoryTarget::from_origin(named).map_err(|_| SourceError::Refused {
3230 message: format!(
3231 "{} names repository {}, which is not a {}/owner/name repository \
3232 source {} can create an issue in; name one that is, or name none",
3233 what(incoming),
3234 named.as_str(),
3235 RepositoryTarget::HOST,
3236 self.name
3237 ),
3238 })?;
3239 if let Some(parents) = &parents_repository
3240 && parents.owner != target.owner
3241 {
3242 return Err(SourceError::Refused {
3243 message: format!(
3244 "{} names repository {}, owned by {}, but its project's issue is in \
3245 {}, owned by {}, and GitHub files a sub-issue only in a repository \
3246 of the same owner as its parent issue; name a repository of {}, or \
3247 name none",
3248 what(incoming),
3249 target.slug(),
3250 target.owner,
3251 parents.slug(),
3252 parents.owner,
3253 parents.owner
3254 ),
3255 });
3256 }
3257 Ok(target)
3258 }
3259 _ => Ok(parents_repository.unwrap_or_else(|| fallback.clone())),
3260 }
3261 }
3262
3263 /// The node id of the repository `incoming` is being created in, or the refusal naming
3264 /// the item and the repository the token cannot see.
3265 ///
3266 /// Resolved once per command per repository; see [`Self::repository_cache`].
3267 async fn repository_id(
3268 &self,
3269 repository: &RepositoryTarget,
3270 incoming: &Incoming<'_>,
3271 ) -> Result<String, SourceError> {
3272 if let Some(id) = self.repository_cache()?.get(repository).cloned() {
3273 return Ok(id);
3274 }
3275 let data = self
3276 .graphql(
3277 graphql::REPOSITORY,
3278 json!({"owner":repository.owner,"name":repository.name}),
3279 )
3280 .await?;
3281 let node = data
3282 .get("repository")
3283 .filter(|value| !value.is_null())
3284 .ok_or_else(|| SourceError::Refused {
3285 message: format!(
3286 "GitHub repository {} was not found or is not visible to the token, so {} \
3287 {:?} cannot be created in it",
3288 repository.slug(),
3289 incoming.written.kind().describes(),
3290 incoming.title
3291 ),
3292 })?;
3293 let id = required_str(node, "id")?.to_owned();
3294 self.repository_cache()?
3295 .insert(repository.clone(), id.clone());
3296 Ok(id)
3297 }
3298
3299 fn repository_cache(
3300 &self,
3301 ) -> Result<std::sync::MutexGuard<'_, BTreeMap<RepositoryTarget, String>>, SourceError> {
3302 self.repository_cache
3303 .lock()
3304 .map_err(|_| SourceError::Unavailable {
3305 message: "this source's record of the destination repository was left \
3306 inconsistent by an earlier failure; next: run the command again"
3307 .into(),
3308 })
3309 }
3310
3311 /// Create or update one board item, whichever kind it is.
3312 async fn write_item(
3313 &self,
3314 incoming: &Incoming<'_>,
3315 target: Option<&NativeId>,
3316 depends_on: &[DependencyEdge],
3317 ) -> Result<NativeId, SourceError> {
3318 // Refused before anything is read or written: a task or a project titled the way
3319 // this board spells a document would land as an issue this same source reads back
3320 // as a document, so the field this destination cannot carry is named rather than
3321 // written and silently reclassified.
3322 if let Written::Work(kind, _) = incoming.written
3323 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
3324 {
3325 return Err(SourceError::Refused {
3326 message: format!(
3327 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
3328 spells a document, so it would read back as one rather than as a {}; \
3329 retitle it, or copy it as a document",
3330 kind.marker(),
3331 self.name,
3332 kind.marker()
3333 ),
3334 });
3335 }
3336 let board = match target {
3337 Some(target) => match self.board_for_update(target, incoming, depends_on).await? {
3338 Some(board) => board,
3339 None => self.board().await?,
3340 },
3341 None => self.board().await?,
3342 };
3343 let status_target = incoming
3344 .written
3345 .status()
3346 .map(|status| self.resolved_target(status.category))
3347 .transpose()?;
3348 let column = match (incoming.written.status(), status_target.as_ref()) {
3349 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
3350 _ => None,
3351 };
3352 let existing = target
3353 .map(|target| {
3354 board
3355 .items
3356 .iter()
3357 .find(|item| item.id == *target)
3358 .ok_or_else(|| SourceError::Refused {
3359 message: format!("GitHub destination item {} was not found", target.0),
3360 })
3361 })
3362 .transpose()?;
3363 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
3364 if content_kind == ContentKind::DraftIssue {
3365 if let (Some(StatusTarget::Closed(_)), Some(status)) =
3366 (status_target.as_ref(), incoming.written.status())
3367 {
3368 return Err(SourceError::Refused {
3369 message: format!(
3370 "status {} of source {} closes the item's issue, and GitHub draft items \
3371 have no open or closed state",
3372 category_name(status.category),
3373 self.name
3374 ),
3375 });
3376 }
3377 if incoming.parent.is_some() {
3378 return Err(SourceError::Refused {
3379 message: "GitHub draft items cannot be a project's sub-issue".into(),
3380 });
3381 }
3382 }
3383 match existing {
3384 Some(item) if content_kind == ContentKind::Issue => {
3385 if item.labels != incoming.labels {
3386 return Err(SourceError::Refused {
3387 message: "GitHub issue labels differ from the labels being written".into(),
3388 });
3389 }
3390 }
3391 _ => {
3392 if !incoming.labels.is_empty() {
3393 return Err(SourceError::Refused {
3394 message: "GitHub items created by this destination carry no labels".into(),
3395 });
3396 }
3397 }
3398 }
3399
3400 // An existing issue is never moved; a new one is created where the rule says. The
3401 // repository the issue really lives in is what the slot below is written against,
3402 // so a single entry that is where the issue is created travels as no key at all,
3403 // and the read side derives it back from the issue.
3404 let (own_repository, creation_target) = match existing {
3405 Some(item) => (item.own_repository.clone(), None),
3406 None => {
3407 let target = self.creation_target(&board, incoming)?;
3408 let origin = Repository::try_from(target.origin())
3409 .map_err(|message| SourceError::Config { message })?;
3410 (Some(origin), Some(target))
3411 }
3412 };
3413 let (native, fallback) = self
3414 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
3415 .await?;
3416 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
3417 let body = compose_body(incoming.content, &slot)?;
3418 // Read before anything is created, for the reason the field below is: a value
3419 // this destination cannot store has to refuse, and refusing after `createIssue`
3420 // would leave an issue behind that nothing asked for. The engine writes a
3421 // qualified id here; a caller handing this key anything else is told so rather
3422 // than having it silently stored as no origin at all.
3423 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
3424 let origin = match incoming.metadata.get(ORIGIN_KEY) {
3425 None => "",
3426 Some(Value::String(origin)) => origin.as_str(),
3427 Some(other) => {
3428 return Err(SourceError::Refused {
3429 message: format!(
3430 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
3431 is {other}"
3432 ),
3433 });
3434 }
3435 };
3436 // Resolved before anything is created: a board that cannot carry the copy origin
3437 // has to refuse the write, and refusing it after `createIssue` would leave an
3438 // issue behind that nothing asked for.
3439 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
3440 Some(field) => {
3441 if required_str(field, "__typename")? != "ProjectV2Field" {
3442 return Err(SourceError::Refused {
3443 message: format!(
3444 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
3445 ),
3446 });
3447 }
3448 Some(required_str(field, "id")?.to_owned())
3449 }
3450 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
3451 return Err(SourceError::Refused {
3452 message: format!(
3453 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
3454 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
3455 the board"
3456 ),
3457 });
3458 }
3459 None => None,
3460 };
3461
3462 let (content_id, item_id, url) = match existing {
3463 Some(item) => {
3464 self.update_existing(item, incoming, &body, status_target.as_ref())
3465 .await?;
3466 (item.id.clone(), item.item_id.clone(), item.url.clone())
3467 }
3468 None => {
3469 let target = creation_target
3470 .as_ref()
3471 .ok_or_else(|| SourceError::Malformed {
3472 message: "a new item was decided without a repository to create it in"
3473 .into(),
3474 })?;
3475 self.create_and_file_issue(&board, target, incoming, &body, status_target.as_ref())
3476 .await?
3477 }
3478 };
3479
3480 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
3481 // then each board field, the parent and the dependencies — and GitHub can fail at
3482 // any of them. Everything this source can refuse *before* the first of those is
3483 // already checked above, so what is left is GitHub itself failing part way. When it
3484 // does over an item this call created, the issue is taken back: a write that
3485 // refused must not leave an item behind that nobody asked for, and one that does
3486 // makes the retry create a second.
3487 let landed = self
3488 .finish_write(
3489 &board,
3490 incoming,
3491 &content_id,
3492 &item_id,
3493 content_kind,
3494 existing,
3495 origin_field.as_deref(),
3496 origin,
3497 column,
3498 &native,
3499 )
3500 .await;
3501 if let Err(error) = landed {
3502 if existing.is_none() {
3503 // Best effort, and the write's own failure is what the caller is told: a
3504 // refusal naming the tidy-up would hide why the write failed at all.
3505 let _ = self.delete_issue(&content_id).await;
3506 }
3507 return Err(error);
3508 }
3509
3510 // So the rest of this command reads what it just did rather than what the board
3511 // said before it. See `remember_written` for which half takes it.
3512 let remembered = Resolved {
3513 item_id,
3514 id: content_id.clone(),
3515 content_kind,
3516 kind: incoming.written.kind(),
3517 title: incoming.title.to_owned(),
3518 // The visible half of the body this write composed, split back off it the
3519 // way a read splits it — so what this record reports is what a read of the
3520 // same issue reports, rather than the person's text with the metadata slot
3521 // still on the end of it.
3522 body: metadata_body(body.clone())?.0,
3523 // A document has no status of its own; what it reads back as is whatever
3524 // the issue's own state says, which is what a re-read reports.
3525 status: incoming
3526 .written
3527 .status()
3528 .cloned()
3529 .unwrap_or_else(|| Status {
3530 category: StatusCategory::Unknown,
3531 name: "Open".to_owned(),
3532 }),
3533 labels: incoming.labels.to_vec(),
3534 parent: incoming.parent.cloned(),
3535 origin: (!origin.is_empty()).then(|| origin.to_owned()),
3536 // In the update path this is the item's own url, read off `existing` where the
3537 // tuple above was bound, so one expression serves both halves.
3538 url,
3539 created_at: existing.and_then(|item| item.created_at),
3540 updated_at: existing.and_then(|item| item.updated_at),
3541 own_repository,
3542 repositories: incoming.repositories.to_vec(),
3543 slot,
3544 board_id: Some(board.id.clone()),
3545 fields: board
3546 .fields
3547 .get("nodes")
3548 .and_then(Value::as_array)
3549 .cloned()
3550 .unwrap_or_default(),
3551 };
3552 self.remember_written(remembered, existing.is_none())?;
3553 Ok(content_id)
3554 }
3555
3556 /// What an update of an item this board already holds needs of the board, read off that
3557 /// item rather than off the board — or `None` when the item cannot say enough, and the
3558 /// board has to be read after all.
3559 ///
3560 /// An update needs the item it updates, the board's own id for a field write, the
3561 /// definitions of the `Status` and origin fields it writes, and every same-source far end
3562 /// its edges name. The first three ride along on the item's own node read — its board
3563 /// entry names the board's id, and each field value on it carries the definition of the
3564 /// field it is a value of — and each far end is read by its own node id. So updating one
3565 /// item costs a read of that item rather than every page of the board, which is what a
3566 /// copy naming one member out of many is for.
3567 ///
3568 /// **Nothing the item does not say is guessed.** A field this item holds no value of may
3569 /// still be on the board, and a view that read it as absent would refuse a write the
3570 /// board can take, or skip a field write the board needs. So a board this command has
3571 /// already read is used as it is, and an item that does not name its board, holds no
3572 /// value of the origin field, or holds no `Status` value when the write carries a status,
3573 /// sends the write back to reading the board — exactly as every update read it before.
3574 async fn board_for_update(
3575 &self,
3576 target: &NativeId,
3577 incoming: &Incoming<'_>,
3578 depends_on: &[DependencyEdge],
3579 ) -> Result<Option<Board>, SourceError> {
3580 if self.board_cache()?.is_some() {
3581 return Ok(None);
3582 }
3583 let Some(existing) = self.item_by_id(target).await? else {
3584 return Ok(None);
3585 };
3586 let Some(board_id) = existing.board_id.clone() else {
3587 return Ok(None);
3588 };
3589 let defines = |name: &str| {
3590 existing
3591 .fields
3592 .iter()
3593 .any(|field| field.get("name").and_then(Value::as_str) == Some(name))
3594 };
3595 if !defines(ORIGIN_FIELD) || (incoming.written.status().is_some() && !defines("Status")) {
3596 return Ok(None);
3597 }
3598 let fields = json!({"nodes": existing.fields, "pageInfo": {"hasNextPage": false}});
3599 let mut items = vec![existing];
3600 for edge in depends_on {
3601 if !edge
3602 .to
3603 .source()
3604 .is_none_or(|source| source == self.name.as_str())
3605 {
3606 continue;
3607 }
3608 // The first colon, for the reason `partition_edges` gives.
3609 let far = NativeId(if edge.to.is_qualified() {
3610 edge.to
3611 .id()
3612 .split_once(':')
3613 .map_or(edge.to.id(), |(_, native)| native)
3614 .to_owned()
3615 } else {
3616 edge.to.id().to_owned()
3617 });
3618 if items.iter().any(|item| item.id == far) {
3619 continue;
3620 }
3621 match self.item_by_id(&far).await? {
3622 Some(item) => items.push(item),
3623 // Refused against the whole board, in the words that refusal has always had.
3624 None => return Ok(None),
3625 }
3626 }
3627 Ok(Some(Board {
3628 id: board_id,
3629 fields,
3630 items,
3631 }))
3632 }
3633
3634 /// Everything a write does after the item exists: its board fields, its parent, and
3635 /// its dependencies.
3636 ///
3637 /// Split out of `write_item` so there is one place a failure past the point of no
3638 /// return is caught, rather than a tidy-up repeated at each `?` above.
3639 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
3640 // so there is one place a failure past the point of no return is caught, and its
3641 // arguments are exactly the values that tail already had in scope. Bundling them into a
3642 // struct would describe no concept — it would be "the arguments of this function" — and
3643 // would put the whole of `write_item`'s locals behind one more indirection.
3644 #[allow(clippy::too_many_arguments)]
3645 async fn finish_write(
3646 &self,
3647 board: &Board,
3648 incoming: &Incoming<'_>,
3649 content_id: &NativeId,
3650 item_id: &str,
3651 content_kind: ContentKind,
3652 existing: Option<&Resolved>,
3653 origin_field: Option<&str>,
3654 origin: &str,
3655 column: Option<(String, String)>,
3656 native: &[String],
3657 ) -> Result<(), SourceError> {
3658 if let Some(field_id) = origin_field {
3659 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
3660 .await?;
3661 }
3662
3663 if let Some((field_id, option_id)) = column {
3664 self.set_item_field(
3665 &board.id,
3666 item_id,
3667 &field_id,
3668 json!({"singleSelectOptionId":option_id}),
3669 )
3670 .await?;
3671 }
3672
3673 if content_kind == ContentKind::Issue {
3674 self.reparent(
3675 existing.and_then(|item| item.parent.clone()),
3676 content_id,
3677 incoming.parent,
3678 )
3679 .await?;
3680 // A document takes part in no dependency graph, so writing one neither reads
3681 // nor changes the issue's own `blockedBy` relationships. Reconciling them
3682 // against the empty list a document write carries would *delete* whatever
3683 // relationships a person had made on that issue, which is a write nobody
3684 // asked for.
3685 if incoming.written.kind() != BoardKind::Document {
3686 self.reconcile_blocked_by(content_id, native).await?;
3687 }
3688 }
3689 Ok(())
3690 }
3691
3692 /// Delete one issue, which takes its board item with it.
3693 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
3694 let data = self
3695 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3696 .await?;
3697 data.pointer("/deleteIssue/repository")
3698 .filter(|value| !value.is_null())
3699 .ok_or_else(|| SourceError::Malformed {
3700 message: "GitHub issue deletion returned no repository".into(),
3701 })?;
3702 self.forget(id)?;
3703 Ok(())
3704 }
3705
3706 /// Remove one item this copy created, so a copy that could not finish leaves the board
3707 /// as it found it.
3708 ///
3709 /// Deleting the issue takes its board item with it, so there is no second mutation to
3710 /// keep in step. An id the board does not hold is not an error: the item is already
3711 /// gone, which is the state this asks for.
3712 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
3713 let board = self.board().await?;
3714 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
3715 return Ok(());
3716 };
3717 if item.content_kind == ContentKind::DraftIssue {
3718 return Err(SourceError::Refused {
3719 message: format!(
3720 "GitHub item {} is a draft, and this source removes an item by deleting \
3721 its issue; next: remove it from the board by hand",
3722 id.0
3723 ),
3724 });
3725 }
3726 let data = self
3727 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
3728 .await?;
3729 data.pointer("/deleteIssue/repository")
3730 .filter(|value| !value.is_null())
3731 .ok_or_else(|| SourceError::Malformed {
3732 message: "GitHub issue deletion returned no repository".into(),
3733 })?;
3734 self.forget(id)?;
3735 Ok(())
3736 }
3737
3738 /// The issue a comment call on `task` is about, or `None` when this board holds no such
3739 /// task.
3740 ///
3741 /// Resolved exactly as [`TaskSource::get_task`] resolves it, so the comment verbs and a
3742 /// read of the task cannot disagree about which ids name one: a project or a document of
3743 /// this board is not a task here either.
3744 ///
3745 /// A **draft** is a task with nowhere to keep a comment, because GitHub keeps comments on
3746 /// issues and a draft is not one. It is refused rather than answered with an empty page,
3747 /// which would read as a task nobody has commented on yet.
3748 async fn commented_issue(&self, task: &NativeId) -> Result<Option<NativeId>, SourceError> {
3749 let Some(item) = self
3750 .item_by_id(task)
3751 .await?
3752 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3753 else {
3754 return Ok(None);
3755 };
3756 if item.content_kind == ContentKind::DraftIssue {
3757 return Err(SourceError::Refused {
3758 message: format!(
3759 "task {} of source {} is a draft item on the board, and GitHub keeps \
3760 comments on issues alone, so a draft has none to read or write; next: \
3761 convert the draft to an issue on the board, then comment on the issue it \
3762 becomes",
3763 task.0, self.name
3764 ),
3765 });
3766 }
3767 Ok(Some(item.id))
3768 }
3769
3770 /// Whether the comment `comment` is one of `issue`'s own.
3771 ///
3772 /// Read before an edit or a removal is sent, because GitHub's comment mutations take the
3773 /// comment's id and nothing else: a comment id given against the wrong task would
3774 /// otherwise change a comment on some other issue entirely. An id that names nothing, or
3775 /// names something that is not an issue comment, is a comment this task does not have —
3776 /// which is what GitHub refusing to resolve it means too.
3777 async fn comment_is_on(
3778 &self,
3779 issue: &NativeId,
3780 comment: &NativeId,
3781 ) -> Result<bool, SourceError> {
3782 let asked = self
3783 .graphql(graphql::COMMENT_ISSUE, json!({"id":comment.0}))
3784 .await;
3785 let data = match asked {
3786 Ok(data) => data,
3787 Err(error) if unresolvable_node(&error) => return Ok(false),
3788 Err(error) => return Err(error),
3789 };
3790 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
3791 return Ok(false);
3792 };
3793 if optional_str(node, "__typename")? != Some("IssueComment") {
3794 return Ok(false);
3795 }
3796 let on = node.get("issue").ok_or_else(|| SourceError::Malformed {
3797 message: format!("GitHub issue comment {} names no issue", comment.0),
3798 })?;
3799 Ok(required_str(on, "id")? == issue.0)
3800 }
3801
3802 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
3803 async fn partition_edges(
3804 &self,
3805 board: &Board,
3806 near_kind: BoardKind,
3807 near_content: ContentKind,
3808 depends_on: &[DependencyEdge],
3809 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
3810 let mut native = Vec::new();
3811 let mut fallback = Vec::new();
3812 for edge in depends_on {
3813 let same_source = edge
3814 .to
3815 .source()
3816 .is_none_or(|source| source == self.name.as_str());
3817 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
3818 // `DependencyEndpoint::source` both read it that way — and a native id may hold
3819 // colons of its own, so the far end is everything after that one separator.
3820 // Splitting at the last would truncate `work:urn:task:7` to `7`.
3821 let far_id = if edge.to.is_qualified() {
3822 edge.to
3823 .id()
3824 .split_once(':')
3825 .map_or(edge.to.id(), |(_, native)| native)
3826 } else {
3827 edge.to.id()
3828 };
3829 let far = if same_source {
3830 Some(
3831 board
3832 .items
3833 .iter()
3834 .find(|item| item.id.0 == far_id)
3835 .ok_or_else(|| SourceError::Refused {
3836 message: format!("GitHub dependency item {far_id} was not found"),
3837 })?,
3838 )
3839 } else {
3840 None
3841 };
3842 // The caller says which kind the far end is, and this board holds the far end
3843 // itself, so a disagreement is settled here rather than stored: recorded, the
3844 // wrong kind would read back as a cross-level edge that never existed; written
3845 // natively, it would name a relationship of a different level than the caller
3846 // asked for.
3847 //
3848 // A far end this board holds as a *document* fails the same comparison and is
3849 // refused by the same sentence: `ItemKind` has no document variant because
3850 // nothing may point at one, so no caller can name it correctly and the refusal
3851 // is the only honest answer.
3852 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
3853 return Err(SourceError::Refused {
3854 message: format!(
3855 "GitHub dependency item {far_id} is a {} of this board, and this item \
3856 names it as a {}; record the kind it is",
3857 disagreeing.kind.describes(),
3858 edge.to.kind.marker()
3859 ),
3860 });
3861 }
3862 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
3863 // however the far end is spelled — and one classified native here would be
3864 // written nowhere at all, because a draft's native reconciliation never runs.
3865 let native_here = near_content == ContentKind::Issue
3866 && far.is_some_and(|far| {
3867 far.content_kind == ContentKind::Issue
3868 && BoardKind::Work(edge.to.kind) == near_kind
3869 });
3870 if native_here {
3871 native.push(far_id.to_owned());
3872 } else {
3873 fallback.push(edge.clone());
3874 }
3875 }
3876 Ok((native, fallback))
3877 }
3878
3879 async fn update_existing(
3880 &self,
3881 item: &Resolved,
3882 incoming: &Incoming<'_>,
3883 body: &Option<String>,
3884 status_target: Option<&StatusTarget>,
3885 ) -> Result<(), SourceError> {
3886 let title = incoming.written_title();
3887 let (operation, input, pointer) = match item.content_kind {
3888 ContentKind::DraftIssue => (
3889 graphql::UPDATE_DRAFT,
3890 json!({"draftIssueId":item.id.0,"title":title,"body":body}),
3891 "/updateProjectV2DraftIssue/draftIssue",
3892 ),
3893 ContentKind::Issue => (
3894 graphql::UPDATE_ISSUE,
3895 json!({"id":item.id.0,"title":title,"body":body,
3896 "stateInput":state_input(status_target)}),
3897 "/updateIssue/issue",
3898 ),
3899 };
3900 let data = self.graphql(operation, json!({"input":input})).await?;
3901 let returned = data
3902 .pointer(pointer)
3903 .ok_or_else(|| SourceError::Malformed {
3904 message: "GitHub item update returned no item".into(),
3905 })?;
3906 if required_str(returned, "id")? != item.id.0 {
3907 return Err(SourceError::Malformed {
3908 message: "GitHub item update returned the wrong item".into(),
3909 });
3910 }
3911 Ok(())
3912 }
3913
3914 /// Creates one issue, files it on the board, and closes it when the status says so.
3915 ///
3916 /// Three calls rather than one: `createIssue` needs a repository and answers with an
3917 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
3918 /// closed status is a state of the issue rather than a field of the board item.
3919 /// Creates the issue, files it on the board, and reports what a read of it would say:
3920 /// its content id, its board item id, and the web address GitHub gave it.
3921 ///
3922 /// The address comes back here because this is the only place it is known before
3923 /// GitHub's own board read catches up — an item this run created answers the reads
3924 /// that follow it out of the record below, and one remembered without its address
3925 /// would report no location for the rest of the run.
3926 async fn create_and_file_issue(
3927 &self,
3928 board: &Board,
3929 repository: &RepositoryTarget,
3930 incoming: &Incoming<'_>,
3931 body: &Option<String>,
3932 status_target: Option<&StatusTarget>,
3933 ) -> Result<(NativeId, String, Option<String>), SourceError> {
3934 let repository_id = self.repository_id(repository, incoming).await?;
3935 let data = self
3936 .graphql(
3937 graphql::CREATE_ISSUE,
3938 json!({"input":{
3939 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
3940 }}),
3941 )
3942 .await?;
3943 let created = data
3944 .pointer("/createIssue/issue")
3945 .filter(|value| !value.is_null())
3946 .ok_or_else(|| SourceError::Malformed {
3947 message: "GitHub issue creation returned no issue".into(),
3948 })?;
3949 let content_id = NativeId(required_str(created, "id")?.to_owned());
3950 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
3951 // a response without it is not worth failing a landed write over — the item simply
3952 // reports no location until the board read catches up, which is what it did before.
3953 let url = optional_str(created, "url")?.map(str::to_owned);
3954 // The issue exists from here on, so a failure filing it on the board takes it
3955 // back: an issue in the repository that is on no board is an item nobody asked for
3956 // and nothing here would find again.
3957 let added = match self
3958 .graphql(
3959 graphql::ADD_TO_BOARD,
3960 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
3961 )
3962 .await
3963 {
3964 Ok(added) => added,
3965 Err(error) => {
3966 let _ = self.delete_issue(&content_id).await;
3967 return Err(error);
3968 }
3969 };
3970 let item = added
3971 .pointer("/addProjectV2ItemById/item")
3972 .filter(|value| !value.is_null())
3973 .ok_or_else(|| SourceError::Malformed {
3974 message: "GitHub board addition returned no project item".into(),
3975 })?;
3976 if let Some(StatusTarget::Closed(_)) = status_target {
3977 let closed = self
3978 .graphql(
3979 graphql::UPDATE_ISSUE,
3980 json!({"input":{"id":content_id.0,"stateInput":state_input(status_target)}}),
3981 )
3982 .await?;
3983 let returned =
3984 closed
3985 .pointer("/updateIssue/issue")
3986 .ok_or_else(|| SourceError::Malformed {
3987 message: "GitHub item update returned no item".into(),
3988 })?;
3989 if required_str(returned, "id")? != content_id.0 {
3990 return Err(SourceError::Malformed {
3991 message: "GitHub item update returned the wrong item".into(),
3992 });
3993 }
3994 }
3995 Ok((content_id, required_str(item, "id")?.to_owned(), url))
3996 }
3997
3998 /// Move one issue under the project it now belongs to, or out of the one it left.
3999 async fn reparent(
4000 &self,
4001 held: Option<NativeId>,
4002 child: &NativeId,
4003 wanted: Option<&NativeId>,
4004 ) -> Result<(), SourceError> {
4005 if held.as_ref() == wanted {
4006 return Ok(());
4007 }
4008 if let Some(held) = &held {
4009 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
4010 .await?;
4011 }
4012 if let Some(wanted) = wanted {
4013 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
4014 .await?;
4015 }
4016 Ok(())
4017 }
4018
4019 async fn sub_issue(
4020 &self,
4021 operation: &str,
4022 parent: &NativeId,
4023 child: &NativeId,
4024 root: &str,
4025 ) -> Result<(), SourceError> {
4026 let data = self
4027 .graphql(
4028 operation,
4029 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
4030 )
4031 .await?;
4032 let issue =
4033 data.pointer(&format!("/{root}/issue"))
4034 .ok_or_else(|| SourceError::Malformed {
4035 message: "GitHub sub-issue update returned no issue".into(),
4036 })?;
4037 let sub =
4038 data.pointer(&format!("/{root}/subIssue"))
4039 .ok_or_else(|| SourceError::Malformed {
4040 message: "GitHub sub-issue update returned no sub-issue".into(),
4041 })?;
4042 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
4043 return Err(SourceError::Malformed {
4044 message: "GitHub sub-issue update returned the wrong issues".into(),
4045 });
4046 }
4047 Ok(())
4048 }
4049
4050 async fn reconcile_blocked_by(
4051 &self,
4052 content_id: &NativeId,
4053 native: &[String],
4054 ) -> Result<(), SourceError> {
4055 let current = self.native_dependency_ids(content_id).await?;
4056 for (operation, far_id) in current
4057 .iter()
4058 .filter(|id| !native.contains(id))
4059 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
4060 .chain(
4061 native
4062 .iter()
4063 .filter(|id| !current.contains(id))
4064 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
4065 )
4066 {
4067 let data = self
4068 .graphql(
4069 operation,
4070 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
4071 )
4072 .await?;
4073 let root = if operation == graphql::ADD_BLOCKED_BY {
4074 "addBlockedBy"
4075 } else {
4076 "removeBlockedBy"
4077 };
4078 let issue =
4079 data.pointer(&format!("/{root}/issue"))
4080 .ok_or_else(|| SourceError::Malformed {
4081 message: "GitHub dependency update returned no issue".into(),
4082 })?;
4083 let blocker = data
4084 .pointer(&format!("/{root}/blockingIssue"))
4085 .ok_or_else(|| SourceError::Malformed {
4086 message: "GitHub dependency update returned no blocking issue".into(),
4087 })?;
4088 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
4089 {
4090 return Err(SourceError::Malformed {
4091 message: "GitHub dependency update returned the wrong issues".into(),
4092 });
4093 }
4094 }
4095 Ok(())
4096 }
4097}
4098
4099/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
4100enum Reached {
4101 /// An issue this board holds, resolved into everything this source reports about it.
4102 Held(Box<Resolved>),
4103 /// Nothing this board holds: no such node, or a node on some other board.
4104 Nothing,
4105 /// A board draft, which exists only inside the board's own item connection.
4106 Draft,
4107}
4108
4109/// What GitHub says when a string is not a node id it can resolve.
4110///
4111/// Matched because it is the ordinary answer to a project selector naming a project by its
4112/// *name*, and reporting that as a failure would make naming one impossible. It is read
4113/// off the refusal GitHub sent, never guessed from the shape of the string: this source
4114/// does not define the syntax of a GitHub node id and would be wrong about it.
4115const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
4116
4117/// Whether this refusal is GitHub saying the id names no node at all.
4118fn unresolvable_node(error: &SourceError) -> bool {
4119 matches!(error, SourceError::Refused { message }
4120 if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
4121}
4122
4123/// One project name, as a search qualifier which filters on it at the server.
4124///
4125/// Quoted so the whole title is one phrase rather than a bag of words, with the two
4126/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
4127/// the way it documents. A title matched here is still compared for equality afterwards:
4128/// the qualifier narrows what the server sends, and this source decides what it names.
4129fn title_qualifier(name: &str) -> String {
4130 let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
4131 format!("in:title \"{escaped}\"")
4132}
4133
4134/// The board, and every item on it this source reports.
4135#[derive(Clone)]
4136struct Board {
4137 id: String,
4138 fields: Value,
4139 items: Vec<Resolved>,
4140}
4141
4142impl Board {
4143 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
4144 complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
4145 let nodes = fields
4146 .get("nodes")
4147 .and_then(Value::as_array)
4148 .ok_or_else(|| SourceError::Malformed {
4149 message: "GitHub project fields.nodes is not an array".into(),
4150 })?;
4151 Ok(nodes
4152 .iter()
4153 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
4154 }
4155}
4156
4157/// One board item, resolved into everything this source reports about it.
4158#[derive(Clone)]
4159struct Resolved {
4160 item_id: String,
4161 id: NativeId,
4162 content_kind: ContentKind,
4163 kind: BoardKind,
4164 title: String,
4165 body: Option<String>,
4166 status: Status,
4167 labels: Vec<Label>,
4168 parent: Option<NativeId>,
4169 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
4170 origin: Option<String>,
4171 url: Option<String>,
4172 created_at: Option<DateTime<Utc>>,
4173 updated_at: Option<DateTime<Utc>>,
4174 own_repository: Option<Repository>,
4175 repositories: Vec<Repository>,
4176 slot: BTreeMap<String, Value>,
4177 /// The node id of the board this item sits on, when the read that reached it said.
4178 board_id: Option<String>,
4179 /// The definition of every board field this item holds a value of, in the shape a read
4180 /// of the board's own `fields` gives one.
4181 ///
4182 /// Only the fields this item has a value in: a field it holds nothing of is not here,
4183 /// which says nothing about whether the board has it.
4184 fields: Vec<Value>,
4185}
4186
4187impl Resolved {
4188 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
4189 /// in a field of its own, and none of the three keys that are only an encoding.
4190 fn metadata(&self) -> BTreeMap<String, Value> {
4191 let mut metadata = self.slot.clone();
4192 metadata.remove(Repository::METADATA_KEY);
4193 metadata.remove(DependencyEdge::RECORDED_KEY);
4194 metadata.remove(ItemKind::METADATA_KEY);
4195 if let Some(origin) = &self.origin {
4196 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
4197 }
4198 metadata
4199 }
4200
4201 /// Where this item is, as a link a reader can open.
4202 ///
4203 /// A board is a hosted place and every issue on it has a web address, so that address
4204 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
4205 /// of place it is, so a reader knows to open it rather than to read a file out. It
4206 /// does not replace or derive from `url`: the field goes on reporting exactly what it
4207 /// reported before, and this says what that address *is*.
4208 ///
4209 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
4210 /// rather than a third variant, which is the contract's "the source did not say". An
4211 /// issue this run created is not one of those: its address comes back from the
4212 /// creating mutation, so it is somewhere a reader can open from the moment it exists
4213 /// rather than from whenever the board read catches up.
4214 fn location(&self) -> Option<Location> {
4215 self.url.clone().map(Location::Url)
4216 }
4217
4218 fn task(&self) -> Task {
4219 Task {
4220 id: self.id.clone(),
4221 title: self.title.clone(),
4222 content: self.body.clone(),
4223 status: self.status.clone(),
4224 labels: self.labels.clone(),
4225 project: self.parent.clone(),
4226 url: self.url.clone(),
4227 location: self.location(),
4228 created_at: self.created_at,
4229 updated_at: self.updated_at,
4230 metadata: self.metadata(),
4231 repositories: self.repositories.clone(),
4232 }
4233 }
4234
4235 fn project(&self) -> Project {
4236 Project {
4237 id: self.id.clone(),
4238 title: self.title.clone(),
4239 content: self.body.clone(),
4240 status: self.status.clone(),
4241 labels: self.labels.clone(),
4242 url: self.url.clone(),
4243 location: self.location(),
4244 created_at: self.created_at,
4245 updated_at: self.updated_at,
4246 metadata: self.metadata(),
4247 repositories: self.repositories.clone(),
4248 }
4249 }
4250
4251 /// The same issue as a document: the project it is filed under, and no status and no
4252 /// dependencies, because a document is not work.
4253 fn document(&self) -> Document {
4254 Document {
4255 id: self.id.clone(),
4256 title: self.title.clone(),
4257 content: self.body.clone(),
4258 project: self.parent.clone(),
4259 labels: self.labels.clone(),
4260 url: self.url.clone(),
4261 location: self.location(),
4262 created_at: self.created_at,
4263 updated_at: self.updated_at,
4264 metadata: self.metadata(),
4265 repositories: self.repositories.clone(),
4266 }
4267 }
4268}
4269
4270/// What one write is, and the status that comes with being it.
4271///
4272/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
4273/// status and a task or a project always has one, so "a document carrying a status" and
4274/// "a task carrying none" are states a write cannot be in rather than states every use
4275/// site below has to defend against.
4276enum Written<'a> {
4277 /// A document, which is not work and so has no status at all.
4278 Document,
4279 /// A task or a project, and the status it is being written with.
4280 Work(ItemKind, &'a Status),
4281}
4282
4283impl Written<'_> {
4284 /// Which of the board's three kinds this write is.
4285 const fn kind(&self) -> BoardKind {
4286 match self {
4287 Self::Document => BoardKind::Document,
4288 Self::Work(kind, _) => BoardKind::Work(*kind),
4289 }
4290 }
4291
4292 /// The status this write carries. A document carries none, so a write of one says
4293 /// nothing about the issue's open or closed state and selects no board `Status`
4294 /// option.
4295 const fn status(&self) -> Option<&Status> {
4296 match self {
4297 Self::Document => None,
4298 Self::Work(_, status) => Some(status),
4299 }
4300 }
4301}
4302
4303/// The item being written, in the one shape all three write methods reach.
4304struct Incoming<'a> {
4305 written: Written<'a>,
4306 /// The title a person wrote. A document's goes onto the issue with
4307 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
4308 title: &'a str,
4309 content: Option<&'a str>,
4310 labels: &'a [Label],
4311 metadata: &'a BTreeMap<String, Value>,
4312 repositories: &'a [Repository],
4313 parent: Option<&'a NativeId>,
4314}
4315
4316impl Incoming<'_> {
4317 /// The title this write puts on the issue.
4318 fn written_title(&self) -> String {
4319 match self.written {
4320 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
4321 Written::Work(..) => self.title.to_owned(),
4322 }
4323 }
4324}
4325
4326#[derive(Clone, Copy, PartialEq, Eq)]
4327enum ContentKind {
4328 DraftIssue,
4329 Issue,
4330}
4331
4332/// What one board issue is: a document, or the work an [`ItemKind`] names.
4333///
4334/// A type of this source's own rather than an `ItemKind` with a third variant, because
4335/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
4336/// document — the contract keeps a document out of that enum deliberately. Holding the
4337/// board's three answers in one value is what makes every place that asks "which is this?"
4338/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
4339/// two thirds of the board.
4340#[derive(Clone, Copy, PartialEq, Eq)]
4341enum BoardKind {
4342 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
4343 Document,
4344 /// Every other issue, and every draft.
4345 Work(ItemKind),
4346}
4347
4348impl BoardKind {
4349 /// How a refusal names this kind to the person reading it.
4350 const fn describes(self) -> &'static str {
4351 match self {
4352 Self::Document => "document",
4353 Self::Work(kind) => kind.marker(),
4354 }
4355 }
4356}
4357
4358/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
4359///
4360/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
4361/// the shared cross-source journeys assert one answer to one question, so two sources
4362/// that disagree about what "carries the label bug" means fail them.
4363fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
4364 let holds = |name: &String| {
4365 labels
4366 .iter()
4367 .any(|label| label.name.eq_ignore_ascii_case(name))
4368 };
4369 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
4370 && filter.all_of.iter().all(holds)
4371 && !filter.none_of.iter().any(holds)
4372}
4373
4374/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
4375/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
4376fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
4377 statuses.is_empty() || statuses.contains(&category)
4378}
4379
4380/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
4381///
4382/// `content` is the item's own prose — the body with this source's trailing metadata
4383/// comment already taken off — so a search never matches an encoding the author of the
4384/// issue never wrote.
4385fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
4386 let terms = query.terms.to_lowercase();
4387 let in_title = title.to_lowercase().contains(&terms);
4388 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
4389 match query.fields {
4390 TextFields::Title => in_title,
4391 TextFields::Content => in_content,
4392 TextFields::TitleOrContent => in_title || in_content,
4393 }
4394}
4395
4396/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
4397///
4398/// The project predicate is passed separately because a read narrowed to one project has
4399/// already answered it by asking *that project* for its own items — and re-applying it
4400/// there would compare the caller's selector, which may be a project's **name**, against
4401/// the id of the project that name resolved to, and keep nothing. Every other read passes
4402/// `query.project` and applies it here, which is what keeps `projects` a predicate this
4403/// source really does apply.
4404fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
4405 labels_match(&task.labels, &query.labels)
4406 && status_matches(task.status.category, &query.statuses)
4407 && match project {
4408 ProjectFilter::Any => true,
4409 ProjectFilter::Orphans => task.project.is_none(),
4410 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
4411 }
4412 && query
4413 .text
4414 .as_ref()
4415 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
4416}
4417
4418fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
4419 labels_match(&project.labels, &query.labels)
4420 && status_matches(project.status.category, &query.statuses)
4421 && query
4422 .text
4423 .as_ref()
4424 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
4425}
4426
4427/// The same three predicates a task query carries, minus the status filter.
4428///
4429/// A document is not work, so it has no status for one to compare against and the query
4430/// type carries none. The project predicate is the same one — a design issue filed under a
4431/// project issue is in that project, and one filed under nothing is in none — so it is
4432/// spelled the same way here rather than answered differently.
4433fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
4434 labels_match(&document.labels, &query.labels)
4435 && match project {
4436 ProjectFilter::Any => true,
4437 ProjectFilter::Orphans => document.project.is_none(),
4438 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
4439 }
4440 && query
4441 .text
4442 .as_ref()
4443 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
4444}
4445
4446#[async_trait::async_trait]
4447impl TaskSource for GitHubProjectsSource {
4448 fn kind(&self) -> &'static str {
4449 KIND
4450 }
4451 fn capabilities(&self) -> Capabilities {
4452 Capabilities {
4453 projects: Support::Native,
4454 documents: Support::Native,
4455 comments: Support::Native,
4456 orphan_tasks: Support::Native,
4457 filter_by_label: Support::Native,
4458 filter_by_status: Support::Native,
4459 search_title: Support::Native,
4460 search_content: Support::Native,
4461 task_dependencies: DependencySupport::BothDirections,
4462 project_dependencies: DependencySupport::BothDirections,
4463 max_page_size: MAX_PAGE_SIZE,
4464 }
4465 }
4466 async fn health(&self) -> Result<Health, SourceError> {
4467 let board = self.board_page(None, 1).await?;
4468 Ok(Health {
4469 reachable: true,
4470 detail: Some(format!(
4471 "reading GitHub project {}/{} ({})",
4472 self.owner,
4473 self.project_number,
4474 required_str(&board, "title")?
4475 )),
4476 })
4477 }
4478 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
4479 Ok(self
4480 .item_by_id(id)
4481 .await?
4482 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4483 .map(|item| item.task()))
4484 }
4485 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
4486 Ok(self
4487 .item_by_id(id)
4488 .await?
4489 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4490 .map(|item| item.project()))
4491 }
4492 async fn query_tasks(
4493 &self,
4494 query: &TaskQuery,
4495 page: &PageRequest,
4496 ) -> Result<Page<Task>, SourceError> {
4497 validate_page(page)?;
4498 // A read narrowed to one project asks that project for its own tasks, so nothing
4499 // about it costs what the rest of the board holds. Every other task read is a
4500 // question about the whole board and is answered by reading it.
4501 let (held, membership) = match &query.project {
4502 ProjectFilter::Is(project) => (
4503 self.project_children(project).await?,
4504 // Answered by where these items came from; see `task_matches`.
4505 &ProjectFilter::Any,
4506 ),
4507 ProjectFilter::Any | ProjectFilter::Orphans => {
4508 (self.board().await?.items, &query.project)
4509 }
4510 };
4511 // Filtered before paged: a page of a filtered result is a page of the survivors,
4512 // never the survivors of a page.
4513 let tasks = held
4514 .iter()
4515 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
4516 .map(Resolved::task)
4517 .filter(|task| task_matches(task, query, membership))
4518 .collect();
4519 Ok(offset_page(
4520 tasks,
4521 numeric_cursor(page.cursor.as_ref())?,
4522 page.limit.min(MAX_PAGE_SIZE) as usize,
4523 ))
4524 }
4525 async fn query_projects(
4526 &self,
4527 query: &ProjectQuery,
4528 page: &PageRequest,
4529 ) -> Result<Page<Project>, SourceError> {
4530 validate_page(page)?;
4531 // The projects a board holds are found by an issue search scoped to that board,
4532 // never by walking the board's own item connection: what tells a project from a
4533 // task is the `parent` each issue carries, which costs nothing to read.
4534 let projects = self
4535 .board_issues()
4536 .await?
4537 .iter()
4538 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
4539 .map(Resolved::project)
4540 .filter(|project| project_matches(project, query))
4541 .collect();
4542 Ok(offset_page(
4543 projects,
4544 numeric_cursor(page.cursor.as_ref())?,
4545 page.limit.min(MAX_PAGE_SIZE) as usize,
4546 ))
4547 }
4548 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
4549 Ok(self
4550 .item_by_id(id)
4551 .await?
4552 .filter(|item| item.kind == BoardKind::Document)
4553 .map(|item| item.document()))
4554 }
4555 async fn query_documents(
4556 &self,
4557 query: &DocumentQuery,
4558 page: &PageRequest,
4559 ) -> Result<Page<Document>, SourceError> {
4560 validate_page(page)?;
4561 // Narrowed to one project, this is the same sub-issue read a task list scoped to
4562 // that project makes — a document filed under a project is a sub-issue of it too,
4563 // and which of them come back is the kind this caller asked for.
4564 let (held, membership) = match &query.project {
4565 ProjectFilter::Is(project) => (
4566 self.project_children(project).await?,
4567 // Answered by where these items came from; see `task_matches`.
4568 &ProjectFilter::Any,
4569 ),
4570 ProjectFilter::Any | ProjectFilter::Orphans => {
4571 (self.board().await?.items, &query.project)
4572 }
4573 };
4574 // Filtered before paged, exactly as a task read is: a page of a filtered result is
4575 // a page of the survivors, never the survivors of a page.
4576 let documents = held
4577 .iter()
4578 .filter(|item| item.kind == BoardKind::Document)
4579 .map(Resolved::document)
4580 .filter(|document| document_matches(document, query, membership))
4581 .collect();
4582 Ok(offset_page(
4583 documents,
4584 numeric_cursor(page.cursor.as_ref())?,
4585 page.limit.min(MAX_PAGE_SIZE) as usize,
4586 ))
4587 }
4588 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
4589 validate_page(page)?;
4590 let offset = numeric_cursor(page.cursor.as_ref())?;
4591 let mut labels = self
4592 .board()
4593 .await?
4594 .items
4595 .into_iter()
4596 .flat_map(|item| item.labels)
4597 .fold(Vec::new(), |mut all, label| {
4598 if !all.iter().any(|x: &Label| x.id == label.id) {
4599 all.push(label);
4600 }
4601 all
4602 });
4603 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
4604 Ok(offset_page(
4605 labels,
4606 offset,
4607 page.limit.min(MAX_PAGE_SIZE) as usize,
4608 ))
4609 }
4610 async fn task_dependencies(
4611 &self,
4612 id: &NativeId,
4613 direction: Direction,
4614 page: &PageRequest,
4615 ) -> Result<Page<DependencyEdge>, SourceError> {
4616 self.dependencies(id, ItemKind::Task, direction, page).await
4617 }
4618 async fn project_dependencies(
4619 &self,
4620 id: &NativeId,
4621 direction: Direction,
4622 page: &PageRequest,
4623 ) -> Result<Page<DependencyEdge>, SourceError> {
4624 self.dependencies(id, ItemKind::Project, direction, page)
4625 .await
4626 }
4627
4628 fn writes(&self) -> WriteSupport {
4629 WriteSupport::Supported
4630 }
4631
4632 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
4633 self.write_item(
4634 &Incoming {
4635 written: Written::Work(ItemKind::Task, &write.item.status),
4636 title: &write.item.title,
4637 content: write.item.content.as_deref(),
4638 labels: &write.item.labels,
4639 metadata: &write.item.metadata,
4640 repositories: &write.item.repositories,
4641 parent: write.item.project.as_ref(),
4642 },
4643 write.target.as_ref(),
4644 &write.depends_on,
4645 )
4646 .await
4647 }
4648
4649 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
4650 self.write_item(
4651 &Incoming {
4652 written: Written::Work(ItemKind::Project, &write.item.status),
4653 title: &write.item.title,
4654 content: write.item.content.as_deref(),
4655 labels: &write.item.labels,
4656 metadata: &write.item.metadata,
4657 repositories: &write.item.repositories,
4658 parent: None,
4659 },
4660 write.target.as_ref(),
4661 &write.depends_on,
4662 )
4663 .await
4664 }
4665
4666 /// Create or update one document, which is one issue titled the way this board spells
4667 /// a document.
4668 ///
4669 /// Everything else is exactly a task write: caller metadata goes to the same canonical
4670 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
4671 /// or a field this board cannot carry is refused by name rather than dropped, a target
4672 /// naming an issue this board does not hold is refused rather than created, and an
4673 /// issue this call created is taken back when the rest of the write fails.
4674 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
4675 // A document takes part in no dependency graph, so there is no far end to write
4676 // natively and none to record: a caller naming one is told so rather than having it
4677 // stored under the reserved key, where a later read would report an edge the
4678 // contract says cannot exist.
4679 if !write.depends_on.is_empty() {
4680 return Err(SourceError::Refused {
4681 message: format!(
4682 "this write names {} dependencies for a document, and a document takes \
4683 part in no dependency graph; next: put the dependency on the task or \
4684 project the document is about",
4685 write.depends_on.len()
4686 ),
4687 });
4688 }
4689 self.write_item(
4690 &Incoming {
4691 written: Written::Document,
4692 title: &write.item.title,
4693 content: write.item.content.as_deref(),
4694 labels: &write.item.labels,
4695 metadata: &write.item.metadata,
4696 repositories: &write.item.repositories,
4697 parent: write.item.project.as_ref(),
4698 },
4699 write.target.as_ref(),
4700 &[],
4701 )
4702 .await
4703 }
4704
4705 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
4706 self.delete_item(id).await
4707 }
4708
4709 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
4710 self.delete_item(id).await
4711 }
4712
4713 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
4714 self.delete_item(id).await
4715 }
4716
4717 /// One page of the task issue's own comments, walked by GitHub's own cursor.
4718 ///
4719 /// Nothing here filters, so nothing has to be read ahead of the page: the caller's limit is
4720 /// the page GitHub is asked for and GitHub's `endCursor` is the cursor handed back.
4721 async fn task_comments(
4722 &self,
4723 task: &NativeId,
4724 page: &PageRequest,
4725 ) -> Result<Option<Page<Comment>>, SourceError> {
4726 validate_page(page)?;
4727 let Some(issue) = self.commented_issue(task).await? else {
4728 return Ok(None);
4729 };
4730 let after = page.cursor.as_ref().map(|cursor| cursor.0.as_str());
4731 let data = self
4732 .graphql(
4733 graphql::ISSUE_COMMENTS,
4734 json!({"id":issue.0,"first":page.limit.min(MAX_PAGE_SIZE),"after":after}),
4735 )
4736 .await?;
4737 // The issue was there a moment ago; one removed since is no longer a task here.
4738 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
4739 return Ok(None);
4740 };
4741 let connection = node
4742 .get("comments")
4743 .filter(|value| !value.is_null())
4744 .ok_or_else(|| SourceError::Malformed {
4745 message: format!(
4746 "GitHub issue {} answered with no comments connection",
4747 issue.0
4748 ),
4749 })?;
4750 let items = optional_nodes(Some(connection), "issue comments")?
4751 .into_iter()
4752 .flatten()
4753 .map(comment_from)
4754 .collect::<Result<Vec<_>, _>>()?;
4755 let next = next_cursor(connection)?;
4756 if let Some(next) = &next {
4757 validate_cursor_progress(after, &next.0)?;
4758 }
4759 Ok(Some(Page { items, next }))
4760 }
4761
4762 /// Add one comment to the task's issue, as the account the token belongs to.
4763 ///
4764 /// The author is refused before anything is sent — not even the task is read — because
4765 /// no answer GitHub could give would make posting under another name than the one asked
4766 /// for the right outcome.
4767 async fn add_comment(
4768 &self,
4769 task: &NativeId,
4770 comment: &NewComment,
4771 ) -> Result<Option<Comment>, SourceError> {
4772 if let Some(author) = &comment.author {
4773 return Err(SourceError::Refused {
4774 message: format!(
4775 "source {} cannot post a comment as {author:?}: GitHub records the account \
4776 the token signs in as the author of every comment; next: leave --author \
4777 out, and the comment is posted as that account",
4778 self.name
4779 ),
4780 });
4781 }
4782 let Some(issue) = self.commented_issue(task).await? else {
4783 return Ok(None);
4784 };
4785 let data = self
4786 .graphql(
4787 graphql::ADD_COMMENT,
4788 json!({"input":{"subjectId":issue.0,"body":comment.body.as_str()}}),
4789 )
4790 .await?;
4791 let subject = data
4792 .pointer("/addComment/subject")
4793 .filter(|value| !value.is_null())
4794 .ok_or_else(|| SourceError::Malformed {
4795 message: "GitHub comment addition returned no subject".into(),
4796 })?;
4797 if required_str(subject, "id")? != issue.0 {
4798 return Err(SourceError::Malformed {
4799 message: "GitHub comment addition answered about another issue".into(),
4800 });
4801 }
4802 let added = data
4803 .pointer("/addComment/commentEdge/node")
4804 .filter(|value| !value.is_null())
4805 .ok_or_else(|| SourceError::Malformed {
4806 message: "GitHub comment addition returned no comment".into(),
4807 })?;
4808 comment_from(added).map(Some)
4809 }
4810
4811 async fn edit_comment(
4812 &self,
4813 task: &NativeId,
4814 comment: &NativeId,
4815 body: &CommentBody,
4816 ) -> Result<Option<Comment>, SourceError> {
4817 let Some(issue) = self.commented_issue(task).await? else {
4818 return Ok(None);
4819 };
4820 if !self.comment_is_on(&issue, comment).await? {
4821 return Ok(None);
4822 }
4823 let data = self
4824 .graphql(
4825 graphql::UPDATE_COMMENT,
4826 json!({"input":{"id":comment.0,"body":body.as_str()}}),
4827 )
4828 .await?;
4829 let edited = data
4830 .pointer("/updateIssueComment/issueComment")
4831 .filter(|value| !value.is_null())
4832 .ok_or_else(|| SourceError::Malformed {
4833 message: "GitHub comment update returned no comment".into(),
4834 })?;
4835 let edited = comment_from(edited)?;
4836 if edited.id != *comment {
4837 return Err(SourceError::Malformed {
4838 message: "GitHub comment update returned the wrong comment".into(),
4839 });
4840 }
4841 Ok(Some(edited))
4842 }
4843
4844 async fn delete_comment(
4845 &self,
4846 task: &NativeId,
4847 comment: &NativeId,
4848 ) -> Result<Option<NativeId>, SourceError> {
4849 let Some(issue) = self.commented_issue(task).await? else {
4850 return Ok(None);
4851 };
4852 if !self.comment_is_on(&issue, comment).await? {
4853 return Ok(None);
4854 }
4855 let data = self
4856 .graphql(graphql::DELETE_COMMENT, json!({"input":{"id":comment.0}}))
4857 .await?;
4858 // The payload says nothing about the comment it removed, so what is checked is that
4859 // GitHub answered the mutation at all rather than leaving it unanswered.
4860 data.get("deleteIssueComment")
4861 .filter(|value| !value.is_null())
4862 .ok_or_else(|| SourceError::Malformed {
4863 message: "GitHub comment deletion returned no payload".into(),
4864 })?;
4865 Ok(Some(comment.clone()))
4866 }
4867
4868 /// Every request this source has recorded, and what each of GitHub's two budgets was
4869 /// attributed — read off the same accounting the session report is rendered from, so
4870 /// the two cannot count one request two ways.
4871 async fn metering(&self) -> Result<Option<Metering>, SourceError> {
4872 Ok(Some(self.ledger.snapshot().metering()))
4873 }
4874}
4875
4876/// One issue comment as the contract carries it.
4877///
4878/// `author` is absent both when GitHub answers `null` for an account that no longer exists
4879/// and when it answers an actor with no login, because either way the source did not say who
4880/// wrote it — which is what an absent author means, rather than an author called nothing.
4881fn comment_from(value: &Value) -> Result<Comment, SourceError> {
4882 Ok(Comment {
4883 id: NativeId(required_str(value, "id")?.to_owned()),
4884 author: optional_str(value.get("author").unwrap_or(&Value::Null), "login")?
4885 .map(str::to_owned),
4886 created_at: optional_time(value, "createdAt")?,
4887 updated_at: optional_time(value, "updatedAt")?,
4888 body: required_str(value, "body")?.to_owned(),
4889 url: optional_str(value, "url")?.map(str::to_owned),
4890 })
4891}
4892
4893/// Where the recorded tail of a dependency walk resumes; see
4894/// [`GitHubProjectsSource::recorded_edges`].
4895const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
4896
4897/// The board text field this source keeps a copy's origin in.
4898///
4899/// Named after the key it holds, and held to that name by the guard below rather than by
4900/// a reader noticing.
4901const ORIGIN_FIELD: &str = "onetaskgraph.origin";
4902
4903/// The metadata key that field holds.
4904///
4905/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
4906/// constructs or interprets the qualified id it carries. This source names it only to
4907/// route it — a short, typed value belongs in a typed field rather than in the body slot
4908/// a caller's own prose shares.
4909///
4910/// Restated rather than imported, because no plugin crate may depend on the engine. What
4911/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
4912/// target in `check`: it reads the engine's own literal and fails naming the file and the
4913/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
4914/// that creates a second item every run instead of finding the one it wrote — and that is
4915/// too late to learn it.
4916const ORIGIN_KEY: &str = "onetaskgraph.origin";
4917
4918/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
4919///
4920/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
4921/// is derived from the far end, never written down on the near item — so only a forward
4922/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
4923/// it did not come from, and it is told so rather than answered with an empty page that
4924/// reads as a walk which ended.
4925fn recorded_offset(
4926 cursor: Option<&str>,
4927 direction: Direction,
4928) -> Result<Option<usize>, SourceError> {
4929 cursor
4930 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
4931 .map(|offset| {
4932 if direction != Direction::DependsOn {
4933 return Err(SourceError::Config {
4934 message: format!(
4935 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
4936 reverse dependency read never issues; resume it in the direction \
4937 that reported it"
4938 ),
4939 });
4940 }
4941 offset.parse().map_err(|_| SourceError::Config {
4942 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
4943 })
4944 })
4945 .transpose()
4946}
4947
4948fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
4949 let mut page = offset_page(edges, offset, limit.max(1));
4950 page.next = page
4951 .next
4952 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
4953 page
4954}
4955
4956/// The kind of one issue reached through a dependency connection.
4957///
4958/// The same questions the board scan asks, over the fields the dependency document
4959/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
4960/// then anything with sub-issues or the marker is a project.
4961///
4962/// # Errors
4963///
4964/// A far end this board holds as a document is refused rather than reported. The two
4965/// answers that are not refusals would both be wrong: reporting it as a task names an id
4966/// no task read of this source can find, and reporting it as a project names one no
4967/// project read can. There is no third value to return — `ItemKind` has no document
4968/// variant, because nothing may point at a document — so the relationship itself is what
4969/// the person is told about.
4970fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
4971 let id = required_str(value, "id")?;
4972 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
4973 return Err(SourceError::Refused {
4974 message: format!(
4975 "GitHub issue {id} is a document of this board — its title begins \
4976 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
4977 on by one; next: remove that issue's blocking relationship on this board"
4978 ),
4979 });
4980 }
4981 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
4982 if parent.is_some() {
4983 return Ok(ItemKind::Task);
4984 }
4985 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
4986 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
4987 message: format!("GitHub issue {id}: {message}"),
4988 })?;
4989 let sub_issues = sub_issue_total(value)?;
4990 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
4991 ItemKind::Project
4992 } else {
4993 ItemKind::Task
4994 })
4995}
4996
4997/// The `IssueStateUpdateInput` one status target asks for.
4998///
4999/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
5000/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
5001/// a currently-closed issue: without that the item would read back `Unknown` and a copy
5002/// would report a change forever. A document has no status at all, and asks for neither.
5003fn state_input(target: Option<&StatusTarget>) -> Value {
5004 match target {
5005 Some(StatusTarget::Closed(reason)) => {
5006 json!({"value":"CLOSED","stateReason":reason.reason()})
5007 }
5008 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
5009 // A document has no status, so a write of one says nothing about the issue's open
5010 // or closed state rather than forcing it open: `stateInput` is what carries that
5011 // instruction, and an explicit null asks for no change to it.
5012 None => Value::Null,
5013 }
5014}
5015
5016/// The metadata one write stores in the item's body slot.
5017///
5018/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
5019/// rather than carried: the kind marker so an empty project stays readable, the
5020/// repository list only when it is not exactly the issue's own repository, and the far
5021/// ends no relationship here can name.
5022fn slot_metadata(
5023 incoming: &Incoming<'_>,
5024 own_repository: Option<&Repository>,
5025 fallback: &[DependencyEdge],
5026) -> BTreeMap<String, Value> {
5027 let mut metadata = incoming.metadata.clone();
5028 metadata.remove(ORIGIN_KEY);
5029 match incoming.written.kind() {
5030 BoardKind::Work(kind) => metadata.insert(
5031 ItemKind::METADATA_KEY.to_owned(),
5032 Value::String(kind.marker().to_owned()),
5033 ),
5034 // A document is told by its title, so it carries no kind marker: that key names
5035 // what a dependency endpoint points at, and nothing may point at a document.
5036 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
5037 };
5038 let derivable = own_repository
5039 .map(|own| incoming.repositories == [own.clone()])
5040 .unwrap_or(incoming.repositories.is_empty());
5041 if derivable {
5042 metadata.remove(Repository::METADATA_KEY);
5043 } else {
5044 metadata.insert(
5045 Repository::METADATA_KEY.to_owned(),
5046 Value::Array(
5047 incoming
5048 .repositories
5049 .iter()
5050 .map(|repository| Value::String(repository.as_str().to_owned()))
5051 .collect(),
5052 ),
5053 );
5054 }
5055 if fallback.is_empty() {
5056 metadata.remove(DependencyEdge::RECORDED_KEY);
5057 } else {
5058 metadata.insert(
5059 DependencyEdge::RECORDED_KEY.to_owned(),
5060 Value::Array(
5061 fallback
5062 .iter()
5063 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
5064 .collect(),
5065 ),
5066 );
5067 }
5068 metadata
5069}
5070
5071/// Every label one item carries, from its content's own connection and nowhere else.
5072///
5073/// There is no second place to read one from: no document this source sends selects the
5074/// board's built-in `Labels` field, because GitHub derives it from the content and a draft
5075/// cannot carry one at all. The module documentation records the three schema facts that
5076/// settle it.
5077fn labels(content: &Value) -> Result<Vec<Label>, SourceError> {
5078 optional_nodes(content.get("labels"), "content labels")?
5079 .into_iter()
5080 .flatten()
5081 .map(|v| {
5082 Ok(Label {
5083 id: NativeId(required_str(v, "id")?.to_owned()),
5084 name: required_str(v, "name")?.to_owned(),
5085 color: optional_str(v, "color")?.map(str::to_owned),
5086 })
5087 })
5088 .collect()
5089}
5090
5091/// The definition of each board field one item's values are values of, in the shape a read
5092/// of the board's own `fields` gives one.
5093///
5094/// A value names its field through a fragment on that field's own type, so the type is
5095/// known from which kind of value it is: a single-select value's field is a
5096/// `ProjectV2SingleSelectField`, options and all, and a text value's is a `ProjectV2Field`.
5097/// A value whose field carried no id, or an empty one, says nothing usable and is left out.
5098fn field_definitions(field_values: &[Value]) -> Vec<Value> {
5099 field_values
5100 .iter()
5101 .filter_map(|value| {
5102 let field = value.get("field")?.as_object()?;
5103 field.get("id")?.as_str().filter(|id| !id.is_empty())?;
5104 let typename = if value.get("text").is_some() {
5105 "ProjectV2Field"
5106 } else if value.get("name").is_some() {
5107 "ProjectV2SingleSelectField"
5108 } else {
5109 return None;
5110 };
5111 let mut defined = field.clone();
5112 defined.insert("__typename".to_owned(), json!(typename));
5113 Some(Value::Object(defined))
5114 })
5115 .collect()
5116}
5117
5118fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
5119 let Some(node) = field_values
5120 .iter()
5121 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
5122 else {
5123 return Ok(None);
5124 };
5125 Ok(optional_str(node, "text")?.map(str::to_owned))
5126}
5127
5128fn valid_github_owner(owner: &str) -> bool {
5129 !owner.is_empty()
5130 && owner.len() <= 39
5131 && !owner.starts_with('-')
5132 && !owner.ends_with('-')
5133 && !owner.contains("--")
5134 && owner
5135 .bytes()
5136 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
5137}
5138
5139/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
5140/// neither of the two names a path segment already means.
5141fn valid_github_repository_name(name: &str) -> bool {
5142 !name.is_empty()
5143 && name.len() <= 100
5144 && name != "."
5145 && name != ".."
5146 && name
5147 .bytes()
5148 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
5149}
5150
5151fn valid_environment_name(name: &str) -> bool {
5152 let mut bytes = name.bytes();
5153 bytes
5154 .next()
5155 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
5156 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
5157}
5158
5159/// How many sub-issues one issue has.
5160///
5161/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
5162/// absent or non-integer one is a response this source cannot read — and reading it as
5163/// zero would classify a project as a task, which is exactly the mistake the marker
5164/// exists to keep from happening quietly.
5165fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
5166 let summary = issue
5167 .get("subIssuesSummary")
5168 .ok_or_else(|| SourceError::Malformed {
5169 message: "GitHub issue is missing subIssuesSummary".into(),
5170 })?;
5171 summary
5172 .get("total")
5173 .and_then(Value::as_u64)
5174 .ok_or_else(|| SourceError::Malformed {
5175 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
5176 })
5177}
5178
5179fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
5180 value
5181 .get(field)
5182 .and_then(Value::as_str)
5183 .ok_or_else(|| SourceError::Malformed {
5184 message: format!("GitHub response is missing string field {field}"),
5185 })
5186}
5187
5188/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
5189/// needs one — Linear spells them too, in its own description field.
5190///
5191/// Restated rather than shared, because a plugin crate depends on the contract crate and
5192/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
5193/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
5194/// source round-trips its own writes perfectly well under its own spelling.
5195const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
5196const METADATA_CLOSE: &str = "\n-->";
5197
5198/// The visible body and the metadata slot at the end of it.
5199///
5200/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
5201/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
5202/// own content and is left alone.
5203fn metadata_body(
5204 body: Option<String>,
5205) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
5206 let Some(body) = body else {
5207 return Ok((None, BTreeMap::new()));
5208 };
5209 let Some(start) = body.rfind(METADATA_OPEN) else {
5210 return Ok((Some(body), BTreeMap::new()));
5211 };
5212 let encoded_start = start + METADATA_OPEN.len();
5213 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
5214 return Err(SourceError::Malformed {
5215 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
5216 });
5217 };
5218 let encoded_end = encoded_start + relative_end;
5219 if !body[encoded_end + METADATA_CLOSE.len()..].trim().is_empty() {
5220 return Ok((Some(body), BTreeMap::new()));
5221 }
5222 let metadata = serde_json::from_str(&body[encoded_start..encoded_end]).map_err(|error| {
5223 SourceError::Malformed {
5224 message: format!(
5225 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
5226 ),
5227 }
5228 })?;
5229 let visible = body[..start].trim_end();
5230 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
5231}
5232
5233fn compose_body(
5234 content: Option<&str>,
5235 metadata: &BTreeMap<String, Value>,
5236) -> Result<Option<String>, SourceError> {
5237 let visible = content.unwrap_or_default();
5238 if metadata.is_empty() {
5239 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
5240 }
5241 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
5242 message: error.to_string(),
5243 })?;
5244 Ok(Some(if visible.is_empty() {
5245 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
5246 } else {
5247 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
5248 }))
5249}
5250
5251fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
5252 value
5253 .get(field)
5254 .and_then(Value::as_bool)
5255 .ok_or_else(|| SourceError::Malformed {
5256 message: format!("GitHub response is missing boolean field {field}"),
5257 })
5258}
5259fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
5260 match value.get(field) {
5261 None | Some(Value::Null) => Ok(None),
5262 Some(value) => value
5263 .as_str()
5264 .map(Some)
5265 .ok_or_else(|| SourceError::Malformed {
5266 message: format!("GitHub response field {field} is not a string or null"),
5267 }),
5268 }
5269}
5270fn optional_nodes<'a>(
5271 connection: Option<&'a Value>,
5272 name: &str,
5273) -> Result<Option<&'a Vec<Value>>, SourceError> {
5274 match connection {
5275 None | Some(Value::Null) => Ok(None),
5276 Some(value) => value
5277 .get("nodes")
5278 .and_then(Value::as_array)
5279 .map(Some)
5280 .ok_or_else(|| SourceError::Malformed {
5281 message: format!("GitHub {name}.nodes is not an array"),
5282 }),
5283 }
5284}
5285fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
5286 let page_info = connection
5287 .get("pageInfo")
5288 .ok_or_else(|| SourceError::Malformed {
5289 message: format!("GitHub {name} has no pageInfo"),
5290 })?;
5291 if required_bool(page_info, "hasNextPage")? {
5292 return Err(SourceError::Malformed {
5293 message: format!(
5294 "GitHub {name} exceeds the supported nested connection size of {size}"
5295 ),
5296 });
5297 }
5298 Ok(())
5299}
5300fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
5301 optional_str(value, field)?
5302 .map(|timestamp| {
5303 timestamp.parse().map_err(|error| SourceError::Malformed {
5304 message: format!("GitHub response field {field} is not a timestamp: {error}"),
5305 })
5306 })
5307 .transpose()
5308}
5309fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
5310 if page.limit == 0 {
5311 Err(SourceError::Config {
5312 message: "page limit must be at least 1".into(),
5313 })
5314 } else {
5315 Ok(())
5316 }
5317}
5318fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
5319 let page = connection
5320 .get("pageInfo")
5321 .filter(|value| value.is_object())
5322 .ok_or_else(|| SourceError::Malformed {
5323 message: "GitHub connection is missing pageInfo".into(),
5324 })?;
5325 if required_bool(page, "hasNextPage")? {
5326 let cursor = required_str(page, "endCursor")?;
5327 validate_cursor_progress(None, cursor)?;
5328 Ok(Some(Cursor(cursor.into())))
5329 } else {
5330 Ok(None)
5331 }
5332}
5333fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
5334 if next.is_empty() || previous == Some(next) {
5335 Err(SourceError::Malformed {
5336 message: "GitHub pagination cursor is empty or did not advance".into(),
5337 })
5338 } else {
5339 Ok(())
5340 }
5341}
5342fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
5343 cursor.map_or(Ok(0), |c| {
5344 c.0.parse().map_err(|_| SourceError::Config {
5345 message: "page cursor is invalid".into(),
5346 })
5347 })
5348}
5349fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
5350 if offset > items.len() {
5351 return Page::last(vec![]);
5352 }
5353 let tail = items.split_off(offset);
5354 let mut selected = tail;
5355 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
5356 selected.truncate(limit);
5357 Page {
5358 items: selected,
5359 next,
5360 }
5361}