onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so [`GitHubProjectsConfig::repository`] names the one repository this
11//! source creates its project and task issues in; a write without it is refused naming
12//! the field.
13//!
14//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
15//! board has no document type and nothing but issues to hold one in, so the title is the
16//! discriminator and it is the whole of it. The title this source *reports* is the one a
17//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
18//! the body so `content` is what the person wrote — and writing a document puts the prefix
19//! back, so a round trip returns the title that went in.
20//!
21//! **Telling a document from a project from a task.** The design prefix is read **first**:
22//! a document is never a project and never a task, whatever sub-issues it has or does not
23//! have. Only then does the rest apply — a board issue is a project when *either* it has
24//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
25//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
26//! necessary: it is what makes an *empty* project — the state a project copy passes
27//! through between creating the project and filing its first task — readable as a
28//! project, while the sub-issue arm lets a person author a project on the board by hand
29//! with no knowledge of this product's metadata at all. Reading the prefix later than the
30//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
31//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
32//! task nor a document and are ignored.
33//!
34//! **Where an entity is, is a link.** Every project, task and document this source reports
35//! carries a [`Location::Url`] naming the issue's own web address — the same address the
36//! `url` field already reports, in the shape that says a reader can open it. That is the
37//! contrast the location contract exists for: a reader holding an entity from this source
38//! is handed something to link to and one holding an entity from a folder of Markdown is
39//! handed a path, and neither has to know which plugin answered. It does not replace or
40//! derive from `url`; that field goes on reporting what it always reported.
41//!
42//! **Where metadata lives.** Short typed things go to typed fields and native relations:
43//! status to the board's `Status` single-select and the issue's own state, the copy
44//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
45//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
46//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
47//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
48//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
49//! why neither can hold a caller's own prose.
50//!
51//! **Status.** `status_mapping` is per-instance configuration from a status category to
52//! `null`, a board `Status` option name, or a closed state of `completed` or
53//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
54//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
55//! and a mistake destroys every item's status. A status this board cannot represent is a
56//! refusal naming the status and the instance instead.
57//!
58//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
59//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
60//! finished tasks were only moved to a "Done" column would read 0% complete forever.
61//!
62//! # What this source declares, field by field
63//!
64//! One verdict per field of [`Capabilities`], and what `Native` means when this source
65//! says it. *Proven* means a shared journey drives it against the real
66//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
67//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
68//! [`capabilities`](TaskSource::capabilities) from parting.
69//!
70//! | Field | Verdict |
71//! | --- | --- |
72//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
73//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
74//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
75//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
76//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
77//! | `search_title` | **Supported and proven,** over `Issue.title`. |
78//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
79//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
80//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
81//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
82//!
83//! Nothing here is unsupported. `documents` is not a predicate — it says this source has
84//! documents, which it does — and the three facts behind the uniform `Native` on the
85//! predicates beside it are recorded below rather than re-derived, because a reader who
86//! takes `Native` to mean *the remote service filters* will read that uniformity as a
87//! lie.
88//!
89//! First, the plugin contract defines `Support::Native` as *the source applies this
90//! predicate itself*, and says nothing about where it applies it. What the declaration
91//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
92//! — so that the engine may push it down and apply nothing of its own.
93//!
94//! Second, this source can keep that promise for every predicate at no additional API
95//! cost, because whichever of the three reads below answers a query has already read every
96//! item that query could keep before it filters anything. Filtering those items is
97//! in-process work over data already in hand.
98//!
99//! Third, no predicate but `projects` could be pushed into the API even if that were
100//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
101//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
102//! label set, a status column or a substring of a body, and its title qualifier matches
103//! tokens where this source — and the local Markdown source beside it — match substrings,
104//! so pushing a search down would silently *narrow* the answer. What a project filter has
105//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
106//! the issue for them is both cheaper and exact. So there is one predicate this source
107//! applies by asking a narrower question, six it applies in process, and none it is unable
108//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
109//! source has already done, and declaring `projects` native while ignoring the filter
110//! (which this source once did) silently returns another project's tasks, because the
111//! engine trusts the declaration and applies nothing locally.
112//!
113//! # The three ways this source reaches an item, and what each costs
114//!
115//! A board read is charged for what its *nested* connections could return rather than for
116//! what was asked, so one whole-board read costs the same whether the question was about
117//! one project or about all of them. That is why a question about one project is never
118//! answered by reading the board:
119//!
120//! | The question | What is sent | What it costs |
121//! | --- | --- | --- |
122//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
123//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
124//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
125//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
126//!
127//! The board half of an issue — its board item's id, its `Status` option and this
128//! source's origin text field — rides along on `Issue.projectItems` in the first three, so
129//! an item reached any of those ways resolves through the same
130//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
131//! same status, the same labels and the same qualified id. An issue with no entry for
132//! *this* board is not this source's to report, which is what keeps an id naming another
133//! repository's issue from being answered as an item of this board.
134//!
135//! **The board's own `Labels` field is not selected in those three, and nothing is lost by
136//! that.** The board half they read is a fragment `on Issue`, and an issue's labels are
137//! already selected one level up, on the issue itself. A board's `Labels` field is not one
138//! anybody fills in: it is a built-in `ProjectV2FieldType`, it is absent from
139//! `ProjectV2CustomFieldType` so no project can create one, and `ProjectV2FieldValue` —
140//! the whole of what `updateProjectV2ItemFieldValue` accepts — offers no way to write one.
141//! For `Issue` content it *is* the issue's own labels, so selecting it beside them unions a
142//! set with itself. [`graphql::BOARD`] still selects it and must: a board item's content
143//! may be a `DraftIssue`, which has no `labels` of its own to select instead. The four ways
144//! an item is reached are held to reporting one label set by
145//! `an_item_reports_the_same_labels_title_status_and_id_however_it_is_reached` in
146//! `tests/plugin.rs`, which drives each of the four documents against the fixture board.
147//!
148//! The last row is still the board's own item connection, and deliberately: a **draft**
149//! board item is not an issue, so no search and no node read can reach one, and the reads
150//! that have to answer for the whole board are the ones whose cost is the board's size
151//! anyway.
152//!
153//! **What a read may return is capped too, and that cap is on the document rather than on
154//! the board.** GitHub limits the number of nodes **one query may return** to
155//! [`NODE_COUNT_LIMIT`] and refuses a query above that before executing it: the answer is
156//! an error naming the connection the count crossed at, not a slow or a partial result.
157//! Every board this source reads is refused the same way, so no board is too big for these
158//! documents and none is small enough to save one that is over.
159//!
160//! The count is arithmetic over the document's own text: each connection contributes the
161//! `first:` it asks for, counts **multiply** down a nested path and **sum** across sibling
162//! paths. Those are [GitHub's published rules][node-limits] and this workspace does not
163//! restate them — `github-graphql-node-count` implements them, and
164//! [`worst_case_node_count`] under [`largest_page_sizes`] is where every node count here
165//! comes from. `every_document_this_source_sends_stays_under_githubs_node_limit`, in
166//! `tests/node_count.rs`, recomputes every document in [`graphql::DOCUMENTS`] from that
167//! same text on every run and fails naming any that reaches the limit — so a connection
168//! added to a shared fragment is caught there rather than by GitHub.
169//!
170//! What decides those counts is the page sizes: [`MAX_PAGE_SIZE`] on the outer page,
171//! `NESTED_PAGE_SIZE` on the connections hanging off one item, and
172//! `BOARD_ITEMS_PAGE_SIZE` on an issue's board memberships. `$nestedFirst` is spent twice
173//! down one path of a board read, so that constant is effectively squared there, which is
174//! why it is the one the limit is most sensitive to.
175//!
176//! **`nodeCount` and `cost` are two numbers against two limits, and none of this is about
177//! the second.** `nodeCount` is the one above: the most nodes one query may return,
178//! checked per query. `cost` is rate-limit points, metered per hour across everything one
179//! credential does; it is what the two limiters [`Limiter`] tells apart meter, and a
180//! document under [`NODE_COUNT_LIMIT`] says nothing about it.
181//!
182//! [node-limits]: https://docs.github.com/en/graphql/overview/rate-limits-and-node-limits-for-the-graphql-api
183//!
184//! **Where a read-after-write guarantee comes from, since a search index cannot supply
185//! one.** GitHub's issue search is eventually consistent and answers a write made moments
186//! ago with the value from before it. Resolving a node id is not, so a read by id and a
187//! project's own sub-issues are already current. What closes the gap for the search is
188//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
189//! what this process itself wrote, so an item created seconds ago is reported whether or
190//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
191//! and the record dies with the process.
192//!
193//! Filtering happens before paging, so a page of a filtered result is a page of the
194//! survivors rather than the survivors of a page. Label and text matching answer the same
195//! question the same way the local Markdown source's do, so one cross-source expectation
196//! holds for both.
197//!
198//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
199//! has one source, `capabilities`, and the note above is the reasoning behind it rather
200//! than a second copy of it: without the three facts recorded here a reader takes the
201//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
202//! crate's own capabilities test, which pins every field of it against a fully spelled-out
203//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
204//! fails to compile there rather than going unasserted. -->
205//! Required checks use only the local fixture server; the ignored credentialed lane
206//! verifies the current schema, then drives every field of the table above against the
207//! real board. It builds its own fixture there — two projects, one task filed under each,
208//! one filed under neither, a label on one of the three and a closed status on another —
209//! because that shape is what tells an honoured predicate from an ignored one: a board
210//! holding a single project answers a project filter the same way whether or not this
211//! source applies it, which is exactly how the defect above went unseen.
212//!
213//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
214//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
215//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
216//! nominated is what keeps a credentialed write lane off a board and a repository nobody
217//! nominated; it never asks GitHub which project was updated most recently. Before it
218//! starts, the lane also clears any item titled — and any repository label named — the way
219//! it titles and names its own artifacts, which is self-healing after an interrupted run:
220//! a process killed between its writes and its cleanup leaves artifacts the next run
221//! removes.
222//!
223//! **GitHub has two rate limiters and this source is refused by both, so nothing here
224//! treats them as one thing.** The primary budget is the hourly allowance `gh api
225//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
226//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
227//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
228//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
229//! one part of.
230#![deny(missing_docs)]
231
232use std::collections::BTreeMap;
233use std::sync::Mutex;
234use std::time::{Duration, Instant};
235
236use chrono::{DateTime, Utc};
237use onetaskgraph_plugin_api::{
238 Capabilities, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind, DependencySupport,
239 Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label, LabelFilter, Location,
240 NativeId, Page, PageRequest, Project, ProjectFilter, ProjectQuery, Repository, SecretResolver,
241 SourceError, SourceName, SourcePlugin, Status, StatusCategory, Support, Task, TaskQuery,
242 TaskSource, TextFields, TextQuery, WriteSupport,
243};
244use reqwest::{Client, StatusCode, Url};
245use schemars::{Schema, schema_for};
246use secrecy::{ExposeSecret, SecretString};
247use serde::Deserialize;
248use serde_json::{Value, json};
249
250/// The registry name for this plugin.
251pub const KIND: &str = "github-projects";
252/// GitHub's maximum connection page size.
253pub const MAX_PAGE_SIZE: u32 = 100;
254
255/// The most nodes any one document this source sends may be asked to return.
256///
257/// GitHub's own published per-query ceiling, taken from
258/// [`github_graphql_node_count::NODE_LIMIT`] rather than written out again here, so this
259/// workspace cannot hold a stale copy of somebody else's number. A query above it is
260/// **refused before it is executed**, whoever is asking and whatever board they are
261/// asking about — so this is a bound on the documents rather than a budget that runs out.
262///
263/// This is `nodeCount`, the maximum number of nodes *one query may return*. It is not
264/// `cost`, the rate-limit points a call spends against an hourly allowance shared by
265/// everything the credential does. Two numbers, two limits; nothing here is about the
266/// second. The module section on the three ways this source reaches an item says how the
267/// count is arrived at, and which of the page sizes below decide it.
268pub const NODE_COUNT_LIMIT: u64 = github_graphql_node_count::NODE_LIMIT;
269
270/// Nested connection size for the connections that hang off one item.
271///
272/// It multiplies through every document that reaches an item under a page — the count
273/// rules multiply down a nested path — so it is the constant [`NODE_COUNT_LIMIT`] is most
274/// sensitive to. `tests/node_count.rs` is what holds the pair together: it recomputes
275/// every document under these constants and fails naming any that reaches the limit, so
276/// raising this is caught there rather than by GitHub.
277const NESTED_PAGE_SIZE: u32 = 50;
278/// How many of one issue's board memberships are read when an issue is reached directly.
279///
280/// An issue reached through a search or through its own node id carries its board half in
281/// `Issue.projectItems`, and only the entry for *this* board is read. Ten is deliberately
282/// far smaller than [`NESTED_PAGE_SIZE`]: this connection sits under a page of issues, so
283/// its size multiplies through the whole document, and an issue on ten boards at once is
284/// already well past what a person keeps track of. An issue whose entry for this board sits
285/// past it is refused naming the connection rather than reported as not on the board.
286const BOARD_ITEMS_PAGE_SIZE: u32 = 10;
287
288pub use github_graphql_node_count::{NodeCountError, Variables};
289
290/// The largest value this source can bind to each page-size variable its documents name.
291///
292/// Every `first:` in [`graphql`] reads one of these three, and each is capped at the
293/// constant above it wherever a caller's own limit could reach it — `$first` at
294/// [`MAX_PAGE_SIZE`], `$nestedFirst` at `NESTED_PAGE_SIZE`, `$boardItems` at
295/// `BOARD_ITEMS_PAGE_SIZE`. So this is the worst case a caller can drive this source to,
296/// not one configuration of it, which is what makes a bound computed under it a bound on
297/// every read.
298pub fn largest_page_sizes() -> Variables {
299 Variables::from([
300 ("first".to_owned(), MAX_PAGE_SIZE),
301 ("nestedFirst".to_owned(), NESTED_PAGE_SIZE),
302 ("boardItems".to_owned(), BOARD_ITEMS_PAGE_SIZE),
303 ])
304}
305
306/// The most nodes `document` could be asked to return, by GitHub's published rules.
307///
308/// Computed offline from the document's own text under [`largest_page_sizes`] — no
309/// network, no credential and no schema — by
310/// [`github_graphql_node_count::node_count`], which is where the rules themselves live.
311/// A document at or above [`NODE_COUNT_LIMIT`] is one GitHub refuses before executing, so
312/// this is what a check holds every document in [`graphql::DOCUMENTS`] below.
313///
314/// # Errors
315///
316/// Returns the calculation's own [`NodeCountError`] when `document` does not parse, holds
317/// no single operation, or binds a page size this source does not name — each of which is
318/// a defect in the document rather than a number.
319pub fn worst_case_node_count(document: &str) -> Result<u64, NodeCountError> {
320 github_graphql_node_count::node_count(document, &largest_page_sizes())
321}
322
323/// The issue-title prefix that makes a board issue a document.
324///
325/// A GitHub Projects board has no document type — it holds issues — so the discriminator
326/// is the title, and this is the whole of it: an issue whose title begins with these bytes
327/// is a document and every other issue is the task or project the sub-issue rule makes it.
328///
329/// It is spelled **once**, here, and read rather than restated everywhere else — including
330/// by the shared journeys, which take it from this constant so a board fixture cannot
331/// drift from what this source reads. `docs/metadata.md` records the two consequences that
332/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
333/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
334/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
335pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
336
337/// Exact GraphQL query documents issued by this plugin.
338///
339/// Keeping the production documents here lets the pinned-schema test validate the same
340/// bytes that are sent to GitHub, rather than a test-only copy which could drift
341/// independently. No document in this module writes the board itself, and none of them
342/// names `updateProjectV2Field`.
343pub mod graphql {
344 /// Everything this source reads about one issue, wherever it reaches that issue.
345 ///
346 /// A macro rather than a constant so the three documents below can `concat!` it: one
347 /// spelling of these fields is what makes an issue read through the board-scoped
348 /// search, through its own node id, and through its project's sub-issue relationship
349 /// resolve to *the same* item, which is the whole of what
350 /// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
351 ///
352 /// `projectItems` is what carries the board half of an issue: the board item's own id
353 /// and the field values — the `Status` option and this source's origin text field —
354 /// that a `ProjectV2.items` read used to carry. It is asked for on the issue rather
355 /// than on the board, which is what makes the cost of a read proportional to what was
356 /// asked for instead of to the board's size.
357 ///
358 /// It does **not** select the board's `Labels` field value, and that is the whole of
359 /// what keeps the three documents below under [`NODE_COUNT_LIMIT`](super::NODE_COUNT_LIMIT):
360 /// a label connection there sits under `fieldValues` under `projectItems` under a page
361 /// of issues, spending `$nestedFirst` twice down one path, and took
362 /// [`SEARCH_ISSUES`] and [`SUB_ISSUES`] to 2,556,100 nodes against a limit of 500,000.
363 /// No label is lost — this is a fragment `on Issue`, whose own `labels` are selected
364 /// above, and a board's `Labels` field is a built-in mirror of exactly those. The
365 /// module documentation records why that mirroring holds.
366 macro_rules! board_issue {
367 () => {
368 r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
369 labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
370 projectItems(first:$boardItems){nodes{id project{number}
371 fieldValues(first:$nestedFirst){nodes{
372 ... on ProjectV2ItemFieldSingleSelectValue{name field{
373 ... on ProjectV2SingleSelectField{id name options{id name}}
374 }}
375 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
376 }pageInfo{hasNextPage}}}pageInfo{hasNextPage}}}"#
377 };
378 }
379
380 /// Every issue of one board, found by a search scoped to that board.
381 ///
382 /// This is how the projects a board holds are listed, and it selects no `items`
383 /// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
384 /// container walked page by page, so nothing nested inside a board item is paid for.
385 /// Which of the issues it returns is a project is then read off `parent` — GitHub
386 /// accepts `-has:parent` as a search qualifier and silently ignores it, so the
387 /// discriminator has to be applied to the field, which is a scalar on the issue and
388 /// costs nothing.
389 pub const SEARCH_ISSUES: &str = concat!(
390 r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
391 search(query:$search,type:$type,first:$first,after:$after){
392 pageInfo{hasNextPage endCursor}
393 nodes{__typename ...BoardIssue}
394 }
395 }"#,
396 board_issue!()
397 );
398
399 /// One issue by its own node id, which is what a qualified id names here.
400 ///
401 /// Strongly consistent, unlike the search above: GitHub's issue search is an index and
402 /// answers a write made moments ago with the value from before it, and resolving a node
403 /// id does not.
404 pub const ISSUE: &str = concat!(
405 r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
406 node(id:$id){__typename ...BoardIssue}
407 }"#,
408 board_issue!()
409 );
410
411 /// One project's tasks: the sub-issues of the issue that project is.
412 ///
413 /// The work this costs is the project's own size. Nothing about it grows as the board
414 /// gains projects, or as those projects gain tasks.
415 pub const SUB_ISSUES: &str = concat!(
416 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
417 node(id:$id){__typename
418 ... on Issue{subIssues(first:$first,after:$after){
419 pageInfo{hasNextPage endCursor}
420 nodes{__typename ...BoardIssue}
421 }}}
422 }"#,
423 board_issue!()
424 );
425
426 /// Reads the board's fields and one page of its items.
427 pub const BOARD: &str = r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
428 owner:repositoryOwner(login:$owner){
429 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
430 }
431 } fragment Board on ProjectV2 { id title
432 fields(first:$nestedFirst){nodes{
433 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
434 ... on ProjectV2Field{__typename id name}
435 }pageInfo{hasNextPage}}
436 items(first:$first,after:$after){nodes{id fieldValues(first:$nestedFirst){nodes{
437 ... on ProjectV2ItemFieldSingleSelectValue{name field{
438 ... on ProjectV2SingleSelectField{id name options{id name}}
439 }}
440 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
441 ... on ProjectV2ItemFieldLabelValue{labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
442 }pageInfo{hasNextPage}} content{
443 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
444 ... on PullRequest{__typename id}
445 ... on DraftIssue{__typename id title body createdAt updatedAt}
446 }} pageInfo{hasNextPage endCursor}}
447 }"#;
448 /// Resolves the configured repository's node id, which creating an issue requires.
449 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
450 /// Reads both dependency directions for one issue, with each far end's own kind.
451 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
452 ... on Issue{
453 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
454 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
455 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
456 /// Creates one issue in the configured repository.
457 pub const CREATE_ISSUE: &str =
458 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
459 /// Puts an existing issue on the configured board.
460 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
461 /// Updates an issue's visible fields and its open or closed state in one call.
462 pub const UPDATE_ISSUE: &str =
463 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
464 /// Updates an existing draft's user-visible fields.
465 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
466 /// Updates a text or single-select value on one project item.
467 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
468 /// Files one issue under another as a sub-issue, which is what project membership is.
469 pub const ADD_SUB_ISSUE: &str =
470 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
471 /// Takes one issue back out of its parent.
472 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
473 /// Adds GitHub's native issue blocked-by relationship.
474 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
475 /// Removes one native issue blocked-by relationship.
476 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
477 /// Deletes one issue, which takes its board item with it.
478 ///
479 /// The engine sends this in one situation only: undoing a copy that could not finish,
480 /// over the items that same copy created. Deleting the issue removes the board item
481 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
482 pub const DELETE_ISSUE: &str =
483 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
484
485 /// Every document above, with what this source is doing when it sends one.
486 ///
487 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
488 /// name the call that was refused, and a `match` with a catch-all arm would answer a
489 /// document added later with "talking to GitHub" and never say so.
490 ///
491 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
492 /// const` here that this list omits, so the two cannot part — which is the same guard
493 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
494 pub const DOCUMENTS: [(&str, &str); 16] = [
495 (SEARCH_ISSUES, "searching this board's issues"),
496 (ISSUE, "reading one issue"),
497 (SUB_ISSUES, "reading a project's tasks"),
498 (BOARD, "reading the board"),
499 (REPOSITORY, "reading the destination repository"),
500 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
501 (CREATE_ISSUE, "creating an issue"),
502 (ADD_TO_BOARD, "adding an issue to the board"),
503 (UPDATE_ISSUE, "updating an issue"),
504 (UPDATE_DRAFT, "updating a draft item"),
505 (UPDATE_FIELD, "writing a board field"),
506 (ADD_SUB_ISSUE, "filing an issue under its project"),
507 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
508 (ADD_BLOCKED_BY, "recording a dependency"),
509 (REMOVE_BLOCKED_BY, "removing a dependency"),
510 (DELETE_ISSUE, "deleting an issue"),
511 ];
512}
513
514/// Which of GitHub's two rate limiters refused a request.
515///
516/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
517/// secondary one — so an operator told the wrong one takes the wrong next step, which is
518/// the whole reason this is carried rather than collapsed into "rate limited".
519#[derive(Debug, Clone, Copy, PartialEq, Eq)]
520enum Limiter {
521 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
522 Primary,
523 /// The burst limiter over content-generating requests, which nothing reports.
524 Secondary,
525}
526
527/// The wordings GitHub answers a secondary rate limit with.
528///
529/// It sends them under a forbidden status, under a too-many-requests status, and inside
530/// the `errors` of a *successful* response, which is why the text is what this matches on
531/// rather than the status. `abuse detection` is the wording GitHub used before the
532/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
533/// what a burst of content creation is refused with.
534///
535/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
536/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
537/// when, and the drift gate reconciles the two lists both ways. Public for that gate
538/// alone — a caller has no use for it, and matching on a refusal is this source's job.
539pub const SECONDARY_WORDINGS: [&str; 5] = [
540 "secondary rate limit",
541 "temporarily blocked from content creation",
542 "abuse detection",
543 "submitted too quickly",
544 "exceeded a secondary",
545];
546
547/// The wordings GitHub answers an exhausted primary budget with.
548///
549/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
550/// than looked for in the response text. Pinned and gated exactly as
551/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
552pub const PRIMARY_WORDINGS: [&str; 3] = [
553 "api rate limit exceeded",
554 "rate limit exceeded",
555 "rate_limited",
556];
557
558/// What a response *says about itself*, which is the only place a refusal can be read.
559///
560/// Deliberately not the whole response body. A board is a place people write about their
561/// own work, and a task on it titled "the secondary rate limit" would, matched across the
562/// raw text, turn a perfectly good answer into a refusal this source then waited out and
563/// reported. So the item data is never read: what is read is GitHub's own REST-style
564/// `message` envelope, which is what a forbidden status carries, and the `message` and
565/// `type` of each GraphQL error, which is where a *successful* response says it.
566///
567/// A body that is not JSON at all has nothing structured to read, so only a failing
568/// response's own text is taken — a successful response that is not JSON is malformed
569/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
570fn refusal_wording(status: StatusCode, body: &str) -> String {
571 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
572 return if status.is_success() {
573 String::new()
574 } else {
575 body.to_owned()
576 };
577 };
578 let mut said: Vec<&str> = parsed
579 .get("message")
580 .and_then(Value::as_str)
581 .into_iter()
582 .collect();
583 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
584 for error in errors {
585 said.extend(
586 ["message", "type"]
587 .into_iter()
588 .filter_map(|key| error.get(key).and_then(Value::as_str)),
589 );
590 }
591 }
592 said.join("; ")
593}
594
595impl Limiter {
596 /// Which limiter refused this response, or `None` when none of them did.
597 ///
598 /// The wording is read first and the status only decides what carries none of it,
599 /// because GitHub answers a secondary limit with a forbidden status far more often
600 /// than with too-many-requests — while a forbidden status saying nothing about a limit
601 /// really is a credential this token lacks.
602 ///
603 /// A response is a refusal because of its status or its own wording. A spent budget
604 /// only ever explains one; it never turns an answer into a refusal.
605 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
606 let normalized = refusal_wording(status, body).to_ascii_lowercase();
607 if SECONDARY_WORDINGS
608 .iter()
609 .any(|wording| normalized.contains(wording))
610 {
611 return Some(Self::Secondary);
612 }
613 if status == StatusCode::TOO_MANY_REQUESTS {
614 return Some(Self::Primary);
615 }
616 // An exhausted budget *explains* a response that failed; it does not make one that
617 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
618 // request the budget allowed as well as on the ones it then refuses, so reading
619 // the header alone threw away a good answer — and, once refusals were retried,
620 // replayed a request that had already taken effect.
621 if !status.is_success() && budget_exhausted {
622 return Some(Self::Primary);
623 }
624 // A successful response saying it: GitHub reports a GraphQL rate limit in the
625 // `errors` of an HTTP 200, where nothing about the status says so at all.
626 if status.is_success()
627 && PRIMARY_WORDINGS
628 .iter()
629 .any(|wording| normalized.contains(wording))
630 {
631 return Some(Self::Primary);
632 }
633 None
634 }
635
636 /// What this limiter is called where an operator can look it up.
637 const fn name(self) -> &'static str {
638 match self {
639 Self::Primary => "GitHub's primary API rate limit",
640 Self::Secondary => "GitHub's secondary rate limit",
641 }
642 }
643
644 /// What the endpoint an operator would go and check says about this limiter.
645 const fn where_to_look(self) -> &'static str {
646 match self {
647 Self::Primary => {
648 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
649 comes back."
650 }
651 Self::Secondary => {
652 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
653 primary budget and does not report this one, so budget showing there says \
654 nothing about this refusal, and every further attempt extends it."
655 }
656 }
657 }
658
659 /// The next step this limiter actually calls for.
660 const fn what_to_do(self) -> &'static str {
661 match self {
662 Self::Primary => {
663 "wait for the reset `gh api rate_limit` reports, then run the command again."
664 }
665 Self::Secondary => {
666 "leave this board alone for a few minutes, then run the command again — or \
667 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
668 }
669 }
670 }
671}
672
673/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
674#[derive(Debug, Clone, Copy)]
675struct Limited {
676 limiter: Limiter,
677 hint: Option<u64>,
678}
679
680impl Limited {
681 /// What the caller is told once this source has waited as long as it may.
682 ///
683 /// Both limiters report as [`SourceError::RateLimited`], because that is what
684 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
685 /// about *which* limiter it was makes it a different kind of failure. What differs is
686 /// the operator's next step, and that is what the message carries — a secondary
687 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
688 /// budget looks fine, and then back to retry the very burst that was refused.
689 fn exhausted(
690 self,
691 doing: &str,
692 waits: u32,
693 waited: Duration,
694 needed: Duration,
695 budget: Duration,
696 ) -> SourceError {
697 SourceError::RateLimited {
698 retry_after_seconds: self.hint,
699 message: Some(format!(
700 "{} refused this source while {doing}; it waited {} out over {} and was refused \
701 again, and the next wait of {} would take it past the {} one call may spend \
702 waiting. {} next: {}",
703 self.limiter.name(),
704 plural(waits, "refusal"),
705 seconds(waited),
706 seconds(needed),
707 seconds(budget),
708 self.limiter.where_to_look(),
709 self.limiter.what_to_do(),
710 )),
711 }
712 }
713}
714
715/// One attempt's outcome: an error to report, or a rate limit to wait out.
716enum Attempt {
717 Failed(SourceError),
718 Limited(Limited),
719}
720
721fn plural(count: u32, thing: &str) -> String {
722 if count == 1 {
723 format!("{count} {thing}")
724 } else {
725 format!("{count} {thing}s")
726 }
727}
728
729fn seconds(duration: Duration) -> String {
730 format!("{:.1}s", duration.as_secs_f64())
731}
732
733/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
734///
735/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
736/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
737/// header, and neither is what makes a response a refusal — so the whole cost of one this
738/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
739/// instead. Refusing the response over the header would turn a readable refusal into an
740/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
741fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
742 value
743 .and_then(|value| value.to_str().ok())
744 .and_then(|value| value.trim().parse::<u64>().ok())
745}
746
747/// Every mutation this source sends creates content — an issue, a board item, a field of
748/// one, a sub-issue link, a dependency — and no query in [`graphql::DOCUMENTS`] does, so
749/// what the secondary limiter counts and what the keyword says are the same set. That is
750/// what makes the keyword a sound test rather than a convenient one.
751fn is_mutation(query: &str) -> bool {
752 query.trim_start().starts_with("mutation")
753}
754
755/// What this source was doing, for a diagnostic that has to say so.
756///
757/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
758/// a document added without a description is caught by that list's own gate instead of
759/// falling through to the vague arm below.
760fn operation_description(query: &str) -> &'static str {
761 graphql::DOCUMENTS
762 .iter()
763 .find(|(document, _)| *document == query)
764 .map_or("talking to GitHub", |(_, doing)| *doing)
765}
766
767/// GitHub's published ceiling on content-generating requests, per minute.
768///
769/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
770/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
771/// from it, so a pacing value checked only against itself cannot go stale here.
772pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
773/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
774/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
775/// governs; it is pinned beside its sibling so the gate would notice either one moving.
776pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
777/// Shortest interval between two content-creating mutations, in milliseconds.
778///
779/// GitHub documents two secondary limits on content-generating requests:
780/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
781/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
782/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
783/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
784/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
785/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
786/// deliberately *not* what this paces at. An installation that wants the hourly bound
787/// honoured for a long sequence of copies says so through
788/// `pacing.min_mutation_interval_ms`.
789pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
790/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
791///
792/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
793/// own advice for a secondary limit — wait, and wait longer each time — without spending
794/// the first minute of a transient refusal doing nothing.
795pub const RETRY_BACKOFF_MS: u64 = 1_000;
796/// Total time one call may spend waiting out rate limits before it reports a failure.
797///
798/// Two minutes is long enough to ride out the refusals a paced copy still collects and
799/// short enough that a command an operator is watching returns. The bound is what makes
800/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
801/// the limiter, not in a process nobody can tell from a wedged one.
802pub const RETRY_BUDGET_MS: u64 = 120_000;
803
804fn default_token_env() -> String {
805 "GH_PROJECTS_TOKEN".to_owned()
806}
807fn default_endpoint() -> String {
808 "https://api.github.com/graphql".to_owned()
809}
810
811/// Where one status category lands on this board.
812///
813/// `null` — an absent value — disables the category for this instance, and using a
814/// disabled status is a refusal naming the status and the instance.
815#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
816#[serde(untagged)]
817pub enum StatusTargetConfig {
818 /// The name of a `Status` single-select option already on the board.
819 Column(ColumnName),
820 /// A closed issue state, whose reason is what tells done from cancelled.
821 Closed {
822 /// The `IssueClosedStateReason` to close with.
823 closed: ClosedState,
824 },
825}
826
827/// The name of a `Status` single-select option on the board.
828///
829/// Validated on the way in rather than checked later, so a blank option name — which
830/// nothing on a board can be — is a state this type cannot hold.
831#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
832#[serde(try_from = "String")]
833pub struct ColumnName(String);
834
835impl ColumnName {
836 /// The option name, as the board spells it.
837 fn as_str(&self) -> &str {
838 &self.0
839 }
840}
841
842impl TryFrom<String> for ColumnName {
843 type Error = String;
844
845 fn try_from(name: String) -> Result<Self, Self::Error> {
846 if name.trim().is_empty() {
847 return Err("a status_mapping option name cannot be blank".to_owned());
848 }
849 Ok(Self(name))
850 }
851}
852
853/// The two closed states this product can mean.
854///
855/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
856/// work nor abandoned work, so nothing here ever writes it.
857#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
858#[serde(rename_all = "kebab-case")]
859pub enum ClosedState {
860 /// `COMPLETED` — precisely done.
861 Completed,
862 /// `NOT_PLANNED` — precisely cancelled.
863 NotPlanned,
864}
865
866impl ClosedState {
867 const fn reason(self) -> &'static str {
868 match self {
869 Self::Completed => "COMPLETED",
870 Self::NotPlanned => "NOT_PLANNED",
871 }
872 }
873}
874
875/// Configuration for one GitHub Projects v2 board.
876#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
877#[serde(default, deny_unknown_fields)]
878pub struct GitHubProjectsConfig {
879 /// Login of the user or organization which owns the board.
880 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
881 /// The project number shown in the board's GitHub URL.
882 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
883 /// `owner/name` of the one repository this source creates its issues in.
884 ///
885 /// A board has no repository of its own and `createIssue` requires one, so a write
886 /// without this is refused naming the field. Reads never need it.
887 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
888 /// Environment variable containing a fine-grained token with Projects and Issues
889 /// read/write plus Pull requests read-only access for every repository represented on
890 /// the board.
891 #[serde(default = "default_token_env")]
892 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
893 /// GraphQL endpoint. GitHub Enterprise installations may override it.
894 #[serde(default = "default_endpoint")]
895 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
896 /// Per-instance mapping from a status category to where it lands on this board.
897 ///
898 /// A category this does not mention keeps its shipped default: `backlog` to
899 /// "Backlog", `todo` to "Todo", `in-progress` to "In Progress", `done` to closed as
900 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
901 /// disabled.
902 #[serde(default)]
903 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
904 /// How fast this source writes, and how long it waits out a rate-limit refusal.
905 ///
906 /// Every field keeps its shipped default when it is absent, and the defaults are
907 /// GitHub's own published limits rather than taste. See [`Pacing`].
908 #[serde(default)]
909 pub pacing: PacingConfig,
910}
911
912/// How fast this source writes, and how long it waits out a rate-limit refusal.
913///
914/// Configurable because a GitHub Enterprise installation sets its own limits and an
915/// operator who has already been refused may want to go slower still — not because the
916/// defaults are guesses.
917#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
918#[serde(default, deny_unknown_fields)]
919pub struct PacingConfig {
920 /// Shortest interval between two content-creating mutations, in milliseconds.
921 ///
922 /// Zero sends them as fast as they are asked for, which is what a fixture server on
923 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
924 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
925 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
926 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
927 /// zero while there is a budget to spend, because a schedule of zero-length waits
928 /// consumes none of it and so never ends.
929 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
930 /// Total time one call may spend waiting out rate limits, in milliseconds.
931 ///
932 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
933 /// the bound is what makes this a wait rather than a hang.
934 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
935}
936
937/// The largest any pacing setting may be, in milliseconds.
938///
939/// One hour. GitHub's own harshest published bound on content-generating requests works
940/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
941/// anything a real limit asks for, and past it the settings stop describing pacing at all:
942/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
943/// and an interval beyond it is a command that never sends its second mutation. It also
944/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
945/// what an `Instant` can hold on every platform.
946pub const MAX_PACING_MS: u64 = 3_600_000;
947
948/// [`PacingConfig`] with every default resolved and every value checked, which is what the
949/// source holds.
950#[derive(Debug, Clone, Copy)]
951struct Pacing {
952 min_mutation_interval: Duration,
953 retry_backoff: Duration,
954 retry_budget: Duration,
955}
956
957impl Pacing {
958 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
959 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
960 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
961 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
962 message: format!(
963 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
964 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
965 GitHub's own harshest published limit"
966 ),
967 }),
968 Some(value) => Ok(Duration::from_millis(value)),
969 None => Ok(Duration::from_millis(default)),
970 };
971 let retry_backoff = bounded(
972 config.retry_backoff_ms,
973 RETRY_BACKOFF_MS,
974 "retry_backoff_ms",
975 )?;
976 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
977 if retry_backoff.is_zero() && !retry_budget.is_zero() {
978 return Err(SourceError::Config {
979 message: format!(
980 "pacing.retry_backoff_ms of source {instance} is 0 while \
981 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
982 none of that budget, so it would retry a refusal forever. Set a backoff of \
983 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
984 waiting at all",
985 retry_budget.as_millis()
986 ),
987 });
988 }
989 Ok(Self {
990 min_mutation_interval: bounded(
991 config.min_mutation_interval_ms,
992 MIN_MUTATION_INTERVAL_MS,
993 "min_mutation_interval_ms",
994 )?,
995 retry_backoff,
996 retry_budget,
997 })
998 }
999}
1000
1001/// Factory for [`GitHubProjectsSource`].
1002#[derive(Debug, Clone, Copy, Default)]
1003pub struct Plugin;
1004
1005impl SourcePlugin for Plugin {
1006 fn kind(&self) -> &'static str {
1007 KIND
1008 }
1009 fn config_schema(&self) -> Schema {
1010 schema_for!(GitHubProjectsConfig)
1011 }
1012 fn build(
1013 &self,
1014 name: &SourceName,
1015 config: &Value,
1016 secrets: &dyn SecretResolver,
1017 ) -> Result<Box<dyn TaskSource>, SourceError> {
1018 let config: GitHubProjectsConfig =
1019 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
1020 message: format!("source {name}: {e}"),
1021 })?;
1022 let source =
1023 GitHubProjectsSource::new(name, config, secrets).map_err(|error| match error {
1024 SourceError::Config { message } => SourceError::Config {
1025 message: format!("source {name}: {message}"),
1026 },
1027 SourceError::Auth { message } => SourceError::Auth {
1028 message: format!("source {name}: {message}"),
1029 },
1030 other => other,
1031 })?;
1032 Ok(Box::new(source))
1033 }
1034}
1035
1036/// Where a status category lands on this board, once configuration is resolved.
1037#[derive(Debug, Clone, PartialEq, Eq)]
1038enum StatusTarget {
1039 /// Not usable against this instance.
1040 Disabled,
1041 /// The board's `Status` option of this name.
1042 Column(ColumnName),
1043 /// A closed issue, with the reason that says which closed it means.
1044 Closed(ClosedState),
1045}
1046
1047/// Every status category, in the order the vocabulary declares them.
1048///
1049/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
1050/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
1051/// added to the shared vocabulary fails to compile until it is named there, and this
1052/// crate's suite reconciles this list against that enum's own derived schema, which is
1053/// generated from the variants rather than written beside them. The schema is what
1054/// catches a list left one short — a list checking only the positions it already holds
1055/// would pass while every mapping indexed by the new position panicked.
1056pub const CATEGORIES: [StatusCategory; 7] = [
1057 StatusCategory::Draft,
1058 StatusCategory::Backlog,
1059 StatusCategory::Todo,
1060 StatusCategory::InProgress,
1061 StatusCategory::Done,
1062 StatusCategory::Cancelled,
1063 StatusCategory::Unknown,
1064];
1065
1066/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
1067#[must_use]
1068pub const fn category_position(category: StatusCategory) -> usize {
1069 match category {
1070 StatusCategory::Draft => 0,
1071 StatusCategory::Backlog => 1,
1072 StatusCategory::Todo => 2,
1073 StatusCategory::InProgress => 3,
1074 StatusCategory::Done => 4,
1075 StatusCategory::Cancelled => 5,
1076 StatusCategory::Unknown => 6,
1077 }
1078}
1079
1080/// The spelling a status category is configured and reported under.
1081fn category_name(category: StatusCategory) -> &'static str {
1082 match category {
1083 StatusCategory::Draft => "draft",
1084 StatusCategory::Backlog => "backlog",
1085 StatusCategory::Todo => "todo",
1086 StatusCategory::InProgress => "in-progress",
1087 StatusCategory::Done => "done",
1088 StatusCategory::Cancelled => "cancelled",
1089 StatusCategory::Unknown => "unknown",
1090 }
1091}
1092
1093/// A shipped default's option name.
1094///
1095/// The literals below are this file's own and non-blank, and they are validated by the
1096/// one constructor a configured name goes through rather than beside it.
1097fn shipped_column(name: &'static str) -> ColumnName {
1098 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
1099}
1100
1101/// The shipped default for one category, before this instance's configuration.
1102fn shipped_default(category: StatusCategory) -> StatusTarget {
1103 match category {
1104 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
1105 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
1106 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
1107 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
1108 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
1109 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
1110 }
1111}
1112
1113/// This instance's complete category-to-target mapping, read in both directions.
1114///
1115/// One target per category, held at that category's own [`category_position`], so a
1116/// category missing from the mapping, named twice in it, or filed out of order is a
1117/// state this type cannot hold rather than one [`Self::target`] has to defend against.
1118#[derive(Debug, Clone)]
1119struct StatusMapping {
1120 targets: [StatusTarget; CATEGORIES.len()],
1121}
1122
1123impl StatusMapping {
1124 fn resolve(
1125 configured: BTreeMap<String, Option<StatusTargetConfig>>,
1126 instance: &SourceName,
1127 ) -> Result<Self, SourceError> {
1128 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
1129 for (key, value) in configured {
1130 let category = CATEGORIES
1131 .iter()
1132 .find(|category| category_name(**category) == key)
1133 .ok_or_else(|| SourceError::Config {
1134 message: format!(
1135 "status_mapping names {key:?}, which is not a status category of source \
1136 {instance}; the categories are {}",
1137 CATEGORIES
1138 .iter()
1139 .map(|category| category_name(*category))
1140 .collect::<Vec<_>>()
1141 .join(", ")
1142 ),
1143 })?;
1144 overrides.insert(category_name(*category), value);
1145 }
1146 // `CATEGORIES[position] == category` for every category — the crate's suite
1147 // asserts it — so mapping the list in order fills each category's own slot.
1148 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
1149 None => shipped_default(category),
1150 Some(None) => StatusTarget::Disabled,
1151 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
1152 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
1153 });
1154 let mapping = Self { targets };
1155 for (index, category) in CATEGORIES.into_iter().enumerate() {
1156 let StatusTarget::Column(option) = mapping.target(category) else {
1157 continue;
1158 };
1159 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
1160 matches!(mapping.target(**earlier), StatusTarget::Column(name)
1161 if name.as_str().eq_ignore_ascii_case(option.as_str()))
1162 }) {
1163 return Err(SourceError::Config {
1164 message: format!(
1165 "status_mapping of source {instance} sends both {} and {} to the board \
1166 option {:?}; one option cannot read back as two categories",
1167 category_name(*other),
1168 category_name(category),
1169 option.as_str()
1170 ),
1171 });
1172 }
1173 }
1174 Ok(mapping)
1175 }
1176
1177 fn target(&self, category: StatusCategory) -> &StatusTarget {
1178 &self.targets[category_position(category)]
1179 }
1180
1181 /// The category a board option name reports, or `None` when nothing maps to it.
1182 fn category_of(&self, option: &str) -> Option<StatusCategory> {
1183 CATEGORIES.into_iter().find(|category| {
1184 matches!(self.target(*category), StatusTarget::Column(name)
1185 if name.as_str().eq_ignore_ascii_case(option))
1186 })
1187 }
1188}
1189
1190/// The one repository this source creates issues in.
1191#[derive(Debug, Clone)]
1192struct RepositoryTarget {
1193 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1194 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1195}
1196
1197impl RepositoryTarget {
1198 fn parse(value: &str) -> Result<Self, SourceError> {
1199 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
1200 message: format!(
1201 "repository must be spelled owner/name; {value:?} names no repository"
1202 ),
1203 })?;
1204 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
1205 return Err(SourceError::Config {
1206 message: format!(
1207 "repository must be spelled owner/name with a GitHub login and one \
1208 repository name; {value:?} is not"
1209 ),
1210 });
1211 }
1212 Ok(Self {
1213 owner: owner.to_owned(),
1214 name: name.to_owned(),
1215 })
1216 }
1217
1218 fn origin(&self) -> String {
1219 format!("github.com/{}/{}", self.owner, self.name)
1220 }
1221}
1222
1223/// A source which reads GitHub afresh for every operation.
1224pub struct GitHubProjectsSource {
1225 /// This source's configured name, used both to tell a far end naming this source
1226 /// from one naming a system it knows nothing about, and to name the instance a
1227 /// status refusal is about.
1228 name: SourceName,
1229 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
1230 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
1231 repository: Option<RepositoryTarget>,
1232 endpoint: Url,
1233 token: SecretString,
1234 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1235 statuses: StatusMapping,
1236 client: Client,
1237 /// Every item this source has created since it was built, in the order it created
1238 /// them.
1239 ///
1240 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1241 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1242 /// a copy resolving a dependency on an item it had just created refused it as not
1243 /// found. A board read is completed from this — an item remembered here and absent from
1244 /// the read is added back, because the board really does hold it and only the read is
1245 /// behind.
1246 ///
1247 /// It is not a cache of a user's work: nothing is remembered that this process did not
1248 /// itself just write, it lives and dies with the process, and it is never consulted for
1249 /// an item this source did not create.
1250 created: Mutex<Vec<Resolved>>,
1251 /// How fast this source writes, and how long it waits out a refusal.
1252 pacing: Pacing,
1253 /// When the last content-creating mutation finished, or the moment the furthest-out
1254 /// reserved slot releases the next one, whichever is later — so the one after it can be
1255 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1256 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1257 /// what it is measured from.
1258 last_mutation: Mutex<Option<Instant>>,
1259 /// The board as this process last read it, for the length of one command.
1260 ///
1261 /// A copy of a project used to re-read the whole board, paged, before writing each of
1262 /// its items, which is by far the largest part of a copy's request count and none of
1263 /// its work. Nothing else changes this board while a command runs — this source's own
1264 /// writes are the only writer — so one read answers them all.
1265 ///
1266 /// It is not a store of a user's work and it is not the cache the no-persistence
1267 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1268 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1269 /// an item this command created and then depends on resolves whether or not GitHub's
1270 /// own eventually-consistent read has caught up. A write to an item already on the
1271 /// board updates the entry here too, so what this holds is the last read plus this
1272 /// process's own writes rather than a snapshot taken before them.
1273 board_cache: Mutex<Option<Board>>,
1274 /// The destination repository's node id, resolved once rather than per issue created.
1275 ///
1276 /// A repository's node id does not change, and re-reading it for every issue of a copy
1277 /// spent one request per item on an answer this source already had.
1278 repository_cache: Mutex<Option<String>>,
1279}
1280
1281impl GitHubProjectsSource {
1282 /// Validate configuration and capture the named credential without exposing it.
1283 ///
1284 /// # Errors
1285 ///
1286 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1287 /// [`SourceError::Auth`] when the named credential is missing or empty.
1288 pub fn new(
1289 name: &SourceName,
1290 config: GitHubProjectsConfig,
1291 secrets: &dyn SecretResolver,
1292 ) -> Result<Self, SourceError> {
1293 if !valid_github_owner(&config.owner) {
1294 return Err(SourceError::Config {
1295 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1296 });
1297 }
1298 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1299 return Err(SourceError::Config {
1300 message: format!("project_number must be between 1 and {}", i32::MAX),
1301 });
1302 }
1303 if !valid_environment_name(&config.token_env) {
1304 return Err(SourceError::Config {
1305 message: "token_env must be a valid environment-variable name".into(),
1306 });
1307 }
1308 let repository = config
1309 .repository
1310 .as_deref()
1311 .map(RepositoryTarget::parse)
1312 .transpose()?;
1313 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1314 message: format!("endpoint is not a valid URL: {e}"),
1315 })?;
1316 if endpoint.scheme() != "https"
1317 && !(endpoint.scheme() == "http"
1318 && endpoint
1319 .host_str()
1320 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1321 {
1322 return Err(SourceError::Config {
1323 message:
1324 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1325 .into(),
1326 });
1327 }
1328 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1329 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1330 })?;
1331 Ok(Self {
1332 name: name.clone(),
1333 owner: config.owner,
1334 project_number: config.project_number,
1335 repository,
1336 endpoint,
1337 token,
1338 credential_name: config.token_env,
1339 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1340 client: Client::builder()
1341 .user_agent("onetaskgraph")
1342 .build()
1343 .map_err(|e| SourceError::Config {
1344 message: format!("cannot build HTTP client: {e}"),
1345 })?,
1346 created: Mutex::new(Vec::new()),
1347 pacing: Pacing::resolve(config.pacing, name)?,
1348 last_mutation: Mutex::new(None),
1349 board_cache: Mutex::new(None),
1350 repository_cache: Mutex::new(None),
1351 })
1352 }
1353
1354 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1355 /// rate limit rather than handing it straight back as an error.
1356 ///
1357 /// Retrying is safe for every document here, including the mutations, and the reason
1358 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1359 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1360 /// this replays has already taken effect. An outcome this source cannot know — the
1361 /// send failed, or the body could not be read, so the mutation may well have landed —
1362 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1363 /// attempt. A duplicate write would come from replaying one of those, and none is
1364 /// replayed.
1365 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1366 let doing = operation_description(query);
1367 let mut waited = Duration::ZERO;
1368 let mut waits = 0_u32;
1369 let mut backoff = self.pacing.retry_backoff;
1370 loop {
1371 if is_mutation(query) {
1372 let spacing = self.reserve_mutation_slot();
1373 if !spacing.is_zero() {
1374 tokio::time::sleep(spacing).await;
1375 }
1376 }
1377 let attempt = self.send_once(query, &variables).await;
1378 if is_mutation(query) {
1379 self.finish_mutation();
1380 }
1381 let limited = match attempt {
1382 Ok(data) => return Ok(data),
1383 Err(Attempt::Failed(error)) => return Err(error),
1384 Err(Attempt::Limited(limited)) => limited,
1385 };
1386 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1387 // move that extends a secondary limit, so a hint below the schedule's own next
1388 // wait is raised to it.
1389 let wait = match limited.hint {
1390 Some(hint) => Duration::from_secs(hint).max(backoff),
1391 None => backoff,
1392 };
1393 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1394 // A wait of nothing spends none of the budget, so it is exhaustion rather
1395 // than a retry. `Pacing::resolve` rules out every way of configuring one
1396 // except a budget of zero, where reporting the first refusal is the ask.
1397 if wait.is_zero() || wait > remaining {
1398 return Err(limited.exhausted(
1399 doing,
1400 waits,
1401 waited,
1402 wait,
1403 self.pacing.retry_budget,
1404 ));
1405 }
1406 tokio::time::sleep(wait).await;
1407 waited += wait;
1408 waits += 1;
1409 backoff = backoff.saturating_mul(2);
1410 }
1411 }
1412
1413 /// The next moment a content-creating mutation may leave this source, as a wait from
1414 /// now.
1415 ///
1416 /// The slot is reserved under the lock and the waiting happens outside it, so two
1417 /// callers take two slots rather than the same one — and no lock is held across an
1418 /// await.
1419 ///
1420 /// The moment it is spaced from is the previous mutation's *completion*, which
1421 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1422 /// own is the wrong thing to measure from.
1423 fn reserve_mutation_slot(&self) -> Duration {
1424 if self.pacing.min_mutation_interval.is_zero() {
1425 return Duration::ZERO;
1426 }
1427 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1428 // it would turn an earlier failure into a second one for no gain.
1429 let mut last = self
1430 .last_mutation
1431 .lock()
1432 .unwrap_or_else(std::sync::PoisonError::into_inner);
1433 let now = Instant::now();
1434 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1435 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1436 let at = last.map_or(now, |previous| {
1437 previous
1438 .checked_add(self.pacing.min_mutation_interval)
1439 .map_or(now, |earliest| earliest.max(now))
1440 });
1441 *last = Some(at);
1442 at.saturating_duration_since(now)
1443 }
1444
1445 /// Record that a content-creating mutation has finished, so the next one is spaced
1446 /// from here rather than from the moment this one was released.
1447 ///
1448 /// This source can only choose when a request *departs*; the limiter counts when it
1449 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1450 /// departure from the last therefore hands the limiter a gap of the interval less that
1451 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1452 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1453 /// slower machine while passing on a quick one.
1454 ///
1455 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1456 /// previous request had already arrived before its response came back, so its arrival
1457 /// is no later than this moment, and the next mutation is released at least the
1458 /// interval after this moment and arrives no earlier than it is released: the gap the
1459 /// limiter measures is therefore at least the interval, whatever transit costs and on
1460 /// whatever platform. The price is that a mutation's own round trip no longer counts
1461 /// towards its spacing, which makes this source slightly slower than the configured
1462 /// rate rather than slightly faster — the safe side of a limit that punishes being
1463 /// wrong by refusing reads for the next fifty minutes.
1464 ///
1465 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1466 /// and one that never left costs only a wait nobody needed.
1467 fn finish_mutation(&self) {
1468 if self.pacing.min_mutation_interval.is_zero() {
1469 return;
1470 }
1471 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1472 let mut last = self
1473 .last_mutation
1474 .lock()
1475 .unwrap_or_else(std::sync::PoisonError::into_inner);
1476 let now = Instant::now();
1477 // `max` rather than an assignment: a concurrent caller may already have reserved a
1478 // slot further out, and completing this request must never pull that slot back in.
1479 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1480 }
1481
1482 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1483 /// failure that waiting cannot help.
1484 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1485 let response = self
1486 .client
1487 .post(self.endpoint.clone())
1488 .bearer_auth(self.token.expose_secret())
1489 .json(&json!({"query": query, "variables": variables}))
1490 .send()
1491 .await
1492 .map_err(|e| {
1493 Attempt::Failed(SourceError::Unavailable {
1494 message: format!("GitHub GraphQL request failed: {e}"),
1495 })
1496 })?;
1497 let status = response.status();
1498 let header = |name: &str| whole_seconds(response.headers().get(name));
1499 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
1500 // that are not text at all — is "not known to be exhausted". This never makes a
1501 // response a refusal on its own: it says which limiter a refusal is attributed to
1502 // and where its hint comes from, so a value this cannot read costs a hint rather
1503 // than an answer.
1504 let exhausted = response
1505 .headers()
1506 .get("x-ratelimit-remaining")
1507 .and_then(|value| value.to_str().ok())
1508 == Some("0");
1509 // `retry-after` is what GitHub asks for when it asks; when it does not and the
1510 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
1511 // which is the same question answered as an absolute time. Nothing else here is a
1512 // hint, and a schedule is what answers a refusal that carries none.
1513 let hint = header("retry-after").or_else(|| {
1514 exhausted
1515 .then(|| header("x-ratelimit-reset"))
1516 .flatten()
1517 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
1518 });
1519 // Read before it is parsed, because the evidence which tells a secondary rate
1520 // limit from a rejected credential is in the body of a response whose status says
1521 // only "forbidden" — and a non-success response was never parsed at all.
1522 let body = response.text().await.map_err(|e| {
1523 Attempt::Failed(SourceError::Unavailable {
1524 message: format!("GitHub GraphQL response could not be read: {e}"),
1525 })
1526 })?;
1527 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
1528 return Err(Attempt::Limited(Limited { limiter, hint }));
1529 }
1530 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
1531 return Err(Attempt::Failed(SourceError::Auth {
1532 message: format!(
1533 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
1534 ),
1535 }));
1536 }
1537 if !status.is_success() {
1538 return Err(Attempt::Failed(SourceError::Unavailable {
1539 message: format!("GitHub GraphQL returned HTTP {status}"),
1540 }));
1541 }
1542 self.answer(&body).map_err(Attempt::Failed)
1543 }
1544
1545 /// What one successful HTTP response says, once its GraphQL errors are read.
1546 fn answer(&self, body: &str) -> Result<Value, SourceError> {
1547 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
1548 message: format!("GitHub returned invalid JSON: {e}"),
1549 })?;
1550 let errors = body
1551 .get("errors")
1552 .map(|value| {
1553 value.as_array().ok_or_else(|| SourceError::Malformed {
1554 message: "GitHub response errors is not an array".into(),
1555 })
1556 })
1557 .transpose()?;
1558 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
1559 let messages = errors
1560 .iter()
1561 .filter_map(|e| e.get("message").and_then(Value::as_str))
1562 .collect::<Vec<_>>()
1563 .join("; ");
1564 let message = if messages.is_empty() {
1565 "GitHub returned GraphQL errors".into()
1566 } else {
1567 messages
1568 };
1569 let normalized = message.to_ascii_lowercase();
1570 if normalized.contains("resource not accessible") || normalized.contains("scope") {
1571 return Err(SourceError::Auth {
1572 message: format!(
1573 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
1574 self.credential_name
1575 ),
1576 });
1577 }
1578 return Err(SourceError::Refused { message });
1579 }
1580 body.get("data")
1581 .filter(|data| data.is_object())
1582 .cloned()
1583 .ok_or_else(|| SourceError::Malformed {
1584 message: "GitHub response has no data object".into(),
1585 })
1586 }
1587
1588 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
1589 // GraphQL cannot independently page them inside the outer item page. This source page is
1590 // deliberately bounded at that published maximum; the live drift journey exercises it.
1591 async fn board_page(
1592 &self,
1593 items_after: Option<&str>,
1594 items_first: u32,
1595 ) -> Result<Value, SourceError> {
1596 let data = self
1597 .graphql(
1598 graphql::BOARD,
1599 json!({"owner":self.owner,"number":self.project_number,
1600 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
1601 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
1602 )
1603 .await?;
1604 data.pointer("/owner/projectV2")
1605 .filter(|v| !v.is_null())
1606 .cloned()
1607 .ok_or_else(|| SourceError::Refused {
1608 message: format!(
1609 "GitHub project {}/{} was not found or is not visible to the token",
1610 self.owner, self.project_number
1611 ),
1612 })
1613 }
1614
1615 /// The search that finds the issues of this board, narrowed by `also` when it is
1616 /// given.
1617 ///
1618 /// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
1619 /// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
1620 /// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
1621 /// here on purpose — GitHub accepts it and silently ignores it, so a project is told
1622 /// from a task by the `parent` field each issue carries rather than by the search.
1623 fn board_search(&self, also: Option<&str>) -> String {
1624 let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
1625 match also {
1626 Some(also) => format!("{scope} {also}"),
1627 None => scope,
1628 }
1629 }
1630
1631 /// One issue this source reached directly, as the board item a read of the board would
1632 /// have produced — or `None` when this board does not hold it.
1633 ///
1634 /// The board half of an issue rides along on `Issue.projectItems`, so the value handed
1635 /// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
1636 /// item's own id, that item's field values, and the issue as its content. One resolver
1637 /// for both routes is what makes an issue read through a search, through its own node
1638 /// id, or through its project's sub-issues report the same title, the same status, the
1639 /// same labels and the same qualified id.
1640 ///
1641 /// An issue with no entry for *this* board is not this source's to report, which is
1642 /// what keeps an id naming some other repository's issue from being answered as an item
1643 /// of this board.
1644 fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
1645 if optional_str(issue, "__typename")? != Some("Issue") {
1646 return Ok(None);
1647 }
1648 let memberships = issue
1649 .get("projectItems")
1650 .ok_or_else(|| SourceError::Malformed {
1651 message: "GitHub issue is missing projectItems".into(),
1652 })?;
1653 let nodes = memberships
1654 .get("nodes")
1655 .and_then(Value::as_array)
1656 .ok_or_else(|| SourceError::Malformed {
1657 message: "GitHub issue projectItems.nodes is not an array".into(),
1658 })?;
1659 let held = nodes.iter().find(|node| {
1660 node.pointer("/project/number").and_then(Value::as_u64)
1661 == Some(u64::from(self.project_number))
1662 });
1663 let Some(held) = held else {
1664 // Only now: an issue whose entry for this board sits past the page asked for
1665 // would otherwise read as an issue this board does not hold, which is the one
1666 // wrong answer available here.
1667 complete_connection(
1668 memberships,
1669 "issue board memberships",
1670 BOARD_ITEMS_PAGE_SIZE,
1671 )?;
1672 return Ok(None);
1673 };
1674 let item = json!({
1675 "id": required_str(held, "id")?,
1676 "fieldValues": held.get("fieldValues"),
1677 "content": issue,
1678 });
1679 self.resolve(&item)
1680 }
1681
1682 /// One page of a board-scoped issue search, and where the next page resumes.
1683 async fn search_page(
1684 &self,
1685 search: &str,
1686 first: u32,
1687 after: Option<&str>,
1688 ) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
1689 let data = self
1690 .graphql(
1691 graphql::SEARCH_ISSUES,
1692 json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
1693 "after":after,"nestedFirst":NESTED_PAGE_SIZE,
1694 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
1695 )
1696 .await?;
1697 let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
1698 message: "GitHub search response has no search connection".into(),
1699 })?;
1700 let mut found = Vec::new();
1701 for node in connection
1702 .get("nodes")
1703 .and_then(Value::as_array)
1704 .ok_or_else(|| SourceError::Malformed {
1705 message: "GitHub search nodes is not an array".into(),
1706 })?
1707 {
1708 if let Some(resolved) = self.resolve_issue(node)? {
1709 found.push(resolved);
1710 }
1711 }
1712 let info = connection
1713 .get("pageInfo")
1714 .ok_or_else(|| SourceError::Malformed {
1715 message: "GitHub search connection has no pageInfo".into(),
1716 })?;
1717 let next = required_bool(info, "hasNextPage")?
1718 .then(|| required_str(info, "endCursor"))
1719 .transpose()?
1720 .map(str::to_owned);
1721 if let Some(next) = &next {
1722 validate_cursor_progress(after, next)?;
1723 }
1724 Ok((found, next))
1725 }
1726
1727 /// Every issue this board holds, walked to exhaustion, completed with what this run
1728 /// wrote.
1729 ///
1730 /// The completion is not an optimisation and it is not a cache: GitHub's issue search
1731 /// is an index and is eventually consistent, so an issue this run created seconds ago
1732 /// is routinely absent from it, and a project listed straight after being written would
1733 /// otherwise be missing from its own board. What is added back is only what this
1734 /// process itself wrote, out of [`Self::created`], which lives and dies with the
1735 /// process.
1736 async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
1737 let mut after: Option<String> = None;
1738 let mut found = Vec::new();
1739 let search = self.board_search(None);
1740 loop {
1741 let (page, next) = self
1742 .search_page(&search, MAX_PAGE_SIZE, after.as_deref())
1743 .await?;
1744 found.extend(page);
1745 match next {
1746 Some(next) => after = Some(next),
1747 None => break,
1748 }
1749 }
1750 self.completed_with_written(found, |_| true)
1751 }
1752
1753 /// `found`, with everything this run wrote that `keep` accepts and the read did not
1754 /// report.
1755 ///
1756 /// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
1757 /// at all: the search index is behind, and a node read of an item filed moments ago can
1758 /// be too.
1759 fn completed_with_written(
1760 &self,
1761 mut found: Vec<Resolved>,
1762 keep: impl Fn(&Resolved) -> bool,
1763 ) -> Result<Vec<Resolved>, SourceError> {
1764 for own in self.created()?.iter().filter(|own| keep(own)) {
1765 if !found.iter().any(|item| item.id == own.id) {
1766 found.push(own.clone());
1767 }
1768 }
1769 Ok(found)
1770 }
1771
1772 /// What resolving one node id reached.
1773 ///
1774 /// Three answers rather than an `Option`, because a board *draft* is none of the other
1775 /// two: it is not an issue, it has no node of its own this source can read the board
1776 /// half off, and its only home is the board's own item connection — so a read of one
1777 /// is completed from there rather than reported as nothing.
1778 async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
1779 let asked = self
1780 .graphql(
1781 graphql::ISSUE,
1782 json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
1783 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
1784 )
1785 .await;
1786 let data = match asked {
1787 Ok(data) => data,
1788 // A string that is not a node id at all is not a failure to report: it is an id
1789 // this board does not hold, which is what every read of one already answers.
1790 Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
1791 Err(error) => return Err(error),
1792 };
1793 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
1794 return Ok(Reached::Nothing);
1795 };
1796 if optional_str(node, "__typename")? == Some("DraftIssue") {
1797 return Ok(Reached::Draft);
1798 }
1799 Ok(match self.resolve_issue(node)? {
1800 Some(item) => Reached::Held(Box::new(item)),
1801 None => Reached::Nothing,
1802 })
1803 }
1804
1805 /// One item of this board by its own id, whatever kind it is.
1806 ///
1807 /// Resolved from the identifier alone: no search, board-wide or otherwise. What this
1808 /// run wrote is read first, because a node read of an item created moments ago can
1809 /// still be behind the board field values written onto it — see [`Self::created`].
1810 async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
1811 if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
1812 return Ok(Some(own.clone()));
1813 }
1814 match self.reach(id).await? {
1815 Reached::Held(item) => Ok(Some(*item)),
1816 Reached::Nothing => Ok(None),
1817 // The one read that still costs the board: a draft lives nowhere else.
1818 Reached::Draft => Ok(self
1819 .board()
1820 .await?
1821 .items
1822 .into_iter()
1823 .find(|item| item.id == *id)),
1824 }
1825 }
1826
1827 /// Everything filed under one issue of this board, walked to exhaustion — or `None`
1828 /// when that id names nothing here with a sub-issue relationship to walk.
1829 ///
1830 /// `None` and an empty answer are different: `None` is *this is not an issue of this
1831 /// GitHub*, which is what sends a project selector on to be read as a name, and an
1832 /// empty vector is a project that holds nothing.
1833 async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
1834 let mut after: Option<String> = None;
1835 let mut children = Vec::new();
1836 loop {
1837 let asked = self
1838 .graphql(
1839 graphql::SUB_ISSUES,
1840 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
1841 "nestedFirst":NESTED_PAGE_SIZE,
1842 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
1843 )
1844 .await;
1845 let data = match asked {
1846 Ok(data) => data,
1847 // A string that is not a node id at all is not a failure to report: it is
1848 // the ordinary answer to a selector naming a project by its name.
1849 Err(error) if unresolvable_node(&error) => return Ok(None),
1850 Err(error) => return Err(error),
1851 };
1852 let Some(connection) = data
1853 .pointer("/node/subIssues")
1854 .filter(|value| !value.is_null())
1855 else {
1856 // No such node, or one with no sub-issue relationship — a board draft is
1857 // the one this board can really hold.
1858 return Ok(None);
1859 };
1860 for node in connection
1861 .get("nodes")
1862 .and_then(Value::as_array)
1863 .ok_or_else(|| SourceError::Malformed {
1864 message: "GitHub subIssues.nodes is not an array".into(),
1865 })?
1866 {
1867 if let Some(resolved) = self.resolve_issue(node)? {
1868 children.push(resolved);
1869 }
1870 }
1871 let info = connection
1872 .get("pageInfo")
1873 .ok_or_else(|| SourceError::Malformed {
1874 message: "GitHub subIssues connection has no pageInfo".into(),
1875 })?;
1876 let next = required_bool(info, "hasNextPage")?
1877 .then(|| required_str(info, "endCursor"))
1878 .transpose()?;
1879 match next {
1880 Some(next) => {
1881 validate_cursor_progress(after.as_deref(), next)?;
1882 after = Some(next.to_owned());
1883 }
1884 None => return Ok(Some(children)),
1885 }
1886 }
1887 }
1888
1889 /// Which issue of this board a project *name* is, or `None` when none is.
1890 ///
1891 /// One bounded query which filters on that name at the server, rather than a walk of
1892 /// every issue the board holds. The name is compared again here: the qualifier narrows
1893 /// what GitHub sends, and this source decides what it names.
1894 async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
1895 let search = self.board_search(Some(&title_qualifier(name)));
1896 let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
1897 Ok(candidates
1898 .into_iter()
1899 .find(|item| {
1900 item.kind == BoardKind::Work(ItemKind::Project)
1901 && item.title.eq_ignore_ascii_case(name)
1902 })
1903 .map(|item| item.id))
1904 }
1905
1906 /// Everything filed under one project of this board: the sub-issues of the issue that
1907 /// project is.
1908 ///
1909 /// Tasks *and* documents, because a document filed under a project is a sub-issue of it
1910 /// too — the caller keeps the kind it asked for. Nothing about this grows as the board
1911 /// gains projects, or as another project gains tasks.
1912 ///
1913 /// A qualified id names the issue and is asked for its sub-issues directly: one
1914 /// request, no search of any kind. Only a selector GitHub cannot resolve that way is
1915 /// read as a project *name*, which costs the one bounded search
1916 /// [`Self::project_by_name`] makes.
1917 async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
1918 let (project, children) = match self.sub_issues(selector).await? {
1919 Some(children) => (selector.clone(), children),
1920 None => match self.project_by_name(&selector.0).await? {
1921 Some(project) => {
1922 let children = self.sub_issues(&project).await?.unwrap_or_default();
1923 (project, children)
1924 }
1925 None => return Ok(Vec::new()),
1926 },
1927 };
1928 self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
1929 }
1930
1931 /// Every item on the board, with the one board identity they all share.
1932 ///
1933 /// See [`Self::board_cache`]. The completion from `created` happens on every call
1934 /// rather than once, which is what the cache could otherwise have broken.
1935 async fn board(&self) -> Result<Board, SourceError> {
1936 let cached = self.board_cache()?.clone();
1937 let mut board = match cached {
1938 Some(board) => board,
1939 None => {
1940 let read = self.read_board().await?;
1941 *self.board_cache()? = Some(read.clone());
1942 read
1943 }
1944 };
1945 for own in self.created()?.iter() {
1946 if !board.items.iter().any(|item| item.id == own.id) {
1947 board.items.push(own.clone());
1948 }
1949 }
1950 Ok(board)
1951 }
1952
1953 /// This process's own view of the board, or the refusal a poisoned lock is.
1954 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
1955 self.board_cache
1956 .lock()
1957 .map_err(|_| SourceError::Unavailable {
1958 message: "this source's view of the board was left inconsistent by an earlier \
1959 failure; next: run the command again"
1960 .into(),
1961 })
1962 }
1963
1964 /// Bring this process's own view of the board up to an item it has just written.
1965 ///
1966 /// A created item goes to `created`, which is what completes a board read GitHub's own
1967 /// eventual consistency has left behind. An item that was already there is replaced
1968 /// where it sits, so a second write of it in the same command reads its real parent
1969 /// rather than the one it had before the first write.
1970 ///
1971 /// "Where it sits" is two places, and missing the first leaves a stale record that
1972 /// wins: an item this same run created is held in `created` and not in the cached
1973 /// board, and `board` completes the cached board *from* `created`, so replacing only
1974 /// the cached copy of such an item replaces nothing and the read still reports the
1975 /// title it was created with.
1976 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
1977 if created {
1978 self.created()?.push(item);
1979 return Ok(());
1980 }
1981 {
1982 let mut own = self.created()?;
1983 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
1984 *held = item;
1985 return Ok(());
1986 }
1987 }
1988 if let Some(board) = self.board_cache()?.as_mut()
1989 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
1990 {
1991 *held = item;
1992 }
1993 Ok(())
1994 }
1995
1996 /// Forget one item this process has just deleted, from both halves of its own view.
1997 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
1998 self.created()?.retain(|own| own.id != *id);
1999 if let Some(board) = self.board_cache()?.as_mut() {
2000 board.items.retain(|item| item.id != *id);
2001 }
2002 Ok(())
2003 }
2004
2005 /// Every page of the board, read from GitHub.
2006 async fn read_board(&self) -> Result<Board, SourceError> {
2007 let mut after: Option<String> = None;
2008 let mut items = Vec::new();
2009 let mut board;
2010 loop {
2011 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
2012 for item in page
2013 .pointer("/items/nodes")
2014 .and_then(Value::as_array)
2015 .ok_or_else(|| SourceError::Malformed {
2016 message: "GitHub project items.nodes is not an array".into(),
2017 })?
2018 {
2019 if let Some(resolved) = self.resolve(item)? {
2020 items.push(resolved);
2021 }
2022 }
2023 let info = page
2024 .pointer("/items/pageInfo")
2025 .ok_or_else(|| SourceError::Malformed {
2026 message: "GitHub project items have no pageInfo".into(),
2027 })?;
2028 let has_next = required_bool(info, "hasNextPage")?;
2029 let next = has_next
2030 .then(|| required_str(info, "endCursor"))
2031 .transpose()?;
2032 board = page.clone();
2033 match next {
2034 Some(next) => {
2035 validate_cursor_progress(after.as_deref(), next)?;
2036 after = Some(next.to_owned());
2037 }
2038 None => break,
2039 }
2040 }
2041 Ok(Board {
2042 id: required_str(&board, "id")?.to_owned(),
2043 fields: board.get("fields").cloned().unwrap_or(Value::Null),
2044 items,
2045 })
2046 }
2047
2048 /// The items this source has created, for completing a board read that is behind.
2049 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
2050 self.created.lock().map_err(|_| SourceError::Unavailable {
2051 message: "this source's record of what it created in this run was left \
2052 inconsistent by an earlier failure; next: run the command again"
2053 .into(),
2054 })
2055 }
2056
2057 /// One board item as this source reports it, or `None` for content it ignores.
2058 ///
2059 /// A pull request is neither a project nor a task — it is somebody's change, not a
2060 /// unit of plan — and an item whose content the token cannot see has nothing to
2061 /// report at all.
2062 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
2063 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
2064 message: "GitHub project item is missing content".into(),
2065 })?;
2066 if content.is_null() {
2067 return Ok(None);
2068 }
2069 let content_kind = match required_str(content, "__typename")? {
2070 "Issue" => ContentKind::Issue,
2071 "DraftIssue" => ContentKind::DraftIssue,
2072 _ => return Ok(None),
2073 };
2074 let field_values = item
2075 .get("fieldValues")
2076 .ok_or_else(|| SourceError::Malformed {
2077 message: "GitHub project item is missing fieldValues".into(),
2078 })?;
2079 complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
2080 let nodes = field_values
2081 .get("nodes")
2082 .and_then(Value::as_array)
2083 .ok_or_else(|| SourceError::Malformed {
2084 message: "GitHub project item fieldValues.nodes is not an array".into(),
2085 })?;
2086 if let Some(labels) = content.get("labels") {
2087 complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
2088 }
2089 for field_value in nodes {
2090 if let Some(labels) = field_value.get("labels") {
2091 complete_connection(labels, "project item field labels", NESTED_PAGE_SIZE)?;
2092 }
2093 }
2094 let (body, slot) = metadata_body(optional_str(content, "body")?.map(str::to_owned))?;
2095 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
2096 .map(|id| NativeId(id.to_owned()));
2097 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
2098 // to read one from; it is a task, and never a project.
2099 let sub_issues = match content_kind {
2100 ContentKind::Issue => sub_issue_total(content)?,
2101 ContentKind::DraftIssue => 0,
2102 };
2103 let content_id = required_str(content, "id")?;
2104 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
2105 message: format!("GitHub issue {content_id}: {message}"),
2106 })?;
2107 let raw_title = required_str(content, "title")?;
2108 // The design prefix is read *first*, before either of the two rules that separate
2109 // a project from a task. A document is not work whatever sub-issues it has and
2110 // whatever marker it carries, and reading the prefix later would make a design
2111 // issue with none of either an empty project.
2112 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
2113 BoardKind::Document
2114 } else if parent.is_some() {
2115 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
2116 // under a project is that project's task even when it has sub-issues of its
2117 // own.
2118 BoardKind::Work(ItemKind::Task)
2119 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
2120 BoardKind::Work(ItemKind::Project)
2121 } else {
2122 BoardKind::Work(ItemKind::Task)
2123 };
2124 // The title a person wrote, which for a document is the one without the prefix —
2125 // the same way `content` above is the body without this source's metadata slot.
2126 let title = match kind {
2127 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
2128 BoardKind::Work(_) => raw_title.to_owned(),
2129 };
2130 let own_repository = content
2131 .pointer("/repository/nameWithOwner")
2132 .and_then(Value::as_str)
2133 .map(|origin| Repository::try_from(format!("github.com/{origin}")))
2134 .transpose()
2135 .map_err(|message| SourceError::Malformed { message })?;
2136 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
2137 Repository::from_metadata(&slot)
2138 .map_err(|message| SourceError::Malformed { message })?
2139 } else {
2140 own_repository.clone().into_iter().collect()
2141 };
2142 Ok(Some(Resolved {
2143 item_id: required_str(item, "id")?.to_owned(),
2144 id: NativeId(content_id.to_owned()),
2145 content_kind,
2146 kind,
2147 title,
2148 body: body.filter(|value| !value.is_empty()),
2149 status: self.status(item, content)?,
2150 labels: labels(content, nodes)?,
2151 parent,
2152 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
2153 url: optional_str(content, "url")?.map(str::to_owned),
2154 created_at: optional_time(content, "createdAt")?,
2155 updated_at: optional_time(content, "updatedAt")?,
2156 own_repository,
2157 repositories,
2158 slot,
2159 }))
2160 }
2161
2162 /// The status one board item reports.
2163 ///
2164 /// The closed state decides the category and the `Status` option decides the name, so
2165 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
2166 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
2167 /// duplicate is not finished work, and calling it done is a lie the next copy would
2168 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
2169 /// it is read permissively rather than refused — reads are faithful, and refusals
2170 /// belong on writes.
2171 fn status(&self, item: &Value, content: &Value) -> Result<Status, SourceError> {
2172 let nodes = item
2173 .pointer("/fieldValues/nodes")
2174 .and_then(Value::as_array)
2175 .expect("resolve validates fieldValues.nodes before mapping status");
2176 let option = nodes
2177 .iter()
2178 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
2179 .map(|value| required_str(value, "name"))
2180 .transpose()?;
2181 let state = optional_str(content, "state")?;
2182 if state == Some("CLOSED") {
2183 let category = match optional_str(content, "stateReason")? {
2184 None | Some("COMPLETED") => StatusCategory::Done,
2185 Some("NOT_PLANNED") => StatusCategory::Cancelled,
2186 Some(_) => StatusCategory::Unknown,
2187 };
2188 let fallback = match category {
2189 StatusCategory::Done => "Done",
2190 StatusCategory::Cancelled => "Cancelled",
2191 _ => "Closed",
2192 };
2193 return Ok(Status {
2194 category,
2195 name: option.unwrap_or(fallback).to_owned(),
2196 });
2197 }
2198 let name = option.unwrap_or("Open").to_owned();
2199 Ok(Status {
2200 category: self
2201 .statuses
2202 .category_of(&name)
2203 .unwrap_or(StatusCategory::Unknown),
2204 name,
2205 })
2206 }
2207
2208 /// The board Status option this write selects, or the refusal that says why not.
2209 ///
2210 /// For a column target the option is what the status *is*, so a board that has no such
2211 /// option is a refusal naming the status and the instance. For a closed target the
2212 /// issue's own state carries the category, and the option carries only the name a
2213 /// reader reports — so an option spelled the way this status is spelled is selected
2214 /// when the board has one, and nothing is refused when it does not.
2215 fn column_for(
2216 &self,
2217 board: &Board,
2218 status: &Status,
2219 target: &StatusTarget,
2220 ) -> Result<Option<(String, String)>, SourceError> {
2221 let (wanted, required) = match target {
2222 StatusTarget::Column(wanted) => (wanted.as_str(), true),
2223 StatusTarget::Closed(_) => (status.name.as_str(), false),
2224 StatusTarget::Disabled => return Ok(None),
2225 };
2226 let missing = |detail: &str| SourceError::Refused {
2227 message: format!(
2228 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
2229 category_name(status.category),
2230 self.name,
2231 category_name(status.category)
2232 ),
2233 };
2234 let Some(field) = Board::field(&board.fields, "Status")? else {
2235 return if required {
2236 Err(missing("this board has no Status field"))
2237 } else {
2238 Ok(None)
2239 };
2240 };
2241 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
2242 return if required {
2243 Err(missing(
2244 "this board's Status field is not a single-select field",
2245 ))
2246 } else {
2247 Ok(None)
2248 };
2249 }
2250 let option = field
2251 .get("options")
2252 .and_then(Value::as_array)
2253 .and_then(|options| {
2254 options.iter().find(|option| {
2255 option
2256 .get("name")
2257 .and_then(Value::as_str)
2258 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
2259 })
2260 });
2261 match option {
2262 None if required => Err(missing("this board does not have it")),
2263 None => Ok(None),
2264 Some(option) => Ok(Some((
2265 required_str(field, "id")?.to_owned(),
2266 required_str(option, "id")?.to_owned(),
2267 ))),
2268 }
2269 }
2270
2271 /// This instance's target for a category, refusing one it has disabled.
2272 ///
2273 /// Nothing here mutates the board's option set to make room for a status. GitHub
2274 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
2275 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
2276 /// field and every item's status.
2277 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
2278 let target = self.statuses.target(category).clone();
2279 if target != StatusTarget::Disabled {
2280 return Ok(target);
2281 }
2282 Err(SourceError::Refused {
2283 message: if category == StatusCategory::Draft {
2284 format!(
2285 "status draft is disabled for source {}: draft is incompatible with this \
2286 integration because GitHub draft issues cannot have sub-issues, and this \
2287 source stores a project's tasks as its issue's sub-issues",
2288 self.name
2289 )
2290 } else {
2291 format!(
2292 "status {} is disabled for source {}; set status_mapping.{} of this source \
2293 to a board Status option name or to a closed state",
2294 category_name(category),
2295 self.name,
2296 category_name(category)
2297 )
2298 },
2299 })
2300 }
2301
2302 async fn set_item_field(
2303 &self,
2304 board_id: &str,
2305 item_id: &str,
2306 field_id: &str,
2307 value: Value,
2308 ) -> Result<(), SourceError> {
2309 let data = self
2310 .graphql(
2311 graphql::UPDATE_FIELD,
2312 json!({"input":{
2313 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
2314 }}),
2315 )
2316 .await?;
2317 let returned = data
2318 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
2319 .ok_or_else(|| SourceError::Malformed {
2320 message: "GitHub field update returned no project item".into(),
2321 })?;
2322 if required_str(returned, "id")? != item_id {
2323 return Err(SourceError::Malformed {
2324 message: "GitHub field update returned the wrong project item".into(),
2325 });
2326 }
2327 Ok(())
2328 }
2329
2330 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
2331 let mut after: Option<String> = None;
2332 let mut ids = Vec::new();
2333 loop {
2334 let data = self
2335 .graphql(
2336 graphql::ISSUE_DEPENDENCIES,
2337 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
2338 )
2339 .await?;
2340 let connection =
2341 data.pointer("/node/blockedBy")
2342 .ok_or_else(|| SourceError::Malformed {
2343 message: "GitHub dependency response has no blockedBy connection".into(),
2344 })?;
2345 ids.extend(
2346 connection
2347 .get("nodes")
2348 .and_then(Value::as_array)
2349 .ok_or_else(|| SourceError::Malformed {
2350 message: "GitHub dependency response nodes is not an array".into(),
2351 })?
2352 .iter()
2353 .map(|value| required_str(value, "id").map(str::to_owned))
2354 .collect::<Result<Vec<_>, _>>()?,
2355 );
2356 let next = next_cursor(connection)?;
2357 if let Some(next) = &next {
2358 validate_cursor_progress(after.as_deref(), &next.0)?;
2359 }
2360 after = next.map(|cursor| cursor.0);
2361 if after.is_none() {
2362 return Ok(ids);
2363 }
2364 }
2365 }
2366
2367 async fn dependencies(
2368 &self,
2369 id: &NativeId,
2370 near_kind: ItemKind,
2371 direction: Direction,
2372 page: &PageRequest,
2373 ) -> Result<Page<DependencyEdge>, SourceError> {
2374 validate_page(page)?;
2375 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
2376 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
2377 let recorded = recorded_offset(cursor, direction)?;
2378 // Asked for even in the recorded phase, whose page reads nothing from the
2379 // connection: `__typename` is what says whether this item has a native
2380 // relationship at all, and that is what decides which far ends the reserved key is
2381 // allowed to hold.
2382 let data = self
2383 .graphql(
2384 graphql::ISSUE_DEPENDENCIES,
2385 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
2386 "after":if recorded.is_some() {None} else {cursor}}),
2387 )
2388 .await?;
2389 let node =
2390 data.get("node")
2391 .filter(|v| !v.is_null())
2392 .ok_or_else(|| SourceError::Refused {
2393 message: format!(
2394 "GitHub item {} was not found or does not support dependencies",
2395 id.0
2396 ),
2397 })?;
2398 let connection_name = match direction {
2399 Direction::DependsOn => "blockedBy",
2400 Direction::DependedOnBy => "blocking",
2401 };
2402 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
2403 // named natively and the reserved key may hold any far end. An issue's connections
2404 // hold issues, and this source reads them at the near item's own level.
2405 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
2406 if let Some(offset) = recorded {
2407 return Ok(recorded_page(
2408 self.recorded_edges(id, near_kind, direction, natively_names)
2409 .await?,
2410 offset,
2411 limit,
2412 ));
2413 }
2414 if natively_names.is_none() {
2415 return Ok(recorded_page(
2416 self.recorded_edges(id, near_kind, direction, natively_names)
2417 .await?,
2418 0,
2419 limit,
2420 ));
2421 }
2422 let connection = node
2423 .get(connection_name)
2424 .ok_or_else(|| SourceError::Malformed {
2425 message: "GitHub dependency response is missing its connection".into(),
2426 })?;
2427 let nodes = connection
2428 .get("nodes")
2429 .and_then(Value::as_array)
2430 .ok_or_else(|| SourceError::Malformed {
2431 message: "GitHub dependency response nodes is not an array".into(),
2432 })?;
2433 // `from` depends on `to`, always. GitHub spells the same relationship from either
2434 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
2435 // it — so the near item is `from` in one direction and `to` in the other.
2436 let items = nodes
2437 .iter()
2438 .map(|value| {
2439 let related = NativeId(required_str(value, "id")?.into());
2440 let related_kind = related_kind(value)?;
2441 let (from, to) = match direction {
2442 Direction::DependsOn => (
2443 DependencyEndpoint::from_native(id.clone(), near_kind),
2444 DependencyEndpoint::from_native(related, related_kind),
2445 ),
2446 Direction::DependedOnBy => (
2447 DependencyEndpoint::from_native(related, related_kind),
2448 DependencyEndpoint::from_native(id.clone(), near_kind),
2449 ),
2450 };
2451 Ok(DependencyEdge {
2452 from,
2453 to,
2454 kind: DependencyKind::Blocks,
2455 })
2456 })
2457 .collect::<Result<Vec<_>, SourceError>>()?;
2458 let mut next = next_cursor(connection)?;
2459 if let Some(next) = &next {
2460 validate_cursor_progress(cursor, &next.0)?;
2461 }
2462 if next.is_none()
2463 && !self
2464 .recorded_edges(id, near_kind, direction, natively_names)
2465 .await?
2466 .is_empty()
2467 {
2468 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
2469 }
2470 Ok(Page { items, next })
2471 }
2472
2473 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
2474 /// a far end in another source has to live: no GitHub issue relationship can name one.
2475 ///
2476 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
2477 /// source never writes one down.
2478 ///
2479 /// The metadata lives in the item's own body slot, so reading it costs one board scan.
2480 /// That is why it happens once the native connection is spent rather than on every
2481 /// page.
2482 async fn recorded_edges(
2483 &self,
2484 id: &NativeId,
2485 near_kind: ItemKind,
2486 direction: Direction,
2487 natively_names: Option<ItemKind>,
2488 ) -> Result<Vec<DependencyEdge>, SourceError> {
2489 if direction != Direction::DependsOn {
2490 return Ok(Vec::new());
2491 }
2492 let Some(item) = self
2493 .board()
2494 .await?
2495 .items
2496 .into_iter()
2497 .find(|item| item.id == *id)
2498 else {
2499 return Ok(Vec::new());
2500 };
2501 DependencyEdge::recorded(&item.slot, id, near_kind, &self.name, natively_names)
2502 .map_err(|message| SourceError::Malformed { message })
2503 }
2504
2505 /// The configured repository's node id, or the refusal naming the field it needs.
2506 ///
2507 /// Resolved once per command; see [`Self::repository_cache`].
2508 async fn repository_id(&self) -> Result<String, SourceError> {
2509 if let Some(id) = self.repository_cache()?.clone() {
2510 return Ok(id);
2511 }
2512 let repository = self
2513 .repository
2514 .as_ref()
2515 .ok_or_else(|| SourceError::Refused {
2516 message: format!(
2517 "source {} has no repository configured, and a GitHub Projects board has no \
2518 repository of its own to create an issue in; set repository: owner/name on \
2519 this source",
2520 self.name
2521 ),
2522 })?;
2523 let data = self
2524 .graphql(
2525 graphql::REPOSITORY,
2526 json!({"owner":repository.owner,"name":repository.name}),
2527 )
2528 .await?;
2529 let node = data
2530 .get("repository")
2531 .filter(|value| !value.is_null())
2532 .ok_or_else(|| SourceError::Refused {
2533 message: format!(
2534 "GitHub repository {}/{} was not found or is not visible to the token",
2535 repository.owner, repository.name
2536 ),
2537 })?;
2538 let id = required_str(node, "id")?.to_owned();
2539 *self.repository_cache()? = Some(id.clone());
2540 Ok(id)
2541 }
2542
2543 /// This process's own record of the destination repository's node id.
2544 fn repository_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<String>>, SourceError> {
2545 self.repository_cache
2546 .lock()
2547 .map_err(|_| SourceError::Unavailable {
2548 message: "this source's record of the destination repository was left \
2549 inconsistent by an earlier failure; next: run the command again"
2550 .into(),
2551 })
2552 }
2553
2554 /// Create or update one board item, whichever kind it is.
2555 async fn write_item(
2556 &self,
2557 incoming: &Incoming<'_>,
2558 target: Option<&NativeId>,
2559 depends_on: &[DependencyEdge],
2560 ) -> Result<NativeId, SourceError> {
2561 // Refused before anything is read or written: a task or a project titled the way
2562 // this board spells a document would land as an issue this same source reads back
2563 // as a document, so the field this destination cannot carry is named rather than
2564 // written and silently reclassified.
2565 if let Written::Work(kind, _) = incoming.written
2566 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
2567 {
2568 return Err(SourceError::Refused {
2569 message: format!(
2570 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
2571 spells a document, so it would read back as one rather than as a {}; \
2572 retitle it, or copy it as a document",
2573 kind.marker(),
2574 self.name,
2575 kind.marker()
2576 ),
2577 });
2578 }
2579 let board = self.board().await?;
2580 let status_target = incoming
2581 .written
2582 .status()
2583 .map(|status| self.resolved_target(status.category))
2584 .transpose()?;
2585 let column = match (incoming.written.status(), status_target.as_ref()) {
2586 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
2587 _ => None,
2588 };
2589 let existing = target
2590 .map(|target| {
2591 board
2592 .items
2593 .iter()
2594 .find(|item| item.id == *target)
2595 .ok_or_else(|| SourceError::Refused {
2596 message: format!("GitHub destination item {} was not found", target.0),
2597 })
2598 })
2599 .transpose()?;
2600 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
2601 if content_kind == ContentKind::DraftIssue {
2602 if let (Some(StatusTarget::Closed(_)), Some(status)) =
2603 (status_target.as_ref(), incoming.written.status())
2604 {
2605 return Err(SourceError::Refused {
2606 message: format!(
2607 "status {} of source {} closes the item's issue, and GitHub draft items \
2608 have no open or closed state",
2609 category_name(status.category),
2610 self.name
2611 ),
2612 });
2613 }
2614 if incoming.parent.is_some() {
2615 return Err(SourceError::Refused {
2616 message: "GitHub draft items cannot be a project's sub-issue".into(),
2617 });
2618 }
2619 }
2620 match existing {
2621 Some(item) if content_kind == ContentKind::Issue => {
2622 if item.labels != incoming.labels {
2623 return Err(SourceError::Refused {
2624 message: "GitHub issue labels differ from the labels being written".into(),
2625 });
2626 }
2627 }
2628 _ => {
2629 if !incoming.labels.is_empty() {
2630 return Err(SourceError::Refused {
2631 message: "GitHub items created by this destination carry no labels".into(),
2632 });
2633 }
2634 }
2635 }
2636
2637 let own_repository = match existing {
2638 Some(item) => item.own_repository.clone(),
2639 None => self
2640 .repository
2641 .as_ref()
2642 .map(|repository| Repository::try_from(repository.origin()))
2643 .transpose()
2644 .map_err(|message| SourceError::Config { message })?,
2645 };
2646 let (native, fallback) = self
2647 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
2648 .await?;
2649 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
2650 let body = compose_body(incoming.content, &slot)?;
2651 // Read before anything is created, for the reason the field below is: a value
2652 // this destination cannot store has to refuse, and refusing after `createIssue`
2653 // would leave an issue behind that nothing asked for. The engine writes a
2654 // qualified id here; a caller handing this key anything else is told so rather
2655 // than having it silently stored as no origin at all.
2656 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
2657 let origin = match incoming.metadata.get(ORIGIN_KEY) {
2658 None => "",
2659 Some(Value::String(origin)) => origin.as_str(),
2660 Some(other) => {
2661 return Err(SourceError::Refused {
2662 message: format!(
2663 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
2664 is {other}"
2665 ),
2666 });
2667 }
2668 };
2669 // Resolved before anything is created: a board that cannot carry the copy origin
2670 // has to refuse the write, and refusing it after `createIssue` would leave an
2671 // issue behind that nothing asked for.
2672 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
2673 Some(field) => {
2674 if required_str(field, "__typename")? != "ProjectV2Field" {
2675 return Err(SourceError::Refused {
2676 message: format!(
2677 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
2678 ),
2679 });
2680 }
2681 Some(required_str(field, "id")?.to_owned())
2682 }
2683 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
2684 return Err(SourceError::Refused {
2685 message: format!(
2686 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
2687 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
2688 the board"
2689 ),
2690 });
2691 }
2692 None => None,
2693 };
2694
2695 let (content_id, item_id, url) = match existing {
2696 Some(item) => {
2697 self.update_existing(item, incoming, &body, status_target.as_ref())
2698 .await?;
2699 (item.id.clone(), item.item_id.clone(), item.url.clone())
2700 }
2701 None => {
2702 self.create_and_file_issue(&board, incoming, &body, status_target.as_ref())
2703 .await?
2704 }
2705 };
2706
2707 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
2708 // then each board field, the parent and the dependencies — and GitHub can fail at
2709 // any of them. Everything this source can refuse *before* the first of those is
2710 // already checked above, so what is left is GitHub itself failing part way. When it
2711 // does over an item this call created, the issue is taken back: a write that
2712 // refused must not leave an item behind that nobody asked for, and one that does
2713 // makes the retry create a second.
2714 let landed = self
2715 .finish_write(
2716 &board,
2717 incoming,
2718 &content_id,
2719 &item_id,
2720 content_kind,
2721 existing,
2722 origin_field.as_deref(),
2723 origin,
2724 column,
2725 &native,
2726 )
2727 .await;
2728 if let Err(error) = landed {
2729 if existing.is_none() {
2730 // Best effort, and the write's own failure is what the caller is told: a
2731 // refusal naming the tidy-up would hide why the write failed at all.
2732 let _ = self.delete_issue(&content_id).await;
2733 }
2734 return Err(error);
2735 }
2736
2737 // So the rest of this command reads what it just did rather than what the board
2738 // said before it. See `remember_written` for which half takes it.
2739 let remembered = Resolved {
2740 item_id,
2741 id: content_id.clone(),
2742 content_kind,
2743 kind: incoming.written.kind(),
2744 title: incoming.title.to_owned(),
2745 // The visible half of the body this write composed, split back off it the
2746 // way a read splits it — so what this record reports is what a read of the
2747 // same issue reports, rather than the person's text with the metadata slot
2748 // still on the end of it.
2749 body: metadata_body(body.clone())?.0,
2750 // A document has no status of its own; what it reads back as is whatever
2751 // the issue's own state says, which is what a re-read reports.
2752 status: incoming
2753 .written
2754 .status()
2755 .cloned()
2756 .unwrap_or_else(|| Status {
2757 category: StatusCategory::Unknown,
2758 name: "Open".to_owned(),
2759 }),
2760 labels: incoming.labels.to_vec(),
2761 parent: incoming.parent.cloned(),
2762 origin: (!origin.is_empty()).then(|| origin.to_owned()),
2763 // In the update path this is the item's own url, read off `existing` where the
2764 // tuple above was bound, so one expression serves both halves.
2765 url,
2766 created_at: existing.and_then(|item| item.created_at),
2767 updated_at: existing.and_then(|item| item.updated_at),
2768 own_repository,
2769 repositories: incoming.repositories.to_vec(),
2770 slot,
2771 };
2772 self.remember_written(remembered, existing.is_none())?;
2773 Ok(content_id)
2774 }
2775
2776 /// Everything a write does after the item exists: its board fields, its parent, and
2777 /// its dependencies.
2778 ///
2779 /// Split out of `write_item` so there is one place a failure past the point of no
2780 /// return is caught, rather than a tidy-up repeated at each `?` above.
2781 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
2782 // so there is one place a failure past the point of no return is caught, and its
2783 // arguments are exactly the values that tail already had in scope. Bundling them into a
2784 // struct would describe no concept — it would be "the arguments of this function" — and
2785 // would put the whole of `write_item`'s locals behind one more indirection.
2786 #[allow(clippy::too_many_arguments)]
2787 async fn finish_write(
2788 &self,
2789 board: &Board,
2790 incoming: &Incoming<'_>,
2791 content_id: &NativeId,
2792 item_id: &str,
2793 content_kind: ContentKind,
2794 existing: Option<&Resolved>,
2795 origin_field: Option<&str>,
2796 origin: &str,
2797 column: Option<(String, String)>,
2798 native: &[String],
2799 ) -> Result<(), SourceError> {
2800 if let Some(field_id) = origin_field {
2801 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
2802 .await?;
2803 }
2804
2805 if let Some((field_id, option_id)) = column {
2806 self.set_item_field(
2807 &board.id,
2808 item_id,
2809 &field_id,
2810 json!({"singleSelectOptionId":option_id}),
2811 )
2812 .await?;
2813 }
2814
2815 if content_kind == ContentKind::Issue {
2816 self.reparent(
2817 existing.and_then(|item| item.parent.clone()),
2818 content_id,
2819 incoming.parent,
2820 )
2821 .await?;
2822 // A document takes part in no dependency graph, so writing one neither reads
2823 // nor changes the issue's own `blockedBy` relationships. Reconciling them
2824 // against the empty list a document write carries would *delete* whatever
2825 // relationships a person had made on that issue, which is a write nobody
2826 // asked for.
2827 if incoming.written.kind() != BoardKind::Document {
2828 self.reconcile_blocked_by(content_id, native).await?;
2829 }
2830 }
2831 Ok(())
2832 }
2833
2834 /// Delete one issue, which takes its board item with it.
2835 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
2836 let data = self
2837 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
2838 .await?;
2839 data.pointer("/deleteIssue/repository")
2840 .filter(|value| !value.is_null())
2841 .ok_or_else(|| SourceError::Malformed {
2842 message: "GitHub issue deletion returned no repository".into(),
2843 })?;
2844 self.forget(id)?;
2845 Ok(())
2846 }
2847
2848 /// Remove one item this copy created, so a copy that could not finish leaves the board
2849 /// as it found it.
2850 ///
2851 /// Deleting the issue takes its board item with it, so there is no second mutation to
2852 /// keep in step. An id the board does not hold is not an error: the item is already
2853 /// gone, which is the state this asks for.
2854 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
2855 let board = self.board().await?;
2856 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
2857 return Ok(());
2858 };
2859 if item.content_kind == ContentKind::DraftIssue {
2860 return Err(SourceError::Refused {
2861 message: format!(
2862 "GitHub item {} is a draft, and this source removes an item by deleting \
2863 its issue; next: remove it from the board by hand",
2864 id.0
2865 ),
2866 });
2867 }
2868 let data = self
2869 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
2870 .await?;
2871 data.pointer("/deleteIssue/repository")
2872 .filter(|value| !value.is_null())
2873 .ok_or_else(|| SourceError::Malformed {
2874 message: "GitHub issue deletion returned no repository".into(),
2875 })?;
2876 self.forget(id)?;
2877 Ok(())
2878 }
2879
2880 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
2881 async fn partition_edges(
2882 &self,
2883 board: &Board,
2884 near_kind: BoardKind,
2885 near_content: ContentKind,
2886 depends_on: &[DependencyEdge],
2887 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
2888 let mut native = Vec::new();
2889 let mut fallback = Vec::new();
2890 for edge in depends_on {
2891 let same_source = edge
2892 .to
2893 .source()
2894 .is_none_or(|source| source == self.name.as_str());
2895 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
2896 // `DependencyEndpoint::source` both read it that way — and a native id may hold
2897 // colons of its own, so the far end is everything after that one separator.
2898 // Splitting at the last would truncate `work:urn:task:7` to `7`.
2899 let far_id = if edge.to.is_qualified() {
2900 edge.to
2901 .id()
2902 .split_once(':')
2903 .map_or(edge.to.id(), |(_, native)| native)
2904 } else {
2905 edge.to.id()
2906 };
2907 let far = if same_source {
2908 Some(
2909 board
2910 .items
2911 .iter()
2912 .find(|item| item.id.0 == far_id)
2913 .ok_or_else(|| SourceError::Refused {
2914 message: format!("GitHub dependency item {far_id} was not found"),
2915 })?,
2916 )
2917 } else {
2918 None
2919 };
2920 // The caller says which kind the far end is, and this board holds the far end
2921 // itself, so a disagreement is settled here rather than stored: recorded, the
2922 // wrong kind would read back as a cross-level edge that never existed; written
2923 // natively, it would name a relationship of a different level than the caller
2924 // asked for.
2925 //
2926 // A far end this board holds as a *document* fails the same comparison and is
2927 // refused by the same sentence: `ItemKind` has no document variant because
2928 // nothing may point at one, so no caller can name it correctly and the refusal
2929 // is the only honest answer.
2930 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
2931 return Err(SourceError::Refused {
2932 message: format!(
2933 "GitHub dependency item {far_id} is a {} of this board, and this item \
2934 names it as a {}; record the kind it is",
2935 disagreeing.kind.describes(),
2936 edge.to.kind.marker()
2937 ),
2938 });
2939 }
2940 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
2941 // however the far end is spelled — and one classified native here would be
2942 // written nowhere at all, because a draft's native reconciliation never runs.
2943 let native_here = near_content == ContentKind::Issue
2944 && far.is_some_and(|far| {
2945 far.content_kind == ContentKind::Issue
2946 && BoardKind::Work(edge.to.kind) == near_kind
2947 });
2948 if native_here {
2949 native.push(far_id.to_owned());
2950 } else {
2951 fallback.push(edge.clone());
2952 }
2953 }
2954 Ok((native, fallback))
2955 }
2956
2957 async fn update_existing(
2958 &self,
2959 item: &Resolved,
2960 incoming: &Incoming<'_>,
2961 body: &Option<String>,
2962 status_target: Option<&StatusTarget>,
2963 ) -> Result<(), SourceError> {
2964 let title = incoming.written_title();
2965 let (operation, input, pointer) = match item.content_kind {
2966 ContentKind::DraftIssue => (
2967 graphql::UPDATE_DRAFT,
2968 json!({"draftIssueId":item.id.0,"title":title,"body":body}),
2969 "/updateProjectV2DraftIssue/draftIssue",
2970 ),
2971 ContentKind::Issue => (
2972 graphql::UPDATE_ISSUE,
2973 json!({"id":item.id.0,"title":title,"body":body,
2974 "stateInput":state_input(status_target)}),
2975 "/updateIssue/issue",
2976 ),
2977 };
2978 let data = self.graphql(operation, json!({"input":input})).await?;
2979 let returned = data
2980 .pointer(pointer)
2981 .ok_or_else(|| SourceError::Malformed {
2982 message: "GitHub item update returned no item".into(),
2983 })?;
2984 if required_str(returned, "id")? != item.id.0 {
2985 return Err(SourceError::Malformed {
2986 message: "GitHub item update returned the wrong item".into(),
2987 });
2988 }
2989 Ok(())
2990 }
2991
2992 /// Creates one issue, files it on the board, and closes it when the status says so.
2993 ///
2994 /// Three calls rather than one: `createIssue` needs a repository and answers with an
2995 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
2996 /// closed status is a state of the issue rather than a field of the board item.
2997 /// Creates the issue, files it on the board, and reports what a read of it would say:
2998 /// its content id, its board item id, and the web address GitHub gave it.
2999 ///
3000 /// The address comes back here because this is the only place it is known before
3001 /// GitHub's own board read catches up — an item this run created answers the reads
3002 /// that follow it out of the record below, and one remembered without its address
3003 /// would report no location for the rest of the run.
3004 async fn create_and_file_issue(
3005 &self,
3006 board: &Board,
3007 incoming: &Incoming<'_>,
3008 body: &Option<String>,
3009 status_target: Option<&StatusTarget>,
3010 ) -> Result<(NativeId, String, Option<String>), SourceError> {
3011 let repository_id = self.repository_id().await?;
3012 let data = self
3013 .graphql(
3014 graphql::CREATE_ISSUE,
3015 json!({"input":{
3016 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
3017 }}),
3018 )
3019 .await?;
3020 let created = data
3021 .pointer("/createIssue/issue")
3022 .filter(|value| !value.is_null())
3023 .ok_or_else(|| SourceError::Malformed {
3024 message: "GitHub issue creation returned no issue".into(),
3025 })?;
3026 let content_id = NativeId(required_str(created, "id")?.to_owned());
3027 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
3028 // a response without it is not worth failing a landed write over — the item simply
3029 // reports no location until the board read catches up, which is what it did before.
3030 let url = optional_str(created, "url")?.map(str::to_owned);
3031 // The issue exists from here on, so a failure filing it on the board takes it
3032 // back: an issue in the repository that is on no board is an item nobody asked for
3033 // and nothing here would find again.
3034 let added = match self
3035 .graphql(
3036 graphql::ADD_TO_BOARD,
3037 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
3038 )
3039 .await
3040 {
3041 Ok(added) => added,
3042 Err(error) => {
3043 let _ = self.delete_issue(&content_id).await;
3044 return Err(error);
3045 }
3046 };
3047 let item = added
3048 .pointer("/addProjectV2ItemById/item")
3049 .filter(|value| !value.is_null())
3050 .ok_or_else(|| SourceError::Malformed {
3051 message: "GitHub board addition returned no project item".into(),
3052 })?;
3053 if let Some(StatusTarget::Closed(_)) = status_target {
3054 let closed = self
3055 .graphql(
3056 graphql::UPDATE_ISSUE,
3057 json!({"input":{"id":content_id.0,"stateInput":state_input(status_target)}}),
3058 )
3059 .await?;
3060 let returned =
3061 closed
3062 .pointer("/updateIssue/issue")
3063 .ok_or_else(|| SourceError::Malformed {
3064 message: "GitHub item update returned no item".into(),
3065 })?;
3066 if required_str(returned, "id")? != content_id.0 {
3067 return Err(SourceError::Malformed {
3068 message: "GitHub item update returned the wrong item".into(),
3069 });
3070 }
3071 }
3072 Ok((content_id, required_str(item, "id")?.to_owned(), url))
3073 }
3074
3075 /// Move one issue under the project it now belongs to, or out of the one it left.
3076 async fn reparent(
3077 &self,
3078 held: Option<NativeId>,
3079 child: &NativeId,
3080 wanted: Option<&NativeId>,
3081 ) -> Result<(), SourceError> {
3082 if held.as_ref() == wanted {
3083 return Ok(());
3084 }
3085 if let Some(held) = &held {
3086 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
3087 .await?;
3088 }
3089 if let Some(wanted) = wanted {
3090 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
3091 .await?;
3092 }
3093 Ok(())
3094 }
3095
3096 async fn sub_issue(
3097 &self,
3098 operation: &str,
3099 parent: &NativeId,
3100 child: &NativeId,
3101 root: &str,
3102 ) -> Result<(), SourceError> {
3103 let data = self
3104 .graphql(
3105 operation,
3106 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
3107 )
3108 .await?;
3109 let issue =
3110 data.pointer(&format!("/{root}/issue"))
3111 .ok_or_else(|| SourceError::Malformed {
3112 message: "GitHub sub-issue update returned no issue".into(),
3113 })?;
3114 let sub =
3115 data.pointer(&format!("/{root}/subIssue"))
3116 .ok_or_else(|| SourceError::Malformed {
3117 message: "GitHub sub-issue update returned no sub-issue".into(),
3118 })?;
3119 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
3120 return Err(SourceError::Malformed {
3121 message: "GitHub sub-issue update returned the wrong issues".into(),
3122 });
3123 }
3124 Ok(())
3125 }
3126
3127 async fn reconcile_blocked_by(
3128 &self,
3129 content_id: &NativeId,
3130 native: &[String],
3131 ) -> Result<(), SourceError> {
3132 let current = self.native_dependency_ids(content_id).await?;
3133 for (operation, far_id) in current
3134 .iter()
3135 .filter(|id| !native.contains(id))
3136 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
3137 .chain(
3138 native
3139 .iter()
3140 .filter(|id| !current.contains(id))
3141 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
3142 )
3143 {
3144 let data = self
3145 .graphql(
3146 operation,
3147 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
3148 )
3149 .await?;
3150 let root = if operation == graphql::ADD_BLOCKED_BY {
3151 "addBlockedBy"
3152 } else {
3153 "removeBlockedBy"
3154 };
3155 let issue =
3156 data.pointer(&format!("/{root}/issue"))
3157 .ok_or_else(|| SourceError::Malformed {
3158 message: "GitHub dependency update returned no issue".into(),
3159 })?;
3160 let blocker = data
3161 .pointer(&format!("/{root}/blockingIssue"))
3162 .ok_or_else(|| SourceError::Malformed {
3163 message: "GitHub dependency update returned no blocking issue".into(),
3164 })?;
3165 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
3166 {
3167 return Err(SourceError::Malformed {
3168 message: "GitHub dependency update returned the wrong issues".into(),
3169 });
3170 }
3171 }
3172 Ok(())
3173 }
3174}
3175
3176/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
3177enum Reached {
3178 /// An issue this board holds, resolved into everything this source reports about it.
3179 Held(Box<Resolved>),
3180 /// Nothing this board holds: no such node, or a node on some other board.
3181 Nothing,
3182 /// A board draft, which exists only inside the board's own item connection.
3183 Draft,
3184}
3185
3186/// What GitHub says when a string is not a node id it can resolve.
3187///
3188/// Matched because it is the ordinary answer to a project selector naming a project by its
3189/// *name*, and reporting that as a failure would make naming one impossible. It is read
3190/// off the refusal GitHub sent, never guessed from the shape of the string: this source
3191/// does not define the syntax of a GitHub node id and would be wrong about it.
3192const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
3193
3194/// Whether this refusal is GitHub saying the id names no node at all.
3195fn unresolvable_node(error: &SourceError) -> bool {
3196 matches!(error, SourceError::Refused { message }
3197 if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
3198}
3199
3200/// One project name, as a search qualifier which filters on it at the server.
3201///
3202/// Quoted so the whole title is one phrase rather than a bag of words, with the two
3203/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
3204/// the way it documents. A title matched here is still compared for equality afterwards:
3205/// the qualifier narrows what the server sends, and this source decides what it names.
3206fn title_qualifier(name: &str) -> String {
3207 let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
3208 format!("in:title \"{escaped}\"")
3209}
3210
3211/// The board, and every item on it this source reports.
3212#[derive(Clone)]
3213struct Board {
3214 id: String,
3215 fields: Value,
3216 items: Vec<Resolved>,
3217}
3218
3219impl Board {
3220 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
3221 complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
3222 let nodes = fields
3223 .get("nodes")
3224 .and_then(Value::as_array)
3225 .ok_or_else(|| SourceError::Malformed {
3226 message: "GitHub project fields.nodes is not an array".into(),
3227 })?;
3228 Ok(nodes
3229 .iter()
3230 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
3231 }
3232}
3233
3234/// One board item, resolved into everything this source reports about it.
3235#[derive(Clone)]
3236struct Resolved {
3237 item_id: String,
3238 id: NativeId,
3239 content_kind: ContentKind,
3240 kind: BoardKind,
3241 title: String,
3242 body: Option<String>,
3243 status: Status,
3244 labels: Vec<Label>,
3245 parent: Option<NativeId>,
3246 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
3247 origin: Option<String>,
3248 url: Option<String>,
3249 created_at: Option<DateTime<Utc>>,
3250 updated_at: Option<DateTime<Utc>>,
3251 own_repository: Option<Repository>,
3252 repositories: Vec<Repository>,
3253 slot: BTreeMap<String, Value>,
3254}
3255
3256impl Resolved {
3257 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
3258 /// in a field of its own, and none of the three keys that are only an encoding.
3259 fn metadata(&self) -> BTreeMap<String, Value> {
3260 let mut metadata = self.slot.clone();
3261 metadata.remove(Repository::METADATA_KEY);
3262 metadata.remove(DependencyEdge::RECORDED_KEY);
3263 metadata.remove(ItemKind::METADATA_KEY);
3264 if let Some(origin) = &self.origin {
3265 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
3266 }
3267 metadata
3268 }
3269
3270 /// Where this item is, as a link a reader can open.
3271 ///
3272 /// A board is a hosted place and every issue on it has a web address, so that address
3273 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
3274 /// of place it is, so a reader knows to open it rather than to read a file out. It
3275 /// does not replace or derive from `url`: the field goes on reporting exactly what it
3276 /// reported before, and this says what that address *is*.
3277 ///
3278 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
3279 /// rather than a third variant, which is the contract's "the source did not say". An
3280 /// issue this run created is not one of those: its address comes back from the
3281 /// creating mutation, so it is somewhere a reader can open from the moment it exists
3282 /// rather than from whenever the board read catches up.
3283 fn location(&self) -> Option<Location> {
3284 self.url.clone().map(Location::Url)
3285 }
3286
3287 fn task(&self) -> Task {
3288 Task {
3289 id: self.id.clone(),
3290 title: self.title.clone(),
3291 content: self.body.clone(),
3292 status: self.status.clone(),
3293 labels: self.labels.clone(),
3294 project: self.parent.clone(),
3295 url: self.url.clone(),
3296 location: self.location(),
3297 created_at: self.created_at,
3298 updated_at: self.updated_at,
3299 metadata: self.metadata(),
3300 repositories: self.repositories.clone(),
3301 }
3302 }
3303
3304 fn project(&self) -> Project {
3305 Project {
3306 id: self.id.clone(),
3307 title: self.title.clone(),
3308 content: self.body.clone(),
3309 status: self.status.clone(),
3310 labels: self.labels.clone(),
3311 url: self.url.clone(),
3312 location: self.location(),
3313 created_at: self.created_at,
3314 updated_at: self.updated_at,
3315 metadata: self.metadata(),
3316 repositories: self.repositories.clone(),
3317 }
3318 }
3319
3320 /// The same issue as a document: the project it is filed under, and no status and no
3321 /// dependencies, because a document is not work.
3322 fn document(&self) -> Document {
3323 Document {
3324 id: self.id.clone(),
3325 title: self.title.clone(),
3326 content: self.body.clone(),
3327 project: self.parent.clone(),
3328 labels: self.labels.clone(),
3329 url: self.url.clone(),
3330 location: self.location(),
3331 created_at: self.created_at,
3332 updated_at: self.updated_at,
3333 metadata: self.metadata(),
3334 repositories: self.repositories.clone(),
3335 }
3336 }
3337}
3338
3339/// What one write is, and the status that comes with being it.
3340///
3341/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
3342/// status and a task or a project always has one, so "a document carrying a status" and
3343/// "a task carrying none" are states a write cannot be in rather than states every use
3344/// site below has to defend against.
3345enum Written<'a> {
3346 /// A document, which is not work and so has no status at all.
3347 Document,
3348 /// A task or a project, and the status it is being written with.
3349 Work(ItemKind, &'a Status),
3350}
3351
3352impl Written<'_> {
3353 /// Which of the board's three kinds this write is.
3354 const fn kind(&self) -> BoardKind {
3355 match self {
3356 Self::Document => BoardKind::Document,
3357 Self::Work(kind, _) => BoardKind::Work(*kind),
3358 }
3359 }
3360
3361 /// The status this write carries. A document carries none, so a write of one says
3362 /// nothing about the issue's open or closed state and selects no board `Status`
3363 /// option.
3364 const fn status(&self) -> Option<&Status> {
3365 match self {
3366 Self::Document => None,
3367 Self::Work(_, status) => Some(status),
3368 }
3369 }
3370}
3371
3372/// The item being written, in the one shape all three write methods reach.
3373struct Incoming<'a> {
3374 written: Written<'a>,
3375 /// The title a person wrote. A document's goes onto the issue with
3376 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
3377 title: &'a str,
3378 content: Option<&'a str>,
3379 labels: &'a [Label],
3380 metadata: &'a BTreeMap<String, Value>,
3381 repositories: &'a [Repository],
3382 parent: Option<&'a NativeId>,
3383}
3384
3385impl Incoming<'_> {
3386 /// The title this write puts on the issue.
3387 fn written_title(&self) -> String {
3388 match self.written {
3389 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
3390 Written::Work(..) => self.title.to_owned(),
3391 }
3392 }
3393}
3394
3395#[derive(Clone, Copy, PartialEq, Eq)]
3396enum ContentKind {
3397 DraftIssue,
3398 Issue,
3399}
3400
3401/// What one board issue is: a document, or the work an [`ItemKind`] names.
3402///
3403/// A type of this source's own rather than an `ItemKind` with a third variant, because
3404/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
3405/// document — the contract keeps a document out of that enum deliberately. Holding the
3406/// board's three answers in one value is what makes every place that asks "which is this?"
3407/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
3408/// two thirds of the board.
3409#[derive(Clone, Copy, PartialEq, Eq)]
3410enum BoardKind {
3411 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
3412 Document,
3413 /// Every other issue, and every draft.
3414 Work(ItemKind),
3415}
3416
3417impl BoardKind {
3418 /// How a refusal names this kind to the person reading it.
3419 const fn describes(self) -> &'static str {
3420 match self {
3421 Self::Document => "document",
3422 Self::Work(kind) => kind.marker(),
3423 }
3424 }
3425}
3426
3427/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
3428///
3429/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
3430/// the shared cross-source journeys assert one answer to one question, so two sources
3431/// that disagree about what "carries the label bug" means fail them.
3432fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
3433 let holds = |name: &String| {
3434 labels
3435 .iter()
3436 .any(|label| label.name.eq_ignore_ascii_case(name))
3437 };
3438 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
3439 && filter.all_of.iter().all(holds)
3440 && !filter.none_of.iter().any(holds)
3441}
3442
3443/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
3444/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
3445fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
3446 statuses.is_empty() || statuses.contains(&category)
3447}
3448
3449/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
3450///
3451/// `content` is the item's own prose — the body with this source's trailing metadata
3452/// comment already taken off — so a search never matches an encoding the author of the
3453/// issue never wrote.
3454fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
3455 let terms = query.terms.to_lowercase();
3456 let in_title = title.to_lowercase().contains(&terms);
3457 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
3458 match query.fields {
3459 TextFields::Title => in_title,
3460 TextFields::Content => in_content,
3461 TextFields::TitleOrContent => in_title || in_content,
3462 }
3463}
3464
3465/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
3466///
3467/// The project predicate is passed separately because a read narrowed to one project has
3468/// already answered it by asking *that project* for its own items — and re-applying it
3469/// there would compare the caller's selector, which may be a project's **name**, against
3470/// the id of the project that name resolved to, and keep nothing. Every other read passes
3471/// `query.project` and applies it here, which is what keeps `projects` a predicate this
3472/// source really does apply.
3473fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
3474 labels_match(&task.labels, &query.labels)
3475 && status_matches(task.status.category, &query.statuses)
3476 && match project {
3477 ProjectFilter::Any => true,
3478 ProjectFilter::Orphans => task.project.is_none(),
3479 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
3480 }
3481 && query
3482 .text
3483 .as_ref()
3484 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
3485}
3486
3487fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
3488 labels_match(&project.labels, &query.labels)
3489 && status_matches(project.status.category, &query.statuses)
3490 && query
3491 .text
3492 .as_ref()
3493 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
3494}
3495
3496/// The same three predicates a task query carries, minus the status filter.
3497///
3498/// A document is not work, so it has no status for one to compare against and the query
3499/// type carries none. The project predicate is the same one — a design issue filed under a
3500/// project issue is in that project, and one filed under nothing is in none — so it is
3501/// spelled the same way here rather than answered differently.
3502fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
3503 labels_match(&document.labels, &query.labels)
3504 && match project {
3505 ProjectFilter::Any => true,
3506 ProjectFilter::Orphans => document.project.is_none(),
3507 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
3508 }
3509 && query
3510 .text
3511 .as_ref()
3512 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
3513}
3514
3515#[async_trait::async_trait]
3516impl TaskSource for GitHubProjectsSource {
3517 fn kind(&self) -> &'static str {
3518 KIND
3519 }
3520 fn capabilities(&self) -> Capabilities {
3521 Capabilities {
3522 projects: Support::Native,
3523 documents: Support::Native,
3524 orphan_tasks: Support::Native,
3525 filter_by_label: Support::Native,
3526 filter_by_status: Support::Native,
3527 search_title: Support::Native,
3528 search_content: Support::Native,
3529 task_dependencies: DependencySupport::BothDirections,
3530 project_dependencies: DependencySupport::BothDirections,
3531 max_page_size: MAX_PAGE_SIZE,
3532 }
3533 }
3534 async fn health(&self) -> Result<Health, SourceError> {
3535 let board = self.board_page(None, 1).await?;
3536 Ok(Health {
3537 reachable: true,
3538 detail: Some(format!(
3539 "reading GitHub project {}/{} ({})",
3540 self.owner,
3541 self.project_number,
3542 required_str(&board, "title")?
3543 )),
3544 })
3545 }
3546 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
3547 Ok(self
3548 .item_by_id(id)
3549 .await?
3550 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3551 .map(|item| item.task()))
3552 }
3553 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
3554 Ok(self
3555 .item_by_id(id)
3556 .await?
3557 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
3558 .map(|item| item.project()))
3559 }
3560 async fn query_tasks(
3561 &self,
3562 query: &TaskQuery,
3563 page: &PageRequest,
3564 ) -> Result<Page<Task>, SourceError> {
3565 validate_page(page)?;
3566 // A read narrowed to one project asks that project for its own tasks, so nothing
3567 // about it costs what the rest of the board holds. Every other task read is a
3568 // question about the whole board and is answered by reading it.
3569 let (held, membership) = match &query.project {
3570 ProjectFilter::Is(project) => (
3571 self.project_children(project).await?,
3572 // Answered by where these items came from; see `task_matches`.
3573 &ProjectFilter::Any,
3574 ),
3575 ProjectFilter::Any | ProjectFilter::Orphans => {
3576 (self.board().await?.items, &query.project)
3577 }
3578 };
3579 // Filtered before paged: a page of a filtered result is a page of the survivors,
3580 // never the survivors of a page.
3581 let tasks = held
3582 .iter()
3583 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3584 .map(Resolved::task)
3585 .filter(|task| task_matches(task, query, membership))
3586 .collect();
3587 Ok(offset_page(
3588 tasks,
3589 numeric_cursor(page.cursor.as_ref())?,
3590 page.limit.min(MAX_PAGE_SIZE) as usize,
3591 ))
3592 }
3593 async fn query_projects(
3594 &self,
3595 query: &ProjectQuery,
3596 page: &PageRequest,
3597 ) -> Result<Page<Project>, SourceError> {
3598 validate_page(page)?;
3599 // The projects a board holds are found by an issue search scoped to that board,
3600 // never by walking the board's own item connection: what tells a project from a
3601 // task is the `parent` each issue carries, which costs nothing to read.
3602 let projects = self
3603 .board_issues()
3604 .await?
3605 .iter()
3606 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
3607 .map(Resolved::project)
3608 .filter(|project| project_matches(project, query))
3609 .collect();
3610 Ok(offset_page(
3611 projects,
3612 numeric_cursor(page.cursor.as_ref())?,
3613 page.limit.min(MAX_PAGE_SIZE) as usize,
3614 ))
3615 }
3616 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
3617 Ok(self
3618 .item_by_id(id)
3619 .await?
3620 .filter(|item| item.kind == BoardKind::Document)
3621 .map(|item| item.document()))
3622 }
3623 async fn query_documents(
3624 &self,
3625 query: &DocumentQuery,
3626 page: &PageRequest,
3627 ) -> Result<Page<Document>, SourceError> {
3628 validate_page(page)?;
3629 // Narrowed to one project, this is the same sub-issue read a task list scoped to
3630 // that project makes — a document filed under a project is a sub-issue of it too,
3631 // and which of them come back is the kind this caller asked for.
3632 let (held, membership) = match &query.project {
3633 ProjectFilter::Is(project) => (
3634 self.project_children(project).await?,
3635 // Answered by where these items came from; see `task_matches`.
3636 &ProjectFilter::Any,
3637 ),
3638 ProjectFilter::Any | ProjectFilter::Orphans => {
3639 (self.board().await?.items, &query.project)
3640 }
3641 };
3642 // Filtered before paged, exactly as a task read is: a page of a filtered result is
3643 // a page of the survivors, never the survivors of a page.
3644 let documents = held
3645 .iter()
3646 .filter(|item| item.kind == BoardKind::Document)
3647 .map(Resolved::document)
3648 .filter(|document| document_matches(document, query, membership))
3649 .collect();
3650 Ok(offset_page(
3651 documents,
3652 numeric_cursor(page.cursor.as_ref())?,
3653 page.limit.min(MAX_PAGE_SIZE) as usize,
3654 ))
3655 }
3656 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
3657 validate_page(page)?;
3658 let offset = numeric_cursor(page.cursor.as_ref())?;
3659 let mut labels = self
3660 .board()
3661 .await?
3662 .items
3663 .into_iter()
3664 .flat_map(|item| item.labels)
3665 .fold(Vec::new(), |mut all, label| {
3666 if !all.iter().any(|x: &Label| x.id == label.id) {
3667 all.push(label);
3668 }
3669 all
3670 });
3671 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
3672 Ok(offset_page(
3673 labels,
3674 offset,
3675 page.limit.min(MAX_PAGE_SIZE) as usize,
3676 ))
3677 }
3678 async fn task_dependencies(
3679 &self,
3680 id: &NativeId,
3681 direction: Direction,
3682 page: &PageRequest,
3683 ) -> Result<Page<DependencyEdge>, SourceError> {
3684 self.dependencies(id, ItemKind::Task, direction, page).await
3685 }
3686 async fn project_dependencies(
3687 &self,
3688 id: &NativeId,
3689 direction: Direction,
3690 page: &PageRequest,
3691 ) -> Result<Page<DependencyEdge>, SourceError> {
3692 self.dependencies(id, ItemKind::Project, direction, page)
3693 .await
3694 }
3695
3696 fn writes(&self) -> WriteSupport {
3697 WriteSupport::Supported
3698 }
3699
3700 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
3701 self.write_item(
3702 &Incoming {
3703 written: Written::Work(ItemKind::Task, &write.item.status),
3704 title: &write.item.title,
3705 content: write.item.content.as_deref(),
3706 labels: &write.item.labels,
3707 metadata: &write.item.metadata,
3708 repositories: &write.item.repositories,
3709 parent: write.item.project.as_ref(),
3710 },
3711 write.target.as_ref(),
3712 &write.depends_on,
3713 )
3714 .await
3715 }
3716
3717 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
3718 self.write_item(
3719 &Incoming {
3720 written: Written::Work(ItemKind::Project, &write.item.status),
3721 title: &write.item.title,
3722 content: write.item.content.as_deref(),
3723 labels: &write.item.labels,
3724 metadata: &write.item.metadata,
3725 repositories: &write.item.repositories,
3726 parent: None,
3727 },
3728 write.target.as_ref(),
3729 &write.depends_on,
3730 )
3731 .await
3732 }
3733
3734 /// Create or update one document, which is one issue titled the way this board spells
3735 /// a document.
3736 ///
3737 /// Everything else is exactly a task write: caller metadata goes to the same canonical
3738 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
3739 /// or a field this board cannot carry is refused by name rather than dropped, a target
3740 /// naming an issue this board does not hold is refused rather than created, and an
3741 /// issue this call created is taken back when the rest of the write fails.
3742 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
3743 // A document takes part in no dependency graph, so there is no far end to write
3744 // natively and none to record: a caller naming one is told so rather than having it
3745 // stored under the reserved key, where a later read would report an edge the
3746 // contract says cannot exist.
3747 if !write.depends_on.is_empty() {
3748 return Err(SourceError::Refused {
3749 message: format!(
3750 "this write names {} dependencies for a document, and a document takes \
3751 part in no dependency graph; next: put the dependency on the task or \
3752 project the document is about",
3753 write.depends_on.len()
3754 ),
3755 });
3756 }
3757 self.write_item(
3758 &Incoming {
3759 written: Written::Document,
3760 title: &write.item.title,
3761 content: write.item.content.as_deref(),
3762 labels: &write.item.labels,
3763 metadata: &write.item.metadata,
3764 repositories: &write.item.repositories,
3765 parent: write.item.project.as_ref(),
3766 },
3767 write.target.as_ref(),
3768 &[],
3769 )
3770 .await
3771 }
3772
3773 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
3774 self.delete_item(id).await
3775 }
3776
3777 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
3778 self.delete_item(id).await
3779 }
3780
3781 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
3782 self.delete_item(id).await
3783 }
3784}
3785
3786/// Where the recorded tail of a dependency walk resumes; see
3787/// [`GitHubProjectsSource::recorded_edges`].
3788const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
3789
3790/// The board text field this source keeps a copy's origin in.
3791///
3792/// Named after the key it holds, and held to that name by the guard below rather than by
3793/// a reader noticing.
3794const ORIGIN_FIELD: &str = "onetaskgraph.origin";
3795
3796/// The metadata key that field holds.
3797///
3798/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
3799/// constructs or interprets the qualified id it carries. This source names it only to
3800/// route it — a short, typed value belongs in a typed field rather than in the body slot
3801/// a caller's own prose shares.
3802///
3803/// Restated rather than imported, because no plugin crate may depend on the engine. What
3804/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
3805/// target in `check`: it reads the engine's own literal and fails naming the file and the
3806/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
3807/// that creates a second item every run instead of finding the one it wrote — and that is
3808/// too late to learn it.
3809const ORIGIN_KEY: &str = "onetaskgraph.origin";
3810
3811/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
3812///
3813/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
3814/// is derived from the far end, never written down on the near item — so only a forward
3815/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
3816/// it did not come from, and it is told so rather than answered with an empty page that
3817/// reads as a walk which ended.
3818fn recorded_offset(
3819 cursor: Option<&str>,
3820 direction: Direction,
3821) -> Result<Option<usize>, SourceError> {
3822 cursor
3823 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
3824 .map(|offset| {
3825 if direction != Direction::DependsOn {
3826 return Err(SourceError::Config {
3827 message: format!(
3828 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
3829 reverse dependency read never issues; resume it in the direction \
3830 that reported it"
3831 ),
3832 });
3833 }
3834 offset.parse().map_err(|_| SourceError::Config {
3835 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
3836 })
3837 })
3838 .transpose()
3839}
3840
3841fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
3842 let mut page = offset_page(edges, offset, limit.max(1));
3843 page.next = page
3844 .next
3845 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
3846 page
3847}
3848
3849/// The kind of one issue reached through a dependency connection.
3850///
3851/// The same questions the board scan asks, over the fields the dependency document
3852/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
3853/// then anything with sub-issues or the marker is a project.
3854///
3855/// # Errors
3856///
3857/// A far end this board holds as a document is refused rather than reported. The two
3858/// answers that are not refusals would both be wrong: reporting it as a task names an id
3859/// no task read of this source can find, and reporting it as a project names one no
3860/// project read can. There is no third value to return — `ItemKind` has no document
3861/// variant, because nothing may point at a document — so the relationship itself is what
3862/// the person is told about.
3863fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
3864 let id = required_str(value, "id")?;
3865 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
3866 return Err(SourceError::Refused {
3867 message: format!(
3868 "GitHub issue {id} is a document of this board — its title begins \
3869 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
3870 on by one; next: remove that issue's blocking relationship on this board"
3871 ),
3872 });
3873 }
3874 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
3875 if parent.is_some() {
3876 return Ok(ItemKind::Task);
3877 }
3878 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
3879 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
3880 message: format!("GitHub issue {id}: {message}"),
3881 })?;
3882 let sub_issues = sub_issue_total(value)?;
3883 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
3884 ItemKind::Project
3885 } else {
3886 ItemKind::Task
3887 })
3888}
3889
3890/// The `IssueStateUpdateInput` one status target asks for.
3891///
3892/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
3893/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
3894/// a currently-closed issue: without that the item would read back `Unknown` and a copy
3895/// would report a change forever. A document has no status at all, and asks for neither.
3896fn state_input(target: Option<&StatusTarget>) -> Value {
3897 match target {
3898 Some(StatusTarget::Closed(reason)) => {
3899 json!({"value":"CLOSED","stateReason":reason.reason()})
3900 }
3901 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
3902 // A document has no status, so a write of one says nothing about the issue's open
3903 // or closed state rather than forcing it open: `stateInput` is what carries that
3904 // instruction, and an explicit null asks for no change to it.
3905 None => Value::Null,
3906 }
3907}
3908
3909/// The metadata one write stores in the item's body slot.
3910///
3911/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
3912/// rather than carried: the kind marker so an empty project stays readable, the
3913/// repository list only when it is not exactly the issue's own repository, and the far
3914/// ends no relationship here can name.
3915fn slot_metadata(
3916 incoming: &Incoming<'_>,
3917 own_repository: Option<&Repository>,
3918 fallback: &[DependencyEdge],
3919) -> BTreeMap<String, Value> {
3920 let mut metadata = incoming.metadata.clone();
3921 metadata.remove(ORIGIN_KEY);
3922 match incoming.written.kind() {
3923 BoardKind::Work(kind) => metadata.insert(
3924 ItemKind::METADATA_KEY.to_owned(),
3925 Value::String(kind.marker().to_owned()),
3926 ),
3927 // A document is told by its title, so it carries no kind marker: that key names
3928 // what a dependency endpoint points at, and nothing may point at a document.
3929 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
3930 };
3931 let derivable = own_repository
3932 .map(|own| incoming.repositories == [own.clone()])
3933 .unwrap_or(incoming.repositories.is_empty());
3934 if derivable {
3935 metadata.remove(Repository::METADATA_KEY);
3936 } else {
3937 metadata.insert(
3938 Repository::METADATA_KEY.to_owned(),
3939 Value::Array(
3940 incoming
3941 .repositories
3942 .iter()
3943 .map(|repository| Value::String(repository.as_str().to_owned()))
3944 .collect(),
3945 ),
3946 );
3947 }
3948 if fallback.is_empty() {
3949 metadata.remove(DependencyEdge::RECORDED_KEY);
3950 } else {
3951 metadata.insert(
3952 DependencyEdge::RECORDED_KEY.to_owned(),
3953 Value::Array(
3954 fallback
3955 .iter()
3956 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
3957 .collect(),
3958 ),
3959 );
3960 }
3961 metadata
3962}
3963
3964fn labels(content: &Value, field_values: &[Value]) -> Result<Vec<Label>, SourceError> {
3965 let direct = optional_nodes(content.get("labels"), "content labels")?;
3966 let field = field_values
3967 .iter()
3968 .find_map(|value| value.get("labels"))
3969 .map(|labels| optional_nodes(Some(labels), "field labels"))
3970 .transpose()?
3971 .flatten();
3972 let labels = direct
3973 .into_iter()
3974 .flatten()
3975 .chain(field.into_iter().flatten())
3976 .map(|v| {
3977 Ok(Label {
3978 id: NativeId(required_str(v, "id")?.to_owned()),
3979 name: required_str(v, "name")?.to_owned(),
3980 color: optional_str(v, "color")?.map(str::to_owned),
3981 })
3982 })
3983 .collect::<Result<Vec<_>, SourceError>>()?
3984 .into_iter()
3985 .fold(Vec::new(), |mut labels, label| {
3986 if !labels.iter().any(|x: &Label| x.id == label.id) {
3987 labels.push(label);
3988 }
3989 labels
3990 });
3991 Ok(labels)
3992}
3993
3994fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
3995 let Some(node) = field_values
3996 .iter()
3997 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
3998 else {
3999 return Ok(None);
4000 };
4001 Ok(optional_str(node, "text")?.map(str::to_owned))
4002}
4003
4004fn valid_github_owner(owner: &str) -> bool {
4005 !owner.is_empty()
4006 && owner.len() <= 39
4007 && !owner.starts_with('-')
4008 && !owner.ends_with('-')
4009 && !owner.contains("--")
4010 && owner
4011 .bytes()
4012 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
4013}
4014
4015/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
4016/// neither of the two names a path segment already means.
4017fn valid_github_repository_name(name: &str) -> bool {
4018 !name.is_empty()
4019 && name.len() <= 100
4020 && name != "."
4021 && name != ".."
4022 && name
4023 .bytes()
4024 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
4025}
4026
4027fn valid_environment_name(name: &str) -> bool {
4028 let mut bytes = name.bytes();
4029 bytes
4030 .next()
4031 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
4032 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
4033}
4034
4035/// How many sub-issues one issue has.
4036///
4037/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
4038/// absent or non-integer one is a response this source cannot read — and reading it as
4039/// zero would classify a project as a task, which is exactly the mistake the marker
4040/// exists to keep from happening quietly.
4041fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
4042 let summary = issue
4043 .get("subIssuesSummary")
4044 .ok_or_else(|| SourceError::Malformed {
4045 message: "GitHub issue is missing subIssuesSummary".into(),
4046 })?;
4047 summary
4048 .get("total")
4049 .and_then(Value::as_u64)
4050 .ok_or_else(|| SourceError::Malformed {
4051 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
4052 })
4053}
4054
4055fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
4056 value
4057 .get(field)
4058 .and_then(Value::as_str)
4059 .ok_or_else(|| SourceError::Malformed {
4060 message: format!("GitHub response is missing string field {field}"),
4061 })
4062}
4063
4064/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
4065/// needs one — Linear spells them too, in its own description field.
4066///
4067/// Restated rather than shared, because a plugin crate depends on the contract crate and
4068/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
4069/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
4070/// source round-trips its own writes perfectly well under its own spelling.
4071const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
4072const METADATA_CLOSE: &str = "\n-->";
4073
4074/// The visible body and the metadata slot at the end of it.
4075///
4076/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
4077/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
4078/// own content and is left alone.
4079fn metadata_body(
4080 body: Option<String>,
4081) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
4082 let Some(body) = body else {
4083 return Ok((None, BTreeMap::new()));
4084 };
4085 let Some(start) = body.rfind(METADATA_OPEN) else {
4086 return Ok((Some(body), BTreeMap::new()));
4087 };
4088 let encoded_start = start + METADATA_OPEN.len();
4089 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
4090 return Err(SourceError::Malformed {
4091 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
4092 });
4093 };
4094 let encoded_end = encoded_start + relative_end;
4095 if !body[encoded_end + METADATA_CLOSE.len()..].trim().is_empty() {
4096 return Ok((Some(body), BTreeMap::new()));
4097 }
4098 let metadata = serde_json::from_str(&body[encoded_start..encoded_end]).map_err(|error| {
4099 SourceError::Malformed {
4100 message: format!(
4101 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
4102 ),
4103 }
4104 })?;
4105 let visible = body[..start].trim_end();
4106 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
4107}
4108
4109fn compose_body(
4110 content: Option<&str>,
4111 metadata: &BTreeMap<String, Value>,
4112) -> Result<Option<String>, SourceError> {
4113 let visible = content.unwrap_or_default();
4114 if metadata.is_empty() {
4115 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
4116 }
4117 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
4118 message: error.to_string(),
4119 })?;
4120 Ok(Some(if visible.is_empty() {
4121 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4122 } else {
4123 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4124 }))
4125}
4126
4127fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
4128 value
4129 .get(field)
4130 .and_then(Value::as_bool)
4131 .ok_or_else(|| SourceError::Malformed {
4132 message: format!("GitHub response is missing boolean field {field}"),
4133 })
4134}
4135fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
4136 match value.get(field) {
4137 None | Some(Value::Null) => Ok(None),
4138 Some(value) => value
4139 .as_str()
4140 .map(Some)
4141 .ok_or_else(|| SourceError::Malformed {
4142 message: format!("GitHub response field {field} is not a string or null"),
4143 }),
4144 }
4145}
4146fn optional_nodes<'a>(
4147 connection: Option<&'a Value>,
4148 name: &str,
4149) -> Result<Option<&'a Vec<Value>>, SourceError> {
4150 match connection {
4151 None | Some(Value::Null) => Ok(None),
4152 Some(value) => value
4153 .get("nodes")
4154 .and_then(Value::as_array)
4155 .map(Some)
4156 .ok_or_else(|| SourceError::Malformed {
4157 message: format!("GitHub {name}.nodes is not an array"),
4158 }),
4159 }
4160}
4161fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
4162 let page_info = connection
4163 .get("pageInfo")
4164 .ok_or_else(|| SourceError::Malformed {
4165 message: format!("GitHub {name} has no pageInfo"),
4166 })?;
4167 if required_bool(page_info, "hasNextPage")? {
4168 return Err(SourceError::Malformed {
4169 message: format!(
4170 "GitHub {name} exceeds the supported nested connection size of {size}"
4171 ),
4172 });
4173 }
4174 Ok(())
4175}
4176fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
4177 optional_str(value, field)?
4178 .map(|timestamp| {
4179 timestamp.parse().map_err(|error| SourceError::Malformed {
4180 message: format!("GitHub response field {field} is not a timestamp: {error}"),
4181 })
4182 })
4183 .transpose()
4184}
4185fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
4186 if page.limit == 0 {
4187 Err(SourceError::Config {
4188 message: "page limit must be at least 1".into(),
4189 })
4190 } else {
4191 Ok(())
4192 }
4193}
4194fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
4195 let page = connection
4196 .get("pageInfo")
4197 .filter(|value| value.is_object())
4198 .ok_or_else(|| SourceError::Malformed {
4199 message: "GitHub connection is missing pageInfo".into(),
4200 })?;
4201 if required_bool(page, "hasNextPage")? {
4202 let cursor = required_str(page, "endCursor")?;
4203 validate_cursor_progress(None, cursor)?;
4204 Ok(Some(Cursor(cursor.into())))
4205 } else {
4206 Ok(None)
4207 }
4208}
4209fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
4210 if next.is_empty() || previous == Some(next) {
4211 Err(SourceError::Malformed {
4212 message: "GitHub pagination cursor is empty or did not advance".into(),
4213 })
4214 } else {
4215 Ok(())
4216 }
4217}
4218fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
4219 cursor.map_or(Ok(0), |c| {
4220 c.0.parse().map_err(|_| SourceError::Config {
4221 message: "page cursor is invalid".into(),
4222 })
4223 })
4224}
4225fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
4226 if offset > items.len() {
4227 return Page::last(vec![]);
4228 }
4229 let tail = items.split_off(offset);
4230 let mut selected = tail;
4231 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
4232 selected.truncate(limit);
4233 Page {
4234 items: selected,
4235 next,
4236 }
4237}