onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so [`GitHubProjectsConfig::repository`] names the one repository this
11//! source creates its project and task issues in; a write without it is refused naming
12//! the field.
13//!
14//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
15//! board has no document type and nothing but issues to hold one in, so the title is the
16//! discriminator and it is the whole of it. The title this source *reports* is the one a
17//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
18//! the body so `content` is what the person wrote — and writing a document puts the prefix
19//! back, so a round trip returns the title that went in.
20//!
21//! **Telling a document from a project from a task.** The design prefix is read **first**:
22//! a document is never a project and never a task, whatever sub-issues it has or does not
23//! have. Only then does the rest apply — a board issue is a project when *either* it has
24//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
25//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
26//! necessary: it is what makes an *empty* project — the state a project copy passes
27//! through between creating the project and filing its first task — readable as a
28//! project, while the sub-issue arm lets a person author a project on the board by hand
29//! with no knowledge of this product's metadata at all. Reading the prefix later than the
30//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
31//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
32//! task nor a document and are ignored.
33//!
34//! **Where an entity is, is a link.** Every project, task and document this source reports
35//! carries a [`Location::Url`] naming the issue's own web address — the same address the
36//! `url` field already reports, in the shape that says a reader can open it. That is the
37//! contrast the location contract exists for: a reader holding an entity from this source
38//! is handed something to link to and one holding an entity from a folder of Markdown is
39//! handed a path, and neither has to know which plugin answered. It does not replace or
40//! derive from `url`; that field goes on reporting what it always reported.
41//!
42//! **Where metadata lives.** Short typed things go to typed fields and native relations:
43//! status to the board's `Status` single-select and the issue's own state, the copy
44//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
45//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
46//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
47//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
48//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
49//! why neither can hold a caller's own prose.
50//!
51//! **Status.** `status_mapping` is per-instance configuration from a status category to
52//! `null`, a board `Status` option name, or a closed state of `completed` or
53//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
54//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
55//! and a mistake destroys every item's status. A status this board cannot represent is a
56//! refusal naming the status and the instance instead.
57//!
58//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
59//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
60//! finished tasks were only moved to a "Done" column would read 0% complete forever.
61//!
62//! # What this source declares, field by field
63//!
64//! One verdict per field of [`Capabilities`], and what `Native` means when this source
65//! says it. *Proven* means a shared journey drives it against the real
66//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
67//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
68//! [`capabilities`](TaskSource::capabilities) from parting.
69//!
70//! | Field | Verdict |
71//! | --- | --- |
72//! | `projects` | **Supported and proven,** and the one predicate here that is pushed down rather than applied in process: a task's project is the issue it is a sub-issue of, so a listing scoped to one *asks that issue* for its own sub-issues. This is the field that was declared and then not applied, which silently returned another project's tasks. |
73//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
74//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
75//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
76//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
77//! | `search_title` | **Supported and proven,** over `Issue.title`. |
78//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
79//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
80//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
81//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
82//!
83//! Nothing here is unsupported. `documents` is not a predicate — it says this source has
84//! documents, which it does — and the three facts behind the uniform `Native` on the
85//! predicates beside it are recorded below rather than re-derived, because a reader who
86//! takes `Native` to mean *the remote service filters* will read that uniformity as a
87//! lie.
88//!
89//! First, the plugin contract defines `Support::Native` as *the source applies this
90//! predicate itself*, and says nothing about where it applies it. What the declaration
91//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
92//! — so that the engine may push it down and apply nothing of its own.
93//!
94//! Second, this source can keep that promise for every predicate at no additional API
95//! cost, because whichever of the three reads below answers a query has already read every
96//! item that query could keep before it filters anything. Filtering those items is
97//! in-process work over data already in hand.
98//!
99//! Third, no predicate but `projects` could be pushed into the API even if that were
100//! wanted, and `projects` is pushed down: `ProjectV2.items` takes `first` and `after` and
101//! offers no filter argument of any kind, GitHub's issue search offers no qualifier for a
102//! label set, a status column or a substring of a body, and its title qualifier matches
103//! tokens where this source — and the local Markdown source beside it — match substrings,
104//! so pushing a search down would silently *narrow* the answer. What a project filter has
105//! instead is a relationship: a project's tasks are that issue's sub-issues, and asking
106//! the issue for them is both cheaper and exact. So there is one predicate this source
107//! applies by asking a narrower question, six it applies in process, and none it is unable
108//! to apply. Declaring one `Unsupported` would make the engine compensate for work this
109//! source has already done, and declaring `projects` native while ignoring the filter
110//! (which this source once did) silently returns another project's tasks, because the
111//! engine trusts the declaration and applies nothing locally.
112//!
113//! # The three ways this source reaches an item, and what each costs
114//!
115//! A board read is charged for what its *nested* connections could return rather than for
116//! what was asked, so one whole-board read costs the same whether the question was about
117//! one project or about all of them. That is why a question about one project is never
118//! answered by reading the board:
119//!
120//! | The question | What is sent | What it costs |
121//! | --- | --- | --- |
122//! | one item, by its own id | [`graphql::ISSUE`] — `node(id:)` | the item |
123//! | one project's tasks or documents | [`graphql::SUB_ISSUES`] — that issue's own `subIssues` | that project |
124//! | which projects this board holds | [`graphql::SEARCH_ISSUES`] — an issue search scoped to the board | the board's issues, without their board items |
125//! | every task, every document, every label | [`graphql::BOARD`] — the board's own `items` | the board |
126//!
127//! The board half of an issue — its board item's id, its `Status` option, this source's
128//! origin text field, its board label field — rides along on `Issue.projectItems` in the
129//! first three, so an item reached any of those ways resolves through the same
130//! [`GitHubProjectsSource::resolve`] the board walk uses and reports the same title, the
131//! same status, the same labels and the same qualified id. An issue with no entry for
132//! *this* board is not this source's to report, which is what keeps an id naming another
133//! repository's issue from being answered as an item of this board.
134//!
135//! The last row is still the board's own item connection, and deliberately: a **draft**
136//! board item is not an issue, so no search and no node read can reach one, and the reads
137//! that have to answer for the whole board are the ones whose cost is the board's size
138//! anyway.
139//!
140//! **Where a read-after-write guarantee comes from, since a search index cannot supply
141//! one.** GitHub's issue search is eventually consistent and answers a write made moments
142//! ago with the value from before it. Resolving a node id is not, so a read by id and a
143//! project's own sub-issues are already current. What closes the gap for the search is
144//! [`GitHubProjectsSource::created`]: every read this source answers is completed with
145//! what this process itself wrote, so an item created seconds ago is reported whether or
146//! not GitHub's index has caught up. Nothing else is remembered, nothing is written down,
147//! and the record dies with the process.
148//!
149//! Filtering happens before paging, so a page of a filtered result is a page of the
150//! survivors rather than the survivors of a page. Label and text matching answer the same
151//! question the same way the local Markdown source's do, so one cross-source expectation
152//! holds for both.
153//!
154//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
155//! has one source, `capabilities`, and the note above is the reasoning behind it rather
156//! than a second copy of it: without the three facts recorded here a reader takes the
157//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
158//! crate's own capabilities test, which pins every field of it against a fully spelled-out
159//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
160//! fails to compile there rather than going unasserted. -->
161//! Required checks use only the local fixture server; the ignored credentialed lane
162//! verifies the current schema, then drives every field of the table above against the
163//! real board. It builds its own fixture there — two projects, one task filed under each,
164//! one filed under neither, a label on one of the three and a closed status on another —
165//! because that shape is what tells an honoured predicate from an ignored one: a board
166//! holding a single project answers a project filter the same way whether or not this
167//! source applies it, which is exactly how the defect above went unseen.
168//!
169//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
170//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
171//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
172//! nominated is what keeps a credentialed write lane off a board and a repository nobody
173//! nominated; it never asks GitHub which project was updated most recently. Before it
174//! starts, the lane also clears any item titled — and any repository label named — the way
175//! it titles and names its own artifacts, which is self-healing after an interrupted run:
176//! a process killed between its writes and its cleanup leaves artifacts the next run
177//! removes.
178//!
179//! **GitHub has two rate limiters and this source is refused by both, so nothing here
180//! treats them as one thing.** The primary budget is the hourly allowance `gh api
181//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
182//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
183//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
184//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
185//! one part of.
186#![deny(missing_docs)]
187
188use std::collections::BTreeMap;
189use std::sync::Mutex;
190use std::time::{Duration, Instant};
191
192use chrono::{DateTime, Utc};
193use onetaskgraph_plugin_api::{
194 Capabilities, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind, DependencySupport,
195 Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label, LabelFilter, Location,
196 NativeId, Page, PageRequest, Project, ProjectFilter, ProjectQuery, Repository, SecretResolver,
197 SourceError, SourceName, SourcePlugin, Status, StatusCategory, Support, Task, TaskQuery,
198 TaskSource, TextFields, TextQuery, WriteSupport,
199};
200use reqwest::{Client, StatusCode, Url};
201use schemars::{Schema, schema_for};
202use secrecy::{ExposeSecret, SecretString};
203use serde::Deserialize;
204use serde_json::{Value, json};
205
206/// The registry name for this plugin.
207pub const KIND: &str = "github-projects";
208/// GitHub's maximum connection page size.
209pub const MAX_PAGE_SIZE: u32 = 100;
210/// Nested connection size which keeps GitHub's worst-case query below its node limit.
211const NESTED_PAGE_SIZE: u32 = 50;
212/// How many of one issue's board memberships are read when an issue is reached directly.
213///
214/// An issue reached through a search or through its own node id carries its board half in
215/// `Issue.projectItems`, and only the entry for *this* board is read. Ten is deliberately
216/// far smaller than [`NESTED_PAGE_SIZE`]: this connection sits under a page of issues, so
217/// its size multiplies through the whole document, and an issue on ten boards at once is
218/// already well past what a person keeps track of. An issue whose entry for this board sits
219/// past it is refused naming the connection rather than reported as not on the board.
220const BOARD_ITEMS_PAGE_SIZE: u32 = 10;
221
222/// The issue-title prefix that makes a board issue a document.
223///
224/// A GitHub Projects board has no document type — it holds issues — so the discriminator
225/// is the title, and this is the whole of it: an issue whose title begins with these bytes
226/// is a document and every other issue is the task or project the sub-issue rule makes it.
227///
228/// It is spelled **once**, here, and read rather than restated everywhere else — including
229/// by the shared journeys, which take it from this constant so a board fixture cannot
230/// drift from what this source reads. `docs/metadata.md` records the two consequences that
231/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
232/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
233/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
234pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
235
236/// Exact GraphQL query documents issued by this plugin.
237///
238/// Keeping the production documents here lets the pinned-schema test validate the same
239/// bytes that are sent to GitHub, rather than a test-only copy which could drift
240/// independently. No document in this module writes the board itself, and none of them
241/// names `updateProjectV2Field`.
242pub mod graphql {
243 /// Everything this source reads about one issue, wherever it reaches that issue.
244 ///
245 /// A macro rather than a constant so the three documents below can `concat!` it: one
246 /// spelling of these fields is what makes an issue read through the board-scoped
247 /// search, through its own node id, and through its project's sub-issue relationship
248 /// resolve to *the same* item, which is the whole of what
249 /// [`GitHubProjectsSource::resolve_issue`](super::GitHubProjectsSource) relies on.
250 ///
251 /// `projectItems` is what carries the board half of an issue: the board item's own id
252 /// and the field values — the `Status` option, this source's origin text field, and any
253 /// board label field — that a `ProjectV2.items` read used to carry. It is asked for on
254 /// the issue rather than on the board, which is what makes the cost of a read
255 /// proportional to what was asked for instead of to the board's size.
256 macro_rules! board_issue {
257 () => {
258 r#" fragment BoardIssue on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total}
259 labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}
260 projectItems(first:$boardItems){nodes{id project{number}
261 fieldValues(first:$nestedFirst){nodes{
262 ... on ProjectV2ItemFieldSingleSelectValue{name field{
263 ... on ProjectV2SingleSelectField{id name options{id name}}
264 }}
265 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
266 ... on ProjectV2ItemFieldLabelValue{labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
267 }pageInfo{hasNextPage}}}pageInfo{hasNextPage}}}"#
268 };
269 }
270
271 /// Every issue of one board, found by a search scoped to that board.
272 ///
273 /// This is how the projects a board holds are listed, and it selects no `items`
274 /// connection on `ProjectV2`: the board is a *qualifier of the search* rather than a
275 /// container walked page by page, so nothing nested inside a board item is paid for.
276 /// Which of the issues it returns is a project is then read off `parent` — GitHub
277 /// accepts `-has:parent` as a search qualifier and silently ignores it, so the
278 /// discriminator has to be applied to the field, which is a scalar on the issue and
279 /// costs nothing.
280 pub const SEARCH_ISSUES: &str = concat!(
281 r#"query($search:String!,$type:SearchType!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
282 search(query:$search,type:$type,first:$first,after:$after){
283 pageInfo{hasNextPage endCursor}
284 nodes{__typename ...BoardIssue}
285 }
286 }"#,
287 board_issue!()
288 );
289
290 /// One issue by its own node id, which is what a qualified id names here.
291 ///
292 /// Strongly consistent, unlike the search above: GitHub's issue search is an index and
293 /// answers a write made moments ago with the value from before it, and resolving a node
294 /// id does not.
295 pub const ISSUE: &str = concat!(
296 r#"query($id:ID!,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
297 node(id:$id){__typename ...BoardIssue}
298 }"#,
299 board_issue!()
300 );
301
302 /// One project's tasks: the sub-issues of the issue that project is.
303 ///
304 /// The work this costs is the project's own size. Nothing about it grows as the board
305 /// gains projects, or as those projects gain tasks.
306 pub const SUB_ISSUES: &str = concat!(
307 r#"query($id:ID!,$first:Int!,$after:String,$nestedFirst:Int!,$boardItems:Int!,$duplicates:Boolean!){
308 node(id:$id){__typename
309 ... on Issue{subIssues(first:$first,after:$after){
310 pageInfo{hasNextPage endCursor}
311 nodes{__typename ...BoardIssue}
312 }}}
313 }"#,
314 board_issue!()
315 );
316
317 /// Reads the board's fields and one page of its items.
318 pub const BOARD: &str = r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
319 owner:repositoryOwner(login:$owner){
320 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
321 }
322 } fragment Board on ProjectV2 { id title
323 fields(first:$nestedFirst){nodes{
324 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
325 ... on ProjectV2Field{__typename id name}
326 }pageInfo{hasNextPage}}
327 items(first:$first,after:$after){nodes{id fieldValues(first:$nestedFirst){nodes{
328 ... on ProjectV2ItemFieldSingleSelectValue{name field{
329 ... on ProjectV2SingleSelectField{id name options{id name}}
330 }}
331 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
332 ... on ProjectV2ItemFieldLabelValue{labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
333 }pageInfo{hasNextPage}} content{
334 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
335 ... on PullRequest{__typename id}
336 ... on DraftIssue{__typename id title body createdAt updatedAt}
337 }} pageInfo{hasNextPage endCursor}}
338 }"#;
339 /// Resolves the configured repository's node id, which creating an issue requires.
340 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
341 /// Reads both dependency directions for one issue, with each far end's own kind.
342 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
343 ... on Issue{
344 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
345 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
346 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
347 /// Creates one issue in the configured repository.
348 pub const CREATE_ISSUE: &str =
349 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
350 /// Puts an existing issue on the configured board.
351 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
352 /// Updates an issue's visible fields and its open or closed state in one call.
353 pub const UPDATE_ISSUE: &str =
354 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
355 /// Updates an existing draft's user-visible fields.
356 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
357 /// Updates a text or single-select value on one project item.
358 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
359 /// Files one issue under another as a sub-issue, which is what project membership is.
360 pub const ADD_SUB_ISSUE: &str =
361 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
362 /// Takes one issue back out of its parent.
363 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
364 /// Adds GitHub's native issue blocked-by relationship.
365 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
366 /// Removes one native issue blocked-by relationship.
367 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
368 /// Deletes one issue, which takes its board item with it.
369 ///
370 /// The engine sends this in one situation only: undoing a copy that could not finish,
371 /// over the items that same copy created. Deleting the issue removes the board item
372 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
373 pub const DELETE_ISSUE: &str =
374 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
375
376 /// Every document above, with what this source is doing when it sends one.
377 ///
378 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
379 /// name the call that was refused, and a `match` with a catch-all arm would answer a
380 /// document added later with "talking to GitHub" and never say so.
381 ///
382 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
383 /// const` here that this list omits, so the two cannot part — which is the same guard
384 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
385 pub const DOCUMENTS: [(&str, &str); 16] = [
386 (SEARCH_ISSUES, "searching this board's issues"),
387 (ISSUE, "reading one issue"),
388 (SUB_ISSUES, "reading a project's tasks"),
389 (BOARD, "reading the board"),
390 (REPOSITORY, "reading the destination repository"),
391 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
392 (CREATE_ISSUE, "creating an issue"),
393 (ADD_TO_BOARD, "adding an issue to the board"),
394 (UPDATE_ISSUE, "updating an issue"),
395 (UPDATE_DRAFT, "updating a draft item"),
396 (UPDATE_FIELD, "writing a board field"),
397 (ADD_SUB_ISSUE, "filing an issue under its project"),
398 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
399 (ADD_BLOCKED_BY, "recording a dependency"),
400 (REMOVE_BLOCKED_BY, "removing a dependency"),
401 (DELETE_ISSUE, "deleting an issue"),
402 ];
403}
404
405/// Which of GitHub's two rate limiters refused a request.
406///
407/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
408/// secondary one — so an operator told the wrong one takes the wrong next step, which is
409/// the whole reason this is carried rather than collapsed into "rate limited".
410#[derive(Debug, Clone, Copy, PartialEq, Eq)]
411enum Limiter {
412 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
413 Primary,
414 /// The burst limiter over content-generating requests, which nothing reports.
415 Secondary,
416}
417
418/// The wordings GitHub answers a secondary rate limit with.
419///
420/// It sends them under a forbidden status, under a too-many-requests status, and inside
421/// the `errors` of a *successful* response, which is why the text is what this matches on
422/// rather than the status. `abuse detection` is the wording GitHub used before the
423/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
424/// what a burst of content creation is refused with.
425///
426/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
427/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
428/// when, and the drift gate reconciles the two lists both ways. Public for that gate
429/// alone — a caller has no use for it, and matching on a refusal is this source's job.
430pub const SECONDARY_WORDINGS: [&str; 5] = [
431 "secondary rate limit",
432 "temporarily blocked from content creation",
433 "abuse detection",
434 "submitted too quickly",
435 "exceeded a secondary",
436];
437
438/// The wordings GitHub answers an exhausted primary budget with.
439///
440/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
441/// than looked for in the response text. Pinned and gated exactly as
442/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
443pub const PRIMARY_WORDINGS: [&str; 3] = [
444 "api rate limit exceeded",
445 "rate limit exceeded",
446 "rate_limited",
447];
448
449/// What a response *says about itself*, which is the only place a refusal can be read.
450///
451/// Deliberately not the whole response body. A board is a place people write about their
452/// own work, and a task on it titled "the secondary rate limit" would, matched across the
453/// raw text, turn a perfectly good answer into a refusal this source then waited out and
454/// reported. So the item data is never read: what is read is GitHub's own REST-style
455/// `message` envelope, which is what a forbidden status carries, and the `message` and
456/// `type` of each GraphQL error, which is where a *successful* response says it.
457///
458/// A body that is not JSON at all has nothing structured to read, so only a failing
459/// response's own text is taken — a successful response that is not JSON is malformed
460/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
461fn refusal_wording(status: StatusCode, body: &str) -> String {
462 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
463 return if status.is_success() {
464 String::new()
465 } else {
466 body.to_owned()
467 };
468 };
469 let mut said: Vec<&str> = parsed
470 .get("message")
471 .and_then(Value::as_str)
472 .into_iter()
473 .collect();
474 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
475 for error in errors {
476 said.extend(
477 ["message", "type"]
478 .into_iter()
479 .filter_map(|key| error.get(key).and_then(Value::as_str)),
480 );
481 }
482 }
483 said.join("; ")
484}
485
486impl Limiter {
487 /// Which limiter refused this response, or `None` when none of them did.
488 ///
489 /// The wording is read first and the status only decides what carries none of it,
490 /// because GitHub answers a secondary limit with a forbidden status far more often
491 /// than with too-many-requests — while a forbidden status saying nothing about a limit
492 /// really is a credential this token lacks.
493 ///
494 /// A response is a refusal because of its status or its own wording. A spent budget
495 /// only ever explains one; it never turns an answer into a refusal.
496 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
497 let normalized = refusal_wording(status, body).to_ascii_lowercase();
498 if SECONDARY_WORDINGS
499 .iter()
500 .any(|wording| normalized.contains(wording))
501 {
502 return Some(Self::Secondary);
503 }
504 if status == StatusCode::TOO_MANY_REQUESTS {
505 return Some(Self::Primary);
506 }
507 // An exhausted budget *explains* a response that failed; it does not make one that
508 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
509 // request the budget allowed as well as on the ones it then refuses, so reading
510 // the header alone threw away a good answer — and, once refusals were retried,
511 // replayed a request that had already taken effect.
512 if !status.is_success() && budget_exhausted {
513 return Some(Self::Primary);
514 }
515 // A successful response saying it: GitHub reports a GraphQL rate limit in the
516 // `errors` of an HTTP 200, where nothing about the status says so at all.
517 if status.is_success()
518 && PRIMARY_WORDINGS
519 .iter()
520 .any(|wording| normalized.contains(wording))
521 {
522 return Some(Self::Primary);
523 }
524 None
525 }
526
527 /// What this limiter is called where an operator can look it up.
528 const fn name(self) -> &'static str {
529 match self {
530 Self::Primary => "GitHub's primary API rate limit",
531 Self::Secondary => "GitHub's secondary rate limit",
532 }
533 }
534
535 /// What the endpoint an operator would go and check says about this limiter.
536 const fn where_to_look(self) -> &'static str {
537 match self {
538 Self::Primary => {
539 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
540 comes back."
541 }
542 Self::Secondary => {
543 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
544 primary budget and does not report this one, so budget showing there says \
545 nothing about this refusal, and every further attempt extends it."
546 }
547 }
548 }
549
550 /// The next step this limiter actually calls for.
551 const fn what_to_do(self) -> &'static str {
552 match self {
553 Self::Primary => {
554 "wait for the reset `gh api rate_limit` reports, then run the command again."
555 }
556 Self::Secondary => {
557 "leave this board alone for a few minutes, then run the command again — or \
558 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
559 }
560 }
561 }
562}
563
564/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
565#[derive(Debug, Clone, Copy)]
566struct Limited {
567 limiter: Limiter,
568 hint: Option<u64>,
569}
570
571impl Limited {
572 /// What the caller is told once this source has waited as long as it may.
573 ///
574 /// Both limiters report as [`SourceError::RateLimited`], because that is what
575 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
576 /// about *which* limiter it was makes it a different kind of failure. What differs is
577 /// the operator's next step, and that is what the message carries — a secondary
578 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
579 /// budget looks fine, and then back to retry the very burst that was refused.
580 fn exhausted(
581 self,
582 doing: &str,
583 waits: u32,
584 waited: Duration,
585 needed: Duration,
586 budget: Duration,
587 ) -> SourceError {
588 SourceError::RateLimited {
589 retry_after_seconds: self.hint,
590 message: Some(format!(
591 "{} refused this source while {doing}; it waited {} out over {} and was refused \
592 again, and the next wait of {} would take it past the {} one call may spend \
593 waiting. {} next: {}",
594 self.limiter.name(),
595 plural(waits, "refusal"),
596 seconds(waited),
597 seconds(needed),
598 seconds(budget),
599 self.limiter.where_to_look(),
600 self.limiter.what_to_do(),
601 )),
602 }
603 }
604}
605
606/// One attempt's outcome: an error to report, or a rate limit to wait out.
607enum Attempt {
608 Failed(SourceError),
609 Limited(Limited),
610}
611
612fn plural(count: u32, thing: &str) -> String {
613 if count == 1 {
614 format!("{count} {thing}")
615 } else {
616 format!("{count} {thing}s")
617 }
618}
619
620fn seconds(duration: Duration) -> String {
621 format!("{:.1}s", duration.as_secs_f64())
622}
623
624/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
625///
626/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
627/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
628/// header, and neither is what makes a response a refusal — so the whole cost of one this
629/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
630/// instead. Refusing the response over the header would turn a readable refusal into an
631/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
632fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
633 value
634 .and_then(|value| value.to_str().ok())
635 .and_then(|value| value.trim().parse::<u64>().ok())
636}
637
638/// Every mutation this source sends creates content — an issue, a board item, a field of
639/// one, a sub-issue link, a dependency — and no query in [`graphql::DOCUMENTS`] does, so
640/// what the secondary limiter counts and what the keyword says are the same set. That is
641/// what makes the keyword a sound test rather than a convenient one.
642fn is_mutation(query: &str) -> bool {
643 query.trim_start().starts_with("mutation")
644}
645
646/// What this source was doing, for a diagnostic that has to say so.
647///
648/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
649/// a document added without a description is caught by that list's own gate instead of
650/// falling through to the vague arm below.
651fn operation_description(query: &str) -> &'static str {
652 graphql::DOCUMENTS
653 .iter()
654 .find(|(document, _)| *document == query)
655 .map_or("talking to GitHub", |(_, doing)| *doing)
656}
657
658/// GitHub's published ceiling on content-generating requests, per minute.
659///
660/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
661/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
662/// from it, so a pacing value checked only against itself cannot go stale here.
663pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
664/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
665/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
666/// governs; it is pinned beside its sibling so the gate would notice either one moving.
667pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
668/// Shortest interval between two content-creating mutations, in milliseconds.
669///
670/// GitHub documents two secondary limits on content-generating requests:
671/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
672/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
673/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
674/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
675/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
676/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
677/// deliberately *not* what this paces at. An installation that wants the hourly bound
678/// honoured for a long sequence of copies says so through
679/// `pacing.min_mutation_interval_ms`.
680pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
681/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
682///
683/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
684/// own advice for a secondary limit — wait, and wait longer each time — without spending
685/// the first minute of a transient refusal doing nothing.
686pub const RETRY_BACKOFF_MS: u64 = 1_000;
687/// Total time one call may spend waiting out rate limits before it reports a failure.
688///
689/// Two minutes is long enough to ride out the refusals a paced copy still collects and
690/// short enough that a command an operator is watching returns. The bound is what makes
691/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
692/// the limiter, not in a process nobody can tell from a wedged one.
693pub const RETRY_BUDGET_MS: u64 = 120_000;
694
695fn default_token_env() -> String {
696 "GH_PROJECTS_TOKEN".to_owned()
697}
698fn default_endpoint() -> String {
699 "https://api.github.com/graphql".to_owned()
700}
701
702/// Where one status category lands on this board.
703///
704/// `null` — an absent value — disables the category for this instance, and using a
705/// disabled status is a refusal naming the status and the instance.
706#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
707#[serde(untagged)]
708pub enum StatusTargetConfig {
709 /// The name of a `Status` single-select option already on the board.
710 Column(ColumnName),
711 /// A closed issue state, whose reason is what tells done from cancelled.
712 Closed {
713 /// The `IssueClosedStateReason` to close with.
714 closed: ClosedState,
715 },
716}
717
718/// The name of a `Status` single-select option on the board.
719///
720/// Validated on the way in rather than checked later, so a blank option name — which
721/// nothing on a board can be — is a state this type cannot hold.
722#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
723#[serde(try_from = "String")]
724pub struct ColumnName(String);
725
726impl ColumnName {
727 /// The option name, as the board spells it.
728 fn as_str(&self) -> &str {
729 &self.0
730 }
731}
732
733impl TryFrom<String> for ColumnName {
734 type Error = String;
735
736 fn try_from(name: String) -> Result<Self, Self::Error> {
737 if name.trim().is_empty() {
738 return Err("a status_mapping option name cannot be blank".to_owned());
739 }
740 Ok(Self(name))
741 }
742}
743
744/// The two closed states this product can mean.
745///
746/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
747/// work nor abandoned work, so nothing here ever writes it.
748#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
749#[serde(rename_all = "kebab-case")]
750pub enum ClosedState {
751 /// `COMPLETED` — precisely done.
752 Completed,
753 /// `NOT_PLANNED` — precisely cancelled.
754 NotPlanned,
755}
756
757impl ClosedState {
758 const fn reason(self) -> &'static str {
759 match self {
760 Self::Completed => "COMPLETED",
761 Self::NotPlanned => "NOT_PLANNED",
762 }
763 }
764}
765
766/// Configuration for one GitHub Projects v2 board.
767#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
768#[serde(default, deny_unknown_fields)]
769pub struct GitHubProjectsConfig {
770 /// Login of the user or organization which owns the board.
771 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
772 /// The project number shown in the board's GitHub URL.
773 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
774 /// `owner/name` of the one repository this source creates its issues in.
775 ///
776 /// A board has no repository of its own and `createIssue` requires one, so a write
777 /// without this is refused naming the field. Reads never need it.
778 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
779 /// Environment variable containing a fine-grained token with Projects and Issues
780 /// read/write plus Pull requests read-only access for every repository represented on
781 /// the board.
782 #[serde(default = "default_token_env")]
783 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
784 /// GraphQL endpoint. GitHub Enterprise installations may override it.
785 #[serde(default = "default_endpoint")]
786 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
787 /// Per-instance mapping from a status category to where it lands on this board.
788 ///
789 /// A category this does not mention keeps its shipped default: `backlog` to
790 /// "Backlog", `todo` to "Todo", `in-progress` to "In Progress", `done` to closed as
791 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
792 /// disabled.
793 #[serde(default)]
794 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
795 /// How fast this source writes, and how long it waits out a rate-limit refusal.
796 ///
797 /// Every field keeps its shipped default when it is absent, and the defaults are
798 /// GitHub's own published limits rather than taste. See [`Pacing`].
799 #[serde(default)]
800 pub pacing: PacingConfig,
801}
802
803/// How fast this source writes, and how long it waits out a rate-limit refusal.
804///
805/// Configurable because a GitHub Enterprise installation sets its own limits and an
806/// operator who has already been refused may want to go slower still — not because the
807/// defaults are guesses.
808#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
809#[serde(default, deny_unknown_fields)]
810pub struct PacingConfig {
811 /// Shortest interval between two content-creating mutations, in milliseconds.
812 ///
813 /// Zero sends them as fast as they are asked for, which is what a fixture server on
814 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
815 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
816 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
817 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
818 /// zero while there is a budget to spend, because a schedule of zero-length waits
819 /// consumes none of it and so never ends.
820 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
821 /// Total time one call may spend waiting out rate limits, in milliseconds.
822 ///
823 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
824 /// the bound is what makes this a wait rather than a hang.
825 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
826}
827
828/// The largest any pacing setting may be, in milliseconds.
829///
830/// One hour. GitHub's own harshest published bound on content-generating requests works
831/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
832/// anything a real limit asks for, and past it the settings stop describing pacing at all:
833/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
834/// and an interval beyond it is a command that never sends its second mutation. It also
835/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
836/// what an `Instant` can hold on every platform.
837pub const MAX_PACING_MS: u64 = 3_600_000;
838
839/// [`PacingConfig`] with every default resolved and every value checked, which is what the
840/// source holds.
841#[derive(Debug, Clone, Copy)]
842struct Pacing {
843 min_mutation_interval: Duration,
844 retry_backoff: Duration,
845 retry_budget: Duration,
846}
847
848impl Pacing {
849 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
850 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
851 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
852 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
853 message: format!(
854 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
855 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
856 GitHub's own harshest published limit"
857 ),
858 }),
859 Some(value) => Ok(Duration::from_millis(value)),
860 None => Ok(Duration::from_millis(default)),
861 };
862 let retry_backoff = bounded(
863 config.retry_backoff_ms,
864 RETRY_BACKOFF_MS,
865 "retry_backoff_ms",
866 )?;
867 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
868 if retry_backoff.is_zero() && !retry_budget.is_zero() {
869 return Err(SourceError::Config {
870 message: format!(
871 "pacing.retry_backoff_ms of source {instance} is 0 while \
872 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
873 none of that budget, so it would retry a refusal forever. Set a backoff of \
874 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
875 waiting at all",
876 retry_budget.as_millis()
877 ),
878 });
879 }
880 Ok(Self {
881 min_mutation_interval: bounded(
882 config.min_mutation_interval_ms,
883 MIN_MUTATION_INTERVAL_MS,
884 "min_mutation_interval_ms",
885 )?,
886 retry_backoff,
887 retry_budget,
888 })
889 }
890}
891
892/// Factory for [`GitHubProjectsSource`].
893#[derive(Debug, Clone, Copy, Default)]
894pub struct Plugin;
895
896impl SourcePlugin for Plugin {
897 fn kind(&self) -> &'static str {
898 KIND
899 }
900 fn config_schema(&self) -> Schema {
901 schema_for!(GitHubProjectsConfig)
902 }
903 fn build(
904 &self,
905 name: &SourceName,
906 config: &Value,
907 secrets: &dyn SecretResolver,
908 ) -> Result<Box<dyn TaskSource>, SourceError> {
909 let config: GitHubProjectsConfig =
910 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
911 message: format!("source {name}: {e}"),
912 })?;
913 let source =
914 GitHubProjectsSource::new(name, config, secrets).map_err(|error| match error {
915 SourceError::Config { message } => SourceError::Config {
916 message: format!("source {name}: {message}"),
917 },
918 SourceError::Auth { message } => SourceError::Auth {
919 message: format!("source {name}: {message}"),
920 },
921 other => other,
922 })?;
923 Ok(Box::new(source))
924 }
925}
926
927/// Where a status category lands on this board, once configuration is resolved.
928#[derive(Debug, Clone, PartialEq, Eq)]
929enum StatusTarget {
930 /// Not usable against this instance.
931 Disabled,
932 /// The board's `Status` option of this name.
933 Column(ColumnName),
934 /// A closed issue, with the reason that says which closed it means.
935 Closed(ClosedState),
936}
937
938/// Every status category, in the order the vocabulary declares them.
939///
940/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
941/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
942/// added to the shared vocabulary fails to compile until it is named there, and this
943/// crate's suite reconciles this list against that enum's own derived schema, which is
944/// generated from the variants rather than written beside them. The schema is what
945/// catches a list left one short — a list checking only the positions it already holds
946/// would pass while every mapping indexed by the new position panicked.
947pub const CATEGORIES: [StatusCategory; 7] = [
948 StatusCategory::Draft,
949 StatusCategory::Backlog,
950 StatusCategory::Todo,
951 StatusCategory::InProgress,
952 StatusCategory::Done,
953 StatusCategory::Cancelled,
954 StatusCategory::Unknown,
955];
956
957/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
958#[must_use]
959pub const fn category_position(category: StatusCategory) -> usize {
960 match category {
961 StatusCategory::Draft => 0,
962 StatusCategory::Backlog => 1,
963 StatusCategory::Todo => 2,
964 StatusCategory::InProgress => 3,
965 StatusCategory::Done => 4,
966 StatusCategory::Cancelled => 5,
967 StatusCategory::Unknown => 6,
968 }
969}
970
971/// The spelling a status category is configured and reported under.
972fn category_name(category: StatusCategory) -> &'static str {
973 match category {
974 StatusCategory::Draft => "draft",
975 StatusCategory::Backlog => "backlog",
976 StatusCategory::Todo => "todo",
977 StatusCategory::InProgress => "in-progress",
978 StatusCategory::Done => "done",
979 StatusCategory::Cancelled => "cancelled",
980 StatusCategory::Unknown => "unknown",
981 }
982}
983
984/// A shipped default's option name.
985///
986/// The literals below are this file's own and non-blank, and they are validated by the
987/// one constructor a configured name goes through rather than beside it.
988fn shipped_column(name: &'static str) -> ColumnName {
989 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
990}
991
992/// The shipped default for one category, before this instance's configuration.
993fn shipped_default(category: StatusCategory) -> StatusTarget {
994 match category {
995 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
996 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
997 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
998 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
999 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
1000 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
1001 }
1002}
1003
1004/// This instance's complete category-to-target mapping, read in both directions.
1005///
1006/// One target per category, held at that category's own [`category_position`], so a
1007/// category missing from the mapping, named twice in it, or filed out of order is a
1008/// state this type cannot hold rather than one [`Self::target`] has to defend against.
1009#[derive(Debug, Clone)]
1010struct StatusMapping {
1011 targets: [StatusTarget; CATEGORIES.len()],
1012}
1013
1014impl StatusMapping {
1015 fn resolve(
1016 configured: BTreeMap<String, Option<StatusTargetConfig>>,
1017 instance: &SourceName,
1018 ) -> Result<Self, SourceError> {
1019 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
1020 for (key, value) in configured {
1021 let category = CATEGORIES
1022 .iter()
1023 .find(|category| category_name(**category) == key)
1024 .ok_or_else(|| SourceError::Config {
1025 message: format!(
1026 "status_mapping names {key:?}, which is not a status category of source \
1027 {instance}; the categories are {}",
1028 CATEGORIES
1029 .iter()
1030 .map(|category| category_name(*category))
1031 .collect::<Vec<_>>()
1032 .join(", ")
1033 ),
1034 })?;
1035 overrides.insert(category_name(*category), value);
1036 }
1037 // `CATEGORIES[position] == category` for every category — the crate's suite
1038 // asserts it — so mapping the list in order fills each category's own slot.
1039 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
1040 None => shipped_default(category),
1041 Some(None) => StatusTarget::Disabled,
1042 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
1043 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
1044 });
1045 let mapping = Self { targets };
1046 for (index, category) in CATEGORIES.into_iter().enumerate() {
1047 let StatusTarget::Column(option) = mapping.target(category) else {
1048 continue;
1049 };
1050 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
1051 matches!(mapping.target(**earlier), StatusTarget::Column(name)
1052 if name.as_str().eq_ignore_ascii_case(option.as_str()))
1053 }) {
1054 return Err(SourceError::Config {
1055 message: format!(
1056 "status_mapping of source {instance} sends both {} and {} to the board \
1057 option {:?}; one option cannot read back as two categories",
1058 category_name(*other),
1059 category_name(category),
1060 option.as_str()
1061 ),
1062 });
1063 }
1064 }
1065 Ok(mapping)
1066 }
1067
1068 fn target(&self, category: StatusCategory) -> &StatusTarget {
1069 &self.targets[category_position(category)]
1070 }
1071
1072 /// The category a board option name reports, or `None` when nothing maps to it.
1073 fn category_of(&self, option: &str) -> Option<StatusCategory> {
1074 CATEGORIES.into_iter().find(|category| {
1075 matches!(self.target(*category), StatusTarget::Column(name)
1076 if name.as_str().eq_ignore_ascii_case(option))
1077 })
1078 }
1079}
1080
1081/// The one repository this source creates issues in.
1082#[derive(Debug, Clone)]
1083struct RepositoryTarget {
1084 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1085 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
1086}
1087
1088impl RepositoryTarget {
1089 fn parse(value: &str) -> Result<Self, SourceError> {
1090 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
1091 message: format!(
1092 "repository must be spelled owner/name; {value:?} names no repository"
1093 ),
1094 })?;
1095 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
1096 return Err(SourceError::Config {
1097 message: format!(
1098 "repository must be spelled owner/name with a GitHub login and one \
1099 repository name; {value:?} is not"
1100 ),
1101 });
1102 }
1103 Ok(Self {
1104 owner: owner.to_owned(),
1105 name: name.to_owned(),
1106 })
1107 }
1108
1109 fn origin(&self) -> String {
1110 format!("github.com/{}/{}", self.owner, self.name)
1111 }
1112}
1113
1114/// A source which reads GitHub afresh for every operation.
1115pub struct GitHubProjectsSource {
1116 /// This source's configured name, used both to tell a far end naming this source
1117 /// from one naming a system it knows nothing about, and to name the instance a
1118 /// status refusal is about.
1119 name: SourceName,
1120 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
1121 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
1122 repository: Option<RepositoryTarget>,
1123 endpoint: Url,
1124 token: SecretString,
1125 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1126 statuses: StatusMapping,
1127 client: Client,
1128 /// Every item this source has created since it was built, in the order it created
1129 /// them.
1130 ///
1131 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1132 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1133 /// a copy resolving a dependency on an item it had just created refused it as not
1134 /// found. A board read is completed from this — an item remembered here and absent from
1135 /// the read is added back, because the board really does hold it and only the read is
1136 /// behind.
1137 ///
1138 /// It is not a cache of a user's work: nothing is remembered that this process did not
1139 /// itself just write, it lives and dies with the process, and it is never consulted for
1140 /// an item this source did not create.
1141 created: Mutex<Vec<Resolved>>,
1142 /// How fast this source writes, and how long it waits out a refusal.
1143 pacing: Pacing,
1144 /// When the last content-creating mutation finished, or the moment the furthest-out
1145 /// reserved slot releases the next one, whichever is later — so the one after it can be
1146 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1147 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1148 /// what it is measured from.
1149 last_mutation: Mutex<Option<Instant>>,
1150 /// The board as this process last read it, for the length of one command.
1151 ///
1152 /// A copy of a project used to re-read the whole board, paged, before writing each of
1153 /// its items, which is by far the largest part of a copy's request count and none of
1154 /// its work. Nothing else changes this board while a command runs — this source's own
1155 /// writes are the only writer — so one read answers them all.
1156 ///
1157 /// It is not a store of a user's work and it is not the cache the no-persistence
1158 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1159 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1160 /// an item this command created and then depends on resolves whether or not GitHub's
1161 /// own eventually-consistent read has caught up. A write to an item already on the
1162 /// board updates the entry here too, so what this holds is the last read plus this
1163 /// process's own writes rather than a snapshot taken before them.
1164 board_cache: Mutex<Option<Board>>,
1165 /// The destination repository's node id, resolved once rather than per issue created.
1166 ///
1167 /// A repository's node id does not change, and re-reading it for every issue of a copy
1168 /// spent one request per item on an answer this source already had.
1169 repository_cache: Mutex<Option<String>>,
1170}
1171
1172impl GitHubProjectsSource {
1173 /// Validate configuration and capture the named credential without exposing it.
1174 ///
1175 /// # Errors
1176 ///
1177 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1178 /// [`SourceError::Auth`] when the named credential is missing or empty.
1179 pub fn new(
1180 name: &SourceName,
1181 config: GitHubProjectsConfig,
1182 secrets: &dyn SecretResolver,
1183 ) -> Result<Self, SourceError> {
1184 if !valid_github_owner(&config.owner) {
1185 return Err(SourceError::Config {
1186 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1187 });
1188 }
1189 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1190 return Err(SourceError::Config {
1191 message: format!("project_number must be between 1 and {}", i32::MAX),
1192 });
1193 }
1194 if !valid_environment_name(&config.token_env) {
1195 return Err(SourceError::Config {
1196 message: "token_env must be a valid environment-variable name".into(),
1197 });
1198 }
1199 let repository = config
1200 .repository
1201 .as_deref()
1202 .map(RepositoryTarget::parse)
1203 .transpose()?;
1204 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1205 message: format!("endpoint is not a valid URL: {e}"),
1206 })?;
1207 if endpoint.scheme() != "https"
1208 && !(endpoint.scheme() == "http"
1209 && endpoint
1210 .host_str()
1211 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1212 {
1213 return Err(SourceError::Config {
1214 message:
1215 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1216 .into(),
1217 });
1218 }
1219 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1220 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1221 })?;
1222 Ok(Self {
1223 name: name.clone(),
1224 owner: config.owner,
1225 project_number: config.project_number,
1226 repository,
1227 endpoint,
1228 token,
1229 credential_name: config.token_env,
1230 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1231 client: Client::builder()
1232 .user_agent("onetaskgraph")
1233 .build()
1234 .map_err(|e| SourceError::Config {
1235 message: format!("cannot build HTTP client: {e}"),
1236 })?,
1237 created: Mutex::new(Vec::new()),
1238 pacing: Pacing::resolve(config.pacing, name)?,
1239 last_mutation: Mutex::new(None),
1240 board_cache: Mutex::new(None),
1241 repository_cache: Mutex::new(None),
1242 })
1243 }
1244
1245 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1246 /// rate limit rather than handing it straight back as an error.
1247 ///
1248 /// Retrying is safe for every document here, including the mutations, and the reason
1249 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1250 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1251 /// this replays has already taken effect. An outcome this source cannot know — the
1252 /// send failed, or the body could not be read, so the mutation may well have landed —
1253 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1254 /// attempt. A duplicate write would come from replaying one of those, and none is
1255 /// replayed.
1256 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1257 let doing = operation_description(query);
1258 let mut waited = Duration::ZERO;
1259 let mut waits = 0_u32;
1260 let mut backoff = self.pacing.retry_backoff;
1261 loop {
1262 if is_mutation(query) {
1263 let spacing = self.reserve_mutation_slot();
1264 if !spacing.is_zero() {
1265 tokio::time::sleep(spacing).await;
1266 }
1267 }
1268 let attempt = self.send_once(query, &variables).await;
1269 if is_mutation(query) {
1270 self.finish_mutation();
1271 }
1272 let limited = match attempt {
1273 Ok(data) => return Ok(data),
1274 Err(Attempt::Failed(error)) => return Err(error),
1275 Err(Attempt::Limited(limited)) => limited,
1276 };
1277 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1278 // move that extends a secondary limit, so a hint below the schedule's own next
1279 // wait is raised to it.
1280 let wait = match limited.hint {
1281 Some(hint) => Duration::from_secs(hint).max(backoff),
1282 None => backoff,
1283 };
1284 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1285 // A wait of nothing spends none of the budget, so it is exhaustion rather
1286 // than a retry. `Pacing::resolve` rules out every way of configuring one
1287 // except a budget of zero, where reporting the first refusal is the ask.
1288 if wait.is_zero() || wait > remaining {
1289 return Err(limited.exhausted(
1290 doing,
1291 waits,
1292 waited,
1293 wait,
1294 self.pacing.retry_budget,
1295 ));
1296 }
1297 tokio::time::sleep(wait).await;
1298 waited += wait;
1299 waits += 1;
1300 backoff = backoff.saturating_mul(2);
1301 }
1302 }
1303
1304 /// The next moment a content-creating mutation may leave this source, as a wait from
1305 /// now.
1306 ///
1307 /// The slot is reserved under the lock and the waiting happens outside it, so two
1308 /// callers take two slots rather than the same one — and no lock is held across an
1309 /// await.
1310 ///
1311 /// The moment it is spaced from is the previous mutation's *completion*, which
1312 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1313 /// own is the wrong thing to measure from.
1314 fn reserve_mutation_slot(&self) -> Duration {
1315 if self.pacing.min_mutation_interval.is_zero() {
1316 return Duration::ZERO;
1317 }
1318 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1319 // it would turn an earlier failure into a second one for no gain.
1320 let mut last = self
1321 .last_mutation
1322 .lock()
1323 .unwrap_or_else(std::sync::PoisonError::into_inner);
1324 let now = Instant::now();
1325 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1326 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1327 let at = last.map_or(now, |previous| {
1328 previous
1329 .checked_add(self.pacing.min_mutation_interval)
1330 .map_or(now, |earliest| earliest.max(now))
1331 });
1332 *last = Some(at);
1333 at.saturating_duration_since(now)
1334 }
1335
1336 /// Record that a content-creating mutation has finished, so the next one is spaced
1337 /// from here rather than from the moment this one was released.
1338 ///
1339 /// This source can only choose when a request *departs*; the limiter counts when it
1340 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1341 /// departure from the last therefore hands the limiter a gap of the interval less that
1342 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1343 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1344 /// slower machine while passing on a quick one.
1345 ///
1346 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1347 /// previous request had already arrived before its response came back, so its arrival
1348 /// is no later than this moment, and the next mutation is released at least the
1349 /// interval after this moment and arrives no earlier than it is released: the gap the
1350 /// limiter measures is therefore at least the interval, whatever transit costs and on
1351 /// whatever platform. The price is that a mutation's own round trip no longer counts
1352 /// towards its spacing, which makes this source slightly slower than the configured
1353 /// rate rather than slightly faster — the safe side of a limit that punishes being
1354 /// wrong by refusing reads for the next fifty minutes.
1355 ///
1356 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1357 /// and one that never left costs only a wait nobody needed.
1358 fn finish_mutation(&self) {
1359 if self.pacing.min_mutation_interval.is_zero() {
1360 return;
1361 }
1362 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1363 let mut last = self
1364 .last_mutation
1365 .lock()
1366 .unwrap_or_else(std::sync::PoisonError::into_inner);
1367 let now = Instant::now();
1368 // `max` rather than an assignment: a concurrent caller may already have reserved a
1369 // slot further out, and completing this request must never pull that slot back in.
1370 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1371 }
1372
1373 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1374 /// failure that waiting cannot help.
1375 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1376 let response = self
1377 .client
1378 .post(self.endpoint.clone())
1379 .bearer_auth(self.token.expose_secret())
1380 .json(&json!({"query": query, "variables": variables}))
1381 .send()
1382 .await
1383 .map_err(|e| {
1384 Attempt::Failed(SourceError::Unavailable {
1385 message: format!("GitHub GraphQL request failed: {e}"),
1386 })
1387 })?;
1388 let status = response.status();
1389 let header = |name: &str| whole_seconds(response.headers().get(name));
1390 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
1391 // that are not text at all — is "not known to be exhausted". This never makes a
1392 // response a refusal on its own: it says which limiter a refusal is attributed to
1393 // and where its hint comes from, so a value this cannot read costs a hint rather
1394 // than an answer.
1395 let exhausted = response
1396 .headers()
1397 .get("x-ratelimit-remaining")
1398 .and_then(|value| value.to_str().ok())
1399 == Some("0");
1400 // `retry-after` is what GitHub asks for when it asks; when it does not and the
1401 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
1402 // which is the same question answered as an absolute time. Nothing else here is a
1403 // hint, and a schedule is what answers a refusal that carries none.
1404 let hint = header("retry-after").or_else(|| {
1405 exhausted
1406 .then(|| header("x-ratelimit-reset"))
1407 .flatten()
1408 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
1409 });
1410 // Read before it is parsed, because the evidence which tells a secondary rate
1411 // limit from a rejected credential is in the body of a response whose status says
1412 // only "forbidden" — and a non-success response was never parsed at all.
1413 let body = response.text().await.map_err(|e| {
1414 Attempt::Failed(SourceError::Unavailable {
1415 message: format!("GitHub GraphQL response could not be read: {e}"),
1416 })
1417 })?;
1418 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
1419 return Err(Attempt::Limited(Limited { limiter, hint }));
1420 }
1421 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
1422 return Err(Attempt::Failed(SourceError::Auth {
1423 message: format!(
1424 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
1425 ),
1426 }));
1427 }
1428 if !status.is_success() {
1429 return Err(Attempt::Failed(SourceError::Unavailable {
1430 message: format!("GitHub GraphQL returned HTTP {status}"),
1431 }));
1432 }
1433 self.answer(&body).map_err(Attempt::Failed)
1434 }
1435
1436 /// What one successful HTTP response says, once its GraphQL errors are read.
1437 fn answer(&self, body: &str) -> Result<Value, SourceError> {
1438 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
1439 message: format!("GitHub returned invalid JSON: {e}"),
1440 })?;
1441 let errors = body
1442 .get("errors")
1443 .map(|value| {
1444 value.as_array().ok_or_else(|| SourceError::Malformed {
1445 message: "GitHub response errors is not an array".into(),
1446 })
1447 })
1448 .transpose()?;
1449 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
1450 let messages = errors
1451 .iter()
1452 .filter_map(|e| e.get("message").and_then(Value::as_str))
1453 .collect::<Vec<_>>()
1454 .join("; ");
1455 let message = if messages.is_empty() {
1456 "GitHub returned GraphQL errors".into()
1457 } else {
1458 messages
1459 };
1460 let normalized = message.to_ascii_lowercase();
1461 if normalized.contains("resource not accessible") || normalized.contains("scope") {
1462 return Err(SourceError::Auth {
1463 message: format!(
1464 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
1465 self.credential_name
1466 ),
1467 });
1468 }
1469 return Err(SourceError::Refused { message });
1470 }
1471 body.get("data")
1472 .filter(|data| data.is_object())
1473 .cloned()
1474 .ok_or_else(|| SourceError::Malformed {
1475 message: "GitHub response has no data object".into(),
1476 })
1477 }
1478
1479 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
1480 // GraphQL cannot independently page them inside the outer item page. This source page is
1481 // deliberately bounded at that published maximum; the live drift journey exercises it.
1482 async fn board_page(
1483 &self,
1484 items_after: Option<&str>,
1485 items_first: u32,
1486 ) -> Result<Value, SourceError> {
1487 let data = self
1488 .graphql(
1489 graphql::BOARD,
1490 json!({"owner":self.owner,"number":self.project_number,
1491 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
1492 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
1493 )
1494 .await?;
1495 data.pointer("/owner/projectV2")
1496 .filter(|v| !v.is_null())
1497 .cloned()
1498 .ok_or_else(|| SourceError::Refused {
1499 message: format!(
1500 "GitHub project {}/{} was not found or is not visible to the token",
1501 self.owner, self.project_number
1502 ),
1503 })
1504 }
1505
1506 /// The search that finds the issues of this board, narrowed by `also` when it is
1507 /// given.
1508 ///
1509 /// `project:owner/number` is what scopes a search to one board, and `is:issue` is what
1510 /// keeps pull requests out of it: GitHub's `ISSUE` search type covers both, and a pull
1511 /// request is somebody's change rather than a unit of plan. `-has:parent` is *not*
1512 /// here on purpose — GitHub accepts it and silently ignores it, so a project is told
1513 /// from a task by the `parent` field each issue carries rather than by the search.
1514 fn board_search(&self, also: Option<&str>) -> String {
1515 let scope = format!("project:{}/{} is:issue", self.owner, self.project_number);
1516 match also {
1517 Some(also) => format!("{scope} {also}"),
1518 None => scope,
1519 }
1520 }
1521
1522 /// One issue this source reached directly, as the board item a read of the board would
1523 /// have produced — or `None` when this board does not hold it.
1524 ///
1525 /// The board half of an issue rides along on `Issue.projectItems`, so the value handed
1526 /// to [`Self::resolve`] is the very shape a `ProjectV2.items` read gives it: the board
1527 /// item's own id, that item's field values, and the issue as its content. One resolver
1528 /// for both routes is what makes an issue read through a search, through its own node
1529 /// id, or through its project's sub-issues report the same title, the same status, the
1530 /// same labels and the same qualified id.
1531 ///
1532 /// An issue with no entry for *this* board is not this source's to report, which is
1533 /// what keeps an id naming some other repository's issue from being answered as an item
1534 /// of this board.
1535 fn resolve_issue(&self, issue: &Value) -> Result<Option<Resolved>, SourceError> {
1536 if optional_str(issue, "__typename")? != Some("Issue") {
1537 return Ok(None);
1538 }
1539 let memberships = issue
1540 .get("projectItems")
1541 .ok_or_else(|| SourceError::Malformed {
1542 message: "GitHub issue is missing projectItems".into(),
1543 })?;
1544 let nodes = memberships
1545 .get("nodes")
1546 .and_then(Value::as_array)
1547 .ok_or_else(|| SourceError::Malformed {
1548 message: "GitHub issue projectItems.nodes is not an array".into(),
1549 })?;
1550 let held = nodes.iter().find(|node| {
1551 node.pointer("/project/number").and_then(Value::as_u64)
1552 == Some(u64::from(self.project_number))
1553 });
1554 let Some(held) = held else {
1555 // Only now: an issue whose entry for this board sits past the page asked for
1556 // would otherwise read as an issue this board does not hold, which is the one
1557 // wrong answer available here.
1558 complete_connection(
1559 memberships,
1560 "issue board memberships",
1561 BOARD_ITEMS_PAGE_SIZE,
1562 )?;
1563 return Ok(None);
1564 };
1565 let item = json!({
1566 "id": required_str(held, "id")?,
1567 "fieldValues": held.get("fieldValues"),
1568 "content": issue,
1569 });
1570 self.resolve(&item)
1571 }
1572
1573 /// One page of a board-scoped issue search, and where the next page resumes.
1574 async fn search_page(
1575 &self,
1576 search: &str,
1577 first: u32,
1578 after: Option<&str>,
1579 ) -> Result<(Vec<Resolved>, Option<String>), SourceError> {
1580 let data = self
1581 .graphql(
1582 graphql::SEARCH_ISSUES,
1583 json!({"search":search,"type":"ISSUE","first":first.min(MAX_PAGE_SIZE),
1584 "after":after,"nestedFirst":NESTED_PAGE_SIZE,
1585 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
1586 )
1587 .await?;
1588 let connection = data.get("search").ok_or_else(|| SourceError::Malformed {
1589 message: "GitHub search response has no search connection".into(),
1590 })?;
1591 let mut found = Vec::new();
1592 for node in connection
1593 .get("nodes")
1594 .and_then(Value::as_array)
1595 .ok_or_else(|| SourceError::Malformed {
1596 message: "GitHub search nodes is not an array".into(),
1597 })?
1598 {
1599 if let Some(resolved) = self.resolve_issue(node)? {
1600 found.push(resolved);
1601 }
1602 }
1603 let info = connection
1604 .get("pageInfo")
1605 .ok_or_else(|| SourceError::Malformed {
1606 message: "GitHub search connection has no pageInfo".into(),
1607 })?;
1608 let next = required_bool(info, "hasNextPage")?
1609 .then(|| required_str(info, "endCursor"))
1610 .transpose()?
1611 .map(str::to_owned);
1612 if let Some(next) = &next {
1613 validate_cursor_progress(after, next)?;
1614 }
1615 Ok((found, next))
1616 }
1617
1618 /// Every issue this board holds, walked to exhaustion, completed with what this run
1619 /// wrote.
1620 ///
1621 /// The completion is not an optimisation and it is not a cache: GitHub's issue search
1622 /// is an index and is eventually consistent, so an issue this run created seconds ago
1623 /// is routinely absent from it, and a project listed straight after being written would
1624 /// otherwise be missing from its own board. What is added back is only what this
1625 /// process itself wrote, out of [`Self::created`], which lives and dies with the
1626 /// process.
1627 async fn board_issues(&self) -> Result<Vec<Resolved>, SourceError> {
1628 let mut after: Option<String> = None;
1629 let mut found = Vec::new();
1630 let search = self.board_search(None);
1631 loop {
1632 let (page, next) = self
1633 .search_page(&search, MAX_PAGE_SIZE, after.as_deref())
1634 .await?;
1635 found.extend(page);
1636 match next {
1637 Some(next) => after = Some(next),
1638 None => break,
1639 }
1640 }
1641 self.completed_with_written(found, |_| true)
1642 }
1643
1644 /// `found`, with everything this run wrote that `keep` accepts and the read did not
1645 /// report.
1646 ///
1647 /// See [`Self::created`] and [`Self::board_issues`] for why a read has to be completed
1648 /// at all: the search index is behind, and a node read of an item filed moments ago can
1649 /// be too.
1650 fn completed_with_written(
1651 &self,
1652 mut found: Vec<Resolved>,
1653 keep: impl Fn(&Resolved) -> bool,
1654 ) -> Result<Vec<Resolved>, SourceError> {
1655 for own in self.created()?.iter().filter(|own| keep(own)) {
1656 if !found.iter().any(|item| item.id == own.id) {
1657 found.push(own.clone());
1658 }
1659 }
1660 Ok(found)
1661 }
1662
1663 /// What resolving one node id reached.
1664 ///
1665 /// Three answers rather than an `Option`, because a board *draft* is none of the other
1666 /// two: it is not an issue, it has no node of its own this source can read the board
1667 /// half off, and its only home is the board's own item connection — so a read of one
1668 /// is completed from there rather than reported as nothing.
1669 async fn reach(&self, id: &NativeId) -> Result<Reached, SourceError> {
1670 let asked = self
1671 .graphql(
1672 graphql::ISSUE,
1673 json!({"id":id.0,"nestedFirst":NESTED_PAGE_SIZE,
1674 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
1675 )
1676 .await;
1677 let data = match asked {
1678 Ok(data) => data,
1679 // A string that is not a node id at all is not a failure to report: it is an id
1680 // this board does not hold, which is what every read of one already answers.
1681 Err(error) if unresolvable_node(&error) => return Ok(Reached::Nothing),
1682 Err(error) => return Err(error),
1683 };
1684 let Some(node) = data.get("node").filter(|value| !value.is_null()) else {
1685 return Ok(Reached::Nothing);
1686 };
1687 if optional_str(node, "__typename")? == Some("DraftIssue") {
1688 return Ok(Reached::Draft);
1689 }
1690 Ok(match self.resolve_issue(node)? {
1691 Some(item) => Reached::Held(Box::new(item)),
1692 None => Reached::Nothing,
1693 })
1694 }
1695
1696 /// One item of this board by its own id, whatever kind it is.
1697 ///
1698 /// Resolved from the identifier alone: no search, board-wide or otherwise. What this
1699 /// run wrote is read first, because a node read of an item created moments ago can
1700 /// still be behind the board field values written onto it — see [`Self::created`].
1701 async fn item_by_id(&self, id: &NativeId) -> Result<Option<Resolved>, SourceError> {
1702 if let Some(own) = self.created()?.iter().find(|own| own.id == *id) {
1703 return Ok(Some(own.clone()));
1704 }
1705 match self.reach(id).await? {
1706 Reached::Held(item) => Ok(Some(*item)),
1707 Reached::Nothing => Ok(None),
1708 // The one read that still costs the board: a draft lives nowhere else.
1709 Reached::Draft => Ok(self
1710 .board()
1711 .await?
1712 .items
1713 .into_iter()
1714 .find(|item| item.id == *id)),
1715 }
1716 }
1717
1718 /// Everything filed under one issue of this board, walked to exhaustion — or `None`
1719 /// when that id names nothing here with a sub-issue relationship to walk.
1720 ///
1721 /// `None` and an empty answer are different: `None` is *this is not an issue of this
1722 /// GitHub*, which is what sends a project selector on to be read as a name, and an
1723 /// empty vector is a project that holds nothing.
1724 async fn sub_issues(&self, id: &NativeId) -> Result<Option<Vec<Resolved>>, SourceError> {
1725 let mut after: Option<String> = None;
1726 let mut children = Vec::new();
1727 loop {
1728 let asked = self
1729 .graphql(
1730 graphql::SUB_ISSUES,
1731 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after,
1732 "nestedFirst":NESTED_PAGE_SIZE,
1733 "boardItems":BOARD_ITEMS_PAGE_SIZE,"duplicates":true}),
1734 )
1735 .await;
1736 let data = match asked {
1737 Ok(data) => data,
1738 // A string that is not a node id at all is not a failure to report: it is
1739 // the ordinary answer to a selector naming a project by its name.
1740 Err(error) if unresolvable_node(&error) => return Ok(None),
1741 Err(error) => return Err(error),
1742 };
1743 let Some(connection) = data
1744 .pointer("/node/subIssues")
1745 .filter(|value| !value.is_null())
1746 else {
1747 // No such node, or one with no sub-issue relationship — a board draft is
1748 // the one this board can really hold.
1749 return Ok(None);
1750 };
1751 for node in connection
1752 .get("nodes")
1753 .and_then(Value::as_array)
1754 .ok_or_else(|| SourceError::Malformed {
1755 message: "GitHub subIssues.nodes is not an array".into(),
1756 })?
1757 {
1758 if let Some(resolved) = self.resolve_issue(node)? {
1759 children.push(resolved);
1760 }
1761 }
1762 let info = connection
1763 .get("pageInfo")
1764 .ok_or_else(|| SourceError::Malformed {
1765 message: "GitHub subIssues connection has no pageInfo".into(),
1766 })?;
1767 let next = required_bool(info, "hasNextPage")?
1768 .then(|| required_str(info, "endCursor"))
1769 .transpose()?;
1770 match next {
1771 Some(next) => {
1772 validate_cursor_progress(after.as_deref(), next)?;
1773 after = Some(next.to_owned());
1774 }
1775 None => return Ok(Some(children)),
1776 }
1777 }
1778 }
1779
1780 /// Which issue of this board a project *name* is, or `None` when none is.
1781 ///
1782 /// One bounded query which filters on that name at the server, rather than a walk of
1783 /// every issue the board holds. The name is compared again here: the qualifier narrows
1784 /// what GitHub sends, and this source decides what it names.
1785 async fn project_by_name(&self, name: &str) -> Result<Option<NativeId>, SourceError> {
1786 let search = self.board_search(Some(&title_qualifier(name)));
1787 let (candidates, _) = self.search_page(&search, MAX_PAGE_SIZE, None).await?;
1788 Ok(candidates
1789 .into_iter()
1790 .find(|item| {
1791 item.kind == BoardKind::Work(ItemKind::Project)
1792 && item.title.eq_ignore_ascii_case(name)
1793 })
1794 .map(|item| item.id))
1795 }
1796
1797 /// Everything filed under one project of this board: the sub-issues of the issue that
1798 /// project is.
1799 ///
1800 /// Tasks *and* documents, because a document filed under a project is a sub-issue of it
1801 /// too — the caller keeps the kind it asked for. Nothing about this grows as the board
1802 /// gains projects, or as another project gains tasks.
1803 ///
1804 /// A qualified id names the issue and is asked for its sub-issues directly: one
1805 /// request, no search of any kind. Only a selector GitHub cannot resolve that way is
1806 /// read as a project *name*, which costs the one bounded search
1807 /// [`Self::project_by_name`] makes.
1808 async fn project_children(&self, selector: &NativeId) -> Result<Vec<Resolved>, SourceError> {
1809 let (project, children) = match self.sub_issues(selector).await? {
1810 Some(children) => (selector.clone(), children),
1811 None => match self.project_by_name(&selector.0).await? {
1812 Some(project) => {
1813 let children = self.sub_issues(&project).await?.unwrap_or_default();
1814 (project, children)
1815 }
1816 None => return Ok(Vec::new()),
1817 },
1818 };
1819 self.completed_with_written(children, |own| own.parent.as_ref() == Some(&project))
1820 }
1821
1822 /// Every item on the board, with the one board identity they all share.
1823 ///
1824 /// See [`Self::board_cache`]. The completion from `created` happens on every call
1825 /// rather than once, which is what the cache could otherwise have broken.
1826 async fn board(&self) -> Result<Board, SourceError> {
1827 let cached = self.board_cache()?.clone();
1828 let mut board = match cached {
1829 Some(board) => board,
1830 None => {
1831 let read = self.read_board().await?;
1832 *self.board_cache()? = Some(read.clone());
1833 read
1834 }
1835 };
1836 for own in self.created()?.iter() {
1837 if !board.items.iter().any(|item| item.id == own.id) {
1838 board.items.push(own.clone());
1839 }
1840 }
1841 Ok(board)
1842 }
1843
1844 /// This process's own view of the board, or the refusal a poisoned lock is.
1845 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
1846 self.board_cache
1847 .lock()
1848 .map_err(|_| SourceError::Unavailable {
1849 message: "this source's view of the board was left inconsistent by an earlier \
1850 failure; next: run the command again"
1851 .into(),
1852 })
1853 }
1854
1855 /// Bring this process's own view of the board up to an item it has just written.
1856 ///
1857 /// A created item goes to `created`, which is what completes a board read GitHub's own
1858 /// eventual consistency has left behind. An item that was already there is replaced
1859 /// where it sits, so a second write of it in the same command reads its real parent
1860 /// rather than the one it had before the first write.
1861 ///
1862 /// "Where it sits" is two places, and missing the first leaves a stale record that
1863 /// wins: an item this same run created is held in `created` and not in the cached
1864 /// board, and `board` completes the cached board *from* `created`, so replacing only
1865 /// the cached copy of such an item replaces nothing and the read still reports the
1866 /// title it was created with.
1867 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
1868 if created {
1869 self.created()?.push(item);
1870 return Ok(());
1871 }
1872 {
1873 let mut own = self.created()?;
1874 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
1875 *held = item;
1876 return Ok(());
1877 }
1878 }
1879 if let Some(board) = self.board_cache()?.as_mut()
1880 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
1881 {
1882 *held = item;
1883 }
1884 Ok(())
1885 }
1886
1887 /// Forget one item this process has just deleted, from both halves of its own view.
1888 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
1889 self.created()?.retain(|own| own.id != *id);
1890 if let Some(board) = self.board_cache()?.as_mut() {
1891 board.items.retain(|item| item.id != *id);
1892 }
1893 Ok(())
1894 }
1895
1896 /// Every page of the board, read from GitHub.
1897 async fn read_board(&self) -> Result<Board, SourceError> {
1898 let mut after: Option<String> = None;
1899 let mut items = Vec::new();
1900 let mut board;
1901 loop {
1902 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
1903 for item in page
1904 .pointer("/items/nodes")
1905 .and_then(Value::as_array)
1906 .ok_or_else(|| SourceError::Malformed {
1907 message: "GitHub project items.nodes is not an array".into(),
1908 })?
1909 {
1910 if let Some(resolved) = self.resolve(item)? {
1911 items.push(resolved);
1912 }
1913 }
1914 let info = page
1915 .pointer("/items/pageInfo")
1916 .ok_or_else(|| SourceError::Malformed {
1917 message: "GitHub project items have no pageInfo".into(),
1918 })?;
1919 let has_next = required_bool(info, "hasNextPage")?;
1920 let next = has_next
1921 .then(|| required_str(info, "endCursor"))
1922 .transpose()?;
1923 board = page.clone();
1924 match next {
1925 Some(next) => {
1926 validate_cursor_progress(after.as_deref(), next)?;
1927 after = Some(next.to_owned());
1928 }
1929 None => break,
1930 }
1931 }
1932 Ok(Board {
1933 id: required_str(&board, "id")?.to_owned(),
1934 fields: board.get("fields").cloned().unwrap_or(Value::Null),
1935 items,
1936 })
1937 }
1938
1939 /// The items this source has created, for completing a board read that is behind.
1940 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
1941 self.created.lock().map_err(|_| SourceError::Unavailable {
1942 message: "this source's record of what it created in this run was left \
1943 inconsistent by an earlier failure; next: run the command again"
1944 .into(),
1945 })
1946 }
1947
1948 /// One board item as this source reports it, or `None` for content it ignores.
1949 ///
1950 /// A pull request is neither a project nor a task — it is somebody's change, not a
1951 /// unit of plan — and an item whose content the token cannot see has nothing to
1952 /// report at all.
1953 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
1954 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
1955 message: "GitHub project item is missing content".into(),
1956 })?;
1957 if content.is_null() {
1958 return Ok(None);
1959 }
1960 let content_kind = match required_str(content, "__typename")? {
1961 "Issue" => ContentKind::Issue,
1962 "DraftIssue" => ContentKind::DraftIssue,
1963 _ => return Ok(None),
1964 };
1965 let field_values = item
1966 .get("fieldValues")
1967 .ok_or_else(|| SourceError::Malformed {
1968 message: "GitHub project item is missing fieldValues".into(),
1969 })?;
1970 complete_connection(field_values, "project item field values", NESTED_PAGE_SIZE)?;
1971 let nodes = field_values
1972 .get("nodes")
1973 .and_then(Value::as_array)
1974 .ok_or_else(|| SourceError::Malformed {
1975 message: "GitHub project item fieldValues.nodes is not an array".into(),
1976 })?;
1977 if let Some(labels) = content.get("labels") {
1978 complete_connection(labels, "content labels", NESTED_PAGE_SIZE)?;
1979 }
1980 for field_value in nodes {
1981 if let Some(labels) = field_value.get("labels") {
1982 complete_connection(labels, "project item field labels", NESTED_PAGE_SIZE)?;
1983 }
1984 }
1985 let (body, slot) = metadata_body(optional_str(content, "body")?.map(str::to_owned))?;
1986 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
1987 .map(|id| NativeId(id.to_owned()));
1988 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
1989 // to read one from; it is a task, and never a project.
1990 let sub_issues = match content_kind {
1991 ContentKind::Issue => sub_issue_total(content)?,
1992 ContentKind::DraftIssue => 0,
1993 };
1994 let content_id = required_str(content, "id")?;
1995 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
1996 message: format!("GitHub issue {content_id}: {message}"),
1997 })?;
1998 let raw_title = required_str(content, "title")?;
1999 // The design prefix is read *first*, before either of the two rules that separate
2000 // a project from a task. A document is not work whatever sub-issues it has and
2001 // whatever marker it carries, and reading the prefix later would make a design
2002 // issue with none of either an empty project.
2003 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
2004 BoardKind::Document
2005 } else if parent.is_some() {
2006 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
2007 // under a project is that project's task even when it has sub-issues of its
2008 // own.
2009 BoardKind::Work(ItemKind::Task)
2010 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
2011 BoardKind::Work(ItemKind::Project)
2012 } else {
2013 BoardKind::Work(ItemKind::Task)
2014 };
2015 // The title a person wrote, which for a document is the one without the prefix —
2016 // the same way `content` above is the body without this source's metadata slot.
2017 let title = match kind {
2018 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
2019 BoardKind::Work(_) => raw_title.to_owned(),
2020 };
2021 let own_repository = content
2022 .pointer("/repository/nameWithOwner")
2023 .and_then(Value::as_str)
2024 .map(|origin| Repository::try_from(format!("github.com/{origin}")))
2025 .transpose()
2026 .map_err(|message| SourceError::Malformed { message })?;
2027 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
2028 Repository::from_metadata(&slot)
2029 .map_err(|message| SourceError::Malformed { message })?
2030 } else {
2031 own_repository.clone().into_iter().collect()
2032 };
2033 Ok(Some(Resolved {
2034 item_id: required_str(item, "id")?.to_owned(),
2035 id: NativeId(content_id.to_owned()),
2036 content_kind,
2037 kind,
2038 title,
2039 body: body.filter(|value| !value.is_empty()),
2040 status: self.status(item, content)?,
2041 labels: labels(content, nodes)?,
2042 parent,
2043 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
2044 url: optional_str(content, "url")?.map(str::to_owned),
2045 created_at: optional_time(content, "createdAt")?,
2046 updated_at: optional_time(content, "updatedAt")?,
2047 own_repository,
2048 repositories,
2049 slot,
2050 }))
2051 }
2052
2053 /// The status one board item reports.
2054 ///
2055 /// The closed state decides the category and the `Status` option decides the name, so
2056 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
2057 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
2058 /// duplicate is not finished work, and calling it done is a lie the next copy would
2059 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
2060 /// it is read permissively rather than refused — reads are faithful, and refusals
2061 /// belong on writes.
2062 fn status(&self, item: &Value, content: &Value) -> Result<Status, SourceError> {
2063 let nodes = item
2064 .pointer("/fieldValues/nodes")
2065 .and_then(Value::as_array)
2066 .expect("resolve validates fieldValues.nodes before mapping status");
2067 let option = nodes
2068 .iter()
2069 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
2070 .map(|value| required_str(value, "name"))
2071 .transpose()?;
2072 let state = optional_str(content, "state")?;
2073 if state == Some("CLOSED") {
2074 let category = match optional_str(content, "stateReason")? {
2075 None | Some("COMPLETED") => StatusCategory::Done,
2076 Some("NOT_PLANNED") => StatusCategory::Cancelled,
2077 Some(_) => StatusCategory::Unknown,
2078 };
2079 let fallback = match category {
2080 StatusCategory::Done => "Done",
2081 StatusCategory::Cancelled => "Cancelled",
2082 _ => "Closed",
2083 };
2084 return Ok(Status {
2085 category,
2086 name: option.unwrap_or(fallback).to_owned(),
2087 });
2088 }
2089 let name = option.unwrap_or("Open").to_owned();
2090 Ok(Status {
2091 category: self
2092 .statuses
2093 .category_of(&name)
2094 .unwrap_or(StatusCategory::Unknown),
2095 name,
2096 })
2097 }
2098
2099 /// The board Status option this write selects, or the refusal that says why not.
2100 ///
2101 /// For a column target the option is what the status *is*, so a board that has no such
2102 /// option is a refusal naming the status and the instance. For a closed target the
2103 /// issue's own state carries the category, and the option carries only the name a
2104 /// reader reports — so an option spelled the way this status is spelled is selected
2105 /// when the board has one, and nothing is refused when it does not.
2106 fn column_for(
2107 &self,
2108 board: &Board,
2109 status: &Status,
2110 target: &StatusTarget,
2111 ) -> Result<Option<(String, String)>, SourceError> {
2112 let (wanted, required) = match target {
2113 StatusTarget::Column(wanted) => (wanted.as_str(), true),
2114 StatusTarget::Closed(_) => (status.name.as_str(), false),
2115 StatusTarget::Disabled => return Ok(None),
2116 };
2117 let missing = |detail: &str| SourceError::Refused {
2118 message: format!(
2119 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
2120 category_name(status.category),
2121 self.name,
2122 category_name(status.category)
2123 ),
2124 };
2125 let Some(field) = Board::field(&board.fields, "Status")? else {
2126 return if required {
2127 Err(missing("this board has no Status field"))
2128 } else {
2129 Ok(None)
2130 };
2131 };
2132 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
2133 return if required {
2134 Err(missing(
2135 "this board's Status field is not a single-select field",
2136 ))
2137 } else {
2138 Ok(None)
2139 };
2140 }
2141 let option = field
2142 .get("options")
2143 .and_then(Value::as_array)
2144 .and_then(|options| {
2145 options.iter().find(|option| {
2146 option
2147 .get("name")
2148 .and_then(Value::as_str)
2149 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
2150 })
2151 });
2152 match option {
2153 None if required => Err(missing("this board does not have it")),
2154 None => Ok(None),
2155 Some(option) => Ok(Some((
2156 required_str(field, "id")?.to_owned(),
2157 required_str(option, "id")?.to_owned(),
2158 ))),
2159 }
2160 }
2161
2162 /// This instance's target for a category, refusing one it has disabled.
2163 ///
2164 /// Nothing here mutates the board's option set to make room for a status. GitHub
2165 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
2166 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
2167 /// field and every item's status.
2168 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
2169 let target = self.statuses.target(category).clone();
2170 if target != StatusTarget::Disabled {
2171 return Ok(target);
2172 }
2173 Err(SourceError::Refused {
2174 message: if category == StatusCategory::Draft {
2175 format!(
2176 "status draft is disabled for source {}: draft is incompatible with this \
2177 integration because GitHub draft issues cannot have sub-issues, and this \
2178 source stores a project's tasks as its issue's sub-issues",
2179 self.name
2180 )
2181 } else {
2182 format!(
2183 "status {} is disabled for source {}; set status_mapping.{} of this source \
2184 to a board Status option name or to a closed state",
2185 category_name(category),
2186 self.name,
2187 category_name(category)
2188 )
2189 },
2190 })
2191 }
2192
2193 async fn set_item_field(
2194 &self,
2195 board_id: &str,
2196 item_id: &str,
2197 field_id: &str,
2198 value: Value,
2199 ) -> Result<(), SourceError> {
2200 let data = self
2201 .graphql(
2202 graphql::UPDATE_FIELD,
2203 json!({"input":{
2204 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
2205 }}),
2206 )
2207 .await?;
2208 let returned = data
2209 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
2210 .ok_or_else(|| SourceError::Malformed {
2211 message: "GitHub field update returned no project item".into(),
2212 })?;
2213 if required_str(returned, "id")? != item_id {
2214 return Err(SourceError::Malformed {
2215 message: "GitHub field update returned the wrong project item".into(),
2216 });
2217 }
2218 Ok(())
2219 }
2220
2221 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
2222 let mut after: Option<String> = None;
2223 let mut ids = Vec::new();
2224 loop {
2225 let data = self
2226 .graphql(
2227 graphql::ISSUE_DEPENDENCIES,
2228 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
2229 )
2230 .await?;
2231 let connection =
2232 data.pointer("/node/blockedBy")
2233 .ok_or_else(|| SourceError::Malformed {
2234 message: "GitHub dependency response has no blockedBy connection".into(),
2235 })?;
2236 ids.extend(
2237 connection
2238 .get("nodes")
2239 .and_then(Value::as_array)
2240 .ok_or_else(|| SourceError::Malformed {
2241 message: "GitHub dependency response nodes is not an array".into(),
2242 })?
2243 .iter()
2244 .map(|value| required_str(value, "id").map(str::to_owned))
2245 .collect::<Result<Vec<_>, _>>()?,
2246 );
2247 let next = next_cursor(connection)?;
2248 if let Some(next) = &next {
2249 validate_cursor_progress(after.as_deref(), &next.0)?;
2250 }
2251 after = next.map(|cursor| cursor.0);
2252 if after.is_none() {
2253 return Ok(ids);
2254 }
2255 }
2256 }
2257
2258 async fn dependencies(
2259 &self,
2260 id: &NativeId,
2261 near_kind: ItemKind,
2262 direction: Direction,
2263 page: &PageRequest,
2264 ) -> Result<Page<DependencyEdge>, SourceError> {
2265 validate_page(page)?;
2266 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
2267 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
2268 let recorded = recorded_offset(cursor, direction)?;
2269 // Asked for even in the recorded phase, whose page reads nothing from the
2270 // connection: `__typename` is what says whether this item has a native
2271 // relationship at all, and that is what decides which far ends the reserved key is
2272 // allowed to hold.
2273 let data = self
2274 .graphql(
2275 graphql::ISSUE_DEPENDENCIES,
2276 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
2277 "after":if recorded.is_some() {None} else {cursor}}),
2278 )
2279 .await?;
2280 let node =
2281 data.get("node")
2282 .filter(|v| !v.is_null())
2283 .ok_or_else(|| SourceError::Refused {
2284 message: format!(
2285 "GitHub item {} was not found or does not support dependencies",
2286 id.0
2287 ),
2288 })?;
2289 let connection_name = match direction {
2290 Direction::DependsOn => "blockedBy",
2291 Direction::DependedOnBy => "blocking",
2292 };
2293 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
2294 // named natively and the reserved key may hold any far end. An issue's connections
2295 // hold issues, and this source reads them at the near item's own level.
2296 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
2297 if let Some(offset) = recorded {
2298 return Ok(recorded_page(
2299 self.recorded_edges(id, near_kind, direction, natively_names)
2300 .await?,
2301 offset,
2302 limit,
2303 ));
2304 }
2305 if natively_names.is_none() {
2306 return Ok(recorded_page(
2307 self.recorded_edges(id, near_kind, direction, natively_names)
2308 .await?,
2309 0,
2310 limit,
2311 ));
2312 }
2313 let connection = node
2314 .get(connection_name)
2315 .ok_or_else(|| SourceError::Malformed {
2316 message: "GitHub dependency response is missing its connection".into(),
2317 })?;
2318 let nodes = connection
2319 .get("nodes")
2320 .and_then(Value::as_array)
2321 .ok_or_else(|| SourceError::Malformed {
2322 message: "GitHub dependency response nodes is not an array".into(),
2323 })?;
2324 // `from` depends on `to`, always. GitHub spells the same relationship from either
2325 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
2326 // it — so the near item is `from` in one direction and `to` in the other.
2327 let items = nodes
2328 .iter()
2329 .map(|value| {
2330 let related = NativeId(required_str(value, "id")?.into());
2331 let related_kind = related_kind(value)?;
2332 let (from, to) = match direction {
2333 Direction::DependsOn => (
2334 DependencyEndpoint::from_native(id.clone(), near_kind),
2335 DependencyEndpoint::from_native(related, related_kind),
2336 ),
2337 Direction::DependedOnBy => (
2338 DependencyEndpoint::from_native(related, related_kind),
2339 DependencyEndpoint::from_native(id.clone(), near_kind),
2340 ),
2341 };
2342 Ok(DependencyEdge {
2343 from,
2344 to,
2345 kind: DependencyKind::Blocks,
2346 })
2347 })
2348 .collect::<Result<Vec<_>, SourceError>>()?;
2349 let mut next = next_cursor(connection)?;
2350 if let Some(next) = &next {
2351 validate_cursor_progress(cursor, &next.0)?;
2352 }
2353 if next.is_none()
2354 && !self
2355 .recorded_edges(id, near_kind, direction, natively_names)
2356 .await?
2357 .is_empty()
2358 {
2359 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
2360 }
2361 Ok(Page { items, next })
2362 }
2363
2364 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
2365 /// a far end in another source has to live: no GitHub issue relationship can name one.
2366 ///
2367 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
2368 /// source never writes one down.
2369 ///
2370 /// The metadata lives in the item's own body slot, so reading it costs one board scan.
2371 /// That is why it happens once the native connection is spent rather than on every
2372 /// page.
2373 async fn recorded_edges(
2374 &self,
2375 id: &NativeId,
2376 near_kind: ItemKind,
2377 direction: Direction,
2378 natively_names: Option<ItemKind>,
2379 ) -> Result<Vec<DependencyEdge>, SourceError> {
2380 if direction != Direction::DependsOn {
2381 return Ok(Vec::new());
2382 }
2383 let Some(item) = self
2384 .board()
2385 .await?
2386 .items
2387 .into_iter()
2388 .find(|item| item.id == *id)
2389 else {
2390 return Ok(Vec::new());
2391 };
2392 DependencyEdge::recorded(&item.slot, id, near_kind, &self.name, natively_names)
2393 .map_err(|message| SourceError::Malformed { message })
2394 }
2395
2396 /// The configured repository's node id, or the refusal naming the field it needs.
2397 ///
2398 /// Resolved once per command; see [`Self::repository_cache`].
2399 async fn repository_id(&self) -> Result<String, SourceError> {
2400 if let Some(id) = self.repository_cache()?.clone() {
2401 return Ok(id);
2402 }
2403 let repository = self
2404 .repository
2405 .as_ref()
2406 .ok_or_else(|| SourceError::Refused {
2407 message: format!(
2408 "source {} has no repository configured, and a GitHub Projects board has no \
2409 repository of its own to create an issue in; set repository: owner/name on \
2410 this source",
2411 self.name
2412 ),
2413 })?;
2414 let data = self
2415 .graphql(
2416 graphql::REPOSITORY,
2417 json!({"owner":repository.owner,"name":repository.name}),
2418 )
2419 .await?;
2420 let node = data
2421 .get("repository")
2422 .filter(|value| !value.is_null())
2423 .ok_or_else(|| SourceError::Refused {
2424 message: format!(
2425 "GitHub repository {}/{} was not found or is not visible to the token",
2426 repository.owner, repository.name
2427 ),
2428 })?;
2429 let id = required_str(node, "id")?.to_owned();
2430 *self.repository_cache()? = Some(id.clone());
2431 Ok(id)
2432 }
2433
2434 /// This process's own record of the destination repository's node id.
2435 fn repository_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<String>>, SourceError> {
2436 self.repository_cache
2437 .lock()
2438 .map_err(|_| SourceError::Unavailable {
2439 message: "this source's record of the destination repository was left \
2440 inconsistent by an earlier failure; next: run the command again"
2441 .into(),
2442 })
2443 }
2444
2445 /// Create or update one board item, whichever kind it is.
2446 async fn write_item(
2447 &self,
2448 incoming: &Incoming<'_>,
2449 target: Option<&NativeId>,
2450 depends_on: &[DependencyEdge],
2451 ) -> Result<NativeId, SourceError> {
2452 // Refused before anything is read or written: a task or a project titled the way
2453 // this board spells a document would land as an issue this same source reads back
2454 // as a document, so the field this destination cannot carry is named rather than
2455 // written and silently reclassified.
2456 if let Written::Work(kind, _) = incoming.written
2457 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
2458 {
2459 return Err(SourceError::Refused {
2460 message: format!(
2461 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
2462 spells a document, so it would read back as one rather than as a {}; \
2463 retitle it, or copy it as a document",
2464 kind.marker(),
2465 self.name,
2466 kind.marker()
2467 ),
2468 });
2469 }
2470 let board = self.board().await?;
2471 let status_target = incoming
2472 .written
2473 .status()
2474 .map(|status| self.resolved_target(status.category))
2475 .transpose()?;
2476 let column = match (incoming.written.status(), status_target.as_ref()) {
2477 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
2478 _ => None,
2479 };
2480 let existing = target
2481 .map(|target| {
2482 board
2483 .items
2484 .iter()
2485 .find(|item| item.id == *target)
2486 .ok_or_else(|| SourceError::Refused {
2487 message: format!("GitHub destination item {} was not found", target.0),
2488 })
2489 })
2490 .transpose()?;
2491 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
2492 if content_kind == ContentKind::DraftIssue {
2493 if let (Some(StatusTarget::Closed(_)), Some(status)) =
2494 (status_target.as_ref(), incoming.written.status())
2495 {
2496 return Err(SourceError::Refused {
2497 message: format!(
2498 "status {} of source {} closes the item's issue, and GitHub draft items \
2499 have no open or closed state",
2500 category_name(status.category),
2501 self.name
2502 ),
2503 });
2504 }
2505 if incoming.parent.is_some() {
2506 return Err(SourceError::Refused {
2507 message: "GitHub draft items cannot be a project's sub-issue".into(),
2508 });
2509 }
2510 }
2511 match existing {
2512 Some(item) if content_kind == ContentKind::Issue => {
2513 if item.labels != incoming.labels {
2514 return Err(SourceError::Refused {
2515 message: "GitHub issue labels differ from the labels being written".into(),
2516 });
2517 }
2518 }
2519 _ => {
2520 if !incoming.labels.is_empty() {
2521 return Err(SourceError::Refused {
2522 message: "GitHub items created by this destination carry no labels".into(),
2523 });
2524 }
2525 }
2526 }
2527
2528 let own_repository = match existing {
2529 Some(item) => item.own_repository.clone(),
2530 None => self
2531 .repository
2532 .as_ref()
2533 .map(|repository| Repository::try_from(repository.origin()))
2534 .transpose()
2535 .map_err(|message| SourceError::Config { message })?,
2536 };
2537 let (native, fallback) = self
2538 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
2539 .await?;
2540 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
2541 let body = compose_body(incoming.content, &slot)?;
2542 // Read before anything is created, for the reason the field below is: a value
2543 // this destination cannot store has to refuse, and refusing after `createIssue`
2544 // would leave an issue behind that nothing asked for. The engine writes a
2545 // qualified id here; a caller handing this key anything else is told so rather
2546 // than having it silently stored as no origin at all.
2547 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
2548 let origin = match incoming.metadata.get(ORIGIN_KEY) {
2549 None => "",
2550 Some(Value::String(origin)) => origin.as_str(),
2551 Some(other) => {
2552 return Err(SourceError::Refused {
2553 message: format!(
2554 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
2555 is {other}"
2556 ),
2557 });
2558 }
2559 };
2560 // Resolved before anything is created: a board that cannot carry the copy origin
2561 // has to refuse the write, and refusing it after `createIssue` would leave an
2562 // issue behind that nothing asked for.
2563 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
2564 Some(field) => {
2565 if required_str(field, "__typename")? != "ProjectV2Field" {
2566 return Err(SourceError::Refused {
2567 message: format!(
2568 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
2569 ),
2570 });
2571 }
2572 Some(required_str(field, "id")?.to_owned())
2573 }
2574 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
2575 return Err(SourceError::Refused {
2576 message: format!(
2577 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
2578 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
2579 the board"
2580 ),
2581 });
2582 }
2583 None => None,
2584 };
2585
2586 let (content_id, item_id, url) = match existing {
2587 Some(item) => {
2588 self.update_existing(item, incoming, &body, status_target.as_ref())
2589 .await?;
2590 (item.id.clone(), item.item_id.clone(), item.url.clone())
2591 }
2592 None => {
2593 self.create_and_file_issue(&board, incoming, &body, status_target.as_ref())
2594 .await?
2595 }
2596 };
2597
2598 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
2599 // then each board field, the parent and the dependencies — and GitHub can fail at
2600 // any of them. Everything this source can refuse *before* the first of those is
2601 // already checked above, so what is left is GitHub itself failing part way. When it
2602 // does over an item this call created, the issue is taken back: a write that
2603 // refused must not leave an item behind that nobody asked for, and one that does
2604 // makes the retry create a second.
2605 let landed = self
2606 .finish_write(
2607 &board,
2608 incoming,
2609 &content_id,
2610 &item_id,
2611 content_kind,
2612 existing,
2613 origin_field.as_deref(),
2614 origin,
2615 column,
2616 &native,
2617 )
2618 .await;
2619 if let Err(error) = landed {
2620 if existing.is_none() {
2621 // Best effort, and the write's own failure is what the caller is told: a
2622 // refusal naming the tidy-up would hide why the write failed at all.
2623 let _ = self.delete_issue(&content_id).await;
2624 }
2625 return Err(error);
2626 }
2627
2628 // So the rest of this command reads what it just did rather than what the board
2629 // said before it. See `remember_written` for which half takes it.
2630 let remembered = Resolved {
2631 item_id,
2632 id: content_id.clone(),
2633 content_kind,
2634 kind: incoming.written.kind(),
2635 title: incoming.title.to_owned(),
2636 // The visible half of the body this write composed, split back off it the
2637 // way a read splits it — so what this record reports is what a read of the
2638 // same issue reports, rather than the person's text with the metadata slot
2639 // still on the end of it.
2640 body: metadata_body(body.clone())?.0,
2641 // A document has no status of its own; what it reads back as is whatever
2642 // the issue's own state says, which is what a re-read reports.
2643 status: incoming
2644 .written
2645 .status()
2646 .cloned()
2647 .unwrap_or_else(|| Status {
2648 category: StatusCategory::Unknown,
2649 name: "Open".to_owned(),
2650 }),
2651 labels: incoming.labels.to_vec(),
2652 parent: incoming.parent.cloned(),
2653 origin: (!origin.is_empty()).then(|| origin.to_owned()),
2654 // In the update path this is the item's own url, read off `existing` where the
2655 // tuple above was bound, so one expression serves both halves.
2656 url,
2657 created_at: existing.and_then(|item| item.created_at),
2658 updated_at: existing.and_then(|item| item.updated_at),
2659 own_repository,
2660 repositories: incoming.repositories.to_vec(),
2661 slot,
2662 };
2663 self.remember_written(remembered, existing.is_none())?;
2664 Ok(content_id)
2665 }
2666
2667 /// Everything a write does after the item exists: its board fields, its parent, and
2668 /// its dependencies.
2669 ///
2670 /// Split out of `write_item` so there is one place a failure past the point of no
2671 /// return is caught, rather than a tidy-up repeated at each `?` above.
2672 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
2673 // so there is one place a failure past the point of no return is caught, and its
2674 // arguments are exactly the values that tail already had in scope. Bundling them into a
2675 // struct would describe no concept — it would be "the arguments of this function" — and
2676 // would put the whole of `write_item`'s locals behind one more indirection.
2677 #[allow(clippy::too_many_arguments)]
2678 async fn finish_write(
2679 &self,
2680 board: &Board,
2681 incoming: &Incoming<'_>,
2682 content_id: &NativeId,
2683 item_id: &str,
2684 content_kind: ContentKind,
2685 existing: Option<&Resolved>,
2686 origin_field: Option<&str>,
2687 origin: &str,
2688 column: Option<(String, String)>,
2689 native: &[String],
2690 ) -> Result<(), SourceError> {
2691 if let Some(field_id) = origin_field {
2692 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
2693 .await?;
2694 }
2695
2696 if let Some((field_id, option_id)) = column {
2697 self.set_item_field(
2698 &board.id,
2699 item_id,
2700 &field_id,
2701 json!({"singleSelectOptionId":option_id}),
2702 )
2703 .await?;
2704 }
2705
2706 if content_kind == ContentKind::Issue {
2707 self.reparent(
2708 existing.and_then(|item| item.parent.clone()),
2709 content_id,
2710 incoming.parent,
2711 )
2712 .await?;
2713 // A document takes part in no dependency graph, so writing one neither reads
2714 // nor changes the issue's own `blockedBy` relationships. Reconciling them
2715 // against the empty list a document write carries would *delete* whatever
2716 // relationships a person had made on that issue, which is a write nobody
2717 // asked for.
2718 if incoming.written.kind() != BoardKind::Document {
2719 self.reconcile_blocked_by(content_id, native).await?;
2720 }
2721 }
2722 Ok(())
2723 }
2724
2725 /// Delete one issue, which takes its board item with it.
2726 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
2727 let data = self
2728 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
2729 .await?;
2730 data.pointer("/deleteIssue/repository")
2731 .filter(|value| !value.is_null())
2732 .ok_or_else(|| SourceError::Malformed {
2733 message: "GitHub issue deletion returned no repository".into(),
2734 })?;
2735 self.forget(id)?;
2736 Ok(())
2737 }
2738
2739 /// Remove one item this copy created, so a copy that could not finish leaves the board
2740 /// as it found it.
2741 ///
2742 /// Deleting the issue takes its board item with it, so there is no second mutation to
2743 /// keep in step. An id the board does not hold is not an error: the item is already
2744 /// gone, which is the state this asks for.
2745 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
2746 let board = self.board().await?;
2747 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
2748 return Ok(());
2749 };
2750 if item.content_kind == ContentKind::DraftIssue {
2751 return Err(SourceError::Refused {
2752 message: format!(
2753 "GitHub item {} is a draft, and this source removes an item by deleting \
2754 its issue; next: remove it from the board by hand",
2755 id.0
2756 ),
2757 });
2758 }
2759 let data = self
2760 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
2761 .await?;
2762 data.pointer("/deleteIssue/repository")
2763 .filter(|value| !value.is_null())
2764 .ok_or_else(|| SourceError::Malformed {
2765 message: "GitHub issue deletion returned no repository".into(),
2766 })?;
2767 self.forget(id)?;
2768 Ok(())
2769 }
2770
2771 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
2772 async fn partition_edges(
2773 &self,
2774 board: &Board,
2775 near_kind: BoardKind,
2776 near_content: ContentKind,
2777 depends_on: &[DependencyEdge],
2778 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
2779 let mut native = Vec::new();
2780 let mut fallback = Vec::new();
2781 for edge in depends_on {
2782 let same_source = edge
2783 .to
2784 .source()
2785 .is_none_or(|source| source == self.name.as_str());
2786 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
2787 // `DependencyEndpoint::source` both read it that way — and a native id may hold
2788 // colons of its own, so the far end is everything after that one separator.
2789 // Splitting at the last would truncate `work:urn:task:7` to `7`.
2790 let far_id = if edge.to.is_qualified() {
2791 edge.to
2792 .id()
2793 .split_once(':')
2794 .map_or(edge.to.id(), |(_, native)| native)
2795 } else {
2796 edge.to.id()
2797 };
2798 let far = if same_source {
2799 Some(
2800 board
2801 .items
2802 .iter()
2803 .find(|item| item.id.0 == far_id)
2804 .ok_or_else(|| SourceError::Refused {
2805 message: format!("GitHub dependency item {far_id} was not found"),
2806 })?,
2807 )
2808 } else {
2809 None
2810 };
2811 // The caller says which kind the far end is, and this board holds the far end
2812 // itself, so a disagreement is settled here rather than stored: recorded, the
2813 // wrong kind would read back as a cross-level edge that never existed; written
2814 // natively, it would name a relationship of a different level than the caller
2815 // asked for.
2816 //
2817 // A far end this board holds as a *document* fails the same comparison and is
2818 // refused by the same sentence: `ItemKind` has no document variant because
2819 // nothing may point at one, so no caller can name it correctly and the refusal
2820 // is the only honest answer.
2821 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
2822 return Err(SourceError::Refused {
2823 message: format!(
2824 "GitHub dependency item {far_id} is a {} of this board, and this item \
2825 names it as a {}; record the kind it is",
2826 disagreeing.kind.describes(),
2827 edge.to.kind.marker()
2828 ),
2829 });
2830 }
2831 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
2832 // however the far end is spelled — and one classified native here would be
2833 // written nowhere at all, because a draft's native reconciliation never runs.
2834 let native_here = near_content == ContentKind::Issue
2835 && far.is_some_and(|far| {
2836 far.content_kind == ContentKind::Issue
2837 && BoardKind::Work(edge.to.kind) == near_kind
2838 });
2839 if native_here {
2840 native.push(far_id.to_owned());
2841 } else {
2842 fallback.push(edge.clone());
2843 }
2844 }
2845 Ok((native, fallback))
2846 }
2847
2848 async fn update_existing(
2849 &self,
2850 item: &Resolved,
2851 incoming: &Incoming<'_>,
2852 body: &Option<String>,
2853 status_target: Option<&StatusTarget>,
2854 ) -> Result<(), SourceError> {
2855 let title = incoming.written_title();
2856 let (operation, input, pointer) = match item.content_kind {
2857 ContentKind::DraftIssue => (
2858 graphql::UPDATE_DRAFT,
2859 json!({"draftIssueId":item.id.0,"title":title,"body":body}),
2860 "/updateProjectV2DraftIssue/draftIssue",
2861 ),
2862 ContentKind::Issue => (
2863 graphql::UPDATE_ISSUE,
2864 json!({"id":item.id.0,"title":title,"body":body,
2865 "stateInput":state_input(status_target)}),
2866 "/updateIssue/issue",
2867 ),
2868 };
2869 let data = self.graphql(operation, json!({"input":input})).await?;
2870 let returned = data
2871 .pointer(pointer)
2872 .ok_or_else(|| SourceError::Malformed {
2873 message: "GitHub item update returned no item".into(),
2874 })?;
2875 if required_str(returned, "id")? != item.id.0 {
2876 return Err(SourceError::Malformed {
2877 message: "GitHub item update returned the wrong item".into(),
2878 });
2879 }
2880 Ok(())
2881 }
2882
2883 /// Creates one issue, files it on the board, and closes it when the status says so.
2884 ///
2885 /// Three calls rather than one: `createIssue` needs a repository and answers with an
2886 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
2887 /// closed status is a state of the issue rather than a field of the board item.
2888 /// Creates the issue, files it on the board, and reports what a read of it would say:
2889 /// its content id, its board item id, and the web address GitHub gave it.
2890 ///
2891 /// The address comes back here because this is the only place it is known before
2892 /// GitHub's own board read catches up — an item this run created answers the reads
2893 /// that follow it out of the record below, and one remembered without its address
2894 /// would report no location for the rest of the run.
2895 async fn create_and_file_issue(
2896 &self,
2897 board: &Board,
2898 incoming: &Incoming<'_>,
2899 body: &Option<String>,
2900 status_target: Option<&StatusTarget>,
2901 ) -> Result<(NativeId, String, Option<String>), SourceError> {
2902 let repository_id = self.repository_id().await?;
2903 let data = self
2904 .graphql(
2905 graphql::CREATE_ISSUE,
2906 json!({"input":{
2907 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
2908 }}),
2909 )
2910 .await?;
2911 let created = data
2912 .pointer("/createIssue/issue")
2913 .filter(|value| !value.is_null())
2914 .ok_or_else(|| SourceError::Malformed {
2915 message: "GitHub issue creation returned no issue".into(),
2916 })?;
2917 let content_id = NativeId(required_str(created, "id")?.to_owned());
2918 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
2919 // a response without it is not worth failing a landed write over — the item simply
2920 // reports no location until the board read catches up, which is what it did before.
2921 let url = optional_str(created, "url")?.map(str::to_owned);
2922 // The issue exists from here on, so a failure filing it on the board takes it
2923 // back: an issue in the repository that is on no board is an item nobody asked for
2924 // and nothing here would find again.
2925 let added = match self
2926 .graphql(
2927 graphql::ADD_TO_BOARD,
2928 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
2929 )
2930 .await
2931 {
2932 Ok(added) => added,
2933 Err(error) => {
2934 let _ = self.delete_issue(&content_id).await;
2935 return Err(error);
2936 }
2937 };
2938 let item = added
2939 .pointer("/addProjectV2ItemById/item")
2940 .filter(|value| !value.is_null())
2941 .ok_or_else(|| SourceError::Malformed {
2942 message: "GitHub board addition returned no project item".into(),
2943 })?;
2944 if let Some(StatusTarget::Closed(_)) = status_target {
2945 let closed = self
2946 .graphql(
2947 graphql::UPDATE_ISSUE,
2948 json!({"input":{"id":content_id.0,"stateInput":state_input(status_target)}}),
2949 )
2950 .await?;
2951 let returned =
2952 closed
2953 .pointer("/updateIssue/issue")
2954 .ok_or_else(|| SourceError::Malformed {
2955 message: "GitHub item update returned no item".into(),
2956 })?;
2957 if required_str(returned, "id")? != content_id.0 {
2958 return Err(SourceError::Malformed {
2959 message: "GitHub item update returned the wrong item".into(),
2960 });
2961 }
2962 }
2963 Ok((content_id, required_str(item, "id")?.to_owned(), url))
2964 }
2965
2966 /// Move one issue under the project it now belongs to, or out of the one it left.
2967 async fn reparent(
2968 &self,
2969 held: Option<NativeId>,
2970 child: &NativeId,
2971 wanted: Option<&NativeId>,
2972 ) -> Result<(), SourceError> {
2973 if held.as_ref() == wanted {
2974 return Ok(());
2975 }
2976 if let Some(held) = &held {
2977 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
2978 .await?;
2979 }
2980 if let Some(wanted) = wanted {
2981 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
2982 .await?;
2983 }
2984 Ok(())
2985 }
2986
2987 async fn sub_issue(
2988 &self,
2989 operation: &str,
2990 parent: &NativeId,
2991 child: &NativeId,
2992 root: &str,
2993 ) -> Result<(), SourceError> {
2994 let data = self
2995 .graphql(
2996 operation,
2997 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
2998 )
2999 .await?;
3000 let issue =
3001 data.pointer(&format!("/{root}/issue"))
3002 .ok_or_else(|| SourceError::Malformed {
3003 message: "GitHub sub-issue update returned no issue".into(),
3004 })?;
3005 let sub =
3006 data.pointer(&format!("/{root}/subIssue"))
3007 .ok_or_else(|| SourceError::Malformed {
3008 message: "GitHub sub-issue update returned no sub-issue".into(),
3009 })?;
3010 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
3011 return Err(SourceError::Malformed {
3012 message: "GitHub sub-issue update returned the wrong issues".into(),
3013 });
3014 }
3015 Ok(())
3016 }
3017
3018 async fn reconcile_blocked_by(
3019 &self,
3020 content_id: &NativeId,
3021 native: &[String],
3022 ) -> Result<(), SourceError> {
3023 let current = self.native_dependency_ids(content_id).await?;
3024 for (operation, far_id) in current
3025 .iter()
3026 .filter(|id| !native.contains(id))
3027 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
3028 .chain(
3029 native
3030 .iter()
3031 .filter(|id| !current.contains(id))
3032 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
3033 )
3034 {
3035 let data = self
3036 .graphql(
3037 operation,
3038 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
3039 )
3040 .await?;
3041 let root = if operation == graphql::ADD_BLOCKED_BY {
3042 "addBlockedBy"
3043 } else {
3044 "removeBlockedBy"
3045 };
3046 let issue =
3047 data.pointer(&format!("/{root}/issue"))
3048 .ok_or_else(|| SourceError::Malformed {
3049 message: "GitHub dependency update returned no issue".into(),
3050 })?;
3051 let blocker = data
3052 .pointer(&format!("/{root}/blockingIssue"))
3053 .ok_or_else(|| SourceError::Malformed {
3054 message: "GitHub dependency update returned no blocking issue".into(),
3055 })?;
3056 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
3057 {
3058 return Err(SourceError::Malformed {
3059 message: "GitHub dependency update returned the wrong issues".into(),
3060 });
3061 }
3062 }
3063 Ok(())
3064 }
3065}
3066
3067/// What resolving one node id reached; see [`GitHubProjectsSource::reach`].
3068enum Reached {
3069 /// An issue this board holds, resolved into everything this source reports about it.
3070 Held(Box<Resolved>),
3071 /// Nothing this board holds: no such node, or a node on some other board.
3072 Nothing,
3073 /// A board draft, which exists only inside the board's own item connection.
3074 Draft,
3075}
3076
3077/// What GitHub says when a string is not a node id it can resolve.
3078///
3079/// Matched because it is the ordinary answer to a project selector naming a project by its
3080/// *name*, and reporting that as a failure would make naming one impossible. It is read
3081/// off the refusal GitHub sent, never guessed from the shape of the string: this source
3082/// does not define the syntax of a GitHub node id and would be wrong about it.
3083const UNRESOLVABLE_NODE: &str = "could not resolve to a node";
3084
3085/// Whether this refusal is GitHub saying the id names no node at all.
3086fn unresolvable_node(error: &SourceError) -> bool {
3087 matches!(error, SourceError::Refused { message }
3088 if message.to_ascii_lowercase().contains(UNRESOLVABLE_NODE))
3089}
3090
3091/// One project name, as a search qualifier which filters on it at the server.
3092///
3093/// Quoted so the whole title is one phrase rather than a bag of words, with the two
3094/// characters GitHub's own quoting grammar gives a meaning inside a quoted phrase escaped
3095/// the way it documents. A title matched here is still compared for equality afterwards:
3096/// the qualifier narrows what the server sends, and this source decides what it names.
3097fn title_qualifier(name: &str) -> String {
3098 let escaped = name.replace('\\', "\\\\").replace('"', "\\\"");
3099 format!("in:title \"{escaped}\"")
3100}
3101
3102/// The board, and every item on it this source reports.
3103#[derive(Clone)]
3104struct Board {
3105 id: String,
3106 fields: Value,
3107 items: Vec<Resolved>,
3108}
3109
3110impl Board {
3111 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
3112 complete_connection(fields, "project fields", NESTED_PAGE_SIZE)?;
3113 let nodes = fields
3114 .get("nodes")
3115 .and_then(Value::as_array)
3116 .ok_or_else(|| SourceError::Malformed {
3117 message: "GitHub project fields.nodes is not an array".into(),
3118 })?;
3119 Ok(nodes
3120 .iter()
3121 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
3122 }
3123}
3124
3125/// One board item, resolved into everything this source reports about it.
3126#[derive(Clone)]
3127struct Resolved {
3128 item_id: String,
3129 id: NativeId,
3130 content_kind: ContentKind,
3131 kind: BoardKind,
3132 title: String,
3133 body: Option<String>,
3134 status: Status,
3135 labels: Vec<Label>,
3136 parent: Option<NativeId>,
3137 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
3138 origin: Option<String>,
3139 url: Option<String>,
3140 created_at: Option<DateTime<Utc>>,
3141 updated_at: Option<DateTime<Utc>>,
3142 own_repository: Option<Repository>,
3143 repositories: Vec<Repository>,
3144 slot: BTreeMap<String, Value>,
3145}
3146
3147impl Resolved {
3148 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
3149 /// in a field of its own, and none of the three keys that are only an encoding.
3150 fn metadata(&self) -> BTreeMap<String, Value> {
3151 let mut metadata = self.slot.clone();
3152 metadata.remove(Repository::METADATA_KEY);
3153 metadata.remove(DependencyEdge::RECORDED_KEY);
3154 metadata.remove(ItemKind::METADATA_KEY);
3155 if let Some(origin) = &self.origin {
3156 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
3157 }
3158 metadata
3159 }
3160
3161 /// Where this item is, as a link a reader can open.
3162 ///
3163 /// A board is a hosted place and every issue on it has a web address, so that address
3164 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
3165 /// of place it is, so a reader knows to open it rather than to read a file out. It
3166 /// does not replace or derive from `url`: the field goes on reporting exactly what it
3167 /// reported before, and this says what that address *is*.
3168 ///
3169 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
3170 /// rather than a third variant, which is the contract's "the source did not say". An
3171 /// issue this run created is not one of those: its address comes back from the
3172 /// creating mutation, so it is somewhere a reader can open from the moment it exists
3173 /// rather than from whenever the board read catches up.
3174 fn location(&self) -> Option<Location> {
3175 self.url.clone().map(Location::Url)
3176 }
3177
3178 fn task(&self) -> Task {
3179 Task {
3180 id: self.id.clone(),
3181 title: self.title.clone(),
3182 content: self.body.clone(),
3183 status: self.status.clone(),
3184 labels: self.labels.clone(),
3185 project: self.parent.clone(),
3186 url: self.url.clone(),
3187 location: self.location(),
3188 created_at: self.created_at,
3189 updated_at: self.updated_at,
3190 metadata: self.metadata(),
3191 repositories: self.repositories.clone(),
3192 }
3193 }
3194
3195 fn project(&self) -> Project {
3196 Project {
3197 id: self.id.clone(),
3198 title: self.title.clone(),
3199 content: self.body.clone(),
3200 status: self.status.clone(),
3201 labels: self.labels.clone(),
3202 url: self.url.clone(),
3203 location: self.location(),
3204 created_at: self.created_at,
3205 updated_at: self.updated_at,
3206 metadata: self.metadata(),
3207 repositories: self.repositories.clone(),
3208 }
3209 }
3210
3211 /// The same issue as a document: the project it is filed under, and no status and no
3212 /// dependencies, because a document is not work.
3213 fn document(&self) -> Document {
3214 Document {
3215 id: self.id.clone(),
3216 title: self.title.clone(),
3217 content: self.body.clone(),
3218 project: self.parent.clone(),
3219 labels: self.labels.clone(),
3220 url: self.url.clone(),
3221 location: self.location(),
3222 created_at: self.created_at,
3223 updated_at: self.updated_at,
3224 metadata: self.metadata(),
3225 repositories: self.repositories.clone(),
3226 }
3227 }
3228}
3229
3230/// What one write is, and the status that comes with being it.
3231///
3232/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
3233/// status and a task or a project always has one, so "a document carrying a status" and
3234/// "a task carrying none" are states a write cannot be in rather than states every use
3235/// site below has to defend against.
3236enum Written<'a> {
3237 /// A document, which is not work and so has no status at all.
3238 Document,
3239 /// A task or a project, and the status it is being written with.
3240 Work(ItemKind, &'a Status),
3241}
3242
3243impl Written<'_> {
3244 /// Which of the board's three kinds this write is.
3245 const fn kind(&self) -> BoardKind {
3246 match self {
3247 Self::Document => BoardKind::Document,
3248 Self::Work(kind, _) => BoardKind::Work(*kind),
3249 }
3250 }
3251
3252 /// The status this write carries. A document carries none, so a write of one says
3253 /// nothing about the issue's open or closed state and selects no board `Status`
3254 /// option.
3255 const fn status(&self) -> Option<&Status> {
3256 match self {
3257 Self::Document => None,
3258 Self::Work(_, status) => Some(status),
3259 }
3260 }
3261}
3262
3263/// The item being written, in the one shape all three write methods reach.
3264struct Incoming<'a> {
3265 written: Written<'a>,
3266 /// The title a person wrote. A document's goes onto the issue with
3267 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
3268 title: &'a str,
3269 content: Option<&'a str>,
3270 labels: &'a [Label],
3271 metadata: &'a BTreeMap<String, Value>,
3272 repositories: &'a [Repository],
3273 parent: Option<&'a NativeId>,
3274}
3275
3276impl Incoming<'_> {
3277 /// The title this write puts on the issue.
3278 fn written_title(&self) -> String {
3279 match self.written {
3280 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
3281 Written::Work(..) => self.title.to_owned(),
3282 }
3283 }
3284}
3285
3286#[derive(Clone, Copy, PartialEq, Eq)]
3287enum ContentKind {
3288 DraftIssue,
3289 Issue,
3290}
3291
3292/// What one board issue is: a document, or the work an [`ItemKind`] names.
3293///
3294/// A type of this source's own rather than an `ItemKind` with a third variant, because
3295/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
3296/// document — the contract keeps a document out of that enum deliberately. Holding the
3297/// board's three answers in one value is what makes every place that asks "which is this?"
3298/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
3299/// two thirds of the board.
3300#[derive(Clone, Copy, PartialEq, Eq)]
3301enum BoardKind {
3302 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
3303 Document,
3304 /// Every other issue, and every draft.
3305 Work(ItemKind),
3306}
3307
3308impl BoardKind {
3309 /// How a refusal names this kind to the person reading it.
3310 const fn describes(self) -> &'static str {
3311 match self {
3312 Self::Document => "document",
3313 Self::Work(kind) => kind.marker(),
3314 }
3315 }
3316}
3317
3318/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
3319///
3320/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
3321/// the shared cross-source journeys assert one answer to one question, so two sources
3322/// that disagree about what "carries the label bug" means fail them.
3323fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
3324 let holds = |name: &String| {
3325 labels
3326 .iter()
3327 .any(|label| label.name.eq_ignore_ascii_case(name))
3328 };
3329 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
3330 && filter.all_of.iter().all(holds)
3331 && !filter.none_of.iter().any(holds)
3332}
3333
3334/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
3335/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
3336fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
3337 statuses.is_empty() || statuses.contains(&category)
3338}
3339
3340/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
3341///
3342/// `content` is the item's own prose — the body with this source's trailing metadata
3343/// comment already taken off — so a search never matches an encoding the author of the
3344/// issue never wrote.
3345fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
3346 let terms = query.terms.to_lowercase();
3347 let in_title = title.to_lowercase().contains(&terms);
3348 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
3349 match query.fields {
3350 TextFields::Title => in_title,
3351 TextFields::Content => in_content,
3352 TextFields::TitleOrContent => in_title || in_content,
3353 }
3354}
3355
3356/// Whether `task` satisfies `query`, with `project` deciding the project predicate.
3357///
3358/// The project predicate is passed separately because a read narrowed to one project has
3359/// already answered it by asking *that project* for its own items — and re-applying it
3360/// there would compare the caller's selector, which may be a project's **name**, against
3361/// the id of the project that name resolved to, and keep nothing. Every other read passes
3362/// `query.project` and applies it here, which is what keeps `projects` a predicate this
3363/// source really does apply.
3364fn task_matches(task: &Task, query: &TaskQuery, project: &ProjectFilter) -> bool {
3365 labels_match(&task.labels, &query.labels)
3366 && status_matches(task.status.category, &query.statuses)
3367 && match project {
3368 ProjectFilter::Any => true,
3369 ProjectFilter::Orphans => task.project.is_none(),
3370 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
3371 }
3372 && query
3373 .text
3374 .as_ref()
3375 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
3376}
3377
3378fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
3379 labels_match(&project.labels, &query.labels)
3380 && status_matches(project.status.category, &query.statuses)
3381 && query
3382 .text
3383 .as_ref()
3384 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
3385}
3386
3387/// The same three predicates a task query carries, minus the status filter.
3388///
3389/// A document is not work, so it has no status for one to compare against and the query
3390/// type carries none. The project predicate is the same one — a design issue filed under a
3391/// project issue is in that project, and one filed under nothing is in none — so it is
3392/// spelled the same way here rather than answered differently.
3393fn document_matches(document: &Document, query: &DocumentQuery, project: &ProjectFilter) -> bool {
3394 labels_match(&document.labels, &query.labels)
3395 && match project {
3396 ProjectFilter::Any => true,
3397 ProjectFilter::Orphans => document.project.is_none(),
3398 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
3399 }
3400 && query
3401 .text
3402 .as_ref()
3403 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
3404}
3405
3406#[async_trait::async_trait]
3407impl TaskSource for GitHubProjectsSource {
3408 fn kind(&self) -> &'static str {
3409 KIND
3410 }
3411 fn capabilities(&self) -> Capabilities {
3412 Capabilities {
3413 projects: Support::Native,
3414 documents: Support::Native,
3415 orphan_tasks: Support::Native,
3416 filter_by_label: Support::Native,
3417 filter_by_status: Support::Native,
3418 search_title: Support::Native,
3419 search_content: Support::Native,
3420 task_dependencies: DependencySupport::BothDirections,
3421 project_dependencies: DependencySupport::BothDirections,
3422 max_page_size: MAX_PAGE_SIZE,
3423 }
3424 }
3425 async fn health(&self) -> Result<Health, SourceError> {
3426 let board = self.board_page(None, 1).await?;
3427 Ok(Health {
3428 reachable: true,
3429 detail: Some(format!(
3430 "reading GitHub project {}/{} ({})",
3431 self.owner,
3432 self.project_number,
3433 required_str(&board, "title")?
3434 )),
3435 })
3436 }
3437 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
3438 Ok(self
3439 .item_by_id(id)
3440 .await?
3441 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3442 .map(|item| item.task()))
3443 }
3444 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
3445 Ok(self
3446 .item_by_id(id)
3447 .await?
3448 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
3449 .map(|item| item.project()))
3450 }
3451 async fn query_tasks(
3452 &self,
3453 query: &TaskQuery,
3454 page: &PageRequest,
3455 ) -> Result<Page<Task>, SourceError> {
3456 validate_page(page)?;
3457 // A read narrowed to one project asks that project for its own tasks, so nothing
3458 // about it costs what the rest of the board holds. Every other task read is a
3459 // question about the whole board and is answered by reading it.
3460 let (held, membership) = match &query.project {
3461 ProjectFilter::Is(project) => (
3462 self.project_children(project).await?,
3463 // Answered by where these items came from; see `task_matches`.
3464 &ProjectFilter::Any,
3465 ),
3466 ProjectFilter::Any | ProjectFilter::Orphans => {
3467 (self.board().await?.items, &query.project)
3468 }
3469 };
3470 // Filtered before paged: a page of a filtered result is a page of the survivors,
3471 // never the survivors of a page.
3472 let tasks = held
3473 .iter()
3474 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
3475 .map(Resolved::task)
3476 .filter(|task| task_matches(task, query, membership))
3477 .collect();
3478 Ok(offset_page(
3479 tasks,
3480 numeric_cursor(page.cursor.as_ref())?,
3481 page.limit.min(MAX_PAGE_SIZE) as usize,
3482 ))
3483 }
3484 async fn query_projects(
3485 &self,
3486 query: &ProjectQuery,
3487 page: &PageRequest,
3488 ) -> Result<Page<Project>, SourceError> {
3489 validate_page(page)?;
3490 // The projects a board holds are found by an issue search scoped to that board,
3491 // never by walking the board's own item connection: what tells a project from a
3492 // task is the `parent` each issue carries, which costs nothing to read.
3493 let projects = self
3494 .board_issues()
3495 .await?
3496 .iter()
3497 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
3498 .map(Resolved::project)
3499 .filter(|project| project_matches(project, query))
3500 .collect();
3501 Ok(offset_page(
3502 projects,
3503 numeric_cursor(page.cursor.as_ref())?,
3504 page.limit.min(MAX_PAGE_SIZE) as usize,
3505 ))
3506 }
3507 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
3508 Ok(self
3509 .item_by_id(id)
3510 .await?
3511 .filter(|item| item.kind == BoardKind::Document)
3512 .map(|item| item.document()))
3513 }
3514 async fn query_documents(
3515 &self,
3516 query: &DocumentQuery,
3517 page: &PageRequest,
3518 ) -> Result<Page<Document>, SourceError> {
3519 validate_page(page)?;
3520 // Narrowed to one project, this is the same sub-issue read a task list scoped to
3521 // that project makes — a document filed under a project is a sub-issue of it too,
3522 // and which of them come back is the kind this caller asked for.
3523 let (held, membership) = match &query.project {
3524 ProjectFilter::Is(project) => (
3525 self.project_children(project).await?,
3526 // Answered by where these items came from; see `task_matches`.
3527 &ProjectFilter::Any,
3528 ),
3529 ProjectFilter::Any | ProjectFilter::Orphans => {
3530 (self.board().await?.items, &query.project)
3531 }
3532 };
3533 // Filtered before paged, exactly as a task read is: a page of a filtered result is
3534 // a page of the survivors, never the survivors of a page.
3535 let documents = held
3536 .iter()
3537 .filter(|item| item.kind == BoardKind::Document)
3538 .map(Resolved::document)
3539 .filter(|document| document_matches(document, query, membership))
3540 .collect();
3541 Ok(offset_page(
3542 documents,
3543 numeric_cursor(page.cursor.as_ref())?,
3544 page.limit.min(MAX_PAGE_SIZE) as usize,
3545 ))
3546 }
3547 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
3548 validate_page(page)?;
3549 let offset = numeric_cursor(page.cursor.as_ref())?;
3550 let mut labels = self
3551 .board()
3552 .await?
3553 .items
3554 .into_iter()
3555 .flat_map(|item| item.labels)
3556 .fold(Vec::new(), |mut all, label| {
3557 if !all.iter().any(|x: &Label| x.id == label.id) {
3558 all.push(label);
3559 }
3560 all
3561 });
3562 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
3563 Ok(offset_page(
3564 labels,
3565 offset,
3566 page.limit.min(MAX_PAGE_SIZE) as usize,
3567 ))
3568 }
3569 async fn task_dependencies(
3570 &self,
3571 id: &NativeId,
3572 direction: Direction,
3573 page: &PageRequest,
3574 ) -> Result<Page<DependencyEdge>, SourceError> {
3575 self.dependencies(id, ItemKind::Task, direction, page).await
3576 }
3577 async fn project_dependencies(
3578 &self,
3579 id: &NativeId,
3580 direction: Direction,
3581 page: &PageRequest,
3582 ) -> Result<Page<DependencyEdge>, SourceError> {
3583 self.dependencies(id, ItemKind::Project, direction, page)
3584 .await
3585 }
3586
3587 fn writes(&self) -> WriteSupport {
3588 WriteSupport::Supported
3589 }
3590
3591 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
3592 self.write_item(
3593 &Incoming {
3594 written: Written::Work(ItemKind::Task, &write.item.status),
3595 title: &write.item.title,
3596 content: write.item.content.as_deref(),
3597 labels: &write.item.labels,
3598 metadata: &write.item.metadata,
3599 repositories: &write.item.repositories,
3600 parent: write.item.project.as_ref(),
3601 },
3602 write.target.as_ref(),
3603 &write.depends_on,
3604 )
3605 .await
3606 }
3607
3608 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
3609 self.write_item(
3610 &Incoming {
3611 written: Written::Work(ItemKind::Project, &write.item.status),
3612 title: &write.item.title,
3613 content: write.item.content.as_deref(),
3614 labels: &write.item.labels,
3615 metadata: &write.item.metadata,
3616 repositories: &write.item.repositories,
3617 parent: None,
3618 },
3619 write.target.as_ref(),
3620 &write.depends_on,
3621 )
3622 .await
3623 }
3624
3625 /// Create or update one document, which is one issue titled the way this board spells
3626 /// a document.
3627 ///
3628 /// Everything else is exactly a task write: caller metadata goes to the same canonical
3629 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
3630 /// or a field this board cannot carry is refused by name rather than dropped, a target
3631 /// naming an issue this board does not hold is refused rather than created, and an
3632 /// issue this call created is taken back when the rest of the write fails.
3633 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
3634 // A document takes part in no dependency graph, so there is no far end to write
3635 // natively and none to record: a caller naming one is told so rather than having it
3636 // stored under the reserved key, where a later read would report an edge the
3637 // contract says cannot exist.
3638 if !write.depends_on.is_empty() {
3639 return Err(SourceError::Refused {
3640 message: format!(
3641 "this write names {} dependencies for a document, and a document takes \
3642 part in no dependency graph; next: put the dependency on the task or \
3643 project the document is about",
3644 write.depends_on.len()
3645 ),
3646 });
3647 }
3648 self.write_item(
3649 &Incoming {
3650 written: Written::Document,
3651 title: &write.item.title,
3652 content: write.item.content.as_deref(),
3653 labels: &write.item.labels,
3654 metadata: &write.item.metadata,
3655 repositories: &write.item.repositories,
3656 parent: write.item.project.as_ref(),
3657 },
3658 write.target.as_ref(),
3659 &[],
3660 )
3661 .await
3662 }
3663
3664 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
3665 self.delete_item(id).await
3666 }
3667
3668 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
3669 self.delete_item(id).await
3670 }
3671
3672 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
3673 self.delete_item(id).await
3674 }
3675}
3676
3677/// Where the recorded tail of a dependency walk resumes; see
3678/// [`GitHubProjectsSource::recorded_edges`].
3679const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
3680
3681/// The board text field this source keeps a copy's origin in.
3682///
3683/// Named after the key it holds, and held to that name by the guard below rather than by
3684/// a reader noticing.
3685const ORIGIN_FIELD: &str = "onetaskgraph.origin";
3686
3687/// The metadata key that field holds.
3688///
3689/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
3690/// constructs or interprets the qualified id it carries. This source names it only to
3691/// route it — a short, typed value belongs in a typed field rather than in the body slot
3692/// a caller's own prose shares.
3693///
3694/// Restated rather than imported, because no plugin crate may depend on the engine. What
3695/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
3696/// target in `check`: it reads the engine's own literal and fails naming the file and the
3697/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
3698/// that creates a second item every run instead of finding the one it wrote — and that is
3699/// too late to learn it.
3700const ORIGIN_KEY: &str = "onetaskgraph.origin";
3701
3702/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
3703///
3704/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
3705/// is derived from the far end, never written down on the near item — so only a forward
3706/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
3707/// it did not come from, and it is told so rather than answered with an empty page that
3708/// reads as a walk which ended.
3709fn recorded_offset(
3710 cursor: Option<&str>,
3711 direction: Direction,
3712) -> Result<Option<usize>, SourceError> {
3713 cursor
3714 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
3715 .map(|offset| {
3716 if direction != Direction::DependsOn {
3717 return Err(SourceError::Config {
3718 message: format!(
3719 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
3720 reverse dependency read never issues; resume it in the direction \
3721 that reported it"
3722 ),
3723 });
3724 }
3725 offset.parse().map_err(|_| SourceError::Config {
3726 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
3727 })
3728 })
3729 .transpose()
3730}
3731
3732fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
3733 let mut page = offset_page(edges, offset, limit.max(1));
3734 page.next = page
3735 .next
3736 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
3737 page
3738}
3739
3740/// The kind of one issue reached through a dependency connection.
3741///
3742/// The same questions the board scan asks, over the fields the dependency document
3743/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
3744/// then anything with sub-issues or the marker is a project.
3745///
3746/// # Errors
3747///
3748/// A far end this board holds as a document is refused rather than reported. The two
3749/// answers that are not refusals would both be wrong: reporting it as a task names an id
3750/// no task read of this source can find, and reporting it as a project names one no
3751/// project read can. There is no third value to return — `ItemKind` has no document
3752/// variant, because nothing may point at a document — so the relationship itself is what
3753/// the person is told about.
3754fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
3755 let id = required_str(value, "id")?;
3756 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
3757 return Err(SourceError::Refused {
3758 message: format!(
3759 "GitHub issue {id} is a document of this board — its title begins \
3760 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
3761 on by one; next: remove that issue's blocking relationship on this board"
3762 ),
3763 });
3764 }
3765 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
3766 if parent.is_some() {
3767 return Ok(ItemKind::Task);
3768 }
3769 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
3770 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
3771 message: format!("GitHub issue {id}: {message}"),
3772 })?;
3773 let sub_issues = sub_issue_total(value)?;
3774 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
3775 ItemKind::Project
3776 } else {
3777 ItemKind::Task
3778 })
3779}
3780
3781/// The `IssueStateUpdateInput` one status target asks for.
3782///
3783/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
3784/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
3785/// a currently-closed issue: without that the item would read back `Unknown` and a copy
3786/// would report a change forever. A document has no status at all, and asks for neither.
3787fn state_input(target: Option<&StatusTarget>) -> Value {
3788 match target {
3789 Some(StatusTarget::Closed(reason)) => {
3790 json!({"value":"CLOSED","stateReason":reason.reason()})
3791 }
3792 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
3793 // A document has no status, so a write of one says nothing about the issue's open
3794 // or closed state rather than forcing it open: `stateInput` is what carries that
3795 // instruction, and an explicit null asks for no change to it.
3796 None => Value::Null,
3797 }
3798}
3799
3800/// The metadata one write stores in the item's body slot.
3801///
3802/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
3803/// rather than carried: the kind marker so an empty project stays readable, the
3804/// repository list only when it is not exactly the issue's own repository, and the far
3805/// ends no relationship here can name.
3806fn slot_metadata(
3807 incoming: &Incoming<'_>,
3808 own_repository: Option<&Repository>,
3809 fallback: &[DependencyEdge],
3810) -> BTreeMap<String, Value> {
3811 let mut metadata = incoming.metadata.clone();
3812 metadata.remove(ORIGIN_KEY);
3813 match incoming.written.kind() {
3814 BoardKind::Work(kind) => metadata.insert(
3815 ItemKind::METADATA_KEY.to_owned(),
3816 Value::String(kind.marker().to_owned()),
3817 ),
3818 // A document is told by its title, so it carries no kind marker: that key names
3819 // what a dependency endpoint points at, and nothing may point at a document.
3820 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
3821 };
3822 let derivable = own_repository
3823 .map(|own| incoming.repositories == [own.clone()])
3824 .unwrap_or(incoming.repositories.is_empty());
3825 if derivable {
3826 metadata.remove(Repository::METADATA_KEY);
3827 } else {
3828 metadata.insert(
3829 Repository::METADATA_KEY.to_owned(),
3830 Value::Array(
3831 incoming
3832 .repositories
3833 .iter()
3834 .map(|repository| Value::String(repository.as_str().to_owned()))
3835 .collect(),
3836 ),
3837 );
3838 }
3839 if fallback.is_empty() {
3840 metadata.remove(DependencyEdge::RECORDED_KEY);
3841 } else {
3842 metadata.insert(
3843 DependencyEdge::RECORDED_KEY.to_owned(),
3844 Value::Array(
3845 fallback
3846 .iter()
3847 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
3848 .collect(),
3849 ),
3850 );
3851 }
3852 metadata
3853}
3854
3855fn labels(content: &Value, field_values: &[Value]) -> Result<Vec<Label>, SourceError> {
3856 let direct = optional_nodes(content.get("labels"), "content labels")?;
3857 let field = field_values
3858 .iter()
3859 .find_map(|value| value.get("labels"))
3860 .map(|labels| optional_nodes(Some(labels), "field labels"))
3861 .transpose()?
3862 .flatten();
3863 let labels = direct
3864 .into_iter()
3865 .flatten()
3866 .chain(field.into_iter().flatten())
3867 .map(|v| {
3868 Ok(Label {
3869 id: NativeId(required_str(v, "id")?.to_owned()),
3870 name: required_str(v, "name")?.to_owned(),
3871 color: optional_str(v, "color")?.map(str::to_owned),
3872 })
3873 })
3874 .collect::<Result<Vec<_>, SourceError>>()?
3875 .into_iter()
3876 .fold(Vec::new(), |mut labels, label| {
3877 if !labels.iter().any(|x: &Label| x.id == label.id) {
3878 labels.push(label);
3879 }
3880 labels
3881 });
3882 Ok(labels)
3883}
3884
3885fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
3886 let Some(node) = field_values
3887 .iter()
3888 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
3889 else {
3890 return Ok(None);
3891 };
3892 Ok(optional_str(node, "text")?.map(str::to_owned))
3893}
3894
3895fn valid_github_owner(owner: &str) -> bool {
3896 !owner.is_empty()
3897 && owner.len() <= 39
3898 && !owner.starts_with('-')
3899 && !owner.ends_with('-')
3900 && !owner.contains("--")
3901 && owner
3902 .bytes()
3903 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
3904}
3905
3906/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
3907/// neither of the two names a path segment already means.
3908fn valid_github_repository_name(name: &str) -> bool {
3909 !name.is_empty()
3910 && name.len() <= 100
3911 && name != "."
3912 && name != ".."
3913 && name
3914 .bytes()
3915 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
3916}
3917
3918fn valid_environment_name(name: &str) -> bool {
3919 let mut bytes = name.bytes();
3920 bytes
3921 .next()
3922 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
3923 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
3924}
3925
3926/// How many sub-issues one issue has.
3927///
3928/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
3929/// absent or non-integer one is a response this source cannot read — and reading it as
3930/// zero would classify a project as a task, which is exactly the mistake the marker
3931/// exists to keep from happening quietly.
3932fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
3933 let summary = issue
3934 .get("subIssuesSummary")
3935 .ok_or_else(|| SourceError::Malformed {
3936 message: "GitHub issue is missing subIssuesSummary".into(),
3937 })?;
3938 summary
3939 .get("total")
3940 .and_then(Value::as_u64)
3941 .ok_or_else(|| SourceError::Malformed {
3942 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
3943 })
3944}
3945
3946fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
3947 value
3948 .get(field)
3949 .and_then(Value::as_str)
3950 .ok_or_else(|| SourceError::Malformed {
3951 message: format!("GitHub response is missing string field {field}"),
3952 })
3953}
3954
3955/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
3956/// needs one — Linear spells them too, in its own description field.
3957///
3958/// Restated rather than shared, because a plugin crate depends on the contract crate and
3959/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
3960/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
3961/// source round-trips its own writes perfectly well under its own spelling.
3962const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
3963const METADATA_CLOSE: &str = "\n-->";
3964
3965/// The visible body and the metadata slot at the end of it.
3966///
3967/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
3968/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
3969/// own content and is left alone.
3970fn metadata_body(
3971 body: Option<String>,
3972) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
3973 let Some(body) = body else {
3974 return Ok((None, BTreeMap::new()));
3975 };
3976 let Some(start) = body.rfind(METADATA_OPEN) else {
3977 return Ok((Some(body), BTreeMap::new()));
3978 };
3979 let encoded_start = start + METADATA_OPEN.len();
3980 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
3981 return Err(SourceError::Malformed {
3982 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
3983 });
3984 };
3985 let encoded_end = encoded_start + relative_end;
3986 if !body[encoded_end + METADATA_CLOSE.len()..].trim().is_empty() {
3987 return Ok((Some(body), BTreeMap::new()));
3988 }
3989 let metadata = serde_json::from_str(&body[encoded_start..encoded_end]).map_err(|error| {
3990 SourceError::Malformed {
3991 message: format!(
3992 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
3993 ),
3994 }
3995 })?;
3996 let visible = body[..start].trim_end();
3997 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
3998}
3999
4000fn compose_body(
4001 content: Option<&str>,
4002 metadata: &BTreeMap<String, Value>,
4003) -> Result<Option<String>, SourceError> {
4004 let visible = content.unwrap_or_default();
4005 if metadata.is_empty() {
4006 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
4007 }
4008 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
4009 message: error.to_string(),
4010 })?;
4011 Ok(Some(if visible.is_empty() {
4012 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4013 } else {
4014 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
4015 }))
4016}
4017
4018fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
4019 value
4020 .get(field)
4021 .and_then(Value::as_bool)
4022 .ok_or_else(|| SourceError::Malformed {
4023 message: format!("GitHub response is missing boolean field {field}"),
4024 })
4025}
4026fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
4027 match value.get(field) {
4028 None | Some(Value::Null) => Ok(None),
4029 Some(value) => value
4030 .as_str()
4031 .map(Some)
4032 .ok_or_else(|| SourceError::Malformed {
4033 message: format!("GitHub response field {field} is not a string or null"),
4034 }),
4035 }
4036}
4037fn optional_nodes<'a>(
4038 connection: Option<&'a Value>,
4039 name: &str,
4040) -> Result<Option<&'a Vec<Value>>, SourceError> {
4041 match connection {
4042 None | Some(Value::Null) => Ok(None),
4043 Some(value) => value
4044 .get("nodes")
4045 .and_then(Value::as_array)
4046 .map(Some)
4047 .ok_or_else(|| SourceError::Malformed {
4048 message: format!("GitHub {name}.nodes is not an array"),
4049 }),
4050 }
4051}
4052fn complete_connection(connection: &Value, name: &str, size: u32) -> Result<(), SourceError> {
4053 let page_info = connection
4054 .get("pageInfo")
4055 .ok_or_else(|| SourceError::Malformed {
4056 message: format!("GitHub {name} has no pageInfo"),
4057 })?;
4058 if required_bool(page_info, "hasNextPage")? {
4059 return Err(SourceError::Malformed {
4060 message: format!(
4061 "GitHub {name} exceeds the supported nested connection size of {size}"
4062 ),
4063 });
4064 }
4065 Ok(())
4066}
4067fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
4068 optional_str(value, field)?
4069 .map(|timestamp| {
4070 timestamp.parse().map_err(|error| SourceError::Malformed {
4071 message: format!("GitHub response field {field} is not a timestamp: {error}"),
4072 })
4073 })
4074 .transpose()
4075}
4076fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
4077 if page.limit == 0 {
4078 Err(SourceError::Config {
4079 message: "page limit must be at least 1".into(),
4080 })
4081 } else {
4082 Ok(())
4083 }
4084}
4085fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
4086 let page = connection
4087 .get("pageInfo")
4088 .filter(|value| value.is_object())
4089 .ok_or_else(|| SourceError::Malformed {
4090 message: "GitHub connection is missing pageInfo".into(),
4091 })?;
4092 if required_bool(page, "hasNextPage")? {
4093 let cursor = required_str(page, "endCursor")?;
4094 validate_cursor_progress(None, cursor)?;
4095 Ok(Some(Cursor(cursor.into())))
4096 } else {
4097 Ok(None)
4098 }
4099}
4100fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
4101 if next.is_empty() || previous == Some(next) {
4102 Err(SourceError::Malformed {
4103 message: "GitHub pagination cursor is empty or did not advance".into(),
4104 })
4105 } else {
4106 Ok(())
4107 }
4108}
4109fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
4110 cursor.map_or(Ok(0), |c| {
4111 c.0.parse().map_err(|_| SourceError::Config {
4112 message: "page cursor is invalid".into(),
4113 })
4114 })
4115}
4116fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
4117 if offset > items.len() {
4118 return Page::last(vec![]);
4119 }
4120 let tail = items.split_off(offset);
4121 let mut selected = tail;
4122 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
4123 selected.truncate(limit);
4124 Page {
4125 items: selected,
4126 next,
4127 }
4128}