onetaskgraph_github_projects/lib.rs
1//! A stateless onetaskgraph source over one GitHub Projects v2 board.
2//!
3//! **A board is a container of projects, not a project.** Its own `title`,
4//! `shortDescription` and `readme` are never read as an item's fields and are never
5//! written: nothing in this source can rename the board a user configured.
6//!
7//! **A project is an issue and its tasks are that issue's sub-issues.** GitHub's schema
8//! decides that: `Issue` exposes `parent`, `subIssues` and `subIssuesSummary`, and
9//! `DraftIssue` exposes none of them. Creating an issue needs a `repositoryId`, and a
10//! board has none, so [`GitHubProjectsConfig::repository`] names the one repository this
11//! source creates its project and task issues in; a write without it is refused naming
12//! the field.
13//!
14//! **A document is an ordinary issue whose title begins [`DESIGN_TITLE_PREFIX`].** A
15//! board has no document type and nothing but issues to hold one in, so the title is the
16//! discriminator and it is the whole of it. The title this source *reports* is the one a
17//! person wrote, with the prefix taken off — the same way the metadata slot is taken off
18//! the body so `content` is what the person wrote — and writing a document puts the prefix
19//! back, so a round trip returns the title that went in.
20//!
21//! **Telling a document from a project from a task.** The design prefix is read **first**:
22//! a document is never a project and never a task, whatever sub-issues it has or does not
23//! have. Only then does the rest apply — a board issue is a project when *either* it has
24//! sub-issues *or* it carries [`ItemKind::METADATA_KEY`]; otherwise it is a task. A
25//! sub-issue is always a task, whatever it carries. The marker is sufficient and never
26//! necessary: it is what makes an *empty* project — the state a project copy passes
27//! through between creating the project and filing its first task — readable as a
28//! project, while the sub-issue arm lets a person author a project on the board by hand
29//! with no knowledge of this product's metadata at all. Reading the prefix later than the
30//! sub-issue rule would make a design issue with no sub-issues an empty project, which is
31//! exactly the state that rule exists to catch. Pull requests are neither a project nor a
32//! task nor a document and are ignored.
33//!
34//! **Where an entity is, is a link.** Every project, task and document this source reports
35//! carries a [`Location::Url`] naming the issue's own web address — the same address the
36//! `url` field already reports, in the shape that says a reader can open it. That is the
37//! contrast the location contract exists for: a reader holding an entity from this source
38//! is handed something to link to and one holding an entity from a folder of Markdown is
39//! handed a path, and neither has to know which plugin answered. It does not replace or
40//! derive from `url`; that field goes on reporting what it always reported.
41//!
42//! **Where metadata lives.** Short typed things go to typed fields and native relations:
43//! status to the board's `Status` single-select and the issue's own state, the copy
44//! origin to a source-owned `onetaskgraph.origin` text field, and dependencies to
45//! `blockedBy` and to sub-issue links. Unbounded caller JSON goes in a trailing
46//! `<!-- onetaskgraph.metadata ... -->` comment at the end of the issue body — the same
47//! encoding `docs/metadata.md` settles for Linear, not a second one. A ProjectV2 text
48//! field is length-bounded and `shortDescription` is capped at 300 characters, which is
49//! why neither can hold a caller's own prose.
50//!
51//! **Status.** `status_mapping` is per-instance configuration from a status category to
52//! `null`, a board `Status` option name, or a closed state of `completed` or
53//! `not-planned`. Nothing here ever calls `updateProjectV2Field`: that mutation's
54//! `singleSelectOptions` *overwrites* a field's option set, so no addition is additive
55//! and a mistake destroys every item's status. A status this board cannot represent is a
56//! refusal naming the status and the instance instead.
57//!
58//! `done` closes the issue by default because GitHub derives `subIssuesSummary.completed`
59//! and the board's own `Sub-issues progress` field from closed sub-issues: a plan whose
60//! finished tasks were only moved to a "Done" column would read 0% complete forever.
61//!
62//! # What this source declares, field by field
63//!
64//! One verdict per field of [`Capabilities`], and what `Native` means when this source
65//! says it. *Proven* means a shared journey drives it against the real
66//! binary over this source's own row in `crates/onetaskgraph/tests/e2e/fixtures.rs`, and
67//! `every_row_declares_exactly_what_its_plugin_reports` is what keeps this list and
68//! [`capabilities`](TaskSource::capabilities) from parting.
69//!
70//! | Field | Verdict |
71//! | --- | --- |
72//! | `projects` | **Supported and proven.** A task's project is the issue it is a sub-issue of, and a listing scoped to one keeps the items filed under that issue. This is the field that was declared and then not applied, which silently returned another project's tasks. |
73//! | `documents` | **Supported and proven.** A board holds issues, so a document is one: the issue whose title begins [`DESIGN_TITLE_PREFIX`]. Reads, filters and paging answer on exactly the terms a task read does, and a write puts the prefix back. |
74//! | `orphan_tasks` | **Supported and proven.** A task issue with no `parent` is in no project. |
75//! | `filter_by_label` | **Supported and proven,** over the issue's own labels. |
76//! | `filter_by_status` | **Supported and proven,** over the board's `Status` option and the issue's open or closed state, through this instance's own `status_mapping`. |
77//! | `search_title` | **Supported and proven,** over `Issue.title`. |
78//! | `search_content` | **Supported and proven,** over the visible body — the trailing metadata comment is not part of it. |
79//! | `task_dependencies` | **Supported and proven,** in both directions: `blockedBy` and `blocking`. |
80//! | `project_dependencies` | **Supported and proven,** in both directions, over the same two connections, because a project here is an issue. |
81//! | `max_page_size` | **Supported and proven.** [`MAX_PAGE_SIZE`], GitHub's own connection maximum. |
82//!
83//! Nothing here is unsupported. `documents` is not a predicate — it says this source has
84//! documents, which it does — and the three facts behind the uniform `Native` on the
85//! predicates beside it are recorded below rather than re-derived, because a reader who
86//! takes `Native` to mean *the remote service filters* will read that uniformity as a
87//! lie.
88//!
89//! First, the plugin contract defines `Support::Native` as *the source applies this
90//! predicate itself*, and says nothing about where it applies it. What the declaration
91//! promises the engine is capability rule 1 — a predicate declared `Native` **is** applied
92//! — so that the engine may push it down and apply nothing of its own.
93//!
94//! Second, this source can keep that promise for every predicate at no additional API
95//! cost, because this source's own `board` walk already reads every page of the board
96//! before it answers anything at all. That walk is what `get_task`, `labels` and every
97//! listing already pay for; filtering the items it returns is in-process work over data
98//! already in hand.
99//!
100//! Third, no predicate could be pushed into the API even if that were wanted:
101//! `ProjectV2.items` takes `first` and `after` and offers no filter argument of any kind.
102//! So there is no server-side filtering to declare, and — the other half of the same fact
103//! — there is no predicate here that is genuinely unsupportable. Declaring one
104//! `Unsupported` would make the engine compensate for work this source has already done,
105//! and declaring `projects` native while ignoring the filter (which this source once did)
106//! silently returns another project's tasks, because the engine trusts the declaration and
107//! applies nothing locally.
108//!
109//! Filtering happens before paging, so a page of a filtered result is a page of the
110//! survivors rather than the survivors of a page. Label and text matching answer the same
111//! question the same way the local Markdown source's do, so one cross-source expectation
112//! holds for both.
113//!
114//! <!-- llmlint: ignore[contracts_have_one_source_or_a_drift_gate] The declaration itself
115//! has one source, `capabilities`, and the note above is the reasoning behind it rather
116//! than a second copy of it: without the three facts recorded here a reader takes the
117//! uniform `Native` for a lie and reverts it. The drift gate on the declaration is this
118//! crate's own capabilities test, which pins every field of it against a fully spelled-out
119//! `Capabilities` literal — a struct with no `Default`, so a field added to the contract
120//! fails to compile there rather than going unasserted. -->
121//! Required checks use only the local fixture server; the ignored credentialed lane
122//! verifies the current schema, then drives every field of the table above against the
123//! real board. It builds its own fixture there — two projects, one task filed under each,
124//! one filed under neither, a label on one of the three and a closed status on another —
125//! because that shape is what tells an honoured predicate from an ignored one: a board
126//! holding a single project answers a project filter the same way whether or not this
127//! source applies it, which is exactly how the defect above went unseen.
128//!
129//! That lane writes only to the board `GH_PROJECTS_OWNER` and `GH_PROJECTS_NUMBER` name,
130//! and only into the repository `GH_PROJECTS_REPOSITORY` names, and skips — as it does
131//! without `GH_PROJECTS_TOKEN` — when any of them is absent. Requiring both to be
132//! nominated is what keeps a credentialed write lane off a board and a repository nobody
133//! nominated; it never asks GitHub which project was updated most recently. Before it
134//! starts, the lane also clears any item titled — and any repository label named — the way
135//! it titles and names its own artifacts, which is self-healing after an interrupted run:
136//! a process killed between its writes and its cleanup leaves artifacts the next run
137//! removes.
138//!
139//! **GitHub has two rate limiters and this source is refused by both, so nothing here
140//! treats them as one thing.** The primary budget is the hourly allowance `gh api
141//! rate_limit` reports; the secondary limiter is a burst limiter over content-generating
142//! requests, and *nothing* reports it. Which one refused decides the operator's next step,
143//! so [`Limiter`] is a type rather than a detail, and it is what [`MIN_MUTATION_INTERVAL_MS`],
144//! [`GitHubProjectsSource::board_cache`] and [`GitHubProjectsSource::graphql`] each answer
145//! one part of.
146#![deny(missing_docs)]
147
148use std::collections::BTreeMap;
149use std::sync::Mutex;
150use std::time::{Duration, Instant};
151
152use chrono::{DateTime, Utc};
153use onetaskgraph_plugin_api::{
154 Capabilities, Cursor, DependencyEdge, DependencyEndpoint, DependencyKind, DependencySupport,
155 Direction, Document, DocumentQuery, Health, ItemKind, ItemWrite, Label, LabelFilter, Location,
156 NativeId, Page, PageRequest, Project, ProjectFilter, ProjectQuery, Repository, SecretResolver,
157 SourceError, SourceName, SourcePlugin, Status, StatusCategory, Support, Task, TaskQuery,
158 TaskSource, TextFields, TextQuery, WriteSupport,
159};
160use reqwest::{Client, StatusCode, Url};
161use schemars::{Schema, schema_for};
162use secrecy::{ExposeSecret, SecretString};
163use serde::Deserialize;
164use serde_json::{Value, json};
165
166/// The registry name for this plugin.
167pub const KIND: &str = "github-projects";
168/// GitHub's maximum connection page size.
169pub const MAX_PAGE_SIZE: u32 = 100;
170/// Nested connection size which keeps GitHub's worst-case query below its node limit.
171const NESTED_PAGE_SIZE: u32 = 50;
172
173/// The issue-title prefix that makes a board issue a document.
174///
175/// A GitHub Projects board has no document type — it holds issues — so the discriminator
176/// is the title, and this is the whole of it: an issue whose title begins with these bytes
177/// is a document and every other issue is the task or project the sub-issue rule makes it.
178///
179/// It is spelled **once**, here, and read rather than restated everywhere else — including
180/// by the shared journeys, which take it from this constant so a board fixture cannot
181/// drift from what this source reads. `docs/metadata.md` records the two consequences that
182/// are not obvious from the bytes: the reported title has this prefix taken off, exactly
183/// as the body's metadata slot is taken off `content`, and this prefix is read *before*
184/// the sub-issue rule, so a design issue with no sub-issues is never an empty project.
185pub const DESIGN_TITLE_PREFIX: &str = "DESIGN: ";
186
187/// Exact GraphQL query documents issued by this plugin.
188///
189/// Keeping the production documents here lets the pinned-schema test validate the same
190/// bytes that are sent to GitHub, rather than a test-only copy which could drift
191/// independently. No document in this module writes the board itself, and none of them
192/// names `updateProjectV2Field`.
193pub mod graphql {
194 /// Reads the board's fields and one page of its items.
195 pub const BOARD: &str = r#"query($owner:String!,$number:Int!,$first:Int!,$after:String,$nestedFirst:Int!,$duplicates:Boolean!){
196 owner:repositoryOwner(login:$owner){
197 ... on ProjectV2Owner{projectV2(number:$number){...Board}}
198 }
199 } fragment Board on ProjectV2 { id title
200 fields(first:$nestedFirst){nodes{
201 ... on ProjectV2SingleSelectField{__typename id name options{id name}}
202 ... on ProjectV2Field{__typename id name}
203 }pageInfo{hasNextPage}}
204 items(first:$first,after:$after){nodes{id fieldValues(first:$nestedFirst){nodes{
205 ... on ProjectV2ItemFieldSingleSelectValue{name field{
206 ... on ProjectV2SingleSelectField{id name options{id name}}
207 }}
208 ... on ProjectV2ItemFieldTextValue{text field{... on ProjectV2Field{id name}}}
209 ... on ProjectV2ItemFieldLabelValue{labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
210 }pageInfo{hasNextPage}} content{
211 ... on Issue{__typename id title body url createdAt updatedAt state stateReason(enableDuplicate:$duplicates) repository{nameWithOwner} parent{id} subIssuesSummary{total} labels(first:$nestedFirst){nodes{id name color}pageInfo{hasNextPage}}}
212 ... on PullRequest{__typename id}
213 ... on DraftIssue{__typename id title body createdAt updatedAt}
214 }} pageInfo{hasNextPage endCursor}}
215 }"#;
216 /// Resolves the configured repository's node id, which creating an issue requires.
217 pub const REPOSITORY: &str = r#"query($owner:String!,$name:String!){repository(owner:$owner,name:$name){id nameWithOwner}}"#;
218 /// Reads both dependency directions for one issue, with each far end's own kind.
219 pub const ISSUE_DEPENDENCIES: &str = r#"query($id:ID!,$first:Int!,$after:String){node(id:$id){__typename
220 ... on Issue{
221 blockedBy(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
222 blocking(first:$first,after:$after){nodes{...Related}pageInfo{hasNextPage endCursor}}
223 }}} fragment Related on Issue{id title body parent{id} subIssuesSummary{total}}"#;
224 /// Creates one issue in the configured repository.
225 pub const CREATE_ISSUE: &str =
226 r#"mutation($input:CreateIssueInput!){createIssue(input:$input){issue{id url}}}"#;
227 /// Puts an existing issue on the configured board.
228 pub const ADD_TO_BOARD: &str = r#"mutation($input:AddProjectV2ItemByIdInput!){addProjectV2ItemById(input:$input){item{id}}}"#;
229 /// Updates an issue's visible fields and its open or closed state in one call.
230 pub const UPDATE_ISSUE: &str =
231 r#"mutation($input:UpdateIssueInput!){updateIssue(input:$input){issue{id}}}"#;
232 /// Updates an existing draft's user-visible fields.
233 pub const UPDATE_DRAFT: &str = r#"mutation($input:UpdateProjectV2DraftIssueInput!){updateProjectV2DraftIssue(input:$input){draftIssue{id}}}"#;
234 /// Updates a text or single-select value on one project item.
235 pub const UPDATE_FIELD: &str = r#"mutation($input:UpdateProjectV2ItemFieldValueInput!){updateProjectV2ItemFieldValue(input:$input){projectV2Item{id}}}"#;
236 /// Files one issue under another as a sub-issue, which is what project membership is.
237 pub const ADD_SUB_ISSUE: &str =
238 r#"mutation($input:AddSubIssueInput!){addSubIssue(input:$input){issue{id} subIssue{id}}}"#;
239 /// Takes one issue back out of its parent.
240 pub const REMOVE_SUB_ISSUE: &str = r#"mutation($input:RemoveSubIssueInput!){removeSubIssue(input:$input){issue{id} subIssue{id}}}"#;
241 /// Adds GitHub's native issue blocked-by relationship.
242 pub const ADD_BLOCKED_BY: &str = r#"mutation($input:AddBlockedByInput!){addBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
243 /// Removes one native issue blocked-by relationship.
244 pub const REMOVE_BLOCKED_BY: &str = r#"mutation($input:RemoveBlockedByInput!){removeBlockedBy(input:$input){issue{id} blockingIssue{id}}}"#;
245 /// Deletes one issue, which takes its board item with it.
246 ///
247 /// The engine sends this in one situation only: undoing a copy that could not finish,
248 /// over the items that same copy created. Deleting the issue removes the board item
249 /// too, so there is no second `deleteProjectV2Item` to keep in step with it.
250 pub const DELETE_ISSUE: &str =
251 r#"mutation($input:DeleteIssueInput!){deleteIssue(input:$input){repository{id}}}"#;
252
253 /// Every document above, with what this source is doing when it sends one.
254 ///
255 /// One list rather than a `match` beside the constants: a rate-limit diagnostic has to
256 /// name the call that was refused, and a `match` with a catch-all arm would answer a
257 /// document added later with "talking to GitHub" and never say so.
258 ///
259 /// `documents_are_all_inventoried` reads this file back and fails naming any `pub
260 /// const` here that this list omits, so the two cannot part — which is the same guard
261 /// `CATEGORIES` carries, in the one shape available to a set of `&str` constants.
262 pub const DOCUMENTS: [(&str, &str); 13] = [
263 (BOARD, "reading the board"),
264 (REPOSITORY, "reading the destination repository"),
265 (ISSUE_DEPENDENCIES, "reading an issue's dependencies"),
266 (CREATE_ISSUE, "creating an issue"),
267 (ADD_TO_BOARD, "adding an issue to the board"),
268 (UPDATE_ISSUE, "updating an issue"),
269 (UPDATE_DRAFT, "updating a draft item"),
270 (UPDATE_FIELD, "writing a board field"),
271 (ADD_SUB_ISSUE, "filing an issue under its project"),
272 (REMOVE_SUB_ISSUE, "taking an issue out of its project"),
273 (ADD_BLOCKED_BY, "recording a dependency"),
274 (REMOVE_BLOCKED_BY, "removing a dependency"),
275 (DELETE_ISSUE, "deleting an issue"),
276 ];
277}
278
279/// Which of GitHub's two rate limiters refused a request.
280///
281/// Waiting is the whole answer to the primary budget, and polling is what *extends* the
282/// secondary one — so an operator told the wrong one takes the wrong next step, which is
283/// the whole reason this is carried rather than collapsed into "rate limited".
284#[derive(Debug, Clone, Copy, PartialEq, Eq)]
285enum Limiter {
286 /// The hourly API budget, which `gh api rate_limit` reports and a wait answers.
287 Primary,
288 /// The burst limiter over content-generating requests, which nothing reports.
289 Secondary,
290}
291
292/// The wordings GitHub answers a secondary rate limit with.
293///
294/// It sends them under a forbidden status, under a too-many-requests status, and inside
295/// the `errors` of a *successful* response, which is why the text is what this matches on
296/// rather than the status. `abuse detection` is the wording GitHub used before the
297/// limiter was renamed and still returns from some endpoints; `submitted too quickly` is
298/// what a burst of content creation is refused with.
299///
300/// This is GitHub's vocabulary rather than this source's, so it is pinned rather than
301/// remembered: `tests/fixtures/rate-limits.json` records where each wording was read and
302/// when, and the drift gate reconciles the two lists both ways. Public for that gate
303/// alone — a caller has no use for it, and matching on a refusal is this source's job.
304pub const SECONDARY_WORDINGS: [&str; 5] = [
305 "secondary rate limit",
306 "temporarily blocked from content creation",
307 "abuse detection",
308 "submitted too quickly",
309 "exceeded a secondary",
310];
311
312/// The wordings GitHub answers an exhausted primary budget with.
313///
314/// `rate_limited` is the `type` its GraphQL error carries, which is read as a field rather
315/// than looked for in the response text. Pinned and gated exactly as
316/// [`SECONDARY_WORDINGS`] is, and public for the same one reason.
317pub const PRIMARY_WORDINGS: [&str; 3] = [
318 "api rate limit exceeded",
319 "rate limit exceeded",
320 "rate_limited",
321];
322
323/// What a response *says about itself*, which is the only place a refusal can be read.
324///
325/// Deliberately not the whole response body. A board is a place people write about their
326/// own work, and a task on it titled "the secondary rate limit" would, matched across the
327/// raw text, turn a perfectly good answer into a refusal this source then waited out and
328/// reported. So the item data is never read: what is read is GitHub's own REST-style
329/// `message` envelope, which is what a forbidden status carries, and the `message` and
330/// `type` of each GraphQL error, which is where a *successful* response says it.
331///
332/// A body that is not JSON at all has nothing structured to read, so only a failing
333/// response's own text is taken — a successful response that is not JSON is malformed
334/// rather than refused, and [`GitHubProjectsSource::answer`] says so.
335fn refusal_wording(status: StatusCode, body: &str) -> String {
336 let Ok(parsed) = serde_json::from_str::<Value>(body) else {
337 return if status.is_success() {
338 String::new()
339 } else {
340 body.to_owned()
341 };
342 };
343 let mut said: Vec<&str> = parsed
344 .get("message")
345 .and_then(Value::as_str)
346 .into_iter()
347 .collect();
348 if let Some(errors) = parsed.get("errors").and_then(Value::as_array) {
349 for error in errors {
350 said.extend(
351 ["message", "type"]
352 .into_iter()
353 .filter_map(|key| error.get(key).and_then(Value::as_str)),
354 );
355 }
356 }
357 said.join("; ")
358}
359
360impl Limiter {
361 /// Which limiter refused this response, or `None` when none of them did.
362 ///
363 /// The wording is read first and the status only decides what carries none of it,
364 /// because GitHub answers a secondary limit with a forbidden status far more often
365 /// than with too-many-requests — while a forbidden status saying nothing about a limit
366 /// really is a credential this token lacks.
367 ///
368 /// A response is a refusal because of its status or its own wording. A spent budget
369 /// only ever explains one; it never turns an answer into a refusal.
370 fn classify(status: StatusCode, budget_exhausted: bool, body: &str) -> Option<Self> {
371 let normalized = refusal_wording(status, body).to_ascii_lowercase();
372 if SECONDARY_WORDINGS
373 .iter()
374 .any(|wording| normalized.contains(wording))
375 {
376 return Some(Self::Secondary);
377 }
378 if status == StatusCode::TOO_MANY_REQUESTS {
379 return Some(Self::Primary);
380 }
381 // An exhausted budget *explains* a response that failed; it does not make one that
382 // succeeded into a failure. GitHub sets `x-ratelimit-remaining: 0` on the last
383 // request the budget allowed as well as on the ones it then refuses, so reading
384 // the header alone threw away a good answer — and, once refusals were retried,
385 // replayed a request that had already taken effect.
386 if !status.is_success() && budget_exhausted {
387 return Some(Self::Primary);
388 }
389 // A successful response saying it: GitHub reports a GraphQL rate limit in the
390 // `errors` of an HTTP 200, where nothing about the status says so at all.
391 if status.is_success()
392 && PRIMARY_WORDINGS
393 .iter()
394 .any(|wording| normalized.contains(wording))
395 {
396 return Some(Self::Primary);
397 }
398 None
399 }
400
401 /// What this limiter is called where an operator can look it up.
402 const fn name(self) -> &'static str {
403 match self {
404 Self::Primary => "GitHub's primary API rate limit",
405 Self::Secondary => "GitHub's secondary rate limit",
406 }
407 }
408
409 /// What the endpoint an operator would go and check says about this limiter.
410 const fn where_to_look(self) -> &'static str {
411 match self {
412 Self::Primary => {
413 "That is the budget `gh api rate_limit` reports, so that endpoint says when it \
414 comes back."
415 }
416 Self::Secondary => {
417 "That limiter is not the primary API budget: `gh api rate_limit` reports the \
418 primary budget and does not report this one, so budget showing there says \
419 nothing about this refusal, and every further attempt extends it."
420 }
421 }
422 }
423
424 /// The next step this limiter actually calls for.
425 const fn what_to_do(self) -> &'static str {
426 match self {
427 Self::Primary => {
428 "wait for the reset `gh api rate_limit` reports, then run the command again."
429 }
430 Self::Secondary => {
431 "leave this board alone for a few minutes, then run the command again — or \
432 raise pacing.min_mutation_interval_ms on this source so it writes more slowly."
433 }
434 }
435 }
436}
437
438/// One rate-limit refusal, and the wait GitHub asked for if it asked for one.
439#[derive(Debug, Clone, Copy)]
440struct Limited {
441 limiter: Limiter,
442 hint: Option<u64>,
443}
444
445impl Limited {
446 /// What the caller is told once this source has waited as long as it may.
447 ///
448 /// Both limiters report as [`SourceError::RateLimited`], because that is what
449 /// happened: the kind a caller matches on says a rate limit refused this, and nothing
450 /// about *which* limiter it was makes it a different kind of failure. What differs is
451 /// the operator's next step, and that is what the message carries — a secondary
452 /// refusal read as a primary one sends an operator to `gh api rate_limit`, where the
453 /// budget looks fine, and then back to retry the very burst that was refused.
454 fn exhausted(
455 self,
456 doing: &str,
457 waits: u32,
458 waited: Duration,
459 needed: Duration,
460 budget: Duration,
461 ) -> SourceError {
462 SourceError::RateLimited {
463 retry_after_seconds: self.hint,
464 message: Some(format!(
465 "{} refused this source while {doing}; it waited {} out over {} and was refused \
466 again, and the next wait of {} would take it past the {} one call may spend \
467 waiting. {} next: {}",
468 self.limiter.name(),
469 plural(waits, "refusal"),
470 seconds(waited),
471 seconds(needed),
472 seconds(budget),
473 self.limiter.where_to_look(),
474 self.limiter.what_to_do(),
475 )),
476 }
477 }
478}
479
480/// One attempt's outcome: an error to report, or a rate limit to wait out.
481enum Attempt {
482 Failed(SourceError),
483 Limited(Limited),
484}
485
486fn plural(count: u32, thing: &str) -> String {
487 if count == 1 {
488 format!("{count} {thing}")
489 } else {
490 format!("{count} {thing}s")
491 }
492}
493
494fn seconds(duration: Duration) -> String {
495 format!("{:.1}s", duration.as_secs_f64())
496}
497
498/// A header GitHub spells as a whole number of seconds, or `None` when this one is not.
499///
500/// A value that is present and unreadable is deliberately *not* an error. `retry-after` is
501/// allowed by HTTP to be a date rather than a count, an intermediary can rewrite either
502/// header, and neither is what makes a response a refusal — so the whole cost of one this
503/// cannot read is that the refusal carries no hint and the backing-off schedule answers it
504/// instead. Refusing the response over the header would turn a readable refusal into an
505/// unreadable one, and refusing to *wait* would be the one wrong direction to fail in.
506fn whole_seconds(value: Option<&reqwest::header::HeaderValue>) -> Option<u64> {
507 value
508 .and_then(|value| value.to_str().ok())
509 .and_then(|value| value.trim().parse::<u64>().ok())
510}
511
512/// Every mutation this source sends creates content — an issue, a board item, a field of
513/// one, a sub-issue link, a dependency — and no query in [`graphql::DOCUMENTS`] does, so
514/// what the secondary limiter counts and what the keyword says are the same set. That is
515/// what makes the keyword a sound test rather than a convenient one.
516fn is_mutation(query: &str) -> bool {
517 query.trim_start().starts_with("mutation")
518}
519
520/// What this source was doing, for a diagnostic that has to say so.
521///
522/// Read out of [`graphql::DOCUMENTS`], which is the inventory rather than a copy of it, so
523/// a document added without a description is caught by that list's own gate instead of
524/// falling through to the vague arm below.
525fn operation_description(query: &str) -> &'static str {
526 graphql::DOCUMENTS
527 .iter()
528 .find(|(document, _)| *document == query)
529 .map_or("talking to GitHub", |(_, doing)| *doing)
530}
531
532/// GitHub's published ceiling on content-generating requests, per minute.
533///
534/// Pinned in `tests/fixtures/rate-limits.json` and gated against it, because it is
535/// GitHub's number rather than this source's: [`MIN_MUTATION_INTERVAL_MS`] is *derived*
536/// from it, so a pacing value checked only against itself cannot go stale here.
537pub const CONTENT_CREATION_PER_MINUTE: u64 = 80;
538/// The same ceiling as GitHub publishes it per hour, which this source does **not** pace
539/// at. See [`MIN_MUTATION_INTERVAL_MS`] for why the per-minute bound is the one that
540/// governs; it is pinned beside its sibling so the gate would notice either one moving.
541pub const CONTENT_CREATION_PER_HOUR: u64 = 500;
542/// Shortest interval between two content-creating mutations, in milliseconds.
543///
544/// GitHub documents two secondary limits on content-generating requests:
545/// [`CONTENT_CREATION_PER_MINUTE`] and [`CONTENT_CREATION_PER_HOUR`]. 60000/80 is 750, so
546/// a mutation every 750 ms is the fastest rate that cannot exceed the per-minute bound,
547/// and that is the bound a copy actually trips: a copy of one plan-sized project is a
548/// burst of a few dozen mutations inside a few seconds. The hourly bound works out at one
549/// every 7.2 seconds sustained, which no single copy reaches and which, used as the
550/// spacing here, would turn an ordinary copy into an hour of waiting — so it is
551/// deliberately *not* what this paces at. An installation that wants the hourly bound
552/// honoured for a long sequence of copies says so through
553/// `pacing.min_mutation_interval_ms`.
554pub const MIN_MUTATION_INTERVAL_MS: u64 = 60_000 / CONTENT_CREATION_PER_MINUTE;
555/// First wait when a rate-limit refusal carries no hint; each further wait doubles it.
556///
557/// A doubling schedule from one second reaches a minute in six waits, which is GitHub's
558/// own advice for a secondary limit — wait, and wait longer each time — without spending
559/// the first minute of a transient refusal doing nothing.
560pub const RETRY_BACKOFF_MS: u64 = 1_000;
561/// Total time one call may spend waiting out rate limits before it reports a failure.
562///
563/// Two minutes is long enough to ride out the refusals a paced copy still collects and
564/// short enough that a command an operator is watching returns. The bound is what makes
565/// the wait a wait rather than a hang: a call refused past it ends in a diagnostic naming
566/// the limiter, not in a process nobody can tell from a wedged one.
567pub const RETRY_BUDGET_MS: u64 = 120_000;
568
569fn default_token_env() -> String {
570 "GH_PROJECTS_TOKEN".to_owned()
571}
572fn default_endpoint() -> String {
573 "https://api.github.com/graphql".to_owned()
574}
575
576/// Where one status category lands on this board.
577///
578/// `null` — an absent value — disables the category for this instance, and using a
579/// disabled status is a refusal naming the status and the instance.
580#[derive(Debug, Clone, Deserialize, schemars::JsonSchema)]
581#[serde(untagged)]
582pub enum StatusTargetConfig {
583 /// The name of a `Status` single-select option already on the board.
584 Column(ColumnName),
585 /// A closed issue state, whose reason is what tells done from cancelled.
586 Closed {
587 /// The `IssueClosedStateReason` to close with.
588 closed: ClosedState,
589 },
590}
591
592/// The name of a `Status` single-select option on the board.
593///
594/// Validated on the way in rather than checked later, so a blank option name — which
595/// nothing on a board can be — is a state this type cannot hold.
596#[derive(Debug, Clone, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
597#[serde(try_from = "String")]
598pub struct ColumnName(String);
599
600impl ColumnName {
601 /// The option name, as the board spells it.
602 fn as_str(&self) -> &str {
603 &self.0
604 }
605}
606
607impl TryFrom<String> for ColumnName {
608 type Error = String;
609
610 fn try_from(name: String) -> Result<Self, Self::Error> {
611 if name.trim().is_empty() {
612 return Err("a status_mapping option name cannot be blank".to_owned());
613 }
614 Ok(Self(name))
615 }
616}
617
618/// The two closed states this product can mean.
619///
620/// GitHub's `IssueClosedStateReason` also spells `DUPLICATE`, which is neither finished
621/// work nor abandoned work, so nothing here ever writes it.
622#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, schemars::JsonSchema)]
623#[serde(rename_all = "kebab-case")]
624pub enum ClosedState {
625 /// `COMPLETED` — precisely done.
626 Completed,
627 /// `NOT_PLANNED` — precisely cancelled.
628 NotPlanned,
629}
630
631impl ClosedState {
632 const fn reason(self) -> &'static str {
633 match self {
634 Self::Completed => "COMPLETED",
635 Self::NotPlanned => "NOT_PLANNED",
636 }
637 }
638}
639
640/// Configuration for one GitHub Projects v2 board.
641#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
642#[serde(default, deny_unknown_fields)]
643pub struct GitHubProjectsConfig {
644 /// Login of the user or organization which owns the board.
645 pub owner: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates GitHub's owner grammar before private construction.
646 /// The project number shown in the board's GitHub URL.
647 pub project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` bounds this to a positive GraphQL Int.
648 /// `owner/name` of the one repository this source creates its issues in.
649 ///
650 /// A board has no repository of its own and `createIssue` requires one, so a write
651 /// without this is refused naming the field. Reads never need it.
652 pub repository: Option<String>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the `owner/name` grammar before private construction.
653 /// Environment variable containing a fine-grained token with Projects and Issues
654 /// read/write plus Pull requests read-only access for every repository represented on
655 /// the board.
656 #[serde(default = "default_token_env")]
657 pub token_env: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` validates the environment-variable grammar.
658 /// GraphQL endpoint. GitHub Enterprise installations may override it.
659 #[serde(default = "default_endpoint")]
660 pub endpoint: String, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` converts it to the private validated `Url`.
661 /// Per-instance mapping from a status category to where it lands on this board.
662 ///
663 /// A category this does not mention keeps its shipped default: `backlog` to
664 /// "Backlog", `todo` to "Todo", `in-progress` to "In Progress", `done` to closed as
665 /// completed, `cancelled` to closed as not planned, and `draft` and `unknown`
666 /// disabled.
667 #[serde(default)]
668 pub status_mapping: BTreeMap<String, Option<StatusTargetConfig>>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `new` parses each key into a `StatusCategory` and reports an unknown one against this instance.
669 /// How fast this source writes, and how long it waits out a rate-limit refusal.
670 ///
671 /// Every field keeps its shipped default when it is absent, and the defaults are
672 /// GitHub's own published limits rather than taste. See [`Pacing`].
673 #[serde(default)]
674 pub pacing: PacingConfig,
675}
676
677/// How fast this source writes, and how long it waits out a rate-limit refusal.
678///
679/// Configurable because a GitHub Enterprise installation sets its own limits and an
680/// operator who has already been refused may want to go slower still — not because the
681/// defaults are guesses.
682#[derive(Debug, Clone, Default, Deserialize, schemars::JsonSchema)]
683#[serde(default, deny_unknown_fields)]
684pub struct PacingConfig {
685 /// Shortest interval between two content-creating mutations, in milliseconds.
686 ///
687 /// Zero sends them as fast as they are asked for, which is what a fixture server on
688 /// loopback wants and what no board on github.com does. At most [`MAX_PACING_MS`].
689 pub min_mutation_interval_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
690 /// First wait when a rate-limit refusal carries no hint, in milliseconds. Each
691 /// further wait of the same call doubles it. At most [`MAX_PACING_MS`], and never
692 /// zero while there is a budget to spend, because a schedule of zero-length waits
693 /// consumes none of it and so never ends.
694 pub retry_backoff_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` refuses a non-progressing zero and bounds the rest before the private validated `Pacing` is built.
695 /// Total time one call may spend waiting out rate limits, in milliseconds.
696 ///
697 /// Zero reports the refusal rather than waiting at all. At most [`MAX_PACING_MS`]:
698 /// the bound is what makes this a wait rather than a hang.
699 pub retry_budget_ms: Option<u64>, // llmlint: ignore[invalid_states_unrepresentable] Schema DTO; `Pacing::resolve` bounds it to `MAX_PACING_MS` before the private validated `Pacing` is built.
700}
701
702/// The largest any pacing setting may be, in milliseconds.
703///
704/// One hour. GitHub's own harshest published bound on content-generating requests works
705/// out at one every 7.2 seconds, so an hour is already three orders of magnitude past
706/// anything a real limit asks for, and past it the settings stop describing pacing at all:
707/// a wait budget beyond it is the unbounded wait this whole mechanism exists to replace,
708/// and an interval beyond it is a command that never sends its second mutation. It also
709/// keeps the clock arithmetic in [`GitHubProjectsSource::reserve_mutation_slot`] inside
710/// what an `Instant` can hold on every platform.
711pub const MAX_PACING_MS: u64 = 3_600_000;
712
713/// [`PacingConfig`] with every default resolved and every value checked, which is what the
714/// source holds.
715#[derive(Debug, Clone, Copy)]
716struct Pacing {
717 min_mutation_interval: Duration,
718 retry_backoff: Duration,
719 retry_budget: Duration,
720}
721
722impl Pacing {
723 /// Resolve one instance's pacing, refusing a configuration that would not pace at all.
724 fn resolve(config: PacingConfig, instance: &SourceName) -> Result<Self, SourceError> {
725 let bounded = |value: Option<u64>, default: u64, field: &str| match value {
726 Some(value) if value > MAX_PACING_MS => Err(SourceError::Config {
727 message: format!(
728 "pacing.{field} of source {instance} is {value} ms, and the most any pacing \
729 setting may be is {MAX_PACING_MS} ms — an hour, which is already far past \
730 GitHub's own harshest published limit"
731 ),
732 }),
733 Some(value) => Ok(Duration::from_millis(value)),
734 None => Ok(Duration::from_millis(default)),
735 };
736 let retry_backoff = bounded(
737 config.retry_backoff_ms,
738 RETRY_BACKOFF_MS,
739 "retry_backoff_ms",
740 )?;
741 let retry_budget = bounded(config.retry_budget_ms, RETRY_BUDGET_MS, "retry_budget_ms")?;
742 if retry_backoff.is_zero() && !retry_budget.is_zero() {
743 return Err(SourceError::Config {
744 message: format!(
745 "pacing.retry_backoff_ms of source {instance} is 0 while \
746 pacing.retry_budget_ms is {} ms; a schedule of zero-length waits spends \
747 none of that budget, so it would retry a refusal forever. Set a backoff of \
748 at least 1 ms, or set retry_budget_ms to 0 to report a refusal without \
749 waiting at all",
750 retry_budget.as_millis()
751 ),
752 });
753 }
754 Ok(Self {
755 min_mutation_interval: bounded(
756 config.min_mutation_interval_ms,
757 MIN_MUTATION_INTERVAL_MS,
758 "min_mutation_interval_ms",
759 )?,
760 retry_backoff,
761 retry_budget,
762 })
763 }
764}
765
766/// Factory for [`GitHubProjectsSource`].
767#[derive(Debug, Clone, Copy, Default)]
768pub struct Plugin;
769
770impl SourcePlugin for Plugin {
771 fn kind(&self) -> &'static str {
772 KIND
773 }
774 fn config_schema(&self) -> Schema {
775 schema_for!(GitHubProjectsConfig)
776 }
777 fn build(
778 &self,
779 name: &SourceName,
780 config: &Value,
781 secrets: &dyn SecretResolver,
782 ) -> Result<Box<dyn TaskSource>, SourceError> {
783 let config: GitHubProjectsConfig =
784 serde_json::from_value(config.clone()).map_err(|e| SourceError::Config {
785 message: format!("source {name}: {e}"),
786 })?;
787 let source =
788 GitHubProjectsSource::new(name, config, secrets).map_err(|error| match error {
789 SourceError::Config { message } => SourceError::Config {
790 message: format!("source {name}: {message}"),
791 },
792 SourceError::Auth { message } => SourceError::Auth {
793 message: format!("source {name}: {message}"),
794 },
795 other => other,
796 })?;
797 Ok(Box::new(source))
798 }
799}
800
801/// Where a status category lands on this board, once configuration is resolved.
802#[derive(Debug, Clone, PartialEq, Eq)]
803enum StatusTarget {
804 /// Not usable against this instance.
805 Disabled,
806 /// The board's `Status` option of this name.
807 Column(ColumnName),
808 /// A closed issue, with the reason that says which closed it means.
809 Closed(ClosedState),
810}
811
812/// Every status category, in the order the vocabulary declares them.
813///
814/// This list mirrors `StatusCategory`, so it carries its own drift gate rather than a
815/// reviewer's attention: [`category_position`] is a wildcard-free match, so a variant
816/// added to the shared vocabulary fails to compile until it is named there, and this
817/// crate's suite reconciles this list against that enum's own derived schema, which is
818/// generated from the variants rather than written beside them. The schema is what
819/// catches a list left one short — a list checking only the positions it already holds
820/// would pass while every mapping indexed by the new position panicked.
821pub const CATEGORIES: [StatusCategory; 7] = [
822 StatusCategory::Draft,
823 StatusCategory::Backlog,
824 StatusCategory::Todo,
825 StatusCategory::InProgress,
826 StatusCategory::Done,
827 StatusCategory::Cancelled,
828 StatusCategory::Unknown,
829];
830
831/// Where one category sits in [`CATEGORIES`]; see that list for what this pins.
832#[must_use]
833pub const fn category_position(category: StatusCategory) -> usize {
834 match category {
835 StatusCategory::Draft => 0,
836 StatusCategory::Backlog => 1,
837 StatusCategory::Todo => 2,
838 StatusCategory::InProgress => 3,
839 StatusCategory::Done => 4,
840 StatusCategory::Cancelled => 5,
841 StatusCategory::Unknown => 6,
842 }
843}
844
845/// The spelling a status category is configured and reported under.
846fn category_name(category: StatusCategory) -> &'static str {
847 match category {
848 StatusCategory::Draft => "draft",
849 StatusCategory::Backlog => "backlog",
850 StatusCategory::Todo => "todo",
851 StatusCategory::InProgress => "in-progress",
852 StatusCategory::Done => "done",
853 StatusCategory::Cancelled => "cancelled",
854 StatusCategory::Unknown => "unknown",
855 }
856}
857
858/// A shipped default's option name.
859///
860/// The literals below are this file's own and non-blank, and they are validated by the
861/// one constructor a configured name goes through rather than beside it.
862fn shipped_column(name: &'static str) -> ColumnName {
863 ColumnName::try_from(name.to_owned()).expect("a shipped default names a board option")
864}
865
866/// The shipped default for one category, before this instance's configuration.
867fn shipped_default(category: StatusCategory) -> StatusTarget {
868 match category {
869 StatusCategory::Backlog => StatusTarget::Column(shipped_column("Backlog")),
870 StatusCategory::Todo => StatusTarget::Column(shipped_column("Todo")),
871 StatusCategory::InProgress => StatusTarget::Column(shipped_column("In Progress")),
872 StatusCategory::Done => StatusTarget::Closed(ClosedState::Completed),
873 StatusCategory::Cancelled => StatusTarget::Closed(ClosedState::NotPlanned),
874 StatusCategory::Draft | StatusCategory::Unknown => StatusTarget::Disabled,
875 }
876}
877
878/// This instance's complete category-to-target mapping, read in both directions.
879///
880/// One target per category, held at that category's own [`category_position`], so a
881/// category missing from the mapping, named twice in it, or filed out of order is a
882/// state this type cannot hold rather than one [`Self::target`] has to defend against.
883#[derive(Debug, Clone)]
884struct StatusMapping {
885 targets: [StatusTarget; CATEGORIES.len()],
886}
887
888impl StatusMapping {
889 fn resolve(
890 configured: BTreeMap<String, Option<StatusTargetConfig>>,
891 instance: &SourceName,
892 ) -> Result<Self, SourceError> {
893 let mut overrides: BTreeMap<&'static str, Option<StatusTargetConfig>> = BTreeMap::new();
894 for (key, value) in configured {
895 let category = CATEGORIES
896 .iter()
897 .find(|category| category_name(**category) == key)
898 .ok_or_else(|| SourceError::Config {
899 message: format!(
900 "status_mapping names {key:?}, which is not a status category of source \
901 {instance}; the categories are {}",
902 CATEGORIES
903 .iter()
904 .map(|category| category_name(*category))
905 .collect::<Vec<_>>()
906 .join(", ")
907 ),
908 })?;
909 overrides.insert(category_name(*category), value);
910 }
911 // `CATEGORIES[position] == category` for every category — the crate's suite
912 // asserts it — so mapping the list in order fills each category's own slot.
913 let targets = CATEGORIES.map(|category| match overrides.remove(category_name(category)) {
914 None => shipped_default(category),
915 Some(None) => StatusTarget::Disabled,
916 Some(Some(StatusTargetConfig::Column(option))) => StatusTarget::Column(option),
917 Some(Some(StatusTargetConfig::Closed { closed })) => StatusTarget::Closed(closed),
918 });
919 let mapping = Self { targets };
920 for (index, category) in CATEGORIES.into_iter().enumerate() {
921 let StatusTarget::Column(option) = mapping.target(category) else {
922 continue;
923 };
924 if let Some(other) = CATEGORIES[..index].iter().find(|earlier| {
925 matches!(mapping.target(**earlier), StatusTarget::Column(name)
926 if name.as_str().eq_ignore_ascii_case(option.as_str()))
927 }) {
928 return Err(SourceError::Config {
929 message: format!(
930 "status_mapping of source {instance} sends both {} and {} to the board \
931 option {:?}; one option cannot read back as two categories",
932 category_name(*other),
933 category_name(category),
934 option.as_str()
935 ),
936 });
937 }
938 }
939 Ok(mapping)
940 }
941
942 fn target(&self, category: StatusCategory) -> &StatusTarget {
943 &self.targets[category_position(category)]
944 }
945
946 /// The category a board option name reports, or `None` when nothing maps to it.
947 fn category_of(&self, option: &str) -> Option<StatusCategory> {
948 CATEGORIES.into_iter().find(|category| {
949 matches!(self.target(*category), StatusTarget::Column(name)
950 if name.as_str().eq_ignore_ascii_case(option))
951 })
952 }
953}
954
955/// The one repository this source creates issues in.
956#[derive(Debug, Clone)]
957struct RepositoryTarget {
958 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
959 name: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only after `owner/name` validation in `new`.
960}
961
962impl RepositoryTarget {
963 fn parse(value: &str) -> Result<Self, SourceError> {
964 let (owner, name) = value.split_once('/').ok_or_else(|| SourceError::Config {
965 message: format!(
966 "repository must be spelled owner/name; {value:?} names no repository"
967 ),
968 })?;
969 if !valid_github_owner(owner) || !valid_github_repository_name(name) {
970 return Err(SourceError::Config {
971 message: format!(
972 "repository must be spelled owner/name with a GitHub login and one \
973 repository name; {value:?} is not"
974 ),
975 });
976 }
977 Ok(Self {
978 owner: owner.to_owned(),
979 name: name.to_owned(),
980 })
981 }
982
983 fn origin(&self) -> String {
984 format!("github.com/{}/{}", self.owner, self.name)
985 }
986}
987
988/// A source which reads GitHub afresh for every operation.
989pub struct GitHubProjectsSource {
990 /// This source's configured name, used both to tell a far end naming this source
991 /// from one naming a system it knows nothing about, and to name the instance a
992 /// status refusal is about.
993 name: SourceName,
994 owner: String, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after full GitHub-owner validation.
995 project_number: u32, // llmlint: ignore[invalid_states_unrepresentable] Private, constructed only by `new` after GraphQL-Int validation.
996 repository: Option<RepositoryTarget>,
997 endpoint: Url,
998 token: SecretString,
999 credential_name: String, // llmlint: ignore[invalid_states_unrepresentable] Private diagnostic value constructed only after environment-name validation.
1000 statuses: StatusMapping,
1001 client: Client,
1002 /// Every item this source has created since it was built, in the order it created
1003 /// them.
1004 ///
1005 /// GitHub's `projectV2.items` is eventually consistent: an issue added to a board with
1006 /// `addProjectV2ItemById` is routinely absent from the very next read of that board, so
1007 /// a copy resolving a dependency on an item it had just created refused it as not
1008 /// found. A board read is completed from this — an item remembered here and absent from
1009 /// the read is added back, because the board really does hold it and only the read is
1010 /// behind.
1011 ///
1012 /// It is not a cache of a user's work: nothing is remembered that this process did not
1013 /// itself just write, it lives and dies with the process, and it is never consulted for
1014 /// an item this source did not create.
1015 created: Mutex<Vec<Resolved>>,
1016 /// How fast this source writes, and how long it waits out a refusal.
1017 pacing: Pacing,
1018 /// When the last content-creating mutation finished, or the moment the furthest-out
1019 /// reserved slot releases the next one, whichever is later — so the one after it can be
1020 /// spaced from that. See [`MIN_MUTATION_INTERVAL_MS`] for the interval and
1021 /// [`GitHubProjectsSource::finish_mutation`] for why completion rather than release is
1022 /// what it is measured from.
1023 last_mutation: Mutex<Option<Instant>>,
1024 /// The board as this process last read it, for the length of one command.
1025 ///
1026 /// A copy of a project used to re-read the whole board, paged, before writing each of
1027 /// its items, which is by far the largest part of a copy's request count and none of
1028 /// its work. Nothing else changes this board while a command runs — this source's own
1029 /// writes are the only writer — so one read answers them all.
1030 ///
1031 /// It is not a store of a user's work and it is not the cache the no-persistence
1032 /// invariant forbids: it lives and dies with the process exactly as `created` does,
1033 /// nothing is written down, and [`Self::board`] still completes it from `created`, so
1034 /// an item this command created and then depends on resolves whether or not GitHub's
1035 /// own eventually-consistent read has caught up. A write to an item already on the
1036 /// board updates the entry here too, so what this holds is the last read plus this
1037 /// process's own writes rather than a snapshot taken before them.
1038 board_cache: Mutex<Option<Board>>,
1039 /// The destination repository's node id, resolved once rather than per issue created.
1040 ///
1041 /// A repository's node id does not change, and re-reading it for every issue of a copy
1042 /// spent one request per item on an answer this source already had.
1043 repository_cache: Mutex<Option<String>>,
1044}
1045
1046impl GitHubProjectsSource {
1047 /// Validate configuration and capture the named credential without exposing it.
1048 ///
1049 /// # Errors
1050 ///
1051 /// Returns [`SourceError::Config`] for a configuration this instance cannot use and
1052 /// [`SourceError::Auth`] when the named credential is missing or empty.
1053 pub fn new(
1054 name: &SourceName,
1055 config: GitHubProjectsConfig,
1056 secrets: &dyn SecretResolver,
1057 ) -> Result<Self, SourceError> {
1058 if !valid_github_owner(&config.owner) {
1059 return Err(SourceError::Config {
1060 message: "owner must be 1-39 ASCII letters, digits, or single hyphens, and cannot start or end with a hyphen".into(),
1061 });
1062 }
1063 if config.project_number == 0 || config.project_number > i32::MAX as u32 {
1064 return Err(SourceError::Config {
1065 message: format!("project_number must be between 1 and {}", i32::MAX),
1066 });
1067 }
1068 if !valid_environment_name(&config.token_env) {
1069 return Err(SourceError::Config {
1070 message: "token_env must be a valid environment-variable name".into(),
1071 });
1072 }
1073 let repository = config
1074 .repository
1075 .as_deref()
1076 .map(RepositoryTarget::parse)
1077 .transpose()?;
1078 let endpoint = Url::parse(&config.endpoint).map_err(|e| SourceError::Config {
1079 message: format!("endpoint is not a valid URL: {e}"),
1080 })?;
1081 if endpoint.scheme() != "https"
1082 && !(endpoint.scheme() == "http"
1083 && endpoint
1084 .host_str()
1085 .is_some_and(|h| h == "127.0.0.1" || h == "localhost" || h == "::1"))
1086 {
1087 return Err(SourceError::Config {
1088 message:
1089 "endpoint must use HTTPS (HTTP is accepted only for a loopback test server)"
1090 .into(),
1091 });
1092 }
1093 let token = secrets.get(&config.token_env).filter(|token| !token.expose_secret().trim().is_empty()).ok_or_else(|| SourceError::Auth {
1094 message: format!("environment variable {} is missing or empty; set it to a fine-grained GitHub token granting Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board", config.token_env),
1095 })?;
1096 Ok(Self {
1097 name: name.clone(),
1098 owner: config.owner,
1099 project_number: config.project_number,
1100 repository,
1101 endpoint,
1102 token,
1103 credential_name: config.token_env,
1104 statuses: StatusMapping::resolve(config.status_mapping, name)?,
1105 client: Client::builder()
1106 .user_agent("onetaskgraph")
1107 .build()
1108 .map_err(|e| SourceError::Config {
1109 message: format!("cannot build HTTP client: {e}"),
1110 })?,
1111 created: Mutex::new(Vec::new()),
1112 pacing: Pacing::resolve(config.pacing, name)?,
1113 last_mutation: Mutex::new(None),
1114 board_cache: Mutex::new(None),
1115 repository_cache: Mutex::new(None),
1116 })
1117 }
1118
1119 /// Send one GraphQL document, pacing this source's own mutations and waiting out a
1120 /// rate limit rather than handing it straight back as an error.
1121 ///
1122 /// Retrying is safe for every document here, including the mutations, and the reason
1123 /// is that only a *refusal* is retried: [`Limiter::classify`] rules on a response
1124 /// GitHub sent, and a request GitHub refused for a rate limit did not run, so nothing
1125 /// this replays has already taken effect. An outcome this source cannot know — the
1126 /// send failed, or the body could not be read, so the mutation may well have landed —
1127 /// is [`Attempt::Failed`] in [`send_once`] and leaves this loop without a second
1128 /// attempt. A duplicate write would come from replaying one of those, and none is
1129 /// replayed.
1130 async fn graphql(&self, query: &str, variables: Value) -> Result<Value, SourceError> {
1131 let doing = operation_description(query);
1132 let mut waited = Duration::ZERO;
1133 let mut waits = 0_u32;
1134 let mut backoff = self.pacing.retry_backoff;
1135 loop {
1136 if is_mutation(query) {
1137 let spacing = self.reserve_mutation_slot();
1138 if !spacing.is_zero() {
1139 tokio::time::sleep(spacing).await;
1140 }
1141 }
1142 let attempt = self.send_once(query, &variables).await;
1143 if is_mutation(query) {
1144 self.finish_mutation();
1145 }
1146 let limited = match attempt {
1147 Ok(data) => return Ok(data),
1148 Err(Attempt::Failed(error)) => return Err(error),
1149 Err(Attempt::Limited(limited)) => limited,
1150 };
1151 // GitHub really does send `retry-after: 0`, and retrying at once is the one
1152 // move that extends a secondary limit, so a hint below the schedule's own next
1153 // wait is raised to it.
1154 let wait = match limited.hint {
1155 Some(hint) => Duration::from_secs(hint).max(backoff),
1156 None => backoff,
1157 };
1158 let remaining = self.pacing.retry_budget.saturating_sub(waited);
1159 // A wait of nothing spends none of the budget, so it is exhaustion rather
1160 // than a retry. `Pacing::resolve` rules out every way of configuring one
1161 // except a budget of zero, where reporting the first refusal is the ask.
1162 if wait.is_zero() || wait > remaining {
1163 return Err(limited.exhausted(
1164 doing,
1165 waits,
1166 waited,
1167 wait,
1168 self.pacing.retry_budget,
1169 ));
1170 }
1171 tokio::time::sleep(wait).await;
1172 waited += wait;
1173 waits += 1;
1174 backoff = backoff.saturating_mul(2);
1175 }
1176 }
1177
1178 /// The next moment a content-creating mutation may leave this source, as a wait from
1179 /// now.
1180 ///
1181 /// The slot is reserved under the lock and the waiting happens outside it, so two
1182 /// callers take two slots rather than the same one — and no lock is held across an
1183 /// await.
1184 ///
1185 /// The moment it is spaced from is the previous mutation's *completion*, which
1186 /// [`Self::finish_mutation`] records. See that method for why the release moment on its
1187 /// own is the wrong thing to measure from.
1188 fn reserve_mutation_slot(&self) -> Duration {
1189 if self.pacing.min_mutation_interval.is_zero() {
1190 return Duration::ZERO;
1191 }
1192 // A poisoned lock here costs pacing, not correctness, and refusing the write over
1193 // it would turn an earlier failure into a second one for no gain.
1194 let mut last = self
1195 .last_mutation
1196 .lock()
1197 .unwrap_or_else(std::sync::PoisonError::into_inner);
1198 let now = Instant::now();
1199 // `checked_add` rather than `+`: `Instant + Duration` panics on overflow, and
1200 // pacing is not worth a panic even at a bound `MAX_PACING_MS` already rules out.
1201 let at = last.map_or(now, |previous| {
1202 previous
1203 .checked_add(self.pacing.min_mutation_interval)
1204 .map_or(now, |earliest| earliest.max(now))
1205 });
1206 *last = Some(at);
1207 at.saturating_duration_since(now)
1208 }
1209
1210 /// Record that a content-creating mutation has finished, so the next one is spaced
1211 /// from here rather than from the moment this one was released.
1212 ///
1213 /// This source can only choose when a request *departs*; the limiter counts when it
1214 /// *arrives*, and the two differ by whatever the request spent in transit. Spacing one
1215 /// departure from the last therefore hands the limiter a gap of the interval less that
1216 /// transit, so a source pacing at 750 ms can still be seen arriving faster — which is
1217 /// exactly how a copy paced well inside a board's threshold was refused by it on a
1218 /// slower machine while passing on a quick one.
1219 ///
1220 /// Spacing from completion removes the subtraction rather than budgeting for it. The
1221 /// previous request had already arrived before its response came back, so its arrival
1222 /// is no later than this moment, and the next mutation is released at least the
1223 /// interval after this moment and arrives no earlier than it is released: the gap the
1224 /// limiter measures is therefore at least the interval, whatever transit costs and on
1225 /// whatever platform. The price is that a mutation's own round trip no longer counts
1226 /// towards its spacing, which makes this source slightly slower than the configured
1227 /// rate rather than slightly faster — the safe side of a limit that punishes being
1228 /// wrong by refusing reads for the next fifty minutes.
1229 ///
1230 /// A failed attempt is recorded too: a request refused by the limiter still arrived,
1231 /// and one that never left costs only a wait nobody needed.
1232 fn finish_mutation(&self) {
1233 if self.pacing.min_mutation_interval.is_zero() {
1234 return;
1235 }
1236 // A poisoned lock here costs pacing, not correctness, exactly as in the reservation.
1237 let mut last = self
1238 .last_mutation
1239 .lock()
1240 .unwrap_or_else(std::sync::PoisonError::into_inner);
1241 let now = Instant::now();
1242 // `max` rather than an assignment: a concurrent caller may already have reserved a
1243 // slot further out, and completing this request must never pull that slot back in.
1244 *last = Some(last.map_or(now, |reserved| reserved.max(now)));
1245 }
1246
1247 /// One HTTP attempt, classified into an answer, a rate limit to wait out, or a
1248 /// failure that waiting cannot help.
1249 async fn send_once(&self, query: &str, variables: &Value) -> Result<Value, Attempt> {
1250 let response = self
1251 .client
1252 .post(self.endpoint.clone())
1253 .bearer_auth(self.token.expose_secret())
1254 .json(&json!({"query": query, "variables": variables}))
1255 .send()
1256 .await
1257 .map_err(|e| {
1258 Attempt::Failed(SourceError::Unavailable {
1259 message: format!("GitHub GraphQL request failed: {e}"),
1260 })
1261 })?;
1262 let status = response.status();
1263 let header = |name: &str| whole_seconds(response.headers().get(name));
1264 // Exactly `0` is exhaustion and everything else — a count, an empty value, bytes
1265 // that are not text at all — is "not known to be exhausted". This never makes a
1266 // response a refusal on its own: it says which limiter a refusal is attributed to
1267 // and where its hint comes from, so a value this cannot read costs a hint rather
1268 // than an answer.
1269 let exhausted = response
1270 .headers()
1271 .get("x-ratelimit-remaining")
1272 .and_then(|value| value.to_str().ok())
1273 == Some("0");
1274 // `retry-after` is what GitHub asks for when it asks; when it does not and the
1275 // primary budget is spent, `x-ratelimit-reset` says when that budget comes back,
1276 // which is the same question answered as an absolute time. Nothing else here is a
1277 // hint, and a schedule is what answers a refusal that carries none.
1278 let hint = header("retry-after").or_else(|| {
1279 exhausted
1280 .then(|| header("x-ratelimit-reset"))
1281 .flatten()
1282 .map(|reset| reset.saturating_sub(Utc::now().timestamp().max(0).unsigned_abs()))
1283 });
1284 // Read before it is parsed, because the evidence which tells a secondary rate
1285 // limit from a rejected credential is in the body of a response whose status says
1286 // only "forbidden" — and a non-success response was never parsed at all.
1287 let body = response.text().await.map_err(|e| {
1288 Attempt::Failed(SourceError::Unavailable {
1289 message: format!("GitHub GraphQL response could not be read: {e}"),
1290 })
1291 })?;
1292 if let Some(limiter) = Limiter::classify(status, exhausted, &body) {
1293 return Err(Attempt::Limited(Limited { limiter, hint }));
1294 }
1295 if status == StatusCode::UNAUTHORIZED || status == StatusCode::FORBIDDEN {
1296 return Err(Attempt::Failed(SourceError::Auth {
1297 message: format!(
1298 "GitHub rejected the configured credential with HTTP {status}; grant it Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board"
1299 ),
1300 }));
1301 }
1302 if !status.is_success() {
1303 return Err(Attempt::Failed(SourceError::Unavailable {
1304 message: format!("GitHub GraphQL returned HTTP {status}"),
1305 }));
1306 }
1307 self.answer(&body).map_err(Attempt::Failed)
1308 }
1309
1310 /// What one successful HTTP response says, once its GraphQL errors are read.
1311 fn answer(&self, body: &str) -> Result<Value, SourceError> {
1312 let body: Value = serde_json::from_str(body).map_err(|e| SourceError::Malformed {
1313 message: format!("GitHub returned invalid JSON: {e}"),
1314 })?;
1315 let errors = body
1316 .get("errors")
1317 .map(|value| {
1318 value.as_array().ok_or_else(|| SourceError::Malformed {
1319 message: "GitHub response errors is not an array".into(),
1320 })
1321 })
1322 .transpose()?;
1323 if let Some(errors) = errors.filter(|errors| !errors.is_empty()) {
1324 let messages = errors
1325 .iter()
1326 .filter_map(|e| e.get("message").and_then(Value::as_str))
1327 .collect::<Vec<_>>()
1328 .join("; ");
1329 let message = if messages.is_empty() {
1330 "GitHub returned GraphQL errors".into()
1331 } else {
1332 messages
1333 };
1334 let normalized = message.to_ascii_lowercase();
1335 if normalized.contains("resource not accessible") || normalized.contains("scope") {
1336 return Err(SourceError::Auth {
1337 message: format!(
1338 "{message}; grant {} Projects and Issues read/write plus Pull requests read-only access for every repository represented on the board",
1339 self.credential_name
1340 ),
1341 });
1342 }
1343 return Err(SourceError::Refused { message });
1344 }
1345 body.get("data")
1346 .filter(|data| data.is_object())
1347 .cloned()
1348 .ok_or_else(|| SourceError::Malformed {
1349 message: "GitHub response has no data object".into(),
1350 })
1351 }
1352
1353 // llmlint: ignore[boundary_inputs_validated] GitHub caps nested connections at 100 and
1354 // GraphQL cannot independently page them inside the outer item page. This source page is
1355 // deliberately bounded at that published maximum; the live drift journey exercises it.
1356 async fn board_page(
1357 &self,
1358 items_after: Option<&str>,
1359 items_first: u32,
1360 ) -> Result<Value, SourceError> {
1361 let data = self
1362 .graphql(
1363 graphql::BOARD,
1364 json!({"owner":self.owner,"number":self.project_number,
1365 "first":items_first.min(MAX_PAGE_SIZE),"after":items_after,
1366 "nestedFirst":NESTED_PAGE_SIZE,"duplicates":true}),
1367 )
1368 .await?;
1369 data.pointer("/owner/projectV2")
1370 .filter(|v| !v.is_null())
1371 .cloned()
1372 .ok_or_else(|| SourceError::Refused {
1373 message: format!(
1374 "GitHub project {}/{} was not found or is not visible to the token",
1375 self.owner, self.project_number
1376 ),
1377 })
1378 }
1379
1380 /// Every item on the board, with the one board identity they all share.
1381 ///
1382 /// See [`Self::board_cache`]. The completion from `created` happens on every call
1383 /// rather than once, which is what the cache could otherwise have broken.
1384 async fn board(&self) -> Result<Board, SourceError> {
1385 let cached = self.board_cache()?.clone();
1386 let mut board = match cached {
1387 Some(board) => board,
1388 None => {
1389 let read = self.read_board().await?;
1390 *self.board_cache()? = Some(read.clone());
1391 read
1392 }
1393 };
1394 for own in self.created()?.iter() {
1395 if !board.items.iter().any(|item| item.id == own.id) {
1396 board.items.push(own.clone());
1397 }
1398 }
1399 Ok(board)
1400 }
1401
1402 /// This process's own view of the board, or the refusal a poisoned lock is.
1403 fn board_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<Board>>, SourceError> {
1404 self.board_cache
1405 .lock()
1406 .map_err(|_| SourceError::Unavailable {
1407 message: "this source's view of the board was left inconsistent by an earlier \
1408 failure; next: run the command again"
1409 .into(),
1410 })
1411 }
1412
1413 /// Bring this process's own view of the board up to an item it has just written.
1414 ///
1415 /// A created item goes to `created`, which is what completes a board read GitHub's own
1416 /// eventual consistency has left behind. An item that was already there is replaced
1417 /// where it sits, so a second write of it in the same command reads its real parent
1418 /// rather than the one it had before the first write.
1419 ///
1420 /// "Where it sits" is two places, and missing the first leaves a stale record that
1421 /// wins: an item this same run created is held in `created` and not in the cached
1422 /// board, and `board` completes the cached board *from* `created`, so replacing only
1423 /// the cached copy of such an item replaces nothing and the read still reports the
1424 /// title it was created with.
1425 fn remember_written(&self, item: Resolved, created: bool) -> Result<(), SourceError> {
1426 if created {
1427 self.created()?.push(item);
1428 return Ok(());
1429 }
1430 {
1431 let mut own = self.created()?;
1432 if let Some(held) = own.iter_mut().find(|held| held.id == item.id) {
1433 *held = item;
1434 return Ok(());
1435 }
1436 }
1437 if let Some(board) = self.board_cache()?.as_mut()
1438 && let Some(held) = board.items.iter_mut().find(|held| held.id == item.id)
1439 {
1440 *held = item;
1441 }
1442 Ok(())
1443 }
1444
1445 /// Forget one item this process has just deleted, from both halves of its own view.
1446 fn forget(&self, id: &NativeId) -> Result<(), SourceError> {
1447 self.created()?.retain(|own| own.id != *id);
1448 if let Some(board) = self.board_cache()?.as_mut() {
1449 board.items.retain(|item| item.id != *id);
1450 }
1451 Ok(())
1452 }
1453
1454 /// Every page of the board, read from GitHub.
1455 async fn read_board(&self) -> Result<Board, SourceError> {
1456 let mut after: Option<String> = None;
1457 let mut items = Vec::new();
1458 let mut board;
1459 loop {
1460 let page = self.board_page(after.as_deref(), MAX_PAGE_SIZE).await?;
1461 for item in page
1462 .pointer("/items/nodes")
1463 .and_then(Value::as_array)
1464 .ok_or_else(|| SourceError::Malformed {
1465 message: "GitHub project items.nodes is not an array".into(),
1466 })?
1467 {
1468 if let Some(resolved) = self.resolve(item)? {
1469 items.push(resolved);
1470 }
1471 }
1472 let info = page
1473 .pointer("/items/pageInfo")
1474 .ok_or_else(|| SourceError::Malformed {
1475 message: "GitHub project items have no pageInfo".into(),
1476 })?;
1477 let has_next = required_bool(info, "hasNextPage")?;
1478 let next = has_next
1479 .then(|| required_str(info, "endCursor"))
1480 .transpose()?;
1481 board = page.clone();
1482 match next {
1483 Some(next) => {
1484 validate_cursor_progress(after.as_deref(), next)?;
1485 after = Some(next.to_owned());
1486 }
1487 None => break,
1488 }
1489 }
1490 Ok(Board {
1491 id: required_str(&board, "id")?.to_owned(),
1492 fields: board.get("fields").cloned().unwrap_or(Value::Null),
1493 items,
1494 })
1495 }
1496
1497 /// The items this source has created, for completing a board read that is behind.
1498 fn created(&self) -> Result<std::sync::MutexGuard<'_, Vec<Resolved>>, SourceError> {
1499 self.created.lock().map_err(|_| SourceError::Unavailable {
1500 message: "this source's record of what it created in this run was left \
1501 inconsistent by an earlier failure; next: run the command again"
1502 .into(),
1503 })
1504 }
1505
1506 /// One board item as this source reports it, or `None` for content it ignores.
1507 ///
1508 /// A pull request is neither a project nor a task — it is somebody's change, not a
1509 /// unit of plan — and an item whose content the token cannot see has nothing to
1510 /// report at all.
1511 fn resolve(&self, item: &Value) -> Result<Option<Resolved>, SourceError> {
1512 let content = item.get("content").ok_or_else(|| SourceError::Malformed {
1513 message: "GitHub project item is missing content".into(),
1514 })?;
1515 if content.is_null() {
1516 return Ok(None);
1517 }
1518 let content_kind = match required_str(content, "__typename")? {
1519 "Issue" => ContentKind::Issue,
1520 "DraftIssue" => ContentKind::DraftIssue,
1521 _ => return Ok(None),
1522 };
1523 let field_values = item
1524 .get("fieldValues")
1525 .ok_or_else(|| SourceError::Malformed {
1526 message: "GitHub project item is missing fieldValues".into(),
1527 })?;
1528 complete_connection(field_values, "project item field values")?;
1529 let nodes = field_values
1530 .get("nodes")
1531 .and_then(Value::as_array)
1532 .ok_or_else(|| SourceError::Malformed {
1533 message: "GitHub project item fieldValues.nodes is not an array".into(),
1534 })?;
1535 if let Some(labels) = content.get("labels") {
1536 complete_connection(labels, "content labels")?;
1537 }
1538 for field_value in nodes {
1539 if let Some(labels) = field_value.get("labels") {
1540 complete_connection(labels, "project item field labels")?;
1541 }
1542 }
1543 let (body, slot) = metadata_body(optional_str(content, "body")?.map(str::to_owned))?;
1544 let parent = optional_str(content.get("parent").unwrap_or(&Value::Null), "id")?
1545 .map(|id| NativeId(id.to_owned()));
1546 // A draft has no sub-issues to summarise, and GitHub's schema gives it no field
1547 // to read one from; it is a task, and never a project.
1548 let sub_issues = match content_kind {
1549 ContentKind::Issue => sub_issue_total(content)?,
1550 ContentKind::DraftIssue => 0,
1551 };
1552 let content_id = required_str(content, "id")?;
1553 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
1554 message: format!("GitHub issue {content_id}: {message}"),
1555 })?;
1556 let raw_title = required_str(content, "title")?;
1557 // The design prefix is read *first*, before either of the two rules that separate
1558 // a project from a task. A document is not work whatever sub-issues it has and
1559 // whatever marker it carries, and reading the prefix later would make a design
1560 // issue with none of either an empty project.
1561 let kind = if raw_title.starts_with(DESIGN_TITLE_PREFIX) {
1562 BoardKind::Document
1563 } else if parent.is_some() {
1564 // Being a sub-issue wins outright, and no marker overrides it: an issue filed
1565 // under a project is that project's task even when it has sub-issues of its
1566 // own.
1567 BoardKind::Work(ItemKind::Task)
1568 } else if sub_issues > 0 || marked == Some(ItemKind::Project) {
1569 BoardKind::Work(ItemKind::Project)
1570 } else {
1571 BoardKind::Work(ItemKind::Task)
1572 };
1573 // The title a person wrote, which for a document is the one without the prefix —
1574 // the same way `content` above is the body without this source's metadata slot.
1575 let title = match kind {
1576 BoardKind::Document => raw_title[DESIGN_TITLE_PREFIX.len()..].to_owned(),
1577 BoardKind::Work(_) => raw_title.to_owned(),
1578 };
1579 let own_repository = content
1580 .pointer("/repository/nameWithOwner")
1581 .and_then(Value::as_str)
1582 .map(|origin| Repository::try_from(format!("github.com/{origin}")))
1583 .transpose()
1584 .map_err(|message| SourceError::Malformed { message })?;
1585 let repositories = if slot.contains_key(Repository::METADATA_KEY) {
1586 Repository::from_metadata(&slot)
1587 .map_err(|message| SourceError::Malformed { message })?
1588 } else {
1589 own_repository.clone().into_iter().collect()
1590 };
1591 Ok(Some(Resolved {
1592 item_id: required_str(item, "id")?.to_owned(),
1593 id: NativeId(content_id.to_owned()),
1594 content_kind,
1595 kind,
1596 title,
1597 body: body.filter(|value| !value.is_empty()),
1598 status: self.status(item, content)?,
1599 labels: labels(content, nodes)?,
1600 parent,
1601 origin: text_field(nodes, ORIGIN_FIELD)?.filter(|value| !value.is_empty()),
1602 url: optional_str(content, "url")?.map(str::to_owned),
1603 created_at: optional_time(content, "createdAt")?,
1604 updated_at: optional_time(content, "updatedAt")?,
1605 own_repository,
1606 repositories,
1607 slot,
1608 }))
1609 }
1610
1611 /// The status one board item reports.
1612 ///
1613 /// The closed state decides the category and the `Status` option decides the name, so
1614 /// a closed issue sitting in a "Shipped" column reports `done` named `Shipped`. A
1615 /// closed issue whose reason is `DUPLICATE` or `REOPENED` reports `Unknown`: a
1616 /// duplicate is not finished work, and calling it done is a lie the next copy would
1617 /// write back. `REOPENED`-while-closed is a state this source can never produce, so
1618 /// it is read permissively rather than refused — reads are faithful, and refusals
1619 /// belong on writes.
1620 fn status(&self, item: &Value, content: &Value) -> Result<Status, SourceError> {
1621 let nodes = item
1622 .pointer("/fieldValues/nodes")
1623 .and_then(Value::as_array)
1624 .expect("resolve validates fieldValues.nodes before mapping status");
1625 let option = nodes
1626 .iter()
1627 .find(|value| value.pointer("/field/name").and_then(Value::as_str) == Some("Status"))
1628 .map(|value| required_str(value, "name"))
1629 .transpose()?;
1630 let state = optional_str(content, "state")?;
1631 if state == Some("CLOSED") {
1632 let category = match optional_str(content, "stateReason")? {
1633 None | Some("COMPLETED") => StatusCategory::Done,
1634 Some("NOT_PLANNED") => StatusCategory::Cancelled,
1635 Some(_) => StatusCategory::Unknown,
1636 };
1637 let fallback = match category {
1638 StatusCategory::Done => "Done",
1639 StatusCategory::Cancelled => "Cancelled",
1640 _ => "Closed",
1641 };
1642 return Ok(Status {
1643 category,
1644 name: option.unwrap_or(fallback).to_owned(),
1645 });
1646 }
1647 let name = option.unwrap_or("Open").to_owned();
1648 Ok(Status {
1649 category: self
1650 .statuses
1651 .category_of(&name)
1652 .unwrap_or(StatusCategory::Unknown),
1653 name,
1654 })
1655 }
1656
1657 /// The board Status option this write selects, or the refusal that says why not.
1658 ///
1659 /// For a column target the option is what the status *is*, so a board that has no such
1660 /// option is a refusal naming the status and the instance. For a closed target the
1661 /// issue's own state carries the category, and the option carries only the name a
1662 /// reader reports — so an option spelled the way this status is spelled is selected
1663 /// when the board has one, and nothing is refused when it does not.
1664 fn column_for(
1665 &self,
1666 board: &Board,
1667 status: &Status,
1668 target: &StatusTarget,
1669 ) -> Result<Option<(String, String)>, SourceError> {
1670 let (wanted, required) = match target {
1671 StatusTarget::Column(wanted) => (wanted.as_str(), true),
1672 StatusTarget::Closed(_) => (status.name.as_str(), false),
1673 StatusTarget::Disabled => return Ok(None),
1674 };
1675 let missing = |detail: &str| SourceError::Refused {
1676 message: format!(
1677 "status {} of source {} needs the board Status option {wanted:?}, and {detail}; add that option to the board, or point status_mapping.{} of this source at one it has",
1678 category_name(status.category),
1679 self.name,
1680 category_name(status.category)
1681 ),
1682 };
1683 let Some(field) = Board::field(&board.fields, "Status")? else {
1684 return if required {
1685 Err(missing("this board has no Status field"))
1686 } else {
1687 Ok(None)
1688 };
1689 };
1690 if required_str(field, "__typename")? != "ProjectV2SingleSelectField" {
1691 return if required {
1692 Err(missing(
1693 "this board's Status field is not a single-select field",
1694 ))
1695 } else {
1696 Ok(None)
1697 };
1698 }
1699 let option = field
1700 .get("options")
1701 .and_then(Value::as_array)
1702 .and_then(|options| {
1703 options.iter().find(|option| {
1704 option
1705 .get("name")
1706 .and_then(Value::as_str)
1707 .is_some_and(|name| name.eq_ignore_ascii_case(wanted))
1708 })
1709 });
1710 match option {
1711 None if required => Err(missing("this board does not have it")),
1712 None => Ok(None),
1713 Some(option) => Ok(Some((
1714 required_str(field, "id")?.to_owned(),
1715 required_str(option, "id")?.to_owned(),
1716 ))),
1717 }
1718 }
1719
1720 /// This instance's target for a category, refusing one it has disabled.
1721 ///
1722 /// Nothing here mutates the board's option set to make room for a status. GitHub
1723 /// documents `UpdateProjectV2FieldInput.singleSelectOptions` as *"provided values
1724 /// overwrite existing options"*, so no addition is additive and a mistake destroys the
1725 /// field and every item's status.
1726 fn resolved_target(&self, category: StatusCategory) -> Result<StatusTarget, SourceError> {
1727 let target = self.statuses.target(category).clone();
1728 if target != StatusTarget::Disabled {
1729 return Ok(target);
1730 }
1731 Err(SourceError::Refused {
1732 message: if category == StatusCategory::Draft {
1733 format!(
1734 "status draft is disabled for source {}: draft is incompatible with this \
1735 integration because GitHub draft issues cannot have sub-issues, and this \
1736 source stores a project's tasks as its issue's sub-issues",
1737 self.name
1738 )
1739 } else {
1740 format!(
1741 "status {} is disabled for source {}; set status_mapping.{} of this source \
1742 to a board Status option name or to a closed state",
1743 category_name(category),
1744 self.name,
1745 category_name(category)
1746 )
1747 },
1748 })
1749 }
1750
1751 async fn set_item_field(
1752 &self,
1753 board_id: &str,
1754 item_id: &str,
1755 field_id: &str,
1756 value: Value,
1757 ) -> Result<(), SourceError> {
1758 let data = self
1759 .graphql(
1760 graphql::UPDATE_FIELD,
1761 json!({"input":{
1762 "projectId":board_id,"itemId":item_id,"fieldId":field_id,"value":value
1763 }}),
1764 )
1765 .await?;
1766 let returned = data
1767 .pointer("/updateProjectV2ItemFieldValue/projectV2Item")
1768 .ok_or_else(|| SourceError::Malformed {
1769 message: "GitHub field update returned no project item".into(),
1770 })?;
1771 if required_str(returned, "id")? != item_id {
1772 return Err(SourceError::Malformed {
1773 message: "GitHub field update returned the wrong project item".into(),
1774 });
1775 }
1776 Ok(())
1777 }
1778
1779 async fn native_dependency_ids(&self, id: &NativeId) -> Result<Vec<String>, SourceError> {
1780 let mut after: Option<String> = None;
1781 let mut ids = Vec::new();
1782 loop {
1783 let data = self
1784 .graphql(
1785 graphql::ISSUE_DEPENDENCIES,
1786 json!({"id":id.0,"first":MAX_PAGE_SIZE,"after":after}),
1787 )
1788 .await?;
1789 let connection =
1790 data.pointer("/node/blockedBy")
1791 .ok_or_else(|| SourceError::Malformed {
1792 message: "GitHub dependency response has no blockedBy connection".into(),
1793 })?;
1794 ids.extend(
1795 connection
1796 .get("nodes")
1797 .and_then(Value::as_array)
1798 .ok_or_else(|| SourceError::Malformed {
1799 message: "GitHub dependency response nodes is not an array".into(),
1800 })?
1801 .iter()
1802 .map(|value| required_str(value, "id").map(str::to_owned))
1803 .collect::<Result<Vec<_>, _>>()?,
1804 );
1805 let next = next_cursor(connection)?;
1806 if let Some(next) = &next {
1807 validate_cursor_progress(after.as_deref(), &next.0)?;
1808 }
1809 after = next.map(|cursor| cursor.0);
1810 if after.is_none() {
1811 return Ok(ids);
1812 }
1813 }
1814 }
1815
1816 async fn dependencies(
1817 &self,
1818 id: &NativeId,
1819 near_kind: ItemKind,
1820 direction: Direction,
1821 page: &PageRequest,
1822 ) -> Result<Page<DependencyEdge>, SourceError> {
1823 validate_page(page)?;
1824 let limit = page.limit.min(MAX_PAGE_SIZE) as usize;
1825 let cursor = page.cursor.as_ref().map(|c| c.0.as_str());
1826 let recorded = recorded_offset(cursor, direction)?;
1827 // Asked for even in the recorded phase, whose page reads nothing from the
1828 // connection: `__typename` is what says whether this item has a native
1829 // relationship at all, and that is what decides which far ends the reserved key is
1830 // allowed to hold.
1831 let data = self
1832 .graphql(
1833 graphql::ISSUE_DEPENDENCIES,
1834 json!({"id":id.0,"first":page.limit.min(MAX_PAGE_SIZE),
1835 "after":if recorded.is_some() {None} else {cursor}}),
1836 )
1837 .await?;
1838 let node =
1839 data.get("node")
1840 .filter(|v| !v.is_null())
1841 .ok_or_else(|| SourceError::Refused {
1842 message: format!(
1843 "GitHub item {} was not found or does not support dependencies",
1844 id.0
1845 ),
1846 })?;
1847 let connection_name = match direction {
1848 Direction::DependsOn => "blockedBy",
1849 Direction::DependedOnBy => "blocking",
1850 };
1851 // A draft has neither `blockedBy` nor `blocking`, so nothing it depends on can be
1852 // named natively and the reserved key may hold any far end. An issue's connections
1853 // hold issues, and this source reads them at the near item's own level.
1854 let natively_names = (required_str(node, "__typename")? == "Issue").then_some(near_kind);
1855 if let Some(offset) = recorded {
1856 return Ok(recorded_page(
1857 self.recorded_edges(id, near_kind, direction, natively_names)
1858 .await?,
1859 offset,
1860 limit,
1861 ));
1862 }
1863 if natively_names.is_none() {
1864 return Ok(recorded_page(
1865 self.recorded_edges(id, near_kind, direction, natively_names)
1866 .await?,
1867 0,
1868 limit,
1869 ));
1870 }
1871 let connection = node
1872 .get(connection_name)
1873 .ok_or_else(|| SourceError::Malformed {
1874 message: "GitHub dependency response is missing its connection".into(),
1875 })?;
1876 let nodes = connection
1877 .get("nodes")
1878 .and_then(Value::as_array)
1879 .ok_or_else(|| SourceError::Malformed {
1880 message: "GitHub dependency response nodes is not an array".into(),
1881 })?;
1882 // `from` depends on `to`, always. GitHub spells the same relationship from either
1883 // end — `blockedBy` lists what this item waits on, `blocking` lists what waits on
1884 // it — so the near item is `from` in one direction and `to` in the other.
1885 let items = nodes
1886 .iter()
1887 .map(|value| {
1888 let related = NativeId(required_str(value, "id")?.into());
1889 let related_kind = related_kind(value)?;
1890 let (from, to) = match direction {
1891 Direction::DependsOn => (
1892 DependencyEndpoint::from_native(id.clone(), near_kind),
1893 DependencyEndpoint::from_native(related, related_kind),
1894 ),
1895 Direction::DependedOnBy => (
1896 DependencyEndpoint::from_native(related, related_kind),
1897 DependencyEndpoint::from_native(id.clone(), near_kind),
1898 ),
1899 };
1900 Ok(DependencyEdge {
1901 from,
1902 to,
1903 kind: DependencyKind::Blocks,
1904 })
1905 })
1906 .collect::<Result<Vec<_>, SourceError>>()?;
1907 let mut next = next_cursor(connection)?;
1908 if let Some(next) = &next {
1909 validate_cursor_progress(cursor, &next.0)?;
1910 }
1911 if next.is_none()
1912 && !self
1913 .recorded_edges(id, near_kind, direction, natively_names)
1914 .await?
1915 .is_empty()
1916 {
1917 next = Some(Cursor(format!("{RECORDED_CURSOR}0")));
1918 }
1919 Ok(Page { items, next })
1920 }
1921
1922 /// The edges this item records under [`DependencyEdge::RECORDED_KEY`], which is where
1923 /// a far end in another source has to live: no GitHub issue relationship can name one.
1924 ///
1925 /// Only forwards. The reverse of a recorded edge is derived from the far end, and this
1926 /// source never writes one down.
1927 ///
1928 /// The metadata lives in the item's own body slot, so reading it costs one board scan.
1929 /// That is why it happens once the native connection is spent rather than on every
1930 /// page.
1931 async fn recorded_edges(
1932 &self,
1933 id: &NativeId,
1934 near_kind: ItemKind,
1935 direction: Direction,
1936 natively_names: Option<ItemKind>,
1937 ) -> Result<Vec<DependencyEdge>, SourceError> {
1938 if direction != Direction::DependsOn {
1939 return Ok(Vec::new());
1940 }
1941 let Some(item) = self
1942 .board()
1943 .await?
1944 .items
1945 .into_iter()
1946 .find(|item| item.id == *id)
1947 else {
1948 return Ok(Vec::new());
1949 };
1950 DependencyEdge::recorded(&item.slot, id, near_kind, &self.name, natively_names)
1951 .map_err(|message| SourceError::Malformed { message })
1952 }
1953
1954 /// The configured repository's node id, or the refusal naming the field it needs.
1955 ///
1956 /// Resolved once per command; see [`Self::repository_cache`].
1957 async fn repository_id(&self) -> Result<String, SourceError> {
1958 if let Some(id) = self.repository_cache()?.clone() {
1959 return Ok(id);
1960 }
1961 let repository = self
1962 .repository
1963 .as_ref()
1964 .ok_or_else(|| SourceError::Refused {
1965 message: format!(
1966 "source {} has no repository configured, and a GitHub Projects board has no \
1967 repository of its own to create an issue in; set repository: owner/name on \
1968 this source",
1969 self.name
1970 ),
1971 })?;
1972 let data = self
1973 .graphql(
1974 graphql::REPOSITORY,
1975 json!({"owner":repository.owner,"name":repository.name}),
1976 )
1977 .await?;
1978 let node = data
1979 .get("repository")
1980 .filter(|value| !value.is_null())
1981 .ok_or_else(|| SourceError::Refused {
1982 message: format!(
1983 "GitHub repository {}/{} was not found or is not visible to the token",
1984 repository.owner, repository.name
1985 ),
1986 })?;
1987 let id = required_str(node, "id")?.to_owned();
1988 *self.repository_cache()? = Some(id.clone());
1989 Ok(id)
1990 }
1991
1992 /// This process's own record of the destination repository's node id.
1993 fn repository_cache(&self) -> Result<std::sync::MutexGuard<'_, Option<String>>, SourceError> {
1994 self.repository_cache
1995 .lock()
1996 .map_err(|_| SourceError::Unavailable {
1997 message: "this source's record of the destination repository was left \
1998 inconsistent by an earlier failure; next: run the command again"
1999 .into(),
2000 })
2001 }
2002
2003 /// Create or update one board item, whichever kind it is.
2004 async fn write_item(
2005 &self,
2006 incoming: &Incoming<'_>,
2007 target: Option<&NativeId>,
2008 depends_on: &[DependencyEdge],
2009 ) -> Result<NativeId, SourceError> {
2010 // Refused before anything is read or written: a task or a project titled the way
2011 // this board spells a document would land as an issue this same source reads back
2012 // as a document, so the field this destination cannot carry is named rather than
2013 // written and silently reclassified.
2014 if let Written::Work(kind, _) = incoming.written
2015 && incoming.title.starts_with(DESIGN_TITLE_PREFIX)
2016 {
2017 return Err(SourceError::Refused {
2018 message: format!(
2019 "the title of this {} begins {DESIGN_TITLE_PREFIX:?}, which is how source {} \
2020 spells a document, so it would read back as one rather than as a {}; \
2021 retitle it, or copy it as a document",
2022 kind.marker(),
2023 self.name,
2024 kind.marker()
2025 ),
2026 });
2027 }
2028 let board = self.board().await?;
2029 let status_target = incoming
2030 .written
2031 .status()
2032 .map(|status| self.resolved_target(status.category))
2033 .transpose()?;
2034 let column = match (incoming.written.status(), status_target.as_ref()) {
2035 (Some(status), Some(target)) => self.column_for(&board, status, target)?,
2036 _ => None,
2037 };
2038 let existing = target
2039 .map(|target| {
2040 board
2041 .items
2042 .iter()
2043 .find(|item| item.id == *target)
2044 .ok_or_else(|| SourceError::Refused {
2045 message: format!("GitHub destination item {} was not found", target.0),
2046 })
2047 })
2048 .transpose()?;
2049 let content_kind = existing.map_or(ContentKind::Issue, |item| item.content_kind);
2050 if content_kind == ContentKind::DraftIssue {
2051 if let (Some(StatusTarget::Closed(_)), Some(status)) =
2052 (status_target.as_ref(), incoming.written.status())
2053 {
2054 return Err(SourceError::Refused {
2055 message: format!(
2056 "status {} of source {} closes the item's issue, and GitHub draft items \
2057 have no open or closed state",
2058 category_name(status.category),
2059 self.name
2060 ),
2061 });
2062 }
2063 if incoming.parent.is_some() {
2064 return Err(SourceError::Refused {
2065 message: "GitHub draft items cannot be a project's sub-issue".into(),
2066 });
2067 }
2068 }
2069 match existing {
2070 Some(item) if content_kind == ContentKind::Issue => {
2071 if item.labels != incoming.labels {
2072 return Err(SourceError::Refused {
2073 message: "GitHub issue labels differ from the labels being written".into(),
2074 });
2075 }
2076 }
2077 _ => {
2078 if !incoming.labels.is_empty() {
2079 return Err(SourceError::Refused {
2080 message: "GitHub items created by this destination carry no labels".into(),
2081 });
2082 }
2083 }
2084 }
2085
2086 let own_repository = match existing {
2087 Some(item) => item.own_repository.clone(),
2088 None => self
2089 .repository
2090 .as_ref()
2091 .map(|repository| Repository::try_from(repository.origin()))
2092 .transpose()
2093 .map_err(|message| SourceError::Config { message })?,
2094 };
2095 let (native, fallback) = self
2096 .partition_edges(&board, incoming.written.kind(), content_kind, depends_on)
2097 .await?;
2098 let slot = slot_metadata(incoming, own_repository.as_ref(), &fallback);
2099 let body = compose_body(incoming.content, &slot)?;
2100 // Read before anything is created, for the reason the field below is: a value
2101 // this destination cannot store has to refuse, and refusing after `createIssue`
2102 // would leave an issue behind that nothing asked for. The engine writes a
2103 // qualified id here; a caller handing this key anything else is told so rather
2104 // than having it silently stored as no origin at all.
2105 // llmlint: ignore[boundary_inputs_validated, changed_behavior_has_e2e] The qualified id's syntax is the engine's and not this plugin's to police: `GlobalId` is deliberately absent from the contract crate because a plugin never sees a qualified id (AGENTS.md), no plugin crate may depend on the engine to parse one, and `docs/metadata.md` says the contents of this key are what no plugin constructs or interprets. What this boundary owns is whether the value is a string its text field can hold, and that is what it checks.
2106 let origin = match incoming.metadata.get(ORIGIN_KEY) {
2107 None => "",
2108 Some(Value::String(origin)) => origin.as_str(),
2109 Some(other) => {
2110 return Err(SourceError::Refused {
2111 message: format!(
2112 "{ORIGIN_KEY} holds a qualified id spelled as a string, and this item's \
2113 is {other}"
2114 ),
2115 });
2116 }
2117 };
2118 // Resolved before anything is created: a board that cannot carry the copy origin
2119 // has to refuse the write, and refusing it after `createIssue` would leave an
2120 // issue behind that nothing asked for.
2121 let origin_field = match Board::field(&board.fields, ORIGIN_FIELD)? {
2122 Some(field) => {
2123 if required_str(field, "__typename")? != "ProjectV2Field" {
2124 return Err(SourceError::Refused {
2125 message: format!(
2126 "GitHub board source-owned {ORIGIN_FIELD} field is not a text field"
2127 ),
2128 });
2129 }
2130 Some(required_str(field, "id")?.to_owned())
2131 }
2132 None if incoming.metadata.contains_key(ORIGIN_KEY) => {
2133 return Err(SourceError::Refused {
2134 message: format!(
2135 "GitHub board has no source-owned {ORIGIN_FIELD} text field, and the \
2136 item carries {ORIGIN_KEY}; add a text field named {ORIGIN_FIELD} to \
2137 the board"
2138 ),
2139 });
2140 }
2141 None => None,
2142 };
2143
2144 let (content_id, item_id, url) = match existing {
2145 Some(item) => {
2146 self.update_existing(item, incoming, &body, status_target.as_ref())
2147 .await?;
2148 (item.id.clone(), item.item_id.clone(), item.url.clone())
2149 }
2150 None => {
2151 self.create_and_file_issue(&board, incoming, &body, status_target.as_ref())
2152 .await?
2153 }
2154 };
2155
2156 // Creating an item here is several calls — `createIssue`, `addProjectV2ItemById`,
2157 // then each board field, the parent and the dependencies — and GitHub can fail at
2158 // any of them. Everything this source can refuse *before* the first of those is
2159 // already checked above, so what is left is GitHub itself failing part way. When it
2160 // does over an item this call created, the issue is taken back: a write that
2161 // refused must not leave an item behind that nobody asked for, and one that does
2162 // makes the retry create a second.
2163 let landed = self
2164 .finish_write(
2165 &board,
2166 incoming,
2167 &content_id,
2168 &item_id,
2169 content_kind,
2170 existing,
2171 origin_field.as_deref(),
2172 origin,
2173 column,
2174 &native,
2175 )
2176 .await;
2177 if let Err(error) = landed {
2178 if existing.is_none() {
2179 // Best effort, and the write's own failure is what the caller is told: a
2180 // refusal naming the tidy-up would hide why the write failed at all.
2181 let _ = self.delete_issue(&content_id).await;
2182 }
2183 return Err(error);
2184 }
2185
2186 // So the rest of this command reads what it just did rather than what the board
2187 // said before it. See `remember_written` for which half takes it.
2188 let remembered = Resolved {
2189 item_id,
2190 id: content_id.clone(),
2191 content_kind,
2192 kind: incoming.written.kind(),
2193 title: incoming.title.to_owned(),
2194 // The visible half of the body this write composed, split back off it the
2195 // way a read splits it — so what this record reports is what a read of the
2196 // same issue reports, rather than the person's text with the metadata slot
2197 // still on the end of it.
2198 body: metadata_body(body.clone())?.0,
2199 // A document has no status of its own; what it reads back as is whatever
2200 // the issue's own state says, which is what a re-read reports.
2201 status: incoming
2202 .written
2203 .status()
2204 .cloned()
2205 .unwrap_or_else(|| Status {
2206 category: StatusCategory::Unknown,
2207 name: "Open".to_owned(),
2208 }),
2209 labels: incoming.labels.to_vec(),
2210 parent: incoming.parent.cloned(),
2211 origin: (!origin.is_empty()).then(|| origin.to_owned()),
2212 // In the update path this is the item's own url, read off `existing` where the
2213 // tuple above was bound, so one expression serves both halves.
2214 url,
2215 created_at: existing.and_then(|item| item.created_at),
2216 updated_at: existing.and_then(|item| item.updated_at),
2217 own_repository,
2218 repositories: incoming.repositories.to_vec(),
2219 slot,
2220 };
2221 self.remember_written(remembered, existing.is_none())?;
2222 Ok(content_id)
2223 }
2224
2225 /// Everything a write does after the item exists: its board fields, its parent, and
2226 /// its dependencies.
2227 ///
2228 /// Split out of `write_item` so there is one place a failure past the point of no
2229 /// return is caught, rather than a tidy-up repeated at each `?` above.
2230 // llmlint: ignore[suppressions_justified] This is the tail of `write_item` lifted out
2231 // so there is one place a failure past the point of no return is caught, and its
2232 // arguments are exactly the values that tail already had in scope. Bundling them into a
2233 // struct would describe no concept — it would be "the arguments of this function" — and
2234 // would put the whole of `write_item`'s locals behind one more indirection.
2235 #[allow(clippy::too_many_arguments)]
2236 async fn finish_write(
2237 &self,
2238 board: &Board,
2239 incoming: &Incoming<'_>,
2240 content_id: &NativeId,
2241 item_id: &str,
2242 content_kind: ContentKind,
2243 existing: Option<&Resolved>,
2244 origin_field: Option<&str>,
2245 origin: &str,
2246 column: Option<(String, String)>,
2247 native: &[String],
2248 ) -> Result<(), SourceError> {
2249 if let Some(field_id) = origin_field {
2250 self.set_item_field(&board.id, item_id, field_id, json!({"text":origin}))
2251 .await?;
2252 }
2253
2254 if let Some((field_id, option_id)) = column {
2255 self.set_item_field(
2256 &board.id,
2257 item_id,
2258 &field_id,
2259 json!({"singleSelectOptionId":option_id}),
2260 )
2261 .await?;
2262 }
2263
2264 if content_kind == ContentKind::Issue {
2265 self.reparent(
2266 existing.and_then(|item| item.parent.clone()),
2267 content_id,
2268 incoming.parent,
2269 )
2270 .await?;
2271 // A document takes part in no dependency graph, so writing one neither reads
2272 // nor changes the issue's own `blockedBy` relationships. Reconciling them
2273 // against the empty list a document write carries would *delete* whatever
2274 // relationships a person had made on that issue, which is a write nobody
2275 // asked for.
2276 if incoming.written.kind() != BoardKind::Document {
2277 self.reconcile_blocked_by(content_id, native).await?;
2278 }
2279 }
2280 Ok(())
2281 }
2282
2283 /// Delete one issue, which takes its board item with it.
2284 async fn delete_issue(&self, id: &NativeId) -> Result<(), SourceError> {
2285 let data = self
2286 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
2287 .await?;
2288 data.pointer("/deleteIssue/repository")
2289 .filter(|value| !value.is_null())
2290 .ok_or_else(|| SourceError::Malformed {
2291 message: "GitHub issue deletion returned no repository".into(),
2292 })?;
2293 self.forget(id)?;
2294 Ok(())
2295 }
2296
2297 /// Remove one item this copy created, so a copy that could not finish leaves the board
2298 /// as it found it.
2299 ///
2300 /// Deleting the issue takes its board item with it, so there is no second mutation to
2301 /// keep in step. An id the board does not hold is not an error: the item is already
2302 /// gone, which is the state this asks for.
2303 async fn delete_item(&self, id: &NativeId) -> Result<(), SourceError> {
2304 let board = self.board().await?;
2305 let Some(item) = board.items.iter().find(|item| item.id == *id) else {
2306 return Ok(());
2307 };
2308 if item.content_kind == ContentKind::DraftIssue {
2309 return Err(SourceError::Refused {
2310 message: format!(
2311 "GitHub item {} is a draft, and this source removes an item by deleting \
2312 its issue; next: remove it from the board by hand",
2313 id.0
2314 ),
2315 });
2316 }
2317 let data = self
2318 .graphql(graphql::DELETE_ISSUE, json!({"input":{"issueId":id.0}}))
2319 .await?;
2320 data.pointer("/deleteIssue/repository")
2321 .filter(|value| !value.is_null())
2322 .ok_or_else(|| SourceError::Malformed {
2323 message: "GitHub issue deletion returned no repository".into(),
2324 })?;
2325 self.forget(id)?;
2326 Ok(())
2327 }
2328
2329 /// Which far ends this item's own `blockedBy` relationship holds, and which it cannot.
2330 async fn partition_edges(
2331 &self,
2332 board: &Board,
2333 near_kind: BoardKind,
2334 near_content: ContentKind,
2335 depends_on: &[DependencyEdge],
2336 ) -> Result<(Vec<String>, Vec<DependencyEdge>), SourceError> {
2337 let mut native = Vec::new();
2338 let mut fallback = Vec::new();
2339 for edge in depends_on {
2340 let same_source = edge
2341 .to
2342 .source()
2343 .is_none_or(|source| source == self.name.as_str());
2344 // A qualified id's source segment runs to its *first* colon — `GlobalId` and
2345 // `DependencyEndpoint::source` both read it that way — and a native id may hold
2346 // colons of its own, so the far end is everything after that one separator.
2347 // Splitting at the last would truncate `work:urn:task:7` to `7`.
2348 let far_id = if edge.to.is_qualified() {
2349 edge.to
2350 .id()
2351 .split_once(':')
2352 .map_or(edge.to.id(), |(_, native)| native)
2353 } else {
2354 edge.to.id()
2355 };
2356 let far = if same_source {
2357 Some(
2358 board
2359 .items
2360 .iter()
2361 .find(|item| item.id.0 == far_id)
2362 .ok_or_else(|| SourceError::Refused {
2363 message: format!("GitHub dependency item {far_id} was not found"),
2364 })?,
2365 )
2366 } else {
2367 None
2368 };
2369 // The caller says which kind the far end is, and this board holds the far end
2370 // itself, so a disagreement is settled here rather than stored: recorded, the
2371 // wrong kind would read back as a cross-level edge that never existed; written
2372 // natively, it would name a relationship of a different level than the caller
2373 // asked for.
2374 //
2375 // A far end this board holds as a *document* fails the same comparison and is
2376 // refused by the same sentence: `ItemKind` has no document variant because
2377 // nothing may point at one, so no caller can name it correctly and the refusal
2378 // is the only honest answer.
2379 if let Some(disagreeing) = far.filter(|far| far.kind != BoardKind::Work(edge.to.kind)) {
2380 return Err(SourceError::Refused {
2381 message: format!(
2382 "GitHub dependency item {far_id} is a {} of this board, and this item \
2383 names it as a {}; record the kind it is",
2384 disagreeing.kind.describes(),
2385 edge.to.kind.marker()
2386 ),
2387 });
2388 }
2389 // A draft has neither `blockedBy` nor `blocking`, so no edge of one is native
2390 // however the far end is spelled — and one classified native here would be
2391 // written nowhere at all, because a draft's native reconciliation never runs.
2392 let native_here = near_content == ContentKind::Issue
2393 && far.is_some_and(|far| {
2394 far.content_kind == ContentKind::Issue
2395 && BoardKind::Work(edge.to.kind) == near_kind
2396 });
2397 if native_here {
2398 native.push(far_id.to_owned());
2399 } else {
2400 fallback.push(edge.clone());
2401 }
2402 }
2403 Ok((native, fallback))
2404 }
2405
2406 async fn update_existing(
2407 &self,
2408 item: &Resolved,
2409 incoming: &Incoming<'_>,
2410 body: &Option<String>,
2411 status_target: Option<&StatusTarget>,
2412 ) -> Result<(), SourceError> {
2413 let title = incoming.written_title();
2414 let (operation, input, pointer) = match item.content_kind {
2415 ContentKind::DraftIssue => (
2416 graphql::UPDATE_DRAFT,
2417 json!({"draftIssueId":item.id.0,"title":title,"body":body}),
2418 "/updateProjectV2DraftIssue/draftIssue",
2419 ),
2420 ContentKind::Issue => (
2421 graphql::UPDATE_ISSUE,
2422 json!({"id":item.id.0,"title":title,"body":body,
2423 "stateInput":state_input(status_target)}),
2424 "/updateIssue/issue",
2425 ),
2426 };
2427 let data = self.graphql(operation, json!({"input":input})).await?;
2428 let returned = data
2429 .pointer(pointer)
2430 .ok_or_else(|| SourceError::Malformed {
2431 message: "GitHub item update returned no item".into(),
2432 })?;
2433 if required_str(returned, "id")? != item.id.0 {
2434 return Err(SourceError::Malformed {
2435 message: "GitHub item update returned the wrong item".into(),
2436 });
2437 }
2438 Ok(())
2439 }
2440
2441 /// Creates one issue, files it on the board, and closes it when the status says so.
2442 ///
2443 /// Three calls rather than one: `createIssue` needs a repository and answers with an
2444 /// issue that is on no board, `addProjectV2ItemById` is what puts it there, and a
2445 /// closed status is a state of the issue rather than a field of the board item.
2446 /// Creates the issue, files it on the board, and reports what a read of it would say:
2447 /// its content id, its board item id, and the web address GitHub gave it.
2448 ///
2449 /// The address comes back here because this is the only place it is known before
2450 /// GitHub's own board read catches up — an item this run created answers the reads
2451 /// that follow it out of the record below, and one remembered without its address
2452 /// would report no location for the rest of the run.
2453 async fn create_and_file_issue(
2454 &self,
2455 board: &Board,
2456 incoming: &Incoming<'_>,
2457 body: &Option<String>,
2458 status_target: Option<&StatusTarget>,
2459 ) -> Result<(NativeId, String, Option<String>), SourceError> {
2460 let repository_id = self.repository_id().await?;
2461 let data = self
2462 .graphql(
2463 graphql::CREATE_ISSUE,
2464 json!({"input":{
2465 "repositoryId":repository_id,"title":incoming.written_title(),"body":body
2466 }}),
2467 )
2468 .await?;
2469 let created = data
2470 .pointer("/createIssue/issue")
2471 .filter(|value| !value.is_null())
2472 .ok_or_else(|| SourceError::Malformed {
2473 message: "GitHub issue creation returned no issue".into(),
2474 })?;
2475 let content_id = NativeId(required_str(created, "id")?.to_owned());
2476 // Optional although GitHub's schema makes it non-null: the issue exists by now, so
2477 // a response without it is not worth failing a landed write over — the item simply
2478 // reports no location until the board read catches up, which is what it did before.
2479 let url = optional_str(created, "url")?.map(str::to_owned);
2480 // The issue exists from here on, so a failure filing it on the board takes it
2481 // back: an issue in the repository that is on no board is an item nobody asked for
2482 // and nothing here would find again.
2483 let added = match self
2484 .graphql(
2485 graphql::ADD_TO_BOARD,
2486 json!({"input":{"projectId":board.id,"contentId":content_id.0}}),
2487 )
2488 .await
2489 {
2490 Ok(added) => added,
2491 Err(error) => {
2492 let _ = self.delete_issue(&content_id).await;
2493 return Err(error);
2494 }
2495 };
2496 let item = added
2497 .pointer("/addProjectV2ItemById/item")
2498 .filter(|value| !value.is_null())
2499 .ok_or_else(|| SourceError::Malformed {
2500 message: "GitHub board addition returned no project item".into(),
2501 })?;
2502 if let Some(StatusTarget::Closed(_)) = status_target {
2503 let closed = self
2504 .graphql(
2505 graphql::UPDATE_ISSUE,
2506 json!({"input":{"id":content_id.0,"stateInput":state_input(status_target)}}),
2507 )
2508 .await?;
2509 let returned =
2510 closed
2511 .pointer("/updateIssue/issue")
2512 .ok_or_else(|| SourceError::Malformed {
2513 message: "GitHub item update returned no item".into(),
2514 })?;
2515 if required_str(returned, "id")? != content_id.0 {
2516 return Err(SourceError::Malformed {
2517 message: "GitHub item update returned the wrong item".into(),
2518 });
2519 }
2520 }
2521 Ok((content_id, required_str(item, "id")?.to_owned(), url))
2522 }
2523
2524 /// Move one issue under the project it now belongs to, or out of the one it left.
2525 async fn reparent(
2526 &self,
2527 held: Option<NativeId>,
2528 child: &NativeId,
2529 wanted: Option<&NativeId>,
2530 ) -> Result<(), SourceError> {
2531 if held.as_ref() == wanted {
2532 return Ok(());
2533 }
2534 if let Some(held) = &held {
2535 self.sub_issue(graphql::REMOVE_SUB_ISSUE, held, child, "removeSubIssue")
2536 .await?;
2537 }
2538 if let Some(wanted) = wanted {
2539 self.sub_issue(graphql::ADD_SUB_ISSUE, wanted, child, "addSubIssue")
2540 .await?;
2541 }
2542 Ok(())
2543 }
2544
2545 async fn sub_issue(
2546 &self,
2547 operation: &str,
2548 parent: &NativeId,
2549 child: &NativeId,
2550 root: &str,
2551 ) -> Result<(), SourceError> {
2552 let data = self
2553 .graphql(
2554 operation,
2555 json!({"input":{"issueId":parent.0,"subIssueId":child.0}}),
2556 )
2557 .await?;
2558 let issue =
2559 data.pointer(&format!("/{root}/issue"))
2560 .ok_or_else(|| SourceError::Malformed {
2561 message: "GitHub sub-issue update returned no issue".into(),
2562 })?;
2563 let sub =
2564 data.pointer(&format!("/{root}/subIssue"))
2565 .ok_or_else(|| SourceError::Malformed {
2566 message: "GitHub sub-issue update returned no sub-issue".into(),
2567 })?;
2568 if required_str(issue, "id")? != parent.0 || required_str(sub, "id")? != child.0 {
2569 return Err(SourceError::Malformed {
2570 message: "GitHub sub-issue update returned the wrong issues".into(),
2571 });
2572 }
2573 Ok(())
2574 }
2575
2576 async fn reconcile_blocked_by(
2577 &self,
2578 content_id: &NativeId,
2579 native: &[String],
2580 ) -> Result<(), SourceError> {
2581 let current = self.native_dependency_ids(content_id).await?;
2582 for (operation, far_id) in current
2583 .iter()
2584 .filter(|id| !native.contains(id))
2585 .map(|id| (graphql::REMOVE_BLOCKED_BY, id))
2586 .chain(
2587 native
2588 .iter()
2589 .filter(|id| !current.contains(id))
2590 .map(|id| (graphql::ADD_BLOCKED_BY, id)),
2591 )
2592 {
2593 let data = self
2594 .graphql(
2595 operation,
2596 json!({"input":{"issueId":content_id.0,"blockingIssueId":far_id}}),
2597 )
2598 .await?;
2599 let root = if operation == graphql::ADD_BLOCKED_BY {
2600 "addBlockedBy"
2601 } else {
2602 "removeBlockedBy"
2603 };
2604 let issue =
2605 data.pointer(&format!("/{root}/issue"))
2606 .ok_or_else(|| SourceError::Malformed {
2607 message: "GitHub dependency update returned no issue".into(),
2608 })?;
2609 let blocker = data
2610 .pointer(&format!("/{root}/blockingIssue"))
2611 .ok_or_else(|| SourceError::Malformed {
2612 message: "GitHub dependency update returned no blocking issue".into(),
2613 })?;
2614 if required_str(issue, "id")? != content_id.0 || required_str(blocker, "id")? != far_id
2615 {
2616 return Err(SourceError::Malformed {
2617 message: "GitHub dependency update returned the wrong issues".into(),
2618 });
2619 }
2620 }
2621 Ok(())
2622 }
2623}
2624
2625/// The board, and every item on it this source reports.
2626#[derive(Clone)]
2627struct Board {
2628 id: String,
2629 fields: Value,
2630 items: Vec<Resolved>,
2631}
2632
2633impl Board {
2634 fn field<'a>(fields: &'a Value, name: &str) -> Result<Option<&'a Value>, SourceError> {
2635 complete_connection(fields, "project fields")?;
2636 let nodes = fields
2637 .get("nodes")
2638 .and_then(Value::as_array)
2639 .ok_or_else(|| SourceError::Malformed {
2640 message: "GitHub project fields.nodes is not an array".into(),
2641 })?;
2642 Ok(nodes
2643 .iter()
2644 .find(|field| field.get("name").and_then(Value::as_str) == Some(name)))
2645 }
2646}
2647
2648/// One board item, resolved into everything this source reports about it.
2649#[derive(Clone)]
2650struct Resolved {
2651 item_id: String,
2652 id: NativeId,
2653 content_kind: ContentKind,
2654 kind: BoardKind,
2655 title: String,
2656 body: Option<String>,
2657 status: Status,
2658 labels: Vec<Label>,
2659 parent: Option<NativeId>,
2660 // llmlint: ignore[invalid_states_unrepresentable] The write side's reason, read back: this is the engine's qualified id, taken out of a board text field and handed on untouched. A newtype here would have this plugin define the syntax of an id `docs/metadata.md` says no plugin ever constructs or interprets.
2661 origin: Option<String>,
2662 url: Option<String>,
2663 created_at: Option<DateTime<Utc>>,
2664 updated_at: Option<DateTime<Utc>>,
2665 own_repository: Option<Repository>,
2666 repositories: Vec<Repository>,
2667 slot: BTreeMap<String, Value>,
2668}
2669
2670impl Resolved {
2671 /// The metadata a caller sees: their own keys, plus the copy origin this source keeps
2672 /// in a field of its own, and none of the three keys that are only an encoding.
2673 fn metadata(&self) -> BTreeMap<String, Value> {
2674 let mut metadata = self.slot.clone();
2675 metadata.remove(Repository::METADATA_KEY);
2676 metadata.remove(DependencyEdge::RECORDED_KEY);
2677 metadata.remove(ItemKind::METADATA_KEY);
2678 if let Some(origin) = &self.origin {
2679 metadata.insert(ORIGIN_KEY.to_owned(), Value::String(origin.clone()));
2680 }
2681 metadata
2682 }
2683
2684 /// Where this item is, as a link a reader can open.
2685 ///
2686 /// A board is a hosted place and every issue on it has a web address, so that address
2687 /// is what "where is this?" means here — and [`Location::Url`] is what says which kind
2688 /// of place it is, so a reader knows to open it rather than to read a file out. It
2689 /// does not replace or derive from `url`: the field goes on reporting exactly what it
2690 /// reported before, and this says what that address *is*.
2691 ///
2692 /// An item GitHub gave no `url` for — a draft has none — reports no location at all
2693 /// rather than a third variant, which is the contract's "the source did not say". An
2694 /// issue this run created is not one of those: its address comes back from the
2695 /// creating mutation, so it is somewhere a reader can open from the moment it exists
2696 /// rather than from whenever the board read catches up.
2697 fn location(&self) -> Option<Location> {
2698 self.url.clone().map(Location::Url)
2699 }
2700
2701 fn task(&self) -> Task {
2702 Task {
2703 id: self.id.clone(),
2704 title: self.title.clone(),
2705 content: self.body.clone(),
2706 status: self.status.clone(),
2707 labels: self.labels.clone(),
2708 project: self.parent.clone(),
2709 url: self.url.clone(),
2710 location: self.location(),
2711 created_at: self.created_at,
2712 updated_at: self.updated_at,
2713 metadata: self.metadata(),
2714 repositories: self.repositories.clone(),
2715 }
2716 }
2717
2718 fn project(&self) -> Project {
2719 Project {
2720 id: self.id.clone(),
2721 title: self.title.clone(),
2722 content: self.body.clone(),
2723 status: self.status.clone(),
2724 labels: self.labels.clone(),
2725 url: self.url.clone(),
2726 location: self.location(),
2727 created_at: self.created_at,
2728 updated_at: self.updated_at,
2729 metadata: self.metadata(),
2730 repositories: self.repositories.clone(),
2731 }
2732 }
2733
2734 /// The same issue as a document: the project it is filed under, and no status and no
2735 /// dependencies, because a document is not work.
2736 fn document(&self) -> Document {
2737 Document {
2738 id: self.id.clone(),
2739 title: self.title.clone(),
2740 content: self.body.clone(),
2741 project: self.parent.clone(),
2742 labels: self.labels.clone(),
2743 url: self.url.clone(),
2744 location: self.location(),
2745 created_at: self.created_at,
2746 updated_at: self.updated_at,
2747 metadata: self.metadata(),
2748 repositories: self.repositories.clone(),
2749 }
2750 }
2751}
2752
2753/// What one write is, and the status that comes with being it.
2754///
2755/// One value rather than a [`BoardKind`] beside an `Option<Status>`: a document has no
2756/// status and a task or a project always has one, so "a document carrying a status" and
2757/// "a task carrying none" are states a write cannot be in rather than states every use
2758/// site below has to defend against.
2759enum Written<'a> {
2760 /// A document, which is not work and so has no status at all.
2761 Document,
2762 /// A task or a project, and the status it is being written with.
2763 Work(ItemKind, &'a Status),
2764}
2765
2766impl Written<'_> {
2767 /// Which of the board's three kinds this write is.
2768 const fn kind(&self) -> BoardKind {
2769 match self {
2770 Self::Document => BoardKind::Document,
2771 Self::Work(kind, _) => BoardKind::Work(*kind),
2772 }
2773 }
2774
2775 /// The status this write carries. A document carries none, so a write of one says
2776 /// nothing about the issue's open or closed state and selects no board `Status`
2777 /// option.
2778 const fn status(&self) -> Option<&Status> {
2779 match self {
2780 Self::Document => None,
2781 Self::Work(_, status) => Some(status),
2782 }
2783 }
2784}
2785
2786/// The item being written, in the one shape all three write methods reach.
2787struct Incoming<'a> {
2788 written: Written<'a>,
2789 /// The title a person wrote. A document's goes onto the issue with
2790 /// [`DESIGN_TITLE_PREFIX`] put back, so a round trip returns the title that went in.
2791 title: &'a str,
2792 content: Option<&'a str>,
2793 labels: &'a [Label],
2794 metadata: &'a BTreeMap<String, Value>,
2795 repositories: &'a [Repository],
2796 parent: Option<&'a NativeId>,
2797}
2798
2799impl Incoming<'_> {
2800 /// The title this write puts on the issue.
2801 fn written_title(&self) -> String {
2802 match self.written {
2803 Written::Document => format!("{DESIGN_TITLE_PREFIX}{}", self.title),
2804 Written::Work(..) => self.title.to_owned(),
2805 }
2806 }
2807}
2808
2809#[derive(Clone, Copy, PartialEq, Eq)]
2810enum ContentKind {
2811 DraftIssue,
2812 Issue,
2813}
2814
2815/// What one board issue is: a document, or the work an [`ItemKind`] names.
2816///
2817/// A type of this source's own rather than an `ItemKind` with a third variant, because
2818/// `ItemKind` names what a dependency endpoint points at and nothing may point at a
2819/// document — the contract keeps a document out of that enum deliberately. Holding the
2820/// board's three answers in one value is what makes every place that asks "which is this?"
2821/// answer all three, rather than a `document: bool` beside a `kind` that means nothing for
2822/// two thirds of the board.
2823#[derive(Clone, Copy, PartialEq, Eq)]
2824enum BoardKind {
2825 /// An issue whose title begins [`DESIGN_TITLE_PREFIX`].
2826 Document,
2827 /// Every other issue, and every draft.
2828 Work(ItemKind),
2829}
2830
2831impl BoardKind {
2832 /// How a refusal names this kind to the person reading it.
2833 const fn describes(self) -> &'static str {
2834 match self {
2835 Self::Document => "document",
2836 Self::Work(kind) => kind.marker(),
2837 }
2838 }
2839}
2840
2841/// Whether `labels` satisfies `filter`, matching by name, case-insensitively.
2842///
2843/// This is the local Markdown source's `labels_match`, spelled the same way on purpose:
2844/// the shared cross-source journeys assert one answer to one question, so two sources
2845/// that disagree about what "carries the label bug" means fail them.
2846fn labels_match(labels: &[Label], filter: &LabelFilter) -> bool {
2847 let holds = |name: &String| {
2848 labels
2849 .iter()
2850 .any(|label| label.name.eq_ignore_ascii_case(name))
2851 };
2852 (filter.any_of.is_empty() || filter.any_of.iter().any(holds))
2853 && filter.all_of.iter().all(holds)
2854 && !filter.none_of.iter().any(holds)
2855}
2856
2857/// Whether `category` is one of `statuses`. An empty list is unfiltered rather than
2858/// "keeps nothing", which is what lets a `Vec<StatusCategory>` spell no filter at all.
2859fn status_matches(category: StatusCategory, statuses: &[StatusCategory]) -> bool {
2860 statuses.is_empty() || statuses.contains(&category)
2861}
2862
2863/// Whether `title`/`content` satisfies `query`, matching case-insensitively.
2864///
2865/// `content` is the item's own prose — the body with this source's trailing metadata
2866/// comment already taken off — so a search never matches an encoding the author of the
2867/// issue never wrote.
2868fn text_matches(title: &str, content: Option<&str>, query: &TextQuery) -> bool {
2869 let terms = query.terms.to_lowercase();
2870 let in_title = title.to_lowercase().contains(&terms);
2871 let in_content = content.is_some_and(|body| body.to_lowercase().contains(&terms));
2872 match query.fields {
2873 TextFields::Title => in_title,
2874 TextFields::Content => in_content,
2875 TextFields::TitleOrContent => in_title || in_content,
2876 }
2877}
2878
2879fn task_matches(task: &Task, query: &TaskQuery) -> bool {
2880 labels_match(&task.labels, &query.labels)
2881 && status_matches(task.status.category, &query.statuses)
2882 && match &query.project {
2883 ProjectFilter::Any => true,
2884 ProjectFilter::Orphans => task.project.is_none(),
2885 ProjectFilter::Is(id) => task.project.as_ref() == Some(id),
2886 }
2887 && query
2888 .text
2889 .as_ref()
2890 .is_none_or(|text| text_matches(&task.title, task.content.as_deref(), text))
2891}
2892
2893fn project_matches(project: &Project, query: &ProjectQuery) -> bool {
2894 labels_match(&project.labels, &query.labels)
2895 && status_matches(project.status.category, &query.statuses)
2896 && query
2897 .text
2898 .as_ref()
2899 .is_none_or(|text| text_matches(&project.title, project.content.as_deref(), text))
2900}
2901
2902/// The same three predicates a task query carries, minus the status filter.
2903///
2904/// A document is not work, so it has no status for one to compare against and the query
2905/// type carries none. The project predicate is the same one — a design issue filed under a
2906/// project issue is in that project, and one filed under nothing is in none — so it is
2907/// spelled the same way here rather than answered differently.
2908fn document_matches(document: &Document, query: &DocumentQuery) -> bool {
2909 labels_match(&document.labels, &query.labels)
2910 && match &query.project {
2911 ProjectFilter::Any => true,
2912 ProjectFilter::Orphans => document.project.is_none(),
2913 ProjectFilter::Is(id) => document.project.as_ref() == Some(id),
2914 }
2915 && query
2916 .text
2917 .as_ref()
2918 .is_none_or(|text| text_matches(&document.title, document.content.as_deref(), text))
2919}
2920
2921#[async_trait::async_trait]
2922impl TaskSource for GitHubProjectsSource {
2923 fn kind(&self) -> &'static str {
2924 KIND
2925 }
2926 fn capabilities(&self) -> Capabilities {
2927 Capabilities {
2928 projects: Support::Native,
2929 documents: Support::Native,
2930 orphan_tasks: Support::Native,
2931 filter_by_label: Support::Native,
2932 filter_by_status: Support::Native,
2933 search_title: Support::Native,
2934 search_content: Support::Native,
2935 task_dependencies: DependencySupport::BothDirections,
2936 project_dependencies: DependencySupport::BothDirections,
2937 max_page_size: MAX_PAGE_SIZE,
2938 }
2939 }
2940 async fn health(&self) -> Result<Health, SourceError> {
2941 let board = self.board_page(None, 1).await?;
2942 Ok(Health {
2943 reachable: true,
2944 detail: Some(format!(
2945 "reading GitHub project {}/{} ({})",
2946 self.owner,
2947 self.project_number,
2948 required_str(&board, "title")?
2949 )),
2950 })
2951 }
2952 async fn get_task(&self, id: &NativeId) -> Result<Option<Task>, SourceError> {
2953 Ok(self
2954 .board()
2955 .await?
2956 .items
2957 .iter()
2958 .find(|item| item.id == *id && item.kind == BoardKind::Work(ItemKind::Task))
2959 .map(Resolved::task))
2960 }
2961 async fn get_project(&self, id: &NativeId) -> Result<Option<Project>, SourceError> {
2962 Ok(self
2963 .board()
2964 .await?
2965 .items
2966 .iter()
2967 .find(|item| item.id == *id && item.kind == BoardKind::Work(ItemKind::Project))
2968 .map(Resolved::project))
2969 }
2970 async fn query_tasks(
2971 &self,
2972 query: &TaskQuery,
2973 page: &PageRequest,
2974 ) -> Result<Page<Task>, SourceError> {
2975 validate_page(page)?;
2976 // Filtered before paged: a page of a filtered result is a page of the survivors,
2977 // never the survivors of a page.
2978 let tasks = self
2979 .board()
2980 .await?
2981 .items
2982 .iter()
2983 .filter(|item| item.kind == BoardKind::Work(ItemKind::Task))
2984 .map(Resolved::task)
2985 .filter(|task| task_matches(task, query))
2986 .collect();
2987 Ok(offset_page(
2988 tasks,
2989 numeric_cursor(page.cursor.as_ref())?,
2990 page.limit.min(MAX_PAGE_SIZE) as usize,
2991 ))
2992 }
2993 async fn query_projects(
2994 &self,
2995 query: &ProjectQuery,
2996 page: &PageRequest,
2997 ) -> Result<Page<Project>, SourceError> {
2998 validate_page(page)?;
2999 let projects = self
3000 .board()
3001 .await?
3002 .items
3003 .iter()
3004 .filter(|item| item.kind == BoardKind::Work(ItemKind::Project))
3005 .map(Resolved::project)
3006 .filter(|project| project_matches(project, query))
3007 .collect();
3008 Ok(offset_page(
3009 projects,
3010 numeric_cursor(page.cursor.as_ref())?,
3011 page.limit.min(MAX_PAGE_SIZE) as usize,
3012 ))
3013 }
3014 async fn get_document(&self, id: &NativeId) -> Result<Option<Document>, SourceError> {
3015 Ok(self
3016 .board()
3017 .await?
3018 .items
3019 .iter()
3020 .find(|item| item.id == *id && item.kind == BoardKind::Document)
3021 .map(Resolved::document))
3022 }
3023 async fn query_documents(
3024 &self,
3025 query: &DocumentQuery,
3026 page: &PageRequest,
3027 ) -> Result<Page<Document>, SourceError> {
3028 validate_page(page)?;
3029 // Filtered before paged, exactly as a task read is: a page of a filtered result is
3030 // a page of the survivors, never the survivors of a page.
3031 let documents = self
3032 .board()
3033 .await?
3034 .items
3035 .iter()
3036 .filter(|item| item.kind == BoardKind::Document)
3037 .map(Resolved::document)
3038 .filter(|document| document_matches(document, query))
3039 .collect();
3040 Ok(offset_page(
3041 documents,
3042 numeric_cursor(page.cursor.as_ref())?,
3043 page.limit.min(MAX_PAGE_SIZE) as usize,
3044 ))
3045 }
3046 async fn labels(&self, page: &PageRequest) -> Result<Page<Label>, SourceError> {
3047 validate_page(page)?;
3048 let offset = numeric_cursor(page.cursor.as_ref())?;
3049 let mut labels = self
3050 .board()
3051 .await?
3052 .items
3053 .into_iter()
3054 .flat_map(|item| item.labels)
3055 .fold(Vec::new(), |mut all, label| {
3056 if !all.iter().any(|x: &Label| x.id == label.id) {
3057 all.push(label);
3058 }
3059 all
3060 });
3061 labels.sort_by(|a, b| a.name.cmp(&b.name).then(a.id.0.cmp(&b.id.0)));
3062 Ok(offset_page(
3063 labels,
3064 offset,
3065 page.limit.min(MAX_PAGE_SIZE) as usize,
3066 ))
3067 }
3068 async fn task_dependencies(
3069 &self,
3070 id: &NativeId,
3071 direction: Direction,
3072 page: &PageRequest,
3073 ) -> Result<Page<DependencyEdge>, SourceError> {
3074 self.dependencies(id, ItemKind::Task, direction, page).await
3075 }
3076 async fn project_dependencies(
3077 &self,
3078 id: &NativeId,
3079 direction: Direction,
3080 page: &PageRequest,
3081 ) -> Result<Page<DependencyEdge>, SourceError> {
3082 self.dependencies(id, ItemKind::Project, direction, page)
3083 .await
3084 }
3085
3086 fn writes(&self) -> WriteSupport {
3087 WriteSupport::Supported
3088 }
3089
3090 async fn write_task(&self, write: &ItemWrite<Task>) -> Result<NativeId, SourceError> {
3091 self.write_item(
3092 &Incoming {
3093 written: Written::Work(ItemKind::Task, &write.item.status),
3094 title: &write.item.title,
3095 content: write.item.content.as_deref(),
3096 labels: &write.item.labels,
3097 metadata: &write.item.metadata,
3098 repositories: &write.item.repositories,
3099 parent: write.item.project.as_ref(),
3100 },
3101 write.target.as_ref(),
3102 &write.depends_on,
3103 )
3104 .await
3105 }
3106
3107 async fn write_project(&self, write: &ItemWrite<Project>) -> Result<NativeId, SourceError> {
3108 self.write_item(
3109 &Incoming {
3110 written: Written::Work(ItemKind::Project, &write.item.status),
3111 title: &write.item.title,
3112 content: write.item.content.as_deref(),
3113 labels: &write.item.labels,
3114 metadata: &write.item.metadata,
3115 repositories: &write.item.repositories,
3116 parent: None,
3117 },
3118 write.target.as_ref(),
3119 &write.depends_on,
3120 )
3121 .await
3122 }
3123
3124 /// Create or update one document, which is one issue titled the way this board spells
3125 /// a document.
3126 ///
3127 /// Everything else is exactly a task write: caller metadata goes to the same canonical
3128 /// JSON slot at the end of the body and comes back with its JSON types intact, a key
3129 /// or a field this board cannot carry is refused by name rather than dropped, a target
3130 /// naming an issue this board does not hold is refused rather than created, and an
3131 /// issue this call created is taken back when the rest of the write fails.
3132 async fn write_document(&self, write: &ItemWrite<Document>) -> Result<NativeId, SourceError> {
3133 // A document takes part in no dependency graph, so there is no far end to write
3134 // natively and none to record: a caller naming one is told so rather than having it
3135 // stored under the reserved key, where a later read would report an edge the
3136 // contract says cannot exist.
3137 if !write.depends_on.is_empty() {
3138 return Err(SourceError::Refused {
3139 message: format!(
3140 "this write names {} dependencies for a document, and a document takes \
3141 part in no dependency graph; next: put the dependency on the task or \
3142 project the document is about",
3143 write.depends_on.len()
3144 ),
3145 });
3146 }
3147 self.write_item(
3148 &Incoming {
3149 written: Written::Document,
3150 title: &write.item.title,
3151 content: write.item.content.as_deref(),
3152 labels: &write.item.labels,
3153 metadata: &write.item.metadata,
3154 repositories: &write.item.repositories,
3155 parent: write.item.project.as_ref(),
3156 },
3157 write.target.as_ref(),
3158 &[],
3159 )
3160 .await
3161 }
3162
3163 async fn delete_task(&self, id: &NativeId) -> Result<(), SourceError> {
3164 self.delete_item(id).await
3165 }
3166
3167 async fn delete_project(&self, id: &NativeId) -> Result<(), SourceError> {
3168 self.delete_item(id).await
3169 }
3170
3171 async fn delete_document(&self, id: &NativeId) -> Result<(), SourceError> {
3172 self.delete_item(id).await
3173 }
3174}
3175
3176/// Where the recorded tail of a dependency walk resumes; see
3177/// [`GitHubProjectsSource::recorded_edges`].
3178const RECORDED_CURSOR: &str = "onetaskgraph.depends_on:";
3179
3180/// The board text field this source keeps a copy's origin in.
3181///
3182/// Named after the key it holds, and held to that name by the guard below rather than by
3183/// a reader noticing.
3184const ORIGIN_FIELD: &str = "onetaskgraph.origin";
3185
3186/// The metadata key that field holds.
3187///
3188/// The engine owns this key and spells it once as `GlobalId::ORIGIN_KEY`; a plugin never
3189/// constructs or interprets the qualified id it carries. This source names it only to
3190/// route it — a short, typed value belongs in a typed field rather than in the body slot
3191/// a caller's own prose shares.
3192///
3193/// Restated rather than imported, because no plugin crate may depend on the engine. What
3194/// keeps the two spellings one contract is `scripts/check-origin-key-spelling.sh`, a
3195/// target in `check`: it reads the engine's own literal and fails naming the file and the
3196/// line when a plugin's parts from it either way. Drift here has one symptom — a copy
3197/// that creates a second item every run instead of finding the one it wrote — and that is
3198/// too late to learn it.
3199const ORIGIN_KEY: &str = "onetaskgraph.origin";
3200
3201/// Where a recorded tail resumes, refusing a cursor no walk in `direction` reported.
3202///
3203/// The reserved key holds forward edges and nothing else — the reverse of a recorded edge
3204/// is derived from the far end, never written down on the near item — so only a forward
3205/// walk ever reports one of these cursors. A reverse read carrying one is resuming a walk
3206/// it did not come from, and it is told so rather than answered with an empty page that
3207/// reads as a walk which ended.
3208fn recorded_offset(
3209 cursor: Option<&str>,
3210 direction: Direction,
3211) -> Result<Option<usize>, SourceError> {
3212 cursor
3213 .and_then(|cursor| cursor.strip_prefix(RECORDED_CURSOR))
3214 .map(|offset| {
3215 if direction != Direction::DependsOn {
3216 return Err(SourceError::Config {
3217 message: format!(
3218 "{RECORDED_CURSOR}{offset} resumes recorded forward edges, which a \
3219 reverse dependency read never issues; resume it in the direction \
3220 that reported it"
3221 ),
3222 });
3223 }
3224 offset.parse().map_err(|_| SourceError::Config {
3225 message: format!("{RECORDED_CURSOR}{offset} is not a recorded-edge cursor"),
3226 })
3227 })
3228 .transpose()
3229}
3230
3231fn recorded_page(edges: Vec<DependencyEdge>, offset: usize, limit: usize) -> Page<DependencyEdge> {
3232 let mut page = offset_page(edges, offset, limit.max(1));
3233 page.next = page
3234 .next
3235 .map(|cursor| Cursor(format!("{RECORDED_CURSOR}{}", cursor.0)));
3236 page
3237}
3238
3239/// The kind of one issue reached through a dependency connection.
3240///
3241/// The same questions the board scan asks, over the fields the dependency document
3242/// selects, and in the same order: the design prefix first, then a sub-issue is a task,
3243/// then anything with sub-issues or the marker is a project.
3244///
3245/// # Errors
3246///
3247/// A far end this board holds as a document is refused rather than reported. The two
3248/// answers that are not refusals would both be wrong: reporting it as a task names an id
3249/// no task read of this source can find, and reporting it as a project names one no
3250/// project read can. There is no third value to return — `ItemKind` has no document
3251/// variant, because nothing may point at a document — so the relationship itself is what
3252/// the person is told about.
3253fn related_kind(value: &Value) -> Result<ItemKind, SourceError> {
3254 let id = required_str(value, "id")?;
3255 if required_str(value, "title")?.starts_with(DESIGN_TITLE_PREFIX) {
3256 return Err(SourceError::Refused {
3257 message: format!(
3258 "GitHub issue {id} is a document of this board — its title begins \
3259 {DESIGN_TITLE_PREFIX:?} — and nothing may depend on a document or be depended \
3260 on by one; next: remove that issue's blocking relationship on this board"
3261 ),
3262 });
3263 }
3264 let parent = optional_str(value.get("parent").unwrap_or(&Value::Null), "id")?;
3265 if parent.is_some() {
3266 return Ok(ItemKind::Task);
3267 }
3268 let (_, slot) = metadata_body(optional_str(value, "body")?.map(str::to_owned))?;
3269 let marked = ItemKind::from_metadata(&slot).map_err(|message| SourceError::Malformed {
3270 message: format!("GitHub issue {id}: {message}"),
3271 })?;
3272 let sub_issues = sub_issue_total(value)?;
3273 Ok(if sub_issues > 0 || marked == Some(ItemKind::Project) {
3274 ItemKind::Project
3275 } else {
3276 ItemKind::Task
3277 })
3278}
3279
3280/// The `IssueStateUpdateInput` one status target asks for.
3281///
3282/// `stateInput` and `state` are mutually exclusive on `UpdateIssueInput`, and only this
3283/// one is ever sent. A non-terminal status always asks for `OPEN`, which is what reopens
3284/// a currently-closed issue: without that the item would read back `Unknown` and a copy
3285/// would report a change forever. A document has no status at all, and asks for neither.
3286fn state_input(target: Option<&StatusTarget>) -> Value {
3287 match target {
3288 Some(StatusTarget::Closed(reason)) => {
3289 json!({"value":"CLOSED","stateReason":reason.reason()})
3290 }
3291 Some(StatusTarget::Column(_) | StatusTarget::Disabled) => json!({"value":"OPEN"}),
3292 // A document has no status, so a write of one says nothing about the issue's open
3293 // or closed state rather than forcing it open: `stateInput` is what carries that
3294 // instruction, and an explicit null asks for no change to it.
3295 None => Value::Null,
3296 }
3297}
3298
3299/// The metadata one write stores in the item's body slot.
3300///
3301/// The typed fields travel as themselves, so the three reserved keys are rebuilt here
3302/// rather than carried: the kind marker so an empty project stays readable, the
3303/// repository list only when it is not exactly the issue's own repository, and the far
3304/// ends no relationship here can name.
3305fn slot_metadata(
3306 incoming: &Incoming<'_>,
3307 own_repository: Option<&Repository>,
3308 fallback: &[DependencyEdge],
3309) -> BTreeMap<String, Value> {
3310 let mut metadata = incoming.metadata.clone();
3311 metadata.remove(ORIGIN_KEY);
3312 match incoming.written.kind() {
3313 BoardKind::Work(kind) => metadata.insert(
3314 ItemKind::METADATA_KEY.to_owned(),
3315 Value::String(kind.marker().to_owned()),
3316 ),
3317 // A document is told by its title, so it carries no kind marker: that key names
3318 // what a dependency endpoint points at, and nothing may point at a document.
3319 BoardKind::Document => metadata.remove(ItemKind::METADATA_KEY),
3320 };
3321 let derivable = own_repository
3322 .map(|own| incoming.repositories == [own.clone()])
3323 .unwrap_or(incoming.repositories.is_empty());
3324 if derivable {
3325 metadata.remove(Repository::METADATA_KEY);
3326 } else {
3327 metadata.insert(
3328 Repository::METADATA_KEY.to_owned(),
3329 Value::Array(
3330 incoming
3331 .repositories
3332 .iter()
3333 .map(|repository| Value::String(repository.as_str().to_owned()))
3334 .collect(),
3335 ),
3336 );
3337 }
3338 if fallback.is_empty() {
3339 metadata.remove(DependencyEdge::RECORDED_KEY);
3340 } else {
3341 metadata.insert(
3342 DependencyEdge::RECORDED_KEY.to_owned(),
3343 Value::Array(
3344 fallback
3345 .iter()
3346 .map(|edge| json!({"id":edge.to.id(),"kind":edge.to.kind}))
3347 .collect(),
3348 ),
3349 );
3350 }
3351 metadata
3352}
3353
3354fn labels(content: &Value, field_values: &[Value]) -> Result<Vec<Label>, SourceError> {
3355 let direct = optional_nodes(content.get("labels"), "content labels")?;
3356 let field = field_values
3357 .iter()
3358 .find_map(|value| value.get("labels"))
3359 .map(|labels| optional_nodes(Some(labels), "field labels"))
3360 .transpose()?
3361 .flatten();
3362 let labels = direct
3363 .into_iter()
3364 .flatten()
3365 .chain(field.into_iter().flatten())
3366 .map(|v| {
3367 Ok(Label {
3368 id: NativeId(required_str(v, "id")?.to_owned()),
3369 name: required_str(v, "name")?.to_owned(),
3370 color: optional_str(v, "color")?.map(str::to_owned),
3371 })
3372 })
3373 .collect::<Result<Vec<_>, SourceError>>()?
3374 .into_iter()
3375 .fold(Vec::new(), |mut labels, label| {
3376 if !labels.iter().any(|x: &Label| x.id == label.id) {
3377 labels.push(label);
3378 }
3379 labels
3380 });
3381 Ok(labels)
3382}
3383
3384fn text_field(field_values: &[Value], name: &str) -> Result<Option<String>, SourceError> {
3385 let Some(node) = field_values
3386 .iter()
3387 .find(|node| node.pointer("/field/name").and_then(Value::as_str) == Some(name))
3388 else {
3389 return Ok(None);
3390 };
3391 Ok(optional_str(node, "text")?.map(str::to_owned))
3392}
3393
3394fn valid_github_owner(owner: &str) -> bool {
3395 !owner.is_empty()
3396 && owner.len() <= 39
3397 && !owner.starts_with('-')
3398 && !owner.ends_with('-')
3399 && !owner.contains("--")
3400 && owner
3401 .bytes()
3402 .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-')
3403}
3404
3405/// GitHub's repository-name grammar: 1-100 ASCII letters, digits, `-`, `_` or `.`, and
3406/// neither of the two names a path segment already means.
3407fn valid_github_repository_name(name: &str) -> bool {
3408 !name.is_empty()
3409 && name.len() <= 100
3410 && name != "."
3411 && name != ".."
3412 && name
3413 .bytes()
3414 .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.'))
3415}
3416
3417fn valid_environment_name(name: &str) -> bool {
3418 let mut bytes = name.bytes();
3419 bytes
3420 .next()
3421 .is_some_and(|byte| byte.is_ascii_alphabetic() || byte == b'_')
3422 && bytes.all(|byte| byte.is_ascii_alphanumeric() || byte == b'_')
3423}
3424
3425/// How many sub-issues one issue has.
3426///
3427/// `Issue.subIssuesSummary` is `SubIssuesSummary!` and its `total` is `Int!`, so an
3428/// absent or non-integer one is a response this source cannot read — and reading it as
3429/// zero would classify a project as a task, which is exactly the mistake the marker
3430/// exists to keep from happening quietly.
3431fn sub_issue_total(issue: &Value) -> Result<u64, SourceError> {
3432 let summary = issue
3433 .get("subIssuesSummary")
3434 .ok_or_else(|| SourceError::Malformed {
3435 message: "GitHub issue is missing subIssuesSummary".into(),
3436 })?;
3437 summary
3438 .get("total")
3439 .and_then(Value::as_u64)
3440 .ok_or_else(|| SourceError::Malformed {
3441 message: "GitHub issue subIssuesSummary.total is not an unsigned integer".into(),
3442 })
3443}
3444
3445fn required_str<'a>(value: &'a Value, field: &str) -> Result<&'a str, SourceError> {
3446 value
3447 .get(field)
3448 .and_then(Value::as_str)
3449 .ok_or_else(|| SourceError::Malformed {
3450 message: format!("GitHub response is missing string field {field}"),
3451 })
3452}
3453
3454/// The slot's delimiters, which `docs/metadata.md` settles once for every source that
3455/// needs one — Linear spells them too, in its own description field.
3456///
3457/// Restated rather than shared, because a plugin crate depends on the contract crate and
3458/// nothing else of this workspace. `scripts/check-metadata-slot-encoding.sh`, a target in
3459/// `check`, is what keeps the two one encoding: drift is otherwise quiet, since each
3460/// source round-trips its own writes perfectly well under its own spelling.
3461const METADATA_OPEN: &str = "<!-- onetaskgraph.metadata\n";
3462const METADATA_CLOSE: &str = "\n-->";
3463
3464/// The visible body and the metadata slot at the end of it.
3465///
3466/// The encoding is the one `docs/metadata.md` settles for Linear, which is where its
3467/// reasons are. Only a comment at the very end is a slot; one in the middle is a person's
3468/// own content and is left alone.
3469fn metadata_body(
3470 body: Option<String>,
3471) -> Result<(Option<String>, BTreeMap<String, Value>), SourceError> {
3472 let Some(body) = body else {
3473 return Ok((None, BTreeMap::new()));
3474 };
3475 let Some(start) = body.rfind(METADATA_OPEN) else {
3476 return Ok((Some(body), BTreeMap::new()));
3477 };
3478 let encoded_start = start + METADATA_OPEN.len();
3479 let Some(relative_end) = body[encoded_start..].find(METADATA_CLOSE) else {
3480 return Err(SourceError::Malformed {
3481 message: "unterminated onetaskgraph metadata slot in GitHub issue body".into(),
3482 });
3483 };
3484 let encoded_end = encoded_start + relative_end;
3485 if !body[encoded_end + METADATA_CLOSE.len()..].trim().is_empty() {
3486 return Ok((Some(body), BTreeMap::new()));
3487 }
3488 let metadata = serde_json::from_str(&body[encoded_start..encoded_end]).map_err(|error| {
3489 SourceError::Malformed {
3490 message: format!(
3491 "invalid canonical JSON in GitHub issue onetaskgraph metadata slot: {error}"
3492 ),
3493 }
3494 })?;
3495 let visible = body[..start].trim_end();
3496 Ok(((!visible.is_empty()).then(|| visible.to_owned()), metadata))
3497}
3498
3499fn compose_body(
3500 content: Option<&str>,
3501 metadata: &BTreeMap<String, Value>,
3502) -> Result<Option<String>, SourceError> {
3503 let visible = content.unwrap_or_default();
3504 if metadata.is_empty() {
3505 return Ok((!visible.is_empty()).then(|| visible.to_owned()));
3506 }
3507 let encoded = serde_json::to_string(metadata).map_err(|error| SourceError::Malformed {
3508 message: error.to_string(),
3509 })?;
3510 Ok(Some(if visible.is_empty() {
3511 format!("{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
3512 } else {
3513 format!("{visible}\n\n{METADATA_OPEN}{encoded}{METADATA_CLOSE}")
3514 }))
3515}
3516
3517fn required_bool(value: &Value, field: &str) -> Result<bool, SourceError> {
3518 value
3519 .get(field)
3520 .and_then(Value::as_bool)
3521 .ok_or_else(|| SourceError::Malformed {
3522 message: format!("GitHub response is missing boolean field {field}"),
3523 })
3524}
3525fn optional_str<'a>(value: &'a Value, field: &str) -> Result<Option<&'a str>, SourceError> {
3526 match value.get(field) {
3527 None | Some(Value::Null) => Ok(None),
3528 Some(value) => value
3529 .as_str()
3530 .map(Some)
3531 .ok_or_else(|| SourceError::Malformed {
3532 message: format!("GitHub response field {field} is not a string or null"),
3533 }),
3534 }
3535}
3536fn optional_nodes<'a>(
3537 connection: Option<&'a Value>,
3538 name: &str,
3539) -> Result<Option<&'a Vec<Value>>, SourceError> {
3540 match connection {
3541 None | Some(Value::Null) => Ok(None),
3542 Some(value) => value
3543 .get("nodes")
3544 .and_then(Value::as_array)
3545 .map(Some)
3546 .ok_or_else(|| SourceError::Malformed {
3547 message: format!("GitHub {name}.nodes is not an array"),
3548 }),
3549 }
3550}
3551fn complete_connection(connection: &Value, name: &str) -> Result<(), SourceError> {
3552 let page_info = connection
3553 .get("pageInfo")
3554 .ok_or_else(|| SourceError::Malformed {
3555 message: format!("GitHub {name} has no pageInfo"),
3556 })?;
3557 if required_bool(page_info, "hasNextPage")? {
3558 return Err(SourceError::Malformed {
3559 message: format!(
3560 "GitHub {name} exceeds the supported nested connection size of {NESTED_PAGE_SIZE}"
3561 ),
3562 });
3563 }
3564 Ok(())
3565}
3566fn optional_time(value: &Value, field: &str) -> Result<Option<DateTime<Utc>>, SourceError> {
3567 optional_str(value, field)?
3568 .map(|timestamp| {
3569 timestamp.parse().map_err(|error| SourceError::Malformed {
3570 message: format!("GitHub response field {field} is not a timestamp: {error}"),
3571 })
3572 })
3573 .transpose()
3574}
3575fn validate_page(page: &PageRequest) -> Result<(), SourceError> {
3576 if page.limit == 0 {
3577 Err(SourceError::Config {
3578 message: "page limit must be at least 1".into(),
3579 })
3580 } else {
3581 Ok(())
3582 }
3583}
3584fn next_cursor(connection: &Value) -> Result<Option<Cursor>, SourceError> {
3585 let page = connection
3586 .get("pageInfo")
3587 .filter(|value| value.is_object())
3588 .ok_or_else(|| SourceError::Malformed {
3589 message: "GitHub connection is missing pageInfo".into(),
3590 })?;
3591 if required_bool(page, "hasNextPage")? {
3592 let cursor = required_str(page, "endCursor")?;
3593 validate_cursor_progress(None, cursor)?;
3594 Ok(Some(Cursor(cursor.into())))
3595 } else {
3596 Ok(None)
3597 }
3598}
3599fn validate_cursor_progress(previous: Option<&str>, next: &str) -> Result<(), SourceError> {
3600 if next.is_empty() || previous == Some(next) {
3601 Err(SourceError::Malformed {
3602 message: "GitHub pagination cursor is empty or did not advance".into(),
3603 })
3604 } else {
3605 Ok(())
3606 }
3607}
3608fn numeric_cursor(cursor: Option<&Cursor>) -> Result<usize, SourceError> {
3609 cursor.map_or(Ok(0), |c| {
3610 c.0.parse().map_err(|_| SourceError::Config {
3611 message: "page cursor is invalid".into(),
3612 })
3613 })
3614}
3615fn offset_page<T>(mut items: Vec<T>, offset: usize, limit: usize) -> Page<T> {
3616 if offset > items.len() {
3617 return Page::last(vec![]);
3618 }
3619 let tail = items.split_off(offset);
3620 let mut selected = tail;
3621 let next = (selected.len() > limit).then(|| Cursor((offset + limit).to_string()));
3622 selected.truncate(limit);
3623 Page {
3624 items: selected,
3625 next,
3626 }
3627}