Skip to main content

rusty_rtos_port_host/
lib.rs

1//! The Kairos **host** port: a task gets a real stack from the operating
2//! system.
3//!
4//! # Why this exists
5//!
6//! The Kairos kernel is stackless by design. A blocking call keeps its
7//! locals in the TCB and answers `Wait::Blocked`, meaning "call again when
8//! this task next runs" — which is what lets one kernel run the
9//! conformance corpus on a host, on a simulator and on three different
10//! chips without a line of assembly in between.
11//!
12//! A task written in **C** cannot work that way. `vTaskDelay` has to return
13//! *later*, on the line after, with its locals intact, and locals in C live
14//! on a stack. So the C ABI needs a port that supplies one — and until this
15//! crate there were only two kinds: three bare-metal ports, which need an
16//! emulator or a chip, and the sim port, which is stackless. That is why
17//! K6's kill test ran on QEMU Cortex-M3 before it ran on a host: there was
18//! nowhere on a host for a C task to keep its locals.
19//!
20//! # The shape is FreeRTOS's own
21//!
22//! `portable/ThirdParty/GCC/Posix` runs one pthread per task and lets
23//! exactly one of them run at a time; the MSVC-MinGW port does the same
24//! with Win32 threads. This is that: one OS thread per task, a single
25//! **run permit**, and a tick thread standing in for the timer interrupt.
26//! The OS supplies the stacks and switches them; we supply the policy.
27//!
28//! # The two ways a switch happens
29//!
30//! | | who decides | how the outgoing task stops |
31//! |---|---|---|
32//! | voluntary | the task, through [`pend_switch`] | it waits on its own permit |
33//! | preemptive | the tick thread | the OS freezes the thread where it stands |
34//!
35//! The voluntary path is all a cooperative kernel needs. The preemptive
36//! path is not optional: `integer.c` and `flop.c` in the standard demos
37//! **never block and never yield** — with `configUSE_PREEMPTION` set they
38//! rely on being taken off the CPU — so a host port without it starves
39//! every task below them and their neighbours' checkers report failure. A
40//! cooperative host port would pass the demos that block and quietly fail
41//! the ones that do not, which is the worst of both.
42//!
43//! # Freezing a thread is safe here, and the reason is narrow
44//!
45//! Suspending an arbitrary thread is a famous way to deadlock: freeze it
46//! inside `malloc` and the next thread that allocates waits forever.
47//!
48//! It is safe here because of the ORDER in [`Ticker::interrupt`]: the tick
49//! thread takes the critical-section lock **first**, and every kernel call
50//! and every heap call is inside that lock. A thread that is not holding it
51//! is not inside the kernel and not inside our heap, so there is no lock
52//! for it to be holding when it freezes.
53//!
54//! What that argument does NOT cover is a lock we do not own: the C
55//! runtime's allocator, or `stdout`. A task that calls `printf` or the
56//! system `malloc` directly can still be frozen inside one. The demo task
57//! files do neither — they allocate through `pvPortMalloc`, which is the
58//! Kairos heap behind our own critical section. A cell that prints from a
59//! task must do it inside [`Port::enter_critical`].
60//!
61//! # Unsafe
62//!
63//! The workspace denies `unsafe_code`. Every use here is fenced with
64//! `#[expect(unsafe_code)]` and a justification, and they are all of one
65//! kind: freezing and thawing an OS thread. See `UNSAFE.md`.
66
67use std::sync::atomic::{AtomicBool, AtomicU32, AtomicU64, AtomicUsize, Ordering};
68use std::sync::{Condvar, Mutex, OnceLock};
69use std::time::Duration;
70
71use rusty_rtos_core::isr::Woken;
72use rusty_rtos_core::port::Port;
73
74/// Crate version, for manifests and logs.
75pub const VERSION: &str = env!("CARGO_PKG_VERSION");
76
77/// The most tasks a host cell may have.
78///
79/// Fixed rather than grown, so the slot table is a `static` and a slot's
80/// address never moves — a task thread holds a reference to its own slot
81/// for its whole life.
82pub const MAX_TASKS: usize = 128;
83
84// ------------------------------------------------------ the run permit --
85
86/// One task's slot: its permit, and how it is currently stopped.
87///
88/// A task runs only when it holds the single run permit. It can be stopped
89/// in two different ways and the resume path has to know which, because
90/// thawing a thread that is waiting on a condvar does nothing and
91/// signalling a condvar that a frozen thread is not waiting on does nothing
92/// either. Getting this wrong is a hang, not an error.
93#[derive(Debug)]
94struct Slot {
95    /// Set when this task may run. The task's own thread waits for it.
96    granted: Mutex<bool>,
97    /// Signalled when `granted` becomes true.
98    wake: Condvar,
99    /// The OS thread, once it exists, in whatever form the platform needs
100    /// to freeze it. Zero means "no thread yet".
101    os_thread: AtomicU64,
102    /// True while this task's thread is frozen by the tick thread rather
103    /// than parked on its own condvar.
104    frozen: AtomicBool,
105    /// Which occupant of this slot the thread belongs to.
106    ///
107    /// A slot outlives its tasks -- `death.c` deletes and recreates them
108    /// forever and the kernel hands the freed index straight back -- and a
109    /// deleted task's thread is parked inside the C function it was
110    /// running, from which it will never return. So the slot takes a NEW
111    /// thread and this counter is how the old one knows it is no longer
112    /// the occupant: it wakes with the others, sees a generation that is
113    /// not its own, and parks again forever.
114    ///
115    /// Without it the orphan competes for the run permit with the task
116    /// that replaced it, and both run.
117    generation: AtomicU64,
118}
119
120impl Slot {
121    const fn new() -> Self {
122        Self {
123            granted: Mutex::new(false),
124            wake: Condvar::new(),
125            os_thread: AtomicU64::new(0),
126            frozen: AtomicBool::new(false),
127            generation: AtomicU64::new(0),
128        }
129    }
130}
131
132/// The slot table. One entry per task index, and the index is the
133/// kernel's `TaskHandle::index()`, exactly as on the Cortex-M port.
134fn slots() -> &'static [Slot] {
135    static SLOTS: OnceLock<Vec<Slot>> = OnceLock::new();
136    SLOTS.get_or_init(|| (0..MAX_TASKS).map(|_| Slot::new()).collect())
137}
138
139/// Which task index currently holds the run permit.
140///
141/// This is the host's `pxCurrentTCB`. `usize::MAX` means "nothing is
142/// running yet", the state before the first task starts.
143pub static CURRENT: AtomicUsize = AtomicUsize::new(usize::MAX);
144
145/// Nobody is running.
146pub const NO_TASK: usize = usize::MAX;
147
148// ---------------------------------------------------- the critical lock --
149
150/// The critical section, as a lock the tick thread also takes.
151///
152/// On a chip `cpsid i` masks the timer interrupt. Here there is no
153/// interrupt to mask: the "interrupt" is another OS thread, and the only
154/// way to hold it off is a lock it agrees to take. That agreement is the
155/// whole of this port's interrupt model, and [`Ticker::interrupt`] is the
156/// one place the other side of it lives.
157///
158/// It is a spin lock rather than a `Mutex` because it must be re-entrant
159/// (the kernel nests critical sections freely) and must be releasable from
160/// a different call frame than the one that took it, neither of which a
161/// `MutexGuard` will do.
162#[derive(Debug)]
163struct CriticalLock {
164    /// The thread that holds it, as its OS id. Zero is free.
165    owner: AtomicU64,
166    /// How many times that thread has entered without leaving.
167    nesting: AtomicU32,
168}
169
170static CRITICAL: CriticalLock = CriticalLock {
171    owner: AtomicU64::new(0),
172    nesting: AtomicU32::new(0),
173};
174
175impl CriticalLock {
176    /// Take the lock, or count another level if this thread already has it.
177    fn enter(&self, me: u64) {
178        if self.owner.load(Ordering::Acquire) == me {
179            self.nesting.fetch_add(1, Ordering::Relaxed);
180            return;
181        }
182        while self
183            .owner
184            .compare_exchange_weak(0, me, Ordering::AcqRel, Ordering::Relaxed)
185            .is_err()
186        {
187            // The holder is either running on another core or about to be
188            // scheduled. Either way there is nothing useful to do here.
189            std::thread::yield_now();
190        }
191        self.nesting.store(1, Ordering::Relaxed);
192    }
193
194    /// Give up one level, releasing the lock at zero. `true` when it was
195    /// actually released.
196    fn exit(&self, me: u64) -> bool {
197        if self.owner.load(Ordering::Acquire) != me {
198            // Not ours to release. A port that "restored" a lock it never
199            // took would let the tick thread in halfway through somebody
200            // else's critical section.
201            return false;
202        }
203        let left = self.nesting.load(Ordering::Relaxed).saturating_sub(1);
204        self.nesting.store(left, Ordering::Relaxed);
205        if left == 0 {
206            self.owner.store(0, Ordering::Release);
207            return true;
208        }
209        false
210    }
211
212    /// Give the lock up entirely, whatever depth this thread was at, and
213    /// answer that depth.
214    ///
215    /// A critical section belongs to the TASK, not to the CPU. FreeRTOS's
216    /// own Win32 port saves and restores `uxCriticalNesting` across a
217    /// switch for exactly this reason: the kernel yields from inside its
218    /// own critical sections, and a task that parked while still holding
219    /// this lock would deadlock the task it just handed the CPU to.
220    fn release_all(&self, me: u64) -> u32 {
221        if self.owner.load(Ordering::Acquire) != me {
222            return 0;
223        }
224        let depth = self.nesting.load(Ordering::Relaxed);
225        self.nesting.store(0, Ordering::Relaxed);
226        self.owner.store(0, Ordering::Release);
227        depth
228    }
229
230    /// Take it again at the depth [`release_all`](Self::release_all) gave
231    /// back. Nothing to do for a depth of zero.
232    fn reacquire(&self, me: u64, depth: u32) {
233        if depth == 0 {
234            return;
235        }
236        self.enter(me);
237        self.nesting.store(depth, Ordering::Relaxed);
238    }
239}
240
241// ------------------------------------------------------ the OS backend --
242
243#[cfg(windows)]
244mod backend {
245    //! Windows: `SuspendThread` / `ResumeThread` freeze a thread where it
246    //! stands, which is what makes preemption possible at all.
247
248    use windows_sys::Win32::Foundation::{CloseHandle, HANDLE};
249    use windows_sys::Win32::System::Threading::{ResumeThread, SuspendThread};
250
251    /// Is preemption available on this platform?
252    pub const PREEMPTIVE: bool = true;
253
254    /// The OS handle a `JoinHandle` owns.
255    ///
256    /// Taken from the spawner so the thread has a handle before it has
257    /// run. Borrowed, not owned: the `JoinHandle` must outlive every use,
258    /// which is why the caller keeps it.
259    #[must_use]
260    pub fn thread_handle_of(join: &std::thread::JoinHandle<()>) -> u64 {
261        use std::os::windows::io::AsRawHandle as _;
262        join.as_raw_handle() as u64
263    }
264
265    /// Freeze the thread. `false` if it could not be.
266    #[expect(unsafe_code, reason = "suspending a thread is a syscall")]
267    pub fn freeze(handle: u64) -> bool {
268        if handle == 0 {
269            return false;
270        }
271        // SAFETY: `handle` came from `current_thread` above and is a real
272        // thread handle this process owns.
273        let previous = unsafe { SuspendThread(handle as HANDLE) };
274        previous != u32::MAX
275    }
276
277    /// Thaw the thread. `false` if it could not be.
278    #[expect(unsafe_code, reason = "resuming a thread is a syscall")]
279    pub fn thaw(handle: u64) -> bool {
280        if handle == 0 {
281            return false;
282        }
283        // SAFETY: as `freeze`.
284        let previous = unsafe { ResumeThread(handle as HANDLE) };
285        previous != u32::MAX
286    }
287
288    /// Give the handle back to the OS.
289    #[expect(unsafe_code, reason = "closing a handle is a syscall")]
290    pub fn release(handle: u64) {
291        if handle != 0 {
292            // SAFETY: as `freeze`; the handle is not used again.
293            unsafe { CloseHandle(handle as HANDLE) };
294        }
295    }
296}
297
298#[cfg(unix)]
299mod backend {
300    //! Unix: a thread cannot be frozen from outside, so it freezes
301    //! **itself**.
302    //!
303    //! There is no `SuspendThread` here. What Unix gives instead is a
304    //! signal, and a signal handler is the only code that runs in another
305    //! thread's context without that thread's cooperation — which is
306    //! exactly what preemption means. So `freeze` sends `SIGUSR1` and the
307    //! handler, running in the target, parks the target in `sigsuspend`
308    //! until `SIGUSR2` arrives. It is the shape FreeRTOS's own
309    //! `portable/ThirdParty/GCC/Posix` port uses.
310    //!
311    //! # Freezing has to be SYNCHRONOUS, and that is the whole difficulty
312    //!
313    //! `pthread_kill` returns as soon as the signal is *queued*, not when
314    //! it is handled. If `freeze` returned there, the tick thread would
315    //! grant the CPU to a new task while the old one was still running on
316    //! it — two tasks live at once, which is the one thing a single run
317    //! permit exists to prevent. So `freeze` waits until it can observe
318    //! the target parked, and answers `false` if it never does, in which
319    //! case the caller declines the switch rather than taking it unsafely.
320    //!
321    //! # The lost-wakeup race, and why `SIGUSR2` is blocked in the handler
322    //!
323    //! The handler marks itself `PARKED` and then calls `sigsuspend`. A
324    //! `thaw` landing in the gap between those two would be delivered to a
325    //! thread that is not yet waiting, and the thread would then sleep for
326    //! ever on a wake-up that already happened.
327    //!
328    //! `SIGUSR2` is therefore in the suspend handler's `sa_mask`, so it is
329    //! BLOCKED for the whole handler. A `thaw` arriving in that gap stays
330    //! *pending*; `sigsuspend` atomically unblocks it, which delivers it
331    //! immediately and returns. Nothing is lost, and the loop re-checks
332    //! the state rather than trusting the wake-up — a signal handler that
333    //! trusts a single wake-up is a signal handler with a bug.
334    //!
335    //! # What runs inside a handler
336    //!
337    //! Only async-signal-safe calls: `pthread_self`, `sigfillset`,
338    //! `sigdelset`, `sigsuspend`, and atomic loads over a fixed static
339    //! table. No allocation, no locks, no thread-locals — a Rust
340    //! thread-local can allocate on first touch, which is not safe here.
341
342    use std::sync::Once;
343    use std::sync::atomic::{AtomicU32, AtomicUsize, Ordering};
344
345    /// Is preemption available on this platform?
346    pub const PREEMPTIVE: bool = true;
347
348    /// How many task threads can be registered at once.
349    ///
350    /// An entry is freed by [`release`], which the port calls when a task
351    /// slot is reused or a task is deleted, so this bounds LIVE threads
352    /// and not threads over the run. The C ABI cell runs 60.
353    const MAX_THREADS: usize = 256;
354
355    /// Neither asked to park nor parked.
356    const RUNNING: u32 = 0;
357    /// Signalled, but not yet seen to be parked.
358    const ASKED: u32 = 1;
359    /// Sitting in `sigsuspend`, inside the handler.
360    const PARKED: u32 = 2;
361
362    /// How many times [`freeze`] looks for the target to park before it
363    /// gives up and answers `false`.
364    ///
365    /// This is a bound on a handoff between two runnable threads, not a
366    /// wait for work, so it is generous rather than tuned: the cost of
367    /// being wrong in one direction is a declined switch, and in the other
368    /// it is two tasks running at once.
369    const PARK_SPINS: u32 = 200_000;
370
371    struct Entry {
372        /// The `pthread_t`, widened to a `usize`.
373        ///
374        /// Zero means the entry is free. No Unix gives out a null
375        /// `pthread_t` — on Linux it is a pointer to the thread's own
376        /// descriptor — so zero is safe to spend as the empty marker.
377        tid: AtomicUsize,
378        /// One of [`RUNNING`], [`ASKED`], [`PARKED`].
379        state: AtomicU32,
380    }
381
382    static TABLE: [Entry; MAX_THREADS] = [const {
383        Entry {
384            tid: AtomicUsize::new(0),
385            state: AtomicU32::new(RUNNING),
386        }
387    }; MAX_THREADS];
388
389    /// The entry a handle names, or `None`.
390    ///
391    /// The handle is the table index plus one, so that a handle of zero
392    /// keeps meaning "no thread" the way the Windows backend's does.
393    fn entry_at(handle: u64) -> Option<&'static Entry> {
394        let index = usize::try_from(handle).ok()?.checked_sub(1)?;
395        TABLE.get(index)
396    }
397
398    /// The entry belonging to a `pthread_t`, or `None`.
399    ///
400    /// A linear scan of atomic loads, which is what makes it callable from
401    /// a signal handler. 256 relaxed loads is nothing next to the context
402    /// switch that is about to happen.
403    fn entry_of(tid: usize) -> Option<&'static Entry> {
404        TABLE.iter().find(|e| e.tid.load(Ordering::Acquire) == tid)
405    }
406
407    /// A `pthread_t` as the table's `usize`. It is `usize` on Apple and `u64`
408    /// on Linux, so the cast is needed on one and a no-op on the other;
409    /// clippy's `unnecessary_cast` sees only the platform it is run on.
410    #[allow(
411        clippy::unnecessary_cast,
412        reason = "pthread_t is usize on Apple, u64 on Linux"
413    )]
414    fn tid_of(t: libc::pthread_t) -> usize {
415        t as usize
416    }
417
418    /// Park this thread until someone thaws it. `SIGUSR1`.
419    #[expect(
420        unsafe_code,
421        reason = "a signal handler is the only code that runs in another thread without its cooperation"
422    )]
423    extern "C" fn on_suspend(_signal: core::ffi::c_int) {
424        // First, before anything can touch it: `sigsuspend` below always
425        // sets `errno` (to `EINTR`), and the code this handler interrupted
426        // may be between a failing syscall and reading its `errno`. Put back
427        // on every return by the guard's `Drop`.
428        let _errno = SavedErrno::take();
429        // SAFETY: `pthread_self` reads the calling thread's own id and is
430        // async-signal-safe.
431        let me = tid_of(unsafe { libc::pthread_self() });
432        let Some(entry) = entry_of(me) else {
433            // Not a task thread, or its entry was released while the
434            // signal was in flight. Either way there is nobody to wait
435            // for a thaw that will never come.
436            return;
437        };
438        if entry
439            .state
440            .compare_exchange(ASKED, PARKED, Ordering::AcqRel, Ordering::Acquire)
441            .is_err()
442        {
443            // `freeze` gave up before this handler ran and put the state
444            // back. Parking now would be parking with nobody left to thaw.
445            return;
446        }
447
448        // Everything blocked except the thaw signal, so `sigsuspend` can
449        // only be woken by the thing that is meant to wake it.
450        //
451        // SAFETY: `mask` is a real `sigset_t` about to be initialised by
452        // `sigfillset`; all four calls are async-signal-safe.
453        unsafe {
454            let mut mask: libc::sigset_t = core::mem::zeroed();
455            libc::sigfillset(&raw mut mask);
456            libc::sigdelset(&raw mut mask, libc::SIGUSR2);
457            while entry.state.load(Ordering::Acquire) == PARKED {
458                libc::sigsuspend(&raw const mask);
459            }
460        }
461    }
462
463    /// `errno` as the interrupted code left it, restored when the signal
464    /// handler returns.
465    ///
466    /// A signal handler that calls anything able to set `errno` must put it
467    /// back, or the code it interrupted can read a value it did not cause --
468    /// a failed `read` reporting `EINTR`, or a success reporting an error.
469    /// `on_suspend` calls `sigsuspend`, which always sets it. ThreadSanitizer
470    /// reported exactly this ("signal handler spoils errno") on the host
471    /// port's tests during the hardening audit of 2026-10-01.
472    struct SavedErrno(Option<core::ffi::c_int>);
473
474    impl SavedErrno {
475        #[expect(
476            unsafe_code,
477            reason = "reading errno through the C runtime's per-thread pointer"
478        )]
479        fn take() -> Self {
480            // SAFETY: the pointer is the calling thread's own errno slot,
481            // valid for the life of the thread; reading it is what `errno`
482            // means, and is async-signal-safe.
483            Self(errno_location().map(|p| unsafe { *p }))
484        }
485    }
486
487    impl Drop for SavedErrno {
488        #[expect(
489            unsafe_code,
490            reason = "writing errno through the C runtime's per-thread pointer"
491        )]
492        fn drop(&mut self) {
493            if let (Some(p), Some(value)) = (errno_location(), self.0) {
494                // SAFETY: as `take`: this thread's own errno slot.
495                unsafe { *p = value };
496            }
497        }
498    }
499
500    /// The calling thread's `errno` slot, where the C runtime names it.
501    /// `None` on a Unix this does not know, which then keeps the old
502    /// behaviour rather than failing to build.
503    #[expect(unsafe_code, reason = "the C runtime's errno accessor is an extern fn")]
504    fn errno_location() -> Option<*mut core::ffi::c_int> {
505        #[cfg(any(target_os = "linux", target_os = "android"))]
506        // SAFETY: takes no arguments, returns this thread's errno slot, and
507        // is async-signal-safe.
508        return Some(unsafe { libc::__errno_location() });
509        #[cfg(any(
510            target_os = "macos",
511            target_os = "ios",
512            target_os = "freebsd",
513            target_os = "dragonfly"
514        ))]
515        // SAFETY: as above; this platform's name for the same accessor.
516        return Some(unsafe { libc::__error() });
517        #[cfg(any(target_os = "openbsd", target_os = "netbsd"))]
518        // SAFETY: as above; this platform's name for the same accessor.
519        return Some(unsafe { libc::__errno() });
520        #[allow(unreachable_code)]
521        None
522    }
523
524    /// Wake a parked thread. `SIGUSR2`.
525    ///
526    /// Deliberately empty: the work is done by `sigsuspend` RETURNING. A
527    /// handler that did anything here would be doing it in the wrong
528    /// thread's context for no reason.
529    extern "C" fn on_resume(_signal: core::ffi::c_int) {}
530
531    /// Install both handlers, once for the process.
532    #[expect(unsafe_code, reason = "installing a signal handler is a syscall")]
533    fn install_handlers() {
534        static ONCE: Once = Once::new();
535        ONCE.call_once(|| {
536            // SAFETY: both structures are zeroed, then filled with a real
537            // handler and a real mask before `sigaction` reads them, and
538            // the handlers have C ABI and the right signature.
539            unsafe {
540                let mut suspend: libc::sigaction = core::mem::zeroed();
541                suspend.sa_sigaction = on_suspend as *const () as libc::sighandler_t;
542                libc::sigemptyset(&raw mut suspend.sa_mask);
543                // The lost-wakeup guard: see this module's header.
544                libc::sigaddset(&raw mut suspend.sa_mask, libc::SIGUSR2);
545                // A task frozen mid-`read` should resume the read, not see
546                // `EINTR` it never asked about.
547                suspend.sa_flags = libc::SA_RESTART;
548                libc::sigaction(libc::SIGUSR1, &raw const suspend, core::ptr::null_mut());
549
550                let mut resume: libc::sigaction = core::mem::zeroed();
551                resume.sa_sigaction = on_resume as *const () as libc::sighandler_t;
552                libc::sigemptyset(&raw mut resume.sa_mask);
553                resume.sa_flags = libc::SA_RESTART;
554                libc::sigaction(libc::SIGUSR2, &raw const resume, core::ptr::null_mut());
555            }
556        });
557    }
558
559    /// The OS handle a `JoinHandle` owns, registered so it can be frozen.
560    ///
561    /// Taken from the spawner so the thread has a handle before it has
562    /// run, exactly as on Windows: a thread that registered itself would
563    /// have a window in which it holds the run permit and cannot be
564    /// frozen.
565    #[must_use]
566    pub fn thread_handle_of(join: &std::thread::JoinHandle<()>) -> u64 {
567        use std::os::unix::thread::JoinHandleExt as _;
568
569        install_handlers();
570        let tid = tid_of(join.as_pthread_t());
571        if tid == 0 {
572            return 0;
573        }
574        for (index, entry) in TABLE.iter().enumerate() {
575            // Claim the entry with the store that publishes it, so two
576            // threads spawning at once cannot take the same one.
577            if entry
578                .tid
579                .compare_exchange(0, tid, Ordering::AcqRel, Ordering::Acquire)
580                .is_ok()
581            {
582                entry.state.store(RUNNING, Ordering::Release);
583                return index.saturating_add(1) as u64;
584            }
585        }
586        // Full. Zero is the port's "no thread", so it declines to freeze
587        // this one rather than freezing somebody else.
588        0
589    }
590
591    /// Freeze the thread, and do not answer until it really is frozen.
592    #[expect(unsafe_code, reason = "signalling a thread is a syscall")]
593    pub fn freeze(handle: u64) -> bool {
594        let Some(entry) = entry_at(handle) else {
595            return false;
596        };
597        let tid = entry.tid.load(Ordering::Acquire);
598        if tid == 0 {
599            return false;
600        }
601        if entry
602            .state
603            .compare_exchange(RUNNING, ASKED, Ordering::AcqRel, Ordering::Acquire)
604            .is_err()
605        {
606            // Already asked or already parked. Not this caller's to take.
607            return false;
608        }
609        // SAFETY: `tid` was registered by `thread_handle_of` from a live
610        // `JoinHandle` the port still owns, and cleared by `release`
611        // before that thread's slot is reused.
612        if unsafe { libc::pthread_kill(tid as libc::pthread_t, libc::SIGUSR1) } != 0 {
613            entry.state.store(RUNNING, Ordering::Release);
614            return false;
615        }
616        for _ in 0..PARK_SPINS {
617            if entry.state.load(Ordering::Acquire) == PARKED {
618                return true;
619            }
620            std::thread::yield_now();
621        }
622        // It never parked. Put the state back: the handler, if it runs at
623        // all now, will find its `ASKED -> PARKED` swap refused and return
624        // without parking, which is the unwind this depends on.
625        entry.state.store(RUNNING, Ordering::Release);
626        false
627    }
628
629    /// Thaw the thread. `false` if it was not parked.
630    #[expect(unsafe_code, reason = "signalling a thread is a syscall")]
631    pub fn thaw(handle: u64) -> bool {
632        let Some(entry) = entry_at(handle) else {
633            return false;
634        };
635        let tid = entry.tid.load(Ordering::Acquire);
636        if tid == 0 {
637            return false;
638        }
639        // The state goes back FIRST, so the handler's loop sees `RUNNING`
640        // when the signal wakes it. The other order is a thread that wakes
641        // up, re-reads `PARKED`, and suspends itself again for ever.
642        if entry.state.swap(RUNNING, Ordering::AcqRel) != PARKED {
643            return false;
644        }
645        // SAFETY: as `freeze`.
646        unsafe { libc::pthread_kill(tid as libc::pthread_t, libc::SIGUSR2) == 0 }
647    }
648
649    /// Give the entry back. There is no OS handle to close on Unix.
650    pub fn release(handle: u64) {
651        let Some(entry) = entry_at(handle) else {
652            return;
653        };
654        entry.state.store(RUNNING, Ordering::Release);
655        entry.tid.store(0, Ordering::Release);
656    }
657}
658
659#[cfg(not(any(windows, unix)))]
660mod backend {
661    //! Everywhere else: **cooperative only**, and it says so rather than
662    //! pretending.
663    //!
664    //! Windows and Unix both have a way to stop a thread that is not
665    //! cooperating. A platform that is neither does not necessarily, and
666    //! guessing at one would be worse than reporting the truth: on this
667    //! platform a task that never blocks and never yields will not be
668    //! taken off the CPU, and the demos that rely on preemption
669    //! (`integer.c`, `flop.c`) will starve their neighbours.
670    //!
671    //! [`super::PREEMPTIVE`] is the honest report of that, and a cell
672    //! should check it rather than assume.
673
674    /// Is preemption available on this platform?
675    pub const PREEMPTIVE: bool = false;
676
677    /// No handle to take.
678    #[must_use]
679    pub fn thread_handle_of(_join: &std::thread::JoinHandle<()>) -> u64 {
680        0
681    }
682
683    /// Cannot freeze a thread here.
684    pub fn freeze(_handle: u64) -> bool {
685        false
686    }
687
688    /// Nothing was frozen, so nothing thaws.
689    pub fn thaw(_handle: u64) -> bool {
690        false
691    }
692
693    /// Nothing to release.
694    pub fn release(_handle: u64) {}
695}
696
697/// Whether this platform can take a task off the CPU that does not
698/// cooperate.
699///
700/// A cell that runs the standard demo files should refuse to report a pass
701/// when this is false and `integer`-shaped tasks are in the mix: they
702/// never block, so without preemption everything below them starves and
703/// the failure looks like somebody else's bug.
704pub const PREEMPTIVE: bool = backend::PREEMPTIVE;
705
706// --------------------------------------------------------- the switcher --
707
708/// What the port calls to choose the next task.
709///
710/// It must set [`CURRENT`] to the incoming task's index; whatever it
711/// leaves there is what runs next. The same contract as the Cortex-M
712/// port's `CURRENT_SP_SLOT`, one level up: there the scheduler names a
713/// stack pointer, here it names a thread.
714pub type Scheduler = extern "C" fn();
715
716static SCHEDULER: AtomicUsize = AtomicUsize::new(0);
717
718/// Install the function the port calls to pick the next task.
719pub fn set_scheduler(f: Scheduler) {
720    SCHEDULER.store(f as usize, Ordering::SeqCst);
721}
722
723fn ask_scheduler() {
724    let f = SCHEDULER.load(Ordering::SeqCst);
725    if f != 0 {
726        // SAFETY: `f` was stored by `set_scheduler` from a `Scheduler`,
727        // and nothing else ever writes this.
728        #[expect(unsafe_code, reason = "the installed scheduler is a fn pointer")]
729        let f: Scheduler = unsafe { core::mem::transmute::<usize, Scheduler>(f) };
730        f();
731    }
732}
733
734/// Permits handed to each slot, so a cell can ask who actually got the CPU.
735#[allow(clippy::declare_interior_mutable_const)]
736const NO_GRANTS: AtomicU64 = AtomicU64::new(0);
737static GRANTS: [AtomicU64; MAX_TASKS] = [NO_GRANTS; MAX_TASKS];
738
739/// How many times slot `index` was handed the run permit.
740#[must_use]
741pub fn grants_to(index: usize) -> u64 {
742    GRANTS.get(index).map_or(0, |c| c.load(Ordering::Relaxed))
743}
744
745/// Give `index` the run permit.
746fn grant(index: usize) {
747    if let Some(c) = GRANTS.get(index) {
748        c.fetch_add(1, Ordering::Relaxed);
749    }
750    let Some(slot) = slots().get(index) else {
751        return;
752    };
753    if slot.frozen.swap(false, Ordering::SeqCst) {
754        // It was taken off the CPU by the tick thread, not parked, so it
755        // is not waiting on anything and must be thawed instead.
756        backend::thaw(slot.os_thread.load(Ordering::SeqCst));
757        return;
758    }
759    let Ok(mut granted) = slot.granted.lock() else {
760        return;
761    };
762    *granted = true;
763    // `notify_all`, not `notify_one`: an orphaned thread from an earlier
764    // occupant of this slot may be waiting on the same condvar, and
765    // waking only one of them could wake the wrong one. Each checks its
766    // own generation and the loser goes back to sleep for good.
767    slot.wake.notify_all();
768}
769
770/// Wait until this task has the run permit, or forever if this thread is
771/// no longer the slot's occupant.
772fn await_permit(index: usize, generation: u64) {
773    let Some(slot) = slots().get(index) else {
774        return;
775    };
776    let Ok(granted) = slot.granted.lock() else {
777        return;
778    };
779    take_permit(slot, generation, granted);
780}
781
782/// The waiting half, given a guard the caller already holds.
783///
784/// `pend_switch` must take this lock BEFORE it lets anything else run, so
785/// the wait cannot be a self-contained function -- it has to be handed the
786/// guard.
787fn take_permit(slot: &Slot, generation: u64, mut granted: std::sync::MutexGuard<'_, bool>) {
788    loop {
789        if slot.generation.load(Ordering::SeqCst) != generation {
790            // A later task has this slot. This thread belongs to a task
791            // that no longer exists and must never run again.
792            let Ok(next) = slot.wake.wait(granted) else {
793                return;
794            };
795            granted = next;
796            continue;
797        }
798        if *granted {
799            *granted = false;
800            return;
801        }
802        let Ok(next) = slot.wake.wait(granted) else {
803            return;
804        };
805        granted = next;
806    }
807}
808
809/// `portYIELD()`: ask the scheduler who is next and hand the CPU over.
810///
811/// Returns when this task is next given the permit, which is what makes a
812/// blocking C call possible: the locals are on this thread's own stack and
813/// are still there when it comes back.
814pub fn pend_switch() {
815    let who = me();
816    if CRITICAL.owner.load(Ordering::Acquire) == who {
817        // We are inside a critical section, which on this port means
818        // inside a kernel call. This is `PendSV` being PENDED, not taken:
819        // record it, and `exit_critical` runs it on the way out.
820        //
821        // Taking it here instead re-enters the kernel while an outer call
822        // still holds a `&mut` to it -- and the kernel's idea of the
823        // current task then moves under the feet of the call in progress,
824        // so every API that means "the calling task" names the wrong one.
825        // `GenQTest` ends a test with `vTaskPrioritySet( NULL,
826        // genqMUTEX_LOW_PRIORITY )`, and the task set to low priority was
827        // the HIGH priority one.
828        PENDING.store(true, Ordering::SeqCst);
829        return;
830    }
831    switch_now(who);
832}
833
834/// Set when a switch has been asked for and not yet taken. One flag,
835/// because there is one CPU: this is `ICSR.PENDSVSET`.
836static PENDING: AtomicBool = AtomicBool::new(false);
837
838/// Take the switch. The caller must NOT hold the critical section.
839fn switch_now(who: u64) {
840    // The decision and the handover happen with the tick thread locked
841    // out. Without this the two interleave: a task asks the scheduler, the
842    // tick thread freezes it mid-answer and hands the CPU to somebody
843    // else, and when the frozen task is later thawed it finishes a
844    // handover whose answer is now stale -- granting the permit to a task
845    // that should not have it, so two run at once.
846    //
847    // That race is not theoretical and it is not loud. It shows up as
848    // `eTaskGetState( x ) == eBlocked` failing three statements after a
849    // `vTaskResume`, which reads as a scheduler bug rather than a port
850    // one.
851    CRITICAL.enter(who);
852    let mine = CURRENT.load(Ordering::SeqCst);
853    ask_scheduler();
854    let next = CURRENT.load(Ordering::SeqCst);
855    if next == mine || next == NO_TASK || mine == NO_TASK {
856        CRITICAL.exit(who);
857        return;
858    }
859    let Some(slot) = slots().get(mine) else {
860        CRITICAL.exit(who);
861        return;
862    };
863    // Our own permit lock, taken BEFORE anything else can run, so a grant
864    // aimed at us cannot land between the handover and the wait.
865    let Ok(granted) = slot.granted.lock() else {
866        CRITICAL.exit(who);
867        return;
868    };
869    let generation = slot.generation.load(Ordering::SeqCst);
870    grant(next);
871
872    // Put the critical section down completely while parked, and pick it
873    // up at the same depth on the way back. See `release_all`.
874    let depth = CRITICAL.release_all(who);
875
876    take_permit(slot, generation, granted);
877
878    CRITICAL.reacquire(who, depth);
879    // ...and undo the level this function took on the way in.
880    CRITICAL.exit(who);
881}
882
883/// Give task `index` a thread, parked until it is granted the permit.
884///
885/// `entry` is handed the task index, exactly as `init_stack` does on the
886/// Cortex-M port, so one trampoline can serve every task.
887pub fn init_task(index: usize, entry: extern "C" fn(usize) -> !) {
888    let Some(slot) = slots().get(index) else {
889        return;
890    };
891    // A slot that already had a thread gets a NEW one, and the old one is
892    // orphaned rather than reused: it is parked inside the C function of a
893    // task that no longer exists, and there is no safe way to bring it
894    // back to the top. Bumping the generation is what stops it competing
895    // for the permit.
896    let generation = slot
897        .generation
898        .fetch_add(1, Ordering::SeqCst)
899        .wrapping_add(1);
900    if generation > 1 {
901        ORPHANS.fetch_add(1, Ordering::Relaxed);
902        // Wake the outgoing occupant so it can see its generation has
903        // passed and settle; and give its handle back.
904        //
905        // And clear the two flags that describe the OLD thread, which a new
906        // occupant must not inherit. The tick can freeze a thread in the gap
907        // between its grant and its waking to take it; if that task is then
908        // deleted, the slot still says both "granted" and "frozen":
909        //
910        // * a standing GRANT lets the new thread run the moment it starts,
911        //   uninvited, while the kernel has somebody else current -- on
912        //   Linux, "this thread is task 77, the kernel believes 43";
913        // * a standing FREEZE makes the new occupant's first grant try to
914        //   THAW a thread that is not frozen -- a no-op -- instead of
915        //   handing it the permit, so the kernel's current task runs nowhere
916        //   until the next tick, and every tick it is chosen is lost
917        //   (`TimerDemo`'s exact-tick ISR checks fail).
918        //
919        // Clearing one alone produced the other's failure; both were
920        // measured, with `death.c` deleting a SUICID task the tick had just
921        // interrupted. The counter says how often a reuse found either.
922        if let Ok(mut granted) = slot.granted.lock() {
923            let stale_grant = core::mem::replace(&mut *granted, false);
924            let stale_freeze = slot.frozen.swap(false, Ordering::SeqCst);
925            if stale_grant || stale_freeze {
926                STALE_GRANTS.fetch_add(1, Ordering::Relaxed);
927            }
928            drop(granted);
929            slot.wake.notify_all();
930        }
931        backend::release(slot.os_thread.swap(0, Ordering::SeqCst));
932    }
933    let spawned = std::thread::Builder::new()
934        .name(format!("kairos-task-{index}.{generation}"))
935        .stack_size(STACK_BYTES)
936        .spawn(move || {
937            MY_INDEX.set(Some(index));
938            await_permit(index, generation);
939            entry(index);
940        });
941
942    // Register the thread from HERE, not from inside it.
943    //
944    // A thread that names itself has a window in which it exists, may hold
945    // the run permit, and has no handle -- and a tick landing in that
946    // window cannot freeze it. That is not a missed tick: it is the one
947    // case where the port would have to decline a switch, which is the
948    // path that used to leave the kernel and the port disagreeing about
949    // who was running.
950    //
951    // The `JoinHandle` owns the OS thread handle, so it is kept: dropping
952    // it would close the handle and every later freeze would fail.
953    if let Ok(handle) = spawned {
954        if let Some(slot) = slots().get(index) {
955            slot.os_thread
956                .store(backend::thread_handle_of(&handle), Ordering::SeqCst);
957        }
958        if let Ok(mut live) = threads().lock() {
959            live.push(handle);
960        }
961    }
962}
963
964thread_local! {
965    /// Which task this thread IS, as opposed to which task the kernel
966    /// believes is running.
967    ///
968    /// On a chip those cannot differ: there is one CPU and the kernel's
969    /// `current` is definitionally who is on it. Here they are two
970    /// different facts kept in two places, so they CAN differ -- and when
971    /// they do, every API that means "the calling task" quietly names
972    /// somebody else. Having the thread able to state its own identity is
973    /// what makes that checkable rather than arguable.
974    static MY_INDEX: core::cell::Cell<Option<usize>> = const { core::cell::Cell::new(None) };
975}
976
977/// Is a context switch outstanding but not yet taken? (`ICSR.PENDSVSET`.)
978#[must_use]
979pub fn switch_pending() -> bool {
980    PENDING.load(Ordering::SeqCst)
981}
982
983/// How deep the calling thread is inside critical sections, and whether it
984/// is the holder at all.
985#[must_use]
986pub fn critical_depth() -> (bool, u32) {
987    let mine = CRITICAL.owner.load(Ordering::Acquire) == me();
988    (mine, CRITICAL.nesting.load(Ordering::Relaxed))
989}
990
991/// Which task the CALLING THREAD is, if it is a task thread at all.
992#[must_use]
993pub fn my_index() -> Option<usize> {
994    MY_INDEX.with(core::cell::Cell::get)
995}
996
997/// What the tick thread did, tick by tick.
998///
999/// A port that declines to switch reports the same thing as a port that
1000/// was never asked, and those are different bugs -- so it counts both.
1001static TICKS_SEEN: AtomicU64 = AtomicU64::new(0);
1002static TICKS_WANTED_SWITCH: AtomicU64 = AtomicU64::new(0);
1003static TICKS_COULD_NOT_FREEZE: AtomicU64 = AtomicU64::new(0);
1004static TICKS_SWITCHED: AtomicU64 = AtomicU64::new(0);
1005
1006/// `(ticks, wanted a switch, could not freeze, actually switched)`.
1007#[must_use]
1008pub fn tick_stats() -> (u64, u64, u64, u64) {
1009    (
1010        TICKS_SEEN.load(Ordering::Relaxed),
1011        TICKS_WANTED_SWITCH.load(Ordering::Relaxed),
1012        TICKS_COULD_NOT_FREEZE.load(Ordering::Relaxed),
1013        TICKS_SWITCHED.load(Ordering::Relaxed),
1014    )
1015}
1016
1017/// Bytes of OS stack per task thread.
1018///
1019/// Every orphaned thread keeps one until the process ends, and `death.c`
1020/// makes orphans on purpose, so this is not purely a depth question. The
1021/// deepest a demo task reached on the Cortex-M3 cell, measured by stack
1022/// painting, was 122 words; a host frame is wider but not by three orders
1023/// of magnitude.
1024const STACK_BYTES: usize = 128 * 1024;
1025
1026/// How many task threads have been orphaned by a slot being reused.
1027///
1028/// Reported rather than hidden: each one holds [`STACK_BYTES`] of address
1029/// space for the life of the process, and a cell that makes thousands of
1030/// them should be able to see that it is doing so. Terminating them
1031/// instead is what FreeRTOS's Posix port does with `pthread_cancel`, and
1032/// is the obvious next step for this port.
1033static ORPHANS: AtomicU64 = AtomicU64::new(0);
1034
1035/// Slots handed to a new occupant while a grant to, or a freeze of, the old
1036/// one was still standing (see `init_task`).
1037static STALE_GRANTS: AtomicU64 = AtomicU64::new(0);
1038
1039/// How many slots `init_task` found holding a stale grant or freeze.
1040#[must_use]
1041pub fn stale_grants() -> u64 {
1042    STALE_GRANTS.load(Ordering::Relaxed)
1043}
1044
1045/// How many task threads have been orphaned by a slot being reused.
1046#[must_use]
1047pub fn orphaned_threads() -> u64 {
1048    ORPHANS.load(Ordering::Relaxed)
1049}
1050
1051/// The spawned task threads, kept alive so their OS handles stay valid.
1052fn threads() -> &'static Mutex<Vec<std::thread::JoinHandle<()>>> {
1053    static THREADS: OnceLock<Mutex<Vec<std::thread::JoinHandle<()>>>> = OnceLock::new();
1054    THREADS.get_or_init(|| Mutex::new(Vec::new()))
1055}
1056
1057/// Start the first task and never come back.
1058///
1059/// The calling thread becomes the idle of last resort: it parks, because
1060/// every task now has its own thread and this one has nothing left to do.
1061pub fn start_first_task(index: usize) -> ! {
1062    CURRENT.store(index, Ordering::SeqCst);
1063    grant(index);
1064    loop {
1065        // Long enough that this thread costs nothing, short enough that a
1066        // cell which exits can do so promptly.
1067        std::thread::sleep(Duration::from_millis(50));
1068    }
1069}
1070
1071// ------------------------------------------------------------ the tick --
1072
1073/// What the tick thread calls once per period.
1074///
1075/// It stands where `SysTick` does on a chip: it is called with the
1076/// critical-section lock HELD and the running task frozen, so it may
1077/// touch the kernel freely. `true` asks for a context switch, exactly as
1078/// `xTaskIncrementTick` answering `pdTRUE` does.
1079pub type TickFn = extern "C" fn() -> bool;
1080
1081/// The tick thread's own handle on the system.
1082#[derive(Debug)]
1083pub struct Ticker {
1084    period: Duration,
1085    on_tick: TickFn,
1086}
1087
1088impl Ticker {
1089    /// A ticker that calls `on_tick` every `period`.
1090    #[must_use]
1091    pub const fn new(period: Duration, on_tick: TickFn) -> Self {
1092        Self { period, on_tick }
1093    }
1094
1095    /// Run the ticker on its own thread, forever.
1096    pub fn spawn(self) {
1097        let _ = std::thread::Builder::new()
1098            .name("kairos-tick".to_owned())
1099            .spawn(move || {
1100                loop {
1101                    std::thread::sleep(self.period);
1102                    self.interrupt();
1103                }
1104            });
1105    }
1106
1107    /// One tick, in the order that makes freezing a thread safe.
1108    ///
1109    /// 1. Take the critical-section lock. This WAITS for the running task
1110    ///    to leave any critical section, so when it is held the running
1111    ///    task is outside the kernel and outside our heap.
1112    /// 2. Freeze the running task. It cannot be holding a lock of ours,
1113    ///    by step 1.
1114    /// 3. Do the kernel's tick work and ask who is next.
1115    /// 4. Release the lock, then thaw or grant the incoming task.
1116    ///
1117    /// Steps 1 and 2 in the other order is the classic deadlock: freeze a
1118    /// task inside `pvPortMalloc` and the next task to allocate waits for
1119    /// a lock whose holder is never going to run again.
1120    fn interrupt(&self) {
1121        let me = me();
1122        CRITICAL.enter(me);
1123        IN_TICK.store(true, Ordering::SeqCst);
1124
1125        TICKS_SEEN.fetch_add(1, Ordering::Relaxed);
1126        let running = CURRENT.load(Ordering::SeqCst);
1127        let frozen = self.freeze_running(running);
1128        if !frozen {
1129            TICKS_COULD_NOT_FREEZE.fetch_add(1, Ordering::Relaxed);
1130        }
1131
1132        let want_switch = (self.on_tick)();
1133        if want_switch {
1134            TICKS_WANTED_SWITCH.fetch_add(1, Ordering::Relaxed);
1135        }
1136
1137        // Ask the scheduler ONLY when its answer can be honoured.
1138        //
1139        // The scheduler is not a query: `switch_context` MOVES the
1140        // kernel's idea of the current task. Asking it and then declining
1141        // to switch -- because the running thread could not be frozen --
1142        // leaves the kernel and this port disagreeing about who is
1143        // running, and the running task keeps executing under somebody
1144        // else's identity. Every API that means "the calling task" then
1145        // names the wrong one: `vTaskPrioritySet( NULL, x )` sets a task
1146        // that is not the caller, `uxTaskPriorityGet( NULL )` answers for
1147        // a task that is not asking.
1148        //
1149        // `GenQTest` catches it precisely, because it ends a test with
1150        // `vTaskPrioritySet( NULL, genqMUTEX_LOW_PRIORITY )` -- and the
1151        // task that was wrongly set to low priority was the HIGH priority
1152        // one, three hundred lines from the assertion that noticed.
1153        let next = if want_switch && frozen {
1154            ask_scheduler();
1155            CURRENT.load(Ordering::SeqCst)
1156        } else {
1157            running
1158        };
1159
1160        IN_TICK.store(false, Ordering::SeqCst);
1161        // A switch the kernel asked for during the tick has been taken
1162        // here, or declined here; either way it is not still outstanding.
1163        PENDING.store(false, Ordering::SeqCst);
1164        CRITICAL.exit(me);
1165
1166        if next == running || next == NO_TASK {
1167            // Nothing to change. Put the task back exactly as it was.
1168            //
1169            // A switch the tick wanted and could not take is not lost, only
1170            // deferred: the running task's next kernel call asks the
1171            // scheduler again, and the ready higher-priority task is still
1172            // ready.
1173            if frozen {
1174                if let Some(slot) = slots().get(running) {
1175                    slot.frozen.store(false, Ordering::SeqCst);
1176                    backend::thaw(slot.os_thread.load(Ordering::SeqCst));
1177                }
1178            }
1179            return;
1180        }
1181        TICKS_SWITCHED.fetch_add(1, Ordering::Relaxed);
1182        grant(next);
1183    }
1184
1185    /// Freeze the running task, and record that it was frozen rather than
1186    /// parked so [`grant`] thaws it instead of signalling a condvar
1187    /// nobody is waiting on.
1188    fn freeze_running(&self, running: usize) -> bool {
1189        if running == NO_TASK || !preemption_enabled() {
1190            return false;
1191        }
1192        let Some(slot) = slots().get(running) else {
1193            return false;
1194        };
1195        let handle = slot.os_thread.load(Ordering::SeqCst);
1196        if !backend::freeze(handle) {
1197            return false;
1198        }
1199        slot.frozen.store(true, Ordering::SeqCst);
1200        true
1201    }
1202}
1203
1204/// Turn preemption off at runtime, with `KAIROS_HOST_NO_PREEMPT=1`.
1205///
1206/// Two uses, and both are about not having to take this port's word for
1207/// anything:
1208///
1209/// * **The poison test.** A cell that claims "a task that never yields was
1210///   taken off the CPU" has to be able to show the claim FAILING when the
1211///   mechanism is removed. Without that, the check might be passing for
1212///   some other reason and nobody would know.
1213/// * **Standing in for the platforms that do not have it.** Windows and
1214///   Unix both preempt; a platform that is neither reports `PREEMPTIVE`
1215///   false, and this is how that is reproduced on the machine you have
1216///   rather than discovered on the machine you do not.
1217fn preemption_enabled() -> bool {
1218    static ENABLED: OnceLock<bool> = OnceLock::new();
1219    *ENABLED.get_or_init(|| std::env::var("KAIROS_HOST_NO_PREEMPT").as_deref() != Ok("1"))
1220}
1221
1222/// True while the tick thread is inside [`Ticker::interrupt`].
1223///
1224/// This port's only interrupt context. It is a flag rather than a magic
1225/// owner id in the critical lock because the lock's identity has to be the
1226/// SAME function everywhere -- `pend_switch` compares against it to decide
1227/// whether it is being pended or taken, and a tick thread that held the
1228/// lock under a different name would take a switch from inside the tick.
1229static IN_TICK: AtomicBool = AtomicBool::new(false);
1230
1231/// The OS thread backing task `index`, or zero if it has none yet.
1232///
1233/// A cell reports this the way a bare-metal cell reports a saved stack
1234/// pointer: a slot with an entry point and no thread is a task that is
1235/// SCHEDULABLE WITH NOWHERE TO RUN, which is a real bug shape and one that
1236/// presents a long way from its cause.
1237#[must_use]
1238pub fn thread_of(index: usize) -> u64 {
1239    slots()
1240        .get(index)
1241        .map_or(0, |s| s.os_thread.load(Ordering::SeqCst))
1242}
1243
1244/// Hand a task slot's OS handle back. For a cell that deletes tasks.
1245pub fn release_task(index: usize) {
1246    if let Some(slot) = slots().get(index) {
1247        backend::release(slot.os_thread.swap(0, Ordering::SeqCst));
1248    }
1249}
1250
1251// ------------------------------------------------------ virtual cores --
1252
1253/// Which kernel core the code holding the run permit is running as.
1254///
1255/// This port runs ONE task thread at a time, whatever the kernel's core
1256/// count. A two-core cell keeps that, and takes turns: it tells the port
1257/// which core the permit currently stands for, and the kernel reads it back
1258/// through [`Port::core_id`] -- `portGET_CORE_ID()`. Two cores this way are
1259/// virtual: every two-core path in the kernel is exercised (per-core
1260/// current tasks, cross-core yields, selection that skips a task the other
1261/// core holds), while the kernel is still never entered by two threads at
1262/// once. The policy of WHEN a turn passes belongs to the cell; this is only
1263/// where the answer is kept.
1264static CORE: core::sync::atomic::AtomicU8 = core::sync::atomic::AtomicU8::new(0);
1265
1266/// Make the run permit stand for kernel core `core` from now on.
1267pub fn set_core(core: u8) {
1268    CORE.store(core, Ordering::SeqCst);
1269}
1270
1271/// The kernel core the run permit currently stands for.
1272#[must_use]
1273pub fn core() -> u8 {
1274    CORE.load(Ordering::SeqCst)
1275}
1276
1277// -------------------------------------------------------------- the port --
1278
1279/// The host [`Port`].
1280#[derive(Debug, Default)]
1281pub struct HostPort {
1282    yields: AtomicU64,
1283    ticks: AtomicU64,
1284}
1285
1286impl HostPort {
1287    /// A port with nothing counted yet.
1288    #[must_use]
1289    pub const fn new() -> Self {
1290        Self {
1291            yields: AtomicU64::new(0),
1292            ticks: AtomicU64::new(0),
1293        }
1294    }
1295
1296    /// Ticks delivered so far.
1297    #[must_use]
1298    pub fn tick_count(&self) -> u64 {
1299        self.ticks.load(Ordering::Relaxed)
1300    }
1301
1302    /// `portYIELD()` calls so far.
1303    #[must_use]
1304    pub fn yield_count(&self) -> u64 {
1305        self.yields.load(Ordering::Relaxed)
1306    }
1307
1308    /// Count a tick the ticker delivered.
1309    pub fn note_tick(&self) {
1310        self.ticks.fetch_add(1, Ordering::Relaxed);
1311    }
1312}
1313
1314/// The identity the critical lock uses for the calling thread.
1315///
1316/// A thread id, not a handle: it only has to be unique and non-zero, and
1317/// `ThreadId` is neither a number nor stable across runs, so it is hashed
1318/// into one.
1319fn me() -> u64 {
1320    use std::hash::{Hash as _, Hasher as _};
1321    let mut h = std::collections::hash_map::DefaultHasher::new();
1322    std::thread::current().id().hash(&mut h);
1323    // Zero means "the lock is free", so it is the one value that may not
1324    // be handed out.
1325    match h.finish() {
1326        0 => 1,
1327        other => other,
1328    }
1329}
1330
1331impl Port for HostPort {
1332    /// This port COMMITS the switch.
1333    ///
1334    /// `pend_switch` does not merely note a request: it asks the scheduler,
1335    /// hands the run permit over and parks the caller, so the decision and
1336    /// the change of running task are one step -- exactly what every
1337    /// bare-metal port here declares, for the same reason.
1338    ///
1339    /// Leaving it `false` was a real defect and a quiet one. The kernel
1340    /// then treats itself as STACKLESS and moves `current` inside
1341    /// `port_yield` by calling `switch_context` directly; the port then
1342    /// switches AGAIN on the way out. Two selections per yield, and the
1343    /// round robin advances twice -- which on a ready list of two is the
1344    /// same task every time, for ever.
1345    ///
1346    /// That is how `EventGroups` and `dynamic` stopped dead in the C ABI
1347    /// host cell while a busier demo's equal-priority tasks took a
1348    /// thousandfold share, and how one of two never-yielding tasks in
1349    /// `firmware/host-kernel` got 13.5 million laps against ZERO.
1350    ///
1351    /// The trait's own doc names the other half of the cost: between the
1352    /// kernel moving `current` and the port catching up, "code runs as a
1353    /// task the kernel no longer thinks is current".
1354    const COMMITS_SWITCH: bool = true;
1355
1356    fn yield_now(&self) {
1357        self.yields.fetch_add(1, Ordering::Relaxed);
1358        pend_switch();
1359    }
1360
1361    fn yield_from_isr(&self, woken: Woken) {
1362        if woken.needed() {
1363            self.yields.fetch_add(1, Ordering::Relaxed);
1364            pend_switch();
1365        }
1366    }
1367
1368    fn enter_critical(&self) {
1369        CRITICAL.enter(me());
1370    }
1371
1372    fn exit_critical(&self) {
1373        let who = me();
1374        // Released, not merely counted down: this is the outermost exit,
1375        // which on a chip is where `cpsie i` lets the pending `PendSV`
1376        // finally run. Same place, same reason.
1377        if CRITICAL.exit(who)
1378            && !IN_TICK.load(Ordering::SeqCst)
1379            && PENDING.swap(false, Ordering::SeqCst)
1380        {
1381            switch_now(who);
1382        }
1383    }
1384
1385    fn set_interrupt_mask_from_isr(&self) -> u32 {
1386        let who = me();
1387        let already = CRITICAL.owner.load(Ordering::Acquire) == who;
1388        if !already {
1389            CRITICAL.enter(who);
1390        }
1391        u32::from(already)
1392    }
1393
1394    fn clear_interrupt_mask_from_isr(&self, saved: u32) {
1395        // Restore, do not release: a mask taken inside somebody else's
1396        // critical section must leave that section standing. The Cortex-M
1397        // port learned this the hard way -- an unconditional release there
1398        // ended a caller's critical section halfway through and let a
1399        // half-created task be scheduled.
1400        if saved == 0 {
1401            CRITICAL.exit(me());
1402        }
1403    }
1404
1405    fn in_isr(&self) -> bool {
1406        // The tick thread is this port's only interrupt context, and only
1407        // one context runs at a time -- so if the tick is inside, the
1408        // caller IS the tick.
1409        IN_TICK.load(Ordering::SeqCst)
1410    }
1411
1412    /// `portGET_CORE_ID()`: the core the run permit stands for -- always 0
1413    /// unless a two-core cell has passed a turn ([`set_core`]).
1414    fn core_id(&self) -> u8 {
1415        core()
1416    }
1417
1418    fn count_tick(&self) {
1419        self.note_tick();
1420    }
1421
1422    fn count_yield(&self) {
1423        self.yields.fetch_add(1, Ordering::Relaxed);
1424    }
1425}
1426
1427#[cfg(test)]
1428#[allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)]
1429mod tests {
1430    use super::*;
1431
1432    #[test]
1433    fn a_critical_section_nests_and_only_the_outermost_exit_releases() {
1434        let who = 7;
1435        CRITICAL.enter(who);
1436        assert_ne!(CRITICAL.owner.load(Ordering::Acquire), 0);
1437        CRITICAL.enter(who);
1438        assert!(!CRITICAL.exit(who), "the inner exit must not release");
1439        assert_ne!(CRITICAL.owner.load(Ordering::Acquire), 0);
1440        assert!(CRITICAL.exit(who), "the outer exit releases");
1441        assert_eq!(CRITICAL.owner.load(Ordering::Acquire), 0);
1442    }
1443
1444    #[test]
1445    fn a_thread_cannot_release_a_critical_section_it_does_not_hold() {
1446        CRITICAL.enter(11);
1447        assert!(!CRITICAL.exit(12), "12 does not hold it");
1448        assert_ne!(CRITICAL.owner.load(Ordering::Acquire), 0, "11 still does");
1449        assert!(CRITICAL.exit(11));
1450    }
1451
1452    #[test]
1453    fn the_thread_identity_is_never_the_free_marker() {
1454        assert_ne!(me(), 0, "zero means the lock is free");
1455    }
1456
1457    /// A yield asked for from inside a critical section must be PENDED,
1458    /// not taken -- that is the whole of what `PendSV` being the
1459    /// lowest-priority exception buys on a chip.
1460    #[test]
1461    fn a_yield_inside_a_critical_section_is_pended_rather_than_taken() {
1462        let who = me();
1463        PENDING.store(false, Ordering::SeqCst);
1464        CRITICAL.enter(who);
1465        pend_switch();
1466        assert!(
1467            PENDING.load(Ordering::SeqCst),
1468            "the switch should be outstanding, not taken"
1469        );
1470        PENDING.store(false, Ordering::SeqCst);
1471        assert!(CRITICAL.exit(who));
1472    }
1473
1474    #[test]
1475    fn the_slot_table_is_the_size_it_says() {
1476        assert_eq!(slots().len(), MAX_TASKS);
1477    }
1478
1479    /// The port must not claim preemption it does not have. A cell reads
1480    /// this to decide whether a demo that never yields can be trusted.
1481    #[test]
1482    fn preemption_is_reported_honestly_for_this_platform() {
1483        assert_eq!(PREEMPTIVE, cfg!(any(windows, unix)));
1484    }
1485
1486    /// Freezing a thread that is **running** is the whole of preemption,
1487    /// so it gets a test that needs no kernel, no cell and no emulator.
1488    ///
1489    /// The subject does nothing but increment a counter, which is the
1490    /// point: it never blocks, never yields and never takes a lock, so
1491    /// the ONLY thing that can stop it is the port. If the counter stops
1492    /// moving while it is frozen and moves again after the thaw, the
1493    /// mechanism works; if `freeze` returned `true` without actually
1494    /// stopping it, this fails — and that is the failure worth catching,
1495    /// because a `freeze` that answers before the thread has stopped puts
1496    /// two tasks on the CPU at once and the damage shows up somewhere
1497    /// else entirely.
1498    #[test]
1499    fn a_running_thread_is_frozen_where_it_stands_and_thaws_again() {
1500        if !PREEMPTIVE {
1501            // Nothing to assert on a platform that says it cannot.
1502            return;
1503        }
1504        if cfg!(miri) {
1505            // Miri interprets Rust; it does not run `SuspendThread` and it
1506            // does not deliver signals. This test IS the foreign call, so
1507            // under Miri there is nothing here it can check -- and the
1508            // package's Miri row is a real gate, not one to break with a
1509            // test that was never going to run under it.
1510            return;
1511        }
1512        static SPINS: AtomicU64 = AtomicU64::new(0);
1513        static STOP: AtomicBool = AtomicBool::new(false);
1514
1515        SPINS.store(0, Ordering::SeqCst);
1516        STOP.store(false, Ordering::SeqCst);
1517
1518        let spinner = std::thread::spawn(|| {
1519            while !STOP.load(Ordering::Relaxed) {
1520                SPINS.fetch_add(1, Ordering::Relaxed);
1521                std::hint::spin_loop();
1522            }
1523        });
1524        let handle = backend::thread_handle_of(&spinner);
1525        assert_ne!(handle, 0, "a spawned thread must have a usable handle");
1526
1527        // Wait for it to be genuinely running, so that what gets frozen
1528        // is a thread in its loop rather than one that has not started.
1529        let mut started = false;
1530        for _ in 0..1_000 {
1531            if SPINS.load(Ordering::SeqCst) > 0 {
1532                started = true;
1533                break;
1534            }
1535            std::thread::sleep(Duration::from_millis(1));
1536        }
1537        assert!(started, "the spinner never ran");
1538
1539        assert!(backend::freeze(handle), "freeze refused a running thread");
1540        let at_freeze = SPINS.load(Ordering::SeqCst);
1541        // Long enough that a thread still on the CPU would move the
1542        // counter by millions, so this is not a close-run thing.
1543        std::thread::sleep(Duration::from_millis(50));
1544        assert_eq!(
1545            SPINS.load(Ordering::SeqCst),
1546            at_freeze,
1547            "a frozen thread kept running"
1548        );
1549
1550        assert!(backend::thaw(handle), "thaw refused a frozen thread");
1551        let mut resumed = false;
1552        for _ in 0..1_000 {
1553            if SPINS.load(Ordering::SeqCst) > at_freeze {
1554                resumed = true;
1555                break;
1556            }
1557            std::thread::sleep(Duration::from_millis(1));
1558        }
1559        assert!(resumed, "a thawed thread never ran again");
1560
1561        STOP.store(true, Ordering::SeqCst);
1562        spinner.join().expect("the spinner should end");
1563        backend::release(handle);
1564    }
1565}