Serene Runtime 1.0.0-dev
C runtime for the Serene programming language
Loading...
Searching...
No Matches
fiber.h
Go to the documentation of this file.
1/* -*- C -*-
2 * Serene programming language
3 * Copyright (C) 2019-2026 Sameer Rahmani <lxsameer@lxsameer.com>
4 *
5 * This library is free software: you can redistribute it and/or modify
6 * it under the terms of the GNU Lesser General Public License as published by
7 * the Free Software Foundation, either version 3 of the License, or
8 * (at your option) any later version.
9 *
10 * This library is distributed in the hope that it will be useful,
11 * but WITHOUT ANY WARRANTY; without even the implied warranty of
12 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13 * GNU Lesser General Public License for more details.
14 *
15 * You should have received a copy of the GNU Lesser General Public License
16 * along with this library. If not, see <https://www.gnu.org/licenses/>.
17 */
18
19// TODO(lxsameer): A bocking IO thread poool:
20// The current implementation has a big weakness. Depending on the backend it
21// might to NOT support async operations on regular files (Likes of epoll,
22// kqueue backend; while io_uring and IOCP do). That means that we are facing
23// with a tricky situation.
24//
25// We can never block the reactor since it is single threaded. We need to either
26// offload the blocking IO operation to a threadpool dedicated for such
27// operations or just do the blocking IO in the fibers themselves.
28// Right now we went with the later option since it is much, much simpler to
29// implement, and we didn't want to overengineer. But the better choice seem to
30// be the thread pool option.
31//
32// That being said, it has it's own problems as well. For example:
33// - We need more parked threads that might not be optimal
34// - We can't post completion from the threadpool since SQs/CQs are single
35// producer/consumer and we need to have Multi producer signle user queues.
36// - The current model would be ok for the majority of use cases.
37//
38// So I guess the point of this todo is to investigate the IO threadpool idea,
39// and implement it if makes sense.
40
41/** @file
42 AI Generated (🤦)
43 Fiber subsystem overview
44
45 Stackful, cooperative fibers for the runtime. A fiber is a function plus its
46 own stack, so it can pause at any call depth and resume later. Nothing is
47 preempted. A fiber runs until it explicitly yields, suspends, or finishes.
48
49 Terminology (used throughout the fiber subsystem)
50 fiber a unit of work with its own stack. It can pause and resume.
51 os thread a thread the operating system schedules.
52 worker the state one os thread uses to run fibers. It holds a local
53 queue of ready fibers and a worker's loop. There is one
54 worker per os thread.
55 worker's loop a special fiber that stands in for the os thread. A running
56 fiber switches back to it to hand control over, and it picks
57 the next fiber to run.
58 worker routine the steps an os thread repeats. It takes a fiber, switches
59 into it, deals with how the fiber gave up control, and parks
60 when no fiber is runnable.
61 scheduler there is exactly one. It owns the shared ready queue and the
62 list of live fibers and decides what runs next. It does not
63 run fibers itself, workers do.
64 suspend a fiber steps off every queue and waits to be woken. Its os
65 thread stays free to run other fibers.
66 park an os thread blocks because no fiber is runnable anywhere. A
67 notify wakes it.
68
69 Every fiber stack is the same fixed size (the configured `fiber.stack_size`), one
70 mapping with a guard page that never grows. Uniform size is a deliberate
71 constraint, not an accident. It lets a finished fiber's stack be handed to any
72 later fiber (the stack ring TODO below) without tracking size classes, and it
73 keeps switch and mapping costs predictable.
74
75 Control flow is a hub. A fiber never jumps to another fiber. It switches back
76 to its worker's loop, which picks the next one.
77
78 scheduler ready queue: [F2] -> [F5] -> ...
79 | the os thread takes the head
80 v
81 +-----------> worker's loop --- switch ---> running fiber
82 | |
83 | yield (re-enqueue, run next) ----------------+
84 | suspend (off-queue until srn_fiber_ready) -----+
85 | finish (reap) --------------------------------+
86 +--------------------------------------------------+
87
88 A running fiber gives up control three ways:
89 - yield still runnable, so it moves to the back of the ready queue.
90 - suspend off every queue until some party calls srn_fiber_ready. Fibers,
91 workers, and the reactor may wake it; the reactor is the only
92 waker outside the worker pool (see srn_fiber_ready).
93 - finish its entry returns and the fiber is reaped.
94
95 Signal handling is not part of this subsystem. No scheduler entry point is
96 async signal safe, and by design: a runtime that owns an event loop folds
97 signals into it rather than promising handler safety call by call.
98 `srn_sched_stop` and `srn_sched_drain` take the scheduler mutex and signal
99 its condition variable, so calling them from a signal handler can
100 self-deadlock. A signal-initiated shutdown routes the signal into ordinary
101 thread context first (a dedicated thread in sigwait, or a signalfd
102 registered with the reactor) and calls stop or drain from there.
103
104 The spec book's fibers chapter (docs/spec/fibers.typ) is the long-form
105 reference. Keep this overview in step with it.
106*/
107
108#pragma once
109
110#include <stddef.h>
111#include <stdint.h>
112
113#include "serene/rt/trace.h"
114
115#ifndef __x86_64__
116# error "no fiber context switch for this architecture (need ctx_<arch>.c and switch_<arch>.S)"
117#endif
118
119// Sanitizer detection. AddressSanitizer and ThreadSanitizer each need a little
120// per-fiber bookkeeping (the guarded fields below and the switch path in
121// fiber.c), and a normal build carries none of it. Every translation unit in a
122// build compiles with the same `-fsanitize` flags, so the struct layout stays
123// consistent within that build.
124#if defined(__SANITIZE_ADDRESS__)
125# define SRN_ASAN 1
126#elif defined(__has_feature)
127# if __has_feature(address_sanitizer)
128# define SRN_ASAN 1
129# endif
130#endif
131#ifndef SRN_ASAN
132# define SRN_ASAN 0
133#endif
134
135#if defined(__SANITIZE_THREAD__)
136# define SRN_TSAN 1
137#elif defined(__has_feature)
138# if __has_feature(thread_sanitizer)
139# define SRN_TSAN 1
140# endif
141#endif
142#ifndef SRN_TSAN
143# define SRN_TSAN 0
144#endif
145
146#define FIBER_TRACEPOINT(...) SRN_TRACEPOINT_WITH_GROUP(fiber __VA_OPT__(, ) __VA_ARGS__)
147
148typedef struct srn_engine_t srn_engine_t;
149typedef struct srn_context_t srn_context_t;
151typedef struct srn_scheduler_t srn_scheduler_t;
152typedef struct srn_reactor_t srn_reactor_t;
153typedef size_t srn_worker_id_t;
154
155/**
156 * What a fiber's entry produces, type-erased. It is an alias for `void *`, so
157 * the fiber subsystem carries no dependency on the value model. A caller that
158 * runs fibers over Serene values casts to and from its own type at the
159 * boundary.
160 */
161typedef void *srn_fiber_result_t;
162
163#define srn_fiber_get_scheduler_m(fiber) fiber->ctx->engine->scheduler
164
165// TODO(lxsameer): We need to have a ring with a certain size, which
166// would be like a storage for ready to use stacks. When we're done with a fiber
167// put its stack in the ring and prepare it for another upcoming fiber.
168// it might be even a good idea to allocate a few stacks in advance on start
169// up. This ring should be per thread I think.
170
171/**
172 * Size of the embedded debug name buffer on every fiber, terminator
173 * included. Large enough for the autogenerated `fiber_<id>` spelling of any
174 * 64 bit id.
175 */
176#define SRN_FIBER_NAME_MAX 32U
177
178// -----------------------------------------------------------------------------
179// Saved machine context
180// -----------------------------------------------------------------------------
181/**
182 * The saved context of a suspended fiber is a single word, its stack pointer
183 * at the moment it was switched away from. The callee-saved registers live on
184 * the fiber's own stack, pushed by srn_fiber_swap and popped when it is
185 * resumed.
186 */
190
191// -----------------------------------------------------------------------------
192// Stack
193// -----------------------------------------------------------------------------
194/**
195 * One stack per fiber, mapped with a guard band at the low end so an overflow
196 * faults deterministically instead of corrupting a neighbour. With addresses
197 * increasing to the right, the layout is:
198 *
199 * |... guard band ...|.......... frames ..........|
200 * ^ guard ^ limit ^ start
201 *
202 * `guard` is the mmap base, the lowest page of the region. The band spans
203 * `fiber.guard_pages` pages from the configuration. The stack grows downward,
204 * so frames start at the high end (`start`) and grow toward `limit`; one step
205 * past `limit` lands on the protected guard band and faults.
206 */
207typedef struct srn_fiber_stack_t {
208 /// High end, stack pointer initialises to this address
209 void *start;
210 /// Low end of usable region
211 void *limit;
212 /// Low base of the protected guard band that detects stack overflows
213 void *guard;
215
216// -----------------------------------------------------------------------------
217// Lifecycle
218// -----------------------------------------------------------------------------
219typedef enum srn_fiber_state_e : uint8_t {
220 /// Created, stack mapped, never resumed
222 /// On the run queue, eligible to run
224 /// Currently executing
226 /// Parked off the run queue, awaits srn_fiber_ready
228 /// Entry returned. The result is final
231
232/**
233 * The function a fiber runs. Allocations made through `ctx` land in the
234 * context's block.
235 * TODO(lxsameer): The shape is generic until code generation produces
236 * fibers, at which point it adopts the Serene ABI.
237 * TODO(lxsameer): Should we support two types of entry functions? one
238 * for C calling convention and one for FastC?
239 */
241
242/**
243 * Suspend commit callback. The worker routine runs it on its own side once the
244 * fiber has switched out. It re-checks the awaited condition and only then
245 * registers `self` with whatever will wake it, returning true to stay
246 * suspended, or false to resume `self` at once because the condition already
247 * holds. The order matters, registering before the re-check can leave a live
248 * registration behind a false return, and that stale waker would wake the
249 * fiber's NEXT suspend. It must not switch fibers.
250 */
251typedef bool (*srn_fiber_park_fn)(srn_fiber_t *self, void *arg);
252
254 /// Saved stack pointer (see srn_fiber_ctx_t)
257 /// The lifecycle state. Atomic because a waker may run on a different os
258 /// thread than the worker running the fiber. `srn_fiber_ready` flips
259 /// `SUSPENDED` to `READY` with a CAS while the worker running the fiber reads
260 /// and writes the same field. Making it atomic keeps every access whole, so a
261 /// read can never see an inconsistent value.
265 void *arg;
266 /// Set when state reaches SRN_FIBER_DONE. Type-erased (see
267 /// srn_fiber_result_t), the caller interprets it.
269
270 /// While this fiber is suspending, the commit the worker routine runs once
271 /// the fiber is off the stack, to publish it to its waker (see
272 /// srn_fiber_suspend). Valid only across that one suspend hand-off.
274 void *park_arg;
275
276#if SRN_ASAN
277 /// AddressSanitizer bookkeeping across switches.
278 void *fake_stack;
279#endif
280
281#if SRN_TSAN
282 /// ThreadSanitizer fiber handle. TSan tracks each fiber as its own execution
283 /// context, so the switch path tells it which one is current. Created with
284 /// the fiber and destroyed when it is reaped.
285 void *tsan_fiber;
286#endif
287
288 /// Intrusive link threading this fiber onto one of the scheduler's
289 /// singly-linked lists (the ready run queue, or a wait queue) without a
290 /// separate node allocation, the next pointer lives in the fiber itself. A
291 /// fiber belongs to at most one such list at a time, so one link suffices.
293
294 /// Head of the list of fibers blocked in srn_fiber_wait_for on this fiber.
295 /// They are threaded through their own `link` (a waiter is suspended, so off
296 /// the ready queue, and its `link` is free). When this fiber reaches DONE the
297 /// worker routine wakes them all.
299
300 /// Registry links.
301 ///
302 /// The scheduler keeps every live fiber on one doubly linked list through
303 /// these pointers. It is different from `link`. `link` says where a fiber
304 /// sits in the run order, the registry says the fiber exists at all. A fiber
305 /// joins the list when it is created and leaves when it is reaped, in
306 /// whatever state it is in between -- including SUSPENDED, which sits on no
307 /// queue.
308 ///
309 /// This is how the scheduler accounts for, cleans up, and (later) cancels its
310 /// fibers. Doubly linked so reaping can unlink from the middle in O(1).
313
314 /// Debug name, the caller's choice copied at creation, or autogenerated
315 /// when the caller passed none (see srn_fiber_autoname).
316 /// srn_fiber_init_thread names loop fibers "thread". Embedded so the name
317 /// shares the fiber's lifetime with no separate allocation.
319};
320
321// -----------------------------------------------------------------------------
322// Scheduler
323// -----------------------------------------------------------------------------
324[[nodiscard]] [[gnu::nonnull(1)]]
326
327/**
328 * The one stop tear down of the fiber subsystem, should be called once
329 * `srn_sched_run` has returned. It joins the os threads, then releases the
330 * stack of every fiber still registered (any left unreaped, such as one
331 * suspended with no party to wake it, or one left queued when the run stopped
332 * early), and frees the scheduler's own resources. The fiber structs
333 * themselves live in context blocks and are reclaimed with those blocks, not
334 * here.
335 *
336 * The scheduler is NOT usable after this. A later `srn_sched_run` panics, and
337 * a repeated `srn_sched_shutdown` is a no-op (so the engine can call it
338 * unconditionally even after a caller already did).
339 *
340 * Must be called from outside the pool, never from an os thread that is
341 * running a worker nor from a fiber, and only after `srn_sched_run` has
342 * returned. Calling it on a live pool panics -- stop the pool with
343 * `srn_sched_stop`, let `srn_sched_run` return, then shut down.
344 */
345[[gnu::nonnull(1)]]
347
348/**
349 * Run the scheduler with `nworkers` os threads draining it, returning once the
350 * pool goes quiescent (every os thread parked on an empty queue) or a stop is
351 * requested with `srn_sched_stop`. The calling os thread becomes worker 0, so
352 * it does not return until then. `nworkers` is clamped to at least 1; with 1
353 * it is the calling os thread alone, which keeps execution single-threaded and
354 * cooperatively ordered. The spawned os threads are not joined here --
355 * `srn_sched_shutdown` joins them as part of tearing the subsystem down.
356 */
357[[gnu::nonnull(1)]]
358void srn_sched_run(srn_scheduler_t *sched, size_t nworkers);
359
360/**
361 * Ask a running scheduler to stop. Each worker routine checks the request at
362 * the top of each turn and stops once the fiber it is running yields or
363 * finishes, so a slice in flight is never cut mid-execution. `srn_sched_run`
364 * then returns. Fibers left queued stay unrun and are reclaimed by
365 * `srn_sched_shutdown`. Does not wait. Safe to call from any os thread or a
366 * fiber, but NOT from a signal handler. It takes the scheduler mutex and
367 * signals the condition variable, neither of which is async signal safe.
368 */
369[[gnu::nonnull(1)]]
371
372/**
373 * Ask a running scheduler to wind down gracefully. Unlike `srn_sched_stop`,
374 * the workers keep running every runnable fiber and let in-flight IO complete;
375 * only new IO submissions are fenced, so a fiber's next IO call returns
376 * `-ECANCELED` and the fiber unwinds rather than parking on a fresh op. The
377 * pool then reaches the same quiescence as a natural finish and
378 * `srn_sched_run` returns. An in-flight op that never completes (an idle recv,
379 * a long sleep) stalls the wind-down until it finishes; bounding that needs op
380 * cancellation and is not provided yet. Does not wait. Safe to call from any
381 * os thread or a fiber. A no-op if the scheduler is not running.
382 */
383[[gnu::nonnull(1)]]
385
386/**
387 * Record a fiber in the scheduler's registry of live fibers, where it stays
388 * until it is reaped.
389 */
390[[gnu::nonnull(1, 2)]]
392
393/**
394 * Place a fiber on a scheduler's ready queue, making it eligible to run.
395 */
396[[gnu::nonnull(1, 2)]]
398
399// -----------------------------------------------------------------------------
400// Fibers
401// -----------------------------------------------------------------------------
402/**
403 * Create a fiber that will run entry(ctx, arg), registered with `sched` but
404 * NOT scheduled. The fiber stays NEW until srn_fiber_schedule starts it, so
405 * a caller can build several fibers and wire them up before any of them
406 * runs. Registration at creation means the scheduler reaps every fiber at
407 * shutdown, scheduled or not.
408 *
409 * `name` is a debug label copied into the fiber, truncated to
410 * SRN_FIBER_NAME_MAX - 1 bytes, or nullptr for an autogenerated unique one.
411 * A `stack_size` of 0 selects the configured per fiber default.
412 */
413[[nodiscard]] [[gnu::nonnull(1, 2, 4)]]
415 srn_context_t *ctx, srn_scheduler_t *sched, const char *name, srn_fiber_entry_t entry, void *arg,
416 size_t stack_size
417);
418
419/**
420 * Write the autogenerated debug name for a new fiber into `dst`. Created on
421 * a worker the tag is `f#<worker>:<n>`, the creating worker's id and its
422 * spawn count. Created off the pool it is `f#m:<id>` from the engine wide
423 * object id counter. The worker part is provenance, not placement, a stolen
424 * fiber runs anywhere. srn_fiber_make uses this when given no name.
425 */
426[[gnu::nonnull(1, 2)]]
427void srn_fiber_autoname(srn_engine_t *engine, char *dst, size_t size);
428
429/**
430 * Schedule a NEW fiber, making it eligible to run. A fiber is scheduled
431 * exactly once, scheduling one that is not NEW panics, and waking a
432 * suspended fiber is srn_fiber_ready's job instead.
433 */
434[[gnu::nonnull(1)]]
436
437/**
438 * Make and schedule a fiber with every default, the engine's scheduler, the
439 * configured stack size, and an autogenerated name. The caller keeps
440 * ownership of `arg` and must keep it alive until the fiber is done with
441 * it, srn_fiber_spawn_copy lifts that burden.
442 */
443[[gnu::nonnull(1, 2)]]
445
446/**
447 * srn_fiber_spawn with `size` bytes of `*arg` copied into `ctx` first, so
448 * the fiber owns its argument and the caller's stack frame can die freely.
449 * The copy lives until the context is released, like the fiber itself.
450 */
451[[gnu::nonnull(1, 2, 3)]]
453srn_fiber_spawn_copy(srn_context_t *ctx, srn_fiber_entry_t entry, const void *arg, size_t size);
454
455/**
456 * srn_fiber_spawn_copy for an lvalue, the address and size are taken for
457 * the caller.
458 */
459#define SRN_FIBER_SPAWN_COPY(ctx, entry, value) \
460 srn_fiber_spawn_copy(ctx, entry, &(value), sizeof(value))
461
462/**
463 * Yield cooperatively, re-enqueue the running fiber and run the next ready
464 * one. Acts on the fiber currently running on this thread.
465 */
466void srn_fiber_yield(void);
467
468/**
469 * Park the running fiber until a party calls srn_fiber_ready. The fiber
470 * switches out first. Only then does the scheduler run `commit` (see
471 * srn_fiber_park_fn) to register it and confirm it should stay parked.
472 * Registering only after the park completes is what makes it race free -- a
473 * waker can never observe a half parked fiber. Acts on the fiber currently
474 * running on this thread.
475 */
476[[gnu::nonnull(1)]]
477void srn_fiber_suspend(srn_fiber_park_fn commit, void *arg);
478
479/**
480 * Mark a suspended fiber runnable again, waking it when the event it awaited
481 * occurs. Callable from a fiber, a worker, or the reactor. The reactor is
482 * the only legitimate waker outside the worker pool.
483 */
484// NOTE: Quiescence accounts for runnable fibers and the reactor's inflight ops, nothing else, so a
485// wake from an unrelated os thread can arrive after the run has already ended and be lost, or race
486// the scheduler teardown. A subsystem that wakes fibers from its own threads must be accounted for
487// in quiescence the way the reactor is.
488[[gnu::nonnull(1)]]
489void srn_fiber_ready(srn_fiber_t *fiber);
490
491/**
492 * Block the calling fiber until `target` finishes, then return its result. If the `target` is
493 * already done, returns immediately.
494 */
495[[gnu::nonnull(1)]]
497
498/**
499 * The fiber currently running on this os thread, or null when the calling
500 * thread is not a worker or the worker is running its own loop rather than
501 * a fiber.
502 */
504
505/**
506 * The worker's loop of the worker running on the calling os thread. The fiber
507 * that resumes when the current fiber yields, suspends, or finishes, the
508 * resumer a fiber's launcher hands control back to. Each worker has its own,
509 * so this is valid only on an os thread that is currently running the worker
510 * routine.
511 */
512[[gnu::returns_nonnull]]
514
515// -----------------------------------------------------------------------------
516// Context switch
517// -----------------------------------------------------------------------------
518/**
519 * Save the current execution context into `from`, restore `to`, and resume on
520 * `to`'s stack. Returns, on `from`'s stack, when a fiber later switches back
521 * into `from`.
522 */
523[[gnu::nonnull(1, 2)]]
525
526/**
527 * Initialise a fresh fiber context so the first srn_fiber_swap into it begins
528 * executing fn(arg) on `stack`.
529 *
530 * `fn` is the fiber's entry, a `void (*)(void *)` that runs on the new stack
531 * and receives `arg`. It must NEVER return -- there is no frame beneath it, so
532 * a return traps in the trampoline. Instead it transfers control away with
533 * srn_fiber_switch (to yield) or srn_fiber_switch_final (when finished), and
534 * as its first action it calls srn_fiber_on_entry(). Typical shape:
535 *
536 * static void worker(void *arg) {
537 * srn_fiber_on_entry(&scheduler); // hand the switch to the
538 * sanitizer
539 * ... do the fiber's work using arg ...
540 * srn_fiber_switch_final(&scheduler); // finished, never returns
541 * }
542 */
543[[gnu::nonnull(1, 3)]]
545 srn_fiber_ctx_t *fiber_ctx, srn_fiber_stack_t stack, void (*fn)(void *), void *arg
546);
547
548/**
549 * Switch from `from` to `to`, both live fibers, telling AddressSanitizer about
550 * the stack change. `from` resumes where it left off when something later
551 * switches back into it.
552 */
553[[gnu::nonnull(1, 2)]]
555
556/**
557 * Like srn_fiber_switch, but for a fiber that has finished and must not be
558 * resumed, control transfers to `to` and never comes back. The fiber's entry
559 * function therefore ends at this call -- any code after it is unreachable,
560 * which is why this is [[noreturn]]. Nothing is freed here. The spent fiber's
561 * stack is left frozen and reclaimed later by its owner (the scheduler reaps a
562 * finished fiber and frees its stack via srn_fiber_stack_free).
563 */
564[[noreturn]] [[gnu::nonnull(1)]]
566
567/**
568 * Call as the first action inside a fresh fiber's entry. `from` is the fiber
569 * that started this one (the scheduler, or the bootstrap thread fiber). It
570 * completes the switch for AddressSanitizer and, since `from`'s stack bounds
571 * only become discoverable here, records them into `from` so a later switch
572 * back is tracked. `from` may be null. A no-op when the sanitizer is not in
573 * use.
574 */
576
577/**
578 * Call when a finished fiber is reaped, after it has switched away for the
579 * last time. Releases the fiber's ThreadSanitizer handle. A no-op when the
580 * sanitizer is not in use.
581 */
582[[gnu::nonnull(1)]]
583void srn_fiber_on_reap(srn_fiber_t *fiber);
584
585/**
586 * Represent the calling OS thread as the running fiber ("#0"), so the
587 * scheduler or a test can switch away from it and back. The thread's stack
588 * bounds are not queried here (there is no portable way). Under the sanitizer
589 * they are recorded by the first fiber's srn_fiber_on_entry. The saved context
590 * is written by the first switch away.
591 */
592[[gnu::nonnull(1)]]
594
595// -----------------------------------------------------------------------------
596// Stack provider (rt/fiber/stack_*.c)
597// -----------------------------------------------------------------------------
598/**
599 * Allocate a stack of at least `size` usable bytes, or
600 * SRN_CONFIG_DEFAULT_FIBER_STACK_SIZE when `size` is 0, plus a guard band of
601 * `guard_pages` OS pages below it. `guard_pages` must be at least 1. Platform
602 * specific.
603 */
604[[nodiscard]]
605srn_fiber_stack_t srn_fiber_stack_alloc(size_t size, size_t guard_pages);
606
608
609// -----------------------------------------------------------------------------
610// Helpers
611// -----------------------------------------------------------------------------
613 return (size_t)((char *)s.start - (char *)s.limit);
614}
615
616/**
617 * Return the id of the worker that the calling os thread is running, or
618 * SIZE_MAX when the calling thread is not a worker.
619 */
621
622/**
623 * Whether the scheduler still accepts new IO submissions. True only while
624 * RUNNING; false once `srn_sched_drain` or `srn_sched_stop` has begun winding
625 * the pool down. The IO bridge reads this to fence submissions during a
626 * wind-down, so a fiber's IO call comes back cancelled instead of parking on an
627 * op the wind-down would have to wait out.
628 */
629[[gnu::nonnull(1)]]
631
632/**
633 * Rouse parked workers so the owner of `channel` consumes its completions.
634 * The reactor's notify hook. The wake is currently a broadcast, `channel`
635 * names the worker that must look, but every parked worker is notified and
636 * the ones with nothing to do re-park.
637 */
638void srn_sched_wake_worker(srn_scheduler_t *sched, size_t channel);
void srn_sched_register(srn_scheduler_t *sched, srn_fiber_t *fiber)
Record a fiber in the scheduler's registry of live fibers, where it stays until it is reaped.
Definition scheduler.c:324
size_t srn_worker_id_t
Definition fiber.h:153
srn_fiber_t * srn_fiber_worker_loop(void)
The worker's loop of the worker running on the calling os thread.
Definition scheduler.c:1101
void srn_fiber_switch_final(srn_fiber_t *to)
Like srn_fiber_switch, but for a fiber that has finished and must not be resumed, control transfers t...
Definition fiber.c:87
void srn_fiber_swap(srn_fiber_ctx_t *from, srn_fiber_ctx_t *to)
Save the current execution context into from, restore to, and resume on to's stack.
void srn_sched_wake_worker(srn_scheduler_t *sched, size_t channel)
Rouse parked workers so the owner of channel consumes its completions.
Definition scheduler.c:1171
void srn_fiber_ready(srn_fiber_t *fiber)
Mark a suspended fiber runnable again, waking it when the event it awaited occurs.
Definition scheduler.c:1083
srn_fiber_result_t srn_fiber_wait_for(srn_fiber_t *target)
Block the calling fiber until target finishes, then return its result.
Definition scheduler.c:1130
srn_fiber_t * srn_fiber_spawn_copy(srn_context_t *ctx, srn_fiber_entry_t entry, const void *arg, size_t size)
srn_fiber_spawn with size bytes of *arg copied into ctx first, so the fiber owns its argument and the...
Definition fiber.c:258
srn_fiber_t * srn_fiber_spawn(srn_context_t *ctx, srn_fiber_entry_t entry, void *arg)
Make and schedule a fiber with every default, the engine's scheduler, the configured stack size,...
Definition fiber.c:250
srn_fiber_stack_t srn_fiber_stack_alloc(size_t size, size_t guard_pages)
Allocate a stack of at least size usable bytes, or SRN_CONFIG_DEFAULT_FIBER_STACK_SIZE when size is 0...
static size_t srn_fiber_stack_size(srn_fiber_stack_t s)
Definition fiber.h:612
void srn_fiber_ctx_make(srn_fiber_ctx_t *fiber_ctx, srn_fiber_stack_t stack, void(*fn)(void *), void *arg)
Initialise a fresh fiber context so the first srn_fiber_swap into it begins executing fn(arg) on stac...
void srn_sched_drain(srn_scheduler_t *sched)
Ask a running scheduler to wind down gracefully.
Definition scheduler.c:1002
#define SRN_FIBER_NAME_MAX
Size of the embedded debug name buffer on every fiber, terminator included.
Definition fiber.h:176
void srn_fiber_init_thread(srn_fiber_t *f)
Represent the calling OS thread as the running fiber ("#0"), so the scheduler or a test can switch aw...
Definition fiber.c:153
void srn_sched_shutdown(srn_scheduler_t *sched)
The one stop tear down of the fiber subsystem, should be called once srn_sched_run has returned.
Definition scheduler.c:333
bool srn_sched_accepting_submissions(srn_scheduler_t *sched)
Whether the scheduler still accepts new IO submissions.
Definition scheduler.c:1163
void srn_fiber_stack_free(srn_fiber_stack_t stack)
void srn_sched_stop(srn_scheduler_t *sched)
Ask a running scheduler to stop.
Definition scheduler.c:978
void srn_sched_enqueue(srn_scheduler_t *sched, srn_fiber_t *fiber)
Place a fiber on a scheduler's ready queue, making it eligible to run.
Definition scheduler.c:630
srn_fiber_state_e
Definition fiber.h:219
@ SRN_FIBER_NEW
Created, stack mapped, never resumed.
Definition fiber.h:221
@ SRN_FIBER_RUNNING
Currently executing.
Definition fiber.h:225
@ SRN_FIBER_READY
On the run queue, eligible to run.
Definition fiber.h:223
@ SRN_FIBER_DONE
Entry returned. The result is final.
Definition fiber.h:229
@ SRN_FIBER_SUSPENDED
Parked off the run queue, awaits srn_fiber_ready.
Definition fiber.h:227
srn_fiber_result_t(* srn_fiber_entry_t)(srn_context_t *ctx, void *arg)
The function a fiber runs.
Definition fiber.h:240
srn_scheduler_t * srn_sched_init(srn_engine_t *engine)
Definition scheduler.c:263
srn_fiber_t * srn_fiber_current(void)
The fiber currently running on this os thread, or null when the calling thread is not a worker or the...
Definition scheduler.c:1097
void srn_fiber_switch(srn_fiber_t *from, srn_fiber_t *to)
Switch from from to to, both live fibers, telling AddressSanitizer about the stack change.
Definition fiber.c:65
srn_fiber_t * srn_fiber_make(srn_context_t *ctx, srn_scheduler_t *sched, const char *name, srn_fiber_entry_t entry, void *arg, size_t stack_size)
Create a fiber that will run entry(ctx, arg), registered with sched but NOT scheduled.
Definition fiber.c:202
void srn_fiber_on_entry(srn_fiber_t *from)
Call as the first action inside a fresh fiber's entry.
Definition fiber.c:110
bool(* srn_fiber_park_fn)(srn_fiber_t *self, void *arg)
Suspend commit callback.
Definition fiber.h:251
void srn_fiber_schedule(srn_fiber_t *fiber)
Schedule a NEW fiber, making it eligible to run.
Definition scheduler.c:638
void * srn_fiber_result_t
What a fiber's entry produces, type-erased.
Definition fiber.h:161
enum srn_fiber_state_e srn_fiber_state_t
void srn_sched_run(srn_scheduler_t *sched, size_t nworkers)
Run the scheduler with nworkers os threads draining it, returning once the pool goes quiescent (every...
Definition scheduler.c:875
void srn_fiber_on_reap(srn_fiber_t *fiber)
Call when a finished fiber is reaped, after it has switched away for the last time.
Definition fiber.c:132
void srn_fiber_autoname(srn_engine_t *engine, char *dst, size_t size)
Write the autogenerated debug name for a new fiber into dst.
Definition scheduler.c:1141
void srn_fiber_suspend(srn_fiber_park_fn commit, void *arg)
Park the running fiber until a party calls srn_fiber_ready.
Definition scheduler.c:1063
void srn_fiber_yield(void)
Yield cooperatively, re-enqueue the running fiber and run the next ready one.
Definition scheduler.c:1039
srn_worker_id_t srn_sched_current_worker_id(void)
Return the id of the worker that the calling os thread is running, or SIZE_MAX when the calling threa...
Definition scheduler.c:1157
Engine is a structure to own the long living and main pieces of the compiler.
Definition engine.h:51
The saved context of a suspended fiber is a single word, its stack pointer at the moment it was switc...
Definition fiber.h:187
void * sp
Definition fiber.h:188
One stack per fiber, mapped with a guard band at the low end so an overflow faults deterministically ...
Definition fiber.h:207
void * guard
Low base of the protected guard band that detects stack overflows.
Definition fiber.h:213
void * limit
Low end of usable region.
Definition fiber.h:211
void * start
High end, stack pointer initialises to this address.
Definition fiber.h:209
char name[SRN_FIBER_NAME_MAX]
Debug name, the caller's choice copied at creation, or autogenerated when the caller passed none (see...
Definition fiber.h:318
srn_fiber_entry_t entry
Definition fiber.h:264
_Atomic srn_fiber_state_t state
The lifecycle state.
Definition fiber.h:262
srn_fiber_t * link
Intrusive link threading this fiber onto one of the scheduler's singly-linked lists (the ready run qu...
Definition fiber.h:292
srn_fiber_t * waiters
Head of the list of fibers blocked in srn_fiber_wait_for on this fiber.
Definition fiber.h:298
void * park_arg
Definition fiber.h:274
srn_fiber_result_t result
Set when state reaches SRN_FIBER_DONE.
Definition fiber.h:268
srn_fiber_park_fn park_commit
While this fiber is suspending, the commit the worker routine runs once the fiber is off the stack,...
Definition fiber.h:273
srn_context_t * ctx
Definition fiber.h:263
srn_fiber_stack_t stack
Definition fiber.h:256
srn_fiber_t * reg_prev
Registry links.
Definition fiber.h:311
void * arg
Definition fiber.h:265
srn_fiber_t * reg_next
Definition fiber.h:312
srn_fiber_ctx_t fiber_ctx
Saved stack pointer (see srn_fiber_ctx_t).
Definition fiber.h:255
Platform neutral static tracepoints.