ClientApp
Owns contexts, adapts queue operations, loads the queue DLL, supplies a monitor, and runs demonstration/regression checks.
C++20 / Win32 / Systems programming
Work,
ordered.
Reusable workers. An intrusive priority queue. Explicit ownership from submission to shutdown.
A small native Windows thread pool, explained through its scheduling decisions, synchronization costs, and failure boundaries—not just its happy path.
01 / The problem
Do not create a thread for every small job.
Thread creation involves OS resources, scheduling, stack reservation, and eventual cleanup. A pool amortizes that lifecycle across jobs. Producers describe work; a bounded set of workers executes it. This is a learning implementation, not a replacement for the Windows thread-pool API.
Pooling does not make every workload faster. Tiny jobs can spend more time on allocation, queue locks, semaphore operations, and diagnostic bookkeeping than on useful work. More workers can increase contention. The measurements below expose that tradeoff instead of promising linear scaling.
02 / Architecture
Five runtime projects. Separate responsibilities.
ThreadPool
Owns work wrappers and worker handles. Accepts opaque enqueue/dequeue callbacks; it does not know the priority policy.
PriorityQueue
Three FIFO lists with strict HIGH → MEDIUM → LOW selection, queue-wide locking, and count publication.
Intrusive_LinkedList
Embedded nodes and head/tail operations. Lists retain individual critical sections for standalone use.
Debugger
Source-aware logs, assertions, and timing helpers. Each linking DLL has its own logger state and SRW lock.
A separate C17 console executable, PerformanceTests1, measures the real DLL APIs without changing these five identities. Its sole measurement source is ThreadPoolNativeBenchmarks.c, compiled as C with no C++ test framework. The production DLL implementation remains C++. The browser cannot load the DLLs or submit native work.
03 / Data structure & scheduling
Intrusive links, not a binary heap.
Constant-time insertion
A NODE contains previous/next pointers embedded in the pool's WORK_ITEM. Enqueue appends to one of three tails: O(1), with no separate container-node allocation. The pool still allocates a work wrapper per submission.
Dequeue scans P priority levels: O(P), with P fixed at three. Pointer-based list search is O(n). Pointer chasing, nested locks, extra metadata, and lifetime bookkeeping are real costs.
Precisely what priority guarantees
- 0 = HIGH, 1 = MEDIUM, 2 = LOW.
- Choose the highest nonempty level at dequeue time.
- FIFO within a level means enqueue-commit order, not the order concurrent producers began calling.
- No preemption, aging, or starvation bound.
- Parallel callback start/finish order can differ from dequeue order.
One node, one list membership. Never move or free its enclosing object while linked. Queue cleanup detaches nodes; it does not own the payload.
04 / Synchronization
Critical sections + a semaphore.
Not condition variables.
Submission & selection
- Allocate a work wrapper.
- Acquire the pool lock; reject if stopping.
- Enqueue under queue → list locks; publish pending count.
- Release one semaphore permit; consider worker growth.
- A worker wakes, locks the pool, checks stop, dequeues, and claims a diagnostic slot.
Normal nested order: pool → queue → list. The queue lock makes priority selection and count updates one serialized operation.
Keep user work outside the lock
The worker releases the pool lock before invoking the callback. Completion callbacks also run outside it. A long callback therefore does not hold the queue lock, but all submissions and selections still share the pool lock.
The semaphore counts wake permits; it is not a condition variable or a promise that a queued item exists after a shutdown wake. Worker idle tracking, slot tracking, and completion arbitration acquire the pool lock too.
Core workers remain alive. Extra workers retire after five seconds idle. The hard live-worker cap is 16, including overdue workers. Benchmarks set min = max to isolate 1, 4, or 8 fixed workers.
05 / Ownership & shutdown
Cancel the queue. Join the running work.
- Stop external usersJoin producers and the monitor before shutdown.
- Reject & cancelSet stop; report CANCELLED for queued, unexecuted work.
- Join worker handlesRunning callbacks return naturally; shutdown waits.
- Release ownershipOnly now free contexts, destroy the queue, or unload DLLs.
The pool owns and frees its internal wrapper. The caller owns the context and callback code. A FAILED_STUCK notification reports an overdue callback, not a dead thread: its context must remain alive. Recovery may add a replacement worker within the hard cap.
No TerminateThread, early return, or abandoned synchronization objects. A callback that never returns can block shutdown indefinitely. Do not call shutdown from a callback, let exceptions escape the callback boundary, or destroy the pool concurrently.
06 / Interactive educational simulation
Watch selection - not the Windows scheduler.
Browser model, not a native trace. Durations are logical ticks, not milliseconds. Workers advance deterministically in ID order. The model omits locks, OS scheduling, dynamic growth, and stuck recovery. Its counters are not benchmark results.
Pending queue head → tail
Workers non-preemptive
Recent model events
Try one worker: start a long LOW task, then add HIGH tasks. HIGH waits for the running callback, then jumps ahead of queued LOW work. “Cancel queued + join” never cancels a running callback. Reset starts a new illustrative model, not a native shutdown.
07 / Real native measurements — separate from the model
Measured in C. Displayed here.
No capture loaded. Do not infer timings from the simulation. Download the capture below or reproduce it locally.
Throughput and many-producer stress
Two discarded warmups and seven measured repetitions per scenario. QPC surrounds producer release through producer joins and the last completion notification. Setup, thread creation, pool shutdown, and checksum recomputation are outside this throughput boundary. Every accepted task is still checked individually after all workers are joined.
| Kind | Workers / producers | H / M / L | Tasks / run | Median tasks/s | Time ms min / median / max |
|---|
Thread lifecycle baselines
Serial create-work-join-close: 256 sequential CreateThread calls, each followed by the instrumented callback, exit, wait and handle close. Includes dispatch and join; not bare thread-creation API latency. Workload sizes differ from pool batches, so no direct speedup ratio is claimed.
Pool initialize-shutdown: fixed worker startup through safe shutdown return, including synchronization objects and joins; queue construction/cleanup excluded. Initialization return does not prove all workers are ready. This is not the steady-state throughput interval.
| Boundary | Workers | Iterations / callback | Cycles | Median ms / cycle | Batch ms min / median / max |
|---|
Instrumentation is part of the contract
This capture uses a reproducible NativeBenchmark=true Release build. Trace/info macros and their argument evaluation are removed at compile time in every module, including separately linked logger instances. Ordinary diagnostic builds are unchanged. Stdout redirection alone would not remove formatting, logger locks or debugger output.
Warnings, errors, assertions, allocation, all synchronization, counters, slot tracking, timestamps and harness validation atomics remain. Startup/shutdown still contain blank stdout writes. This is not a claim about diagnostic-build throughput or a measured logging speedup.
What the stress actually validates
See the downloadable capture for per-task validation and measured task totals.
The many-producer case starts 16 producers together, using 8 fixed workers and 48,000 tasks per repetition across all three mixes. Each task ID must be accepted, execute, complete successfully, and match its checksum exactly once. A separate 90-node preloaded check verifies strict priority and FIFO pointer order.
Passing these finite runs is evidence of no lost/duplicated tasks in the exercised runs—not proof for every interleaving.
Download full JSON + environment · Download raw QPC samples (CSV) · Methodology & limitations
08 / Failure cases & boundaries
Make the uncomfortable cases explicit.
| Case | Behavior / caller responsibility |
|---|---|
| Invalid priority, null callback, allocation/enqueue failure | Submission fails. Caller retains context ownership; do not expect a completion notification for rejected work. |
| Continuous HIGH traffic | LOW may starve. No aging, fairness bound, deadline or priority inheritance is implemented. |
| Overdue callback | Monitor can report FAILED_STUCK once and add bounded replacement capacity. Work may still be running; retries need idempotence. |
| Shutdown with queued work | Queued work receives CANCELLED; running callbacks return naturally and are joined. This is not a drain-all policy. |
| Permanent hang / escaping exception | No forcible termination or callback exception isolation. Shutdown can block indefinitely; callbacks must respect the API contract. |
| Unbounded submissions | Workers are capped, but queue capacity is not. Memory use can grow under overload; no backpressure API. |
| OS resource failures | Some initialization/creation failures are checked, but the benchmark does not inject allocator, semaphore or wait failures. Do not claim exhaustive fault tolerance. |
09 / Debugging & evidence
A count of zero is not a thread join.
The lifetime trap
A worker can decrement a live counter and still be logging or returning through pool code. Freeing the pool on that counter alone risks use-after-free. The current shutdown joins retained thread handles before destroying synchronization objects.
RegressionTests.cpp exercises shutdown with gated running work and queued cancellation, both with and without stuck reporting, ten times each. The normal four-build matrix is separate from these performance measurements.
Reproduce the investigation
- Run
Build/Test.ps1 -Stress; keep the build and stdout/stderr logs private. - Break in
ShutdownThreadPooland inspect worker handles and diagnostic slots. - Use Parallel Stacks / Threads to distinguish waiting workers from callbacks still alive.
- Check notification counts and context lifetime, not only the pending queue count.
No profiler trace or debugger screenshot is invented here. QPC results measure elapsed boundaries; they do not attribute CPU time to a particular lock.
The SDK is pinned to 10.0.26100.0: the earlier floating SDK selection produced a manifest-tool startup failure. The build fix retained manifest generation rather than hiding it. See the validation notes and regression source.
10 / Reproduce & inspect
No packages. No benchmark theater.
Native capture
Windows, Visual Studio 2026 Desktop C++ workload (including the C17 compiler), v145 and SDK 10.0.26100.0. No C++ test framework is required. From the repository root:
powershell -File Build/Benchmark.ps1 -Platform x64 -PublishThe script rebuilds all benchmark dependencies into isolated directories, runs the C17 executable, validates its successful exit, C-language metadata and all 182 measured rows, and exports sanitized CSV/JSON. Raw build/test logs stay under ignored artifacts/.
Win32 is also supported by the script; publishing it replaces the displayed capture. Environment metadata on this page always identifies the displayed build.
Static presentation
Open index.html directly, or host this repository as a static site. Local JavaScript data avoids a file:// fetch requirement. No installation, CDN, analytics, or network service is needed.
node --test Build/simulation.test.jsNode is optional and only used to test the browser model. Native binaries and source paths never run in the browser. See presentation and accessibility notes.