# Concurim: Visualizing Python Concurrency
> Building a visual debugging and learning tool for Python threads, processes, and asynchronous programs.
I've found myself writing increasingly concurrent programs, whether for game development tools, game engines, or my experiments with AI. As those programs have grown more complex, I've noticed that the way I normally reason about code doesn't always hold up.
When debugging sequential code, I can usually follow it like a cooking recipe. One instruction follows another, and I can trace the state of the program as I read through it. Even when the code is complicated, there's generally a path I can follow.
Concurrency makes that considerably harder.
With multiple threads, processes, or asynchronous tasks, execution is no longer a single sequence of instructions. Workers can overlap, block, resume, and interact through shared resources. An operation that appears straightforward in isolation might behave differently depending on what another worker is doing at that moment.
I can read through a producer-consumer implementation and understand what each function does. But keeping track of which worker has which item, who's waiting on a lock, and when a queue becomes full requires building a mental picture of the entire system.
I'm a visual learner, and I wanted a better way to develop that intuition.
I've always enjoyed Grant Sanderson's [3Blue1Brown](https://www.3blue1brown.com/) videos, particularly how animation can make an otherwise abstract mathematical concept feel intuitive. His open-source animation engine, [Manim](https://github.com/3b1b/manim), also demonstrated that these sorts of explanations could be generated programmatically.
I started wondering whether I could do something similar with Python concurrency.
What if I could drop a small Python program into a tool, record what happened during execution, and watch its threads, processes, queues, and locks interact as an animation?
That became **Concurim**.
*The one that fell.*
Concurim is an open-source, local-first Python concurrency visualizer. It records an execution and lets you replay it as an interactive animation, inspect individual events, follow workers through source code, and export the results as a video.
The project is available on [GitHub](https://github.com/ej-sanmartin/concurim) and [PyPI](https://pypi.org/project/concurim/).
## Getting started
I wanted the initial experience to be as simple as possible, especially for students experimenting with small Python scripts.
With Python 3.13 or newer, install Concurim from PyPI:
```bash
pip install concurim
```
Then launch it:
```bash
concurim
```
This opens a local interface in your browser.
From there, you can drag and drop a Python file or project folder, review the execution plan, and click **Record execution**.
Once the program finishes, Concurim opens the recording in its viewer. You can play, pause, step forward or backward through events, inspect resources, and watch your program's workers interact.
[](https://github.com/ej-sanmartin/concurim)
There's also a command-line interface for users who prefer it:
```bash
concurim record --open --auto sync example.py
```
This enables automatic tracing of threads, queues, synchronization primitives, and supported pools, then opens the resulting recording.
The interface also includes example programs demonstrating common concurrency patterns, including producer-consumer queues, lock contention, semaphores, thread pools, and asyncio tasks.
My intention was to make the first experience about understanding the program, not spending an afternoon configuring a profiler.
## Recording what actually happened
One of the earliest design decisions was also one of the most consequential: **Concurim should visualize an actual execution rather than simulate what an execution might look like.**
Python concurrency is nondeterministic. Depending on scheduling, timing, and resource contention, two executions of the same program can produce different event orders.
An animation that invents a convenient execution order might look clearer, but it wouldn't necessarily represent the program that ran.
Instead, Concurim records events as the program executes and builds a replay from that event stream.
Conceptually, the system is divided into three stages:
1. **Record:** Instrument supported Python concurrency primitives and capture events during execution.
2. **Replay:** Reduce those events into a sequence of program states.
3. **Visualize:** Render those states as an interactive graph, timeline, or video.
The replay engine is particularly important. Rather than having the browser independently reconstruct the state of every queue or lock, a Python reducer processes the recorded events into frames.
This gives the viewer a consistent representation of the recording. Stepping backward doesn't require reversing arbitrary JavaScript state mutations; the viewer simply displays an earlier frame.
It also made exporting the same recording into different visual formats considerably easier.
There is, however, an important distinction between what the recorder *knows* and what it *infers*.
If Concurim records a thread blocking on a lock, that's an observed event. If the thread hasn't reported blocking, the viewer may treat it as running, but that doesn't mean Concurim has observed the thread actively executing on a CPU.
The interface distinguishes those cases.
That became one of the guiding principles of the project: don't manufacture certainty from incomplete observations.
## Instrumenting concurrency without becoming the problem
Recording concurrency introduces an interesting problem: the recorder itself executes alongside the program being observed.
Instrumentation takes time. Additional function calls, event collection, and synchronization can alter scheduling. In a sufficiently timing-sensitive program, observation can even change whether a race condition appears.
There's no pretending this overhead doesn't exist.
Concurim provides explicit tracing wrappers alongside opt-in automatic instrumentation of supported Python primitives. Automatic patching applies only during recording and is reversed afterward.
The recorder also has to avoid observing its own implementation.
For example, if recording a lock acquisition required acquiring another instrumented lock, which itself triggered another recording event, the recorder could recursively instrument its own synchronization.
To avoid that, internal synchronization uses an uninstrumented lock and a per-thread reentrancy guard. The event buffer is also bounded, and traces explicitly indicate when events have been dropped.
Some details matter more than they initially appear.
For queues, events are recorded from internal mutation operations while the queue's mutex is held. That lets the event stream reflect the order in which items actually entered or left the queue, rather than relying on whichever thread happened to append an event to the collector first.
There are tradeoffs here.
Automatic instrumentation can't see every object, particularly those created before recording begins or inside uninstrumented libraries. Some standard-library behaviors are deliberately excluded rather than approximated incorrectly.
Recording also introduces measurable overhead. In microbenchmarks dominated by queue operations, the slowdown can be substantial, although it's less significant when real work happens between those operations.
For a production observability tool, this would be a major concern. For an educational tool designed to examine small programs offline, I consider it an acceptable tradeoff, provided the limitations are clear.
## A deadlock that couldn't be recorded
One particularly interesting problem appeared while working on deadlock visualization.
Imagine two threads and two locks.
Thread A acquires lock 1, then waits for lock 2. Thread B acquires lock 2, then waits for lock 1.
Neither thread can proceed.
This is exactly the sort of behavior I wanted Concurim to explain visually.
Unfortunately, the initial recording implementation had an obvious problem in retrospect: it wrote the completed trace when the recording context exited.
A deadlocked program never reached that point.
The recorder had already captured the interesting events in memory. It knew which thread had acquired which lock and which acquisitions had started blocking. But because the program never finished, those events weren't written to disk.
The most useful failure to visualize produced no visualization.
Rather than building an elaborate deadlock detector to decide when to save the trace, I changed the persistence behavior.
Concurim now periodically saves a snapshot of the event buffer, approximately once per second, and attempts another save when interrupted by supported termination signals.
If a program hangs, the last saved recording can still be opened and inspected. The viewer marks it as interrupted rather than pretending the execution completed.
From there, the replay engine can identify wait cycles involving observed lock ownership and blocking relationships.
This distinction matters: discovering that two workers are blocked on each other in one recorded execution is not the same thing as proving that every possible execution will deadlock.
Concurim doesn't explore all possible thread interleavings. It reports what happened in the captured run, alongside clearly labeled warnings about patterns that *could* cause problems.
Sometimes a relatively simple change in architecture solves a problem more effectively than introducing more sophisticated detection logic.
## Making the animation understandable
My original inspiration was Manim, but I eventually decided against using it as Concurim's renderer.
The first visualization was a relatively simple producer-queue-consumer diagram built with React and SVG. As the project expanded to support locks, processes, pools, and more complex worker relationships, I adopted React Flow and Motion for the interactive graph.
The browser already had to render nodes, connections, queue items, labels, and animations.
Using Manim for video export would have meant implementing a second renderer capable of reproducing those same visual semantics.
That seemed like unnecessary complexity.
Instead, video export drives a browser through the Chrome DevTools Protocol, advances the visualization frame by frame, and captures the rendered output.
This also let me separate the speed of the recording from the speed of the animation.
A queue item might be transferred in a fraction of a millisecond during execution, but its movement can be animated over several hundred milliseconds to make the handoff understandable.
The animation is intentionally slower and more readable than the program itself.
The recorded event order remains unchanged.
I made similar compromises for the source-code viewer. Exact source locations are available for some instrumented operations, while other execution positions are approximated using periodic sampling.
Sampling isn't perfect. Short-lived instructions can be missed, and blocking operations can be disproportionately represented. Concurim labels approximate positions rather than displaying them as exact observations.
The goal is to help someone understand what their workers are doing, not to imply that a sampled source line is a precise CPU instruction trace.
## Testing nondeterministic systems
Of all the work involved in building Concurim, testing was probably the most frustrating.
There were effectively two different kinds of nondeterminism to deal with.
The first came from concurrent execution itself.
An unreliable concurrency test might start two threads, sleep for a short period, and assert that one specific sequence of events occurred.
That test could pass a hundred times and fail the next time the scheduler behaved differently.
I tried to avoid tests that depended on incidental scheduling order. Instead, tests use synchronization where appropriate and assert properties that should remain true regardless of which worker executes first.
For instance, a bounded queue should respect its capacity, and recorded queue mutations should correspond to the operations that actually occurred. Neither requirement depends on a particular thread always running first.
The second kind of nondeterminism came from rendering.
Initially, some export tests asserted that two generated PNG files were byte-identical.
That sounded reasonable: if the same recording produces the same frame, shouldn't the image be identical?
Not necessarily.
I encountered differences across macOS and Windows environments, as well as rendering variations between browser launches. Font rasterization, antialiasing, and image compression could all affect the generated binary.
A single changed edge pixel could alter the compressed PNG data and cause a test to fail, even when the resulting images were visually indistinguishable.
Waiting for fonts and rendering to settle fixed some failures, but not all of them.
Eventually, I moved toward pixel-based image comparisons with platform-specific baselines and tolerances for small rendering differences.
This preserved the useful property I wanted from the tests: they should catch meaningful regressions, such as a missing node, incorrect event, or broken animation, without failing over an insignificant antialiased edge.
The Windows tests also uncovered platform-specific issues unrelated to concurrency scheduling. One particularly unpleasant failure came from using `os.path.relpath` across different drives. An exception in instrumentation prevented an executor from shutting down normally, leaving a background thread alive and hanging the test suite.
That reinforced another important invariant: **a recorder failure must not prevent the underlying program from performing its normal cleanup.**
These problems were tedious, but they were valuable reminders that determinism is rarely a binary property. A replay can be logically deterministic without every rendered byte being identical across operating systems.
## Building with AI tools
I developed Concurim iteratively over several weeks, using AI coding tools throughout the process.
I primarily used Claude Sonnet when working through more complicated concurrency behavior, instrumentation, and architectural questions. For smaller UI iterations, tests, and general debugging, I frequently switched between Grok and Kimi.
The process was conversational and iterative. I'd work through an approach, implement it, test the behavior, identify failures or limitations, and refine the design.
Having models available made it easier to explore Python internals and different implementation strategies, but it didn't eliminate the need to establish precise invariants or reproduce failures.
In fact, the most difficult problems tended to be the ones where the code looked reasonable but encoded a false assumption: that an event order would always repeat, that a browser would render the same pixels everywhere, or that instrumentation could never interfere with program shutdown.
Those problems required examining what the system was actually doing rather than trusting that a plausible implementation was correct.
I also had to consciously manage scope. It would have been easy to expand Concurim into a general-purpose profiler, trace arbitrary native code, or attempt to model CPU scheduling.
Instead, I kept the project centered on Python's concurrency abstractions and the educational experience I originally wanted.
## What Concurim doesn't do
I think it's important to be explicit about what Concurim isn't.
It isn't intended to replace production profilers, tracing infrastructure, or CI/CD observability systems.
It doesn't reconstruct every possible scheduling interleaving, and it doesn't claim to detect every race condition or deadlock. An execution without a reported hazard doesn't mean the program is free of concurrency bugs.
It also doesn't provide an exact picture of CPU scheduling or which thread owns Python's Global Interpreter Lock at every moment. Those details aren't reliably available through the Python-level instrumentation Concurim uses.
Cross-process event ordering can also be approximate because events from child processes are collected and timestamped by the parent when reports arrive.
Finally, the local recording workflow executes the supplied Python program. It isn't a security sandbox, and programs should be reviewed before running them.
These aren't simply missing features I intend to add someday. Some are deliberate boundaries that help preserve the accuracy and usefulness of the tool.
I'd rather Concurim tell me that it doesn't know something than display a convincing but misleading picture.
## What's next
There are still several areas I'd like to improve.
One is more granular visualization of synchronization behavior, especially lock acquisition and release, contention, and the relationships between waiting workers.
I'm also interested in making concurrency bugs easier to understand visually. Seeing how a particular interleaving produces an unexpected result could be especially helpful for students encountering races for the first time.
The current architecture already records useful events and identifies certain failure patterns, but there's room to make those relationships much more intuitive.
I'd also like to expand the documentation with more practical examples and walkthroughs for integrating Concurim into small projects.
Ultimately, my objective remains fairly modest.
I want to be able to write a small concurrent Python program, drop it into a viewer, and see what each worker actually did: when it started, when it stopped, what it waited for, and how data moved through the system.
When I'm learning a difficult concept, being able to see it can make all the difference.
Concurim is my attempt to make that possible for Python concurrency.
The project is open source under the MIT license. You can find the implementation, examples, documentation, and installation instructions on [GitHub](https://github.com/ej-sanmartin/concurim), or install it directly from [PyPI](https://pypi.org/project/concurim/).