PEP 848 – Generational Incremental Garbage Collection | peps.python.org

The Evolution of Python Memory Management

For decades, CPython has relied on a dual-strategy approach for memory management: reference counting for immediate reclamation and a generational cyclic garbage collector (GC) to handle self-referencing object clusters. While reference counting handles the vast majority of object deallocations, cyclic garbage—where objects refer to each other in a loop—requires a periodic scan of the heap.

As modern Python applications grow in complexity, particularly in fields like data science, artificial intelligence, and large-scale web services, heap sizes have expanded into the gigabyte range. In the current CPython implementation, the cyclic GC must occasionally perform a full scan of the entire heap. For large applications, these scans can lead to noticeable latency spikes, sometimes lasting hundreds of milliseconds, which is unacceptable for real-time systems or highly responsive user interfaces.

The newly proposed collector for Python 3.16 represents a culmination of efforts within the "Faster CPython" initiative. It builds upon lessons learned from a previous attempt at an incremental collector that was briefly introduced and then reverted in Python 3.14 due to concerns regarding peak memory usage. By combining incremental scanning with a refined generational structure, Shannon’s proposal seeks to provide the latency benefits of incrementalism without the memory bloat that plagued previous iterations.

Technical Architecture: A New Generational Paradigm

The core of the proposal is a fundamental redesign of how CPython categorizes and scans objects. The existing collector utilizes three generations (0, 1, and 2), but the new system introduces a more granular "space-based" approach within those generations.

Under the new specification, the heap remains divided into three primary generations: the Nursery, the Aging generation, and the Old generation. However, the internal mechanics differ significantly:

1. The Nursery: This is a single space where new objects are initially created.
2. The Aging Generation: Unlike the current single-space aging generation, the new incremental collector splits this into a configurable number of "half-spaces" (defaulting to ten). This allows for a more gradual promotion of objects, ensuring that only those that truly persist are moved to the most expensive generation to scan.
3. The Old Generation: This generation is divided into "pending" and "visited" spaces. This division is critical for the incremental nature of the collector, allowing it to track which parts of the old heap have been scanned recently and which are awaiting processing.

The algorithm functions by alternating between "young" collections and "incremental old" collections. When the nursery reaches a certain threshold, the collector processes the oldest space in the aging generation. When that aging space is cleared, the nursery’s survivors move into the first aging space. This "conveyor belt" approach ensures that objects are scanned multiple times before reaching the old generation, increasing the likelihood that short-lived cycles are caught early.

Solving the Latency Problem through Incrementalism

The most significant advancement in this PEP is the move toward incremental collection for the old generation. In the current "legacy" collector, a Generation 2 collection requires the interpreter to pause and scan every object in that generation. On a heap with millions of objects, this is a linear-time operation that stops all program execution.

The new incremental collector breaks this massive task into small, manageable chunks. Instead of scanning the whole old generation at once, it identifies a "transitive closure"—a small cluster of reachable objects—and scans only that increment. The collector maintains a "work-to-do" counter to ensure that the rate of scanning keeps pace with the rate of new object allocations.

According to the proposal’s performance analysis, this approach can reduce maximum pause times by as much as 100-fold in certain workloads. Furthermore, the overall time the CPU spends managing the GC is expected to drop by nearly 50%. This is achieved because the collector is more "effective"—it targets areas of the heap where garbage is most likely to have accumulated, rather than wasting cycles scanning live objects that are unlikely to have become unreachable.

Impact on Memory Consumption and Performance Data

A common trade-off in garbage collection design is the balance between latency and memory footprint. Incremental collectors often use more memory because they allow unreachable objects to linger slightly longer before they are identified and reclaimed.

Mark Shannon’s proposal addresses this by utilizing the generational structure to mitigate "floating garbage." By aggressively collecting the nursery and aging spaces, the system ensures that the vast majority of objects are reclaimed quickly. The proposal notes that for small or short-lived applications, memory usage might increase by a bounded amount (approximately 24MB by default) due to the larger young generation spaces.

However, for large-scale, long-running applications, the incremental collector may actually reduce peak memory use. In the legacy system, the old generation is only collected when it grows by 25% of its previous size. On a 10GB heap, this means the system might wait until 2.5GB of new data is added before triggering a collection. The incremental collector, by contrast, performs continuous small sweeps, preventing the old generation from bloating unnecessarily.

Changes to Developer APIs and Backward Compatibility

While the internal mechanics of the GC are changing, the proposal emphasizes maintaining backward compatibility for most Python developers. The standard gc module will remain, and the gc.set_threshold() function will still be used to tune the collector, though the internal meaning of the thresholds will be updated to reflect the new space-based architecture.

One notable change is the deprecation of calling gc.collect(generation) with a specific generation argument. In an incremental, interleaved system, forcing a collection of a specific generation can disrupt the heuristic balance of the collector. Developers are encouraged to use gc.collect() without arguments, which will now trigger a full heap collection to ensure all unreachable cycles are cleared.

For power users and system administrators, a new command-line option -X gc will be introduced. This allows users to toggle between the "incremental" collector and the "legacy" collector, providing a safety net for applications that might have highly specific tuning requirements optimized for the old behavior.

Implications for the Future of Multi-threaded Python

The introduction of the incremental GC is a vital step toward the broader goal of a "free-threaded" CPython (as outlined in PEP 703). The current global interpreter lock (GIL) simplifies memory management because only one thread can modify the object graph at a time. In a multi-threaded environment without a GIL, the garbage collector must be much more robust.

The incremental nature of the new collector provides a framework for concurrent collection, where the GC could potentially run on a separate thread alongside the main application. Shannon notes that while the current proposal is not yet fully concurrent, the "transitive closure" logic and the separation of pending/visited spaces lay the groundwork for a collector that can handle the complexities of multi-threaded object graphs.

Community Reaction and Industry Analysis

The proposal has generated significant discussion on the Python Discourse forums. Early reactions from the community have been largely positive, with developers praising the focus on reducing latency. Industry experts suggest that these changes will make Python even more competitive in the enterprise space, where Java and Go—both of which feature highly optimized incremental collectors—have traditionally held an advantage in low-latency applications.

Some researchers have pointed out that the success of the new GC will depend on its "work-to-do" heuristics. If the collector scans too slowly, memory usage will spike; if it scans too fast, the performance overhead might outweigh the benefits. However, the inclusion of performance analysis by Sergey Miryanov in the proposal provides a data-backed foundation for the default settings.

Chronology of Python’s GC Refinement

• Python 2.0 (2000): Introduction of the original generational cyclic garbage collector.
• Python 3.11 (2022): Significant performance improvements via the "Specializing Adaptive Interpreter."
• Python 3.13 (2024): Introduction of experimental "Free-threading" (No-GIL) support.
• Python 3.14 (Early 2025): Reversion of a non-generational incremental collector due to memory overhead concerns.
• September 2026: PEP draft for the new Incremental Generational GC for Python 3.16.
• Expected October 2027: Scheduled release of Python 3.16 featuring the new collector.

As CPython continues to evolve, the move toward a more sophisticated garbage collector reflects the language’s transition from a scripting tool to a primary driver of global computing infrastructure. By addressing the "pause time" problem, Python 3.16 aims to solidify its position as a high-performance, general-purpose language capable of handling the most demanding modern workloads.