Python Enhancement Proposal for a TOML-based Configuration File Format to Replace the PTH File Mechanism in Site Initialization

In a significant move to modernize the Python interpreter’s startup sequence and enhance the security of the packaging ecosystem, a new proposal has been drafted to overhaul how the language handles site-specific path configurations and initialization code. Authored by veteran Python developer Barry Warsaw and slated for potential inclusion in Python 3.15, the proposal seeks to replace the decades-old .pth file system with a structured, declarative format based on TOML (Tom’s Obvious, Minimal Language). This shift represents one of the most fundamental changes to Python’s internal "site" module in years, aiming to resolve long-standing issues related to security, maintainability, and clarity in package installation.

The proposal introduces a new file naming convention, <package>.site.toml, which will coexist with and eventually supersede the traditional .pth files found in Python’s site-packages directories. By leveraging the tomllib module—which became part of the Python standard library in version 3.11—the new system moves away from the ad-hoc, line-based logic of .pth files toward a rigorous schema that separates directory path extensions from executable initialization logic.

The Historical Context of the PTH Mechanism

To understand the necessity of this transition, one must look back at the evolution of Python’s package management. Since the early days of the language, .pth files have served as a simple mechanism for extending sys.path. When the Python interpreter starts, the site.py module scans the site-packages directory for any files ending in .pth. Each line in such a file is typically treated as a directory path to be added to the interpreter’s search list.

However, a secondary, unintended feature of .pth files became a standard industry practice: any line starting with the word import is executed as Python code. While this allowed developers to perform necessary startup tasks—such as registering namespace packages or configuring specialized environmental hooks—it created a significant security loophole. Because .pth files conflate data (paths) with code (imports), they have become a "dark corner" of the Python ecosystem where arbitrary code can be executed silently upon interpreter startup without explicit user consent or easy auditability.

The limitations of the .pth format are also technical. The format is inherently fragile; it is sensitive to line breaks, lacks support for comments, and offers no structured way to handle metadata. Furthermore, if a .pth file contains an error, the site.py module often aborts the processing of the remainder of that specific file, leading to partially configured environments that are difficult to debug.

Technical Specifications of the site.toml Format

The proposed <package>.site.toml format addresses these deficiencies by introducing a three-section schema: [metadata], [paths], and [entrypoints]. This structure ensures that the interpreter knows exactly what a file intends to do before any action is taken.

The [metadata] section includes a schema_version key. By declaring a version, package authors can ensure forward compatibility. If a future version of Python introduces a new schema that is incompatible with the current proposal, the interpreter can gracefully handle or skip the file based on the version integer.

The [paths] section provides a dedicated space for directory extensions. Unlike the old format, which relied on relative paths that could be ambiguous, the new TOML format supports a hybrid resolution scheme. It introduces the sitedir placeholder, allowing authors to explicitly reference the directory where the TOML file resides. This eliminates the guesswork involved in relative path resolution and provides a foundation for future placeholders like prefix or userbase.

Perhaps the most critical change lies in the [entrypoints] section. This section replaces the "import hack" with a structured list of entry points under the init key. Instead of executing arbitrary strings of code, the system will use the standard package.module:callable syntax. This allows the site module to use the formal importlib machinery to load modules and call specific initialization functions. This approach is not only more readable but also integrates seamlessly with Python’s existing import hooks and logging systems.

A Two-Phase Processing Model

A core innovation of this proposal is the implementation of a two-phase processing model during the interpreter’s startup. In the current .pth system, files are processed one by one, with paths added and code executed immediately upon discovery. This "eager" execution makes it impossible for the system to understand the full state of the environment until the process is finished.

Under the new proposal, the site.py module will first perform a "Discovery and Parsing" phase. All <package>.site.toml files in a given directory are read and parsed into an intermediate data structure. Only after all configurations have been validated does the "Execution" phase begin. During execution, the system follows a strict order:

1. All directory paths from all discovered TOML files are appended to sys.path.
2. Any import lines from legacy .pth files are executed.
3. All init entry points from the new TOML files are executed.

This sequence ensures that all necessary paths are in place before any initialization code runs, significantly reducing "ModuleNotFoundError" issues that plague complex package installations. Furthermore, the alphabetical processing of files within each phase maintains the predictable behavior that developers have come to expect from the legacy system.

Security and Auditability Improvements

From a security perspective, the transition to TOML is a major leap forward. Security researchers have long pointed out that .pth files are an ideal vector for persistence in supply chain attacks. A malicious package can drop a .pth file that executes code every time a user runs python, making the malware difficult to detect without manually inspecting every file in the site-packages directory.

The new TOML format makes initialization logic declarative and visible. Automated security scanners and audit tools can easily parse TOML files to identify which packages are requesting startup execution. By restricting execution to defined entry points rather than arbitrary exec() calls, the attack surface is narrowed. While a malicious package could still point an entry point to a malicious function, the presence of that link is now explicitly documented in a structured format, making it much easier to flag during automated repository scans on platforms like the Python Package Index (PyPI).

Error Handling and Robustness

The proposal significantly improves how Python handles configuration errors. In the legacy system, a single malformed line in a .pth file could halt the entire configuration process for that package. The new specification adopts a "continue on error" philosophy.

If a TOML file fails to parse during the discovery phase, it is skipped, and a warning is issued (visible under verbose mode). Crucially, even a broken .toml file will still supersede a corresponding .pth file. This prevents a "fallback" scenario where a developer might unintentionally run outdated or conflicting logic from a legacy file. During the execution phase, if a specific entry point raises an exception, the traceback is printed to sys.stderr, but the interpreter continues to process the remaining entry points and files. This ensures that a failure in one package does not render the entire Python environment unusable.

Chronology and Implementation Timeline

The journey toward this proposal has been several years in the making, following the broader trend of "TOML-ification" within the Python ecosystem.

• 2016: PEP 518 introduced pyproject.toml, beginning the move away from setup.py.
• 2022: Python 3.11 included tomllib, providing the necessary infrastructure to read TOML without external dependencies.
• 2024-2025: Discussions within the Python Packaging Authority (PyPA) highlighted the need to clean up the "dark corners" of site.py.
• March 2026: The formal draft of the PEP is published by Barry Warsaw, targeting the Python 3.15 release cycle.

The reference implementation, already available as a pull request on the CPython GitHub repository, includes comprehensive modifications to Lib/site.py and a new suite of tests in Lib/test/test_site.py.

Broader Impact on the Ecosystem

For package authors, the migration path is designed to be as painless as possible. The proposal provides clear examples of how to translate legacy .pth hacks into clean TOML declarations. For instance, a line like import my_package; my_package.init() becomes a simple entry in the [entrypoints] section.

Tool makers, including build backends like Flit, Hatch, and Setuptools, as well as installers like pip, will play a crucial role in this transition. These tools will need to be updated to generate .site.toml files instead of .pth files when targeting Python 3.15 and later. The proposal suggests that for a transitional period, packages should ship both files to maintain compatibility with older Python versions, with the understanding that modern interpreters will prioritize the TOML version.

The shift also opens the door for future site-wide policy controls. Because the configuration is now structured, system administrators could theoretically implement policies that disable the [entrypoints] section for certain users or environments while still allowing [paths] to be extended. This level of granular control was impossible with the "all-or-nothing" nature of the legacy system.

In conclusion, the proposal to replace .pth files with <package>.site.toml is more than a simple change in file format. It is a fundamental cleanup of Python’s startup logic that prioritizes security, structure, and reliability. As Python continues to be a dominant force in data science, web development, and system automation, modernizing these foundational components is essential for the long-term health of the language’s vast library ecosystem. By moving to a declarative and auditable system, Python 3.15 aims to provide a more robust environment for developers and a more secure experience for users worldwide.