30 Nov 2025

Understand Python Virtual Environments: How and How-To

As of 2025, isolating system-level Python dependencies from user environments has become the de facto standard practice. Many Linux distributions, including Ubuntu 24.04 LTS, now follow PEP668, which prevents packages installation into the system-wide Python environment, effectively enforcing the use of virtual environments.

In this post, I will start with PEP668 and demonstrate the standard approach to manage Python virtual environments via pip and venv (both are built-in modules shipped with Python installation). Then, I will move on to PEP405 to discuss the implementation details behind these tools—specifically, how Python detects whether it is running inside a virtual environment and how it determines its package search path. Finally, I will share two practical examples of using virtual environments: (1) maintaining a user-level virtual environment for frequently used packages; (2) installing Python applications in isolated environments via pipx.

PEP668: The Enforcement of Virtual Environments

In 2021, PEP668 proposes to mark the system-wide Python environment as externally managed, such that Python-specific tools like pip cannot upgrade, downgrade, or remove any packages in it. This effectively guide end users towards using virtual environments.

For instance, running pip install numpy== on Ubuntu 24.04 LTS will result the following error message.

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.

    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.

    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.

    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

From the above message, we can see that the system-wide Python environment is marked as externally managed and can only be modified by the system package manager apt. This error message occurs when two conditions are met: (1) a Python interpreter is running outside a virtual environment; (2) there exists an EXTERNALLY-MANAGED file in a specific folder; see here for more details.

If we do not want to break the OS-level Python dependencies, the recommended solution is to create a virtual environment and install packages in it with pip.

python3 -m venv test
source test/bin/activate
pip3 install numpy==

The first command creates a virtual environment named test using the built-in tool venv. The second command activates that virtual environment so that pip3 points to the executable under test/bin/. The final command queries all available NumPy versions. The following section is about some implementation details behinds these commands.

PEP405: The Standardization of Virtual Environments

In 2011, PEP405 proposes an official mechanism in Python to support virtual enviroments. Inspired by the existing project virtualenv, PEP405 introduces a new module venv to the standard library which implements the creation of virtual environments.

The command python3 -m venv test calls the venv module to create a virtual environment in a folder ./test, whose content is listed below.

./
├── bin/
│   ├── Activate.ps1
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── pip*
│   ├── pip3*
│   ├── pip3.12*
│   ├── python -> python3*
│   ├── python3 -> /usr/bin/python3*
│   └── python3.12 -> python3*
├── include/
│   └── python3.12/
├── lib/
│   └── python3.12/
├── lib64 -> lib/
└── pyvenv.cfg

7 directories, 11 files

The command source test/bin/activate loads a bash profile to modify some bash variables, effectively activating the created virtual environment. This profile mainly does three things: (1) add test/bin/ to the executable path $PATH; (2) modify the bash prompt to show the name of the virtual environment; (3) set the environment variables VIRTUAL_ENV and VIRTUAL_ENV_PROMPT; (4) provide a function deactivate to undo all these effects.

The folder test/ now serves as an isolated Python environment. When activated, the system-wide Python interpreter and related tools are shadowed by the executables inside test/bin/. Specifically, running pip3 install numpy will install NumPy into test/lib/python3.12/ instead of a system directory like /usr/lib/python3.12/. In addition, packages installed within test/ become accessible by adjusting the Python package search path sys.path. The following subsections discuss in detail how this behavior is implemented.

Detecting virtual environments

So how Python detects whether it is running inside a virtual environment? You may notice that test/bin/python3 is actually a symbolic link pointing to the system-wide Python interpreter /usr/bin/python3. However, running the symbolic link does tells Python that it belongs to virtual environment.

The key is the configuration file test/pyvenv.cfg, which is created along with the creation of the virtual environment. It contains useful information about the environment. Moreover, it will be scanned by the Python interpreter to detect the virtual environment. The precise procedure is described in PEP405.

If a pyvenv.cfg file is found either adjacent to the Python executable or one directory above it (if the executable is a symlink, it is not dereferenced), this file is scanned for lines of the form key = value. If a home key is found, this signifies that the Python binary belongs to a virtual environment, and the value of the home key is the directory containing the Python executable used to create this virtual environment.

The value of the home key in pyvenv.cfg will be assigned to sys.base_prefix and the directory containing pyvenv.cfg will be assigned to sys.prefix. After the initialization, a Python script can use assert sys.prefix == sys.base_prefix to detect the virtual environment.

Determining package search path

When running inside a virtual environment, the initialization result of the package search path sys.path differs slightly from the normal case. Specifically, the site module is responsible for adding site-packages directories to sys.path. According to the documentation, it uses sys.prefix to construct the paths containing site-packages. In this case, however, the value of sys.prefix is the root of the virtual environment, resulting an entry like /abs/path/to/test/lib/python3.12/site-packages.

Example Usage: Maintaining a User-Level Virtual Environment

While it's generally recommended to use separate environments for different projects, maintaining a user-level global environment can still be convenient when:

writing small scripts that are used only once;
starting an interactive Python shell without worrying about dependencies beforehand;
quickly prototyping or developing small projects without managing dependencies.

As an example usage, I maintain a virtual environment at ~/.venv/, which is created by

/usr/bin/python3.12 -m venv --prompt="python3.12" /home/dou/.venv

Whenever needed, I activate this virtual environment and use Python as usual. In my case, I have a handy collection of commonly used packages installed for convenience, such as ipython, numpy, matplotlib, pandas, and torch.

It is worth emphasizing that a created Python virtual environment can be used even without activation. As detailed in the previous section, activating an virtual environment simply adds its bin/ folder to our system path. Moreover, Python's mechanism for detecting virtual environments does not rely on the activation. This can be demonstrated by the following commands.

python3 -c "import sys; print(sys.prefix)" # prints: /usr
/home/dou/.venv/bin/python3 -c "import sys; print(sys.prefix)" # prints: /home/dou/.venv

This is particularly useful when writing small scripts intended to run inside the virtual environment.

#!/home/dou/.venv/bin/python3
import sys
print(sys.prefix, sys.base_prefix)

Save this script as test_python_venv.py and make it executable. Calling ./test_python_venv.py will ensure that it is running inside our virtual environment.

Example Usage: Installing Python Applications via pipx

pipx is a convenient tool for installing and running Python applications in isolated virtual environments.

sudo apt install pipx
pipx install black

In this example, we first install pipx into the system-wide Python environment via apt. Then, pipx install black creates a new virtual environment and installs black into it. Note that pipx installs each package in its own virtual environment. For instance, pipx install livereload will create another virtual environment specifically for livereload. As a result, these two packages reside in separate virtual environments. By default, all virtual environments created by pipx are placed in ~/.local/share/pipx/venvs/, which might look like

./
├── black/
│   ├── bin/
│   ├── include/
│   ├── lib/
│   ├── lib64 -> lib/
│   ├── pipx_metadata.json
│   └── pyvenv.cfg
└── livereload/
    ├── bin/
    ├── include/
    ├── lib/
    ├── lib64 -> lib/
    ├── pipx_metadata.json
    └── pyvenv.cfg

11 directories, 4 files

This guarantees that each application is running in its own environment and will not interfere with each other. For more details, please run pipx list.

pipx is designed for installing Python applications that act as standalone tools. For example, black is a Python code formatter typically used via black 1.py, which formats the given file in-place. If installed via pipx, an executable black.py will be available in ~/.local/bin/. For more details on how pipx works, please see the official documentation.

References and Further Readings

Breuss, M. (2024). Python virtual environments: A primer. Real Python. Link

Pip Documentation. User guide. Retrieved October 13, 2025, from Link

Python Enhancement Proposals. PEP 405 – Python virtual environments. Retrieved October 13, 2025, from Link

Python Enhancement Proposals. PEP 668 – Marking Python base environments as “externally managed”. Retrieved November 30, 2025, from Link

Python Org. The initialization of the sys.path module search path. Retrieved October 13, 2025, from Link

Python Org. Installing Python modules. Retrieved October 13, 2025, from Link

Python Org. site - Site-specific configuration hook. Retrieved November 30, 2025, from Link

Python Org. sys — System-specific parameters and functions. Retrieved October 13, 2025, from Link

Python Org. venv — Creation of virtual environments. Retrieved October 13, 2025, from Link

Ramos, L. P. (2025). uv vs pip: Managing Python Packages and Dependencies. Real Python. Link