Virtual Environments in Python provides an isolated and self contained instance of the version of Python and all other modules or packages that a specific project is dependent on. Virtual Environments appear in the form of directory with the name of the environment we choose and contain the version of Python, scripts, libraries and packages that are available with in the virtual environment.
Table of Contents
- What you will learn
- Working with Virtual Environments
- Create a Virtual Environment
- Activate the Virtual Environment
- Upgrade pip
- Install Modules in activated virtual environment
- Limitations with Virtual EnvironmentsΒ venv
- Freezing the Virtual Environment
- Deactivating the Virtual Environment
- Watch πΊ Demonstration of working with Virtual Environment on CentOS
- Video
What you will learn
In this tutorial we will see a brief introduction to virtual environments. By the end of this tutorial you will understand the following:
- Problems when virtual environments are not used
- What is the need for Virtual Environments
- Why it is suggested to work with virtual environments in Python
- How to create, activate deactivate virtual environment in Linux and Windows
- Install a package in activated virtual environment
- Portability limitation of virtual environments
- How to overcome the Portability limitation withΒ pip freeze
Working with Virtual Environments
Suppose that you are working on two different Python Projects and one of the projects requires the very oldΒ absintheΒ version 0.11 of Flask and another requires version 1.1.1.
The problem occurs if you performΒ pip install flask
Β directly on a PC level. Because the different projects that you work on or Python applications running on a PC require specific versions of Flask. Installing packages or Python modules at the system level may result in malfunctioning of other Python projects that have version specific requirements of their dependent modules. In order to get rid of this problem, Python has a concept of Virtual Environments.
The beauty of working with Python is it allows you to create and work in isolated environments for each of the Python project you work on a PC.
Virtual Environments in Python provides an isolated and self contained instance of the version of Python and all other modules or packages that a specific project is dependent on. Virtual Environments appear in the form of directory with the name of the environment we choose and contain the version of Python, scripts, libraries and packages that are available with in the virtual environment.
Working in activated virtual environments will scope the installation of Python packages to the active virtual environment. This facilitates working on different Python projects on a single PC that require different versions of the same package.
Create a Virtual Environment
The following is the standard command to set up a virtual environment.
python -m venv {name-of-virtualenv}
-m
Β : is the module argument that takes the name of the module.venv
Β : is the name of module to createΒ virtual environment. The standard moduleΒvenv
Β comes default with installation of Python.{name-of-virtualenv}
Β : is the name of the virtual environment of our choice.
Create a Virtual Environment namedΒ envΒ or any name of your choice by typing the following command. Letβs name our virtual environment asΒ env
Create a Python Virtual Environment in Windows OS
python -m venv env
Create a Python Virtual Environment in Unix based OS
python3 -m venv env
The above command tells the Python to useΒ venv
Β module that helps to create the virtual environment with the nameΒ env
.
Virtual Environment Tree Directory Structure
The created virtual environmentΒ envΒ will have the following folder structure
. ββββenv ββββInclude ββββLib β ββββsite-packages β ββββpip β β ββββ_internal β β β ββββcli β β β β ββββ__pycache__ β β β ββββcommands β β β β ββββ__pycache__ β β β ββββdistributions β β β β ββββ__pycache__ β β β ββββmodels β β β β ββββ__pycache__ β β β ββββoperations β β β β ββββ__pycache__ β β β ββββreq β β β β ββββ__pycache__ β β β ββββutils β β β β ββββ__pycache__ β β β ββββvcs β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ_vendor β β β ββββcachecontrol β β β β ββββcaches β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββcertifi β β β β ββββ__pycache__ β β β ββββchardet β β β β ββββcli β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββcolorama β β β β ββββ__pycache__ β β β ββββdistlib β β β β ββββ_backport β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββhtml5lib β β β β ββββfilters β β β β β ββββ__pycache__ β β β β ββββtreeadapters β β β β β ββββ__pycache__ β β β β ββββtreebuilders β β β β β ββββ__pycache__ β β β β ββββtreewalkers β β β β β ββββ__pycache__ β β β β ββββ_trie β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββidna β β β β ββββ__pycache__ β β β ββββlockfile β β β β ββββ__pycache__ β β β ββββmsgpack β β β β ββββ__pycache__ β β β ββββpackaging β β β β ββββ__pycache__ β β β ββββpep517 β β β β ββββ__pycache__ β β β ββββpkg_resources β β β β ββββ__pycache__ β β β ββββprogress β β β β ββββ__pycache__ β β β ββββpytoml β β β β ββββ__pycache__ β β β ββββrequests β β β β ββββ__pycache__ β β β ββββurllib3 β β β β ββββcontrib β β β β β ββββ_securetransport β β β β β β ββββ__pycache__ β β β β β ββββ__pycache__ β β β β ββββpackages β β β β β ββββbackports β β β β β β ββββ__pycache__ β β β β β ββββrfc3986 β β β β β β ββββ__pycache__ β β β β β ββββssl_match_hostname β β β β β β ββββ__pycache__ β β β β β ββββ__pycache__ β β β β ββββutil β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββwebencodings β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ__pycache__ β ββββpip-19.2.3.dist-info β ββββpkg_resources β β ββββextern β β β ββββ__pycache__ β β ββββ_vendor β β β ββββpackaging β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ__pycache__ β ββββsetuptools β β ββββcommand β β β ββββ__pycache__ β β ββββextern β β β ββββ__pycache__ β β ββββ_vendor β β β ββββpackaging β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ__pycache__ β ββββsetuptools-41.2.0.dist-info β ββββ__pycache__ ββββScripts
Activate the Virtual Environment
Once we have the virtual environment created, we need to activate that virtual environment.
Type in the following command based on your operating system to activate the virtual environment.
Activate Python virtual environment for Windows OS
Type the following command to activate Python virtual environment in Windows OS
env\Scripts\activate
Activate Python virtual environment for Unix based OS
Type the following command to activate Python virtual environment in Unix based OS
source ./env/bin/activate
After successful activation of the virtual environment you will see the name of the environment that got created will be prefixed to the command line terminal in rounded brackets. On Windows OS, this looks like
(env) c:\Users\navule\helloflask
If you have given any other name for the virtual environment, you should see the name in closed parentheses accordingly.
Upgrade pip
The first thing you should do after installing and activating a virtual environment is to upgradeΒ pip
.Β pip
Β is a module installer in Python. That means,Β pip
Β will help us to install Python modules so that we can consume it in our application.
pip
Β in itself is a module that comes by default with Python installation. There are frequent upgrades toΒ pip
. Hence it is a best practice to upgrade theΒ pip
Β so that it will always get the dependencies from the trusted sources for our project.
Run the following command to upgradeΒ pip
.
python -m pip install --upgrade pip
Install Modules in activated virtual environment
From now on, you can install any project related modules based on the requirement. Lets say the application needs to have flask module, you perform the module installation by running the command in the virtual environment activated shell as follows:
pip install flask
Now if you look at the site-packages directory you will find the Flask package along with its dependent packages like jinja, werkzeug are also installed.
. ββββenv ββββInclude ββββLib β ββββsite-packages β ββββclick β β ββββ__pycache__ β ββββClick-7.0.dist-info β ββββflask β β ββββjson β β β ββββ__pycache__ β β ββββ__pycache__ β ββββFlask-1.1.1.dist-info β ββββitsdangerous β β ββββ__pycache__ β ββββitsdangerous-1.1.0.dist-info β ββββjinja2 β β ββββ__pycache__ β ββββJinja2-2.11.1.dist-info β ββββmarkupsafe β β ββββ__pycache__ β ββββMarkupSafe-1.1.1.dist-info β ββββpip β β ββββ_internal β β β ββββcli β β β β ββββ__pycache__ β β β ββββcommands β β β β ββββ__pycache__ β β β ββββdistributions β β β β ββββ__pycache__ β β β ββββindex β β β β ββββ__pycache__ β β β ββββmodels β β β β ββββ__pycache__ β β β ββββnetwork β β β β ββββ__pycache__ β β β ββββoperations β β β β ββββbuild β β β β β ββββ__pycache__ β β β β ββββinstall β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββreq β β β β ββββ__pycache__ β β β ββββutils β β β β ββββ__pycache__ β β β ββββvcs β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ_vendor β β β ββββcachecontrol β β β β ββββcaches β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββcertifi β β β β ββββ__pycache__ β β β ββββchardet β β β β ββββcli β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββcolorama β β β β ββββ__pycache__ β β β ββββdistlib β β β β ββββ_backport β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββhtml5lib β β β β ββββfilters β β β β β ββββ__pycache__ β β β β ββββtreeadapters β β β β β ββββ__pycache__ β β β β ββββtreebuilders β β β β β ββββ__pycache__ β β β β ββββtreewalkers β β β β β ββββ__pycache__ β β β β ββββ_trie β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββidna β β β β ββββ__pycache__ β β β ββββmsgpack β β β β ββββ__pycache__ β β β ββββpackaging β β β β ββββ__pycache__ β β β ββββpep517 β β β β ββββ__pycache__ β β β ββββpkg_resources β β β β ββββ__pycache__ β β β ββββprogress β β β β ββββ__pycache__ β β β ββββpytoml β β β β ββββ__pycache__ β β β ββββrequests β β β β ββββ__pycache__ β β β ββββurllib3 β β β β ββββcontrib β β β β β ββββ_securetransport β β β β β β ββββ__pycache__ β β β β β ββββ__pycache__ β β β β ββββpackages β β β β β ββββbackports β β β β β β ββββ__pycache__ β β β β β ββββssl_match_hostname β β β β β β ββββ__pycache__ β β β β β ββββ__pycache__ β β β β ββββutil β β β β β ββββ__pycache__ β β β β ββββ__pycache__ β β β ββββwebencodings β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ__pycache__ β ββββpip-20.0.2.dist-info β ββββpkg_resources β β ββββextern β β β ββββ__pycache__ β β ββββ_vendor β β β ββββpackaging β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ__pycache__ β ββββsetuptools β β ββββcommand β β β ββββ__pycache__ β β ββββextern β β β ββββ__pycache__ β β ββββ_vendor β β β ββββpackaging β β β β ββββ__pycache__ β β β ββββ__pycache__ β β ββββ__pycache__ β ββββsetuptools-41.2.0.dist-info β ββββwerkzeug β β ββββdebug β β β ββββshared β β β ββββ__pycache__ β β ββββmiddleware β β β ββββ__pycache__ β β ββββwrappers β β β ββββ__pycache__ β β ββββ__pycache__ β ββββWerkzeug-1.0.0.dist-info β ββββ__pycache__ ββββScripts
Limitations with Virtual Environments venv
Virtual Environments are not Portable
Virtual environments are not portable. Virtual Environments will not work if you move them to a different directory within the same PC or across different PCs.
Though the virtual environments offer a great isolation to work with different versions of a module across projects by providing isolation between the projects, they completely take out the freedom of portability.
Hence it is not suggested to check-in the virtual environment directory to the source control. And the relative path of the virtual environment directory must be added to theΒ .gitignoreΒ file if you are using git as source control for version tracking.
Freezing the Virtual Environment
To overcome the portability limitation, you can freeze the virtual environment to a text file that has all the module information.
The following command when run in an activated virtual environment will freeze all the modules with version information to a file name of our choice.
pip freeze > requirements.txt
The above command with the help ofΒ pip
Β module freezes the requirements that the active virtual environment is dependent on to the file namedΒ requirements.txt. TheΒ requirements.txtΒ file will be created by pip if it does not already exists and updates if it exists already.
If we look at contents of requirements.txt, it will show something similar to this.
# c:\Users\navule\helloflask\requirements.txt Click==7.0 Flask==1.1.1 itsdangerous==1.1.0 Jinja2==2.11.1 MarkupSafe==1.1.1 Werkzeug==1.0.0
The fileΒ requirements.txtΒ must be checked-in to the version control like git.
Restore the Python packages from requirements.txt
When the repository is ported to the other location or PCs, we can then perform the following activities to restore the packages that our app is dependent on.
- Check out our project on a different PC
- Create a new and fresh virtual environment
- Activate the virtual environment
- Restore/ Install the modules that the app is dependent on fromΒ requirements.txt
Run the following command to restore or install the modules from theΒ requirements.txt
pip install -r requirements.txt
The above command tells the pip module of the activated virtual environment to recursively read and install all the modules mentioned in the requirements.txt
When to perform freeze
The freeze of modules is very important whenever we install a new package via pip. RunningΒ pip freeze > requirements.txt
Β will automatically update the existing requirements file and thus when a new developer or a new deployment happens, the new or updated installer packages will be installed withΒ pip install -r requirements.txt
Deactivating the Virtual Environment
Once you are done working with the application, you are suggested to deactivate the virtual environment. Run the following command to deactivate the virtual environment.
deactivate
Watch πΊ Demonstration of working with Virtual Environment on CentOS
Watch πΊ the real demo of creating Virtual Environment in CentOS.
Video
You can watch this video to understand how to work with virtual environments in Python. This video tutorial uses virtualenv
to create virtual environment.
Pingback: Setup Visual Studio Code for Python Development β TutLinks
Pingback: Developing Python Flask Hello World App from scratch β TutLinks
Pingback: Create and Deploy FastAPI app to Heroku β TutLinks
Pingback: Implementing Async REST APIs in FastAPI with PostgreSQL CRUD β TutLinks
Pingback: Debug FastAPI in VS Code IDE β TutLinks
Pingback: Deploy FastAPI app on Google Cloud Platform β TutLinks
Pingback: Deploy FastAPI on Ubuntu and Serve using Caddy 2 Web Server β TutLinks