Install and Configure Embedded Python on Windows
I wanted to document these steps after downloading embedded Python recently. I extracted it, set the environment path and started the interpreter. Everything seemed to be going smoothly until I tried to exit exit() and it reported back ’exit not found’ or something like that. What!? How can this be? I ended up import sys and using sys.exit() but this sent me on an adventure to determine how to use embedded Python as a regular install.
These steps have been tested on Window 10, Server 2019 and Server 2022.
Download Python
First visit the Python website and select which version you would like to install. For this post I selected Python 3.12.0 but these steps will work with other versions.
After downloading make sure the the MD5 checksum matches. This can be accomplished using PowerShell and the Get-FileHash command.
Get-FileHash python-3.12.0-embed-amd64.zip MD5 I like to copy the sum output and use the browsers find in page function to make sure that is matches (as shown in the screenshot above).
Extract the zip file
Next extract the zip file to your preferred install location. In this case I created c:\python312 this gives a good indication of what is in the directory as well as giving all users access to it.
Add Python to the Path
After it is extracted it’s time to add it to the path so it can be used.
Be sure to add both c:\python312 and c:\python312\Scripts as the Scripts directory is where Python stores other executable files such as pip and virtualenv.
Check that Python is Working
Now that it has been added to the path open a new PowerShell terminal to verify that it is working.
You can check that it is on the path with the following command.
$env:Path You can also try running Python or checking the version as in the image above.
Install PiP
Head over to pypa.io to find the get-pip.py script.
After downloading the file execute using python to install.
python.exe .\get-pip.py You should see the following output.
Create a Virtual Environment
Finally navigate to your development directory and create a virtual environment.
virtualenv.exe venv Activate it with the following command and get started on development!
.\ven\Scripts\activate - Python
- Windows
- PowerShell
Latest Posts
- Using a Kubernetes Configmap in a Pod
- CentOS 9 Automated KVM Install Using Kickstart
- Ansible — Please add this host’s fingerprint to your known_hosts file to manage this host
- Install and Configure Embedded Python on Windows
- Convert Images to WebP using the Linux command line and Docker
Установка pip из архива Windows
Я хочу упаковать питон и все зависимости моего кода в один дистрибутив. Скачал архив с питоном, прописал путь к нему в моих скриптах, работает. Не могу разобраться, как установить другие модули. Если скачивать их и пытаться запустить, получаю ошибку
Traceback (most recent call last): File "./packages/setuptools-51.1/setuptools-51.1.0/setup.py", line 7, in import setuptools ModuleNotFoundError: No module named 'setuptools' Скрипт, который я запускаю:
#! ./python-3.6.5-embed-amd64/python.exe from subprocess import call call(["python", "./packages/setuptools-51.1/setuptools-51.1.0/setup.py"]) Но чтобы поставить тот же setuptools нужен setuptools . Как еще можно установить pip или setuptools из локальной директории?
python-embedded-launcher 0.6
The launcher is a small C program that loads the Python DLL and calls Py_Main with itself as parameter, loading a zipped Python application appended to the exe.
Yes, it is yet an other tool to make standalone Windows applications with Python.
See also py2exe, pyinstaller, cx_Freeze. I guess the basic idea is different from these tools, as we try to combine complete packages with a Python minimal distribution: Python + wheel files. There is no attempt made to search together all the required files, there is no scan for import statements etc.
See also pex, which can grab dependencies from pypi and make zip applications, but that does not bundle the Python interpreter.
Quick Start
- make a setup.py for your application, use scripts and/or entry_points for console_scripts
- run python setup.py bdist_launcher
Done. See result in dist/launcher* .
Usage
The simplest way to use this tool is to add a setup.py to the application that should be distributed.
setup( name="sample_application", description="Small sample application for python-embedded-launcher", # . packages=[. ] setup_requires=[. ] # . entry_points=< 'console_scripts': [ 'app1 = app.core:main', ], >, scripts=['scripts/app2'], )
Then running python setup.py bdist_launcher will do the following steps:
- Run bdist_wheel and then …
- use that wheel file to install the application and all dependencies to a subdirectory within the dist directory.
- Create executables for all scripts and console_scripts entries. When the —icon option is given, the provided icon will be applied to all executables. When the option —python-minimal is used, then the location of the python-minmal directory is overriden and the following step to create a copy is skipped.
- Finally, copy/download a pythonX-minimal distribution to the dist directory (that is, unless —python-minimal was used).
Options for ‘bdist_launcher’ command:
--icon filename of icon to use --python-minimal change the location of the python-minimal location --extend-sys-path (-p) add search pattern(s) for files added to sys.path (separated by ";") --wait-at-exit do not close console window automatically --wait-on-error wait if there is an exception
These options apply to all created launchers (if more than one is generated).
All those options can also be specified in the setup.cfg file (replace — with _ in all option names, drop the leading — ).
The section that applies globally is called [bdist_launcher] but for customization of single files, it is also possible to make such a section per file, e.g. if an example.exe is generated, the corresponding section would be [bdist_launcher.example.exe] .
Advanced Usage
It is also possible to apply the steps to create a distribution manually.
The example directory has demos where these steps are written in a batch file that is ready to run. The description here explains the steps. On Windows, once Python 3 is installed, the Python Launcher py is available, this is what is used here. Otherwise replace py -2 / py -3 with python / python3 . When packaging an app, the same Python version that is packaged, should be used to run the steps here (using py -2 or py -3 accordingly).
Assuming your own project has a setup.py , install to a dist directory:
py -m pip install --prefix=dist --ignore-installed /path/to/your/project
py -m pip install --prefix=dist --ignore-installed -r requirements.txt
Create a Python distribution:
py -2 create_python27_minimal.py -d dist
or for Python 3:
py -3 -m launcher_tool.download_python3_minimal -d dist
Use the launcher tool to write the exe, calling your app:
py -m launcher_tool -o dist/myapp.exe -e mymodule:main
pip will also install scripts in a subdirectory called Scripts . this usually not needed for a packaged app, so this can be deleted.
Variations
Instead of —prefix=dist it is also possible to use —user when the envirionment variable PYTHONUSERBASE is set to dist . This will install into a slightly different subdirectory of dist but lanucher.py also searches this one.
It is also possible download all dependencies as wheels first, so that subsequent runs to create a distribution do not need to download from the Internet (recommended).
Fetch the dependencies once:
py -m pip wheel -w wheels -r requirements.txt
Then use these with —find-links and —no-index options:
py -m pip install --prefix=dist --ignore-installed --find-links=wheels --no-index -r requirements.txt
Alternatives
It is also possible to install pip within the embedded Python distribution and use that distribution itself to install packages:
py -3 -m launcher_tool.download_python3_minimal cd python3-minimal python get-pip.py python -m pip install --find-links=/path/to/wheels --no-index -r requirements.txt cd .. py -3 -m launcher_tool -o myapp.exe -e mymodule:main
First we use py -3 to use the systems Python 3, then python to call the local version in the directory. The first step is installing pip with get-pip.py. Then using this to install more packages. Installing from source may not work, it is recommended to only use wheels for this step.
Python 3’s zipapp module can be used to package the application:
py -3 -m zipapp myapp.py -o myapp.pyz py -3 -m launcher_tool -o myapp.exe --run-path myapp.pyz
Tools
launcher_tool
A tool to combine scripts with the launcher27.exe or launcher3.exe . A script is added with the name __main__.py to a zip file. launcher.py a helper module for the boot script is also appended to the zip. This zip file is appended to the exe. Optionally it can also include other files too.
Used to create a python27-minimal distribution. It copies the Python installation from the system.
Unpack a Python 3 embedded distribution. The data is downloaded from https://www.python.org/downloads/windows/ and cached locally (so that for repeated runs, it does not need to use the Internet again).
Copy the launcher.exe to a file. Used e.g. for customizations using launcher_tool.resource_editor .
A small Windows resource editor that can modify the launcher. It uses Windows API functions to read and write the data.
- adding and editing strings
- retrieving and writing icons
- export resources as (binary) blob
- removing any resource type
- adding any resource type is supported partially (currently limited by data input possibilities)
- dump resources
- dump decoded string table
Attention! It will strip debug data and remove the attached ZIP file! So this tool must be used before the application is appended to the launcher.
Customization
The texts and the location of Python is stored as Windows resource in the launcher*.exe . It is possible to use resource editor tools to patch the exe.
Using launcher_tool.resource_editor it is possible to make small edits on the command line, but it does not support all resource types.
E.g. if there was a common Python package installed under %LOCALAPPDATA% a series of commands like this would create a modified launcher:
python -m launcher_tool.copy_launcher -o %DIST%/myapp.exe python -m launcher_tool.resource_editor %DIST%/myapp.exe edit_strings --set 1:^%LOCALAPPDATA^%\python27-minimal python -m launcher_tool.resource_editor %DIST%/myapp.exe write_icon newicon.ico python -m launcher_tool --append-only %DIST%/myapp.exe -e mymodule:main
Note that ^ is the escape character of cmd.exe when used interactively and makes that the % is not treated specially but as normal text (and the variable is thus not expanded). For some reason %% must be used instead of ^% when these lines are put in a .bat file.
An 3rd party tool would be resourcehacker. It can even edit exe files with attached zip data without destroying them.
Alternatively use the sources here to recompile the binaries, it really just needs a mingw gcc (which is only a few dozens of megabytes large). In that case the launcher*.rc within the src/python* directory are edited with a text editor and compile.bat is used to recreate the exe.
Scenarios
Distribute an application
Bundle Python with an application so that users can use it without having to install Python.
In launcher27.rc set IDS_PYTHONHOME to «%SELF%\\python27-minimal» (this is already the default). This way, the Python distribution is expected at the location of the executable. The environment variable SELF is set automatically by the launcher itself (dirname of abspath of exe).
Common python-minimal package
Multiple tools can use a common copy of Python. e.g. with a package manager. Python can be provided as one package and separate application packages can use that Python distribution to run.
In launcher27.rc set IDS_PYTHONHOME to «%PACKAGE_ROOT%\\python27-minimal» . This way, the Python distribution is expected to be at a fixed location, where the PACKAGE_ROOT variable points at. It is expected to be set by the package manager.
Build
Requires a mingw gcc compiler (see Requirements).
Run compile_all.bat in the src directory.
The python27 and python3 directories contain the sources and a batch file. The compile_all.bat file runs both of them.
The resulting binaries will be placed in the launcher_tool directory so that they are available as data files for the Python tool.
Requirements
To build applications:
- pip and wheel
- requests (for download_python3_minimal )
Running pip install -r requirements.txt will install these.
To build the launcher exe:
- mingw(-64) GCC compiler, e.g. http://tdm-gcc.tdragon.net/ has one.
The either PATH must be set so that gcc can be found or the compile*.bat files have to be edited (they set PATH ).
API
A small helper module called launcher is automatically packaged with the exe. It contains a few helper functions.
Add directories (relative to executable, if existing) to sys.path .
- the direcrtory of the executable
- Python/site-packages
- Python/Lib/site-packages
- Lib/site-packages
These locations are also scanned for .pth files.
Add files matching a pattern (e.g. *.zip , *.whl , *.egg ) to sys.path . the pattern is prefixed with the location of the executable. In case of wheel files, it only works for pure Python wheels and only if they do no access the file system to load data on their own (should use pkgutil). This function is used if the command line option —extend-sys-path is used.
Get original command line via Windows API. Restores sys.argv (which is used by the launcher to pass the location of Python). This function is called by the default boot code ( __main__ ).
Useful for GUI applications, it closes a separate console window if there is one, e.g. when the exe was started by a double click.
Return true if the console window was opened with this process (e.g. the console was opened because the exe was started from the file Explorer).
Wait at exit, but only if console window was opened separately. This function is called automatically if the command line option —wait is used.
Wait if the program terminates with an exception, but only if console window was opened separately. This function is called automatically if the command line option —wait-on-error is used.
Implementation Details
Some random notes…
Python 2 uses “ASCII API” while Python 3 uses “Unicode API”. Thats why separate code for the two launchers exists.
The launcher is compiled as console application, so it opens a console window when started from the explorer. However it is easily closed with a Windows API call and launcher.py , which is added to the application, has a function for that. The advantage is, that applications can be started in a console and one can see the output — and wait for the program to terminate etc.
Starting with Python 3.5, an embedded Python distribution is already available (and used here) for download, see https://docs.python.org/3/using/windows.html#embedded-distribution
While Python 3 has a python3.dll , which would be nice to use, as it would make the launcher independent of the Python version – it won’t work. Py_SetPath is not exposed by that library. As a workaround, the name (e.g. python35 ) is in the resources of launcher3.exe so that it can be changed without recompiling.
Python is loaded dynamically via LoadLibrary . The launcher is not linked against the DLL. This has the advantage that the location of the DLL can be different to the one of the exe and that the DLL name can be provided and edited as resource (only in launcher.exe ). The separation would also allow to check if the VC runtime is installed and direct the user to the download if it is not, but this is not implemented yet.
Why put Python in a subdirectory? Because someone could add the directory containing the exe to the PATH and then the system would potentially find multiple python.exe and pythonXY.dll …
pip install —user installs the packages into a subdirectory PythonXY/site-packages named after the Python version.
pip install —prefix=dist installs the packages to a subdirectory Lib/site-packages .
jtmoon79 / python-embedded-for-Win10.md
Save jtmoon79/ce63fe655b2f544462e70d8e5ec30ff5 to your computer and use it in GitHub Desktop.
fully configuring embedded Python on Windows 10
Update: use PowerShell script PythonEmbed4Win.ps1.
The instructions in this gist have some subtle problems and this gist will not be updated.
About
Embedded Python (Windows embeddable package zip file) will not find installed packages and modules . It requires a few extra steps to fully configure.
These instructions will install, configure, and then create a new virtual python environment based on the embedded distribution.
This Python distribution and any derived virtual environments will run in a portable manner (no need for environment variable hacks, no special registry settings, etc.).
Tested with python-3.9.6-embed-amd64.zip on Windows 10 using powershell.
- Install and configure embedded Python
- Create a new virtual environment based on the embedded distribution
Install and configure embedded Python
It is recommended to run these steps as an Administrator.
Download and unzip
- Download the .zip file
- Unzip to some path (this will use example path C:\python-3.9.6-embed-amd64\ ).
PS > Expand-Archive .\python-3.9.6-embed-amd64.zip -DestinationPath C:\python-3.9.6-embed-amd64 PS > dir Directory: C:\python-3.9.6-embed-amd64 Mode LastWriteTime Length Name ---- ------------- ------ ---- -a---- 8/27/2021 12:12 PM 3406016 libcrypto-1_1.dll -a---- 8/27/2021 12:12 PM 32792 libffi-7.dll -a---- 8/27/2021 12:12 PM 690368 libssl-1_1.dll -a---- 8/27/2021 12:12 PM 32628 LICENSE.txt -a---- 8/27/2021 12:12 PM 192744 pyexpat.pyd -a---- 8/27/2021 12:12 PM 561429 python.cat -a---- 8/27/2021 12:12 PM 101608 python.exe -a---- 8/27/2021 12:12 PM 59624 python3.dll -a---- 8/27/2021 12:12 PM 4485864 python39.dll -a---- 8/27/2021 12:12 PM 2502717 python39.zip -a---- 8/27/2021 12:12 PM 79 python39._pth -a---- 8/27/2021 12:12 PM 100072 pythonw.exe -a---- 8/27/2021 12:12 PM 28904 select.pyd -a---- 8/27/2021 12:12 PM 1538792 sqlite3.dll -a---- 8/27/2021 12:12 PM 1121512 unicodedata.pyd -a---- 8/27/2021 12:12 PM 97160 vcruntime140.dll -a---- 8/27/2021 12:12 PM 37256 vcruntime140_1.dll -a---- 8/27/2021 12:12 PM 29928 winsound.pyd -a---- 8/27/2021 12:12 PM 65256 _asyncio.pyd -a---- 8/27/2021 12:12 PM 86760 _bz2.pyd -a---- 8/27/2021 12:12 PM 127208 _ctypes.pyd -a---- 8/27/2021 12:12 PM 270568 _decimal.pyd -a---- 8/27/2021 12:12 PM 179944 _elementtree.pyd -a---- 8/27/2021 12:12 PM 66280 _hashlib.pyd -a---- 8/27/2021 12:12 PM 163048 _lzma.pyd -a---- 8/27/2021 12:12 PM 40168 _msi.pyd -a---- 8/27/2021 12:12 PM 30440 _multiprocessing.pyd -a---- 8/27/2021 12:12 PM 46312 _overlapped.pyd -a---- 8/27/2021 12:12 PM 29416 _queue.pyd -a---- 8/27/2021 12:12 PM 80616 _socket.pyd -a---- 8/27/2021 12:12 PM 89832 _sqlite3.pyd -a---- 8/27/2021 12:12 PM 155368 _ssl.pyd -a---- 8/27/2021 12:12 PM 23784 _uuid.pyd -a---- 8/27/2021 12:12 PM 47336 _zoneinfo.pyd - Unzip C:\python-3.9.6-embed-amd64\python39.zip to C:\python-3.9.6-embed-amd64\python39
PS> Expand-Archive C:\python-3.9.6-embed-amd64\python39.zip -DestinationPath C:\python-3.9.6-embed-amd64\python39 Without these changes, you may see an error regarding pickle .
error: [Errno 0] Error: 'lib2to3\\Grammar3.6.5.final.0.pickle' The issue is pickle cannot read files within a .zip file. 3a. Optional: delete python39.zip
PS> Remove-Item "C:\python-3.9.6-embed-amd64\python39.zip" adjust paths
Running C:\python-3.9.6-embed-amd64\python -m pip will fail with No module named pip .
Path settings must be adjusted.
-
Change C:\python-3.9.6-embed-amd64\python39._pth to
.\python39 .\Scripts . # importing site will run sitecustomize.py import site Without these changes, you may see error
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings' import sys PS> New-Item -type directory "C:\python-3.9.6-embed-amd64\DLLs" Adding directory DLLs will workaround the module-loading error:
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\python-3.9.6-embed-amd64\\DLLs' PS> C:\python-3.9.6-embed-amd64\python.exe -c "import sys; print(sys.path)" ['C:\\python-3.9.6-embed-amd64\\python39', 'C:\\python-3.9.6-embed-amd64', 'C:\\python-3.9.6-embed-amd64\\Scripts', 'C:\\python-3.9.6-embed-amd64\\lib\\site-packages'] Install pip using get-pip.py
PS> Invoke-WebRequest -OutFile "C:\python-3.9.6-embed-amd64\get-pip.py" "https://bootstrap.pypa.io/get-pip.py" PS> C:\python-3.9.6-embed-amd64\python.exe "C:\python-3.9.6-embed-amd64\get-pip.py" Install virtualenv package
PS> C:\python-3.9.6-embed-amd64\python.exe -m pip install virtualenv Create a new virtual environment based on the embedded distribution
This may be run as a normal user.
From some other directory run
PS> C:\python-3.9.6-embed-amd64\python.exe -m virtualenv --copies ".venv-3.9-embedded" This creates a new fully functioning and fully independent virtual python environment, i.e. does not use symbolic links or Windows Registry entries or $env environment variables.
Activate the virtual environment
PS> .\.venv-3.9-embedded\Scripts\activate.ps1 Verify the expected python.exe is used
PS> Get-Command python.exe | Format-Table -Autosize CommandType Name Version Source ----------- ---- ------- ------ Application python.exe 3.9.6150.1013 C:\Users\user1\My-Project\.venv-3.9-embedded/Scripts\python.exe - Setting up python’s Windows embeddable distribution (properly)
- Python 3.9.6
- Using Python on Windows — Finding Modules
- stackoverflow question: Python Windows embeddable package fails to run «No module named pip» «The system cannot find the path specified: ‘C:\python-3.9.6-embed-amd64\DLLs'»