Python wheel packaging

Owned by R D
Last updated: 2023-12-04
2 min read

Introduction

Installing Python libraries is as simple as pip install name . But how does it work?

Wheels

Python libraries contain a lot of non Python code (e.g. C++, CUDA, other compiled non Python libraries (e.g. DLL and shared objects)). Therefore, to allow installation by pip , the library as a whole is compiled and the output is grouped into a wheel file (extension whl ). The wheel file also contains metadata such as version, Python version compatibility, OS compatibility, and processor architecture compatibility. Every Python library typically has multiple wheel files for different processor architectures for the same Python and library version.

Wheels are uploaded the Python Package Index (PyPi) so that it can be automatically found, downloaded, and installed by pip :

The typical format of a wheel file name is: [pip name]-[version number]-[Python version]-[OS and processor architecture] , where:

  • [pip name] is the name on PyPi (e.g. opencv_python , pyav )

  • [version number] is the version of the Python library (e.g. 4.8.71.8 , 11.4.1 )

  • [Python version] is the supported Python version (e.g. cp37-abi3 , cp38-cp38 )

  • [OS and processor architecture] is the OS version and processor architecture. Example:

    • Windows 32 bit on x86: win32

    • Windows 64 bit on x86-64: win_amd64

    • Mac 64 bit on x86: macosx_10_16_x86_64

    • Max 64 bit on ARM: macosx_11_0_arm64

    • Linux 64 bit on x86-64: manylinux_2_17_x86_64.manylinux2014_x86_64

    • Linux 64 bit on ARM: manylinux_2_17_aarch64.manylinux2014_aarch64

Processor architecture

A processor architecture is the instruction set architecture (ISA) supported by the processor. Some notable ISAs include:

  • Intel x86 (x86)

  • AMD x86-64 (x86-64, AMD64)

  • ARM 64 bit (AArch64, ARM64, A64)

  • ARM 32 bit (AArch32, ARM32, A32)

  • RISC-V

Binary executables are usually only compatible with a single processor architecture, as they are composed of binary instructions that are specific to the ISA.

If a Python wheel is not available for a particular Python version, OS, and processor architecture, pip attempts to build it from source. This is not ideal as it greatly increases installation time, and there may be specific instructions by the maintainer on how to build a wheel for your own use.

Â