Standard way to embed version into python package?

StackOverflow https://stackoverflow.com/questions/458550

  •  19-08-2019
  •  | 
  •  

Question

Is there a standard way to associate version string with a python package in such way that I could do the following?

import foo
print foo.version

I would imagine there's some way to retrieve that data without any extra hardcoding, since minor/major strings are specified in setup.py already. Alternative solution that I found was to have import __version__ in my foo/__init__.py and then have __version__.py generated by setup.py.

Was it helpful?

Solution

Not directly an answer to your question, but you should consider naming it __version__, not version.

This is almost a quasi-standard. Many modules in the standard library use __version__, and this is also used in lots of 3rd-party modules, so it's the quasi-standard.

Usually, __version__ is a string, but sometimes it's also a float or tuple.

Edit: as mentioned by S.Lott (Thank you!), PEP 8 says it explicitly:

Version Bookkeeping

If you have to have Subversion, CVS, or RCS crud in your source file, do it as follows.

    __version__ = "$Revision: 63990 $"
    # $Source$

These lines should be included after the module's docstring, before any other code, separated by a blank line above and below.

You should also make sure that the version number conforms to the format described in PEP 440 (PEP 386 a previous version of this standard).

OTHER TIPS

I use a single _version.py file as the "once cannonical place" to store version information:

  1. It provides a __version__ attribute.

  2. It provides the standard metadata version. Therefore it will be detected by pkg_resources or other tools that parse the package metadata (EGG-INFO and/or PKG-INFO, PEP 0345).

  3. It doesn't import your package (or anything else) when building your package, which can cause problems in some situations. (See the comments below about what problems this can cause.)

  4. There is only one place that the version number is written down, so there is only one place to change it when the version number changes, and there is less chance of inconsistent versions.

Here is how it works: the "one canonical place" to store the version number is a .py file, named "_version.py" which is in your Python package, for example in myniftyapp/_version.py. This file is a Python module, but your setup.py doesn't import it! (That would defeat feature 3.) Instead your setup.py knows that the contents of this file is very simple, something like:

__version__ = "3.6.5"

And so your setup.py opens the file and parses it, with code like:

import re
VERSIONFILE="myniftyapp/_version.py"
verstrline = open(VERSIONFILE, "rt").read()
VSRE = r"^__version__ = ['\"]([^'\"]*)['\"]"
mo = re.search(VSRE, verstrline, re.M)
if mo:
    verstr = mo.group(1)
else:
    raise RuntimeError("Unable to find version string in %s." % (VERSIONFILE,))

Then your setup.py passes that string as the value of the "version" argument to setup(), thus satisfying feature 2.

To satisfy feature 1, you can have your package (at run-time, not at setup time!) import the _version file from myniftyapp/__init__.py like this:

from _version import __version__

Here is an example of this technique that I've been using for years.

The code in that example is a bit more complicated, but the simplified example that I wrote into this comment should be a complete implementation.

Here is example code of importing the version.

If you see anything wrong with this approach, please let me know.

Rewritten 2017-05

After more than ten year of writing Python code and managing various packages I came to the conclusion that DIY is maybe not the best approach.

I started using pbr package for dealing with versioning in my packages. If you are using git as your SCM, this will fit into your workflow like magic, saving your weeks of work (you will be surprised about how complex the issue can be).

As of today pbr is ranked #11 most used python package and reaching this level didn't include any dirty tricks: was only one: fixing a common packaging problem in a very simple way.

pbr can do more of the package maintenance burden, is not limited to versioning but it does not force you to adopt all its benefits.

So to give you an idea about how it looks to adopt pbr in one commit have a look swiching packaging to pbr

Probably you would observed that the version is not stored at all in the repository. PBR does detect it from Git branches and tags.

No need to worry about what happens when you do not have a git repository because pbr does "compile" and cache the version when you package or install the applications, so there is no runtime dependency on git.

Old solution

Here is the best solution I've seen so far and it also explains why:

Inside yourpackage/version.py:

# Store the version here so:
# 1) we don't load dependencies by storing it in __init__.py
# 2) we can import it in setup.py for the same reason
# 3) we can import it into your module module
__version__ = '0.12'

Inside yourpackage/__init__.py:

from .version import __version__

Inside setup.py:

exec(open('yourpackage/version.py').read())
setup(
    ...
    version=__version__,
    ...

If you know another approach that seems to be better let me know.

Per the deferred PEP 396 (Module Version Numbers), there is a proposed way to do this. It describes, with rationale, an (admittedly optional) standard for modules to follow. Here's a snippet:

3) When a module (or package) includes a version number, the version SHOULD be available in the __version__ attribute.

4) For modules which live inside a namespace package, the module SHOULD include the __version__ attribute. The namespace package itself SHOULD NOT include its own __version__ attribute.

5) The __version__ attribute's value SHOULD be a string.

Though this is probably far too late, there is a slightly simpler alternative to the previous answer:

__version_info__ = ('1', '2', '3')
__version__ = '.'.join(__version_info__)

(And it would be fairly simple to convert auto-incrementing portions of version numbers to a string using str().)

Of course, from what I've seen, people tend to use something like the previously-mentioned version when using __version_info__, and as such store it as a tuple of ints; however, I don't quite see the point in doing so, as I doubt there are situations where you would perform mathematical operations such as addition and subtraction on portions of version numbers for any purpose besides curiosity or auto-incrementation (and even then, int() and str() can be used fairly easily). (On the other hand, there is the possibility of someone else's code expecting a numerical tuple rather than a string tuple and thus failing.)

This is, of course, my own view, and I would gladly like others' input on using a numerical tuple.

As shezi reminded me, (lexical) comparisons of number strings do not necessarily have the same result as direct numerical comparisons; leading zeroes would be required to provide for that. So in the end, storing __version_info__ (or whatever it would be called) as a tuple of integer values would allow for more efficient version comparisons.

I use a JSON file in the package dir. This fits Zooko's requirements.

Inside pkg_dir/pkg_info.json:

{"version": "0.1.0"}

Inside setup.py:

from distutils.core import setup
import json

with open('pkg_dir/pkg_info.json') as fp:
    _info = json.load(fp)

setup(
    version=_info['version'],
    ...
    )

Inside pkg_dir/__init__.py:

import json
from os.path import dirname

with open(dirname(__file__) + '/pkg_info.json') as fp:
    _info = json.load(fp)

__version__ = _info['version']

I also put other information in pkg_info.json, like author. I like to use JSON because I can automate management of metadata.

Many of these solutions here ignore git version tags which still means you have to track version in multiple places (bad). I approached this with the following goals:

  • Derive all python version references from a tag in the git repo
  • Automate git tag/push and setup.py upload steps with a single command that takes no inputs.

How it works:

  1. From a make release command, the last tagged version in the git repo is found and incremented. The tag is pushed back to origin.

  2. The Makefile stores the version in src/_version.py where it will be read by setup.py and also included in the release. Do not check _version.py into source control!

  3. setup.py command reads the new version string from package.__version__.

Details:

Makefile

# remove optional 'v' and trailing hash "v1.0-N-HASH" -> "v1.0-N"
git_describe_ver = $(shell git describe --tags | sed -E -e 's/^v//' -e 's/(.*)-.*/\1/')
git_tag_ver      = $(shell git describe --abbrev=0)
next_patch_ver = $(shell python versionbump.py --patch $(call git_tag_ver))
next_minor_ver = $(shell python versionbump.py --minor $(call git_tag_ver))
next_major_ver = $(shell python versionbump.py --major $(call git_tag_ver))

.PHONY: ${MODULE}/_version.py
${MODULE}/_version.py:
    echo '__version__ = "$(call git_describe_ver)"' > $@

.PHONY: release
release: test lint mypy
    git tag -a $(call next_patch_ver)
    $(MAKE) ${MODULE}/_version.py
    python setup.py check sdist upload # (legacy "upload" method)
    # twine upload dist/*  (preferred method)
    git push origin master --tags

The release target always increments the 3rd version digit, but you can use the next_minor_ver or next_major_ver to increment the other digits. The commands rely on the versionbump.py script that is checked into the root of the repo

versionbump.py

"""An auto-increment tool for version strings."""

import sys
import unittest

import click
from click.testing import CliRunner  # type: ignore

__version__ = '0.1'

MIN_DIGITS = 2
MAX_DIGITS = 3


@click.command()
@click.argument('version')
@click.option('--major', 'bump_idx', flag_value=0, help='Increment major number.')
@click.option('--minor', 'bump_idx', flag_value=1, help='Increment minor number.')
@click.option('--patch', 'bump_idx', flag_value=2, default=True, help='Increment patch number.')
def cli(version: str, bump_idx: int) -> None:
    """Bumps a MAJOR.MINOR.PATCH version string at the specified index location or 'patch' digit. An
    optional 'v' prefix is allowed and will be included in the output if found."""
    prefix = version[0] if version[0].isalpha() else ''
    digits = version.lower().lstrip('v').split('.')

    if len(digits) > MAX_DIGITS:
        click.secho('ERROR: Too many digits', fg='red', err=True)
        sys.exit(1)

    digits = (digits + ['0'] * MAX_DIGITS)[:MAX_DIGITS]  # Extend total digits to max.
    digits[bump_idx] = str(int(digits[bump_idx]) + 1)  # Increment the desired digit.

    # Zero rightmost digits after bump position.
    for i in range(bump_idx + 1, MAX_DIGITS):
        digits[i] = '0'
    digits = digits[:max(MIN_DIGITS, bump_idx + 1)]  # Trim rightmost digits.
    click.echo(prefix + '.'.join(digits), nl=False)


if __name__ == '__main__':
    cli()  # pylint: disable=no-value-for-parameter

This does the heavy lifting how to process and increment the version number from git.

__init__.py

The my_module/_version.py file is imported into my_module/__init__.py. Put any static install config here that you want distributed with your module.

from ._version import __version__
__author__ = ''
__email__ = ''

setup.py

The last step is to read the version info from the my_module module.

from setuptools import setup, find_packages

pkg_vars  = {}

with open("{MODULE}/_version.py") as fp:
    exec(fp.read(), pkg_vars)

setup(
    version=pkg_vars['__version__'],
    ...
    ...
)

Of course, for all of this to work you'll have to have at least one version tag in your repo to start.

git tag -a v0.0.1

There doesn't seem to be a standard way to embed a version string in a python package. Most packages I've seen use some variant of your solution, i.e. eitner

  1. Embed the version in setup.py and have setup.py generate a module (e.g. version.py) containing only version info, that's imported by your package, or

  2. The reverse: put the version info in your package itself, and import that to set the version in setup.py

Also worth noting is that as well as __version__ being a semi-std. in python so is __version_info__ which is a tuple, in the simple cases you can just do something like:

__version__ = '1.2.3'
__version_info__ = tuple([ int(num) for num in __version__.split('.')])

...and you can get the __version__ string from a file, or whatever.

I also saw another style:

>>> django.VERSION
(1, 1, 0, 'final', 0)

arrow handles it in an interesting way.

In arrow/__init__.py:

__version__ = '0.8.0'
VERSION = __version__

In setup.py:

def grep(attrname):
    pattern = r"{0}\W*=\W*'([^']+)'".format(attrname)
    strval, = re.findall(pattern, file_text)
    return strval

setup(
    name='arrow',
    version=grep('__version__'),
    # [...]
)

For what it's worth, if you're using NumPy distutils, numpy.distutils.misc_util.Configuration has a make_svn_version_py() method that embeds the revision number inside package.__svn_version__ in the variable version .

  1. Use a version.py file only with __version__ = <VERSION> param in the file. In the setup.py file import the __version__ param and put it's value in the setup.py file like this: version=__version__
  2. Another way is to use just a setup.py file with version=<CURRENT_VERSION> - the CURRENT_VERSION is hardcoded.

Since we don't want to manually change the version in the file every time we create a new tag (ready to release a new package version), we can use the following..

I highly recommend bumpversion package. I've been using it for years to bump a version.

start by adding version=<VERSION> to your setup.py file if you don't have it already.

You should use a short script like this every time you bump a version:

bumpversion (patch|minor|major) - choose only one option
git push
git push --tags

Then add one file per repo called: .bumpversion.cfg:

[bumpversion]
current_version = <CURRENT_TAG>
commit = True
tag = True
tag_name = {new_version}
[bumpversion:file:<RELATIVE_PATH_TO_SETUP_FILE>]

Note:

  • You can use __version__ parameter under version.py file like it was suggested in other posts and update the bumpversion file like this: [bumpversion:file:<RELATIVE_PATH_TO_VERSION_FILE>]
  • You must git commit or git reset everything in your repo, otherwise you'll get a dirty repo error.
  • Make sure that your virtual environment includes the package of bumpversion, without it it will not work.

If you use CVS (or RCS) and want a quick solution, you can use:

__version__ = "$Revision: 1.1 $"[11:-2]
__version_info__ = tuple([int(s) for s in __version__.split(".")])

(Of course, the revision number will be substituted for you by CVS.)

This gives you a print-friendly version and a version info that you can use to check that the module you are importing has at least the expected version:

import my_module
assert my_module.__version_info__ >= (1, 1)
Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top