setup.py, pip - get origin path of execution of python package

Question

I have a python package that I need to install via pip as part of my project's requirements file, but I need to copy some files from my package to the projects installing this package. I'm thinking of doing it using the data_files option in setup() like:

setup(
    name='my_pacakge',
    ...
    data_files=[
        ('example/folder1', ['something1/file1.ext', 'something1/file2.ext']),
        ('folder2', ['something2/file1.ext', 'something2/file2.ext']),
    ]
)

and I woud like these files to be copied into paths relative to the directory where pip install my_package was run from, ideally my project root, leaving a tree as follows

project_root
├── example
│   └── folder1
│       ├── file1.ext
│       └── file2.ext
└── folder2
    ├── file1.ext
    └── file2.ext

but for this I will need to know the absolute path of where pip install my_package was run from. I've tried with inspect.stack, inspect.currentframe, sys.get_frame, inspect.inspect.getouterframes(some_of_these_frames) but this only gives me information about the current location of setup.py at installation time, which is directories like

/private/var/folders/gq/vpylvs0d51162jtyqhtgdc6m0000gn/T/pip-tNcbvb-build
/var/folders/gq/vpylvs0d51162jtyqhtgdc6m0000gn/T/pip-6a8MKS-build/setup.py

which is not useful at all for my purpose.

Now, I'm aware setuptools/setup.py should no be used like this but I really need to pack some files in the installable package and copy/move them to the project root at installation time.

Any suggestions on how to accomplish this?

Thanks! :)

Answer 1

Instead of trying to coerce setuptools to do your bootstrapping, consider tools like cookiecutter which are designed to handle other bootstrapping and setup tasks for a variety of projects.

There are templates available for django as well (see here for example).

Answer 2

use pkg_resources

This package gets installed with setuptools.

From package docstring:

Docstring:
Package resource API
--------------------

A resource is a logical file contained within a package, or a logical
subdirectory thereof.  The package resource API expects resource names
to have their path parts separated with ``/``, *not* whatever the local
path separator is.  Do not use os.path operations to manipulate resource
names being passed into the API.

The package resource API is designed to work with normal filesystem packages,
.egg files, and unpacked .egg files.  It can also work in a limited way with
.zip files and with custom PEP 302 loaders that support the ``get_data()``
method.

In case, you install your package with zip_safe = False, your data will be available in directory.

Installing with zip_safe = True, it will be part of egg. But if you need to use the files from there, it is very easy and pkg_resources provides nice set of functions and methods

• pkg_resources.resource_exists
• pkg_resources.resource_isdir
• pkg_resources.resource_stream - provides file-like object
• pkg_resources.resource_filename
• pkg_resources.resource_listdir
• pkg_resources.resource_string

Note, that this works even for zipsafe True installations.

Just put your data into directory relative to your package direcotry. And then from your code:

import pkg_resources
fname = pkg_resources.resource_filename("your.package.name", "folder/example1.txt")

use it only for statical files

Do not attempt to have there any files, which are changing. Technically it could be possible (with zip_safe = False the files are just there), but it is definitely bad design (permission, updates of packages).

Do not init your project from pip, better provide some appinit command

Using pip to install your data into application directory is real abuse.

I would recommend to define a command installed by our program, like appinit, and use it whenever you need to populate your directory. Commands like buildout (installed by zc.buildout) and others are often using this patter.