Deploying a python application with shared package
-
11-07-2019 - |
Question
I'm thinking how to arrange a deployed python application which will have a
- Executable script located in /usr/bin/ which will provide a CLI to functionality implemented in
- A library installed to wherever the current site-packages directory is.
Now, currently, I have the following directory structure in my sources:
foo.py
foo/
__init__.py
...
which I guess is not the best way to do things. During development, everything works as expected, however when deployed, the "from foo import FooObject" code in foo.py seemingly attempts to import foo.py itself, which is not the behaviour I'm looking for.
So the question is what is the standard practice of orchestrating situations like this? One of the things I could think of is, when installing, rename foo.py to just foo, which stops it from importing itself, but that seems rather awkward...
Another part of the problem, I suppose, is that it's a naming challenge. Perhaps call the executable script foo-bin.py?
Solution
This article is pretty good, and shows you a good way to do it. The second item from the Do list answers your question.
shameless copy paste:
Filesystem structure of a Python project
by Jp Calderone
Do:
- name the directory something related to your project. For example, if your project is named "Twisted", name the top-level directory for its source files
Twisted
. When you do releases, you should include a version number suffix:Twisted-2.5
.- create a directory
Twisted/bin
and put your executables there, if you have any. Don't give them a.py
extension, even if they are Python source files. Don't put any code in them except an import of and call to a main function defined somewhere else in your projects.- If your project is expressable as a single Python source file, then put it into the directory and name it something related to your project. For example,
Twisted/twisted.py
. If you need multiple source files, create a package instead (Twisted/twisted/
, with an emptyTwisted/twisted/__init__.py
) and place your source files in it. For example,Twisted/twisted/internet.py
.- put your unit tests in a sub-package of your package (note - this means that the single Python source file option above was a trick - you always need at least one other file for your unit tests). For example,
Twisted/twisted/test/
. Of course, make it a package withTwisted/twisted/test/__init__.py
. Place tests in files likeTwisted/twisted/test/test_internet.py
.- add
Twisted/README
andTwisted/setup.py
to explain and install your software, respectively, if you're feeling nice.Don't:
- put your source in a directory called
src
orlib
. This makes it hard to run without installing.- put your tests outside of your Python package. This makes it hard to run the tests against an installed version.
- create a package that only has a
__init__.py
and then put all your code into__init__.py
. Just make a module instead of a package, it's simpler.- try to come up with magical hacks to make Python able to import your module or package without having the user add the directory containing it to their import path (either via
PYTHONPATH
or some other mechanism). You will not correctly handle all cases and users will get angry at you when your software doesn't work in their environment.
OTHER TIPS
Distutils supports installing modules, packages, and scripts. If you create a distutils setup.py
which refers to foo
as a package and foo.py
as a script, then foo.py
should get installed to /usr/local/bin
or whatever the appropriate script install path is on the target OS, and the foo
package should get installed to the site_packages
directory.
You should call the executable just foo
, not foo.py
, then attempts to import foo will not use it.
As for naming it properly: this is difficult to answer in the abstract; we would need to know what specifically it does. For example, if it configures and controls, calling it -config or ctl might be appropriate. If it is a shell API for the library, it should have the same name as the library.
Your CLI module is one thing, the package that supports it is another thing. Don't confuse the names withe module foo
(in a file foo.py
) and the package foo
(in a directory foo
with a file __init__.py
).
You have two things named foo
: a module and a package. What else do you want to name foo
? A class? A function? A variable?
Pick a distinctive name for the foo module or the foo package. foolib
, for example, is a popular package name.