How to structure project where a library and application using the library are concurrently developed?

Question

I am planning to develop a new application that will heavily use a library that will be developed from scratch specifically for the application, but made general enough for it to be used for other programs once it's finished.

So the development of this library is going to be heavily driven by the development of the application. It'll get features based on how important that feature is to the paired application. All the same, I want the library to be completely decoupled from the application. The application uses the library, but the library is unaware of the application and at some point of time, I'll want to ensure that the library can easily be used in other applications.

I'm looking for advice on how to structure this. In particular, where the code for the library should be located (same/different repo, some kind of build tool to keep them separate, etc).

Since this may be language dependent (as a result of build tools and other language differences), I am planning to use Python.

Answer 1

First, if your library is generic, it should NOT be influenced by the application. It should provide the functionality that it is designed to provide. Application is based on the library, not the other way around.

Whether you keep it the same repository or different one, doesn't matter - have it a separate project though.

I would do a technical design first and define all the required functionality. Then I would separate the functionality into presentation layer, business layer and data persistent layer. Then I would implement it starting from the bottom (persistence layer).

Answer 2

I'll start with the generic language neutral stuff first, and then I'll cover the python specific bits.

Why are you so sure that there needs to be a seperate library? Is it because different developers working on the code? or are there plans to reuse the library in additional applications? or even to distribute the library on its own to customers.

In general, if your library has its own independent life, then you need to consider how you are going to manage the different release life cycle, and making it a seperate repository would help with that, at the cost of increasing the complexity for the developers working on both projects as part of the same development task. Which way you decide on this will have long term impact on your team, and will have wide reaching consequences in how you optimise your teams work patterns and development practices. both ways can work if implemented well, and both can fail badly if implemented and managed poorly.

With Python, best practice is to release your components as python packages. Whilst it is possible to have multiple packages in the same repository, I probably would push hard for a different repository per package, and then to have multiple python virtual environments. The development environment would have the packages installed using the source files in place, with the release environment would be created by installing the packages as finished packages. Switching between these environments allows easy testing of all possible comnbinations. This is the pythonic way.

Answer 3

It is not a matter of programming language.

If you'll write your library with 'this is general purpose library' in mind you can be successful. But, if I were your I'd start development of both in the same place, in the same repository. Later you can extract this library into another project.