What is the right way to create a build system?

Question

I'm a student, and I don't have much expirience with the building process. In the last few days i've read something about CMake and others build system generators, but other than that I don't know anything about this world.

I realized that i should learn something about this topic because I wanted to use a third party library in a project and I spent a lot of time just to be able to build something. In particular I wanted to try the telegram database library (https://github.com/tdlib/td) and, having followed the installation instructions (I'm using Windows and writing in c++), I got a folder without knowing exactly what to do next. After having learnt a little bit of Cmake, I managed to get the lib, include and bin folders, to copy them in my project and to build everything.

Is this the standard process to integrate an external library in a project? Do I really have to install separately all third party libraries and then integrate them in a single project?

The other solution I can think of is to include the source code of the external libraries in the project and to have a top CMakeLists.txt file that takes care of reaching the CMakeLists.txt files of all the libraries. The problem with this is that, for example, some third party libraries don't give away their source code.

On the other hand, I find the first solution a bit ugly: what if an external library (for example the telegram database library) depends on some other external libraries (in this case zlib, openssl...)? How can i know in advance that a certain lib has to be linked to another specific lib if I don't have a CMakeLists.txt file?

Is there another way / another tool to manage the build system without having to manually write some scripts that take care of all the dependencies?

Answer 1

Disclaimer: I know practically nothing about CMake; therefore, my answer would be about the general build systems instead of CMake specifically.

The goal of a build is to take the source code and to create something which is ready to be deployed or shipped. This means two things:

• There should ideally be no prerequisites on the machine. In other words, you should be able to run a build on a freshly installed OS.

• It should be possible to rebuild a previous version of a system at any moment.

This means that, when it comes to the dependencies:

• Dependencies should automatically be retrieved from somewhere and installed. The source could be virtually anything; for instance Python uses pypi package index and .NET uses NuGet, while Node.js relies on npm, but can also install packages directly from GitHub. apt-get and yum are two other great examples of package management: if you install an application, its dependencies are installed automatically. In most cases, it is possible to have your own index in order to use your personal (or corporate) projects as a dependency.

• Package indexes should give access to all the published versions, not just the latest one.

This is all good in theory, but quite difficult to do properly in practice:

• Where do dependencies stop? Is the operating system itself a dependency? What about configuration? API secret keys? APIs themselves? There are lots of things the build is unable to do (for instance, change your hardware) or is not expected to do (you won't be happy if it re-installs your operating system, wiping out all your data, would you?)

• Circumstances may require the author of a library you use to remove some of the older versions of the library. In many cases, the author retains the intellectual property on the published library, and can remove the whole library without asking your permission.

Do I really have to install separately all third party libraries and then integrate them in a single project?

No, you don't. For CMake, it seems that you need to use find_package. When it comes to package indexes similar to pypi or npm, things get more complex. If I understand it correctly, it's not a job of CMake to go get your dependencies from the Internet; rather, you should be using a C++ package manager. There is a large choice of them.

Easy dependency management for C++ with CMake and Git may also be relevant, and the screenshot from reddit at the beginning of the article seems to be quite illustrative.

The problem with this is that, for example, some third party libraries don't give away their source code.

If they give just the the header files and a binary which is compatible with your system, that's all you need. This being said, having source code and the CMakeLists.txt of the dependency makes it much easier to work with.

How can i know in advance that a certain lib has to be linked to another specific lib if I don't have a CMakeLists.txt file?

The problem is solved for apt-get, pypi, npm or NuGet packages by shipping the list of dependencies with every package. So when you build your own app which uses a dependency, the build process looks at the dependencies of the dependency, and so on, including all the dependency tree.

As I understand it, C++ libraries should be either shipped as a apt-get/yum package, or should provide their own CMakeLists.txt file. If they don't do either, this may be a sign that the library is not mature enough.