Could Free Pascal benefit of something like Apache Maven?

Question

Apache Maven is a very popular build and dependency management tool in the Java open source ecosphere. I did some tests to find out if it can handle compiled Free Pascal / Delphi units and found it easy to implement. So it would be possible to

• release open source libraries precompiled for Free Pascal (or Delphi) in a public Maven repository
• include metadata in this repository which contains dependency information
• use Maven on the command line to download the open source library from the public repository, and automatically resolve all dependencies
• local repositories, working as proxies, could be used to cache frequently used binaries
• automatic checksum generation and verification (provided by Maven) would reduce the risk of downloading corrupted binaries
• source code and even documentation files could be provided with the binaries
• binaries can be provided with or without debug information
• continuous integration servers like Hudson, TeamCity or CruiseControl can be used to build projects whenever changes have been submitted to the source control system and notify developers about build errors

This way of dependency management could be very beneficial for open source projects which use many third party libraries with complex dependencies. It would avoid typical conflicts caused by using wrong versions.

For the developer, the workflow for editing and building a project would be reduced to a minimum:

• checkout the project source from internal version control system
• edit source file(s)
• run mvn package to automatically download all required third party libraries (precompiled units) if they are not yet in the workstation's local repository
• compile and run

The only additional file for Apache Maven which is required in the project folder is the POM.XML file containing the project information.

Edit: while Maven is usable for some of the required tasks, implementing a solution like Maven in native Free Pascal would have some advantages: no Java SDK required, support for all development platforms where Free Pascal is available, maintenance and plugin development in Pascal.

Usage of a Maven-like tool would not be helpful for open source projects only - commercial projects could access and use the artifacts in public Maven repositories in the same way as well.

Maven features are listed at http://maven.apache.org/maven-features.html

---

Update:

one use case could be the build of Lazarus, where Maven would download all required libraries and invoke the compiler with the necessary build path arguments. Changes in the dependencies on lower levels would be propagated automatically up to the parent build.

Possible benefits:

• less time needed to set up a new work station, no manual installation of third party libraries required
• less errors caused by wrong library versions, detection of version conflicts (for example if two libraries depend on different versions of a third library)
• artifacts which are created inhouse can be added to the local maven repository and shared between developers and project, central storage of all artifacts with metadata
• builds are reproducible, just by using the same source and project metadata file (pom.xml)
• can reduce development time and increase project stability

Update #2: FPMake

the FPMake build system for Free Pascal seems to be a tool with much potential, in many details it is quite similar to Maven:

• FPMake is a pascal based build system developed for and distributed with FPC
• FPMake standardizes the building by defining some limits like standard directories
• the command fppkg <packagename> will look in a database for the package, extract it, and then compile fpmake.pp and run it
• it has standard build targets (clean, build, install, ...)
• it can create a 'manifest' file suitable for import into a repository (like mvn deploy or mvn install), the manifest is an XML file which looks very similar to a pom.xml in Maven:

FPMake manifest file:

      <packages>
        <package name="my-package">
          <version major="0" minor="7" micro="6" build="1"/>
          <filename>my-package-0.7.6-1.zip</filename>
          <author>my name</author>
          <license>GPL</license>
          <homepageurl>http://www.freepascal.org/</homepageurl>
          <email>myname@freepascal.org</email>
          <description>this is the package description</description>
          <dependencies>
            <dependency>
              <package packagename="rtl"/>
            </dependency>
          </dependencies>
        </package>    
      </packages>

Answer 1

Freepascal has been working on a package system of its own in a cross between apt-get and freebsd ports style. (download source/build/install automatically), called fppkg. However work has stalled. People investing time are the bottleneck, not people wanting to choose tools.

As far as Maven goes, I don't like auxilary tools that need installation of huge external runtimes. It might be fine for a big major app (like Open Office), but not for an util.

I also prefer a tool that is designed to the FPC reality and workflow. Documentation tools, build tools, download systems, testsuite systems are already all there, it just need a person that dedicates a lot of time into it to make it happen.

Some typical problems when introducing a new technology in a project as FPC, and why it has a tendency to make its own tools:

• need to train 20+ committers in parttime.
• The only COMMON programming language you can assume is Free Pascal. Even Delphi inner workings can't be taken for granted to be known (many committers came directly to FPC or even still via TP or a Mac Pascal)
  • Obviously that makes something with plugins in a different language annoying.
• Bash script is a close second. (g)make third, but already a magnitude less.
• All servers are *nix-like (FreeBSD, OS X, Linux), but not all run Apache. (e.g. my FreeBSD mirror runs XSHTTPD)
• somebody most knowledgable must be dedicated maintainer for a long time. Fix problems, update/ do migrations etc. Perferably more than one for obvious reasons.
• a major pain are Linux distributions (and FreeBSD to a lesser degree), most maintainers of *nix packages are not capable of more than "./configure;make;make install", and must be spoonfed with a near buildable repository and auxilary files.
  • In-distribution packaging of FPC/Lazarus has always been important, and is still increasing
  • All distributions have their own special rules about metadata, depedancies, and how sources must be published. Particularly Debian/Ubuntu is very bureaucratic and slow.
  • Most don't like third party auto-installers on top of their systems (since that bypasses their dependancy control)

This all leads to the effective practice that own tools in Pascal with minimal scripting work best. Some tools used:

• Gmake is mainly used to parameterise the build process on a per directory level, a successor, fpcmake (not really a make derivative despite the name) has begun, but the migration hasn't completed.
• Latex and a latex to html conversion (tex4ht, but debian uses hevea) are used in the documentation building (the non library documentation)
• The community site (netscape community server which uses TCL scripting, a heavy complex application server) has been a trouble ever since it started, but specially lately since the maintainer became less active.
• Mantis has been a problem (specially the email module would crash or lame the server due to the volume), but it has been whipped into shape during successive updates and hard work of several lazarus devels. Currently it is a decent workhorse.
• lazarus.freepascal.org PHPBB forum OTOH is relatively painless since a lot of younger people know how to deal with it.
• The same goes for subversions (though the more advanced scale needs some adjusting, not everybody is deep into the ins and outs of mergetracking)

If somebody was really serious about Maven, I usually would ask him:

• to CRITICIALLY investigate the use for the project. In a very concrete way, with schedule and time estimates. Birds-eye level "everything's possible" overviews are essentialy worthless.
• Give some thought on future change of used technologies. Every technology is eventually replaced, even the in-house ones, in 18 year+ projects. A new technology must not make migrations of other infrastructural components hard or involved. The new technology to end all new technologies doesn't exist.
• Make a migration plan. Migration is often underrated and underestimated.
• And in the end, there is always the 1000000 Euro question, who will do the daily maintenance?

Keep in mind that in a company you just kick the person responsible for the application server. But in an informal environment this is way harder, specially long term, since people's lives, occupations and time spent on the project vary.

Answer 2

Sounds like an interesting plan, but the Delphi community (and FPC even more so, I'd imagine!) values libraries as source far more than precompiled libraries. The general consensus is that anyone who uses a binary-only library is a fool, for two reasons: You can't fix any bugs you find in it, and compiler changes will break compatibility.