How to install a module which includes a different version of Symfony?

Question

I am a developer and maintainer for the CiviCRM project. We've been trying to make a Drupal 8 version CiviCRM, and have a come a long way. We're beating our heads against our collective keyboards trying to figure a major blocker for the project.

CiviCRM has used Symfony for a while, and the version that is included is different than what ships with Drupal.

We can get CiviCRM installed with Drupal 8, but after you install it, we cannot install any other Drupal module.

I believe it boils down to a situation where somehow the CiviCRM version of Symfony loads before the Drupal version, and this causes problems.

Does anyone know of a Drupal 8 module that includes a different version of Symfony than the one that ships with Drupal?

Recently I ran across the Ludwig project. This module allows the registering of namespaces in a class that extends ServiceProviderBase.

Would it be possible for the Drupal 8 version of the CiviCRM module to include a CivicrmServiceProvider.php file, which defines a CivicrmServiceProvider class, and a register() method that adds a container namespace to allow this to work?

Many CiviCRM files have use statements like Drupal starting with Symfony, like here.

We actually put CiviCRM Core into Drupal doc_root/libraries folder, and use the libraries module.

This is the repo for the CiviCRM Drupal module 8.x version, if somebody wants to look at what we got so far. If somebody has the magic elixir for this, I can tell you there would be many happy folks in our community. So if you know how to help us, please do.

CiviCRM does install, and the CiviCRM pages do work. What doesn't work is that after CiviCRM is installed, we can't install other modules via the admin/modules page. As far as I know that's the only thing that is broken. Also installing modules with Drush, after installing CiviCRM, does work.

Trying to install another module after CiviCRM is installed causes the following error:

PHP Fatal error: Call to undefined method Symfony\Component\DependencyInjection\Definition::setFactory() in /var/www/html/civi-for-d8/core/lib/Drupal/Core/DependencyInjection/YamlFileLoader.php on line 206

That's in Drupal 8.3.5. Trying to install CiviCRM for Drupal 8 into a clean Drupal 8.4-dev instance causes the following error:

Drupal\Component\Serialization\Exception\InvalidDataTypeException: The reserved indicator "@" cannot start a plain scalar; you need to quote the scalar at line 8 (near "arguments: [@string_translation, @civicrm.page_state]"). in Drupal\Component\Serialization\YamlSymfony::decode() (line 40 of /var/www/html/drupal84/core/lib/Drupal/Component/Serialization/YamlSymfony.php).

Answer 1

So, I think if CiviCRM were installed into Drupal 8 via composer (ie. composer require civicrm/civicrm-core in the Drupal root) and CiviCRM's use of Symfony was compatible with Symfony 2.8 or 3.x (ie. not using deprecated functionality), this could work.

This would get everything installed in Drupal's vendor directory, rather than having two, and it'd mean that CiviCRM would use the Symfony version in Drupal 8. But if CiviCRM was compatible with later Symfony versions (even if it bundled an older version for Drupal 6 & 7 and other CMS's) it should be fine.

I think?

UPDATED: Yes, it works - I tried it. :-) I originally posted the below in the CiviCRM issue queue (CRM-17652), but re-posting here for completeness.

The big idea:

Since composer is pretty new to a lot of people, I'm going to attempt to go step by step, from some high-level composer stuff all the way to one way it could be done in CiviCRM:

• Composer allows applications to require the libraries it needs (and libraries, of course, can require other libraries).
• Libraries have a composer.json file which says what other libraries it needs and what versions it's compatible with (but not necessarily a specific single version - usually a range of versions, like ^2.4.3 which says a minimum of 2.4.3 and up to (but not including) 3.0.0)
• Applications have a composer.json which similarly describes needed libraries and compatibility with a range of versions, but the range is really to help with updating. An application will also have a composer.lock which is a specific set of individual versions
• Libraries can also have a composer.lock for their own testing or distribution (like building the release tarball with the dependencies bundled), but this is ignored when an application requires the given library (see https://getcomposer.org/doc/02-libraries.md#lock-file)
• When an application wants to require a new library, composer finds an intersection of version compatibility between all the things the application requires (including all the libraries already installed and their dependencies) and the new library, possibly doing some updates to make everything line up (or erroring out if it can't find a compatible mix of versions)
• In this case, CiviCRM is a library, and a particular Drupal 8 site is the application (Drupal core itself is a library)
• CiviCRM could say it "requires" Symfony ^2.5 in its composer.json which means it is compatible with versions 2.5.0 up to (but not including) 3.0.0
• When a Drupal 8 site wants to use CiviCRM, the site admin uses composer require civicrm/civicrm-core to require the CiviCRM library and all it's dependencies. If CiviCRM is compatible with Symfony 2.8 (like used in Drupal 8.3.x) everything will install and work fine, using the single Symfony 2.8 from Drupal. All the dependencies end up in Drupal's vendor directory.
• However, CiviCRM could keep Symfony 2.5 in its composer.lock, which means the tests would use that, and the tarballs for Drupal 6 & 7 and other CMS's would bundle Symfony 2.5

The proposal:

1. Update the composer.json of CiviCRM so it can be used as a library by composer-based CMS's like Drupal 8 (but probably others could move that way in the future - composer is getting quite popular)
2. Make sure CiviCRM core is compatible with Symfony 2.8 and 3.0 (used by Drupal 8.3.x and 8.4.x respectively) but keep the "officially supported" version (currently Symfony 2.5) in the composer.lock for testing and the tarball for distribution. Being compatible with multiple Symfony versions may not be as hard as it sounds - there's a number of libraries out there compatible with both Symfony 2.8 and 3.0. It may just be a matter of avoiding deprecated methods/classes/features! The composer.json will need to be updated to reflect this
3. Use composer to install the CiviCRM library on Drupal 8 rather than copying into the libraries directory. This is becoming the normal way of installing 3rd party PHP libraries in Drupal 8 (this is used extensively by Drupal Commerce, for example)

For composer-based CMS's, I really think this is The Right Way. While this issue is currently affecting Symfony and Drupal, as the PHP community starts using more and more 3rd party libraries via composer, this could very well affect other CMS's with other version conflicts.

Some working code to test:

So, as promised, I actually got this to work to a limited degree :-) I'm totally coming at this from a Drupal/Composer/Symfony perspective -- I don't have a ton of CiviCRM experience, so there's probably some better ways to do my process below. I welcome any advice!

1. Download and install Drupal 8.3.5 (or the latest dev of Drupal 8.4.x!)
2. Go into the root directory in the shell and run these commands to install CiviCRM via composer: https://gist.github.com/dsnopek/56311dbea347874e75180883efabb620
3. If you use Apache, remove the vendor/.htacess file. This is a security measure from Drupal, which prevents resources like CSS/JS being loaded. This will need some collaboration with the Drupal project to figure out a proper solution because removing this file altogether is a bad idea on production. See: vendor/.htaccess blocking CSS/JS assets from composer libraries.
4. Go into the /modules directory and do git clone https://github.com/dsnopek/civicrm-drupal.git --branch composer-library
5. Go to the "Extend" page (/admin/modules) and install the CiviCRM module
6. Clear drupal cache via Drush (drush cr)
7. Log out and log back in again per CRM-19878
8. CiviCRM works! :-)

After all of this, CiviCRM is using Symfony 2.8 from Drupal and the dependencies in Drupal's vendor directory, and isn't loading anything from it's own vendor directory. Huzzah!

I tested enabling the "Telephone" module which failed before these changes (see my steps to reproduce), but works fine with them. :-)

Answer 2

I don't think this is possible.

Drupal 8.4 actually already switched to Symfony 3 although there are still similar discussions related to drush, which has the same problem. see Drush 8.x doesn't install Drupal 8.4.x and Drush master doesn't install Drupal 8.3.x and Symfony components are updated to 3.2.6

It's not possible to load two different symfony versions, either you break your integration or you break Drupal. Maybe symfony3 will not be in 8.4 yet, but the security support for symfony2 will end before Drupal8 security support, so at some point, we will have to switch.

Answer 3

Theoretically the only issues here are the file location and class namespace. Unfortunately the only tools I know of in composer for doing that don't let you specify per VERSION, just per package name.

Have you tried setting it up as a completely separate autoloader?