Separate DB connection for each forked child process using Doctrine

Question

I have a Symfony 2.2 application that has a Command that forks children to process entities in the DB. I'm having a hard time figuring out the proper way to force Doctrine to reconnect in each forked child process.

I finally got one solution working (I can see a new DB connection in my dev.log for each child) but I'm not sure if it's the best way to do it. I pass the container to each child who then creates a new connection and sets the default_connection service with that connection. But this seems a little messy. Any other thoughts on this?

$conn = $this->container->get('doctrine')->getConnection();
$conn2 = \Doctrine\DBAL\DriverManager::getConnection($conn->getParams(), $conn->getConfiguration(), $conn->getEventManager());
$this->container->set('doctrine.dbal.default_connection', $conn2);
$this->doctrine = $this->container->get('doctrine');
$this->doctrine->resetManager();

I don't like the idea of modifying the default_connection, even though this is done in a child process and won't affect the parent. Just seems unclean.

Answer 1

After some research I've figured this out on my own.

My approach above doesn't work fully and is more complicated than needed. The main thing to be aware of is what happens to your database connection when a child exits.

As most developers should know when you fork a process the child gets a copy of everything, including resources like DB connections. However, the tricky part is when a forked child exits it will close those resources. This means the child will close the same DB connection it shared from the parent.

When the child exits the parent will immediately have an invalid DB connection and you'll get an error like "Mysql server has gone away" when you try to do a query in the parent.

One solution to avoid this issue is to make sure the parent always reconnects to the DB right after it forks a child. This means if you fork 100 children your parent is going to have to reconnect 100 times. It's an unavoidable bit of overhead if your children and parent all need to interact with a DB.

So in my code I currently do something like the following right after I fork a child:

$conn = $this->getContainer()->get('doctrine')->getConnection();
$conn->close();
$conn->connect();

If you do a "show processlist" in your mysql client you'll now see multiple connections, one for each forked child.

I've even written a ProcessPool library on github that provides an easy to use API to create lots of worker processes that can return results back to the parent.

Answer 2

In my shell script, I used this before forking, to make sure all connections are closed.

// Close connections
$connections = $this->getContainer()
    ->get('doctrine')
    ->getConnections();
foreach($connections as $conn) {
    $this->logger->debug("Closing connection");
    $conn->close();
}