It turned out I was being an idiot. The site goal uses the plugin defintions declared in the reporting section of the pom. Therefore I had to add my max memory configuration here instead, see below:
...
<build>
...
<pluginManagement>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.8.1</version>
</plugin>
</plugins>
<pluginManagement>
...
</build>
...
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.8.1</version>
<configuration>
<maxmemory>512m</maxmemory>
</configuration>
</plugin>
</plugins>
</reporting>
...
Based on this result, it is my assumption that Maven will not pick up configuration details defined in the build-pluginManagement section of the pom for plugins defined in the reporting section. Cheers.