There are two XPath mistakes in your code, using /root/pubDate
instead of the (correct) current node. This fixed stylesheet provides the desired output:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:key name="years" match="/root/pubDate" use="substring-before(substring-after(substring-after(substring-after(., ' '), ' '), ' '), ' ')" />
<xsl:template match="root">
<rss>
<xsl:apply-templates mode="year" select="pubDate[
generate-id()
=
generate-id(key('years', substring-before(substring-after(substring-after(substring-after(., ' '), ' '), ' '), ' '))[1])
]"/>
</rss>
</xsl:template>
<xsl:template match="root/pubDate" mode="year">
<xsl:variable name="year" select="substring-before(substring-after(substring-after(substring-after(., ' '), ' '), ' '), ' ')"/>
<year handle="{$year}">
<xsl:apply-templates mode="final" select="key('years', $year)"/>
</year>
</xsl:template>
<xsl:template match="root/pubDate" mode="final">
<date>
<xsl:value-of select="." />
</date>
<title><xsl:value-of select="./following-sibling::*[1]" /></title>
</xsl:template>
</xsl:stylesheet>
There are shorter solutions as well. IMHO using different templates is no win in this case, since it doesn't make the code more readable (and re-usability of templates is not required). In addition, I would solve details differently. You might consider the following code, or maybe parts of it:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:key name="pubdates-by-year" match="/root/pubDate" use="substring-before(substring-after(substring-after(substring-after(., ' '), ' '), ' '), ' ')" />
<xsl:template match="root">
<rss>
<xsl:for-each select="pubDate[count(. | key('pubdates-by-year', substring-before(substring-after(substring-after(substring-after(., ' '), ' '), ' '), ' '))[1]) = 1]">
<xsl:variable name="year" select="substring-before(substring-after(substring-after(substring-after(., ' '), ' '), ' '), ' ')"/>
<year handle="{$year}">
<xsl:for-each select="key('pubdates-by-year', $year)">
<xsl:copy-of select="."/>
<xsl:copy-of select="following-sibling::title[1]"/>
</xsl:for-each>
</year>
</xsl:for-each>
</rss>
</xsl:template>
</xsl:stylesheet>