As mentioned in the comments, if you are using XSLT 1.0 then the xsl:for-each-group command is not available. In XSLT 1.0, grouping is usually done using a technique called Muenchian Grouping. It is worth reading it, and understanding it, as it is a very useful technique in XSLT 1.0 once you understand it.
In Muenchian Grouping, you start off by defining a key which will be used to look up the elements in your group. In your case, you are grouping by Category and Code together, and so they key would look like this:
<xsl:key name="GroupByCategoryCode" match="Book" use="concat(Category, '|', @code)"/>
Now, in XSLT 2.0, you would probably write this...
<xsl:for-each-group select="Library/Book" group-by="concat(Category, '|', @code)">
However, in XSLT 1.0 you have to write this (I've added lots of indentation to improve readability)
<xsl:for-each
select="Library/Book
[
generate-id() =
generate-id
(
key('GroupByCategoryCode', concat(Category, '|', @code))[1]
)
]">
(Using xsl:apply-templates here would also work).
What this is going is looking at all Book elements, and checking its combination of "Category" and "Code" to see whether it is the first occurrence of that element in the key. Effectively it will pick the distinct occurences of "Category" and "Code".
In XSLT 2.0, you would use "current-group" to then iterate over all elements in the group. In XSLT 1.0, you would just use the key
<xsl:for-each select="key('GroupByCategoryCode', concat(Category, '|', @code))">
Although, in this particular case, you don't need to do a for-each, as you are just summing up Quantity in each group.
<xsl:value-of
select="sum(key('GroupByCategoryCode', concat(Category, '|', @code))/Quantity)"/>
Or, to improve readability abit...
<xsl:variable name="currentGroup" select="key('GroupByCategoryCode', concat(Category, '|', @code))"/>
<xsl:value-of select="sum($currentGroup/Quantity)"/>
Try this XSLT. To keep things simple (and because I don't know xsl-fo) I am outputting HTML to demonstrate the principle. All you need to do is to replace the HTML tags with their xsl-fo equivalents
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:key name="GroupByCategoryCode" match="Book" use="concat(Category, '|', @code)"/>
<xsl:template match="/">
<html>
<body>
<h1>Books Information</h1>
<table border="1">
<tr>
<th>Category</th>
<th>Book Code</th>
<th>Quantity</th>
<th>Unit Price</th>
<th>Price</th>
</tr>
<xsl:apply-templates select="Library/Book[generate-id() = generate-id(key('GroupByCategoryCode', concat(Category, '|', @code))[1])]">
<xsl:sort select="Category"/>
<xsl:sort select="@code"/>
</xsl:apply-templates>
</table>
</body>
</html>
</xsl:template>
<xsl:template match="Book">
<xsl:variable name="currentGroup" select="key('GroupByCategoryCode', concat(Category, '|', @code))"/>
<tr>
<td>
<xsl:value-of select="Category"/>
</td>
<td>
<xsl:value-of select="@code"/>
</td>
<td>
<xsl:value-of select="sum($currentGroup/Quantity)"/>
</td>
<td>
<xsl:value-of select="Price"/>
</td>
<td>
<xsl:value-of select="sum($currentGroup/Quantity) * Price"/>
</td>
</tr>
</xsl:template>
</xsl:stylesheet>
EDIT: To show the word 'Repeated' instead of the Category for repeated categories, you could view this as grouping by category, and only show the category name for the first one in the group.
So, add the following key to the XSLT, to allow you to look up books by category
<xsl:key name="GroupByCategory" match="Book" use="Category"/>
Then, you need to pick the distinct categories (this should surround the existing xsl:apply-templates)
<xsl:for-each select="Library/Book[generate-id() = generate-id(key('GroupByCategory', Category)[1])]">
Then within the template matching Book you could then determine whether to show the category name or "defined" like so
<xsl:choose>
<xsl:when test="position() = 1">
<xsl:value-of select="Category" />
</xsl:when>
<xsl:otherwise>
<xsl:text>Repeated</xsl:text>
</xsl:otherwise>
</xsl:choose>
Try this XSLT
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:key name="GroupByCategoryCode" match="Book" use="concat(Category, '|', @code)"/>
<xsl:key name="GroupByCategory" match="Book" use="Category"/>
<xsl:template match="/">
<html>
<body>
<h1>Books Information</h1>
<table border="1">
<tr>
<th>Category</th>
<th>Book Code</th>
<th>Quantity</th>
<th>Unit Price</th>
<th>Price</th>
</tr>
<xsl:for-each select="Library/Book[generate-id() = generate-id(key('GroupByCategory', Category)[1])]">
<xsl:sort select="Category"/>
<xsl:apply-templates select="key('GroupByCategory', Category)[generate-id() = generate-id(key('GroupByCategoryCode', concat(Category, '|', @code))[1])]">
<xsl:sort select="@code"/>
</xsl:apply-templates>
</xsl:for-each>
</table>
</body>
</html>
</xsl:template>
<xsl:template match="Book">
<xsl:variable name="Category">
<xsl:choose>
<xsl:when test="position() = 1">
<xsl:value-of select="Category" />
</xsl:when>
<xsl:otherwise>
<xsl:text>Repeated</xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:variable>
<xsl:variable name="currentGroup" select="key('GroupByCategoryCode', concat(Category, '|', @code))"/>
<tr>
<td>
<xsl:value-of select="$Category"/>
</td>
<td>
<xsl:value-of select="@code"/>
</td>
<td>
<xsl:value-of select="sum($currentGroup/Quantity)"/>
</td>
<td>
<xsl:value-of select="Price"/>
</td>
<td>
<xsl:value-of select="sum($currentGroup/Quantity) * Price"/>
</td>
</tr>
</xsl:template>
</xsl:stylesheet>