I believe I have solved it. In the attribute-map.xml I had to add:
<Attribute name="xxxx" id="xxxx">
<AttributeDecoder xsi:type="NameIDAttributeDecoder" formatter="$Name" defaultQualifiers="true"/>
</Attribute>
<Attribute name="yyyy" id="yyyy">
<AttributeDecoder xsi:type="NameIDAttributeDecoder" formatter="$Name" defaultQualifiers="true"/>
</Attribute>
See here: NativeSPAttributeDecoder