How to get attributes of an attribute set that are not present in default attribute set

Question

How can I get the list of attributes of an attribute set that are not present in the default attribute set?

I tried the following codes:

$attributeSetId = Mage::getModel('eav/entity_attribute_set')
            ->load($_product->getAttributeSetId())->getId();
$attributes = Mage::getModel('catalog/product_attribute_api')
            ->items($attributeSetId);

echo '<pre>';
print_r($attributes);
die();

But this returned an array with all the attributes including the attributes of default attribute set whereas I just needed the attributes that only belonged to my custom attribute set.