According to the question's update, it seems the first loop displays all categories in portfolio_categories
from a custom query, most probably including those from category 36 also.
The only way I can think off to not repeat the posts in the second loop is to exclude all category 36 posts from the first loop.
I can't test the code, but here is an idea of how to do it:
Add 3 lines of code after the Loop, like this:
if (have_posts()) : while (have_posts()) : the_post(); // This is the Loop
$Category = get_the_category( $post->ID );
$CatID = $Category[0]->cat_ID ; // The 0 assumes the post has only one category. If there are more, the number must be changed accordingly.
if ($CatID == 36) continue;