This is an example of a "set-within-sets" query. You can construct it for a specific set of users as:
select cu.conversation_id
from conversation_users cu
group by cu.conversation_id
having sum(cu.user = x) > 0 and
sum(cu.user = y) > 0 and
sum(cu.user = z) > 0 and
sum(cu.user not in (x, y, z)) = 0;
Each condition in the having
clause (except the last) checks that a given user is in the conversation. The last checks that no other users are in it.
If you store the users you care about in a temporary table or string (in the horrible comma-separated format), you can still do this:
select cu.conversation_id
from conversation_users cu
group by cu.conversation_id
having count(distinct (case when find_in_set(cu.user, @USERLIST) > 0 then cu.user end)) =
1 + length(@USERLIST) - length(replace(@USERLIST, ',', '') and
count(distinct cu.user_id) = 1 + length(@USERLIST) - length(replace(@USERLIST, ',', '')
What this says is that the number of matching users in the list is equal to the size of the list, and the total number of matching users is the size of the list. These two conditions say that all members of the list are in the conversation.