Presumably, you want people who have taken course 1 but not course 4. You can do this with an aggregation query and a having
clause. The following query returns the learners who have taken one class but not the other:
SELECT ltc.person_id
FROM learners_to_classes AS ltc INNER JOIN
classes c
ON ltc.class_id = c.class_id
WHERE c.deleted = 0
GROUP BY ltc.person_id
HAVING sum(c.course_id = 1) = 1 and
sum(c.course_id = 4) = 0;
If you want just the number of such people, use this as a subquery:
SELECT COUNT(*)
FROM (SELECT ltc.person_id
FROM learners_to_classes AS ltc INNER JOIN
classes c
ON ltc.class_id = c.class_id
WHERE c.deleted = 0
GROUP BY ltc.person_id
HAVING sum(c.course_id = 1) = 1 and
sum(c.course_id = 4) = 0
) l;