First, lazy seqs are not efficient for checking membership. Consider using a set instead of a lazy seq.
If a set is impractical, your solution isn't bad. A couple of possible improvements:
"Not empty" is a bit awkward. Just using seq is enough to get a nil-or-truthy value that your users can use in an if.You can wrap that in boolean if you want true or false.
Since you only care about the first match, you can use some instead of filter and seq.
A convenient way to write an equality predicate is with a literal set, like #{key}, though if key is nil this will always return nil whether nil is found our not.
All together that gives you:
(defn lazy-contains? [col key]
(some #{key} col))