I haven't found a built-in way to do this, but you could accomplish it with your own simple check. Here's an example Thor command that does what you want.
desc "command", "A command that does something."
option :list, aliases: 'l', type: :array
option :file, aliases: 'f'
def list
if options[:list] && options[:file]
puts "Use only one, --list(-l) or --file(-f)"
exit(0)
end
# Place the functions of the command here
end
Hope this helps!