Check the output of mount
and if it is definitely not mounted, just unplug it. There is no problem if no process has access to it.
You may be worried that lsusb
still display your device, but that is because that command show the usb devices that are available - that is, plugged in.
If you have doubts, try to plug a regular flash usb stick, umount it, (with sudo umount /dev/your_usb_device
) and then do lsusb
before unplugging it. You'll notice it will still appear there.