I think you are looking for an embedded database
Good ebmedded
With embedded you don't need a separate install of the database.
As for security.
I don't see how Express would have more security than an embedded.
If the USB pen is lost then they are gong to be able to hack the data on it.
You could encrypt with a long key.
Your best security is don't lose the USB.
The problem you have with a key is that if it is short enough for a human to enter then it can be hacked by brute force. If you put a long key on a CD so it is hard to crack then you are stuck with don't lose the CD. If you look at HIPPA if you are an open network then you have to use an encryption key of a certain length. But that is not data going over a network (I assume).
Is the device you are putting USB into trusted? If not game over.