After a lot of research I couldn't find a built-in way to do this in FFMPEG.
My solution instead was to encrypt and decrypt the file before i used it with FFMPEG.
I did however find a function in libavutil/aes.h which helped my do the actual encryption:
void av_aes_crypt (AVAES *a, uint8_t *dst, const uint8_t *src, int count, uint8_t *iv, int decrypt)