"Human sort" is a mad man's illusion available only on hypothetical AI systems, but only when having proper context knowledge.
To this end you should rather use some quick regexp to extract meta-data from file names and use them to order files:
files<-c("file9.txt","file10.txt");
as.numeric(gsub('^file([0123456789]*)\\.txt$','\\1',files))->fileNum;
files[order(fileNum)]