Turns out it was a simple fix to get things working on AWS:
FileSystem fs = distCache.getFileSystem(conf);
AWS could then see the shards under that directory, and it executed just fine. I still don't know why this was necessary for AWS to work when the previous code in my question functioned just fine on a standard cluster, but there you have it.