Rejoining chunked HTML 5 uploads

Question

We've built a HTML5 uploader using file api from scratch with no outside libraries and it was working fine until we tried to upload a file > 150MB. At this point we realised we need to upload the file as chunks.

The Javascript side of things is being sorted by someone else who is splitting the files and and uploading. I'm working on the PHP side of things and need to know what I need to be looking at to work out how to rejoin these chunks.

Do I need to modify the headers of each chunk?

How should I queue chunks while waiting for the next one?

Are we doing this the long way?

Answer 1

Looks to me like you need to talk with the gui doing the javascript side or at least get the finished scripts from him and take a look at the code, quite likely the chunk number will be sent in the one of headers for the request.

Since you'll be receiving chunks in different php scripts I suggest you store them in the database or in the filesystem, it will be a bit harder with the filesystem as you need to also add the chunk number, if you really wanna go the filesystem path I suggest you store them in a folder having the file name as it's name and the chunks as 0...chunk_count-1 as files inside that folder.

You should also receive the total number of chunks through a header and after uploading compare how many chunks you have. When you have all the chunks just assemble them in the order of their chunk id. Care should be taken if you use the filesystem for this that all the files are actually fully written (typically if total size matches original size). A database makes this much simpler as you can insert a row in it when you have all the data.

A workaround for the filesystem way to make sure all your files are completely written is to write them in a different folder and move them to the destination folder upon completing the write.. both folders should be on the same filesystem so the move doesn't generate a copy + delete instead.