Delimiter to use within a query string value

Question

I need to accept a list of file names in a query string. ie:

http://someSite/someApp/myUtil.ashx?files=file1.txt|file2.bmp|file3.doc

Do you have any recommendations on what delimiter to use?

Answer 1

If they're filenames, a good choice would be a character which is disallowed in filenames. Suggestions so far included , | & which are generally allowed in filenames and therefore might lead to ambiguities. / on the other hand is generally not allowed, not even on Windows. It is allowed in URIs, and it has no special meaning in query strings.

Example:

http://someSite/someApp/myUtil.ashx?files=file1.txt|file2.bmp|file3.doc is bad because it may refer to the valid file file1.txt|file2.bmp.

http://someSite/someApp/myUtil.ashx?files=file1.txt/file2.bmp/file3.doc unambiguously refers to 3 files.

Answer 2

Having query parameters multiple times is legal, and the only way to guarantee no parsing problems in all cases:

http://someSite/someApp/myUtil.ashx?file=file1.txt&file=file2.bmp&file=file3.doc

The semicolon ; must be URI encoded if part of a filename (turned to %3B), yet not if it is separating query parameters which is its reserved use.

See section 2.2 of this rfc:

2.2. Reserved Characters

URIs include components and subcomponents that are delimited by characters in the "reserved" set. These characters are called "reserved" because they may (or may not) be defined as delimiters by the generic syntax, by each scheme-specific syntax, or by the implementation-specific syntax of a URI's dereferencing algorithm. If data for a URI component would conflict with a reserved character's purpose as a delimiter, then the conflicting data must be percent-encoded before the URI is formed.

 reserved    = gen-delims / sub-delims

 gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"

 sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
             / "*" / "+" / "," / ";" / "="

Answer 3

I would recommend making each file its own query parameter, i.e.

myUtil.ashx?file1=file1.txt&file2=file2.bmp&file3=file3.doc

This way you can just use standard query parsing and loop

Answer 4

Do you need to list the filenames as a string? Most languages accepts arrays in the querystring so you could write it like

http://someSite/someApp/myUtil.ashx?files[]=file1.txt&files[]=file2.bmp&files[]=file3.doc

If it doesn't, or you can't use for some other reason, you should stick to a delimiter that is either not allowed or unusual in a filename. Pipe (|) is a good one, otherwise you could urlencode an invisible character since they are quite easy to use in coding, but harder to actually include in a filename.

I usually use arrays when possible and pipe otherwise.

Answer 5

I've always used double pipes "||". I don't have any good evidence to back up why this is a good choice other than 10 years of web programming and it's never been an issue.

Answer 6

This is one common problem. How i handled it was: I created a method which accepted a list of strings, then found a character that was not in any of the strings. (I did this by a simple concatenation of the strings, then testing for various characters.) Once a character was found, concatenated all the strings together but also prepended the string with the separation character. So in the given question, one example wud be: http://someSite/someApp/myUtil.ashx?files=|file1.txt|file2.bmp|file3.doc and another wud be: http://someSite/someApp/myUtil.ashx?files=,file1.txt,file2.bmp,file3.doc But since i actually use a method that guarantees my separator character is not in the rest of the strings, it is safe. It was a bit of work to create the first time, but i've used it MANY times in various applications.

Answer 7

I think I would consider using commas or semicolons.

Answer 8

I would build on MSalters answer by saying, to generalize, the best delimiter is one that is invalid to the items in the list. For example, if your list is prices, a comma is a bad delimiter because it can be confused with the values. For that reason, as most these answers suggest, I think a good general purpose delimiter is probably "|" as it is rarely a valid value. "/" is maybe not the best delimiter generally as it is valid for paths sometimes.