Alternative to query string and cookies when sending data to server?

Question

I have a small (or perhaps not so small) issue with some ASP.NET/Castle Monorail sites for a product I work on. This is a fairly legacy system (written well before my time), and it uses GET requests with a considerable amount of information in the query string. We have recently run into query string length limitations, and for the amount of data we need to transfer to the server, storing the data temporarily in a cookie is also not plausible (we are already well past the 4096 byte limit per cookie, and we set a lot of cookies, so we are likely near or at the limit of cookies per domain as well.)

I am wondering if there are any alternatives, besides POST (in some cases changing to a POST request may be possible, but likely not in all), that might solve this problem. I'm hoping someone else here on StackOverflow has run into similar issues, and has some wizardly solution (i.e. compress data with javascript, encode as base64, pass to a single query string item? Just not sure if there are any libraries that can compress data with javascript in a way compatible with built-in compression classes in .NET 3.5.)

UPDATE:

Solution I ended up choosing was to POST to a temporary controller. This temp controller pulled the large list of data, stuck it in a shared Session (the production servers are in a large multi-bank server farm that does not use sticky sessions/IPs), and performed a GET to the actual controller, which pulled the data from the shared Session. Not the most performant solution, but it solved the problem.

Answer 1

There are several Base64 encode plugins for jQuery, but that doesn't help because Base64 generally makes the data longer, not shorter.

Depending on the data, I'd look at some other text compression techniques. Wikipedia lists a few:

• Context Tree Weighting method (CTW)
• Burrows-Wheeler transform (block sorting preprocessing that makes compression more efficient)
• LZ77 (used by DEFLATE)
• LZW

Here's an LZW implementation for Javascript.

Huffman encoding is based on letter frequency, so it might not be appropriate for your data. You'll probably have to escape the result to make it URL-safe.

Answer 2

Beyond switching to POST your options are going to be limited. Base64 is going to increase the size of the data not decrease it. You mention compressing the data into a single query string item... maybe instead you could split the data into 2 separate requests using Ajax? I'm not sure if that's feasible for your situation, though.

Answer 3

I would suggest using JSON as the data transfer format if an array like query string is generated as follows:

params[]=sth&params[]=sth2&params[]=sth3

This would be

{params:['sth1', 'sth2', 'sth3']}

You can sssign this JSON string to a single letter variable like below

p={params:['sth1', 'sth2', 'sth3']}

Something even better would be compressing and Base64 encoding the whole query string. Since you generate the query string on the server side(I assume this but same thing can be done in JS by using the same comp/decomp algorithms) you can use any builtin compression algorithm with your programming language such as gzip. After Base64 encoding the compressed data, yes it will expand a bit but not as much as url encoding expands it.

Answer 4

I would not rely too much on javascript. Compacting the query string would be the first natural thing to do; but if you're way ahead those limits, you should try mentaining sessions. You start up fresh with default state info, then gradually reflect the state on the server into the user's session with POST data provided, so you partition all that data into many handlers (read pages).

Answer 5

The textbook answer is to use POST. Why would this not be possible? The reason POST was invented was pretty much to get around length limits in GET.

Another possibility depending on the details is to have multiple screens, each with a subset of the data, and store the data from each on the server and connect them up at the end.

Answer 6

Is the query string limitation on the client or the server?

If the issue is just that the client will not send the long GET string, perhaps you can POST to a proxy server which then forwards the request as a GET.

Answer 7

Is this to hold state information between pages? If so, an option could be to use a cookie with it's value a GUID as such:

_SESSIONID=3F2504E0-4F89-11D3-9A0C-0305E82C3301

And store the session information server-side. The session is uniquely identified with a GUID, so it is then easy to retrieve the large amount of information directly on the server.

All you then need to send/retrieve client-side is that session id, as well as client generated information (GET/POST fields).