Good ways to test a unit that communicates via HTTP

Question

Often, I find myself wanting to write a unit test for a portion of code that accesses HTTP resources as part of its normal function. Have you found any good ways to write these kinds of tests?

Answer 1

Extract the part that accesses the HTTP resources out of your main code. Create an interface for that new component, In your test, mock the interface and return data that you can control reliably.

You can test the HTTP access as an integration test.

Answer 2

This is typically a function I would mock out for the tests... I don't like my tests depending on anything external... even worse if it is an external resource I have no control over (such as a 3rd party website).

Databases is one of the few external resources I often won't mock... I use DBUnit instead.

Answer 3

I recently had to write a component that accessed a wiki and did some basic text scraping. The majority of tests I wrote validated the correct HTTP response code. As far as validating the actual resource goes, I would save an offline version of a known resource and check that the algorithm is gathering/processing the correct data.

Answer 4

Depending on which language or framework you're using, it may be straightforward to start up a locally-running HTTP server which serves up the resources you want.