How can I unit test a class which requires a web service call?

Question

I'm trying to test a class which calls some Hadoop web services. The code is pretty much of the form:

method() {
    ...use Jersey client to create WebResource...
    ...make request...
    ...do something with response...
}

e.g. there is a create directory method, a create folder method etc.

Given that the code is dealing with an external web service that I don't have control over, how can I unit test this? I could try and mock the web service client/responses but that breaks the guideline I've seen a lot recently: "Don't mock objects you don't own". I could set up a dummy web service implementation - would that still constitute a "unit test" or would it then be an integration test? Is it just not possible to unit test at this low a level - how would a TDD practitioner go about this?

Answer 1

In my opinion you should mock the webservice calls if this is a unit test, as opposed to an integration test.

Your unit test should not test whether the external webservice is working, or whether your integration with it is correct. Without getting too dogmatic about TDD, note that a side effect of turning your unit test into an integration test is that it's likely to run slower, and you want fast unit tests.

Also, if the webservice is temporarily down or working incorrectly, should this cause your unit test to fail? It doesn't seem right. Your unit test should fail for only one reason: if there is a bug in the code in that "unit".

The only portion of code that is relevant here is ...do something with response.... Mock the rest.

Answer 2

I disagree with "don't mock objects that you don't own" when your are unit testing.

Mocks purpose of existence is the fact that there will be modules, libraries, classes we will not own.

My suggestion for your scenario is mock the web service call.

Setup the mock in such a way that it returns data back to your module.
Make sure you cover all scenario eg when the data returned back is null, when the data returned back is valid etc.

And for the code that you own, your responsibility as a developer is to ensure that the code you are authoring performs as expected in all scenarios.

Answer 3

I would use something like EasyMock for this test. Mocking frameworks are an ideal way to remove outside dependencies on a class and gives you total control over the result of outside dependencies during tests. To extend your example a little:

class WebClass {

private WebServiceInterface webserviceInterface;

    void method(){
        R result = webServiceInterface.performWebServiceCall();
        ... do something with result
    }

    public void setWebServiceInterface(WebServiceInterface webServiceInterface){
        this.webServiceInterface = webServiceInterface;
    }
}


interface WebServiceInterface {

   R performWebServiceCall();

}


class WebClassTest {

private WebServiceInterface mock;    
private R sampleResult = new R();

    @Before
    public void before(){
        mock = EasyMock.createMock(WebServiceInterface.class);
    }


    @Test
    public void test() {
        WebClass classUnderTest = new WebClass();
        EasyMock.expect(mock.performWebServiceCall()).andReturn(sampleResult);
        classUnderTest.setWebServiceInterface(mock);
        classUnderTest.method();
        EasyMock.verify(mock);
    }
}

The first thing you'll need to do is extract the logic in your class where you use Jersey to get a WebResource and call the web service into a separate class. Creating an Interface for this class will allow you to create a mock to which you can then dictate behaviour.

Once this interface is created, you can create a mock using EasyMock, which will return a specified object accoring to your test case. The example above is a simplification of how to structure a basic mocked test and how your interface will work.

For more information on mocking frameworks, please see this question. Also, this example assumes the use of Java but mocking frameworks are available in all languages and although they be implemented differently, they will work in generally the same way

Answer 4

Mocks are acceptable in this case, but you don't need it. Instead of unit testing method(), instead unit test just the portion that handles the response.

Extract a function that takes ResponseData (of whatever kind is appropriate) and then performs the action.

Instead of mocking, now you just construct a ResponseData object and pass it in.

You can leave the calling of the service to full integration tests - that will cover method() in total

Answer 5

What I've done, and it works:

1. Have all code call webservices through proxy.
2. The proxy calls a class that statically knows if we are using proxy or not and redirects accordingly. Mocks are just HashMaps that for each request returns a given reply.
3. Run the tests several times in this order:

3.1 First all webservices are tested. From each machine, even developer's machines. These are the real webservices, but running in development envioronment. This means webservices can never be down or reply erroneous values, because otherwise every developer complains that he can't compile.

3.2 Then all unit tests internal to the application are run. This means that all webservices are mocked and tested runnning the same tests as 3.1 (anbde they should pass too, otherwise out mocks are wrong), and being invoked by the real application as if they were really being used. If the mocks are wrong, you can run the test in 3.1 and record those (request, reply) values in a HashMap.

3.3 Then the same tests as 3.2 are run, but this time against the real webservices running in the development environment.

After all these has been completed, for the real production environment you just need to provide the real address for each webservice. Hopefully this doesn't require too much change in the configuration.