In functional testing, should I compare all tabular data rendered in the browser with the one coming from the DB?

Question

I'm working on a test plan for a website where some tests are taking the following path:

1. Hit the requested URI and get the data rendered inside some table(20 rows per page).
2. Make a database query to get the data that is supposed to be rendered in that table.
3. Compare the 2 data row by row, they should match.

Is that a correct way of doing functional testing? If that request was an Ajax request, what will be the answer also? Would the answer differ for integration testing?

I have some reason that makes me believe that this is wrong somehow.... still need your opinions guys!

Answer 1

Yes, this could be a productive test. Either you have a fixed data set or you don't.

If you have a fixed data set, this is much easier to test, because all you're doing is comparing against a fixed output.

If you don't have a fixed data set, then you need to duplicate the business logic, effectively duplicating the work already done by the developer. Then you have two sets of logic to maintain.

The second is the best approach because you get two ways of doing the same thing, effectively a peer review of the specification and code. It's also very expensive in terms of time and resources, which is why most people choose to have a fixed data set.

To answer your question, if your business logic in the query is simple, then you can get a test very easily. However, the value that the test brings isn't great, because you aren't testing very much.

If the business logic is complex, you are getting more value from the test, but it's going to be harder to maintain in the long term.

For me, what your test does bring is a simple integration test that proves that the system reads correctly from the database, and displays the data correctly. This is a good test, even better if it is automated.

Answer 2

This seems fine for functional testing. Integration testing in my mind has to do with the testing of different technologies or components that are supposed to work together which is generally broader than functional testing. But of course this sort of testing could also be considered integration testing, depending on how your application is put together and where the testing is happening in the lifecycle of your development. For example it may be that in order for this site to work you have to put together a few components that were developed independently; this might be one of the tests to validate that the integration works.

Don't see how this being Ajax or not has anything to do with making the answer different.

Answer 3

I will likely be a dissenting opinion here, but I don't consider this to be a productive test. What you are doing is simply duplicating the code which produces the page. And any time you introduce duplicated code (even across departments) you'll be looking at defects cropping up long-term.

It is far better to load the DB with known data (either through the app, or directly) and then check that the output matches what you'd expect. This also ensures that your DB layer, or DB itself, hasn't modified the data in a way you do not expect.

That is:

1. Load known data (preferably through the app itself)
2. Load the requested URI
3. Check that displayed data matches your known data

Answer 4

This kind of test could be good for testing a large set of data with relatively little tester effort if there is not much developer logic between the database and the display to the end user. Our team has done this on a number of occasions, and it is especially useful for running large quantities of real production data through our tests to be sure that actual scenarios are handled as expected. Do make sure you do at least a little fixed input testing for rare scenarios that might be especially likely to be handled differently in the DB and on the web page - null values, special characters, and other oddities.

Personally, I would call this "integration testing", since you are testing the integration of the DB and the web site, and not "functional testing". For "functional testing", I'd probably want to make a mock of the datasource (e.g., the database) that will provide pre-written sets of data in the format you expect.

Having said that, if I had high confidence in the validity of the DB data and if the logic between the DB query and the web page display was very small and low-risk, I would probably not bother with the mock and would let the integration test cover the functionality as well. I don't know that testing the functionality and integration separately would be a big quality win in this case, and there are likely better things you could do with the available testing time. If there is a lot of logic around this data, you should probably test the integration separately from the functionality. Additional integration testing would probably include things like, "What if the database can't be reached?" and "What if the database is slow?".

While this technique will work with Ajax, make sure your testing tools will work with Ajax. Specifically, think about how you will capture the database query results and how you will gather the results displayed on the web page.

I'm assuming that the validity of the data in the query is being tested elsewhere, since you mentioned that this was just one type of test in the test plan. I'm also just discussing integration with the database and this report and not other features or components, and not other aspects of testing (performance, security. etc.), since that was the scope of your question.