It's good practice to clean your test db all the time, your test should be self contained and not rely on anything left over in the db from a previous test.
That being said, yes, you should also set up fixtures or helper function that insert whatever necessary data you need in your test db.
You can do this setup once for each test file. DB tests are expensive, time wise, so if you can get away with mocking calls to db you should do that, but it all depends on what you are trying to test.