Automate clearing everything (database tables, log files, etc.) and starting from a fresh state during development?

Question

The Issue

When I'm fixing a bug during web development projects, I often find myself cleaning out the existing (tainted) database records, clearing out log files, etc.

I do this so that I can start from a fresh state to ensure that none of the previous buggy run's records will affect my new run.

Dropping tables, clearing log files and caches, etc. get pretty tedious to do everytime the server is restarted, so right now I have a simple shell script that I run called drop.sh that automates that process. It looks something like:

#!/bin/bash
mongo <dbname> --eval "db.dropDatabase()"
cd /project/directory
rm ./logfile1.log
touch logfile1.log
rm ./logfile2.log
touch logfile2.log

But I feel this is very clunky and also not sure whether to include this script in the .gitignore.

My Question

I'm a new developer who has no real world experience yet (student). I'm guessing that this isn't what's actually done in industry, and wanted to ask what the best practice is to automate something like this.

Should I be using a Makefile instead that can clear out the server state?

Should I be using automated build software like Jenkins or something?

---

In case it matters, the two projects I'm working on right now that deals with this issue is:

1. A python script that works with MongoDB database
2. A Node.js + Express server + PostgreSQL database for a full-stack webapp

Answer 1

Dropping tables, clearing log files and caches, etc. get pretty tedious to do every time the server is restarted, so right now I have a simple shell script that I run called drop.sh

Any task you find to be tedious or repetitive, it's likely to be so for whoever inherits the project or whoever is working alongside with you.

Automating repetitive task by implementing your own tools is (implicitly) what you are paid for. You are not only professional by doing this, you are a good one because these tools make you productive.

But I feel this is very clunky and also not sure whether to include this script in the .gitignore.

Any script should be in the right folder within the project and in the SCM. Absolutely. Because you will make others as productive as you by sharing these tools. Moreover, others might contribute to the improvement of these tools, making the day even more productive.

But it's not only about to be productive. It's also about knowledge/code reuse. These shells (if written properly) can be re-used. As you said, you can reuse them for CI.

It's only "clunky" when it's not well implemented or when it's not well documented. Introduce all these shells in the README.md file!

Should I be using a Makefile instead that can clear out the server state?

You should perhaps implement both! If your project is the one of those that Makefile allows 3rd parties to compile and assemble quickly, then go ahead. If you Makefile solves your compiling, assembling or packaging then go ahead. But don't mix them up. Don't make a God Shell/Makefile.

Resetting dev environment is one job, compiling and packaging is another job. Releasing the binary is a different job and rolling out the binary is... a different job too. For simplicity make 1 tool for each so you can orchestrate them in the order that best serves you. It's also easier to reason about if you have a tool that does only one thing.

Back again to the beginning, documentation is a must. Make sure you document well-enough these tools.

---

^{1: This is the main reason why I programme these shells, why I make them part of the project and why they are in the SCM. Because this way, I can tell Jenkins _- do use this shell to do this task - _ rather than rewriting the whole task in a different language. Again. It's likely whatever works in my local env, will work in Jenkins (master or slave doesn't matter). If I work on Windows, I implement the respective .cmd shells.}

Answer 2

Your approach looks reasonable. Automating repeated tasks is absolutely good practice, and clearing out development databases helps create a reproducible environment. Doing it with a makefile instead of a shell script wouldn't really be better since there is no network of dependencies.