How to improve my development strategy?

Question

I am writing a program for Windows, which uses two databases. One is very big (dozens of tables and thousands of records) and comes from another program, but I only download data from it. Second is "mine" and only used by my program (dozen of tables and hundreds of records). I have a repository to keep track of changes. I develop this application only by myself, and when some new functionality or a fix is ready, I publish it to a network hard drive as a ClickOnce application, which means that other people in the company get automatic updates, whenever I publish a new one. So far it has been working fine, but now I need to add a new module, which is going to increase the scope of the project by let's say 50% more. Now I want to implement some better methods to divide the program into development and production. How should I do this in the most efficient way?

1. I thought of making a second branch on my repository called "production", and only switching to it when I want to publish something, and merging it with the main branch. But when a fix would be needed, while I am working on something in the main branch, I don't know how to do it, other than writing the fix on production branch, and copying to the main branch manually, so that the partially written code of something else would not get transferred to the production branch.
2. My second idea is to create a second folder and copy the project to it, but then I would have to manually copy the code from one to another.
3. My third problem is with the database. Should I create a copy of "my" smaller database, for example, once a day, and use the copied version for development, and leave the original version only for people using the released application?

Can someone guide me on how to advance my project from being a one-person-amateur-styled-single-instance-manual-changes-etc-everything, to something which would resemble a bit more professional approach to developing an application which would be easily adaptable to new functionalities, maybe new people joining me in development, keeping production and development separate, avoid some big mistakes and crashes on the working database, and so on?

Answer 1

This answer cannot delve into every question you asked to the fullest degree, as your question is pretty much touching on every modern development practice. But I can steer you in the right direction, because I believe you have the right spirit but simply lack the reference material to implement these changes to your development process.

I've bolded some words to point out important concepts that you should look up if you want more information on it.

1. I thought of making a second branch on my repository called "production", and only switching to it when I want to publish something, and merging it with the main branch. But when a fix would be needed, while I am working on something in the main branch, I don't know how to do it, other than writing the fix on production branch, and copying to the main branch manually, so that the partially written code of something else would not get transferred to the production branch.

Really look into versioning systems, most popularly Git. Using separate branches is a very good idea, but it requires more forethought than your current plan reveals. There are different branching strategies and it's not clear which one would suit your situation best.

The simplest but clean branching strategy is to maintain a master branch which is ready for release at all times, it should never be in a broken or "in progress" state. The rest of the development then takes place on either one particular branch (often called the dev branch), or multiple branches (feature branches). I prefer feature branches but different circumstances might require different strategies.

You can find more information online (for example here)

2. My second idea is to create a second folder and copy the project to it, but then I would have to manually copy the code from one to another.

Don't copy/paste. As you've already pointed out, it's going to lead to you needing to maintain two different sources. And that's even in the optimistic outcome where the sources are similar enough that you can copy/paste from one to the other, because that's definitely not a given for any non-trivial change you're developing.

This ties into Git (or similar versioning systems) again. There is value to having separate versions (e.g. production vs development) but using branches is going to make it a lot easier to keep these separate versions in sync with one another (when you need them to be). Though a git merge isn't always the most pleasant of tasks, it's still a whole lot better than doing it all manually.

3. My third problem is with the database. Should I create a copy of "my" smaller database, for example, once a day, and use the copied version for development, and leave the original version only for people using the released application?

Yes, your development database should be completely separate from the production database. Never mess with production while developing. That is a hard rule because it will otherwise eventually blow up in your face.

Whether your dev database is filled with dummy data or a copy from production, is up to you. There are external considerations here such as privacy concerns which can skew the decision.

In general, I prefer working with dummy data for everyday development purposes, as it ensures I work in a clean environment which I control.

However, if you're often debugging issues in production code, and privacy isn't a concern, it's valuable to be able to debug your code against the real world data - which means that you'd ideally be able to take a backup from production at will (i.e. at the time the bug is reported), restore it locally, and use that for your local debug session.

Additionally, there is definitely value in testing your application (after development, before release) against some real data - but this requires a rigorous test strategy, which I surmise you don't have right now. This is not a hard rule, I've worked on plenty of projects where tests against dummy data were considered sufficient. It depends on how easily the application can break based on bad (or unexpected) data.

Testing strategies are a whole different topic to discuss. It's unclear right now how much you do test, and whether you have any automated tests (I'd hazard a guess that you don't, but it's a guess). For starters, I'd suggest looking into unit testing, and subsequently integration testing.

---

Overall, I personally think that your drive to improve and prevent common development obstacles is a very good one. You've clearly put in effort to come up with solutions to the problem's you're having (or foreseeing), but you've glossed over the fact that there have been many others before you who have tackled these problems, and there are many conventional solutions already available.

The degree of information you need is more than a StackExchange Q&A can address; so I suggest instead doing some online research. The bolded keywords I provided should steer you in the right direction on the search engine of your choice.