How to reduce the merge-redundancy due to testing in our Git workflow?

Question

We have are a small team of 6 people working with Scrum and Git.

We are developing a quite complex webapplication with the ZK Framework. It uses Java and a MySQL database.

We have adapted the "Git workflow" by Vincent Driessen from:

https://nvie.com/posts/a-successful-git-branching-model/

This results in the following diagram, showing our model:

When we develop a new feature, we branch off of the "develop"-branch.

After a feature has been completed, we do 2 stages of testing - internally (the developers) and externally (the customers that suggested the feature).

For the customer-test, we have a "sandbox", which is more or less a 1:1 copy of our live-system. It has it's own database and branch.

It has it's own branch "sandbox" and is sort of a "pre-release".

Our problem:

We have 1 person doing the merges from the feature-branch into the sandbox-branch, including solving all merge-conflicts.

(We know this is not scalable and soon every developer might need to merge + conflict-solve their own feature-branches. We are still learning and adapting)

Then the customer goes into the sandbox, tests the feature and says either "fail" or "success".

1) Fail - we fix the feature.

2) Success - we close the feature and merge it into develop.

And this is where we are not sure if this process is optimal - you basically need to merge + conflict-solve at least twice, because while testing on the sandbox, the develop-branch keeps getting new commits, which often can result in the feature-/sandbox-version being different from the develop-verson -> merge-conflicts.

Is there a way to solve this more elegantly?

And / Or is there a way to reduce the merge-conflicts?

No data flows back from sandbox into the feature. The arrows labeled "Test: Fail / Success" are merely used to show that the customer tested it and gave his "okay".

We also do not develop on the sandbox.

Answer 1

As your image shows, the Sandbox branch can contain commits from (multiple) unfinished features. This means that in practice, what gets tested is never the same as what gets merged into develop nor the result of merging a feature into develop.

Your features may have been independent enough that this didn't bite you, but someday it will. And the fact that you encounter merge conflicts on both merges suggests that one of two things is happening and neither is desirable

• features get a failed test result due to an earlier failure on a related (unfinished) feature
• code from an unfinished feature gets merged into develop together with another feature

As a way forward, I would recommend that you abandon the Sandbox branch. Instead, you configure the sandbox environment such that code from any branch can be installed on it. And next to that, you implement a reservation mechanism, so that people know when the customer is testing a feature and they shouldn't change what software is running in the sandbox environment.

Answer 2

I don't have access to the image but from your explanation I understand that "sandbox" is a fork of "master" where features are merged to be tested.

There are multiple ways to solve this depending on your system and process

1. Scrap "sandbox", customer test on "develop". You still branch from develop for fixes in case of customer feedback. Stop merge into "develop" when approaching a release and focus on fixing feature. When all features on "develop" are green merge "develop" in a release branch. Reopen merge on develop.
2. Have customer tests on feature branches. You need to be able to deploy differently per customer. Easy for standalone software (like a game), harder for complex systems.
3. Have "sandbox" forked from develop. Only merge "ready" features in sandbox. When all features are OK, you can merge on develop. Problem is you risk to continually have untested feature (by customer) that prevent merging.

IMO the best solution is 2. but is not always an option. 1. is using develop as a beta branch but may force some developer to keep some feature branch branch open if they miss the window to merge before a release.

Answer 3

Others have pointed out that your sandbox branch is flawed, your diagram even shows code being merged into and passing its test when the code is in a failed state from the previous feature.

However, I think you have bigger problems

[feature is merged into sandbox] Then the customer goes into the sandbox, tests the feature and says either "fail" or "success".

Having customer acceptance per feature, regardless of branching strategy, is going to be super hard and cause lots of merge problems.

The reason is that features are not developed one at a time, or in isolation from each other.

If you test feature A and it fails, and test feature B and merge it in. Now feature A has to be retested with the B changes and then with C, D etc if they don't complete fast enough. Before you know it you have a super long lived feature branch which is always having to merge and test

Unless you work on a single feature at a time You have to have some grouping of features into releases and test the release rather than individual features.

Pulling a feature from a release is a big decision, because of the knock on consequences on the other features. You can't let the customer decide at the last minute. Get them to agree the features they want in the next release.

Answer 4

Merge into develop and use feature flags to hide the feature. Deploy develop to the sandbox environment and enable those feature flags there. Develop can still be deployed to live with other changes as necessary, just don't enable the new feature. This means you are only merging once.

Answer 5

As some have said this way of working is problematic but threre are a few things that you can do to minimize the problem:

1. Keep branches as short as posible. Long-lived branches will be problematic. It is better to have more small merges that will result in less conflicts. You can split your changes in smaller tasks but this will require a customer who is fast to approve.

2. Reduce Work in Progress to prevent openning a new branch until the last one is finished. This is good thing in itself as work in progress is a kind of waste.

3. Work on the same branch. 6 people in 6 different branches means a lot of conflicts. Pair programing means only 3 branches with shorter duration. Nothing prevents the 6 of you to work on the same branch.

4. Merge changes from develop frequently into every branch.

5. Use feature flags.

6. Add a third criteria: "Fail" and "Success" are not enough. You can include a new decission "Needs changes" that would allow the merge but will requiere you to fix the bugs or make the appropiate improvements. Incremental improvement will go in the spirit of Agile.

7. Use automatic build and tests. This should detect a lot of the problems caused by new commits. That way the client won't need a long testing process. Less errors will also reduce the "Fails" assesments and that means shorter branches. And hopefully the client will start to trust you so their evaluation will only focus on their business preferences.

8. Choose to work on features that would minimize code conflicts. Surely you will have an idea about what modules would be changed so you can select be strategic to minimize merge.