What is the best GIT workflow with CI/CD with submodules in a master to a test environment and a stable branch to a production environment?

Question

We have a several GIT repos in Azure DevOps with .NET Core web applications that are related to each other with the use of submodule (we used autocreation of NuGet Packages in a private NuGet store, but this was hard to debug and maintain).

And we also have a master branch with CI/CD pipeline to a test environment and a Stable branch with CI/CD to a production environment.

Without submodules, the workflow is clear to me:

1. We work on a feature in the branches made in the Master branch and commit to the Master branch until the feature is complete and the master branch is free of bugs
2. We do a merge from Master to Stable branch and the CI/CD releases to production

But with submodules this gets trickier.

Here is a sketch of our setup:

• Web App - Master branch

  • Submodule 1 with custom Libraries - Master branch
    • Submodule 2 with custom Libraries - Master branch

In order to prevent unfinished features from a master branch of one of the submodules to get into the Stable branch of the Web App I think the setup for the Stable branch should look like this:

• Web App - Stable branch

  • Submodule 1 with custom Libraries - Stable branch
    • Submodule 2 with custom Libraries - Stable branch

But to make this work, we would need to change the submodules each time we merge Master to Stable.

Is there a better workflow for working with submodules in different git branches? Or is there a good method to change the submodules to the Stable branches of those repositories?

Edit:

If you are downvoting, please let me know why in a comment so I can correct my mistake. Anonymous downvoting is in my opinion 'not done', I always (on stackoverflow) comment as to why I downvote.

Answer 1

We ended up using submodules, I discovered that you have the option to add a branch name in the .gitmodules file through this SO post.

So I add the last line of this code snippet to all my master branches

[submodule "SubmoduleTestRepo"]
    path = SubmoduleTestRepo
    url = https://github.com/jzaccone/SubmoduleTestRepo.git
    branch = master

And when merging a master to stable I just update that to

[submodule "SubmoduleTestRepo"]
    path = SubmoduleTestRepo
    url = https://github.com/jzaccone/SubmoduleTestRepo.git
    branch = stable

of course this does mean that if changes where made to a nested submodule, I first need to merge that master branch to the stable branch of that git repo, but none the less I keep my great debugging and editing flow I am used to and I get to use the same base projects in multiple other projects. Both things are essential to our workflow.

Thanks for helping me in the right direction.

Answer 2

To be honest, I've always struggled to find good workflows to use when dealing with submodules, specifically because of the reason you've pointed out:

But to make this work, we would need to change the submodules each time we merge Master to Stable.

I am not a fan of having to coordinate a ton of merges across a bunch of repos in order to implement day-to-day changes. Do the main repo and the submodules often change in tandem? Are the submodules used in other projects as well, or are they only used by the project in the main repo? Would you be open to considering getting rid of the submodules and just going with a single repo? I think your workflow would be as clear as you described if you ditch the submodules.

There are situations where submodules are necessary, but if you decide that they add more work & complexity than value, it is possible to roll the submodules into your main repo while preserving history. In the main repo, remove the submodules entirely, and commit that. Then, in the submodule repos, move the contents of the repo into a folder named the same thing as the submodule, and commit that. Finally, go back to the main repo, add the sub module repos as remotes, and then merge whatever-you-named-the-remote/master into your main repo branch with the option --allow-unrelated-histories.

Answer 3

I would have two requirements of shared code

1. My own builds are repeatable
2. I am not caught unaware by new bugs or breaking changes / I can actively choose when to advance to a newer version of the dependency

These are usually fulfilled by locking down the version of the dependency, and some sort of versioning scheme. Nuget is good here because I will always get the same version binaries unless I initiate a change, and it follows semantic versioning so I have a rough idea what level of changes I am getting.

Just tracking a stable branch of the dependency in my mind isn't good enough. I think of stable as proven, not not changing at all.

Git submodules apparently can track branches other than master ... but not commit hashes, which would be ideal to ward against unintended changes.

Maybe a nuget packages with symbols would do the trick (for debugging)?