top of page

Real Agile Git Strategy on AWS

Updated: Jul 20, 2023


$ agile == continuous

The following article covers real world Agile Branching / Tagging Strategy and Git Flow from an AWS DevOps perspective using native AWS CI/CD AWS CodePipeline.




i think this diagram pretty much sums it up…




i love this diagram… the rebbe would say, the wise will understand.




But seriously,


Why is this diagram so true and so brilliant?


What do the hot poops & toilet paper emoji’s have anything to do with it, what does it actually mean?


I’ll explain;


TO VIEW OUR AI GENERATED EXPLAINER VIDEOS CLICK THE LINKS BELOW:


OUR 3 PART SERIES OF AGILE GIT STRATEGY ON AWS:


PART 1:




PART 2:




PART 3:





Waterfall:



In traditional Waterfall development, you are working hard towards a big software release.


To use an overloaded agile term in waterfall context ironically - you are working towards an Epic release!


So, one big giant poop! Packed with everything in it! And then after we release, all we are doing is spending time on fixing - which translates into a couple of big giant toilet papers! Cleaning up our crap for the next couple of months!




Bad Agile:



In the Bad Agile diagram, we see a bit more frequent releases, which makes us think & believe that we are more agile, and working in an agile way, and not waterfall, but essentially, we are still releasing too slow based on scheduled release versions / milestones / or some sort of roadmap with pre-set / pre-determined dates for our releases, and then incrementally adding & pushing things in a bad agile way - for example in an accumulated release at the end of sprints - and then releases accumulate ( bigger and bigger piles of poop), at which point we have accumulated so much technical debt and bugs to fix that we end up spending the rest of the time cleaning up in “epic” waterfall fashion (more big toilet papers)



This is real - in some cases, for example government systems, software releases may be done once or twice a year! In other Waterfall & Bad Agile shops maybe once a quarter, or once a month, nevertheless you are accumulating a significant amount of change, and releasing at a relatively low frequency, where after your “epic” release you are mostly dealing with bug fixes and fixing things that you didn’t expect, as releasing to the wild is never the same as testing in a vacuum.



Real Agile:



In Real Agile, you are releasing continuously!

That is the key word, “continuous” as in CI/CD - continuous integration / continuous delivery / continuous deployment - dealing with little bit every day continuously - every day releasing - every day fixing - every day enhancing - every day, all day merging, pushing, tagging, testing, fixing, deploying, rinse & repeat - many little poops, many little toilet papers, lots of small releases, lots & lots of continuous wiping! That is real agile - it ain’t pretty - but it’s fast and continuous - and whenever something is ready it’s ready, and release it.


Real Agile process was designed this way because 90% of the time developers will miss the estimated LOE, which then just creates totally misaligned roadmaps and inaccurate gantt charts which are totally anti-agile in principle. Don’t get me wrong, i am not saying there isn’t value in creating a roadmap with dates and milestones, but that should absolutely not dictate the velocity of actual development in the day to day. It should only serve as direction, to communicate the goals we are working towards with some sort of priority and timeline, but nothing more than that.





OK! Switching gears!


Now that we understand that we really want to work in Real Agile fashion, let’s bring these concepts down to earth and see how this translates to actual work, tools, commands, flow, pipelines, day to day, on the ground…



So from here we are going to discuss what Real Agile Git Strategy looks like on AWS.


We will discuss Branching/ Tagging Strategy and Git Flow from an AWS DevOps perspective using native AWS CI/CD & AWS CodePipeline approaches.


Ready? Let’s begin;





For a moment, i want you to throw away all your existing perceived notions on git flow, and how you have been using git up till now, and open your mind for a moment to a different, maybe new, maybe simple concept that is different than what you are doing today, and accept that this can possibly change your life for the better.


Let’s agree for a moment - we only have and need one branch, call it master, call it main.


One main branch.


No more branches per features, no more branches per story, no branch per ticket, no branch per epic, nada, no more branches period.


Stay with me for a moment, just one single main branch, ok? Good. Up to here understood.


Now, every day when we login to code, we follow the following process / commands:



Our Daily Git Commands & Routine:



git pull (before start work)
  code - do work
git pull (before commit)
[ IMPORTANT! here if conflict; merge ff - using fast forward ]
git push origin master
git tag v1.1 (optional tag on milestones / features / releases to be tested)
git push origin v1.1



you can increment tags, like v1.1.0 or v1.1.1 or v1.1a, v1.1b

so many ways to increment minor versions and small incremental fixes, have a standard but be loose, ex:



v1.1a-golden-button
v1.1b-silver-button
v1.1c-bugfix-CMS123




So what are we doing here?

First thing that probably stood out to you is, push to master, and you probably reacted like:

“Seriously?”





So, the answer is YES!

every day, all day, we are pushing to master.

Yes, every day, all day we are dealing with merges using fast forward approach.

Yes, we can do this!

Yes we can!


“Seriously, no branches, no feature branches, no dev branch, no QA branch, no pull requests, no PRs?”

Nope - no need - KIS


Why? This is REAL AGILE.

How?

If we do this the right way, through a properly setup pipeline, with controlled build, test, and deploy steps, we will be successful.


So let’s discuss that, but first, i want to quickly formalize and summarize what we have just proposed thus far in more official language:


Trunk Based Development

Essentially what we are doing, is called Trunk Based Development, more info here:


Please make sure to read & understand at least the introduction page where it outlines all the points, pros / cons of this approach vs git feature branch PR workflow.


To emphasize again, just so the idea is understood, the fundamental key point is - to avoid merge hell on complex accumulated merges - the key is working in master all day.





the key is working in master all day.





And here it is in AWS's words…



See Full AWS Article here:



In addition, to be successful using this strategy, we need to emphasize to all developers, not only the overall trunk based dev strategy, but also the specifics of how to merge / roll back & roll forward using the tags and codepipeline that we will be setting up, and we need to practice & document this for everyone to use as a reference till it becomes natural to them.



For example,


To Roll Back -


We will ALWAYS be doing:

* git revert command approach <<


$ git revert 5945jb5

And NOT git checkout + reset hard + rebase = which will rewrite history and change the master to the tip of some branch.

We NEVER want to rewrite history.



To Merge / daily pulls -


We ALWAYS want to use git pull + git fast forward option:


$ git pull
$ git merge --ff-only

Again, this ensures that commits are replayed in the proper order, maintaining a consistent chronological history within the version control system. If a fast forward merge failed, then that is good, and perfectly fine, and means that something in the underlying code we were originally trying to fix has changed, and we need to go back and resolve as expected.

More information on how this strategy works with CodePipeline and AWS CI/CD specifically here:



Pipelines




OK, back to how this translates into actual AWS CodePipeline.


As we mentioned, we need to ensure our process is built the right way, with pipeline approval steps for build, test, & deploy.


The pipeline approval process is key to the success of Trunk Based Development on AWS!


So what does this actually look like using AWS CodePipeline Approval steps?



Notice!

<approval before build>

<approval after build, before deploy>


Why?


By having a separate approval step before the deploy and after the build, it allows us to choose when we want to deploy and when we want to test the actual build / code compilation process, whether the code compiles without errors, before we deploy. We can run unit tests, code quality gate testing, and further automated testing at this build step, before we deploy.


This is key!

Plus, this approach saves money, because we can decide when we actually want to build, on-demand, which costs money since you are running a CodeBuild instance to run your buildspec.yaml code build process.







Concerns:


FAQ & Arguments & Common Concerns that always come up when discussing this:



The main question / concern:

“...

We have scenarios where the whole user story / PR feature is complete and tested, but due to Upstream issues or downstream issues e.g other teams not being ready to deploy their changes, we have to back out a series of related changes that are spread across multiple commits to master, so the whole story needs to be reverted, the whole feature that was built needs to be backed out, what do we do? Won’t a master only approach make this difficult and impossible to cleanly rip out a feature?

…”



Answer:

The truth is, when implemented correctly & if development is done & managed correctly you wouldn't need to back out anything...in other words we move only forward, always.


Ask yourself, do you ever have a scenario where 2 or more developers are working on the same exact file?


Most likely your answer will be “no, each developer is working on separate files” - in which case ripping out your feature should be easy and simple with an incremental delete commit on top of your current position in master. You shouldn’t need now to go back in time to rip out the feature if it was in a separate files to begin with. Good development should introduce the new feature code into the existing code through design patterns & best practices such as dependency injection, isolation, modularity & extensibility, whereby you should be able to identify & delete newly introduced isolated code by deleting the isolated lib / function / class / file easily.


In short, if it is not the same file or it is a set of totally separate files, backing out the change using a forward delete approach, should be easy.


If your answer was, “yes - we do have many cases where developers are working on the same exact files at the same exact time” - then you really need to ask yourself, are you properly distributing, coordinating & prioritizing the work? This is where you should use your daily agile scrum ceremonies to coordinate the work in a way where developers can make progress without stepping on each others toes. This is the development managers responsibility.


In these cases you need to solve a management / discipline issue more than a technical issue whether you decide on trunk or PR based strategy.




Summary:

No matter what, we are dealing with poop ;-P

j/k


If you read the title of this article and said to yourself, oh no, not this argument again, trust me, i feel ya, and i know man…it’s tiring & draining, and it never ends…


Hopefully this article gave you my $0.18, and showed you what worked for me at least, and you can choose your own poison ;-)


We discussed:

  • Trunk Based Development vs Git feature branch PR workflow

  • We Described the recommended native AWS approach using Long lasting branches

  • We covered the actual day to day commands

  • We showed what AWS CodePipeline looks like with approval processes for build, test, deploy stages.


We said our daily dev commands are:


git pull (before start work)
  code - do work - develop
git pull (before your new commit)
[ here if conflict; merge ff - using fast forward ]
git push origin master
git tag v1.1 (optional tag on milestones / features / releases to be tested)
git push origin v1.1


If a merge conflict arises during any one of the git pulls then a fast forward approach should be selected as the merge strategy for git.


This is where, as explained in the referenced articles - multiple, frequent, daily merges, is the key to avoid merge hell....no need to halt work in order for a single dev manager (bottleneck) to review every merge in some “epic” Pull Request.


Release control & empower all your developers to do this together in a coordinated & collaborative fashion.


Reviews are valuable, however if our process is built the right way, with pipeline approval steps for build, test, & deploy, then major reviews should happen maybe only when we want to promote code to upper environments, so basically on major tags, but even there, we can argue that that is unnecessary as we gain stability & confidence in our process.


The pipeline approval process is one of the keys!


There is much more to discuss, argue, compare, & measure pros/cons of the different approaches to development...


...for example: "ok, but instead of completely radical change to only one master branch, can we still have a dev/qa branch hooked up to a dev/qa pipeline, in a separate account, so prod master is totally isolated since we are more comfortable this way"...and the answer is sure!


And remember that every vendor wants to push their tooling on you, and GitHub wants you to use GitHub Actions, and others wants you to use Jenkins, or some other CI/CD tool...however, my opinion, if you are already in AWS, use AWS the way it was designed to be used, and use the native AWS CI/CD tooling the right way, the simple way, don't overcomplicate things, and get the job done.

Additionally, there are many hybrids and variations to pure master only / trunk based development approach....in the end of the day adjusting the process to fit exactly what the team understands and is comfortable with is the right way to go to be successful.



Good luck & G-D bless,







CLICK TO VIEW OUR YOUTUBE CHANNEL!!





243 views0 comments
bottom of page