The problem is, you don’t know what good looks like - till you either a) see good with your own eyes and/or b) make all the mistakes possible till you figure out what good looks like on your own, based on all the good, the bad & the ugly that you built yourself…((and then comes the over / re- engineering problem but that’s a different blog topic…))
As for me?…all of the above…i made the ugly, painful mistakes, then after lessons learned, started to build the right way, and was fortunate enough to have my hacks verified & validated by the industry best, in the real world…
As an AWS cloud consultant working for AWS Advanced & Gold Partners, I get to see many people’s “backyards“….I step into many AWS environments, all types of organizations; commercial & government, while everyone is trying to run to adopt the cloud, and transform their on-prem applications to cloud native, or “lift & shift” as-is, or build new applications and each with their ideas on how to make a beautiful backyard…
Well, lemme tell ya somtn….i saw some pretty messy, disorganized, out of control backyards out there and it ain’t pretty!
Tons of money spent & wasted on doing things the wrong way, futile arguments between inexperienced, uncertified, unqualified engineers & architects arguing what to build, how to build, one way more broken than the other….
((....think Jinga towards the end of the game before it is about to collapse with a bunch of people running around it executing manual processes…yikes! ))
So today, in this post, i am going to share “What Good looks like” imo, and explain why, and how it is different than the bad & ugly i have seen out there….and if in the future after this, you want to know “What Great looks like” check out this post too;
OK, Ready? Let’s go…
The Cloud Application Chicken & Egg problem
So the first thing a cloud architect really should focus on solving, is - solving the cloud application chicken & egg problem in a repeatable way; i’ll explain:
You need 2 things to start:
a) You need a CI/CD IaC pipeline to create your infrastructure from code in a repeatable way.
b) You need the setup in step (a) - your pipelines & starting code - to also be stood up using IaC in a repeatable fashion.
Why?
Why is this so important? Repeatability! it's all about repeatability! Repeatability leads to high availability, which leads to durability, which leads to success!
it's all about repeatability!
So which one comes first? (a) or (b)
(( and how do you achieve this? ))
see the chicken 🐔 & 🥚 egg problem here?
You need an installer of some sorts to bootstrap everything and you also need to bootstrap the bootstrapping, you need some place to start …so what are we going to use? A tool? A script? the ol’ setup.sh? or installShield.exe ;-)
In this post, we are going to KIS and use my recommended, favorite, built-in, 100% native, AWS Tool for the job...>>>
>> the chicken >> AWS CodeStar !!!!!
In my previous post, An Intro to CodeStar, The Most Important AWS service you never heard of, we covered standing up a full serverless Well Architected stack using 100% native AWS CI/CD & IaC.
We used the following AWS services included, and most importantly, integrated, in the AWS CodeStar serverless project blueprint:
AWS CodeStar: the project wrapper & bootstrapper << the tool of tools
AWS CodePipeline
AWS CodeCommit
AWS CodeDeploy
AWS CloudFormation
AWS Cloud9 IDE - see why cloud9 here
AWS IAM
AWS Lambda
AWS API Gateway
(* if you are not familiar with codestar, i recommend to check this post out first *)
How?
OK, got it, codestar is the way to go, but How does CodeStar actually solve the cloud application pipeline bootstrapping Chicken & Egg problem specifically & technically?
To answer this question, let's take a look at what happens within CloudFormation when we spin up a CodeStar project. (Remember, behind the scenes cloudformation drives all provisioning, including codestar projects)
In CloudFormation for every CodeStar Project created, we have at least 2 stacks:
1. THE CHICKEN :: The CodeStar Project Stack itself;
awscodestar-lambda-python
this stack is not directly update-able, rather updates are driven from AWS CodeStar console when creating or updating the codestar project itself. Typically this stack runs just once, unless you are adding Cloud environments, or making other codestar project configuration level changes such as adding integrations which we discussed earlier. Do not try to modify or update this stack directly in cloudformation, as you may break your codestar project. This init CodeStar CFT stack creates & hooks up the initial CodePipeline to all the build & IaC files, including everything else that can be considered initial project plumbing.
2. THE EGG :: The CodeStar Infrastructure Stack;
awscodestar-lambda-python-infrastructure
the stack that is created from the template.yml included in the codestar project git / codecommit repo. This is where you iterate as many times as you need & develop, and make your own changes & modifications, and build your own stack components.
The line is virtually drawn on this - these 2 stacks. Here it is visually in an architecture diagram:
So to conclude this point, by using AWS CodeStar, we use a 100% native AWS tool, to bootstrap our project, and all it's related components (pipeline, repo, roles, permissions, etc), in a repeatable, standard way, where i as a devops engineer, do not need to maintain a separate / custom set of tools or scripts to do this type of bootstraping of an aws project / infra stack.
** more
To dive deeper into the anatomy of the codestar project itself, the files in the egg, the code, the cloudformation templates, buildspec.yml and more, please continue reading here:
Summary
In this article we expanded upon the AWS CodeStar service and the codestar approach to bootstrapping aws infrastructure as code projects using 100% native AWS CI/CD tools included & wrapped by CodeStar including AWS CodePipeline, CodeBuild, CodeDeploy, CloudFormation and more.
The CodeStar approach
We described why this is the right way to manage IaC projects,
We discussed where the lines are drawn to solve the chicken 🐔 & 🥚 egg cloud application bootstrapping problem,
We concluded that CodeStar will bootstrap / start you off the right way, then you will continue to iterate on your stack & pipeline to build your solution.
Stay Strong & Happy Building!!!
Now go start iterating on some eggs! ;-)
You got some powerful chicken in your hands now that can produce many many good eggs stacks!
*****
Extra:
What bad & ugly look like...
i had to add a word about this too...
what to look for, what signals bad, what are some anti patterns and common mistakes;
if you are building lambdas manually, zipping up files, and uploading them to s3 manually or through zip and s3 cp commands in your build, you are doing it wrong...
if you are not using aws cloudformation package command in your buildspec.yml you are doing it wrong...
if you are manually updating cloudformation stacks, entering stack params manually, or pressing any cloudformation buttons manually, you are doing it wrong...
if you are moving files between environments manually, you are doing it wrong...
if you are not working with version control, bad, wrong,
if you are doing all this through cli, or shell scripts, not good, ugly,
if you are using jenkins, probably ugly,
====
Next recommended reading:
dive deeper into the anatomy of the codestar project itself...
now that you know what good looks like…let me show you What Great Looks Like...
Comments