Amazon AWS SysOps – CloudFormation for SysOps Part 4

19. CloudFormation DeletionPolicy

All right, let’s learn how to retain data on Deletes so we can put something called a Deletion Policy on any resource we want to control what happens when the confirmation templates gets deleted. So we can say Deletion Policy equals retain and we’ll specify on any resource we want. And in that case confirmation will preserve or backup that resource, which is kind of neat. And we can specify to keep a resource, we’ll specify Retain and it works for anything thing, for resources, for nested stacks, for anything you want. There’s another one called deletion policy equals snapshots.

 And this one is quite cool because it only applies to EBS volumes, Elastic ache cluster, replication groups, resub instanced cluster, and redshift cluster. But what we get is that the actual instance gets deleted for whatever I just named here, but it will make a snapshot just before deleting it. Hence we can keep our data. So if you have a dev environment and for some reason you want to delete the dev environment, but you want to keep the data for later use and analysis, maybe you want to use a Deletion policy equals snapshots. Finally deletion.

Policy equals delete. That’s the default behavior. That’s what we would do. And by the way, there is this a few other exception. For DB cluster, the default policy is snapshots. But for the rest of the resources and confirmation templates, deletion Policy equals Delete. And if you wanted to delete an estro bucket, something you should know is that you first need to delete everything in that bucket, make it empty before you can delete the estro bucket yourself. So let’s go have a look at how retaining data on deletes work. So I’m going to open Deletionpolicy. YAML.

And very simply, we have a My security group. This is a security group that enables SSH access on port 22 from anywhere. And the Deletion policy for this one is Retain. So we won’t lose that security group when we delete our stack. And we have an EBS volume and it’s an easy two volume. And this time the Deletion policy is snapshots.

And the properties is that the availability zone is going to be in USG one A, the size going to be 1GB and it’s going to be a GB two. So when we delete our stack, we should expect the security group to remain the EBS volume to go away, but a snapshot to be made of it. So let’s go and create a stack. I will upload a template file and I will choose my Deletion Policy stack. All right, excellent. Click on next and I’ll call it Deletion Policy. And I’m really bad at writing right now, but whatever. And we’ll click on Next and click on Create Stack. And so very quickly for us, this will provision an EBS volume and a security group.

So as we can see now, both my security group and my EBS volume are in Create complete stage. So in my resources. If I refresh this, I have an EBS volume and I have a security group. So here they are. I’m going to close the rest. All right. Here’s my EBS. Volume 1GB. GP two. And here is my Deletion, my security group right here.

So now what I’m going to do is go to my stack and click on the stack name and then click on Action Delete Stack and this will delete the stack itself. So click on delete. And so from what we’ve seen before, when we delete a cloud formation stack, everything should go away. But because we have specified, if you remember, a deletion policy retained for the security group and snapshot for the EBC Two volume, we shall see a very different behavior. So let’s go have a look. Here we get a deletion in progress. And if we look at the events, as we can see, my EBS volume is getting snapshot. So it is actually creating a snapshot for us. And my security group was in status delete skipped. So this is kind of neat. So now if we can see our volume for now is still here.

But if I click on snapshots we can see that there is a 1GB snapshot that was created for me automatically by cloud formation. So let’s go back to confirmation stack, refresh everything. And so we can see now that the snapshot is in create Complete. So now what should happen is that now that it is in Create Complete, my EBS volume should be in delete in progress. So it’s going to be actually deleted because it’s been snapshotted. So let’s go back to the volumes and as we can see, we do not have any EBS volumes anymore because it actually has been deleted. So it’s a cool way to expose how things work. And then finally our entire stack is in delete complete stage and so we have no more stacks to display.

So the outcome of this is that we have a security group that still is there and we have a snapshots that get created for us for this EBS volume automatically. So what you need to remember though is that once you do these things well, you need to delete these things manually. So we’ll delete the snapshot manually to clean up properly and I’ll also delete the security group manually to delete properly. But it’s a quite a neat trick to see. And remember, we get deletion policy, retain snapshot and by default delete. All right, that’s it. I will see you in the next lecture.

20. CloudFormation TerminationProtection

Very, very quick lecture. But if you wanted to protect your stacks against this accidental deletes, you can use something called termination protection. The same way we have it for easy two instances, we also have it for confirmation templates. So let’s see that’s quickly how it works. All right, so I’m going to create a stack and I will upload a template file and I’ll just upload the very first one, just EC two YAML. Click on next, click on next. I’ll just need to name it. I’ll name it Test and the next, and at the bottom create stack. So here we go. We’re creating our stack and as we know already, this is going to create yet another easy to instance.

Okay, so now our stack has been created and as we can see, we’re in create complete. So if I go back to my stack, I am able to delete it, but I won’t this time. I will edit the termination protection and I will say right now it is disabled, but I will enable my termination protection so I can’t delete this accidentally. So let’s try to delete it right now. I say action, delete stack and it says you can’t delete it because termination protection is enabled. So to delete the stack, I’m going to first go ahead edit determination protection, disable it, and now after disabling it, I can go ahead and delete my stack. So that’s it, that’s all I want to show you a quick one, but I hope that was helpful and I will see you in the next lecture.

21. ASG – CloudFormation CreationPolicy

Okay, so let’s have a look at how we can create auto scaling groups using cloud formation. Because this is something you need to know. So let’s go into create stack and then we’ll choose a template file and that template file will be the one you get from the code under auto scaling and cloud formation. And we’ll start with zero creation policy for ASG. So let’s go quickly see what this does in my code editor. So in this template I have a parameter and it’s using the SSM public parameter store to get the AMI ID for the latest Amazon Linux two. Then we create an auto scaling group and for the AZ it’s getting them from the region I made already for the launch configuration. It’s a reference to the launch configuration I defined below. The desired capacity is going to be three instances.

The Min size is going to be one and the max size is going to be four. But how do we know if this auto scaling group is correctly being launched? We want to make sure that the instances are able to signal the fact that they have launched correctly, so that we know that this creation of the auto scaling group itself has worked as well. So we can attach a creation policy to your Oto scaling group so that we are waiting for resources to signal and we are waiting for three resources to signal a correct number. So this is the same as my desired capacity, obviously.

And we’re saying we are willing to accept 15 minutes of wait before these instances come up. And if that timeout happens, then the whole auto scaling group will fail to launch. So this is really good because if you have a very complicated launch configuration with a lot of easy to user data in it and something fails within your easy to user data, you want to make sure that this auto scaling group does receive a fail signal. And if it does, then this entire thing will just roll back and stop working. So if we scroll down and look at the launch configuration, it’s very simple. For the image ID and the AMI, it’s using the latest Linux AMI ID that I get from the parameters. And then I will scroll down the instance types, the T two microbe and the user data is very simple. It’s the base 64 of this entire script file and which does something very simple.

It gets the bootstrap scripts and then it uses the CFN signal script to say to my stack, my stack name for the resource auto scaling group which is defined right here for the resource auto scaling group and the region you’re in signal the fact that you are successful. And so this launch config attached to this auto scaling group on the three instances we have, will do three signals onto the auto scaling group and hopefully will work. So let’s go back into confirmation. We’re going to click on Next. I’ll call it demo ASG CFN. And I’ll click on next. I’ll scroll down and click on Next and create my stack in the very bottom. Here we go. And now my creation is in progress. And as we can see here, the Oto Scaling Group is being created. And if we go to my Otisala Group UI, it has been created.

But the CFN stack itself is not in Create Complete state because we are waiting for three signals to come from these three easy to instances that are being created for me. So we need to wait for these three easy two instances to use the user data script and then signal back into confirmation their success. And when they do signal their success, it hits. Then we will go into create complete and so we can see here that there are three events that happened in this timeline. And the event says receive the success signal with a unique idi blah, which is the instance ID here.

So we have three different instance IDs. And so the three instances here, here and here have signaled their success to Cloud Watch, to Cloud the Formation. Sorry. And so as Cloud Formation has had the success and it was waiting for three success counts as part of the creation policy, then what we get out of it is a Create Complete for this ASG. So remember going into the exam, okay, if you want to make sure that the instances are properly configured when being created for the first creation of an ASG, then you need to make sure to set a creation policy for your ASG itself. Okay? Well, that’s it for this lecture. In the next lecture, we’ll look at updates policy.

22. ASG – CloudFormation UpdatePolicy

Okay so now let’s go back into our auto scaling group. So say this is great, this is looking good and these have been launched using this launch configuration right here and say we want to update that auto scaling group, right? So why don’t we go ahead and have a try at it. So I’ve created this one, one ASG update policy and so as you can see this is the exact same template as before. Okay? This is the exact same thing for my auto scaling group but the launch configuration is a little, a little bit different. I’ve updated the launch configuration such as now it says Hello World before doing the Yam update and then doing the signal. So I’m going to try to update my auto scaling group with this and see what happens. So if I go into confirmation and I click on update I’m going to replace the current template and I’m going to upload a file and I’ll choose the first one and I’m going to click on Next.

And so what this will do is that this will create a new launch configuration. Let’s verify that. So yes there is a new launch configuration and my auto scaling group is going to be modified to have that launch configuration as well. So I’ll click on update stack and the update is now in progress. So let me wait for it to happen. And now the update is complete. So let’s get back into our auto scaling group. I’m going to refresh this page. So yes this launch configuration has been updated. It. Start with one J, nine G. Okay. And if I go to my instances and I look at them well these instances are not using the latest launch configuration so somehow these instances have not been updated. My auto scaling group itself has been updated using this launch configuration but my instances in this auto scaling group have not been updated and as such 1 may wonder well how can I make sure that my auto scaling group does update my instances whenever I change my launch configurations? Because that may be something you desire. And for this we’re going to use the update policy attributes. And so the update policy is applied to an auto scaling group a lambda alias or elastic cache replication group. But for this lecture we’ll just consider the auto scaling group and so we’ll specify an update policy and with this update policy we can specify three things. We can specify whether what we want replacing updates so auto scaling, replacing updates and auto scaling rolling updates and finally some attribute called auto scaling schedule action. So you need to remember this going to the exam there are three things you can specify for your update policy replacing update, rolling updates and the auto scaling schedule actions. So let’s go and have a deep dive into one of those to get started.

So we’ll first look at the rolling one. So if you look at this template it’s the same as before. The parameters are the same, the auto scaling group is the same, except that now we have added an update policy to it and this update policy is specifying an auto scaling rolling update. So this rolling update is saying that the minimum instances in service must be one and the max batch size to update this auto scaling group is two. So because we have three instances, that means that two will be taken down for rolling updates, one will be in service and maximum two at a time. So this is how fast we want this update to happen. Then we specify a pass time which is how much pause we want to give for the new instances to come up and give us a signal.

So 1 minute and if we don’t receive a signal in 1 minute then we have to roll back. And do we want to wait on resource signals? True yes, we want to wait for a signal for this update policy. And additionally we set up an auto scaling schedule action to prevent the fact that schedule actions should modify the Min max and desired for confirmation in the ASG. So that means that for example, if you have conflicting schedule actions while you’re doing an update in confirmation, this may lead to bite behavior. So to resolve against this you specify this block of auto scaling schedule action and ignore unmodified group size properties. True, but I think what really matters going into the exam is knowing that the auto scaling rolling update exists. And as the name indicates, it will do a rolling update from within the same auto scaling group. So why don’t we test this one?

And by the way, the launch configuration again has changed. We now say hello world two so that we trigger some kind of updates. So let’s go back into confirmation. I’m going to update this one and I’ll replace the current template. I’ll upload a file and this file will be number two update rolling policy. And so click on Next, click on Next and then finally click on Update Stack. So this will create a new launch configuration again but this time it will update the launch configuration on our ISG. It will also make sure that the instances in here get replaced because of this update policy we have specified. So for it to happen, here we go. We can look at it now.

So this auto scaling group, let me show you. Here we go. So the launch company has been updated. So we have a new one. And here the auto scaling group is saying okay, the rolling update has been initiated. We’ll terminate three obsolete instances in batches of two while keeping at least one instance in service and we’ll wait on resource signals with a timeout of 1 minute when the new instances are added to the autoscaling group.

So this is quite a nice way to show us what we need to have based on the code that we have specified right here. Okay, so now let’s look at the next status. It says, okay, temporarily setting the auto scaling group min size and desired capacity to three. So it has changed that. So if we refresh this, the mid size is now three. Then we go again in here, sorry, in confirmation then it says, okay, terminating two instances and replacing with two new instances. As you can see it did have to terminate instances first and then creating new instances. I’ll refresh this to see the signals. So okay, here we are. Then it says successfully terminated instances. So that’s 67% progress for this rolling update. Then the new instances are added to the auto scanning group and it’s waiting for the two resource signals with a timeout of PT 1 minute and has received one signal. So it’s probably going to receive the second one very, very soon. Yeah, it has received the second one. So now two instances have successfully been updated. So this one and this one are updated and it’s going to do the last batch in my thing. So it’s going to terminate this instance and replace it with a new one. So here we go.

A new one is pending already and so again if I refresh this it has terminated the instance and it will wait for a success signal from that instance. And when the instance has given us a success signal then everything will be working. So let’s wait for it to happen. And now it says that the last instance was successfully terminated and it was waiting 1 minute to receive a success signal and then it received the signal. So it says, okay, update complete and we have just performed our first rolling upgrade of our auto scaling group. So that’s one way of doing things. And you have to remember this going into the exam, you can perform a rolling update for your auto scaling using an update policy. But the last thing I want to show you as part of this lecture is that you can have a replacement policy. So instead of doing a rolling update, we’re going to do a replacing update. So remember this is the second kind of update you can do and we’ll say will replace True.

And this replacing update, what it does is that it will say, okay, create an entirely new auto scaling group and then when that auto scaling group has successfully passed the creation policy, this one then terminate the old auto scaling group. So let’s try this out here. We’re going to create a new auto scaling group. So this is more of like an immutable type of update. So I’m going to update this, replace the current template, upload a file and I will choose the number three. So this should create a new auto scaling group altogether. So let’s have a look and see if that works. I’m going to update my stack. Here we go. So now the update is in progress. Okay. And there’s new resources to be needing to be created. So the resource creation have been initiated. And if we go back to the auto scaling group UI and I refresh this, now we can see that we have a third auto scaling group, I mean a second one as part of this confirmation template, but a third in my UI.

So this new auto scaling group has been created. It has the same settings as the other one and it wants three instances. But this one has the new launch configuration while this one has the old launch configuration. So now we’re waiting for three instances to be started so they’re pending states. And because this is a new auto scaling group and we have three new instances, then again the creation policy we have specified will take place. So if I refresh this, it should start talking about my creation policy. Or maybe if it doesn’t, at least it’s taken into accounts. Trust me. So let me wait for everything to happen and let’s see what the UI says us. So here we go. We have received one, two and three success signal from these three instances that were created in this new auto scaling group. And now what will happen is that the update is incomplete and now there’s a clean up happening and it’s going to delete my old auto scaling group.

So if I refresh this and refresh this page right here, we can see that, yes, this old auto scaling group right now is terminating all the instances. And then when it has terminated all the instances, then we’ll only have the new auto scaling group in place. And as such, we’ll have performed a replaced type of update, which is an immutable type of update. Okay? So this is really important to see, and I think quite good to see the different update policies. So if you don’t specify anything, it will just update the launch configuration but won’t do anything to the instances. If you specify a rolling update for your update policy, then that means that instances will be created and terminated within the same Otto scanning group. And if you specify a replace update, it will create a new auto scaling group altogether and place instances in them.

So finally, there’s one document that you need to make sure to read is how can I make sure that my auto scaling group is updated the right way during a confirmation stack update and they describe a few issues that you can have and the resolution. So I do encourage you to read that document because I think it’s quite an interesting one to really understand how the update policy do work in the real world.

I don’t think it’s something you have to know by heart for the exam. But if you read through this document and understand it, that means that you have understood how the update policy work. And that means that my job is complete and that you’ll be ready for the exam on that topic. Okay, so that’s it for this lecture. I will see you in the next lecture.