SAP-C02 Amazon AWS Certified Solutions Architect Professional – New Domain 3 – Migration Planning part 3

6. AWS Application Discovery Service

Hey everyone and welcome back. Now in today’s video we will be discussing about the Application Discovery Service. Now the Application Discovery Service, as the name suggests, helps enterprise customers to plan their migration projects by gathering information about their Onpremise data centers. Now typically if you speak about enterprises, they might have hundreds to thousands of servers within that data center. And in the due process of migration, understanding the types of servers, understanding their network dependencies is one of the major tasks and it takes a lot of time during those phases. So application discovery service really helps there. So one of the great feature about this product is its network dependency chart.

So whenever you are running Application Discovery Service, you see it basically gives you the network dependency chart something similar to what you see in the diagram. So if application server A is connecting to B, b is connecting to C, so the entire chart is automatically prepared by the Application Discovery Service. So when you plan the migration, you know that if you migrate just one server, then what are the other dependencies as well, which you need to migrate to AWS cloud. So this is one good feature. And the next good feature which I like about Application Discovery Service is its suggestion about right instance type. So with the agent based installation, typically it can look into the amount of CPU, the amount of memory that the Onpremise server is consuming and based on that it can automatically recommend you which instance type in easy to build that server fit in.

So basically for Application Discovery Service it supports both agentless discovery and agent based discovery. Now, the agentless service needs installation of Discovery connector in VMware. Vcenter. This is important. So basically if you want to have this nice network dependency graph, then you have to go ahead and install the Discovery connector which is supported in VMware V center. If you go with the agent based installation you will not really get this nice little graph here, but you will certainly get information related to what are the right easy to instance types and few more when you migrate to AWS. So let me quickly show you an overview on how exactly this might look like.

So I’m in my EC to console, let’s go to the Application Discovery Service and within here it basically gives you the overview about the agents which are currently associated. So here it basically says that there are two agents among which one is healthy and one is in the shutdown state. So you can go ahead and click on view two agents. All right, so these are the two agents here, among which you will see that the second agent is currently running. So now when you go to servers, it will basically give you the agent. And let’s click here and here you see it basically gives more detail about that specific agent. So currently I have one server in Digital Ocean where I had installed the agent there. So it does not really have a VMware virtualization, so these fields would be empty. However, on the right hand side it basically gives you the CPU type, the operating system, which is the centaur.

Then it tells about the memory, which is the free disk size. You have the total amount of memory, which is two GB, the total amount of disk here, and also it gives you various performance related matrix associated with that server. So now if you go on the left hand side there is a tab for EC two Instance Recommendation. Let’s go ahead here and here we can export a result based on CSV. So let’s say that there are 102 instances within your on premise and you want to migrate them to AWS. So you want to see which is the Right Instance type for all of those 100 on premise servers based on their current CPU and memory utilization. So this specific feature really helps here, where you can select the CPU Ram Sizing. Ideally it should be on average utilization or you can even have a maximum utilization there. But let me just select Average here.

Then again you can select the Tenancy, the pricing model, and if you would like, you can even exclude the certain EC to instance type like Burst Table instances, if specifically these instances are critical and would be running in production. So once you have selected this, you can go ahead and click on Export Recommendation. So it says now that the Recommendation file should be downloaded automatically and this is a compressed file which got downloaded. So let’s click here and this is basically a CSV file. So I’ll just double click here. All right. So now, since only one agent was installed and the data was collected accordingly, now, if you look into the Recommendation section here. So this is the Recommendation easy to instance model. And here the recommended one is T three as small. So this automatic basically recommends the Right Instance type and it really helps the migration engineers who might want to select the Right Instance type based on the application utilization in onpremise servers.

7. Overview of Database Migration Service

Hey everyone and welcome back. In today’s video we will be discussing about the database migration service. Now the AWS database Migration Service, which is also referred as DMs, basically helps customers migrate their databases to AWS quickly and securely. Now, a lot of organizations and enterprises are moving to AWS and most of them have their databases in the online onpremise or in their data center. So they might want to migrate their database to AWS. So DMs is one of the services which makes things much more simpler. Now, during the migration phase, the source database remains fully operational, hence minimizing downtime to application that rely on that onpremise databases.

Now, one of the great features of the DMs that it supports the homogeneous migrations such as Oracle to Oracle, as well as heterogeneous migrations between multiple database platform such as Oracle to Microsoft SQL Server, to Amazon Aurora. So this is one great feature of it. Now if you want to understand this in a simplistic format, let’s say that you have Aurora database and this acts as your source and you might want to migrate the data within the database maybe to Aurora in a different region to Amazon RDS for MySQL as well as MySQL on premise. So what you do, you put a database migration service in between this DMs can talk to Aurora, it can talk to, let’s say, MySQL on premise.

And depending upon the configuration that you set, it can go ahead and migrate whatever data that is present in this database to any other database which is acting as a target. So there are two important terminology here. One is the source and second is the target. Now, from the name itself, we know that source is basically where the source database is and target is where you want to migrate this source database into. So let me quickly show you a demo on how exactly this might look like so it becomes easier for us to understand. So for our demo, what I had done is I had two EC two instance. Now do remember that DMs is not specific to RDS. It can even migrate the database from one EC two instance to a second EC two instance. So what I had done, I had created two EC two instance.

The first one which is easy to MySQL one that was the source one. So easy to MySQL is the source database. The EC to MySQL two was the target. And we had a database migration service which took the database related information and data from the source and moved it to the target. So if you want to see, this is how the DMs dashboard really looks like. So you have the database migration task. So if you would see this is a DB migration task and if you go a bit down, it basically tells you the schema name and also the tables which were migrated as part of the task. Now, during year there are two terminologies that if you would see one is Temp Source EC two. So this is the source. If you would see this is the source and you have the EC to MySQL two.

This acts as the destination here. So data from the source will be migrated to the EC two instance, which is the Temp Desk EC two. All right? And this is how things would look like. So let’s do one thing. I am in my source EC two instance. Let me quickly log in here. So if I quickly do a Show databases, you will see that there is a specific database called Employees. And this is the database that let’s say we want to migrate. So now this is the destination server. And let me quickly open this up. So if I do a Show databases here, you will see that you have the same database called as Employees. So this database from the source to this destination easy to instance, had been migrated with the DMs. Now let’s do one thing, let’s drop this database.

All right, let me quickly log in again. I have a few connection issues anyways, so let’s go. Show database is here and let’s drop the database Employees. All right, so now if you do a Show database is here, you will see that you no longer have an employee database here. So just to reiterate, the Employee database is currently in EC to MySQL and we had earlier migrated it to EC to MySQL two with the DMs. Now, what I have done is I have removed the Employees database from the EC to MySQL two and we’ll have a quick demo with the DMs to see on how the Employees database from the first instance is migrated to the second instance. So let’s go to the database migration task. I’ll create a new task here. Let’s give it a name of KP Labs. Hyphen Demo.

So you have a replication instance. Replication instance is basically the instance which has the DMs software appliance. All right? So the source endpoint here we have already discussed. The source endpoint is where the database is currently present. So I’ll say temp source Two. So Temp source II is basically the first EC to instance and the target database endpoint is Temp Desktwo and Temp disk II is the second EC two instance. All right? So I’ll say migrate existing data and within the advanced settings you can configure various controls. We’ll just ignore this as of now, let’s add a new selection rule where we have to specify the Schema.

Schema is basically which database because a single EC two instance can have multiple database. So currently within this EC two instance, there are two database. One is employees and second is test. So I’ll select that I want to migrate the Employees database from first EC to instance to second EC to instance. All right? And let’s go ahead and create a task. So currently the task is in the creating stage it might take a little amount of time. Again, it depends upon the size of the database. So let’s quickly wait for a minute here. So it has been few minutes and now you will see the status of DMs went from creating to load complete and if you would open up your DMs task and if you go a bit down, it basically shows you the schema name as well as various table names here.

All right. So in order to verify whether things are working as expected, let’s log into the destination EC two instance. So this is the destination server and let’s do a show databases here. And now you will see that you have the database of employees which is created and if you would go inside employees and show tables, it will basically show you the tables associated with the database. So I hope at a high level overview you understood what exactly DMs does. Now do remember that DMs is not just specific to RDS, it can definitely move. Let’s say I have an easy two instance and this EC two instance has a database and you want to migrate that database to RDS. That can specifically be done. You can even migrate database between EC two instance or between on premise and EC two. And as we have discussed, it supports both homogeneous as well as heterogeneous migrations.

8. Creating our first DMS task for Migration

Hey everyone and welcome back. Now, in the earlier video we had a great demo about what exactly DMs is and how exactly it functions in terms of database migrations. So in today’s video we will do the practical so we’ll start things from scratch so that we understand each and every step associated with DMs. Now, the first thing that we need to do is we need to have two EC two instances. So these are the two EC two instances that we had created as part of the demo. So let’s do one thing let’s go ahead and create two new EC Two instances. So these EC two instances will be based on Amazon Linux Two AMI I’ll use T two micro let’s go to review and launch and I’ll launch it all right, so let’s quickly verify the details I’ll call it as KP Labs Hyphen DMs Hyphen source so it’s easy to identify.

So this is a T two MicroBase instance and it has a specific public IP. All right? So now once you have verify the details you can go ahead and launch one more instance like this so I’ll say launch more like this and we’ll go ahead and launch the EC two instance. So this time I’ll change the instance name. So let’s quickly verify here so this is the first EC two instance we had launched and this is the second one. So let’s change the name of the second instance to Kplabs DMs. Let’s call it as desk. It’s easier to identify. All right, so both of them has a security group of Launch Wizard six so let’s quickly open this up. So currently within inbound you have 22 on so we’ll be able to log into that EC two instance make sure you change the security group to your source IP. So let’s quickly log into the first instance here so I’ll quickly log in via SSH all right and let’s go ahead and install Mario DB server.

All right, once you have done that, go ahead and start the Mariodb service. So by default it is in the stopped state. Let’s go ahead and start it all right so now if you do a status you will see that the Mario DB service is active and running so now you can go ahead and do a MySQL secure installation so currently there is no password for root. Let’s set a root password here all right I’ll remove the anonymous user not really required disabled root login remotely I’ll just put it as no for the time being and remove test database I’ll put it as no since this is a demo, we’ll just keep it pretty simple. All right, so we have successfully created so now let’s go ahead and log in. So I’ll put the password here and if I do a show databases you will see that currently there are four database which are available so let’s run the same step for the second EC two instance here.

So this is the second EC To instance which is the destination. Let’s log in here. All right, let’s go ahead and install a Mariodb once. Then I’ll start a Mariodb service and we do the MySQL seeker installation. So I’ll leave everything similar to what we had done in the source instance. So the password and everything remains the same here. All right. So now the next thing that we need to do is we have to create a DMs. So within the database migration service. So whenever you click on the DMs, so let me quickly show you. So when you select the database migration service from console, you will see that the first option here is create a replication instance. And this is quite important. So basically you need to create a replication instance. And this instance is basically responsible for migrating your data from the source to the target.

So let’s click on create a replication instance. I’ll call it as KP Labs Hyphen replication instance. So the instance class I’ll select it as T two micro allocated Storage. Let me just put it as ten GB. Now I can create it inside the VPC. Let’s verify everything. So you can even select the availability zone if required. So that’s about it. So this step is pretty simple. I’ll go ahead and create a replication instance. All right, so this is a replication instance which is creating. Now, this replication instance should be able to connect both to the source and to the destination. This is very important. If the replication instance cannot connect to source, cannot connect to destination either way, then there is no point of migrating the data because this is the instance which acts as a middleman and it does the migration.

So since this replication instance is created inside the VPC, it will have both the public IP address as well as the private IP address. So let’s quickly wait for the instance to be created. So the current state here is creating. All right, so it has been few minutes here and our replication instance, the status if you would see, is now available. So now if I just click on the replication instance here, you will see the details about this. So it says that the class is T two micro. You have the engine version, it is also publicly accessible. And from the cloud watch matrix you will see the amount of memory, CPU utilization and storage specific matrix here. Now, along with that, there are two important things that you will see here. One is the public IP address and second is the private IP address.

Now, this is important because if you would see that this DMs instance would be able or should be able to connect to both source and the target. So within the security group you will have to do the whitelisting so that this DMs instance can connect there. Now, within my EC two instances, if you would see both of these EC two instances here, they share the same security group. So let’s click over here and within the inbound, let’s go ahead and add an inbound on 3306. And I’ll whitelist the private IP address because this DMs instance is launched within the VPC, I’ll whitelist the private IP address of the DMs. Great. Along with that, the next thing here is the VPC security group. So this DMs instance also has its own security group. Now, if you would just filter by the security group.

This is the security group associated with DMs. This verify whether the inbound and the outbound has the right required rules. So in my case, the outbound has so that should not be the problem here. Great, our DMs is now ready. Now let’s do one thing. Let’s go to the source instance and import a sample database that we’ll be using for our demo purposes. So this is my sample EC two instance. Let’s do one thing. I’ll go ahead and install Gate. So basically we’ll be pulling a Git repository which has certain sample database that we’ll import. Now, I have certain sample commands that I have documented. So let’s quickly run those commands. So this will go ahead and clone the repository.

All right, so the specific repo is currently cloned. Now, if you go under test DB, you will see that you have the employees SQL here. So let’s go ahead and import it. I’ll say employees. All right, let’s give it a password. So it is going ahead and creating the database structure and also loading the appropriate information within the tables.Great. So now if you quickly log in here and do a show databases, you will see that you have one database called as Employees. Now, if you currently use this database and do a show tables, you will see that it has multiple amount of tables here. So let’s go back to the DMs and let’s go to the dashboard. All right. So within here, the next important part is to configure the endpoints. So let’s select the endpoints.

Now, these are the endpoints that we had used earlier. Now, we have already discussed the concept of source and target. In also our case, we have two instance. This is a source where we just imported our database and this is a target where we want to migrate to. So let’s go ahead and create an endpoint here. So you’ll have to do it two times. One for the source and second for the target. So for the endpoint Identifier, let’s call it as Kplabs Hyphen source instance. The source engine here would be MariaDB. Now the server name. So for the server name we can give the IP address of our instance here. I give the IP address, you can give the port. Let’s go and give the username and password. All right. Now one great thing here is you can test the connection.

Now before you can go ahead and test the connection you have to make sure that root login is allowed from a specific IP address here. So within the security group so here we had whitelisted the IP address of the DMs and this same IP address needs to be whitelisted at the Maradb level as well. So within the notepad you have this sample command. So this command basically grants all privileged to root at the rate IP address identified by the password that you had set. So let’s copy this specific IP address here and let me replace it. I’ll copy this command and within my marriage DB, I’ll grant the access here.

Now, the same thing you have to do on both the instances because the DMs will be connecting both to the source as well as the destination. So this is the target one. So let’s quickly log in here and I’ll run the same command. Once you have done that, make sure you flush the privileges great. So once you have done that, let’s quickly go ahead and test the connection. So I’ll run a test, so the current status is testing. Let’s quickly wait for a moment here. All right, so now if you would see that the test here is failed. Now, the reason why it has failed is because the replication instance here that we had selected was the one that we had used for the demo and we had not really white listed this one.

So I go ahead and change it to the one that we just created in this demo. Let’s go ahead and run the test once again and when you run the test once again, you see it basically gives you an error. Now, generally whenever you run a test, it automatically goes ahead and saves this endpoint. So if I can quickly show you the end point, you will see that this specific endpoint is automatically created. So this is a little non expected because here we are not really creating the end point, we are just testing it. So this is one thing that you should know. So for the time being, let’s do one thing, I’ll call it as Temp and we’ll go ahead and run a test once again. So it is running the test and here you see it automatically created the end point even before we clicked on the Create endpoint button.

So I believe this is something that will be improved at the later stage, but something that you should know. Great. So now you see the test status is successful. You can go ahead and create an endpoint now. All right, so this is our first endpoint which is Kplab source instance Ten. So now the DMs replication instance can connect to the source. Now the same thing, you have to use it for the target as well. So now let’s go ahead and create one more endpoint. This time the endpoint type would be target. I’ll call it as Kplabs demo dest Temp, the engine of the target again is Mario DB. So I’ll just select MariaDB. Here the server name. Let’s copy the server name here. I’ll use the private IP address. Port is 3306. I’ll fill in the username and password.

Let’s test the endpoint connection. I’ll use the replication instance as kplab’s replication instance here and let’s run the test. Great. So the test is successful. Let’s go ahead and create the endpoint. Great. So now we have the DMs replication instance created. We have the source endpoint created. We have the target endpoint created. Now the next thing is we have to set up this replication related configuration. So let’s go back. And now we’ll go to the database migration task again. These are the two that we had used for the demo earlier. Let’s go ahead and create a new task. Let’s call it as kplabs. Now here you have to select the replication instance. Our application instance is kplab’s replication instance source endpoint is the kplab source instance temp target endpoint is the Kplabs demo desk Temp.

All right. So these are the endpoint that we had just created. Let’s go a bit down. So here you will be able to configure certain settings. We are more interested in the selection rules here. So let’s add a new selection rule. You can specify the schema depending upon the amount of databases you are, you’ll be able to see the databases. We are more interested in the employee’s database here. You can even go to advanced settings and configure certain controls. Here. We’ll avoid it and we’ll keep it simple and we’ll go ahead and create a task. So you have the Kplabs practical task which is currently on the creation stage. Let’s wait for a moment here. So after a moment, your status would have changed from creating to starting. So if you can open this up, you can see the exact details within the table statistics.

So this is quite useful. So let’s quickly wait for a moment here till the time it is migrating the data. Great. So our load is now completed. That basically means the migration has completed. So if you go back to the database migration task, you will see that the progress is now 100%. Now, within the target instance, if you want to verify whether the things have been working correctly or not, you can do a show databases and you will see that you have an additional employee database which is now created. And this database should have the data similar to what was present within the source DB. So that’s the highlevel overview about the database migration service in terms of the practical aspect. I hope this video has been informative for you and I look forward to seeing in the next video.