Hue, Splunk, and Flume There are several other technologies that are associated with EMR that you need to at least know what they are. You’re not going to need a whole lot of depth on these, so I’m just going to spend one slide on each one. The important things here are what each technology does and how it integrates with AWS. Those are the key points. Let’s start with Hue. Hue stands for Hadoop User Experience, and basically this is the front end for your entire cluster. It’s like…

Pig on EMR Let’s briefly talk about Apache pig. That is also an important part of the Hadoop ecosystem that comes preinstalled on Amazon EMR. So Pig Arose is sort of an alternative interface to MapReduce. So it recognizes that writing code for Mappers and reducers using MapReduce takes a long time. And pig introduced instead a language called Pig Latin. This is basically a scripting language that lets you use SQL style syntax to define your map and reduce. So instead of writing Java code for a MapReduce code,…

EMR Promises; Intro to Hadoop Let’s talk about some of the guarantees EMR makes to you. What is that? EMR charges you by the hour in addition to EC two charges. So you are paying for every hour that your cluster is running, whether you’re using it or not. So it is not a serverless kind of thing. You are provisioning a fixed amount of servers and you are paying for those servers for as long as they’re running. Like we talked about, you can use transient clusters to actually…

Glue Costs and Anti-Patterns Let’s talk about Glue’s cost model real quick. Again, it is serverless, so you’re just paying for the capacity that you’re consuming. You will be billed by the minute for your crawler jobs and ETL jobs. So you’re just billed by CPU time, basically, for any crawlers that are actually extracting information from your schema, and any ETL jobs that are transforming that data on demand. The first million objects stored and accessed are free for the Glue data cap log, and any development endpoints for…

[Exercise] AWS Lambda So now that we know what AWS Lambda is all about, we can finally finish up our order history app requirement here. We’re just going to fill in this last missing piece here. Already. We’ve created our fake server logs on an EC Two instance. We’ve put those into a kinesis data stream using the Kinesis agent, and we have a DynamoDB table set up to receive that data that an app could then talk to to actually retrieve customer order information from an end user. The…

What is AWS Lambda? Let’s talk about AWS lambda. Lambda is a serverless data processing tool that you can use. Let’s talk about what that means. So what is lambda? Basically, it’s a way to run little snippets of code in the cloud. So if you have a little bit of code in pretty much any language you can imagine, lambda can run that for you without you having to worry about what servers to run it on. So you don’t have to go and provision a bunch of EC…

DynamoDB Security So now let’s talk about DynamoDB security. Basically, you have VPC endpoints to access DynamoDB without the Internet. You can get access fully controlled by IAM. You can get encryption at rest using KS, encryption in transit using SSL and TLS. So it’s basically as secure as any AWS service. You get backup and restore feature available, so you can do a point in time. Restore? Just like RDS, there’s no performance impact. We can do backup and restores. You can define global tables, which are multiregional, fully…

The alert log Part 2 Hello everyone and welcome back. I hope that you are doing fine. In the previous lecture we took an overview about the alert look, today we will navigate and we will see the alert logs. So I open my virtual machine. I will go to to home. Then I will go to other location computer one app Oracle. Then I will go to Diag. Then I will go to the product RDBMS. Then the oracle and oracle again. So this is the directory. Actually you…

Default scope in alter statement part 2 Hello everyone and welcome back. In the previous lecture we talked about the default scope in Altar statement. Now let’s do a very nice exercise. I open my SQL developer and my virtual machine is up and running. And also the listener is up and running. My service services is up and running. Now I will do a very nice exercise. First thing I have a parameter called open underscore cursors. And this parameter specifies the maximum number of open cursors a session…

creating non default name for pfile and startup Hello everyone and welcome back. In this lecture we will do a very nice exercise. We will create a p file. But we will create a p file with non-default name. For example, we will create a fleetest aura. Then we will start up the database using this p file. Now we will see how we can do this. First thing I will make a right click and I will open a terminal. I will make SQL plus as sysdba. So this…