SPLK-1003 Splunk Enterprise Certified Admin – Installation and Configuration of Splunk Components Part 2

11. Installation of Splunk Indexers

And when it is starting there will be always a statement or a phrase. Splunk is specified by. As we can see it says another one where we’ll be starting in the index. This phrase is randomly picked and there are a couple of phrases like big data superhero and picking needle in a stack. These kind of key phrases that when Splunk every time it starts it pops up one of these and it is checking for prerequisites. That is our ports availability. 8000 for web, 80 89 for management, 80 65 for our app server and Kvstore for 8191. It created the web directory which we know before installation. The word directory will not be but post installation or the first start where directory will be created.

Now we have installed our Splunk searcher and we have successfully started up. Let us try to access this. So what is the IP of this? 52 40 since searched complete package or the Splunk Enterprise full package. We will have the Splunk web component also by default. When you log in for the first time, this is your login screen where the username is admin and the password is change me as soon as you hit enter by logging in. Enter the new password. Let me enter my new password. That should be it. Yes. So you change the password and login. This will be our welcome screen for the first time. Now we have set up all this and we know what is this screen and what each menu means from our previous tutorial.

This shows our Splunk search ed has been installed but it is not configured to function as a searcher. But our instance is up. Let us follow the same procedure to bring up our indexes. This is our indexes we have installed using Privilege account. Let me log into my application account. That is splunk. I’ll be using the complete path. This is our Splunk home bin directory. Splunk start. We know there will be a long license that will be showed. If we don’t want that to be shown to us, we’ll just add one more parameter that is accept license. If you type this, it will automatically accept the license and start our Splunk instance. As you can see this it didn’t display as the license key but it completed rest of the steps. Now we have our index set up and running. Let us change the password for the same. We all know by now that 8000 port is used for Splunk web. Is it taking so long? Is it up? This IP seems to be correct. Let me check quickly whether I have any firewalls rules enabled on this. I have applied some new firewall rules. Let me try to reload. Yes, I think there was a firewall that was configured to block this connection. This is our index and we are logging in for the first time with change me password and once logged in I’m changing my password. Our indexer is also updated.

12. Installation of Splunk Heavy Forwarders and Deployment Servers

Let me bring up our AV folder and deployment server. This is our deployment server and this is our AV folder. I started both of them parallels. These are typically how your production environment looks. Nothing much changes because always whatever the server you’re using, you’ll finally use some putty session to log into them and configure the same. This is the actual the configuration or the real time experience. What you’d be seeing now we have installed all the four servers. Let us change those passwords. What is my AV forwarder? IP? This is my IP.

Before logging in. Let me check h Firewall Networking Security Group. Change it to default sign. This is my ebi folder or the component of Splunk which will be configuring it as evolved. This is as of now for my reference, I have named all these components. But we’ll see how to specify to Splunk to act them as the specified component. Always make sure when you first install to change the passwords so that we’ll see that the functionality or everything is working fine with our installation. So that’s our AV forwarder. This is our deployment server. I think the same Firewall issue on the deployment server. Let me quickly fix it. It should be able to load now. Yes no. So this is our final instance. Change me. New password. New password. Okay, now we have four instances running. Let me quickly show you how to make this Https. .

There are a couple of ways to configure Https. Because as of now you can see this is a plain Http connection which is not secure. It’s better to change our Splunk instances for Https communication. The one way to do this is go to Settings server settings and you’ll be able to see the screen where you have general settings and I’ll enable SSL here so that my Splunk web will be running Https. This is one way of doing this. So as soon as you have updated settings to use SSL, you need to restart to restart again. There are two methods in this while changing this. For this instance we’ll be using GUI mode. For the second instance we’ll be using complete CLI mode.

I clicked on messages click here to restart to make my new server or the new instance to run on Https, let me restart my Splunk. This is my 247. That should be my index at any time. If you want to see the status of your Splunk restart or what is the status current status, you can type this command opt Splunk Bin splunk and status as an argument. This is your splunk home. Bin is where all the executable are stored and Splunk is the name of the utility that is used to check for the status. Hit Enter. It will show Splunk is running. Let us see it’s up. Yes, it is up. Since our self signed certificate, it is safe to see proceed to our IP. Because we know this is hosted by us. And this is our splunk instance. And this is a self signed certificate. Let us log in with our new password. This is done now. We have changed it successfully to Https, although it is our self signed.

13. Enable SSL on Splunk Enterprise Instance

In our previous tutorial we have seen how to install Splunk on Linux and on Windows using a universal forwarder and we have also seen how to enable Https for a Splunk search app. This is our Splunk indexer. We will go through a couple of other methods of enabling SSR. This is our Splunk indexer machine. Let us verify. So this is the 57 and 57 which is presently running with Http. So we will see other methods of enabling Http of Splunk instance. For that we need to start editing our configuration. So what we’ll do we’ll go to Splunk home first, then by the directory structure of Splunk which we have gone through a couple of courses behind. We know all the configuration are stayed under, etc. We know there is a system directory which is highly critical and holds all the Splunk configuration. And we know there is a default directory where all this Splunk default configuration required to start a Splunk instance is present. So we’ll not modify that directory.

Let us see if we have a local. Yes we have a local directory and we have a couple of inbuilt configuration files which are generated as part of installation. We’ll go through all the configuration probably in the later part of our course. But as of now we need to see how to enable our Splunk SSL using editing of configuration file. The SSL configuration is in a file called Web cone which as of now it is not present. We are going to create our web conf. I’m using VI as a text editor throughout this course. You can also use nougat or probably you can also use some other text editors to edit the file and upload it to this location on Splunk. Make sure you are on the right location.

This is the location opt Splunk etc and system local VI Web. This is the file name and we’ll edit this configuration which says Settings and enable Splunk web SSL set to one. One is nothing but true to explain this configuration. This is our configuration throughout the Splunk. The configuration syntax is almost same for all the configuration file. It starts with square braces saying configuration file or configuration type. You can call them as and it starts with parameter and its value. This can be boolean that is true or false or constant or regular expression.

There are multiple options for the value but the parameter will always be one of the Splunk configuration that we are going to change. So this is the common configuration how the Splunk configurations are edited. This collectively is known as a stanza in a configuration file. Each configuration file is a set of multiple stancers. That means it contains multiple different configuration. Let us go through an example of web conf which is by default present in Splunk.

So we have edited our own configuration that is in local but we’ll check the web that is available in default that is provided as part of our installation package. So this command I’m using it just to see the contents of our default web con so that we can go through what all configurations are there probably quickly here splunk clearly says to do not edit these files. That is the reason we are just seeing the contents of those files. If we edit any of this configuration or make any changes which are not compatible with Splunk, we might be unable to bring up the Splunk instance until we fix those problem. It says it has a default configuration Stanza.

All this Stanzas or the definition of standards is the set of configurations that are defined. So there is a setting Stanza which says to start web server by default it is set to true. One stands for true and zero stands for false. You can either mention the text true or False which is mentioned as year. See here. By default it says enable Splunk web SSL to false. That is the reason as soon as we start our Splunk we can access it over Http rather than Https. It is by default set to false which we have changed it to. There is a local file and this is our local file. We have changed it to true. One is nothing but a True. We have made it enabled now even in the web. While we are enabling Https for our Splunk web, we saw we got a restart message under the messages. So we need to restart any time you edit a configuration file in Splunk there are a couple of other configuration which just requires a reload of configurations instead of complete restart of Splunk.

That we’ll be looking at the advanced section of this course to how to reload the configuration without restarting the Splunk. But few of the configuration like enabling SSH changing ports. These require restart of Splunk no matter what. Now we are using this command to restart Splunk. It is again Splunk home bin directory and Splunk utility with the argument restart.

By the time it restart we should be able to see our Splunk in Https. But remember we have made this through CLI, not completely CLI. It is the Linux CLI, not the Splunk CLI. We have edited configuration in the system local. The configuration name is web conf where we have mentioned enable Splunk web SSL to true or one. Let us see our configuration as reflected or not. So it has disconnected from the server during the restart. We know there is no Http anymore. So it is Http. Yes, as you can see this is self signed certificate as a browser precaution it wants us not to proceed. But we know this is a site hosted by us and it is a self signed certificate. We are clicking Proceed. Let me log in with my new password now we are index is also up and running on Https.

14. Enabling SSL from CLI

In our previous tutorial we have seen how to install all the components of Splunk and enable Https on them. We have seen two methods to enable Https. One is using Web that is clicking on Settings server settings and just checking the button of enable SSL. It will be under general settings. Enable SSL. That is first part. The second part we saw was by editing configuration file that is Web conf under System local by setting enable Splunk web SSL to drop. So that was the two methods we saw. If you found some of them were too difficult or it was too easy, there is one more easiest option to enable Splunk SSL that is using Splunk CLI. To invoke Splunk CLI you need to call the Splunk utility. That is the same utility that we called for starting, stopping or restarting Splunk service. This is our AV forwarder which as you can see still runs on Http.

So still it is running on Http. This is our third method in enabling SSL. There is a simple command that is enable web iPhone SSL. You can type this handy enter. It will ask for the username and password of the admin, not the OAS. So make sure whenever you enter Splunk utility it is asking or looking for Splunk user password. With admin privilege I’ll enter my Splunk admin user and password. It says you need to restart the Splunk server for your changes to take effect. Of course we saw either by using configuration edit or Splunk UI, we restarted it to take it effect. Now we used Splunk CLI command to enable Web SSL configuration and it popped up with a prompt saying you need to restart your Splunk service. Let’s go ahead and restart the Splunk service.

Once this has restart you’d be able to see now we’ll be able to access our AV forwarder instance in Https. So there is no more Http, that is Https. See? Since this is the first time I’ll proceed with accepting the certificate. Now we have seen three methods of enabling Https. One via your splunk web. Two via editing configuration. Three. The most easiest one. Splunk. CLI. Now, during this course or any further course that will be going on from now on you’ll be able to see me showing you all these three steps of doing each configuration. If you are configuring a searcher, we will see how we can configure in Web CLI and editing configuration. Similarly, if we are configuring a V folder, we’ll see how to edit configuration, how to edit using CLI, how to make the configuration from Web console. We’ll be going through all these kind of three methods in order to understand, plan and you can choose to learn whichever is more convenient or easy to start with.

15. Index, Indexes and Indexers

In our previous lectures we have seen how to install Splunk, how to install a Splunk universal forwarder on Windows, and how to enable SSL on Splunk other components using three methods. Now, let us proceed with the indexer configuration. Since indexer is the core component of Splunk, let us start with the indexer. From previous modules, we know that what an indexer does it passes the data that is received from the AV forwarder or unicorn forwarder and it is the one which stores the data post processing. The indexer contains multiple logical storages known as index. The index are like small blocks in a big train. For example, each block will hold specific data and it is of specific size. Example one index can be of name Windows, which consists of all the Windows data. Similarly, another index can be named as Linux which holds all the Linux related data.

These Windows and Linux, which are custom defined indexes holding specific data, can be of custom size, like let’s say 100 GB. So each index that is Windows holds under GB. Linux can hold another 100 GB, whereas internal index like underscore audit underscore internal can be of lesser size, but they can also exist on the same storage. To make it simpler, I’ve created a small diagram that is a visual file for better understanding of what are the different components like how the indexes are stored on an indexer. Let’s say this is a total indexer storage, that is the outermost container and there is a Windows index and there is a Linux index. As you can see, they both are like of similar size. Considering our example of 100 GB, these two are sitting on the same storage, but they are logically separated by Splunk under size or it can be even location. This will be under different folder, this will be under different folder.

Both the folders can hold up to 100 GB of data. Similarly, this is our indexer storage which by default will be under Splunk 100 score home. That is, we know by now that we’ll bropt splunk directory and under VAR lib Splunk here it will hold all the database that is total index storage, all the databases or the indexes that are created under Splunk. So this will be our complete index location. Similarly, we have under this location Windows, Linux underscore audit and internal databases which are sharing the same storage but can be of different sizes.

This is how typically the underlying storage of these indexes work. One of the most confusing parts to understand is index, indexes, indexes and indexing which all seems to be kind of similar. Let me help you clear out these terms. To define an index is one logical separation for storing data based on technology or teams or even organization level. If you are building a Splunk for a shared environment, probably you can define each index based on individual company names. What are indexes? Then we understood from our diagram these individual blocks within the same storage can be called as index. Now, what is indexes? It typically represents a group of index present on the same storage we call windows as a index. Collection of these are known as indexes. The next term is indexer. Indexer we have defined earlier it is a component of splunk which carries out indexing process.