SPLK-1003 Splunk Enterprise Certified Admin – Splunk Post Installation Activities : Knowledge Objects Part 3

11. Props Extract Command

Since we have seen how to extract fields using interactive field extractor and how to use Rex command to extract the fields on the fly that is on the searcher and how to use Rejects in our searches. Now let’s see how to make this field extraction permanent so that any user should be able to view these fields and make visualization based on these fields. To make that we need to edit Propsandranscopes. To edit, let me go to our searcher. This is our searcher. I’m a splunk user. I’ll go to Splunk home. I’ll change the font size so that looks much detail. So I’ll go to etc system local and I’ll create a new Props con. As you can see, we’ve added two lines these from our previous tutorial. We know this rejects matches the IP address and it adds a field name under IP address. To make it clear, I have added field name as IP address that is extracted from the Props. com.

This is followed by a source type of our logs. As you can see in our search we have mentioned for the source type of these logs we’ll be extracting this as the IP address from our previous Rejects condition. As we know, this is the beginning condition, this is the ending condition and this is our matching condition for the IP address and this will be our field name. We know that by now. Let me save this file. Once saved this file, we can rerun this search and find any fields that are named as IP address. As of now, there is nothing.

So in order to reflect your newly added field, there are two ways. One, you can restart your Splunk instance so that the newly added props cons will be reflected and picked up during the Splunk starting. Instead of that, the better way is using the command Extract reload is equal to True. This makes sure all the extractions in the props and transforms are reloaded before the search is beginning. Just do extract reload and now the search has been completed, let’s look for a new field. So it is IP address underscore Props. As you can see, our rejx is correctly extracting the IP address from our logs using this IP address once it has been extracted.

Now, if you rerun the search without extract command, also you should be able to see the same field because the props and transforms are already reloaded and we’ll be able to see these fields now in a selected fields. Now, if any users comes up and restarts or researches the same source type, he will be able to get this as part of his default fields. Now we got our IP address from our props. This is one method of doing it. There are other methods where you can use. At present, we have used in our Props. com a method known as Extract command. We will see other method, something called as a report and transforms. We’ll see how we can add these two in our next lecture.

12. Props Report and Transforms

Now we have understood how to extract fields and make it available for all the users so that we’ll be placing it under props. com that is using our extract command. Now let us see how we can do the same using a report. The syntax is report iPhone. I’ll call this as you can call this function the name as anything let’s say I’ll say report IP is equal to this will be your function name which will be defined in transforms. com. Let’s say IP underscore extraction in transforms this name can be anything but make sure you remember this name I’ll copy it save this file.

Now we need to edit one more file called transforms. com in order to define these fields. So here the stanza name will be the function name whatever we defined there. Once you have added the function name the next will be your parameters that is rejects equal to whatever the regex that we have created couple. Of lectures before and I’ll paste the same. And there is format which says which field should be assigned to. I’ll change this name to Field name to transforms so that we’ll understand it better in our GUI once it has extracted. So this is the common syntax, it says this is the match value and whichever matches the first argument assign it to field name called IP address underscore transforms.

As you can see now we are doing the same field extraction which we did in props in our previous lecture using a single command. This is using report function. Why this has been made complicated let’s say this Extract command is good for extracting individual fields, or probably a couple of fields, but if you use Report command, you can write a complete syntax with additional options which are available as part of your transforms.

That is Dlim where you can specify custom delimiter like Pi or Comma, or even a white space where you can mention any number of fields and you can keep on assigning these fields to n number of fields. So if you want to extract fields bulk I would recommend you to go for report and if you are extracting individual fields then it should be fine to use extract command. Now we have added our props and transforms again you can either restart your splunk service so that the edited configuration will reflect or you can simply use the command extract reload is equal to true.

So once these fields are extracted we’ll be able to see our new transforms IP address sorry see as you can see now we have got our newly added field transforms which does the same function since we have added the same rejects. But we have followed a different method in extracting this as you can see one we did via a single command, another one via multiple steps but our results are same. As I said, extract can be used for extracting individual or probably a couple of fields but report can be used for bulk extraction of the entire log.

13. Props.conf Location

Now we know about props and transforms and how to extract these fields, the next important thing is where to deploy these files. We will see how we can deploy these field extractions from our next part of the lecture where we’ll be completely dealing with managing configurations via deployment server under. Also, if you have a question of where to place these props and transforms, always use your local location. That is either system local or apps local, depending upon the context you are using this field extraction for. No matter where these props come, it will always be available. It can always be made available to all the users in Splunk. All the objects by default are private, but at any point of time you can make it available for all other Splunk users.

14. Eventtypes Creation and permission

We have understood more about Splunk installation and its component and also more about extracting fields. Now let us understand the various knowledge objects of Splunk. The first knowledge object in our discussion is Event types. When I see knowledge objects, they are nothing, but it is a method of enriching your data in Splunk so that user can add additional values to their data for getting more information of the data that is present inside your Splunk, and also to teach Splunk about the information that has a regular system admin. Or any person who is regularly interacting with those data would have much better information. We will be seeing how to add this information which is available with the respective individual into Splunk.

Let’s say I’m a system admin. I know what this IP belongs to, which part of my data center it exists, and which department is using this one. The same information can be taught to Splunk so that anybody using this system in their logs will also be aware of where this information or where this server has been placed. Now looking at Event type, it is a user defined field that represents a group of events. These events are grouped by similarity of their technology or the conditions at which these events occur. Let us jump into our lab and see how we out of these event types in our Splunk. This is our search app. I’ll write a basic query index is equal to main source type is equal to we have seen previously. I’ll stick to access is equal to combined search for last 30 days. I got the results as I integrated or as I uploaded this data. I know these are my access logs of Splunk.

So what I’ll do, I’ll go to save as create an event type and I will give any access underscore logs so that anytime if I mention event type is equal to access lock, I’ll get these results no matter what tags will be discussing in our next section. And you can also choose which color these tags should be or Event type should be of. For now, let us leave it to default. I’ll save this. I’ll click on done. Once it is saved, I’ll remove my search and I’ll just type event type is equal to access underscore logs. As you can see now I didn’t type my full query, but still I got the same results. This is because we have saved it as an event type. In order to see all your event types go to Settings, click on Event Types. I’ll open it in a separate tab. This is my general practice so that my search is not disturbed.

These are some of the event times by default which are present. As you can see, we have created our own access underscore logs which the search string is this one and the owner is admin. Presently it is private. If you want to share this with other team members or other Splunk users. You can click on permission and you can choose what permission the other users will be having. Either they will be having just the read permission or even write permission. I’ll use anybody using this app which is the default app. Or you can also choose any apps anywhere in this plunk they are searching. Everybody has read permission admin as right and the power user has right privileges. So I’ll choose these two roles as write privileges and I can save it. As you can see now the sharing permission got changed to Global. This event type was created under Search app. That is our search app. It was created by Admin. Attend anytime. If you want to disable this, you can disable this event types. If you want to clone and modify, you can clone this event types.

15. Eventtypes Use Case

Similarly, let me go back to my previous search. This is our previous search by default. If I run this search, I should be able to get a new field that is our Event Type which we created right now. As you can see, I can get additional field which has says it is an access log. Let me make some more Event types so that we’ll get better understanding. There is a field called Status which represents Http status in our access log. I’ll say Status not equal to 200. That means these are Event types that are not 200. I’ll save as seen access Logs nontwohundred Request okay, I’ve saved that one. I’ll save Access log only with 200 request save Event type access underscore logs underscore 200 quest that means they were successfully given a response back.

Now let us go back to our main search and rerun the search and let’s see how many Event types will be getting. So we have still one. Let me refresh this because it is reloading from my browser cache. Once I reload from the server, you should be able to see more Event types. Because we have including Status 200 and other requests. Let us see if our Event types are created successfully. Yes, they are created.

This is our Status 200 and Non 200. As you can see now we have two Event types that are Non 200 and Access log 200. Similarly, this can act as your additional field where you can filter multiple events. Instead of writing a long queries, you’ll be able to write just single query. See, instead of writing all 200 requests now I can just type Event Type is equal to Access log underscore 200. I’ll get the same results instead of writing the complete search, event Type will be your field name and value will be the name of the Event Type that you are given in your Event Type definition. So as you can see, this much big search can be called as Internal Search Term which will give all the searches that are used. One more thing about Event Type is like whenever you create Event Types, it will go in turn and create a file known as Event Typesconf. Let me quickly get into that folder we have created under Search Local. It should be under event types. Conf. As you can see, we have created access logs. This is the name and this is the search for that Event Type. So all the Event types will be under your Event Types conf.