Send Specific Events To A Specific Index

Send Specific Events To A Specific Index

Hi guys !!

Today, we will show you how to send specific events to a specific index.

You all know that by default all events go to the index called  “main” . If you want the events to go to a particular index then we write an attribute index=<Particular_Name_Of_Index> under the Monitor stanza. Also we know that if we want to  break lines in seperate events then we use an attribute SHOULD_LINEMERGE = false under the particular stanza in props.conf. But Today we will show you new and interesting things.

Suppose we monitor a file or a application log. This application generates different type of logs.

And we will distinguish them and store them different indexes. If this type of situation  arises then how to handle it .This is today’s topic and we will discuss it below.

Suppose this is the sample file –

I am a kolkata boy
I know KOLKATA is a good city to live
Hi Behala, do you have dada with you ?
yes I have data with me
I am happy here
behala is in kolkata
kolkata is not in behala
hi kolkata
kolkata is  big as respect of behala
behala is small as respect of kolkata
behala jio jio
Behala jio jio

In this situation suppose the line which content “kolkata” that goes to a index name as ‘kolkata’ and the line which content “behala” that goes to a index name as ‘behala’ . Rest of all the data goes to ‘restdata’ index.

First of all ,save the sample data in a file named “new.txt”. Then go to the following path in UF.

$SPLUNK_HOME$/etc/system/local

And open the inputs.conf.

Here,I will mention the path of the file where  the sample data is kept,like here I have specified the path of new.txt and metadata(not mandatory).

[monitor:///root/new.txt]
index = restdata
sourcetype=kol
 

NOTE: We have to define props.conf and transforms.conf in HF.

Then we will go  to the below defined location in HF.

$SPLUNK_HOME$/etc/system/local

In this location we have to define props.conf and transforms.conf.

props.conf:

[kol]
SHOULD_LINEMERGE=false
TRANSFORMS-area=BEHALA,KOLKATA

In square bracket which will be written that is the sourcetype which is mentioned in inputs.conf in UF. This is stanza.Here Two attributes are defined.First is SHOULD_LINEMERGE = false, it is used to break the lines in events.If it’s true then total lines will be in one event but if it’s set to false then the lines are  not merged i.e.one line in one event.

Second attribute is used for creating the class name or a unique identifier which we have used to create stanza in transforms.conf. “BEHALA” and “KOLKATA” are the unique identifier names in props.conf.

Then we will configure transforms.conf and here we will define the regex of the pattern which we need.

transforms.conf:

[KOLKATA]
REGEX=(?i)Kolkata
DEST_KEY=_MetaData:Index
FORMAT=kolkata

[BEHALA]
REGEX=(?i)Behala
DEST_KEY=_MetaData:Index
FORMAT=behala

In the above case which line contains “behala” that has gone to index name as behala. And which contains “kolkata” that goes in index say behala. Other lines which neither contain “behala ” nor contain “kolkata” that will go  in restdata index. For this we have used DEST_KEY = _MetaData:Index. To know more about the usages of DEST_KEY you can follow DEST_KEY = _MetaData:Index.

Then restart the splunk server  –

./splunk restart 

NOTE: Before restarting the splunk, check if the indexes are present or not.The indexes names are Behala , Kolkata , Restdata .

See We are getting the events as desired.

We are getting only those events where kolkata is present. See we are getting some events where kolkata and behala is present. It is happening because it will give highest preference that stanza which had been defined in transforms.conf .

We are getting only those events where behala is present.

We are getting only those events where neither kolkata is present nor behala is present.

Hope, this has helped you in achieving the below requirement without fail:

Send Specific Events To A Specific Index

Happy Splunking  !!