DELIMS

DELIMS

Hi guys,

We all know that at the time of indexing when the data is  getting stored into indexers , Splunk software parses the data stream into a series of events. Now ,we can perform different actions on those events. For parsing and filtering we use two configuration files i.e. props.conf and transforms.conf  in Heavy Forwarders. But ,we  can use also configure props.conf and transforms.conf in Search Head(SH) when we want to perform Search Time Field Extraction .

Today ,I am going to show you the how to use the attribute DELIMS in transforms.conf of SH to perform Search Time Field Extraction.

Below is the sample data from which I want to extract field at search time,

NAME:   SARADA MAJUMDER
AGE:    23
HOMETOWN:       BEHALA

We know that in splunk we can create our own custom field at index time and search time both.

The easiest method of creating field is writing <field_name>=<field_value> in your data.But, if at the place of “=” sign if you will write any other sign, the field is not going to be created. But, by using DELIMS attribute in transforms.conf we can easily make our own custom field as per our requirement using any delimiter in between <field_name> and <field_value>

But,we can use DELIMS only at the time of search time field extraction only.(not applicable for Index Time Field Extraction).

Lets see, how:-

Step:1

First, you have to go to the  location where you want save the sample data and there you have to create a file where you want to save your data .

Here, I have created one file called data.txt in /tmp location. You can use any other location or any existing file for storing you data.

Step-2

In the next step we will configure inputs.conf in Universal Forwarder(UF), where I will give the absolute path of  data.txt ,  index name and mention the metadata(host,source,sourcetype)[but it is not mandatory to define metadata]

Here, I have specified index =name

Step:3

Now we will configure the props.conf. As, it is search time field extraction we will configure the  props.conf in Search Head(SH). You can find the props.conf in following path, $SPLUNK_HOME$/etc/system/local.

In props.conf write,

[date]
SHOULD_LINEMERGE=false
REPORT-class=abc

As you can see, I have mentioned here the sourcetype=date, then in props.conf I have to mention the sourcetype in stanza.

You can see the attribute SHOULD_LINEMERGE=false. It will only help to break each line in different events.

Now, the second attribute is REPORT-class=abc(the general format is REPORT-<class_name>-<unique_stanza_name>. So, here the mentioned class name is ‘class’(you can give any string) and the unique_stanza_name is abc(you can give any string).Now, the stanza_name you have to specify in transforms.conf. Lets see how in the next step.

**we use REPORT in props.conf  for search time field extraction.

Step:4

Here the transforms.conf will be configured in SH.

[abc]
DELIMS="\n",":\t"

[abc] is transformation name given as stanza

We use DELIMS in the place of REGEX when we want to deal with ASCII-only delimiter based field extractions, where there are delimiters such as commas, tabs, paces, colons, line breaks etc in between field and value pairs .

**Delimiters must be specified within double coats(DELIMS=“\n”,”:\t”)

The first set of quoted delimiters extracts the field/value pairs and the second set of quoted delimiter acts as the separator between the  field name and its  corresponding value.

Step:5

After configuring configuration files you always should restart splunk in SH and UF, so that all the changes will be will be updated.

Step:6

After restarting splunk you just have to go to location of data.txt and the use the command [vi data.txt]

and write the sample data into it.

Step:7

So,as you can see all the events in Search Head and in the selected field panel you can see  three fields are created i.e. AGE,HOMETOWN and NAME

Lets see, in details in the next screenshot.

Here,you can see, one field is created named AGE which contains the value 23 (as specified in the sample data).

So, you can see, one field is created named HOMETOWN which contains the value BEHALA(as specified in the sample data).

Here, you can see, one field is created named NAME which contains the value SARADA MAJUMDER (as specified in the sample data).

So, what has happened here .As I have mentioned above by default we can create custom filed in splunk only when there is “equal to” sign in between field_name and field_value .

But , here using DELIMS attribute we have created custom fields where there is no equal to sign in between field_name and field_value.

For that, in DELIMS attribute I have specified “\n” as the separator between two fields and “:\t” as the separator between the field_name and field_value.

We know that “\n” represents new line .So, as in my sample data the fields are created in new lines that’s why I have mentioned “\n” in DELIMS attribute as separator between the fields.

Now,“:\t” represents colon tab. So, as in my sample data the field name and field value are separated by colon tab that’s why I have mentioned “:\t” as the separator between filed name and filed value and as per our requirement the fields are created with the specified field values.

Hope, this has helped you in achieving the below requirement without fail:

DELIMS

Happy Splunking  !!