Usage of Splunk commands : REX

Usage of Splunk commands: REX

This topic is going to explain to you the Rex command in Splunk with lots of interesting Splunk Rex examples

Usage of Splunk Rex command is as follows :

• Rex command in splunk is used for field extraction in the search head.
• This command is used to extract the fields using regular expressions.
• This command is also used for replacing or substitute characters or digits in the fields by the sed expression.
• You have to specify any field with it otherwise the regular expression will be applied to the _raw field.

Find below the skeleton of the usage of the Splunk “rex” Command :

rex field=<fieldname>  [(regex-expression) ] [ mode=sed <sed-expression>]

Basic syntax of the Splunk rex command

Suppose we have a data which is coming from any of the indexes. Below  we have given a sample data.

127.0.0.1 – admin [08/Sep/2018:12:46:37.223 +0530] “GET /en-US/splunkd/__raw/services/search/shelper?output_mode=json&snippet= true&snippetEmbedJS=

false&namespace= search&search=search+index%3D_internal+sourcetype%

3Dsplunkd_ui_access+ %7C+rex+field%3D_raw+%22.*%5Cs%2B%5C%5B

(%3F%3CDATE%3E%5Cd %2B%5C%2F%5Cw%2B%5C%2F%5Cd%2B)

%5C%3A%5Cd%2B.*%22& useTypeahead=true&show

CommandHelp=true&show CommandHistory=true&

showFieldInfo=false&_= 1536390799087 HTTP/1.1″ 200 29647 “-” “

Mozilla/5.0 (Windows NT 6.3;  WOW64) AppleWebKit/537.36

(KHTML, like Gecko) Chrome/ 68.0.3440.106 Safari

/537.36″ – 96c643368d868c21de48395bc54c65d6 11ms

*************************************************************************************

Splunk Rex Examples 1:

Suppose we want to extract 08/Sep/2018 as DATE.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw ".*\s+\[(?<DATE>\d+\/\w+\/\d+)\:\d+.*"
| table DATE
| dedup DATE

Result:

Explanation:

In the above query, we are getting data from the “_internal” index and the sourcetype name is “splunkd_ui_access”. We have extracted the date from the raw log so we have put “field=_raw” with the “rex” command and the new field name is “DATE”. Then by the “table” command, we have taken “DATE” and by the “dedup” command we have removed the duplicate values.

*************************************************************************************

Splunk Rex Examples 2:

Suppose we want to extract 127.0.0.1 as IP.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw "(?<IP>\d+\.\d+\.\d+\.\d+)\s+.*"
| table IP
| dedup IP

Result:

Explanation:

In the above query, we are getting data from the “_internal” index and the sourcetype name is “splunkd_ui_access”. We have extracted the ip from the raw log so we have put “field=_raw” with the “rex” command and the new field name is “IP”. Then by the “table” command, we have taken “IP” and by the “dedup” command we have removed the duplicate values.

*************************************************************************************

Splunk Rex Examples 3:

We can extract multiple fields by the Splunk rex command.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw "(?<IP>\d+\.\d+\.\d+\.\d+)\s+.*\[(?<DATE>\d+\/\w+\/\d+)\:\d+.*"
| table DATE,IP
| dedup DATE,IP

Result:

Explanation:

In the above query, we are getting data from the “_internal” index and the sourcetype name is “splunkd_ui_access”. We have extracted the IP and date from the raw log so we have put “field=_raw” with the “rex” command and the new field names are “IP” and “DATE”. Then by the “table” command, we have taken the “IP” and “DATE”. At last, by the “dedup “ command we have removed the duplicate values.

*************************************************************************************

Splunk Rex Example 4:

Suppose we want to extract the first two characters from any of the existing fields.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=method "(?<FIRST_TWO_LETTER>\w\w).*"
| table method , FIRST_TWO_LETTER
| dedup method , FIRST_TWO_LETTER

Result:

Explanation:

In the above query “method” is an existing field name in “_internal” index and the sourcetype name is “splunkd_ui_access” .

Here we have extracted the first two characters from each of the “method” field values by the “rex” command. So we have given “field=method” for extracting the values from the “method” field and the new field name is “FIRST_TWO_LETTER”. By the “table” command we have taken “method” and “FIRST_TWO_LETTER “. At last by the “dedup” command we have taken the unique values.

*************************************************************************************

Splunk Rex Example 5:

By the Splunk rex command we can also replace characters in a field.

Query:

index=_internal sourcetype=splunkd_ui_access
| eval CLIENT_IP=clientip
| rex field=clientip mode=sed "s/(\d{3})/XXX/g"
| table CLIENT_IP,clientip
|dedup CLIENT_IP,clientip

Result:

Explanation:

In the above query we are getting data from the “_internal” index and sourcetype name is “splunkd_ui_access”. Here “clientip” is the existing field name and by the “eval” command we have taken the values of “clientip” into the “CLIENT_IP” field.Here we have used “mode=sed” with the “rex” command for replacing the first part of the “clientip” field by “XXX” .For that we have written a sed expression.By the “table” command we have taken the “clientip” and “CLIENT_IP” field.At last by the “dedup” command we have removed the duplicate values.

Now you can effectively utilize  Splunk “rex”  command in  your daily use to meet your requirement !!

Hope you are now comfortable in : Usage of Splunk commands  : REX

HAPPY SPLUNKING !!