Spread our blog

Usage of Splunk commands: REX

This topic is going to explain to you the Rex command in Splunk with lots of interesting Splunk Rex examples

Usage of Splunk Rex command is as follows :

  • Rex command in splunk is used for field extraction in the search head.
  • This command is used to extract the fields using regular expressions.
  • This command is also used for replacing or substitute characters or digits in the fields by the sed expression.
  • You have to specify any field with it otherwise the regular expression will be applied to the _raw field.
Find below the skeleton of the usage of the Splunk “rex” Command :

rex field=<fieldname>  [(regex-expression) ] [ mode=sed <sed-expression>]

Basic syntax of the Splunk rex command

rex

Suppose we have a data which is coming from any of the indexes. Below  we have given a sample data.

127.0.0.1 – admin [08/Sep/2018:12:46:37.223 +0530] “GET /en-US/splunkd/__raw/services/search/shelper?output_mode=json&snippet= true&snippetEmbedJS=
false&namespace= search&search=search+index%3D_internal+sourcetype%
3Dsplunkd_ui_access+ %7C+rex+field%3D_raw+%22.*%5Cs%2B%5C%5B
(%3F%3CDATE%3E%5Cd %2B%5C%2F%5Cw%2B%5C%2F%5Cd%2B)
%5C%3A%5Cd%2B.*%22& useTypeahead=true&show
CommandHelp=true&show CommandHistory=true&
showFieldInfo=false&_= 1536390799087 HTTP/1.1″ 200 29647 “-” “
Mozilla/5.0 (Windows NT 6.3;  WOW64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/ 68.0.3440.106 Safari
/537.36″ – 96c643368d868c21de48395bc54c65d6 11ms

*************************************************************************************

Splunk Rex Examples 1:

Suppose we want to extract 08/Sep/2018 as DATE.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw ".*\s+\[(?<DATE>\d+\/\w+\/\d+)\:\d+.*"
| table DATE
| dedup DATE

Result:

rexdate

Explanation:

In the above query, we are getting data from the “_internal” index and the sourcetype name is “splunkd_ui_access”. We have extracted the date from the raw log so we have put “field=_raw” with the “rex” command and the new field name is “DATE”. Then by the “table” command, we have taken “DATE” and by the “dedup” command we have removed the duplicate values.

*************************************************************************************

You can also know about :  Usage of Splunk commands  : IPLOCATION

Splunk Rex Examples 2:

Suppose we want to extract 127.0.0.1 as IP.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw "(?<IP>\d+\.\d+\.\d+\.\d+)\s+.*"
| table IP
| dedup IP

Result:

rexip

Explanation:

In the above query, we are getting data from the “_internal” index and the sourcetype name is “splunkd_ui_access”. We have extracted the ip from the raw log so we have put “field=_raw” with the “rex” command and the new field name is “IP”. Then by the “table” command, we have taken “IP” and by the “dedup” command we have removed the duplicate values.

*************************************************************************************

Splunk Rex Examples 3:

We can extract multiple fields by the Splunk rex command.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw "(?<IP>\d+\.\d+\.\d+\.\d+)\s+.*\[(?<DATE>\d+\/\w+\/\d+)\:\d+.*"
| table DATE,IP
| dedup DATE,IP

Result:

rexmulti

Explanation:

In the above query, we are getting data from the “_internal” index and the sourcetype name is “splunkd_ui_access”. We have extracted the IP and date from the raw log so we have put “field=_raw” with the “rex” command and the new field names are “IP” and “DATE”. Then by the “table” command, we have taken the “IP” and “DATE”. At last, by the “dedup “ command we have removed the duplicate values.

*************************************************************************************

Splunk Rex Example 4:

Suppose we want to extract the first two characters from any of the existing fields.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=method "(?<FIRST_TWO_LETTER>\w\w).*"
| table method , FIRST_TWO_LETTER
| dedup method , FIRST_TWO_LETTER

Result:

rexmethod

Explanation:

In the above query “method” is an existing field name in “_internal” index and the sourcetype name is “splunkd_ui_access” .

Here we have extracted the first two characters from each of the “method” field values by the “rex” command. So we have given “field=method” for extracting the values from the “method” field and the new field name is “FIRST_TWO_LETTER”. By the “table” command we have taken “method” and “FIRST_TWO_LETTER “. At last by the “dedup” command we have taken the unique values.

*************************************************************************************

You can also know about :  Usage Of Splunk Commands : MULTIKV

Splunk Rex Example 5:

By the Splunk rex command we can also replace characters in a field.

Query:

index=_internal sourcetype=splunkd_ui_access
| eval CLIENT_IP=clientip
| rex field=clientip mode=sed "s/(\d{3})/XXX/g"
| table CLIENT_IP,clientip
|dedup CLIENT_IP,clientip

Result:

rexsed

Explanation:

In the above query we are getting data from the “_internal” index and sourcetype name is “splunkd_ui_access”. Here “clientip” is the existing field name and by the “eval” command we have taken the values of “clientip” into the “CLIENT_IP” field.Here we have used “mode=sed” with the “rex” command for replacing the first part of the “clientip” field by “XXX” .For that we have written a sed expression.By the “table” command we have taken the “clientip” and “CLIENT_IP” field.At last by the “dedup” command we have removed the duplicate values.

Now you can effectively utilize  Splunk “rex”  command in  your daily use to meet your requirement !!

Hope you are now comfortable in : Usage of Splunk commands  : REX

 

HAPPY SPLUNKING !!

What’s your Reaction?
+1
+1
+1
+1
+1
+1
+1

Spread our blog

8 COMMENTS

  1. […] In the above query we are getting data from replace index and sourcetype name in replacelog. After that we have used field and mode attribute with rex command. Then we have used a regular expression. Here “s” is used for substituting after “/” we have to use  regex or string which we want to substitute ( Raj ). Then again we have used one “/”, after this we have to write regex or string (RAJA) which will come in place of substituted portion. At last “/g” is used for globally. So in all the events Raj will replaced by RAJA in our case. You can find more information from Usage of Splunk commands  : REX . […]

  2. HI Abhay, can you please explain the example 1 in more details please?

    As this rex field=_raw “.*\s+\[(?\d+\/\w+\/\d+)\:\d+.*” syntax us not clear for me to understand, thanks a lot

  3. […] Suppose we have data like this, Where we want to extract all counts, highlighted in the red box in the above figure. If you see carefully then you can notice that all the events are in same pattern i.e. number of count for (A/B/C) : (<digit>), here we want to extract all the digits <digit> in a one field. Now if we write normal regex like this then what will happen? If you want to learn about rex command then click here. […]

LEAVE A REPLY

Please enter your comment!
Please enter your name here