Usage of Splunk commands  : REX

Usage of Splunk commands : REX is as follows

 

  • Rex command is used for field extraction in the search head.
  • This command is used to extract the fields using regular expression.
  • This command is also used for replace or substitute characters or digit in the fields by the sed expression.
  • You have to specify any field with it otherwise the regular expression will be applied to the _raw field.

 

Find below the skeleton of the usage of the command “rex” in SPLUNK :

 

rex field=<fieldname>  [(regex-expression) ] [ mode=sed <sed-expression>]

 

Basic syntax of the rex command

rex

Suppose we have a data which is coming from any of the indexes. Below  we have given a sample data.

127.0.0.1 – admin [08/Sep/2018:12:46:37.223 +0530] “GET /en-US/splunkd/__raw/services/search/shelper?output_mode=json&snippet= true&snippetEmbedJS=
false&namespace= search&search=search+index%3D_internal+sourcetype%
3Dsplunkd_ui_access+ %7C+rex+field%3D_raw+%22.*%5Cs%2B%5C%5B
(%3F%3CDATE%3E%5Cd %2B%5C%2F%5Cw%2B%5C%2F%5Cd%2B)
%5C%3A%5Cd%2B.*%22& useTypeahead=true&show
CommandHelp=true&show CommandHistory=true&
showFieldInfo=false&_= 1536390799087 HTTP/1.1″ 200 29647 “-” “
Mozilla/5.0 (Windows NT 6.3;  WOW64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/ 68.0.3440.106 Safari
/537.36″ – 96c643368d868c21de48395bc54c65d6 11ms

*************************************************************************************

Example 1:

Suppose we want to extract 08/Sep/2018 as DATE.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw ".*\s+\[(?<DATE>\d+\/\w+\/\d+)\:\d+.*"
| table DATE
| dedup DATE

Result:

rexdate

Explanation:

In the above query we are getting data from the “_internal” index and sourcetype name is “splunkd_ui_access”. We have extracted the date from the raw log so we have put “field=_raw” with the “rex” command and the new field name is “DATE”. Then by the “table” command we have taken “DATE” and by the “dedup” command we have removed the duplicate values.

*************************************************************************************

Example 2:

Suppose we want to extract 127.0.0.1 as IP.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw "(?<IP>\d+\.\d+\.\d+\.\d+)\s+.*"
| table IP
| dedup IP

Result:

rexip

Explanation:

In the above query we are getting data from the “_internal” index and sourcetype name is “splunkd_ui_access”. We have extracted the ip from the raw log so we have put “field=_raw” with the “rex” command and the new field name is “IP”. Then by the “table” command we have taken “IP” and by the “dedup” command we have removed the duplicate values.

*************************************************************************************

Example 3:

We  can extract multiple fields by the rex command.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=_raw "(?<IP>\d+\.\d+\.\d+\.\d+)\s+.*\[(?<DATE>\d+\/\w+\/\d+)\:\d+.*"
| table DATE,IP
| dedup DATE,IP

Result:

rexmulti

Explanation:

In the above query we are getting data from the “_internal” index and sourcetype name is “splunkd_ui_access”. We have extracted the ip and date  from the raw log so we have put “field=_raw” with the “rex” command and the new field names are “IP” and “DATE”. Then by the “table” command we have taken the “IP” and “DATE”. At last by the “dedup “ command we have removed the duplicate values.

*************************************************************************************

Example 4:

Suppose we want to extract first two character from any of the existing field.

Query:

index=_internal sourcetype=splunkd_ui_access
| rex field=method "(?<FIRST_TWO_LETTER>\w\w).*"
| table method , FIRST_TWO_LETTER
| dedup method , FIRST_TWO_LETTER

Result:

rexmethod

Explanation:

In the above query “method” is an existing field name in “_internal” index and sourcetype name is “splunkd_ui_access” .

Here we have extracted the first two character from each of the “method” field values by the “rex” command . So we have given “field=method” for extracting the values from the “method” field and the new field name is “FIRST_TWO_LETTER”. By the “table” command we have taken “method” and “FIRST_TWO_LETTER “. At last by the “dedup” command we have taken the unique values.

*************************************************************************************

Example 5:

By the rex command we can also replace characters in a field.

Query:

index=_internal sourcetype=splunkd_ui_access
| eval CLIENT_IP=clientip
| rex field=clientip mode=sed "s/(\d{3})/XXX/g"
| table CLIENT_IP,clientip
|dedup CLIENT_IP,clientip

Result:

rexsed

Explanation:

In the above query we are getting data from the “_internal” index and sourcetype name is “splunkd_ui_access”. Here “clientip” is the existing field name and by the “eval” command we have taken the values of “clientip” into the “CLIENT_IP” field.Here we have used “mode=sed” with the “rex” command for replacing the first part of the “clientip” field by “XXX” .For that we have written a sed expression.By the “table” command we have taken the “clientip” and “CLIENT_IP” field.At last by the “dedup” command we have removed the duplicate values.

 

Now you can effectively utilize “rex”  command in  your daily use to meet your requirement !!

 

 Hope you are now comfortable in : Usage of Splunk commands  : REX

 

HAPPY SPLUNKING !!

 

Advertisements