followTail attribute in Splunk

This post covers some special cases/scenarios in which the attribute ‘followTail’ may come handy.

followTail: An attribute which makes splunk ignore older contents of the file while tailing and ingesting only the new contents. Splunk by default sets ‘followTail=0′ for any inputs.

–> When you don’t want to index the old contents of your file, may be due to the reason that  ingesting the old contents which are of no use are also going to count against your license.

• For instance, the below picture depicts a file with some old contents that we don’t want to ingest in splunk.

• Creating a new input with ‘followTail=1‘ via inputs.conf to forward data from /tmp/abhay.txt

• Restart splunk instance after making changes to  inputs.conf, check if splunk ingested any data from the source.  

NOTE: Since we are using the ‘followTail’ attribute no old data/content must be ingested from that source (/tmp/abhay.txt)

• Adding some new contents to the file

• Now, you can see the new contents being indexed in splunk…RIGHT!!!

–> When You want to re-index your data into splunk but also want to avoid data re-indexing for some of the inputs.

• For instance, the contents of the file ‘test_file.txt’ shouldn’t be re-indexed into splunk even if we delete the fishbucket.

• Below image shows the data of this file in splunk

• Setting ‘followTail=1’ in inputs.conf for this source to avoid re-indexing of the file contents.

Now, deleting the fishbucket to see whether re-indexing takes place (To know more about fishbucket, see our blog post Fishbucket in splunk and Re-index your data into splunk )

• After Restarting splunk adding new contents to the file

VOILA!! only new content being ingested by splunk (No Re-indexing)

Thanks, for going through this post, hope!! this post was helpful enough.

Happy Splunking!!