Fishbucket in Splunk

Have you ever heard the term fishbucket ? well, some may be aware of this term some maybe not, but anyways you are going to learn  probably something interesting from this post..so keep reading.

What is fishbucket?

The fishbucket is a sub-directory in splunk which monitors/tracks internally how far the  content of your file is indexed in splunk, from where to resume indexing.The fishbucket sub-directory achieves this feature using its two contents seek pointers and CRCs. To see the contents of fishbucket, search “index=_thefishbucket” in your splunk GUI (contents can only be seen in the older versions of splunk). 

>Cyclic Redundancy Check(CRC) and Seek Address/Pointer:

The splunk monitoring processor selects and reads the first 256 bytes of a new file, after that it hashes this data into a begin and end cyclic redundancy check (CRC), this functions as a fingerprint representing the file content. Splunk uses this CRC to look up an entry in a database that contains all the beginning CRCs of files it has seen before. If successful, the lookup returns a few values, but the important ones are a seekAddress/pointers, meaning the number of bytes into the known file that Splunk has already read, and a seekCRC which is a fingerprint of the data at that location.

The default location of fishbucket sub-directory is $SPLUNK_HOME/splunk/var/lib/ .Also, this index is not intended for normal splunkers, is more of a tool for splunk engineers to help them troubleshoot file input issues. 

Hope!! That’s all you needed to know for now, about splunk fishbucket and concepts related to it. we are going to post more stuffs related to the fishbucket, so stay tuned.

Happy Splunking!!