Re-index your data into Splunk
Sometimes, due to some unavoidable reasons data loss may occur while indexing or partial indexing may take place, then you might want to re-index all your data again into Splunk. Following are the techniques to re-index your data:
If you want to clean your existing data from any of your index before going for re-indexing process, use the commands below.
i) # cd $SPLUNK_HOME/bin ii) # ./splunk stop iii) # ./splunk clean eventdata -index <index-name> ( to clean All Indexes, just drop off -index <index-name> ) iv) # ./splunk start
PROCESS 1: Remove/delete the fishbucket sub-directory which should re-index all your data in all of your indexes.
CAUTION : Deleting the fishbucket sub-directory will re-index data coming into all your indexes from that splunk forwarder/instance, thus may severely impact your license usage.
To delete/remove the fishbucket:
1. Move to the directory /opt/splunk/var/lib/splunk (on the instance forwarding data)
2. Delete/Remove the sub-directory fishbucket
See the pictures below for further reference,
–> index contents before deleting the fishbucket
–> deleting/removing the fish bucket
i) #cd $SPLUNK_HOME/var/lib/splunk ii) #rm -rf fishbucket
–> restart your splunk instance ($SPLUNK_HOME/bin/splunk restart)
Now, as soon as your files are updated on the application server, the whole contents of your files will be re-indexed into splunk in their corresponding indexes.
PROCESS 2: Re-index data without deleting the fishbucket/re-index contents of any specific file
There may be situations, when you only want to re-index the data for a particular file, then you can use the command given below to reset btprobe (run the command on the splunk instance forwarding data)
btprobe: It queries the fishbucket for checkpoints stored by monitor inputs. Any changes you make to the fishbucket using btprobe takes effect only after a restart.
CAUTION: You must stop your splunk instance before using btprobe.
i) #cd $SPLUNK_HOME/bin ii) #./splunk stop iii)#./splunk cmd btprobe -d $SPLUNK_HOME/var/lib/splunk/fishbucket/splunk_private_db --file <File Path> --reset iv) #./splunk start
Below are the screenshots for your reference (using the same index for better understanding ) ….
–> index contents before resetting btprobe
–> resetting the btprobe
–> This should re-index the contents of your file
PROCESS 3: Re-index your file contents based on timestamp for which data has not been indexed in splunk
Suppose due to some reason data coming from the file went missing/didn’t get indexed for the timestamps 09/29/18 5:05:XX to 09/29/18 10:12:XX , but after that the indexing process is working normally, then
1) Copy the file contents which haven’t been indexed ( from timestamp 09/29/18 5:05:XX to 09/29/18 10:12:XX ) to a temporary file, say tmp_file.txt
2) Create a new input stanza in “inputs.conf ” for tmp_file.txt
[monitor://absolute path of tmp_file.txt] index = your_index_name sourcetype = source_type_name
3) Restart your splunk instance
Congrats!! NOW, You have the data indexed that was missing from splunk previously.
NOTE: For cases, when you don’t want to Re-index the contents of some specific file(s), even when you delete the fishbucket, visit our post followTail attribute in splunk
Thanks For Reading!! More contents coming soon, subscribe if you haven’t yet, to keep yourself updated with the latest posts on this blog!!!