Failed to start KV Store process. See mongod.log and splunkd.log for details

Failed to start KV Store process. See mongod.log and splunkd.log for details

Hello everyone !!!

Today we have come with a new and interesting topic of Splunk which will help you in troubleshooting purposes. You may see Failed to start KV Store process. See mongod.log and splunkd.log for details when we open Splunk. Now today we will let you know how to fix this error. The error messages will look like the below images.

1

2

Now we will show you how to solve this one. See the below steps to solve this issue.

Step 1: Open the CLI of this Search Head. Go to bin directory of Splunk.

#cd $SPLKUNK_HOME/bin

3

Step 2: See the status of KV store by using the following command.

#./splunk show kvstore-status -auth <user_name>:<password>

4

You may find the status as failed.

5

Step 3: Check the FQDN ( Fully Qualified Domain Name ) of your server by using the following command.


#hostname --fqdn

You can see the FQDN of this server. Copy this FQDN because we need it shortly.

6

Step 4: Now create a new SSL certificate in the directory called $SPLUNK_HOME$/etc/auth. Run the below command to create an SSL certificate for this server.

#./splunk createssl server -cert 3072 -d /opt/splunk/etc/auth -n server -c <FQDN>

7

We have used FQDN which we had copied earlier.

You can see that one new .pem file has been generated.

8

NOTE : Run this command if the SH is in distributed environment not in cluster environment. We will come back soon with the  solution of  this issue in cluster environment.

Step 5: Restart Splunk once.

#./splunk restart

9

Step 6: Now check the status of kvstore again.

#./splunk show kvstore-status -auth <user_name>:<password>

10

You can see the status as ready along with other information.

11

Step 7: If you again login to Splunk, the error messages will be gone as it is solved.

12

Hope from the next time you will find this error  in Splunk you will be able to  solve it immediately. Also remember the thing what we have mentioned above in the NOTE.

Happy Splunking !!

Advertisements

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.