Spread our blog

Failed to start KV Store process. See mongod.log and splunkd.log for details

Hello everyone !!!

Today we have come with a new and interesting topic of Splunk which will help you in troubleshooting purposes. You may see Failed to start KV Store process. See mongod.log and splunkd.log for details when we open Splunk. Now today we will let you know how to fix this error. The error messages will look like the below images.

1

2

Now we will show you how to solve this one. See the below steps to solve this issue.

Step 1: Open the CLI of this Search Head. Go to bin directory of Splunk.

#cd $SPLKUNK_HOME/bin

3

Step 2: See the status of KV store by using the following command.

#./splunk show kvstore-status -auth <user_name>:<password>

4

You may find the status as failed.

5

Step 3: Check the FQDN ( Fully Qualified Domain Name ) of your server by using the following command.

#hostname --fqdn

You can see the FQDN of this server. Copy this FQDN because we need it shortly.

6

Step 4: Now create a new SSL certificate in the directory called $SPLUNK_HOME$/etc/auth. Run the below command to create an SSL certificate for this server.

#./splunk createssl server -cert 3072 -d /opt/splunk/etc/auth -n server -c <FQDN>

7

We have used FQDN which we had copied earlier.

You can see that one new .pem file has been generated.

8

NOTE : Run this command if the SH is in distributed environment not in cluster environment. We will come back soon with the  solution of  this issue in cluster environment.

Step 5: Restart Splunk once.

#./splunk restart

9

Step 6: Now check the status of kvstore again.

#./splunk show kvstore-status -auth <user_name>:<password>

10

You can see the status as ready along with other information.

You can also know about :  Clone Data to 2 Different Groups of Indexers in Splunk

11

Step 7: If you again login to Splunk, the error messages will be gone as it is solved.

12

Hope from the next time you will find this error  in Splunk you will be able to  solve it immediately. Also remember the thing what we have mentioned above in the NOTE.

Happy Splunking !!

What’s your Reaction?
+1
+1
+1
7
+1
2
+1
+1
+1
1

Spread our blog

5 COMMENTS

  1. It does not work for me. When I ran this command : #./splunk createssl server -cert 3072 -d /opt/splunk/etc/auth -n server -c

    got this :
    Parameters:

    audit-keys|server-cert|web-cert [1024|2048|3072|4096]

    (Defaults to 2048)

    Any advise?

    • There is a typo in the guide.
      Original: ./splunk createssl server -cert 3072 -d /opt/splunk/etc/auth -n server -c

      It should be server-cert with no space.

      Correct: ./splunk createssl server-cert 3072 -d /opt/splunk/etc/auth -n server -c

LEAVE A REPLY

Please enter your comment!
Please enter your name here