What happens when License Master goes Down?
License Master: A License master controls license slaves associated with it,through it you can define license stacks,pools,add license and manage your license slaves.
License Slave: A license slave is a Splunk Enterprise instance controlled by a license master. The license master provides its slaves access to Splunk Enterprise licenses and, in the case of indexers, associated licensing volume.
Relation between the license master and license slaves:
Once you have configured a license master instance and added licenses slaves to it, the license slaves communicate their usage to the license master every minute. If the license master is down or becomes unavailable for any reason, the license slave starts a 72 hour timer. If the license slave cannot communicate with the license master for 72 hours, search is blocked on the license slave (although indexing continues). Users will not be able to search data in the indexes on the license slave until the slave can reach the license master again.
For knowing if a license slave has been unable to reach the license master, look for an event that contains failed to transfer rows in splunkd.log or search for it in the _internal index.
To check the “last successful contact time” of your License Master & any License slave , in the license slave Navigate to:
Settings -> Licensing
You can also use the query shown below to verify the connection between your License Master & slaves, In-case when your License Master is down you can see the similar events as shown in the pic below.
09-28-2018 11:11:19.545 +0000 ERROR LMTracker – failed to send rows, reason=’Unable to connect to license master=https://35.237.235.119:8089 Error connecting: Connect Timeout
Now, we also verified that indexing and searching are not affected when License Master is down, till 72 hours, once the timer runs out, indexing on the slaves continue but the search operation is blocked as mentioned earlier.
So,what needs to be done??
Make sure your License Master is up before 72 hours so that your splunk deployment isn’t affected, if the problem with your License Master is severe, and can’t be resolved in this time period,then consult splunk support,they will help you out with it.
Thanks for going through this post, you can also find a lot of other interesting stuffs related to splunk on this blog.
Happy Splunking!!
