Kibana4 apache configuration with authentication

We need to configure httpd to listen on port 80 which does a reverse proxy to localhost on 5601.
Make sure kibana is listening only on internal port before this settings are done .


        DocumentRoot /var/www/auth

            AuthType Basic
            AuthName "Authentication Required"
            AuthUserFile "/var/www/auth/htpasswd"
            Require valid-user

        ProxyPass / http://localhost:5601/
        ProxyPassReverse / http://localhost:5601/

Use below command to create a htpasswd file and enable authentication 
htpasswd -c /var/www/auth/htpasswd devops

Monitoring of kibana and logstash services and start them if process not found

Kibana and logstash might run out of memory and application might get closed or get terminated. In that case we will be running the below script to monitor services and start them if not running .
There are other parameters that need to be verified before going for this option like java heap size etc.



var1=`ps -ef|grep -i $LOGSTASH|sed -e '/grep/d'`

if [ -z "$var1" ]; then

nohup /opt/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf &

echo "Process logstash not found" | mail -s "Process logstash not found creating a new one"


echo "process logstash  found."


var2=`ps -ef|grep -i $KIBANA|sed -e '/grep/d'`

if [ -z "$var2" ]; then

nohup /opt/kibana/bin/kibana status &

echo "Process kibana  not found" | mail -s "Process kibana not found creating a new one"


echo "process kibana found."


Chef provision a esxi virtual machine or a virtual server

We need to install esxi knife plugin on our workstation

gem install knife-esx

Add below params in knife.rb

knife[:esx_host] = "esx"
knife[:esx_username] = "root"
knife[:esx_password] = "password of your esxi server"

Need to create a ubuntu template ( or custom OS template as per requirement )
Below example shows how to provision a ubuntu template .

ssh root@esxi

mkdir -p /vmfs/volumes/datastore1/esx-gem/templates

vmkfstools -i /vmfs/volumes/datastore1/ubuntu-12.10-x64_template/*.vmdk –diskformat thin /vmfs/volumes/datastore1/esx-gem/templates/ubuntu-12.10-x64_template.vmdk

esx template list ( should display the template we installed )

knife esx vm create --vm-name server1 --use-template ubuntu-12.10-x64_template --verbose true --distro ubuntu12.04-19-gems --vm-memory 2048 -x provision -i ~/.ssh/id_rsa

How to use chef encrypted databags

Note: Please comment the below line in knife.rb file else it will automatically decrypt values
and show when we use "knife data bag show" command
I initially thought knife was not encrypting my values and had to debug ..which took lot of time.

[root@ec2-test .chef]# cat knife.rb | grep secret
#knife[:secret_file] ='/root/encrypted_data_bag_secret'

knife data bag create  --secret-file /root/.chef/encrypted_data_bag_secret testbag password

  "id": "password",
  "password": "this is test password key"


[root@ec2-test .chef]# knife data bag show testbag password

id:       password
  cipher:         aes-256-cbc
  encrypted_data: KMHzeFQwfm0wWeHFymxfJsMo425CP+wlwoZ6xN7waVlgUNOmRrr/+jOtDLIN
  iv:             TYPYnSqYTcmU8ZWE2sIt4A==
  version:        1

Once encrypted if you try to edit the encrypted databag it shows like this 

  "name": "data_bag_item_testbag_password",
  "json_class": "Chef::DataBagItem",
  "chef_type": "data_bag_item",
  "data_bag": "testbag",
  "raw_data": {
    "id": "password",
    "password": {
      "encrypted_data": "KMHzeFQwfm0wWeHFymxfJsMo425CP+wlwoZ6xN7waVlgUNOmRrr/+jOtDLIN\ns7Xl\n",
      "iv": "TYPYnSqYTcmU8ZWE2sIt4A==\n",
      "version": 1,
      "cipher": "aes-256-cbc"

To show the decrypted values we can use 

[root@ec2-test .chef]# knife data bag show  testbag password --secret-file /root/.chef/encrypted_data_bag_secret

id:       password
password: this is test password key

Some important commands related to data bags

Generate random secret key

openssl rand -base64 512 >encrypted_data_bag_secret

Generate random password

date | md5 in mac 
date | md5sum in any linux machine