Bootstrapping Autoscaled Instances With Chef

How To Bootstrap Instances With Chef Launched By Autoscaling

Problem:
In Autoscaling, instances come and go at any point of time based on your autoscaling policy but those instances won't have chef-client installed and won't be connected to chef-server. 

Solutions:

1. Load Chef and the configuration into a custom Amazon Machine Image and use this AMI instead of the default base image provided by AWS.

OR

2. Harness the use of "userdata" in AWS:

Place configuration files - "client.rb", "validation.pem", "init.json", "chef_my_org.crt" in a bucket (mybucket). "init.json" is the first role to be applied on the node. Your can get these files from the other server where chef-client is already running.

Use AWS IAM Roles to provide access to the S3 bucket. Create a role and attache the following policy to it:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:Get*",
                "s3:List*"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:s3:::mybucket/*"
        }
    ]
}

Place the following userdata in the launch configuration

#!/bin/bash
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
wget http://sourceforge.net/projects/s3tools/files/s3cmd/1.5.0-beta1/s3cmd-1.5.0-beta1.tar.gz
tar xvfz s3cmd-1.5.0-beta1.tar.gz
cd s3cmd-1.5.0-beta1/
./s3cmd --config /s3cmd-1.5.0-beta1/s3cfg ls s3://mybucket/
./s3cmd --config /s3cmd-1.5.0-beta1/s3cfg --force get s3://mybucket/config_chef_client.sh
chmod +x config_chef_client.sh
./config_chef_client.sh

In the above userdata we are taking config_chef_client.sh script from mybucket and then executing it. The config_chef_client.sh will install chef and place the files client.rb, validation.pem and init.json to the required location and will run the chef client. Below is the config_chef_client.sh script:

#!/bin/bash
# Install chef
curl -L https://www.opscode.com/chef/install.sh | sudo bash
mkdir /etc/chef
mkdir -p /etc/chef/trusted_certs
 
# Get chef files from S3
./s3cmd --config /s3cmd-1.5.0-beta1/s3cfg ls s3://chef-autoconfig/
./s3cmd --config /s3cmd-1.5.0-beta1/s3cfg --force get s3://mybucket/client.rb /etc/chef/client.rb
./s3cmd --config /s3cmd-1.5.0-beta1/s3cfg --force get s3://mybucket/validation.pem /etc/chef/validation.pem
./s3cmd --config /s3cmd-1.5.0-beta1/s3cfg --force get s3://mybucket/init.json /etc/chef/init.json
./s3cmd --config /s3cmd-1.5.0-beta1/s3cfg --force get s3://mybucket/chef_my_org.crt /etc/chef/trusted_certs/chef_my_org.crt
chef-client -j /etc/chef/init.json

Now, you can go ahead launching the instance but make sure to place the userdata either to launch-config or to the instance itself and also make sure to apply the role with the corresponding policy.