Sunday 29 October 2017

How To Setup Kubernetes Cluster On AWS Using KOPS

Setup Kubernetes Cluster

#What is Kubernetes?

"Kubernetes is an open source system for automating deployments, scaling and management of containerised applications."
In this post, we will learn how to setup kubernetes cluster on AWS using KOPS (kubernetes operations).
NOTE: Before proceeding, we assume that you have a basic understanding about Kubernetes and AWS.

#Prerequisites:

  • Ubuntu instance
  • AWS-cli setup
  • S3 bucket

#Video Tutorial:


#Install kubectl

On your ubuntu instance, make sure AWS cli is setup and also the kops binary. We shall also need kubectl (Kubernetes cli)

- macOS: brew install kubernetes-cli

- Linux:
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl

chmod +x ./kubectl

sudo mv ./kubectl /usr/local/bin/kubectl
Next, we will install kops on ubuntu instance:

wget https://github.com/kubernetes/kops/releases/download/1.6.1/kops-linux-amd64
chmod +x kops-linux-amd64
sudo mv kops-linux-amd64 /usr/local/bin/kops

#Create Route53 domain for the cluster

kubernetes uses DNS for discovery inside the cluster so that you can reach out kubernetes API server from clients.
create a hosted zone on Route53, say, k8s.appychip.vpc. The API server endpoint will then be api.k8s.appychip.vpc

#Create a S3 bucket

Now, create a S3 bucket which will keep the configuration for the cluster:

$ aws s3 mb s3://clusters.k8s.appychip.vpc
Expose environment variable:
$ export KOPS_STATE_STORE=s3://clusters.k8s.appychip.vpc

#Create Kubernetes Cluster

Now we’re ready to create a cluster. You can reuse existing VPC (kops will create a new subnet in this VPC) by providing the vpc-id option:

$ kops create cluster --cloud=aws --zones=us-east-1d --name=useast1.k8s.appychip.vpc --dns-zone=appychip.vpc --dns private

NOTE: Make sure you have ssh keys already generated otherwise it will throw an error.

To actually create cluster run:

kops update cluster useast1.k8s.appychip.vpc --yes
This will do all the required stuff of creating the VPC, subnets, autoscaling-groups, nodes etc. which you can observe in the output. If you want to review what all things going to happen when this command would be run then run the above command without --yes option. Without --yes option, it will print the action it is going to perform without actually doing it.
You can then edit the cluster settings with one of these commands:
  • List clusters with: kops get cluster
  • Edit this cluster with: kops edit cluster useast1.k8s.appychip.vpc
  • Edit your node instance group: kops edit ig --name=useast1.k8s.appychip.vpc nodes
  • Edit your master instance group: kops edit ig --name=useast1.k8s.appychip.vpc master-us-east-1d
Then wait, it takes quite some time for the instances to boot and the DNS entries to be added in the zone. Once everything is up you should be able to get the kubernetes nodes:
$ kubectl get nodes

NAME                          STATUS AGE VERSION
ip-172-20-33-144.ec2.internal Ready  4m  v1.6.2
ip-172-20-39-78.ec2.internal  Ready  1m  v1.6.2
ip-172-20-45-174.ec2.internal Ready  2m  v1.6.2
To enable the Kubernetes UI you need to install the UI service:
$ kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml
Then you can use the kubctl proxy to access the UI from your machine:
$ kubectl proxy --port=8080 &
The UI should now be available at http://localhost:8080

#Deploying Nginx Container

To test our new Kubernetes cluster, we could deploy a simple service made up of some nginx containers:

Create an nginx deployment:
$ kubectl run sample-nginx --image=nginx --replicas=2 --port=80
$ kubectl get pods

NAME                       READY     STATUS    RESTARTS   AGE
sample-nginx-379829228-xb9y3   1/1       Running   0          10s
sample-nginx-379829228-yhd25   1/1       Running   0          10s

$ kubectl get deployments

NAME       DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
sample-nginx   2         2         2            2           29s
Expose the deployment as service. This will create an ELB in front of those 2 containers and allow us to publicly access them:
$ kubectl expose deployment sample-nginx --port=80 --type=LoadBalancer

$ kubectl get services -o wide

NAME         CLUSTER-IP      EXTERNAL-IP                                                              PORT(S)   AGE       SELECTOR
kubernetes   100.64.0.1      <none>                                                                   443/TCP   25m       <none>
sample-nginx     100.70.129.69   adca6650a60e611e7a66612ae64874d4-175711331.us-east-1.elb.amazonaws.com/   80/TCP    19m       run=sample-nginx
There is an ELB running on http://adca6650a60e611e7a66612ae64874d4-175711331.us-east-1.elb.amazonaws.com with our nginx containers behind it:
You can also view the UI by accessing master node. Hit master node's IP/Domain in browser, it will ask for credentials. Run command kubectl config view to see the credentials.

Now, to get rid of the cluster we can completely remove all AWS resources with:

26 comments:

  1. Thanks for giving a great information about DevOps Good Explination nice Article
    anyone want to learn advance devops tools or devops online training
    DevOps Online Training

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. Hi, I have created a kops cluster by following the tutorial. When I tried to validate, It's been throwing an error "unexpected error during validation: unable to resolve Kubernetes cluster API URL dns: lookup api.ovgu-thesis.de on [fd00::ca0e:14ff:feac:d12b]:53: server misbehaving". I did dig NS also. It's been creating the pre-records. Can you help me out?

    ReplyDelete
  4. Hi - where did you specify in create cluster command that the setup needs to have 1 master and 2 nodes of medium .

    ReplyDelete
  5. I followed artical and I'm trying to install kubernetes cluster on AWS(ubuntu,redhat,Amazon Linux) but kubectl is not working I have opened all the required ports still i'm facing issues
    error: The connection to the server localhost:8080 was refused - did you specify the right host or port?(I opened the port 8080)
    and I have given required IAM roles but still i am unable to create the cluster .

    while i tried to create cluster but effected with error.
    error:error running task "IAMRolePolicy/masters.useast1.k8s.appychip.vpc" (8m49s remaining to succeed): error rendering PolicyDocument: error opening resource: DNS ZoneID not set

    ReplyDelete

  6. Excellent blog with top quality information and enjoyed reading waiting for next blog thank you.
    Data Analytics Course Online

    ReplyDelete
  7. We are used to the fact that we know only religious and public holidays and celebrate only them.Iamlinkfeeder Iamlinkfeeder Iamlinkfeeder

    ReplyDelete
  8. The Extraordinary blog went amazed with the content that they have developed in a very descriptive manner. This type of content surely ensures the participants explore themselves. Hope you deliver the same near the future as well. Gratitude to the blogger for the efforts.

    Machine Learning Course in Bangalore


    ReplyDelete
  9. In the last few years, web and hosting technologies have made a lot of leaps. Technologies have quickly become redundant and web hosting companies have tried to offer new and improved services to their clients. https://onohosting.com/

    ReplyDelete
  10. KGF 2 Release Date : Directed by Prashanth Neel. With Yash, Sanjay Dutt, Raveena Tandon, Prakash Raj. The blood-soaked land of Kolar Gold Fields

    ReplyDelete
  11. Thanks so much for sharing this incredible information it's really very helpful I really appreciate your hard work for clearing this article once again thanks

    It would be great if you leave your review on this article that will help me to modify this article for better User experience thanks in advance.

    Fat around women breast

    ReplyDelete
  12. DIYAM Impex Our Company Lab Grown Diamond Manufacturer. We have gone from strength to strength over the years, having expanded from our core business of diamond manufacturing to Real Estate, Renewable Energy and Venture Capital. Diamond has its many utility and its industrial value is enhanced by our effective services. We now focus exclusively on Lab Grown Diamonds. DIYAM IMPEX has grown to become a globally trusted and respected player in the diamond industry over the last five decades. Our expertise lies in our ability to produce a consistent supply of quality polished diamonds in all shapes and sizes.

    ReplyDelete
  13. Thank for sharing wonderful blog, it is great information. anyone looking SEO Company in Inida

    ReplyDelete
  14. best blog i have ever read , anyone looking for an digital marketing expert in delhi This

    ReplyDelete
  15. A neat and clean house is a symbol of a healthy life. If you are not happy with the maid’s cleaning tasks then consult us. We, Safaiwale provide you the best world-class standards deep House Cleaning Services In Dwarkaat affordable rates with flexible times.

    ReplyDelete
  16. It is Very Useful information about Fast Food. If you love Fast Food Visit Xero Degrees

    ReplyDelete
  17. This comment has been removed by the author.

    ReplyDelete
  18. This comment has been removed by the author.

    ReplyDelete
  19. This post is very easy to read and appreciate without leaving any details out.
    Deep home cleaning services in Zirakpur

    ReplyDelete
  20. Unified communications and Ip Pbx includes the connection of various communication systems both for the collaboration tools as the digital workforce.

    ReplyDelete

 

Copyright @ 2013 Appychip.

Designed by Appychip & YouTube Channel