05 May 2018

Kubernetes in a private AWS network

I have been working on setting up a kubernetes cluster on AWS. Usually, the setup isn't difficult and there are many tools that can assist, for example, kops, spray, conjure-up and probably many others I am forgetting. The problem I had with these tools is that they are configured to create all the resources a kubernetes cluster might need. For example, trying to use kops to create a cluster in a private subnet fails if no Internet gateway exists, thus it will try to create an internet gateway. But what if you are in a corporate environment and no internet gateway has been provisioned? What if the internet breakout should go through your own data center? Now your options are limited to:

  1. Setting up the cluster manually (hard way)

  2. Dig deep into the kops / spray code and modify it to do what you need 

Obviously, both options are time-consuming. What if there is another way? The semi-automated way. Actually, the creators of kubernetes thought about the case where flexibility and customizability are needed. That's why they gave us kubeadm currently my go-to tool for provisioning and managing kubernetes clusters. I call it the semi-automated way because I have to ssh into the master / nodes and issue kubeadm commands and write some config files but with a little bash scripting and terraform knowledge it's rather easy to automate everything.

I already wrote the automation for you and you can get started by cloning my repo. I'll explain briefly in this post how you can use the code.

  1. Replace the ssh keys in the ssh/ folder with your own or generate new ones.

  2. Take a look at input.tf and change it to your liking. If you want to install Kubernetes 1.10.1, just change the version variable. Obviously here you will need to define the proxy you want your kubernetes cluster to run behind. Declare the proxy variable like: http://PROXY:PORT

  3. Go into the state folder and provision the resources there, e.g. cd state/ && terraform init && terraform plan / apply

  4. Go back and provision the kubernetes cluster with terraform, e.g. terraform init && terraform plan && terraform apply

  5. The previous step should give you the master node IP, (actually the LB IP). Use the IP to configure kubectl

In the background what the code does is utilize redsocks to forward all the traffic from the containers / host through your corporate proxy. Redsocks will not route internal cluster traffic through your proxy. The traffic rules are configured here if you want to have a look.

Using redoscks is essential in this setup. If you decide not to use it you will need to configure every node and every application / container with proxy configurations. This will quickly lead to maintenance nightmares and hard to debug networking problems.


The code assumes you are provisioning a kubernetes cluster with Ubuntu 16.04 as host OS. Further, it assumes that you already have the networking provisioned (VPC, subnets...)

It can also be made better by adding additional checks. It would also be nice to have a choice of CNI, currently it supports only flannel.
Feel free to copy and edit as you like.


9  question_answer

  • said on 12 March 2020 04:56

    Pastilla Cialis Disfuncion Erectil <a href=http://apcialisle.com/#>Cialis</a> Super Avana <a href=http://apcialisle.com/#>online pharmacy</a> Isotretinoin Order

  • said on 09 March 2020 15:13

    Hydrochlorothiazide 25 Mg For Sale <a href=http://apcialisle.com/#>cheap cialis online</a> Cialis O Viagra Cual Es Mejor <a href=http://apcialisle.com/#>cialis 5 mg best price usa</a> Order Now Doryx Boise

  • said on 11 March 2020 13:32

    Levitra Generico Comprare <a href=http://apcialisle.com/#>Buy Cialis</a> Direct Amoxicilina Get In Internet Without Perscription <a href=http://apcialisle.com/#>Buy Cialis</a> Hydrochlorothiazide Want To Buy Mastercard On Line

  • said on 18 March 2020 18:27

    Cialis 20mg Lilly Preisvergleich <a href=https://apcialisle.com/#>viagra vs cialis</a> Erectile Pills Over The Counter <a href=https://apcialisle.com/#>Cialis</a> Cialis 40mm

  • said on 20 March 2020 13:24

    Will Cephalexin Make Canines Tired <a href=https://apcialisle.com/#>Cialis</a> Celebrex Discount Card Pfizer <a href=https://apcialisle.com/#>Cialis</a> Medrol Dose Pack No Prescription

  • said on 21 March 2020 10:22

    Buy Now Online Zentel <a href=https://apcialisle.com/#>generic cialis online canada</a> Cialis Modo De Empleo <a href=https://apcialisle.com/#>purchase cialis online cheap</a> Was Ist Viagra Fur Manner

  • said on 18 March 2020 01:36

    Cheap Brand Name Viagra <a href=https://apcialisle.com/#>Cialis</a> Non Generic Viagra Online Pharmacy <a href=https://apcialisle.com/#>cheap cialis</a> Buy Viagra 100mg Uk

  • said on 25 March 2020 00:56

    Viagra Bestellen Afhalen <a href=https://apcialisle.com/#>Cialis</a> Venta Cialis Barcelona <a href=https://apcialisle.com/#>Cialis</a> Levitra Medicament En Baisse

  • said on 03 April 2020 04:39

    Can I Take Sudafed With Amoxicillin <a href=http://cialibuy.com/#>cialis canada</a> Forum Gel Kamagra <a href=http://cialibuy.com/#>Cialis</a> buy roche accutane online uk

Leave a comment

Your email address will not be published. It's only used for gravatars