16 September 2017

AWS Network Load Balancer

Tags: AWS Cloud

The other day I received an email from AWS with their latest announcements. I went through the email and saw that AWS now offers a new kind of ELB namely the Network Load Balancer (NLB). Compared to the ALB and the ELB classic the NLB is a layer 4 load balancer (transport). When I clicked the link, the AWS webpage states the following "The Network Load Balancer for the Elastic Load Balancing service is designed to handle millions of requests per second while maintaining ultra-low latencies" and I thought to myself why didn't AWS offer this ELB a year ago when I needed a network load balancer :)

A year ago I worked on a project where the requirements were to capture NetFlow traffic and query CPEs (Customer Premises Equipment) of their status through the SNMP protocol. The SNMP part wasn't that big of a challenge but the NetFlow was more of a challenge. Having 10.000 CPE each with at least 10 interfaces bombing your backend is quite a challenge. One of the requirements was data integrity since the data would be used in a commercial manner. Our initial design consisted of having 5x EC2 instances with NetFlow parsers that would parse and forward (produce) the data to an Apache Kafka cluster. The following picture shows a simplified design

The problem with this design is that you have to guess the number of EC2 instances needed to parse and forward the NetFlow data. The reason we couldn't use an AutoScalingGroup (ASG) was that in the router itself NetFlow is configured with an IP, for example in Cisco
ipflow-export destination 2055

This command would configure a Cisco router to send NetFlow to the EC2 instance with IP on port 2055. The Cisco router can take IPs and hostname records as valid destinations but not DNS names.
What we did was we configured 2000 CPEs per EC2 instance. There was a lot of manual tweaking involved since some CPEs generate more flow data than others.
After reading the announcement from AWS I immediately started digging deeper into their NLB service. As it turns out it can solve all the problems we had. We can position the NLB in front of the EC2 instances and let the NLB handle the routing of the UDP NetFlow data. The NLB also enables us to stop trying to predict the number of required EC2 instances. We can now put the EC2 instances in an ASG and scale in and out with the ingested data! But the biggest win, in my opinion, is the need to manually configure the CPEs with the IPs of EC2 instances. With AWS NLB we get a static IP per availability zone! :=)

I welcome the addition of NLB and think the NLB will enable other people working with IoT or similar customer endpoint devices to abstract a layer out of their architecture and let people focus more on their core problems.

8  question_answer

  • said on 05 May 2019 07:22

    Amoxicillin Gaviscon Order Cialis In Usa Vente De Tadalis Sx En France <a href=http://bestviaonline.com>viagra</a> Progesterone In Australia Luton Wirkung Von Viagra Nebenwirkungen

  • said on 29 July 2019 20:31

    Precio Levitra 20 Mg Comprar Cialis Online Paypal Overnight Shipping Of Levaquin <a href=http://buycialcheap.com>cialis 40 mg</a> Cephalexin Food Interactions Breast Cancer Propecia Viagra Da Dolor De Cabeza

  • said on 13 March 2020 13:21

    Amoxil Dosage <a href=http://apcialisle.com/#>Buy Cialis</a> Cialis Serve Ricetta <a href=http://apcialisle.com/#>Buy Cialis</a> Purchasing Isotretinoin Skin Health Medication

  • said on 13 March 2020 16:58

    Cialis Viagra Vendita <a href=http://apcialisle.com/#>Cialis</a> Buy Doxycycline No Rx <a href=http://apcialisle.com/#>Cialis</a> Cialis Natural

  • said on 12 February 2020 08:35

  • said on 08 March 2020 19:38

    Viagra Generic 100mg <a href=http://apcialisle.com/#>Cialis</a> Utilisation Et Effet Du Viagra <a href=http://apcialisle.com/#>Cialis</a> Achat Cialis Moins Cher

  • said on 18 March 2020 05:25

    cheap viagra cheap viagra <a href="http://lalisa.ru #">generic viagra </a> viagra generic viagra pills ufiouas0209sss viagra generic viagra pills <a href=http://stroyrynok-krd.ru>viagra generic </a> viagra pills viagra generic

  • said on 08 April 2020 22:04

Leave a comment

Your email address will not be published. It's only used for gravatars