{"id":4201,"date":"2017-11-22T02:00:26","date_gmt":"2017-11-22T10:00:26","guid":{"rendered":"http:\/\/softwareengineeringdaily.com\/?p=4201"},"modified":"2021-11-03T01:22:05","modified_gmt":"2021-11-03T08:22:05","slug":"load-balancing-at-scale-with-vivek-panyam","status":"publish","type":"post","link":"https:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/","title":{"rendered":"Load Balancing at Scale with Vivek Panyam"},"content":{"rendered":"<p><img data-attachment-id=\"2475\" data-permalink=\"https:\/\/softwareengineeringdaily.com\/2016\/04\/19\/googles-container-management-brendan-burns\/brendan-burns\/\" data-orig-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2016\/04\/brendan-burns.jpg?fit=175%2C175&amp;ssl=1\" data-orig-size=\"175,175\" data-comments-opened=\"0\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"brendan-burns\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2016\/04\/brendan-burns.jpg?fit=175%2C175&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2016\/04\/brendan-burns.jpg?fit=175%2C175&amp;ssl=1\" decoding=\"async\" loading=\"lazy\" class=\"alignright size-full wp-image-2475\" style=\"border-radius: 50%; border: 1px solid #000000; max-width: 175px; max-height: 175px;\" src=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2017\/11\/vivekpanyam.jpg?resize=175%2C175&#038;ssl=1\" width=\"175\" height=\"175\" data-recalc-dims=\"1\" \/><\/p>\n<p><span style=\"font-weight: 400;\">Facebook serves interactive content to billions of users. Google serves query requests on the world\u2019s biggest search engine. Uber handles a significant percentage of the transportation within the United States. These services are handling radically different types of traffic, but many of the techniques they use to balance loads are similar. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Vivek Panyam is an engineer with Uber, and he previously interned at Google and Facebook. In a popular <\/span><a href=\"https:\/\/blog.vivekpanyam.com\/scaling-a-web-service-load-balancing\/\"><span style=\"font-weight: 400;\">blog post about load balancing at scale<\/span><\/a><span style=\"font-weight: 400;\">, he described how a large company scales up a popular service. The methods for scaling up load balancing are simple, but effective&#8211;and they help to illustrate how load balancing works at different layers of the networking stack.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Let\u2019s say you have a simple service where a user makes a request, and your service sends them a response with a cat picture. Your service starts to get popular, and begins timing out and failing to send a response to users.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">When your service starts to get overwhelmed, you can scale up load by creating another service instance that is a copy of your cat picture service. Now you have two service instances, and you can use a layer 7 load balancer to route traffic evenly between those two service instances. You can keep adding service instances as the load scales and have the load distributed among those new instances. <\/span><\/p>\n<p><span style=\"font-weight: 400;\">Eventually, your L7 load balancer is handling so much traffic itself that you can\u2019t put any more service instances in front of it. So you have to set up another L7 load balancer, and put an L4 load balancer in front of those L7 load balancers. You can scale up that tier of L7 load balancers, each of which is balancing traffic across a set of your service instances. But eventually, even your L4 load balancer gets overwhelmed with requests for cat pictures. You have to set up another tier, this time with L3 load balancing\u2026<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In this episode, Vivek gives a clear description for how load balancing works. We also review the 7 networking layers before discussing why there are different types of load balancers associated with the different networking layers.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Facebook serves interactive content to billions of users. Google serves query requests on the world\u2019s biggest search engine. Uber handles a significant percentage of the transportation within the United States. These services are handling radically different types of traffic, but many of the techniques they use to balance loads are similar. Vivek Panyam is an<\/p>\n","protected":false},"author":3,"featured_media":4205,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"[podcast] Load Balancing at Scale with @VivekPanyam @UberATG #Scalability","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[1363,1079,14],"tags":[1595,250,40],"jetpack_publicize_connections":[],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v21.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Load Balancing at Scale with Vivek Panyam - Software Engineering Daily<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Load Balancing at Scale with Vivek Panyam - Software Engineering Daily\" \/>\n<meta property=\"og:description\" content=\"Facebook serves interactive content to billions of users. Google serves query requests on the world\u2019s biggest search engine. Uber handles a significant percentage of the transportation within the United States. These services are handling radically different types of traffic, but many of the techniques they use to balance loads are similar. Vivek Panyam is an\" \/>\n<meta property=\"og:url\" content=\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/\" \/>\n<meta property=\"og:site_name\" content=\"Software Engineering Daily\" \/>\n<meta property=\"article:published_time\" content=\"2017-11-22T10:00:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-11-03T08:22:05+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2017\/11\/layer_7_load_balancing.png?fit=699%2C315\" \/>\n\t<meta property=\"og:image:width\" content=\"699\" \/>\n\t<meta property=\"og:image:height\" content=\"315\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"SE Daily\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@software_daily\" \/>\n<meta name=\"twitter:site\" content=\"@software_daily\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"SE Daily\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/#article\",\"isPartOf\":{\"@id\":\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/\"},\"author\":{\"name\":\"SE Daily\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8\"},\"headline\":\"Load Balancing at Scale with Vivek Panyam\",\"datePublished\":\"2017-11-22T10:00:26+00:00\",\"dateModified\":\"2021-11-03T08:22:05+00:00\",\"mainEntityOfPage\":{\"@id\":\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/\"},\"wordCount\":364,\"publisher\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\"},\"keywords\":[\"load balancing\",\"Scalability\",\"Uber\"],\"articleSection\":[\"All Content\",\"Cloud Engineering\",\"Podcast\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/\",\"url\":\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/\",\"name\":\"Load Balancing at Scale with Vivek Panyam - Software Engineering Daily\",\"isPartOf\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#website\"},\"datePublished\":\"2017-11-22T10:00:26+00:00\",\"dateModified\":\"2021-11-03T08:22:05+00:00\",\"breadcrumb\":{\"@id\":\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/softwareengineeringdaily.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Load Balancing at Scale with Vivek Panyam\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#website\",\"url\":\"https:\/\/softwareengineeringdaily.com\/\",\"name\":\"Software Engineering Daily\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/softwareengineeringdaily.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\",\"name\":\"Software Engineering Daily\",\"url\":\"https:\/\/softwareengineeringdaily.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1\",\"width\":296,\"height\":139,\"caption\":\"Software Engineering Daily\"},\"image\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/twitter.com\/software_daily\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8\",\"name\":\"SE Daily\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b92f4cf3dc4d94f73834f83e2a22a372?s=96&d=retro&r=pg\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b92f4cf3dc4d94f73834f83e2a22a372?s=96&d=retro&r=pg\",\"caption\":\"SE Daily\"},\"description\":\"The SE Daily podcast.\",\"sameAs\":[\"https:\/\/softwareengineeringdaily.com\"],\"url\":\"https:\/\/softwareengineeringdaily.com\/author\/erikawho\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Load Balancing at Scale with Vivek Panyam - Software Engineering Daily","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/","og_locale":"en_US","og_type":"article","og_title":"Load Balancing at Scale with Vivek Panyam - Software Engineering Daily","og_description":"Facebook serves interactive content to billions of users. Google serves query requests on the world\u2019s biggest search engine. Uber handles a significant percentage of the transportation within the United States. These services are handling radically different types of traffic, but many of the techniques they use to balance loads are similar. Vivek Panyam is an","og_url":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/","og_site_name":"Software Engineering Daily","article_published_time":"2017-11-22T10:00:26+00:00","article_modified_time":"2021-11-03T08:22:05+00:00","og_image":[{"width":699,"height":315,"url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2017\/11\/layer_7_load_balancing.png?fit=699%2C315","type":"image\/png"}],"author":"SE Daily","twitter_card":"summary_large_image","twitter_creator":"@software_daily","twitter_site":"@software_daily","twitter_misc":{"Written by":"SE Daily","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/#article","isPartOf":{"@id":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/"},"author":{"name":"SE Daily","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8"},"headline":"Load Balancing at Scale with Vivek Panyam","datePublished":"2017-11-22T10:00:26+00:00","dateModified":"2021-11-03T08:22:05+00:00","mainEntityOfPage":{"@id":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/"},"wordCount":364,"publisher":{"@id":"https:\/\/softwareengineeringdaily.com\/#organization"},"keywords":["load balancing","Scalability","Uber"],"articleSection":["All Content","Cloud Engineering","Podcast"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/","url":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/","name":"Load Balancing at Scale with Vivek Panyam - Software Engineering Daily","isPartOf":{"@id":"https:\/\/softwareengineeringdaily.com\/#website"},"datePublished":"2017-11-22T10:00:26+00:00","dateModified":"2021-11-03T08:22:05+00:00","breadcrumb":{"@id":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/softwareengineeringdaily.com\/2017\/11\/22\/load-balancing-at-scale-with-vivek-panyam\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/softwareengineeringdaily.com\/"},{"@type":"ListItem","position":2,"name":"Load Balancing at Scale with Vivek Panyam"}]},{"@type":"WebSite","@id":"https:\/\/softwareengineeringdaily.com\/#website","url":"https:\/\/softwareengineeringdaily.com\/","name":"Software Engineering Daily","description":"","publisher":{"@id":"https:\/\/softwareengineeringdaily.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/softwareengineeringdaily.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/softwareengineeringdaily.com\/#organization","name":"Software Engineering Daily","url":"https:\/\/softwareengineeringdaily.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/","url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1","contentUrl":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1","width":296,"height":139,"caption":"Software Engineering Daily"},"image":{"@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/twitter.com\/software_daily"]},{"@type":"Person","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/822f06fe7d6f895baba29a9c0a3aa6c8","name":"SE Daily","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b92f4cf3dc4d94f73834f83e2a22a372?s=96&d=retro&r=pg","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b92f4cf3dc4d94f73834f83e2a22a372?s=96&d=retro&r=pg","caption":"SE Daily"},"description":"The SE Daily podcast.","sameAs":["https:\/\/softwareengineeringdaily.com"],"url":"https:\/\/softwareengineeringdaily.com\/author\/erikawho\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2017\/11\/layer_7_load_balancing.png?fit=699%2C315&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p7GuoD-15L","_links":{"self":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts\/4201"}],"collection":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/comments?post=4201"}],"version-history":[{"count":0,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts\/4201\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/media\/4205"}],"wp:attachment":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/media?parent=4201"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/categories?post=4201"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/tags?post=4201"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}