{"id":10426,"date":"2021-02-04T08:00:53","date_gmt":"2021-02-04T16:00:53","guid":{"rendered":"http:\/\/softwareengineeringdaily.com\/?p=10426"},"modified":"2021-02-04T13:42:07","modified_gmt":"2021-02-04T21:42:07","slug":"debunking-the-three-pillars-of-observability-myth","status":"publish","type":"post","link":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/","title":{"rendered":"Debunking the ‘Three Pillars of Observability’ Myth"},"content":{"rendered":"

Have you heard about the “three pillars of observability” yet? No? The story goes like this:<\/span><\/p>\n

If you’re using microservices, you already know that they’re nearly impossible to understand using a conventional monitoring toolchain: since microservices were literally <\/span>designed<\/span><\/i> to prevent “<\/span>two-pizza<\/span><\/a>” DevOps teams from knowing about each other, it turns out that it’s incredibly difficult for any individual to understand the whole, or even their nearest neighbor in the service graph.<\/span><\/p>\n

Well, Google and Facebook, and Netflix were building microservices before they were called microservices, and I read on Twitter that they already solved all of these problems\u2026 phew! They did it using <\/span>Metrics<\/b><\/a>, Logging, and <\/b>Distributed Tracing<\/b><\/a>, so you should, too \u2013 those are called “<\/span>The Three Pillars of Observability<\/span><\/i>,” and you probably even know what they look like already:<\/span><\/p>\n

\"\"<\/p>\n

Metrics!<\/span><\/i><\/p>\n

\"\"<\/p>\n

Logs!<\/span><\/i><\/p>\n

\"\"<\/p>\n

Traces!<\/span><\/i><\/p>\n

So, if you want to solve <\/span>observability<\/span><\/a> problems like Google and Facebook, and Twitter, it’s simple\u2026 find a metrics provider, a logging provider, a tracing provider, and voila: your DevOps teams will bask in the light of an observable distributed system.<\/span><\/p>\n

Fatal flaws<\/b><\/h2>\n

Perhaps the above is hyperbolic. Still, for those who deployed “the three pillars” as bare technologies, the initial excitement dissipated quickly as fatal flaws emerged.<\/span><\/p>\n

Metrics and cardinality<\/b><\/h3>\n

For Metrics, we all needed to learn a new vocab word: <\/span>cardinality<\/span><\/i><\/a>. The beauty of metrics is that they make it easy to see when something bad happened: the metric looks like a squiggly line, and you can see it jump up (or down) when something bad happens. But diagnosing those anomalous moments is deeply difficult using metrics alone\u2026 the best we can do is to “drill down,” which usually means grouping the metric by a tag, hoping that a specific tag value explains the anomaly, then filtering by that tag and iterating on the drill-down process.<\/span><\/p>\n

“Cardinality” refers to the number of elements in a set. In the case of metrics, cardinality refers to the number of values for any particular metric tag. If there are 5 values, we’re probably fine; 50 might be ok; 500 is probably too expensive; and once we get into the thousands, you simply can’t justify the ROI. Unfortunately, many real-world tags have thousands or millions of values (e.g., user-id, container-id, and so forth), so metrics often prove to be a dead end from an investigative standpoint.<\/span><\/p>\n

Logging volumes with microservices<\/b><\/h3>\n

For Logs, the problem is simpler to describe: they just become too expensive, period. I was at a speaker dinner before a monitoring conference last year, and one of the other presenters \u2013 a really smart, reputable individual who ran the logging efforts for one of today’s most iconic tech companies \u2013 was giving a talk the following day about how to approach logging in a microservices environment. I was excited about the topic and asked him what his basic thesis was. He said, “Oh, it’s very simple: don’t log things anymore.”<\/span><\/p>\n

It’s easy to understand why: if we want to use logs to account for individual transactions (like we used to in the days of a monolithic web server’s request logs), we would need to pay for the following:<\/span><\/p>\n

\"\"<\/p>\n

Logging systems can’t afford to store data about every transaction anymore because the cost of those transactional logs is <\/span>proportional to the number of microservices touched by an average transaction<\/b>. Not to mention that the logs themselves are less useful (independent of cost) due to the analytical need to understand concurrency and causality in microservice transactions. So conventional logging isn’t sufficient in our brave new architectural world.<\/span><\/p>\n

Tracing and foreknowledge<\/b><\/h3>\n

Which brings us to “Distributed Tracing,” a technology specifically developed to address the above problem with logging systems. I built out the <\/span>Dapper<\/span><\/a> project at Google myself. It certainly had its uses, especially for steady-state latency analysis, but we dealt with the data volume problem by applying braindead, entirely random, and very aggressive sampling. This has long been the elephant in the room for distributed tracing, and it’s the reason why Dapper was awkward to apply in on-call scenarios.<\/span><\/p>\n

The obvious answer would be to avoid sampling altogether. For scaled-out microservices, though, the cost is a non-starter. It’s more realistic to <\/span>defer<\/span><\/i> the sampling decision until the transaction has completed: this is an improvement, though that approach masks a crucial question: which traces should we sample, anyway? If we’re restricting our analysis to individual traces, we typically focus on “the slow ones” or those that result in an error; however, performance and reliability problems in production software are typically a byproduct of interference between transactions, and understanding that <\/span>interference<\/span><\/i> involves much more sophisticated sampling strategies that aggregate across related traces that contend for the same resources.<\/span><\/p>\n

In any case, a single distributed trace is occasionally useful, but a bit of a hail mary. Sampling the right distributed traces and extracting meaningful, accessible insights is a broader challenge, yet much more valuable.<\/span><\/p>\n

And about emulating Google (et al.) in the first place\u2026<\/b><\/h3>\n

Another issue with “The Three Pillars” is the very notion that we should always aspire to build software that’s appropriate for the “planet-scale” infrastructure at Google (or Facebook, or Twitter, and so on). Long story short: <\/span>don’t emulate Google<\/b>. This is not a dig on Google \u2013 there are some brilliant people there, and they’ve done some terrific work given their requirements.<\/span><\/p>\n

But! Google’s technologies are built to scale like crazy, and that isn’t necessarily “good”: Jeff Dean (one of those brilliant Googlers who deserves all of the accolades \u2013 he even <\/span>has his own meme<\/span><\/a>) would sometimes talk about how it’s nearly impossible to design a software system that’s appropriate for more than 3-4 orders of magnitude of scale. Further, there is a natural tension between a system’s scalability and its feature set.<\/span><\/p>\n

Google’s microservices generate about 5 <\/span>billion<\/span><\/i> RPCs per second; building observability tools that scale to 5B RPCs\/sec, therefore, boils down to <\/span>building observability tools that are profoundly feature-poor<\/b>. If your organization is doing more like 5 <\/span>million<\/span><\/i> RPCs\/sec, that’s still quite impressive, but you should almost certainly not use what Google uses: at 1\/1000th the scale, you can afford much more powerful features.<\/span><\/p>\n

Bits vs Benefits<\/b><\/h2>\n

So each “pillar” has a fatal flaw (or three), and that’s a problem. The other problem is even more fundamental: Metrics, Logs, and Distributed Traces are <\/span>just bits<\/i><\/b>. Each describes a particular type of data structure, and when we think of them, we tend to think of the most trivial visualization of those data structures: metrics look like squiggly lines; logs look like a chronological listing of formatted strings; and traces look like those nested waterfall timing diagrams.<\/span><\/p>\n

None of the above directly addresses a particular pain point, use case, or business need. That is, with the “three pillars” orthodoxy, <\/span>we implicitly delegate the extraordinarily complex task of actually analyzing the metric, log, and trace data as “an exercise to the reader<\/b>.” And, given the fatal flaws above and the subtle interactions and co-dependencies between these three types of data, our observability suffers greatly as a result.<\/span><\/p>\n

Towards a Better Model<\/b><\/h2>\n

So metrics, logs, and traces aren\u2019t \u201cthe observability,\u201d they\u2019re just the telemetry. Then what is observability? If the raw data isn\u2019t intrinsically valuable, what is? Is Observability just a new word for Monitoring? And can we get it with a single fancy database?<\/span><\/p>\n

These are all excellent questions with non-obvious answers. They\u2019re also out of scope for this article, but you can learn more about all of this here:\u00a0Observability Won\u2019t Replace Monitoring (Because it Shouldn\u2019t)<\/a><\/u>. <\/span>There is a way to make telemetry valuable \u2013 transformative, even \u2013\u00a0but we\u2019ll need a better model for Observability first.<\/p>\n","protected":false},"excerpt":{"rendered":"

Have you heard about the “three pillars of observability” yet? No? The story goes like this: If you’re using microservices, you already know that they’re nearly impossible to understand using a conventional monitoring toolchain: since microservices were literally designed to prevent “two-pizza” DevOps teams from knowing about each other, it turns out that it’s incredibly<\/p>\n","protected":false},"author":61,"featured_media":10442,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_newsletter_tier_id":0,"footnotes":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[1363,83,2143],"tags":[1785,494,1786,436,1497,20,4363],"jetpack_publicize_connections":[],"acf":[],"yoast_head":"\nDebunking the 'Three Pillars of Observability' Myth - Software Engineering Daily<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Debunking the 'Three Pillars of Observability' Myth - Software Engineering Daily\" \/>\n<meta property=\"og:description\" content=\"Have you heard about the “three pillars of observability” yet? No? The story goes like this: If you’re using microservices, you already know that they’re nearly impossible to understand using a conventional monitoring toolchain: since microservices were literally designed to prevent “two-pizza” DevOps teams from knowing about each other, it turns out that it’s incredibly\" \/>\n<meta property=\"og:url\" content=\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/\" \/>\n<meta property=\"og:site_name\" content=\"Software Engineering Daily\" \/>\n<meta property=\"article:author\" content=\"https:\/\/twitter.com\/el_bhs\" \/>\n<meta property=\"article:published_time\" content=\"2021-02-04T16:00:53+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-02-04T21:42:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/02\/lightstep-beware-observability-dogma-hero.png?fit=2040%2C1344&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"2040\" \/>\n\t<meta property=\"og:image:height\" content=\"1344\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Ben Sigelman\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@el_bhs\" \/>\n<meta name=\"twitter:site\" content=\"@software_daily\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Ben Sigelman\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/\"},\"author\":{\"name\":\"Ben Sigelman\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/e68033e70b4c5c46638550d041e66b56\"},\"headline\":\"Debunking the ‘Three Pillars of Observability’ Myth\",\"datePublished\":\"2021-02-04T16:00:53+00:00\",\"dateModified\":\"2021-02-04T21:42:07+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/\"},\"wordCount\":1304,\"publisher\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\"},\"keywords\":[\"Ben Sigelman\",\"Distributed Tracing\",\"Lightstep\",\"Logging\",\"metrics\",\"microservices\",\"three pillars of observability\"],\"articleSection\":[\"All Content\",\"Exclusive Articles\",\"Exclusive Content\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/\",\"url\":\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/\",\"name\":\"Debunking the 'Three Pillars of Observability' Myth - Software Engineering Daily\",\"isPartOf\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#website\"},\"datePublished\":\"2021-02-04T16:00:53+00:00\",\"dateModified\":\"2021-02-04T21:42:07+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/softwareengineeringdaily.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Debunking the ‘Three Pillars of Observability’ Myth\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#website\",\"url\":\"https:\/\/softwareengineeringdaily.com\/\",\"name\":\"Software Engineering Daily\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/softwareengineeringdaily.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#organization\",\"name\":\"Software Engineering Daily\",\"url\":\"https:\/\/softwareengineeringdaily.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1\",\"width\":296,\"height\":139,\"caption\":\"Software Engineering Daily\"},\"image\":{\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/twitter.com\/software_daily\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/e68033e70b4c5c46638550d041e66b56\",\"name\":\"Ben Sigelman\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/4c3d2ea5d2731ddbee5ab2a9b71ae52f?s=96&d=retro&r=pg\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/4c3d2ea5d2731ddbee5ab2a9b71ae52f?s=96&d=retro&r=pg\",\"caption\":\"Ben Sigelman\"},\"description\":\"Ben Sigelman is a cofounder and the CEO at Lightstep, a co-creator of Dapper (Google\u2019s distributed tracing system), and co-creator of the OpenTracing and OpenTelemetry projects (both part of the CNCF).\",\"sameAs\":[\"https:\/\/lightstep.com\/\",\"https:\/\/twitter.com\/el_bhs\",\"https:\/\/www.linkedin.com\/in\/bensigelman\/\",\"https:\/\/www.youtube.com\/lightstephq\"],\"url\":\"https:\/\/softwareengineeringdaily.com\/author\/bsigelman\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Debunking the 'Three Pillars of Observability' Myth - Software Engineering Daily","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/","og_locale":"en_US","og_type":"article","og_title":"Debunking the 'Three Pillars of Observability' Myth - Software Engineering Daily","og_description":"Have you heard about the “three pillars of observability” yet? No? The story goes like this: If you’re using microservices, you already know that they’re nearly impossible to understand using a conventional monitoring toolchain: since microservices were literally designed to prevent “two-pizza” DevOps teams from knowing about each other, it turns out that it’s incredibly","og_url":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/","og_site_name":"Software Engineering Daily","article_author":"https:\/\/twitter.com\/el_bhs","article_published_time":"2021-02-04T16:00:53+00:00","article_modified_time":"2021-02-04T21:42:07+00:00","og_image":[{"width":2040,"height":1344,"url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/02\/lightstep-beware-observability-dogma-hero.png?fit=2040%2C1344&ssl=1","type":"image\/png"}],"author":"Ben Sigelman","twitter_card":"summary_large_image","twitter_creator":"@el_bhs","twitter_site":"@software_daily","twitter_misc":{"Written by":"Ben Sigelman","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/#article","isPartOf":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/"},"author":{"name":"Ben Sigelman","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/e68033e70b4c5c46638550d041e66b56"},"headline":"Debunking the ‘Three Pillars of Observability’ Myth","datePublished":"2021-02-04T16:00:53+00:00","dateModified":"2021-02-04T21:42:07+00:00","mainEntityOfPage":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/"},"wordCount":1304,"publisher":{"@id":"https:\/\/softwareengineeringdaily.com\/#organization"},"keywords":["Ben Sigelman","Distributed Tracing","Lightstep","Logging","metrics","microservices","three pillars of observability"],"articleSection":["All Content","Exclusive Articles","Exclusive Content"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/","url":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/","name":"Debunking the 'Three Pillars of Observability' Myth - Software Engineering Daily","isPartOf":{"@id":"https:\/\/softwareengineeringdaily.com\/#website"},"datePublished":"2021-02-04T16:00:53+00:00","dateModified":"2021-02-04T21:42:07+00:00","breadcrumb":{"@id":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/softwareengineeringdaily.com\/2021\/02\/04\/debunking-the-three-pillars-of-observability-myth\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/softwareengineeringdaily.com\/"},{"@type":"ListItem","position":2,"name":"Debunking the ‘Three Pillars of Observability’ Myth"}]},{"@type":"WebSite","@id":"https:\/\/softwareengineeringdaily.com\/#website","url":"https:\/\/softwareengineeringdaily.com\/","name":"Software Engineering Daily","description":"","publisher":{"@id":"https:\/\/softwareengineeringdaily.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/softwareengineeringdaily.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/softwareengineeringdaily.com\/#organization","name":"Software Engineering Daily","url":"https:\/\/softwareengineeringdaily.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/","url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1","contentUrl":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2022\/01\/cropped-logo-new.png?fit=296%2C139&ssl=1","width":296,"height":139,"caption":"Software Engineering Daily"},"image":{"@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/twitter.com\/software_daily"]},{"@type":"Person","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/e68033e70b4c5c46638550d041e66b56","name":"Ben Sigelman","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/softwareengineeringdaily.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/4c3d2ea5d2731ddbee5ab2a9b71ae52f?s=96&d=retro&r=pg","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4c3d2ea5d2731ddbee5ab2a9b71ae52f?s=96&d=retro&r=pg","caption":"Ben Sigelman"},"description":"Ben Sigelman is a cofounder and the CEO at Lightstep, a co-creator of Dapper (Google\u2019s distributed tracing system), and co-creator of the OpenTracing and OpenTelemetry projects (both part of the CNCF).","sameAs":["https:\/\/lightstep.com\/","https:\/\/twitter.com\/el_bhs","https:\/\/www.linkedin.com\/in\/bensigelman\/","https:\/\/www.youtube.com\/lightstephq"],"url":"https:\/\/softwareengineeringdaily.com\/author\/bsigelman\/"}]}},"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/softwareengineeringdaily.com\/wp-content\/uploads\/2021\/02\/lightstep-beware-observability-dogma-hero.png?fit=2040%2C1344&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p7GuoD-2Ia","_links":{"self":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts\/10426"}],"collection":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/users\/61"}],"replies":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/comments?post=10426"}],"version-history":[{"count":0,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/posts\/10426\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/media\/10442"}],"wp:attachment":[{"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/media?parent=10426"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/categories?post=10426"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/softwareengineeringdaily.com\/wp-json\/wp\/v2\/tags?post=10426"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}