EnvoyCon North America 2022

Over the last year, Envoy's upstream feature set has grown dramatically due to the productionization of Envoy Mobile. New features such as automatic upstream protocol selection using ALPN, HTTP/3 upstream with seamless TCP fail-over, and Happy Eyeballs support have enriched not just Envoy Mobile, but Envoy as a dynamic forward proxy. This talk will run through these new features, and how they can improve your Envoy deployments.

How can Envoy protect itself from OOMs? Envoy has a number of different protection mechanisms out-of-the-box -- how do they work? When should you use them and how should they be configured? Let's find out! Kevin will conclude with some experimental results using these protection mechanisms.

Migrations are one of the most challenging tasks we do as infrastructure engineers.
These are sometimes long, tedious and come with many technical challenges of their own.
At Slack, we switched from HAProxy to Envoy Proxy for all ingress traffic. Overall, this migration was a success, and did not cause any downtime, but even so, we ran into several interesting edge cases that caused minor problems, such as failing a small percentage of requests, or increasing latency for requests, or sometimes an unhappy bot.

Troubleshooting these sorts of 'gray' failures can be difficult, so this talk will discuss some of those facepalm moments: how they were detected, steps taken to investigate them, and how they were solved.

Takeaways from this talk include a specific set of approaches for debugging such problems with Envoy Proxy and other web proxies that we learnt via these events along with some engineering practices that eases the stress during a large migration.

Monitoring and debugging modern cloud-based applications is challenging due to their highly distributed nature. End-to-end distributed tracing (tracing individual calls through a request's lifecycle) has emerged to be essential in a developer's toolbox as they're critical to describing a request's flow through a microservice. Unfortunately, to enable distributed tracing, each service along the request tree needs to propagate a global request ID to help link the related requests (i.e., link incoming requests with spawned backend requests). Enabling such header propagation can be non-trivial for large microservices with 100+ independent services or for legacy apps where instrumentation is hard. This talk explores whether distributed tracing can be made more accessible by eliminating the need for application instrumentation. We describe a method that combines observations external to the app (using an envoy-based service mesh) with timing analysis of the requests to construct end-to-end traces. In an evaluation with a simple microservice, this preliminary method boosts trace reconstruction accuracy to 96% (compared to 77% for a baseline), and can help answer useful developer queries.

Majority of Envoy’s access control policies are defined using the path component of the request URL. How can we be sure that URL Path Based Access Policies (PBAC) can not be bypassed? Two nominally different URL paths may in fact identify the same resource, and need to be normalized to their canonical form, before comparison. The normalization is standardized in RFC 3986, however it may not be enough to ensure safety of the access control. In this talk we explore the effects of URL path normalization on request access control, Envoy’s configuration options for path normalization and general principles for ensuring the safety of the PBAC policies.

Effective traffic management is the key to allow customers to ensure that their micro-services and overall architecture are highly available and highly reliable . How to design the rate limiting system for distributed systems handling millions of requests per second at Google scale? How to make the rate limiting system smarter to ensure fair sharing between various clients and handle the service surge in a particular region? This talk will go into details about the design of this more scalable, intelligent and performant rate limiting service and how your own service can benefit from it.

Envoy allows for modifying HTTP headers when sending requests upstream and responses downstream. The syntax used for creating header content is the same as for creating access log’s entry. But surprisingly, only a small subset of formatters were available for header modification compared to the full set of access log’s formatters. This has changed and now all formatters can be used in both, in header manipulation and in access logs. This talk describes logic used when creating access logs and when manipulating headers. It also explains why using the same formatter in the access log can render different results than using it in the header. Note: This feature is still under development, but I hope that it will be finished and merged to main before the conference. See https://github.com/envoyproxy/envoy/pull/21932