EnvoyCon North America 2022

There are two locations at Huntington Place where you can go through Health + Safety to show proof of vaccination or negative COVID-19 test and pick up your badge:

• Corner entrance on the cityside @ the corner of W Congress St. and Washington Blvd.
• Riverside entrance @ Atwater St. (along the Riverwalk)

CNCF will provide free eMed testing kits on-site from Sunday, October 23 – Friday, October 28 for those that need to provide a negative COVID-19 test prior to entering the event. There will not be space to take the test where you pick it up, so please plan to test in an alternate location (i.e, your hotel room) with reliable internet. You must test within 1-day of picking up your KubeCon + CloudNativeCon North America name badge.

In addition, antigen COVID-19 tests will be available for any attendee that would like to test throughout the week.

eMed Test Kit Pickup Location

• Fort Pontchartrain Wyndham Hotel | Lobby Level, Pontchartrain Room, located directly across the street from Huntington Place.
• Tests will not be available at Huntington Place Convention Center

eMed Test Kit 

• The eMed test kit includes (1) BinaxNow COVID-19 antigen test 
• The test is administered by a virtual proctor via the eMed app

Prepare for Your Test in Advance
1. Create an eMed Account or Use an Existing eMed Account https://core.emed.com/procedure/begin?client_id=dsA1oAynCVLjz7o2S239g&scope=emed-binaxnow
*Save time on-site and complete this step ahead of time.
2. Give yourself plenty of time to pick up and take the test. From start to finish, the testing process takes 20-30 minutes.
3. A step by step process to take the virtually proctored eMed test will be provided when you pick up your test on-site. 
4. Once you’ve taken the test you will receive digital results (shared via email and in the eMed app) to share upon entry to KubeCon + CloudNativeCon North America. 
5. The following data will be shared with the Linux Foundation: date of birth, name, email address, testing result. Your information will be kept confidential. If you do not want to share this data with the Linux Foundation, please unselect this box in the eMed app.

Envoy has become a cornerstone of the Cloud Native ecosystem and is seeing widespread adoption across all kinds of companies. We'll briefly outline the vision, supported by standards from folks at NIST, of Envoy as the engine for a Cloud Native Security Kernel -- the heart of a secure cloud native system.

JPMorgan Chase application security architecture follows least privilege network and micro-segmentation principles. For instance, incoming requests from external users need to be validated in a designated security zone before it can be forwarded to upstream endpoints in a different security zone. Communication between these segments is highly regulated and involves various identity providers and different levels of authentication and authorization checks including token validations and exchanges.



In this talk you will learn how we deal with these complexities leveraging standard Envoy routing capabilities as well as Envoy filters such as JWT Authentication and External Authorization. AWS X-Ray Tracer is leveraged for added observability. For our token exchange requirements we utilize the External Processor filter with a Golang gRPC implementation leveraging Unix Domain Sockets (UDS) for improved performance and robustness. After validating the authentication status for a given incoming request we mint new tokens and inject them into the upstream request. The External Processor filter also us a clean way to logically separate standard routing requirements from very specific token exchange needs.



Envoy has become a strategic tool for operating in an elevated security requirements and the resulting additional traffic management complexities. We have been able to replace expensive, inefficient, and hard to maintain custom proxy implementations with Envoy and the External Processor filter. As our teams investigate Istio adoption, Envoy provides us added long term viability since we should be able to port our custom extensions into a service mesh environment. We have realized substantial cost savings on top of improved performance, agility, resource efficiency, and maintainability. Based on initial interest from other teams we see our Envoy-centric traffic management approach as an evolving pattern in our broader organization.

Over the last decade, infrastructure has been moving away from monolithic centralized servers and increasingly towards end users, with a focus on Edge Computing to run code as close to the people who are accessing it, wherever they are in the world. Envoy Mobile has pushed Envoy beyond the edge, all the way to your fingertips, unlocking a world of possibilities (and challenges) by being able to run Envoy on every node in the network chain from app to service and back again. Come learn how we’ve adapted Envoy to run as native embedded libraries for iOS and Android that feel right at home no matter the platform ecosystem; how we narrowed and in some cases exceeded the performance gap with established mobile networking libraries; what mobile-specific use cases or problem areas we discovered along the way, and how we solved for them; how Envoy’s rich observability tools helped us roll out safely to billions of requests a day; and how this is just the beginning of pushing mobile networking to the next level.

If you're interested in using Envoy as an API gateway or Kubernetes ingress, this is your session. Envoy Gateway is a new project within the Envoy ecosystem that was announced at KubeCon EU 2022. The goal of the project is to attract more users to Envoy by lowering barriers to adoption through expressive, extensible, role-oriented APIs that support a multitude of traffic routing use cases. Agenda 1. Envoy Gateway Introduction & Demo 2. v0.2.0 Release Highlights 3. Roadmap 4. Q&A

Over the last year, Envoy's upstream feature set has grown dramatically due to the productionization of Envoy Mobile. New features such as automatic upstream protocol selection using ALPN, HTTP/3 upstream with seamless TCP fail-over, and Happy Eyeballs support have enriched not just Envoy Mobile, but Envoy as a dynamic forward proxy. This talk will run through these new features, and how they can improve your Envoy deployments.

How can Envoy protect itself from OOMs? Envoy has a number of different protection mechanisms out-of-the-box -- how do they work? When should you use them and how should they be configured? Let's find out! Kevin will conclude with some experimental results using these protection mechanisms.

Migrations are one of the most challenging tasks we do as infrastructure engineers.
These are sometimes long, tedious and come with many technical challenges of their own.
At Slack, we switched from HAProxy to Envoy Proxy for all ingress traffic. Overall, this migration was a success, and did not cause any downtime, but even so, we ran into several interesting edge cases that caused minor problems, such as failing a small percentage of requests, or increasing latency for requests, or sometimes an unhappy bot.

Troubleshooting these sorts of 'gray' failures can be difficult, so this talk will discuss some of those facepalm moments: how they were detected, steps taken to investigate them, and how they were solved.

Takeaways from this talk include a specific set of approaches for debugging such problems with Envoy Proxy and other web proxies that we learnt via these events along with some engineering practices that eases the stress during a large migration.

Monitoring and debugging modern cloud-based applications is challenging due to their highly distributed nature. End-to-end distributed tracing (tracing individual calls through a request's lifecycle) has emerged to be essential in a developer's toolbox as they're critical to describing a request's flow through a microservice. Unfortunately, to enable distributed tracing, each service along the request tree needs to propagate a global request ID to help link the related requests (i.e., link incoming requests with spawned backend requests). Enabling such header propagation can be non-trivial for large microservices with 100+ independent services or for legacy apps where instrumentation is hard. This talk explores whether distributed tracing can be made more accessible by eliminating the need for application instrumentation. We describe a method that combines observations external to the app (using an envoy-based service mesh) with timing analysis of the requests to construct end-to-end traces. In an evaluation with a simple microservice, this preliminary method boosts trace reconstruction accuracy to 96% (compared to 77% for a baseline), and can help answer useful developer queries.

To scale Lyft’s engineers' productivity, the Lyft developer experience team pivoted away from costly and hard to maintain custom individual environments, and rebuilt the development environment around a shared staging environment.

The goal was to enable Lyft developers to run multiple versions of the same service (both in Kubernetes and on their laptop) without stepping on each other's toes. To do this the team designed a scriptable ingress proxy as well as used custom Envoy filters within Lyft’s service mesh to route traffic to these isolated instances, inject and propagate custom metadata, and offload the traffic to custom developer tools. The end result gives engineers special networking debugging superpowers during development.

In this talk, you’ll follow Lyft’s progression designing and developing these different components. You’ll deep dive into the custom Envoy filters and how they combine tracing, Original DST clusters, a custom xDS control plane, and local tooling to build this new developer experience. We will also cover realized benefits to Lyft’s engineering productivity as well as problems encountered along the way.

How can we be assured of the correctness and safety of the many Envoy parsers and state machines in the presence of untrusted or adversarial input? While developers cover main scenarios using tests, complex edge cases may be missed. Adversaries may be able to exploit these cases to trigger denial of service attacks, access Envoy process memory remotely, or trigger remote execution of malicious code. Fuzzing is an automated software testing technique that provides randomized input to the system under test (SUT). Some tests may use a variety of sanitizers to check for violations of memory safety, check for invariants expressed as assert statements or abnormal program terminations or timeouts. Other tests may compare behavior of different SUTs to the same input. This talk will include an overview of different fuzzers in Envoy, the OSS-Fuzz infrastructure for running fuzz tests, some bugs fuzz tests discovered, and examples of creating specific fuzz tests for ESF components in Envoy.

Various Envoy based products and projects exist to provide API gateway features. But what to do if needing to replace a highly used traditional enterprise API management product. Not every project has a compatible set of policies with the same behaviors. Not to mention the differences on the API management side (developer portal, policy registry, API enablement workflows, etc). If having an extensive library of policy rules and configurations, and deep integration with an existing product, it makes sense to recreate the much needed policies for an Envoy based replacement and create a new management portal in line with the existing setup for a smooth and controlled takeout.


So how do we build out our Envoy based API management platform to replace the traditional API management ecosystem with full traffic load within a year? What have we learned so far about the ecosystem and how can it help you decide what to do when in a similar situation? We’ll discuss our journey, integrating the go-control-plane to drive xDS APIs, creating custom C++ & WASM filters to recreate heavily used policies, integrating external break-out services, building a dynamic filter chain allowing for branching and callouts, as well as translating and integrating with existing configuration stores and formats.

This talk is about hardware enablement to Envoy. What are the challenges there and how they could be solved? CryptoMb, QAT and DLB contrib extensions are used as examples.

Majority of Envoy’s access control policies are defined using the path component of the request URL. How can we be sure that URL Path Based Access Policies (PBAC) can not be bypassed? Two nominally different URL paths may in fact identify the same resource, and need to be normalized to their canonical form, before comparison. The normalization is standardized in RFC 3986, however it may not be enough to ensure safety of the access control. In this talk we explore the effects of URL path normalization on request access control, Envoy’s configuration options for path normalization and general principles for ensuring the safety of the PBAC policies.

Effective traffic management is the key to allow customers to ensure that their micro-services and overall architecture are highly available and highly reliable . How to design the rate limiting system for distributed systems handling millions of requests per second at Google scale? How to make the rate limiting system smarter to ensure fair sharing between various clients and handle the service surge in a particular region? This talk will go into details about the design of this more scalable, intelligent and performant rate limiting service and how your own service can benefit from it.

Envoy allows for modifying HTTP headers when sending requests upstream and responses downstream. The syntax used for creating header content is the same as for creating access log’s entry. But surprisingly, only a small subset of formatters were available for header modification compared to the full set of access log’s formatters. This has changed and now all formatters can be used in both, in header manipulation and in access logs. This talk describes logic used when creating access logs and when manipulating headers. It also explains why using the same formatter in the access log can render different results than using it in the header. Note: This feature is still under development, but I hope that it will be finished and merged to main before the conference. See https://github.com/envoyproxy/envoy/pull/21932

Join us onsite for drinks and appetizers with fellow co-located attendees from Monday's CNCF-hosted Co-located Events.

Network with attendees from:

BackstageCon North America hosted by CNCF
Cloud Native SecurityCon North America hosted by CNCF
Cloud Native Telco Day North America hosted by CNCF
Cloud Native Wasm Day North America hosted by CNCF
eBPF Day North America hosted by CNCF
KnativeCon North America hosted by CNCF
EnvoyCon North America hosted by CNCF
Kubernetes Batch + HPC Day North America hosted by CNCF
Open Observability Day North America hosted by CNCF