The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon North America 2022 - Detroit, MI + Virtual and add this Co-Located event to your registration to participate in these sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.
Please note: This schedule is automatically displayed in Eastern Daylight Time (EDT), UTC -4. To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date."
Migrations are one of the most challenging tasks we do as infrastructure engineers. These are sometimes long, tedious and come with many technical challenges of their own. At Slack, we switched from HAProxy to Envoy Proxy for all ingress traffic. Overall, this migration was a success, and did not cause any downtime, but even so, we ran into several interesting edge cases that caused minor problems, such as failing a small percentage of requests, or increasing latency for requests, or sometimes an unhappy bot.
Troubleshooting these sorts of 'gray' failures can be difficult, so this talk will discuss some of those facepalm moments: how they were detected, steps taken to investigate them, and how they were solved.
Takeaways from this talk include a specific set of approaches for debugging such problems with Envoy Proxy and other web proxies that we learnt via these events along with some engineering practices that eases the stress during a large migration.
