Stream Processing
Presentations
Data Engineering Open Space
Details to follow.
Scaling DB Access for Billions of Queries Per Day @PayPal
As microservices scale and proliferate, they add increasing load on databases in terms of connections and resource usage. Open sourced in the Go programming language, Hera (High Efficiency Reliable Access to data stores) scales thousands of PayPal’s applications with connection...
Past Presentations
Adopting Stream Processing for Instrumentation
In the midst of building a multi-datacenter, multi-tenant instrumentation and visibility system, we arrived at stream processing as an alternative to storing, forwarding, and post-processing metrics as traditional systems do. However, the streaming paradigm is alien to many engineers and...
Streaming Microservices: Contracts & Compatibility
In a world of microservices that communicate via unbounded streams of events, schemas are the contracts between the services. Having an agreed contract allows the teams developing those services to move fast, by reducing the risk involved in making changes. Yet delivering events with schema...
Survival of the Fittest - Streaming Architectures
“Perfect is the enemy of good” - Voltaire On the journey through life, we learn and adapt via trial and error - software development is no different. We realize and accept that we won’t build the perfect solution the first time around, it takes many iterations. At...