DataEng
Presentations
Scaling DB Access for Billions of Queries Per Day @PayPal
As microservices scale and proliferate, they add increasing load on databases in terms of connections and resource usage. Open sourced in the Go programming language, Hera (High Efficiency Reliable Access to data stores) scales thousands of PayPal’s applications with connection...
Peloton - Uber's Webscale Unified Scheduler on Mesos & Kubernetes
With the increasing scale of Uber’s business, efficient use of cluster resources is important to reduce the cost per trip. As we have learned when operating Mesos clusters in production, it is a challenge to overcommit resources for latency-sensitive services due to their large spread of...
Data Engineering Open Space
Details to follow.
Building and Operating a Serverless Data Pipeline
At Intent our machine learning platform processes real-time and historical data to predict user intent on billions of page views a month. At the heart of this system is a serverless data pipeline that allows us to gather, process, store, and analyze data from disparate data sources....
Interviews
Scaling DB Access for Billions of Queries Per Day @PayPal
What is the focus of your work today?
Petrica: I am working on developing Hera, which is High Efficiency Reliable Access to data sources, basically a proxy to databases. We support Oracle and MySQL. Hera helps PayPal scale.
Read Full InterviewBuilding and Operating a Serverless Data Pipeline
What is the focus of your work these days?
I'm the lead of the data platform team at Intent. Intent is a data science company that helps commerce sites maximize the value for each person who visits their site. Our main product is an ad network that runs on travel sites. On the data platform team we oversee our data pipeline, that collects data from ad servers and various...
Read Full Interview