API Design for ETL - Lessons from Nearly 100 Data Integrations in the Wild
Presented by: Dan Mosora
In the API Economy, users increasingly want to analyze and connect more of their business data, from small startup shops to multinational corporations. Thinking about making data available through an API? There are innumerable ways to slice the problem. How do you implement pagination? What data model makes sense? What level of authentication is right to be secure, but usable?
At Stitch, we've built and reviewed almost 100 data integrations in Python for the Singer project, most of which interact with APIs, and have had good experiences ...and not-so-good experiences. This talk will explore these lessons through APIs in the wild, cover the challenges that we've faced when integrating with an API for ETL purposes, and highlight the qualities that make an API a joy to interact with.