· beginner
Data Engineering Foundations
How data gets from a source system to a table someone can trust: what a pipeline is for, why the raw copy is kept, and the handful of failures that account for most bad numbers.
Written in house, from the work rather than from a syllabus. Every example is one we would use ourselves, and every quiz question explains why its answer is right.
What you’ll learn
- Explain where a transformation should run, and why the industry moved it
- Decide honestly whether a use case needs streaming or a schedule
- Make a pipeline safe to re-run, and know why that matters more than it sounds
- State the grain of a table, and spot the tables that have none
- Design what happens when a task fails at 3am
- Choose data quality checks that get acted on rather than ignored
Before you start
- · Comfort with SQL SELECT, GROUP BY and JOIN
- · No prior pipeline or infrastructure experience