Core engineering
Hire Data Engineers
Engineers who make the numbers trustworthy, so decisions stop being arguments about whose dashboard is right.
A data engineer builds the pipelines and warehouse models that make analytics trustworthy: ingestion, transformation, testing, lineage and documentation. Hire one when teams disagree about basic numbers or pipelines fail silently. Expect $70 to $150 per hour by seniority and scale.
What this role actually does
The symptom of a missing data engineer is always the same: two teams present different values for the same metric and nobody can reconstruct which is correct. Underneath that are usually untested transformations, undocumented business logic embedded in dashboards, and schema changes upstream that nobody was warned about.
A data engineer fixes this structurally, with modelled and version-controlled transformations, tests that fail loudly when assumptions break, documented lineage from source to dashboard, and contracts with the upstream services that produce the data.
Core skills to look for
- dbt for modelled, tested, version-controlled transformations
- Orchestration with Airflow, Dagster or an equivalent scheduler
- Warehouse depth in Snowflake, BigQuery or Databricks, including cost control
- Dimensional modelling and the discipline to keep it consistent
- Data quality testing and contracts with upstream producers
- Streaming ingestion with Kafka where batch is genuinely insufficient
- SQL well beyond the intermediate level
When you need to hire this role
- Two teams report different numbers for the same metric
- Pipelines fail silently and stale data reaches dashboards unnoticed
- Nightly processing no longer fits inside the night
- Warehouse spend is rising without a clear explanation
- Analysts spend most of their time cleaning data rather than analysing it
How to assess candidates for this role
Ask how they handle upstream schema changes
Strong answers involve contracts, tests and alerting. Weak answers involve finding out when a dashboard breaks.
Test modelling judgement
Give a messy domain and ask for a dimensional model. Look for grain discipline and explicit handling of slowly changing dimensions.
Probe cost awareness
Ask how they would reduce warehouse spend. Expect partitioning, clustering, materialisation strategy and incremental models.
Check testing practice
Ask what they test in a pipeline. Anyone who says row counts only has not maintained a warehouse others depend on.
Typical rate bands
| Level | Experience | Typical rate | What they own |
|---|---|---|---|
| Mid-level | 3-5 years | $70-100/hr | Builds and maintains models within an existing platform |
| Senior | 6-9 years | $100-130/hr | Owns warehouse architecture, testing and data contracts |
| Staff / Principal | 10+ years | $130-150/hr | Designs the platform, governance and organisation-wide standards |
Frequently asked questions
What does a data engineer do?
They build the systems that move, model and validate data: ingestion, transformation, testing, orchestration and documentation. The goal is that anyone querying the warehouse gets a trustworthy answer without needing to know the pipeline's history.
What is the difference between a data engineer and an analytics engineer?
An analytics engineer focuses on modelling inside the warehouse, mostly with dbt and SQL. A data engineer also owns ingestion, orchestration and infrastructure. Smaller teams often need one person covering both.
Why do our teams report different numbers?
Almost always because business logic lives in dashboards rather than in tested, version-controlled models. Centralising definitions in the warehouse with tests and documented lineage is the structural fix.
How much does a data engineer cost?
Typical vetted contract rates are $70 to $100 per hour at mid-level, $100 to $130 at senior and $130 to $150 for staff and principal, varying with data volume and platform complexity.
Do we need streaming or is batch enough?
Batch is enough far more often than teams assume. Streaming is justified when a decision genuinely cannot wait, and it carries real operational cost. A good candidate will interrogate the requirement before recommending it.