Ask
154
@starschema_sy ·

Interviewers keep asking if I have built ETL, what I built is Fivetran plus dbt, is that the same thing

Two years in this role, I own ingestion from six sources into Snowflake, roughly ninety dbt models and an Airflow schedule that ties it together. In interviews I get asked about ETL tooling and I never know whether to say yes and then explain, or say no and lose the question. Is the transform before load versus after load distinction actually load bearing anywhere in practice, or is it just vocabulary that shops use to signal which decade they hired in?

8 answers Share
Report

Answering anonymously — a moderator will review it first.

  • @yaml_yusuf · last mo. · 2 replies

    Be aware the word means different things in different buildings and that is why the question feels weird. In a lot of enterprises ETL is not a description of data flow, it is the name of a department and a toolchain: SSIS packages, Informatica jobs, a scheduler somebody has run since 2011, and a team whose job title contains the letters. If the interviewer came up through that world, hearing no from you means you have never touched their stack, and hearing yes plus a description of Fivetran and dbt tells them exactly what you know. Answer the question they meant rather than the one about acronyms.

    138
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @backfill_bram · last mo.

      This. Ask them what their current stack is before you answer and the whole thing becomes a conversation instead of a test.

      44
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @nightbus_nina · last mo. · 3 replies

    Say yes, then say what you built, and do not apologise for the tooling. ELT is ETL with the warehouse rented as the transform engine, and that is genuinely a different set of tradeoffs rather than a rebrand. Where the order actually matters: if you are not allowed to land raw data at all, because it contains card numbers or health data or something a contract forbids you from storing, then transform has to happen before load and no amount of dbt will get you out of that. Same if the source charges you by row extracted or the raw volume would cost more to store than the value it holds. Outside those cases the distinction is mostly about where your compute bill lands.

    172
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @drip_dana · last mo.

      It is also the one interviewers are usually fishing for. They want to know whether you have hit a constraint or only ever had a green field.

      46
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
    • @starschema_sy · last mo.

      The compliance case is a good one to have ready, because it is the only version where the answer is not just preference.

      41
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @backfill_bram · last mo.

    Politely disagreeing that it is only vocabulary. The one place it bites operationally is replay. If your transform runs after load and the raw landing zone is immutable and dated, you can rebuild any output from history whenever a bug is found, which is enormously valuable and is most of why the pattern won. If you transform before load and only keep the output, then discovering a bug in the transform in March means the January numbers are simply wrong forever and you cannot recover them. That is not a vocabulary difference, that is whether your pipeline is recoverable.

    124
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @hemline_hank · last mo.

    The thing I would actually put in an interview answer is idempotency, not the acronym. Anyone can name their tools. Being able to say how a rerun of yesterday's load behaves, what happens if it half fails, and how you would backfill a schema change across two years of history puts you in a different bucket regardless of which letters your pipeline uses.

    107
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @uptime_hoarder · last mo.

    Yes, you have built ETL. The T moved. Describe the sources, the volumes and how you handle reruns and nobody will ask about the letters again.

    69
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report