Support Microsoft Fabric dbt Job as a recognized orchestrator #2161
Description
Summary
Request to add Microsoft Fabric dbt Job as a supported orchestrator in Elementary's job metadata collection, alongside the currently supported dbt_cloud, airflow, dagster, prefect, and github_actions.
Context
Microsoft Fabric introduced native dbt job support (Public Preview, December 2025). Unlike dbt Cloud or Airflow, a Fabric dbt Job is a fully managed workspace item — Microsoft provides a versioned dbt Job Runtime (currently dbt Core v1.9) that handles execution, scheduling, and monitoring entirely within the Fabric platform. Users author models and configure jobs via the Fabric UI; there is no CLI invocation to customize.
This is fundamentally different from other supported orchestrators:
| Orchestrator | How dbt is invoked | Can pass --vars? | Exposes env vars? |
|---|---|---|---|
| Airflow | User-authored CLI / Operator | ✅ Yes | ✅ Yes (limited) |
| dbt Cloud | Managed, but API-accessible | ✅ Via API | ✅ DBT_CLOUD_JOB_ID, etc. |
| Fabric dbt Job | Fully managed by Microsoft | ❌ Not directly | ❓ Unknown / undocumented |
What I'd like Elementary to support
Enable Elementary to collect job metadata when dbt runs inside a Microsoft Fabric dbt Job, by:
- Adding fabric_dbt_job (or microsoft_fabric) to the recognized orchestrator values in the Variables supported format table
- Mapping any Fabric-injected runtime env vars (e.g. a hypothetical FABRIC_JOB_ID, FABRIC_RUN_ID, FABRIC_WORKSPACE_ID) to Elementary's standard fields: job_id, job_run_id, job_url, job_run_url
- Documenting any workaround (e.g. using Elementary's --vars override via a pre/post-hook if Fabric ever exposes that capability)
Key open question / dependency
The main blocker is whether Microsoft Fabric exposes job/run context as environment variables inside the managed dbt runtime. This is currently undocumented in Fabric's public docs. It would be helpful if the Elementary team could:
- Confirm whether Fabric runtime env vars are accessible to dbt at execution time
- Or flag this as a dependency on Microsoft exposing those variables (similar to how Kestra support was discussed in Support Kestra orchestrator #1430)
If Fabric does expose env vars, the Elementary-side implementation should be straightforward, following the same pattern used for dbt_cloud and github_actions.
Why this matters
Fabric dbt Jobs are growing in adoption among Microsoft-stack teams (Azure Data Warehouse, Fabric Warehouse). These users currently have no way to correlate Elementary data quality alerts with the specific Fabric job run that produced the data, which is one of Elementary's core value propositions. Supporting this orchestrator would unlock Elementary for a large segment of enterprise Microsoft/Azure users.
References
- Elementary docs: https://docs.elementary-data.com/oss/guides/collect-job-data
- Microsoft Fabric dbt Job overview: https://learn.microsoft.com/en-us/fabric/data-factory/dbt-job-overview
- Related issue (Kestra): Support Kestra orchestrator #1430
- Fabric dbt Job announcement: https://blog.fabric.microsoft.com/en-US/blog/dbt-job-in-microsoft-fabric-ship-trustworthy-sql-models-faster-preview/