Unknown field DAB pipeline definition: auto_full_refresh_policy

### Describe the issue
The Databricks docs describe a [feature](https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/full-refresh#auto-full-refresh-policy):

> **Auto full refresh policy**
> 
> To help maintain data consistency without manual intervention, an auto full refresh policy lets you automatically trigger a full refresh when the pipeline encounters unsupported DDL operations:
> 
>  * Table truncate
>  * Incompatible schema changes (for example, data type changes)
>  * Column renames
>  * Column additions with default values
>
> Without auto full refresh enabled, you must manually trigger a full refresh when these operations occur.
> 
> **Configuration parameters**
> 
> Configure auto full refresh at the pipeline level or table level in your pipeline specification:
> 
> Parameter | Type | Description | Default
> --- | --- | --- | ---
> `enabled` | Boolean | Whether auto full refresh is enabled. | `false`
> `min_interval_hours` | Integer | Minimum wait interval in hours between full refreshes. The system waits for this interval since the last snapshot before initiating a new auto full refresh. | 24
> 
> You can configure auto full refresh at multiple levels:
> 
> * Pipeline level: In `ingestion_definition.table_configuration.auto_full_refresh_policy`
> * Table level: In `ingestion_definition.objects[].table.table_configuration.auto_full_refresh_policy`
>
> Table-level configuration overrides pipeline-level configuration.

### Steps to reproduce the behavior

1. `databricks bundle init default-sql`
2. Add the following to `resources/salesforce.pipeline.yml`:

    ```yml
    resources:
      pipelines:
        salesforce_ingestion_pipeline:
          name: Salesforce Ingestion Pipeline
          channel: CURRENT
          catalog: ${var.catalog}
          schema: ${var.schema}
          ingestion_definition:
            source_type: SALESFORCE
            connection_name: salesforce_${bundle.target}
            table_configuration:
              # https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/full-refresh#auto-full-refresh-policy
              auto_full_refresh_policy:
                enabled: true
            objects:
            - table:
                source_schema: objects
                source_table: Contact
                destination_catalog: ${var.catalog}
                destination_schema: ${var.schema}
    ```

3. `databricks bundle validate`

### Expected Behavior
The field should validate (and subsequently deploy) fine.

### Actual Behavior

```console
$ databricks bundle validate
Warning: unknown field: auto_full_refresh_policy
  at resources.pipelines.salesforce_ingestion_pipeline.ingestion_definition.table_configuration
  in resources/salesforce.pipeline.yml:13:11

...
Found 1 warning
```

### OS and CLI version
- MacOS Sequoia 15.7.1 (24G231)
- Databricks CLI v0.281.0

### Debug Logs

<details>
<summary> <code>databricks bundle validate --log-level=debug</code> </summary>

```
$ databricks bundle validate --log-level=debug
09:32:54 Info: start pid=10994 version=0.281.0 args="databricks, bundle, validate, --log-level=debug"
09:32:54 Debug: Found bundle root at /Users/<redacted>/code/mn-deed/sql_project (file /Users/tim_vergenz/code/mn-deed/sql_project/databricks.yml) pid=10994
09:32:54 Info: Phase: load pid=10994
09:32:54 Debug: Apply pid=10994 mutator=EntryPoint
09:32:54 Debug: Apply pid=10994 mutator=scripts.preinit
09:32:54 Debug: No script defined for preinit, skipping pid=10994 mutator=scripts.preinit
09:32:54 Debug: Apply pid=10994 mutator=ProcessRootIncludes
09:32:54 Debug: Apply pid=10994 mutator=ProcessRootIncludes mutator=ProcessInclude(resources/salesforce.pipeline.yml)
Warning: unknown field: auto_full_refresh_policy
  at resources.pipelines.salesforce_ingestion_pipeline.ingestion_definition.table_configuration
  in resources/salesforce.pipeline.yml:13:11

09:32:54 Debug: Apply pid=10994 mutator=ProcessRootIncludes mutator=ProcessInclude(resources/sql_project_sql.job.yml)
09:32:54 Debug: Apply pid=10994 mutator=VerifyCliVersion
09:32:54 Debug: Apply pid=10994 mutator=EnvironmentsToTargets
09:32:54 Debug: Apply pid=10994 mutator=ComputeIdToClusterId
09:32:54 Debug: Apply pid=10994 mutator=InitializeVariables
09:32:54 Debug: Apply pid=10994 mutator=DefineDefaultTarget(default)
09:32:54 Debug: Apply pid=10994 mutator=validate:unique_resource_keys
09:32:54 Debug: Apply pid=10994 mutator=SelectDefaultTarget
09:32:54 Debug: Apply pid=10994 mutator=SelectDefaultTarget mutator=SelectTarget(dev)
09:32:54 Debug: Loading profile dev because of host match pid=10994
09:32:54 Debug: Apply pid=10994 mutator=<func>
09:32:54 Info: Phase: initialize pid=10994
09:32:54 Debug: Apply pid=10994 mutator=validate:AllResourcesHaveValues
09:32:54 Debug: Apply pid=10994 mutator=validate:interpolation_in_auth_config
09:32:54 Debug: Apply pid=10994 mutator=validate:no_interpolation_in_bundle_name
09:32:54 Debug: Apply pid=10994 mutator=validate:scripts
09:32:54 Debug: Apply pid=10994 mutator=RewriteSyncPaths
09:32:54 Debug: Apply pid=10994 mutator=SyncDefaultPath
09:32:54 Debug: Apply pid=10994 mutator=SyncInferRoot
09:32:54 Debug: Apply pid=10994 mutator=InitializeCache
09:32:54 Debug: Apply pid=10994 mutator=PopulateCurrentUser
09:32:54 Debug: [Local Cache] using cache key: 533944c9e3f1668ead77997928b1c968079607f29315f1b153fee25060deedc9 pid=10994 mutator=PopulateCurrentUser
09:32:54 Debug: [Local Cache] cache hit pid=10994 mutator=PopulateCurrentUser
09:32:54 Debug: GET /api/2.0/preview/scim/v2/Me
< HTTP/2.0 200 OK
< {
<   "active": true,
<   "displayName": "Tim Vergenz",
<   "emails": [
<     {
<       "primary": true,
<       "type": "work",
<       "value": "<redacted>"
<     }
<   ],
<   "groups": [
<     {
<       "$ref": "Groups/269198777609561",
<       "display": "Workspace Administrators",
<       "type": "direct",
<       "value": "269198777609561"
<     },
<     {
<       "$ref": "Groups/661492905282999",
<       "display": "users",
<       "type": "direct",
<       "value": "661492905282999"
<     },
<     {
<       "$ref": "Groups/583108487660070",
<       "display": "admins",
<       "type": "indirect",
<       "value": "583108487660070"
<     }
<   ],
<   "id": "72755211285031",
<   "name": {
<     "familyName": "Vergenz",
<     "givenName": "Tim"
<   },
<   "roles": [
<     {
<       "value": "arn:aws:iam::<redacted>"
<     },
<     {
<       "value": "arn:aws:iam::<redacted>"
<     }
<   ],
<   "schemas": [
<     "urn:ietf:params:scim:schemas:core:2.0:User",
<     "urn:ietf:params:scim:schemas:extension:workspace:2.0:User"
<   ],
<   "userName": "<redacted>"
< } pid=10994 mutator=PopulateCurrentUser sdk=true
09:32:54 Debug: [Local Cache] computed and stored result pid=10994 mutator=PopulateCurrentUser
09:32:54 Debug: Apply pid=10994 mutator=LoadGitDetails
09:32:54 Debug: Apply pid=10994 mutator=ApplySourceLinkedDeploymentPreset
09:32:54 Debug: Apply pid=10994 mutator=DefineDefaultWorkspaceRoot
09:32:54 Debug: Apply pid=10994 mutator=ExpandWorkspaceRoot
09:32:54 Debug: Apply pid=10994 mutator=DefaultWorkspacePaths
09:32:54 Debug: Apply pid=10994 mutator=PrependWorkspacePrefix
09:32:54 Debug: Apply pid=10994 mutator=RewriteWorkspacePrefix
09:32:54 Debug: Apply pid=10994 mutator=SetVariables
09:32:54 Debug: Apply pid=10994 mutator=ResolveVariableReferences
09:32:54 Debug: Apply pid=10994 mutator=ResolveLookupVariables
09:32:54 Debug: Apply pid=10994 mutator=ResolveVariableReferences
09:32:54 Debug: Apply pid=10994 mutator=validate:volume-path
09:32:54 Debug: Apply pid=10994 mutator=ApplyTargetMode
09:32:54 Info: Development mode: disabling deployment lock since bundle.deployment.lock.enabled is not set to true pid=10994 mutator=ApplyTargetMode
09:32:54 Debug: Apply pid=10994 mutator=ConfigureWSFS
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ResolveVariableReferences(resources)
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=LogResourceReferences
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=NormalizePaths
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=TranslatePathsDashboards
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=validate:SingleNodeCluster
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ResolveVariableReferences(resources)
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ExpandPipelineGlobPaths
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=MergeJobClusters
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=MergeJobParameters
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=MergeJobTasks
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=MergePipelineClusters
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=MergeApps
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=CaptureSchemaDependency
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ConfigureDashboardSerializedDashboard
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=JobClustersFixups
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ClusterFixups
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ModelServingEndpointFixups
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=SetRunAs
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=OverrideCompute
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ApplyPresets
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.dashboards.*, parent_path, /Workspace/Users/<redacted>/.bundle/sql_project/dev/resources)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.dashboards.*, embed_credentials, false)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.volumes.*, volume_type, MANAGED)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.alerts.*, parent_path, /Workspace/Users/<redacted>/.bundle/sql_project/dev/resources)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*, name, Untitled)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*, max_concurrent_runs, 1)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.schedule, pause_status, UNPAUSED)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.trigger, pause_status, UNPAUSED)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.continuous, pause_status, UNPAUSED)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.task[*].dbt_task, schema, default)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.task[*].for_each_task.task.dbt_task, schema, default)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients, notebooks, true)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.jobs.*.job_clusters[*].new_cluster.workload_type.clients, jobs, true)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.pipelines.*, edition, ADVANCED)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.pipelines.*, channel, CURRENT)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, auto_stop_mins, 120)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, enable_photon, true)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, max_num_clusters, 1)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.sql_warehouses.*, spot_instance_policy, COST_OPTIMIZED)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.apps.*, description, )"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.clusters.*, autotermination_minutes, 60)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.clusters.*.workload_type.clients, notebooks, true)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator="SetDefaultMutator(resources.clusters.*.workload_type.clients, jobs, true)"
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=DefaultQueueing
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=DashboardFixups
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=ApplyBundlePermissions
09:32:54 Debug: Apply pid=10994 mutator=ProcessStaticResources mutator=FixPermissions
09:32:54 Debug: Apply pid=10994 mutator=PythonMutator(load_resources)
09:32:54 Debug: Apply pid=10994 mutator=PythonMutator(apply_mutators)
09:32:54 Debug: Apply pid=10994 mutator=validate:required
09:32:54 Debug: Apply pid=10994 mutator=validate:enum
09:32:54 Debug: Apply pid=10994 mutator=validate:validate_dashboard_etags
09:32:54 Debug: Apply pid=10994 mutator=CheckPermissions
09:32:54 Debug: Apply pid=10994 mutator=TranslatePaths
09:32:54 Debug: Apply pid=10994 mutator=PythonWrapperWarning
09:32:54 Debug: Apply pid=10994 mutator=ApplyArtifactsDynamicVersion
09:32:54 Debug: Apply pid=10994 mutator=artifacts.Prepare
09:32:54 Info: No local tasks in databricks.yml config, skipping auto detect pid=10994 mutator=artifacts.Prepare
09:32:54 Debug: Apply pid=10994 mutator=apps.Validate
09:32:54 Debug: Apply pid=10994 mutator=ValidateTargetMode
09:32:54 Debug: Apply pid=10994 mutator=ValidateSharedRootPermissions
09:32:54 Debug: Apply pid=10994 mutator=metadata.AnnotateJobs
09:32:54 Debug: Apply pid=10994 mutator=metadata.AnnotatePipelines
09:32:54 Debug: Apply pid=10994 mutator=scripts.postinit
09:32:54 Debug: No script defined for postinit, skipping pid=10994 mutator=scripts.postinit
09:32:54 Debug: ApplyParallel pid=10994 mutator=fast_validate(readonly)
09:32:54 Debug: ApplyParallel pid=10994 mutator=validate:files_to_sync
09:32:54 Debug: ApplyParallel pid=10994 mutator=validate:folder_permissions
09:32:54 Debug: ApplyParallel pid=10994 mutator=validate:validate_sync_patterns
09:32:54 Debug: ApplyParallel pid=10994 mutator=fast_validate(readonly) mutator=validate:job_cluster_key_defined
09:32:54 Debug: ApplyParallel pid=10994 mutator=fast_validate(readonly) mutator=validate:job_task_cluster_spec
09:32:54 Debug: ApplyParallel pid=10994 mutator=fast_validate(readonly) mutator=validate:artifact_paths
09:32:54 Debug: GET /api/2.0/workspace/get-status?path=/Workspace/Users/<redacted>/.bundle/sql_project/dev/files
< HTTP/2.0 200 OK
< {
<   "object_id": 2526332894079824,
<   "object_type": "DIRECTORY",
<   "path": "/Workspace/Users/<redacted>/.bundle/sql_project/dev/files",
<   "resource_id": "2526332894079824"
< } pid=10994 mutator=validate:files_to_sync sdk=true
09:32:54 Debug: Path /Workspace/Users/<redacted>/.bundle/sql_project/dev/files has type directory (ID: 2526332894079824) pid=10994 mutator=validate:files_to_sync
Name: sql_project
Target: dev
Workspace:
  Host: https://<redacted>.databricks.com
  User: <redacted>
  Path: /Workspace/Users/<redacted>/.bundle/sql_project/dev

Found 1 warning
09:32:54 Info: completed execution pid=10994 exit_code=0
09:32:54 Debug: no telemetry logs to upload pid=10994
```

</details>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Unknown field DAB pipeline definition: auto_full_refresh_policy #4168

Describe the issue

Steps to reproduce the behavior

Expected Behavior

Actual Behavior

OS and CLI version

Debug Logs

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Parameter	Type	Description	Default
`enabled`	Boolean	Whether auto full refresh is enabled.	`false`
`min_interval_hours`	Integer	Minimum wait interval in hours between full refreshes. The system waits for this interval since the last snapshot before initiating a new auto full refresh.	24

Unknown field DAB pipeline definition: auto_full_refresh_policy #4168

Description

Describe the issue

Steps to reproduce the behavior

Expected Behavior

Actual Behavior

OS and CLI version

Debug Logs

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions