feat(write): add write pipeline with DataFusion INSERT INTO/OVERWRITE support

Add TableWrite for writing Arrow RecordBatches to Paimon append-only
tables. Each (partition, bucket) pair gets its own DataFileWriter with
direct writes (matching delta-rs DeltaWriter pattern). File rolling
uses tokio::spawn for background close, and prepare_commit uses
try_join_all for parallel finalization across partition writers.

Key components:
- TableWrite: routes batches by partition/bucket, holds DataFileWriters
- DataFileWriter: manages parquet file lifecycle with rolling support
- WriteBuilder: creates TableWrite and TableCommit instances
- PaimonDataSink: DataFusion DataSink integration for INSERT/OVERWRITE
- FormatFileWriter: extended with flush() and in_progress_size()

Configurable options via CoreOptions:
- file.compression (default: zstd)
- target-file-size (default: 256MB)
- write.parquet-buffer-size (default: 256MB)

Includes E2E integration tests for unpartitioned, partitioned,
fixed-bucket, multi-commit, column projection, and bucket filtering.

Author: JingsongLi

Cargo.toml

@@ -34,7 +34,9 @@ arrow-buffer = "57.0"
 arrow-schema = "57.0"
 arrow-cast = "57.0"
 arrow-ord = "57.0"
+arrow-select = "57.0"
 datafusion = "52.3.0"
 datafusion-ffi = "52.3.0"
 parquet = "57.0"
 tokio = "1.39.2"
+tokio-util = "0.7"

crates/integration_tests/Cargo.toml

@@ -26,6 +26,7 @@ homepage.workspace = true
 [dependencies]
 paimon = { path = "../paimon" }
 arrow-array = { workspace = true }
+arrow-schema = { workspace = true }
 tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
 futures = "0.3"