chore(deps): bump arrow 58, datafusion 53, object_store 0.13#6410
Closed
jja725 wants to merge 1 commit intolance-format:mainfrom
Closed
chore(deps): bump arrow 58, datafusion 53, object_store 0.13#6410jja725 wants to merge 1 commit intolance-format:mainfrom
jja725 wants to merge 1 commit intolance-format:mainfrom
Conversation
Upgrade core dependencies to align with the latest Apache DataFusion ecosystem: - arrow 57.0.0 → 58.0.0 - datafusion 52.1.0 → 53.0.0 - object_store 0.12.3 → 0.13 Key API changes addressed: - object_store 0.13: Methods (get, put, head, delete, copy, rename, etc.) moved from ObjectStore trait to ObjectStoreExt extension trait. Added new required trait methods delete_stream and copy_opts. rename_if_not_exists replaced with rename_opts + RenameTargetMode::Create. - datafusion 53: ExecutionPlan::statistics() removed (now has default impl). ExecutionPlan::properties() returns &Arc<PlanProperties> instead of &PlanProperties. SimplifyContext::new(&props) replaced with SimplifyContext::default(). PGBitwiseNot renamed to BitwiseNot. HashJoinExec::try_new added null_aware parameter. - arrow 58: No breaking changes for lance's usage. Note: object_store_opendal 0.55 does not yet support object_store 0.13. Cloud storage features (aws, gcp, azure, oss, tencent, huggingface) will need an opendal update (PR merged upstream: apache/opendal#7243). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Upgrade core dependencies to align with the latest Apache DataFusion ecosystem:
Key API changes addressed
object_store 0.12 → 0.13
get,put,head,delete,copy,rename, etc.) moved fromObjectStoretrait toObjectStoreExtextension traitdelete_stream,copy_optsrename_if_not_existsreplaced withrename_opts+RenameTargetMode::Createdatafusion 52 → 53
ExecutionPlan::statistics()removed (now has default impl)ExecutionPlan::properties()returns&Arc<PlanProperties>instead of&PlanPropertiesSimplifyContext::new(&props)→SimplifyContext::default()UnaryOperator::PGBitwiseNot→UnaryOperator::BitwiseNotHashJoinExec::try_newaddednull_awareparameterarrow 57 → 58
Known issue
object_store_opendal 0.55does not yet supportobject_store 0.13. Cloud storage features (aws,gcp,azure,oss,tencent,huggingface) will need an opendal update once released. The fix is already merged upstream: apache/opendal#7243Test plan
cargo checkpasses for all core crates with--no-default-features🤖 Generated with Claude Code