failures:
---- objstore::minio::extract::large_object_no_filters::extract_from_archive_file_to_bucket_root stdout ----
[ssstar-testing/src/test_data.rs:352] object.checksum_sha256 = Some(
"JOrFYMK1e5u3xBgR1gGVsDrgd9EtDta4cE4Op2WMXXM=",
)
thread 'objstore::minio::extract::large_object_no_filters::extract_from_archive_file_to_bucket_root' panicked at 'assertion failed: `(left == right)`
left: `[36, 234, 197, 96, 194, 181, 123, 155, 183, 196, 24, 17, 214, 1, 149, 176, 58, 224, 119, 209, 45, 14, 214, 184, 112, 78, 14, 167, 101, 140, 93, 115]`,
right: `[203, 148, 32, 60, 63, 136, 43, 214, 1, 232, 138, 58, 62, 219, 15, 38, 58, 203, 171, 99, 168, 106, 35, 225, 53, 226, 155, 159, 139, 169, 163, 62]`: S3 object 'test' (key 'test') hash doesn't match expected value', /media/core/kingston-luks/ssstar/ssstar-testing/src/test_data.rs:481:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Log events from this test:
2023-01-05T09:21:12.561162Z DEBUG ThreadId(541) ssstar_testing::minio: get() found minio server endpoint=127.0.0.1:36313
2023-01-05T09:21:12.568467Z DEBUG ThreadId(541) ssstar_testing::minio: Creating bucket bucket="large-object-no-filters-src"
2023-01-05T09:21:12.568567Z DEBUG ThreadId(541) ssstar_testing::minio: Transformed bucket name into valid and unique bucket ID bucket=d5f0da28-large-object-no-filters-src
2023-01-05T09:21:12.590903Z DEBUG ThreadId(541) ssstar_testing::minio: Bucket created bucket=d5f0da28-large-object-no-filters-src
2023-01-05T09:21:12.591014Z DEBUG ThreadId(541) ssstar_testing::minio: Creating bucket bucket="large-object-no-filters-tgt"
2023-01-05T09:21:12.591062Z DEBUG ThreadId(541) ssstar_testing::minio: Transformed bucket name into valid and unique bucket ID bucket=36e67a5e-large-object-no-filters-tgt
2023-01-05T09:21:12.605707Z DEBUG ThreadId(541) ssstar_testing::minio: Bucket created bucket=36e67a5e-large-object-no-filters-tgt
2023-01-05T09:21:17.207973Z DEBUG ThreadId(541) ssstar::create: Adding archive input url=s3://d5f0da28-large-object-no-filters-src/
2023-01-05T09:21:17.210596Z DEBUG ThreadId(541) ssstar::objstore::s3: Validating access to bucket bucket="d5f0da28-large-object-no-filters-src"
2023-01-05T09:21:17.215496Z DEBUG ThreadId(541) ssstar::objstore::s3: Access to bucket is confirmed bucket="d5f0da28-large-object-no-filters-src" region=None
2023-01-05T09:21:17.218121Z DEBUG ThreadId(541) ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="d5f0da28-large-object-no-filters-src" versioning_enabled=true
2023-01-05T09:21:17.218177Z DEBUG ThreadId(541) ssstar::create: Confirmed bucket access for input url=s3://d5f0da28-large-object-no-filters-src/ bucket=S3Bucket { name: "d5f0da28-large-object-no-filters-src", versioning_enabled: true, region: None, client: "<...>" }
2023-01-05T09:21:17.218204Z DEBUG ThreadId(541) ssstar::create: Adding archive input to job input=CreateArchiveInput { bucket: S3Bucket { name: "d5f0da28-large-object-no-filters-src", versioning_enabled: true, region: None, client: "<...>" }, selector: Bucket }
2023-01-05T09:21:17.218237Z DEBUG ThreadId(541) ssstar::create: Listing objects for all inputs input_count=1
2023-01-05T09:21:17.218307Z DEBUG ThreadId(541) into_input_objects: ssstar::create: Listing all object store objects that match this archive input self=Bucket d5f0da28-large-object-no-filters-src
2023-01-05T09:21:17.218342Z DEBUG ThreadId(541) into_input_objects: ssstar::objstore::s3: Archive input matches everything in an S3 bucket bucket=d5f0da28-large-object-no-filters-src
2023-01-05T09:21:17.226316Z DEBUG ThreadId(541) into_input_objects: ssstar::objstore::s3: Matched all S3 objects in an S3 bucket input_objects=1 bucket=d5f0da28-large-object-no-filters-src
2023-01-05T09:21:17.226388Z DEBUG ThreadId(541) into_input_objects: ssstar::create: Listing matching objects completed self=Bucket d5f0da28-large-object-no-filters-src count=1
2023-01-05T09:21:17.226468Z DEBUG ThreadId(541) ssstar::create: Listed all objects in all inputs object_count=1
2023-01-05T09:21:17.234607Z DEBUG ThreadId(552) read_object_part{key="test" version_id=Some("de2d1e05-ac52-4fbe-961b-ee98d5b0d72a") byte_range=0..8388608}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.236942Z DEBUG ThreadId(552) read_object_part{key="test" version_id=Some("de2d1e05-ac52-4fbe-961b-ee98d5b0d72a") byte_range=8388608..16777216}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.238037Z DEBUG ThreadId(552) read_object_part{key="test" version_id=Some("de2d1e05-ac52-4fbe-961b-ee98d5b0d72a") byte_range=16777216..20971520}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.260554Z DEBUG ThreadId(541) ssstar::create: Reading object and writing to tar archive key=test size=20971520
2023-01-05T09:21:17.381106Z DEBUG ThreadId(541) ssstar::create: Streamed object data into tar archive key=test size=20971520 final_part_number=2
2023-01-05T09:21:17.381205Z DEBUG ThreadId(541) ssstar::create: Completed processing of all input objects
2023-01-05T09:21:17.386282Z DEBUG ThreadId(541) ssstar::objstore::s3: Validating access to bucket bucket="36e67a5e-large-object-no-filters-tgt"
2023-01-05T09:21:17.388162Z DEBUG ThreadId(541) ssstar::objstore::s3: Access to bucket is confirmed bucket="36e67a5e-large-object-no-filters-tgt" region=None
2023-01-05T09:21:17.389633Z DEBUG ThreadId(541) ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="36e67a5e-large-object-no-filters-tgt" versioning_enabled=true
2023-01-05T09:21:17.389899Z INFO ThreadId(541) run{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } } target_bucket="36e67a5e-large-object-no-filters-tgt" target_prefix=}: ssstar::extract: Starting extract archive job self.filters=[] self.archive_size=Some(20973056)
2023-01-05T09:21:17.390104Z DEBUG ThreadId(546) ssstar::extract: Starting tar entry processor task
2023-01-05T09:21:17.390112Z DEBUG ThreadId(674) run{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } } target_bucket="36e67a5e-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } }}: ssstar::extract: Starting blocking tar read task
2023-01-05T09:21:17.390359Z DEBUG ThreadId(674) run{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } } target_bucket="36e67a5e-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } }}:Processing tar entry{path=test len=20971520}: ssstar::extract: No filters are applied so this entry is included
2023-01-05T09:21:17.390439Z DEBUG ThreadId(674) run{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } } target_bucket="36e67a5e-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } }}:Processing tar entry{path=test len=20971520}: ssstar::extract: Processing entry as multi-part
2023-01-05T09:21:17.398629Z DEBUG ThreadId(674) run{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } } target_bucket="36e67a5e-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } }}: ssstar::extract: Completed processing all tar entries
2023-01-05T09:21:17.398724Z DEBUG ThreadId(674) run{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } } target_bucket="36e67a5e-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } }}: ssstar::extract: Blocking tar read task completing successfully
2023-01-05T09:21:17.398878Z DEBUG ThreadId(551) process_tar_entries{target_prefix="" target_bucket="36e67a5e-large-object-no-filters-tgt"}: ssstar::extract: Starting multipart file upload path=test len=20971520 key=test
2023-01-05T09:21:17.401414Z DEBUG ThreadId(551) process_tar_entries{target_prefix="" target_bucket="36e67a5e-large-object-no-filters-tgt"}: ssstar::extract: Uploading file part path=test len=8388608
2023-01-05T09:21:17.402889Z DEBUG ThreadId(551) process_tar_entries{target_prefix="" target_bucket="36e67a5e-large-object-no-filters-tgt"}: ssstar::extract: Uploading file part path=test len=8388608
2023-01-05T09:21:17.403001Z DEBUG ThreadId(551) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}: ssstar::objstore::s3: Commencing multi-part upload
2023-01-05T09:21:17.404314Z DEBUG ThreadId(551) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:17.412995Z DEBUG ThreadId(546) process_tar_entries{target_prefix="" target_bucket="36e67a5e-large-object-no-filters-tgt"}: ssstar::extract: Uploading final file part path=test len=4194304
2023-01-05T09:21:17.414311Z DEBUG ThreadId(546) ssstar::writers: encountered end of duplex stream; worker task exiting
2023-01-05T09:21:18.299316Z DEBUG ThreadId(546) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:18.836985Z DEBUG ThreadId(546) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}:upload chunk{part_number=3 chunk_size=4194304}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:19.109543Z DEBUG ThreadId(546) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="7f5f484de6b97c780ef63d0e5763cf78" sha256="fNqqiz4HIy5jckO3taqlzkyU+wi+nd2fF5knAfUPiQI="
2023-01-05T09:21:19.157617Z DEBUG ThreadId(551) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}:upload chunk{part_number=3 chunk_size=4194304}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="51d4cc6f9915c5ee9ee0aa200dd9a4fc" sha256="dFBIHlyoxPGJ4VlSMYqay9DghFdIWBE9ypf+1YP5zUg="
2023-01-05T09:21:19.169221Z DEBUG ThreadId(551) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="158fb90af4014d106e211e2b16e16f3b" sha256="XzRUhiUsYoETL5lGdqdUbs5PAqqcNymF//OPhNn+Nh8="
2023-01-05T09:21:19.169304Z DEBUG ThreadId(551) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjkwNTQ1ZjJkLTRlNjUtNGY2Yi04MzI4LTE1ZWJjNmYxYTM4OQ" bucket=36e67a5e-large-object-no-filters-tgt}: ssstar::objstore::s3: All parts uploaded; completing multi-part upload total_parts=3 total_bytes=20971520
2023-01-05T09:21:19.173977Z DEBUG ThreadId(546) process_tar_entries{target_prefix="" target_bucket="36e67a5e-large-object-no-filters-tgt"}: ssstar::extract: Finished multipart file upload path=test len=20971520 key=test
2023-01-05T09:21:19.174034Z DEBUG ThreadId(546) process_tar_entries{target_prefix="" target_bucket="36e67a5e-large-object-no-filters-tgt"}: ssstar::extract: Entry sender dropped; no more tar entries to process
2023-01-05T09:21:19.174151Z DEBUG ThreadId(546) ssstar::extract: Tar entry processor task completed successfully
2023-01-05T09:21:19.174252Z INFO ThreadId(541) run{source_archive=File { path: "/tmp/extract-test.xRM9SSeWU5Tc/test.tar", metadata: Metadata { file_type: FileType(FileType { mode: 33188 }), is_dir: false, is_file: true, permissions: Permissions(FilePermissions { mode: 33188 }), modified: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 379078325 }), accessed: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), created: Ok(SystemTime { tv_sec: 1672910477, tv_nsec: 231080322 }), .. } } target_bucket="36e67a5e-large-object-no-filters-tgt" target_prefix=}: ssstar::extract: Finished extract job
---- objstore::minio::extract::large_object_no_filters::extract_from_archive_reader_to_bucket_root stdout ----
[ssstar-testing/src/test_data.rs:352] object.checksum_sha256 = Some(
"Hv+/sEGd/KeOPSnJWNnAZNfRcDobv6Vax7E/ibYN+dg=",
)
thread 'objstore::minio::extract::large_object_no_filters::extract_from_archive_reader_to_bucket_root' panicked at 'assertion failed: `(left == right)`
left: `[30, 255, 191, 176, 65, 157, 252, 167, 142, 61, 41, 201, 88, 217, 192, 100, 215, 209, 112, 58, 27, 191, 165, 90, 199, 177, 63, 137, 182, 13, 249, 216]`,
right: `[197, 73, 189, 91, 244, 117, 46, 235, 161, 29, 17, 240, 75, 58, 190, 107, 150, 8, 170, 40, 102, 105, 21, 121, 136, 3, 231, 129, 31, 146, 49, 74]`: S3 object 'test' (key 'test') hash doesn't match expected value', /media/core/kingston-luks/ssstar/ssstar-testing/src/test_data.rs:481:9
Log events from this test:
2023-01-05T09:21:12.707311Z DEBUG ThreadId(569) ssstar_testing::minio: get() found minio server endpoint=127.0.0.1:36313
2023-01-05T09:21:12.721617Z DEBUG ThreadId(569) ssstar_testing::minio: Creating bucket bucket="large-object-no-filters-src"
2023-01-05T09:21:12.721705Z DEBUG ThreadId(569) ssstar_testing::minio: Transformed bucket name into valid and unique bucket ID bucket=942b2a53-large-object-no-filters-src
2023-01-05T09:21:12.752741Z DEBUG ThreadId(569) ssstar_testing::minio: Bucket created bucket=942b2a53-large-object-no-filters-src
2023-01-05T09:21:12.752869Z DEBUG ThreadId(569) ssstar_testing::minio: Creating bucket bucket="large-object-no-filters-tgt"
2023-01-05T09:21:12.752930Z DEBUG ThreadId(569) ssstar_testing::minio: Transformed bucket name into valid and unique bucket ID bucket=1b40191a-large-object-no-filters-tgt
2023-01-05T09:21:12.774438Z DEBUG ThreadId(569) ssstar_testing::minio: Bucket created bucket=1b40191a-large-object-no-filters-tgt
2023-01-05T09:21:17.160341Z DEBUG ThreadId(569) ssstar::create: Adding archive input url=s3://942b2a53-large-object-no-filters-src/
2023-01-05T09:21:17.163146Z DEBUG ThreadId(569) ssstar::objstore::s3: Validating access to bucket bucket="942b2a53-large-object-no-filters-src"
2023-01-05T09:21:17.165441Z DEBUG ThreadId(569) ssstar::objstore::s3: Access to bucket is confirmed bucket="942b2a53-large-object-no-filters-src" region=None
2023-01-05T09:21:17.167459Z DEBUG ThreadId(569) ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="942b2a53-large-object-no-filters-src" versioning_enabled=true
2023-01-05T09:21:17.167521Z DEBUG ThreadId(569) ssstar::create: Confirmed bucket access for input url=s3://942b2a53-large-object-no-filters-src/ bucket=S3Bucket { name: "942b2a53-large-object-no-filters-src", versioning_enabled: true, region: None, client: "<...>" }
2023-01-05T09:21:17.167562Z DEBUG ThreadId(569) ssstar::create: Adding archive input to job input=CreateArchiveInput { bucket: S3Bucket { name: "942b2a53-large-object-no-filters-src", versioning_enabled: true, region: None, client: "<...>" }, selector: Bucket }
2023-01-05T09:21:17.167608Z DEBUG ThreadId(569) ssstar::create: Listing objects for all inputs input_count=1
2023-01-05T09:21:17.167729Z DEBUG ThreadId(569) into_input_objects: ssstar::create: Listing all object store objects that match this archive input self=Bucket 942b2a53-large-object-no-filters-src
2023-01-05T09:21:17.167778Z DEBUG ThreadId(569) into_input_objects: ssstar::objstore::s3: Archive input matches everything in an S3 bucket bucket=942b2a53-large-object-no-filters-src
2023-01-05T09:21:17.180230Z DEBUG ThreadId(569) into_input_objects: ssstar::objstore::s3: Matched all S3 objects in an S3 bucket input_objects=1 bucket=942b2a53-large-object-no-filters-src
2023-01-05T09:21:17.180277Z DEBUG ThreadId(569) into_input_objects: ssstar::create: Listing matching objects completed self=Bucket 942b2a53-large-object-no-filters-src count=1
2023-01-05T09:21:17.180366Z DEBUG ThreadId(569) ssstar::create: Listed all objects in all inputs object_count=1
2023-01-05T09:21:17.180769Z DEBUG ThreadId(576) read_object_part{key="test" version_id=Some("acbc5ebb-52d3-4229-ade4-2c6e088786a1") byte_range=0..8388608}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.181664Z DEBUG ThreadId(576) read_object_part{key="test" version_id=Some("acbc5ebb-52d3-4229-ade4-2c6e088786a1") byte_range=8388608..16777216}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.182555Z DEBUG ThreadId(576) read_object_part{key="test" version_id=Some("acbc5ebb-52d3-4229-ade4-2c6e088786a1") byte_range=16777216..20971520}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.239663Z DEBUG ThreadId(569) ssstar::create: Reading object and writing to tar archive key=test size=20971520
2023-01-05T09:21:17.372983Z DEBUG ThreadId(569) ssstar::create: Streamed object data into tar archive key=test size=20971520 final_part_number=2
2023-01-05T09:21:17.373153Z DEBUG ThreadId(569) ssstar::create: Completed processing of all input objects
2023-01-05T09:21:17.392166Z DEBUG ThreadId(569) ssstar::objstore::s3: Validating access to bucket bucket="1b40191a-large-object-no-filters-tgt"
2023-01-05T09:21:17.394264Z DEBUG ThreadId(569) ssstar::objstore::s3: Access to bucket is confirmed bucket="1b40191a-large-object-no-filters-tgt" region=None
2023-01-05T09:21:17.395722Z DEBUG ThreadId(569) ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="1b40191a-large-object-no-filters-tgt" versioning_enabled=true
2023-01-05T09:21:17.395921Z INFO ThreadId(569) run{source_archive=Reader("dyn Read") target_bucket="1b40191a-large-object-no-filters-tgt" target_prefix=}: ssstar::extract: Starting extract archive job self.filters=[] self.archive_size=None
2023-01-05T09:21:17.396068Z DEBUG ThreadId(575) ssstar::extract: Starting tar entry processor task
2023-01-05T09:21:17.396069Z DEBUG ThreadId(659) run{source_archive=Reader("dyn Read") target_bucket="1b40191a-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=Reader("dyn Read")}: ssstar::extract: Starting blocking tar read task
2023-01-05T09:21:17.396312Z DEBUG ThreadId(659) run{source_archive=Reader("dyn Read") target_bucket="1b40191a-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=Reader("dyn Read")}:Processing tar entry{path=test len=20971520}: ssstar::extract: No filters are applied so this entry is included
2023-01-05T09:21:17.396372Z DEBUG ThreadId(659) run{source_archive=Reader("dyn Read") target_bucket="1b40191a-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=Reader("dyn Read")}:Processing tar entry{path=test len=20971520}: ssstar::extract: Processing entry as multi-part
2023-01-05T09:21:17.396524Z DEBUG ThreadId(578) process_tar_entries{target_prefix="" target_bucket="1b40191a-large-object-no-filters-tgt"}: ssstar::extract: Starting multipart file upload path=test len=20971520 key=test
2023-01-05T09:21:17.399513Z DEBUG ThreadId(575) process_tar_entries{target_prefix="" target_bucket="1b40191a-large-object-no-filters-tgt"}: ssstar::extract: Uploading file part path=test len=8388608
2023-01-05T09:21:17.402012Z DEBUG ThreadId(582) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}: ssstar::objstore::s3: Commencing multi-part upload
2023-01-05T09:21:17.403807Z DEBUG ThreadId(578) process_tar_entries{target_prefix="" target_bucket="1b40191a-large-object-no-filters-tgt"}: ssstar::extract: Uploading file part path=test len=8388608
2023-01-05T09:21:17.404705Z DEBUG ThreadId(659) run{source_archive=Reader("dyn Read") target_bucket="1b40191a-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=Reader("dyn Read")}: ssstar::extract: Completed processing all tar entries
2023-01-05T09:21:17.404770Z DEBUG ThreadId(659) run{source_archive=Reader("dyn Read") target_bucket="1b40191a-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=Reader("dyn Read")}: ssstar::extract: Blocking tar read task completing successfully
2023-01-05T09:21:17.404961Z DEBUG ThreadId(578) process_tar_entries{target_prefix="" target_bucket="1b40191a-large-object-no-filters-tgt"}: ssstar::extract: Uploading final file part path=test len=4194304
2023-01-05T09:21:17.409838Z DEBUG ThreadId(575) ssstar::writers: encountered end of duplex stream; worker task exiting
2023-01-05T09:21:17.410043Z DEBUG ThreadId(575) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:18.313150Z DEBUG ThreadId(575) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:18.906212Z DEBUG ThreadId(575) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}:upload chunk{part_number=3 chunk_size=4194304}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:19.229606Z DEBUG ThreadId(578) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}:upload chunk{part_number=3 chunk_size=4194304}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="4c34568466e02ee3121680c2614ede27" sha256="jAipFqjheVwBa9dyT4tDXt/F3KycAZE78VRh+LKsuMQ="
2023-01-05T09:21:19.275636Z DEBUG ThreadId(578) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="235135148df76539bcc75028fc7d296c" sha256="idCNISzNqgOBg3Yd/+eL086l4beX/DpJO7xOIc3nLdU="
2023-01-05T09:21:19.289588Z DEBUG ThreadId(578) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="2316297fbbc4ce2304ada31aebdeef0b" sha256="RGicJ57GwOyzxneEAMJTNRSvpwPjm9tQIL266L8rb0g="
2023-01-05T09:21:19.289679Z DEBUG ThreadId(578) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyZDkxNjk3LWJhMzgtNGFhMC04OGJjLTFkZGQ5NWIyMWY1Mg" bucket=1b40191a-large-object-no-filters-tgt}: ssstar::objstore::s3: All parts uploaded; completing multi-part upload total_parts=3 total_bytes=20971520
2023-01-05T09:21:19.294778Z DEBUG ThreadId(582) process_tar_entries{target_prefix="" target_bucket="1b40191a-large-object-no-filters-tgt"}: ssstar::extract: Finished multipart file upload path=test len=20971520 key=test
2023-01-05T09:21:19.294842Z DEBUG ThreadId(582) process_tar_entries{target_prefix="" target_bucket="1b40191a-large-object-no-filters-tgt"}: ssstar::extract: Entry sender dropped; no more tar entries to process
2023-01-05T09:21:19.294952Z DEBUG ThreadId(582) ssstar::extract: Tar entry processor task completed successfully
2023-01-05T09:21:19.295036Z INFO ThreadId(569) run{source_archive=Reader("dyn Read") target_bucket="1b40191a-large-object-no-filters-tgt" target_prefix=}: ssstar::extract: Finished extract job
---- objstore::minio::extract::large_object_no_filters::extract_from_archive_on_bucket_to_bucket_root stdout ----
[ssstar-testing/src/test_data.rs:352] object.checksum_sha256 = Some(
"4Hwz+RBQEn4A3k2kk5NMVo3bU0hqokM6SGWCF+smkh4=",
)
thread 'objstore::minio::extract::large_object_no_filters::extract_from_archive_on_bucket_to_bucket_root' panicked at 'assertion failed: `(left == right)`
left: `[224, 124, 51, 249, 16, 80, 18, 126, 0, 222, 77, 164, 147, 147, 76, 86, 141, 219, 83, 72, 106, 162, 67, 58, 72, 101, 130, 23, 235, 38, 146, 30]`,
right: `[159, 191, 191, 89, 94, 97, 27, 45, 149, 152, 86, 221, 215, 69, 206, 30, 192, 233, 54, 117, 21, 49, 72, 182, 220, 204, 178, 208, 162, 83, 94, 186]`: S3 object 'test' (key 'test') hash doesn't match expected value', /media/core/kingston-luks/ssstar/ssstar-testing/src/test_data.rs:481:9
Log events from this test:
2023-01-05T09:21:12.615415Z DEBUG ThreadId(556) ssstar_testing::minio: get() found minio server endpoint=127.0.0.1:36313
2023-01-05T09:21:12.623747Z DEBUG ThreadId(556) ssstar_testing::minio: Creating bucket bucket="large-object-no-filters-src"
2023-01-05T09:21:12.623895Z DEBUG ThreadId(556) ssstar_testing::minio: Transformed bucket name into valid and unique bucket ID bucket=ded5cceb-large-object-no-filters-src
2023-01-05T09:21:12.656439Z DEBUG ThreadId(556) ssstar_testing::minio: Bucket created bucket=ded5cceb-large-object-no-filters-src
2023-01-05T09:21:12.656565Z DEBUG ThreadId(556) ssstar_testing::minio: Creating bucket bucket="large-object-no-filters-tgt"
2023-01-05T09:21:12.656620Z DEBUG ThreadId(556) ssstar_testing::minio: Transformed bucket name into valid and unique bucket ID bucket=417610ee-large-object-no-filters-tgt
2023-01-05T09:21:12.676288Z DEBUG ThreadId(556) ssstar_testing::minio: Bucket created bucket=417610ee-large-object-no-filters-tgt
2023-01-05T09:21:17.148228Z DEBUG ThreadId(556) ssstar::create: Adding archive input url=s3://ded5cceb-large-object-no-filters-src/
2023-01-05T09:21:17.150818Z DEBUG ThreadId(556) ssstar::objstore::s3: Validating access to bucket bucket="ded5cceb-large-object-no-filters-src"
2023-01-05T09:21:17.153044Z DEBUG ThreadId(556) ssstar::objstore::s3: Access to bucket is confirmed bucket="ded5cceb-large-object-no-filters-src" region=None
2023-01-05T09:21:17.157316Z DEBUG ThreadId(556) ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="ded5cceb-large-object-no-filters-src" versioning_enabled=true
2023-01-05T09:21:17.157382Z DEBUG ThreadId(556) ssstar::create: Confirmed bucket access for input url=s3://ded5cceb-large-object-no-filters-src/ bucket=S3Bucket { name: "ded5cceb-large-object-no-filters-src", versioning_enabled: true, region: None, client: "<...>" }
2023-01-05T09:21:17.157427Z DEBUG ThreadId(556) ssstar::create: Adding archive input to job input=CreateArchiveInput { bucket: S3Bucket { name: "ded5cceb-large-object-no-filters-src", versioning_enabled: true, region: None, client: "<...>" }, selector: Bucket }
2023-01-05T09:21:17.157478Z DEBUG ThreadId(556) ssstar::create: Listing objects for all inputs input_count=1
2023-01-05T09:21:17.157571Z DEBUG ThreadId(556) into_input_objects: ssstar::create: Listing all object store objects that match this archive input self=Bucket ded5cceb-large-object-no-filters-src
2023-01-05T09:21:17.157619Z DEBUG ThreadId(556) into_input_objects: ssstar::objstore::s3: Archive input matches everything in an S3 bucket bucket=ded5cceb-large-object-no-filters-src
2023-01-05T09:21:17.163252Z DEBUG ThreadId(556) into_input_objects: ssstar::objstore::s3: Matched all S3 objects in an S3 bucket input_objects=1 bucket=ded5cceb-large-object-no-filters-src
2023-01-05T09:21:17.163325Z DEBUG ThreadId(556) into_input_objects: ssstar::create: Listing matching objects completed self=Bucket ded5cceb-large-object-no-filters-src count=1
2023-01-05T09:21:17.163441Z DEBUG ThreadId(556) ssstar::create: Listed all objects in all inputs object_count=1
2023-01-05T09:21:17.166492Z DEBUG ThreadId(556) ssstar::objstore::s3: Validating access to bucket bucket="ded5cceb-large-object-no-filters-src"
2023-01-05T09:21:17.168891Z DEBUG ThreadId(556) ssstar::objstore::s3: Access to bucket is confirmed bucket="ded5cceb-large-object-no-filters-src" region=None
2023-01-05T09:21:17.176675Z DEBUG ThreadId(556) ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="ded5cceb-large-object-no-filters-src" versioning_enabled=true
2023-01-05T09:21:17.179524Z DEBUG ThreadId(566) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}: ssstar::objstore::s3: Commencing multi-part upload
2023-01-05T09:21:17.179735Z DEBUG ThreadId(566) read_object_part{key="test" version_id=Some("f228b958-18ed-4b32-bdd0-1b6e5170df81") byte_range=0..8388608}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.180655Z DEBUG ThreadId(566) read_object_part{key="test" version_id=Some("f228b958-18ed-4b32-bdd0-1b6e5170df81") byte_range=8388608..16777216}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.181787Z DEBUG ThreadId(566) read_object_part{key="test" version_id=Some("f228b958-18ed-4b32-bdd0-1b6e5170df81") byte_range=16777216..20971520}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:17.226781Z DEBUG ThreadId(556) ssstar::create: Reading object and writing to tar archive key=test size=20971520
2023-01-05T09:21:17.283245Z DEBUG ThreadId(560) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:17.285523Z DEBUG ThreadId(556) ssstar::create: Streamed object data into tar archive key=test size=20971520 final_part_number=2
2023-01-05T09:21:17.285651Z DEBUG ThreadId(556) ssstar::create: Completed processing of all input objects
2023-01-05T09:21:17.296724Z DEBUG ThreadId(562) ssstar::writers: encountered end of duplex stream; worker task exiting
2023-01-05T09:21:18.159232Z DEBUG ThreadId(560) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:18.758224Z DEBUG ThreadId(566) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}:upload chunk{part_number=3 chunk_size=4195840}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:19.214740Z DEBUG ThreadId(566) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="62596a832fdc3e1a9d29eac2b9019047" sha256="nhSj9N6KuiJsewqRFvcpj2sGeKqNFFfVu+/MQrIMp9o="
2023-01-05T09:21:19.263943Z DEBUG ThreadId(562) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}:upload chunk{part_number=3 chunk_size=4195840}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="2ad9ddd548ce51682b7003a13df3e286" sha256="TapScgUQhSanZi0PUIr2rRxorZZOr1LKRQb+hD6k4wA="
2023-01-05T09:21:19.306664Z DEBUG ThreadId(562) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="0174f95a2087685991564d6220a77ce0" sha256="sddxXHtLFi0hzTTyH+QyGW5jWuFEQwkRFoy//bJBUDU="
2023-01-05T09:21:19.306798Z DEBUG ThreadId(562) multipart_object_writer{key="test.tar" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjM2M2FlNGM5LTBjMGQtNGIyOS1hMzAwLTY2MjM4ZmI2MTU4Zg" bucket=ded5cceb-large-object-no-filters-src}: ssstar::objstore::s3: All parts uploaded; completing multi-part upload total_parts=3 total_bytes=20973056
2023-01-05T09:21:19.311583Z DEBUG ThreadId(556) ssstar::create: Upload of tar archive to object storage completed bytes_written=20973056
2023-01-05T09:21:19.314992Z DEBUG ThreadId(556) ssstar::objstore::s3: Validating access to bucket bucket="417610ee-large-object-no-filters-tgt"
2023-01-05T09:21:19.316626Z DEBUG ThreadId(556) ssstar::objstore::s3: Access to bucket is confirmed bucket="417610ee-large-object-no-filters-tgt" region=None
2023-01-05T09:21:19.317815Z DEBUG ThreadId(556) ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="417610ee-large-object-no-filters-tgt" versioning_enabled=true
2023-01-05T09:21:19.319402Z DEBUG ThreadId(556) into_internal{self=ObjectStorage(Url { scheme: "s3", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("ded5cceb-large-object-no-filters-src")), port: None, path: "/test.tar", query: None, fragment: None })}: ssstar::objstore::s3: Validating access to bucket bucket="ded5cceb-large-object-no-filters-src"
2023-01-05T09:21:19.320787Z DEBUG ThreadId(556) into_internal{self=ObjectStorage(Url { scheme: "s3", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("ded5cceb-large-object-no-filters-src")), port: None, path: "/test.tar", query: None, fragment: None })}: ssstar::objstore::s3: Access to bucket is confirmed bucket="ded5cceb-large-object-no-filters-src" region=None
2023-01-05T09:21:19.321993Z DEBUG ThreadId(556) into_internal{self=ObjectStorage(Url { scheme: "s3", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("ded5cceb-large-object-no-filters-src")), port: None, path: "/test.tar", query: None, fragment: None })}: ssstar::objstore::s3: Checked if bucket versioning is enabled bucket="ded5cceb-large-object-no-filters-src" versioning_enabled=true
2023-01-05T09:21:19.323489Z INFO ThreadId(556) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}: ssstar::extract: Starting extract archive job self.filters=[] self.archive_size=Some(20973056)
2023-01-05T09:21:19.323585Z DEBUG ThreadId(556) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}:read_object{key="test.tar" version_id=None byte_range=0..20973056}: ssstar::objstore::s3: Reading object
2023-01-05T09:21:19.323714Z DEBUG ThreadId(566) ssstar::extract: Starting tar entry processor task
2023-01-05T09:21:19.323744Z DEBUG ThreadId(664) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 }}: ssstar::extract: Starting blocking tar read task
2023-01-05T09:21:19.323767Z DEBUG ThreadId(561) read_object_part{key="test.tar" version_id=None byte_range=0..8388608}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:19.324520Z DEBUG ThreadId(561) read_object_part{key="test.tar" version_id=None byte_range=8388608..16777216}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:19.325275Z DEBUG ThreadId(561) read_object_part{key="test.tar" version_id=None byte_range=16777216..20973056}: ssstar::objstore::s3: Reading partial object
2023-01-05T09:21:19.341362Z DEBUG ThreadId(664) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 }}:Processing tar entry{path=test len=20971520}: ssstar::extract: No filters are applied so this entry is included
2023-01-05T09:21:19.341442Z DEBUG ThreadId(664) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 }}:Processing tar entry{path=test len=20971520}: ssstar::extract: Processing entry as multi-part
2023-01-05T09:21:19.341592Z DEBUG ThreadId(562) process_tar_entries{target_prefix="" target_bucket="417610ee-large-object-no-filters-tgt"}: ssstar::extract: Starting multipart file upload path=test len=20971520 key=test
2023-01-05T09:21:19.344156Z DEBUG ThreadId(568) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}: ssstar::objstore::s3: Commencing multi-part upload
2023-01-05T09:21:19.345256Z DEBUG ThreadId(561) ssstar::objstore::s3: Read object async task exiting key=test.tar
2023-01-05T09:21:19.345324Z DEBUG ThreadId(568) process_tar_entries{target_prefix="" target_bucket="417610ee-large-object-no-filters-tgt"}: ssstar::extract: Uploading file part path=test len=8388608
2023-01-05T09:21:19.348521Z DEBUG ThreadId(568) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:19.349002Z DEBUG ThreadId(561) process_tar_entries{target_prefix="" target_bucket="417610ee-large-object-no-filters-tgt"}: ssstar::extract: Uploading file part path=test len=8388608
2023-01-05T09:21:19.349793Z DEBUG ThreadId(664) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 }}: ssstar::extract: Completed processing all tar entries
2023-01-05T09:21:19.349862Z DEBUG ThreadId(664) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}:read_tar_entries_blocking{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 }}: ssstar::extract: Blocking tar read task completing successfully
2023-01-05T09:21:19.349928Z DEBUG ThreadId(561) process_tar_entries{target_prefix="" target_bucket="417610ee-large-object-no-filters-tgt"}: ssstar::extract: Uploading final file part path=test len=4194304
2023-01-05T09:21:19.353761Z DEBUG ThreadId(561) ssstar::writers: encountered end of duplex stream; worker task exiting
2023-01-05T09:21:19.873379Z DEBUG ThreadId(561) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:20.391471Z DEBUG ThreadId(561) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}:upload chunk{part_number=3 chunk_size=4194304}: ssstar::objstore::s3: Uploading multi-part chunk
2023-01-05T09:21:20.649335Z DEBUG ThreadId(561) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}:upload chunk{part_number=1 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="cd9d737be43363631fd63930e4364265" sha256="HXrcPJax2dkdkfXigZCkE7FvLgnApUiz4Cojq17oHYc="
2023-01-05T09:21:20.681648Z DEBUG ThreadId(562) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}:upload chunk{part_number=3 chunk_size=4194304}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="f8a486ac6fab5d26803408266cfcb779" sha256="x0uxBaO+Kqe3TEYGUlIsYkzNDcR1N3+taaSy/YtJmfo="
2023-01-05T09:21:20.753750Z DEBUG ThreadId(562) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}:upload chunk{part_number=2 chunk_size=8388608}: ssstar::objstore::s3: Uploaded multi-part chunk e_tag="3bfb5b5af065a014d1e45206ac07b0b9" sha256="tS094WSkFca1apZJ5pvEFGRm6awW7YBHUR92WX+z7s8="
2023-01-05T09:21:20.753885Z DEBUG ThreadId(562) multipart_object_writer{key="test" upload_id="YjU1YWQ4ZDMtMWY3NS00ZDgyLWJkNGYtOWQxZDFiM2VlNDhmLjEyODkyMzY1LWYwMjEtNDFlZC04MDMyLWYyOGUwMDgyNDgzNg" bucket=417610ee-large-object-no-filters-tgt}: ssstar::objstore::s3: All parts uploaded; completing multi-part upload total_parts=3 total_bytes=20971520
2023-01-05T09:21:20.758557Z DEBUG ThreadId(560) process_tar_entries{target_prefix="" target_bucket="417610ee-large-object-no-filters-tgt"}: ssstar::extract: Finished multipart file upload path=test len=20971520 key=test
2023-01-05T09:21:20.758630Z DEBUG ThreadId(560) process_tar_entries{target_prefix="" target_bucket="417610ee-large-object-no-filters-tgt"}: ssstar::extract: Entry sender dropped; no more tar entries to process
2023-01-05T09:21:20.758751Z DEBUG ThreadId(560) ssstar::extract: Tar entry processor task completed successfully
2023-01-05T09:21:20.758826Z INFO ThreadId(556) run{source_archive=ObjectStorage { bucket: "ded5cceb-large-object-no-filters-src", key: "test.tar", version_id: None, len: 20973056 } target_bucket="417610ee-large-object-no-filters-tgt" target_prefix=}: ssstar::extract: Finished extract job
failures:
objstore::minio::extract::large_object_no_filters::extract_from_archive_file_to_bucket_root
objstore::minio::extract::large_object_no_filters::extract_from_archive_on_bucket_to_bucket_root
objstore::minio::extract::large_object_no_filters::extract_from_archive_reader_to_bucket_root
test result: FAILED. 39 passed; 3 failed; 3 ignored; 0 measured; 0 filtered out; finished in 16.52s
core@core-MSI:/media/core/kingston-luks/ssstar$
Bug description
I tried to run all tests but some of them fails:
test objstore::minio::extract::large_object_no_filters::extract_from_archive_reader_to_bucket_root ... FAILED
test objstore::minio::extract::large_object_no_filters::extract_from_archive_file_to_bucket_root ... FAILED
test objstore::minio::extract::large_object_no_filters::extract_from_archive_on_bucket_to_bucket_root ... FAILED
full logs:
To Reproduce
I just ran tests as is shown in CONTRIBUTING
Expected behavior
All tests have to be passed
Screenshots
Environment
I have installed minio by steps from https://unixcop.com/install-minio-ubuntu/
OS info:
It is on the latest master commit ff689cd
Additional context
This problem is only in tests, nothing related to production code