Skip to content

Conversation

@yuriyzubov
Copy link
Member

set number of retries = 2
change idle worker timeout from None, to 1800s (30min)

Copilot AI review requested due to automatic review settings July 29, 2025 19:53

This comment was marked as outdated.

@yuriyzubov yuriyzubov requested a review from mzouink July 29, 2025 19:54
id computing inference if chunk is not empty
@yuriyzubov yuriyzubov requested a review from Copilot July 31, 2025 14:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR configures the Daisy task scheduler with improved retry and timeout settings while optimizing block processing efficiency by checking if blocks are already processed before expensive inference computation.

  • Adds retry capability with 2 maximum retries for task resilience
  • Sets worker timeout to 30 minutes (1800 seconds) to prevent indefinite waiting
  • Optimizes processing by checking block completion status earlier in the pipeline

Comment on lines +156 to +157
fill_value = getattr(self.output_arrays[0], 'fill_value', self.dtype(0))
if not (self.output_arrays[0][write_roi] == fill_value).all():
Copy link

Copilot AI Jul 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logic is inverted compared to the original check. The original code used if self.output_arrays[0][block.write_roi].any(): which returns early if any non-zero values exist. The new condition if not (self.output_arrays[0][write_roi] == fill_value).all(): also returns early when any values are not equal to fill_value, but this assumes fill_value represents 'unprocessed' state. If fill_value is 0 and processed blocks contain non-zero values, these conditions are equivalent. However, if fill_value is non-zero or processed blocks can contain the fill_value, this could cause incorrect behavior.

Suggested change
fill_value = getattr(self.output_arrays[0], 'fill_value', self.dtype(0))
if not (self.output_arrays[0][write_roi] == fill_value).all():
# Check if block is already processed before expensive inference computation
if self.output_arrays[0][write_roi].any():

Copilot uses AI. Check for mistakes.
return

# Check if block is already processed before expensive inference computation
fill_value = getattr(self.output_arrays[0], 'fill_value', self.dtype(0))
Copy link

Copilot AI Jul 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The fallback to self.dtype(0) assumes that zero represents unprocessed blocks, but this may not always be correct. Consider using a more explicit approach to determine the unprocessed state, or document this assumption clearly.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants