Context
Currently, the workflow execution counter in Redis uses a 1-hour TTL that is refreshed on every beginExecutionSetStatus call. This design assumes all workflow executions complete within 1 hour.
Problem
If we introduce longer-running workflows in the future, the Redis counter key could expire mid-execution, leading to:
- Negative counter values when
endExecutionSetStatus decrements a missing/expired key
- Incorrect workflow
active status in the database
- Potential state inconsistencies
Proposed Enhancement
Update the Redis TTL handling logic:
beginExecutionSetStatus: Remove any TTL when count > 0 (use PERSIST)endExecutionSetStatus: Only set the INACTIVITY_TIMEOUT TTL when count reaches 0
This ensures the counter persists while executions are active and only starts the inactivity timer once all executions complete.
Current Workaround
The current implementation is acceptable since we don't currently support workflows that run longer than 1 hour. This issue tracks the enhancement for future scalability.
References
Context
Currently, the workflow execution counter in Redis uses a 1-hour TTL that is refreshed on every
beginExecutionSetStatuscall. This design assumes all workflow executions complete within 1 hour.Problem
If we introduce longer-running workflows in the future, the Redis counter key could expire mid-execution, leading to:
endExecutionSetStatusdecrements a missing/expired keyactivestatus in the databaseProposed Enhancement
Update the Redis TTL handling logic:
beginExecutionSetStatus: Remove any TTL when count > 0 (usePERSIST)endExecutionSetStatus: Only set the INACTIVITY_TIMEOUT TTL when count reaches 0This ensures the counter persists while executions are active and only starts the inactivity timer once all executions complete.
Current Workaround
The current implementation is acceptable since we don't currently support workflows that run longer than 1 hour. This issue tracks the enhancement for future scalability.
References
apps/worker/src/processor/helper.ts