Skip to content

Optionally copy files from step to runner #291

@vvanouytsel

Description

@vvanouytsel

The consensus is that all steps in a job should be able to share files. Previously this was done via a PersistentVolume and since v0.8 this is handled via copying files from the workflow pod to the runner pod and vice versa.

However, if you have a job that only contains of steps that do not run a step container, you do not need the copy functionality because everything runs in the same workflow pod.

Assume that you have 8 of these 'run' steps in a job, and you are working with a node project with quite some dependencies, you will notice that your node_module directory gets quite large. Currently for every step, your entire working directory (including your node_modules) is copied over the the runner. For then to be copied back to the same workflow pod. This adds some massive delays that can be avoided. I know you can work around this by putting all those 'run' steps in a single step, but that does not seem very elegant to me.

Can we make it so that the copy functionality is only executed if there is at least one container action in your job. Because if there is no container action present in your job, I do not see the value of copying everything to the runner and back to the workflow pod.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions