Improve DOS copy for data exporter#381
Conversation
> Based on issue SANDAG#374 (SANDAG#374). Copying model data from C: drive back to T: drive has been taking longer and occasionally failing. This PR modifies the script to improve copy speed of all non-EMME specific files and prioritizes copy of all non-EMME specific files over EMME files and directories. > ## Types of changes > What types of changes does your code introduce? _Put an `x` in the boxes that apply_ > > * [ ] Bugfix (non-breaking change which fixes an issue) > * [x] New feature (non-breaking change which adds functionality) > * [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) > * [ ] Documentation Update (if none of the other choices apply) > > ## How has this been tested? > Please describe the tests that you ran to verify your changes. > > * [x] Ran a partial 2050 run with data export step included. > > ## Checklist: > * [x] My code follows the style guidelines of this project > * [x] I have performed a self-review of my own code > * [x] I have commented my code, particularly in hard-to-understand areas > * [ ] I have made corresponding changes to the documentation > * [ ] My changes generate no new warnings > > ## Further comments > The new copy method has different error codes when compared to the previous method. A return code other than 0 or 1 is not an immediate flag, but should be examined.
fixed error return code check
|
@jchu-sandag could we please put the emme_bank copy command in a try-except bracket so that the runs still completes successfully but warns (in the log file?) the user that manual copying is required? |
@jchu-sandag and @aber-sandag based on yesterday's conversation, it looks like we don't want to do this and prefer that model crashes, right? |
This is already in a try-except block - https://github.com/SANDAG/ABM/blob/main/src/main/emme/toolbox/master_run.py#L840 |
4 participants
Proposed changes
Based on issue #374. Copying model data from C: drive back to T: drive has been taking longer and occasionally failing. This PR modifies the script to improve copy speed of all non-EMME specific files and prioritizes copy of all non-EMME specific files over EMME files and directories.
Impact
Replaces
xcopyDOS command withrobocopy(with a copy speed of ~7,500 MegaBytes/min).Types of changes
What types of changes does your code introduce to ABM?
Put an
xin the boxes that applyHow has this been tested?
Please describe the tests that you ran to verify your changes.
Checklist:
Further comments
The new copy method has different error codes when compared to the previous method. A return code other than 0 or 1 is not an immediate flag and is coded to not raise an error but should be examined. Error code 16 (no files were copied over) will be raised.