Skip to content

More aggressive pattern copy + rsync replacement.#987

Open
PortalG wants to merge 1 commit intosmartcmd:mainfrom
PortalG:better-copy
Open

More aggressive pattern copy + rsync replacement.#987
PortalG wants to merge 1 commit intosmartcmd:mainfrom
PortalG:better-copy

Conversation

@PortalG
Copy link
Contributor

@PortalG PortalG commented Mar 8, 2026

Description

There are three issues with the current copy configuration that have been bothering me:

  1. There are WAY too many unused files that end up in the output directory.
  2. The more compatible tar and cp utilities in Unix should've been used from the start rather than rsync. (kudos to @izzint)
  3. Patching Windows64Media is no longer necessary and causes duplicates, as other PRs have manually pushed all extra DLCs and other media into the folder instead.

This PR aims to fix the three inconveniences above.

Changes

  1. Add more file and directory exclusion patters to exclude every last file that the game doesn't need to run or function. (DLCs are considered a necessity for functionality in this context)
  2. Replace rsync calls with @izzint's tar and cp implementation from consider replacing rsync with cp and tar in CMakeLists.txt #558.
  3. Remove calls to copy DurangoMedia to Windows64Media, preventing duplicate DLCs and unnecessary file changes.

Previous Behavior

Robocopy and rsync would copy too many unused assets, take a long time to complete, and cause two different DLCs to be present. (note: all examples below are on an old drive to measure multithreaded performance better)

image image image

Root Cause

These issues weren't originally accounted for as my original PR was just to make sure that I and others could have a reliable, automated asset copy system. However, now the project is reaching a point were it should be more production ready. When making a software build, including assets that are unused adds unnecessary bloat and mess. It is always a good idea to strive towards clean builds as much as possible. Having two of the same DLC pop up twice is irritating for the user as well, and can be frustrating.

New Behavior

Robocopy and the new cp and tar utilities have better exclusion patterns and ignores (hopefully) every unnecessary file, prevents DLCs and other files from being copied twice, and keeps asset copies fast and clean:

image image image

Fix Implementation

After tinkering with the CMakeLists.txt file for a bit, I figured out more files and directories that aren't needed for the game to function on top of the original exclusions:

  • .xui and .inl source code files.
  • .cd project files.
  • .resx and .loc language files. (I don't actually know why these aren't used, but the game had no trouble displaying any text so I assume they are just leftovers)
  • .lib flies. (this one should have been obvious, but evidently I missed it in my first PR)
  • .swf files because everything is already compressed in the Media.arc files, so it's pointless.
  • .msscmp files because the format is currently being abandoned for .ogg files.
  • .wav files that are, for some reason, present but go completely unused.
  • Extra media files like MediaOrbis.arc, MediaDurango.arc, along all the PlayStation ones.
  • sce_sys and TROPDIR directories because they are data folders for PlayStation only.
  • Graphics, DummyTexturePack, and GameConfig directories.
  • ReadMe.txt and Network Implementation Notes.txt files.

Along with using cp and tar instead of rsync, I also made sure that /Windows64/GameHDD is always present to prevent unexpected crashes when trying to save the game.

AI Use Disclosure

All code written by me and @izzint, no AI was used.

Related Issues

  • consider replacing rsync with cp and tar in CMakeLists.txt #558
    I am only including @izzint's code in this PR because I have been waiting for 3 days for their pull request to get merged and it still hasn't. It would be pointless and a nuisance to force @izzint to adapt to these new changes rather than just building on top of her changes. All credit goes to her for the cp and tar implementation. Good find!

One final note, I would advise you test this on your own machine to make sure you it works before merging. I am honestly surprised with how much elimination I got away with, so I wouldn't be surprised if I missed something important.

@GabsPuNs
Copy link

GabsPuNs commented Mar 9, 2026

The DLCs where placed in Windows64Media because they are modified version of the xbox one DLCs, with 720p support and SkinHDWin and SkinWin instead of SkinHDDurango, sounds are in high res while xbox one ver is still the original, durangomedia is not used anymore since 2 or 3 days ago, thats why you cant see DLCs duplicated in Nightly Builds

@GabsPuNs
Copy link

GabsPuNs commented Mar 9, 2026

Oh, CMake was never modified, I guess that's why you get duplicates when compiling with CMake

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants