Feature request: Renaming of files

[fippu82]

@engswee, hi!
I realized that SAP has a strange way to create the .iflw files in downloaded Iflow artifacts (regardless of via Web UI or API). The filename of the .iflw file is wrong if the Iflow was previously copied from another one. In this case, the filename is equal the original Iflow that this flow is copied from.
The result is a confusing repository with filenames that don't correspond the actual Iflow name. The correct Iflow name however is used for the root folder of each Iflow in the package (and is visible of course in the .project and MANIFEST files).

I tried to overcome this by adding some renaming logic as a shell script that is called in my pipeline and then commits/pushes the renamed files after flashpipe snapshot ran. That works well, however, next time flashpipe runs, it updates everything in the repo again due to the changed filename. So it's back and forth from wrong to correct filename each time the pipeline runs. This clutters up the commits...

Would it be possible to ignore filename changes of the .iflw file in the diff command that flashpipe executes? This name usually can't be changed by the user, so it should be safe to ignore the change.

Ideally, I suggest to build in the correction of the filenames directly in flashpipe too.

Kind regards,
Philippe

[fippu82]

PS: I contacted SAP about this and after checking with their devs they said that this is intended behavior... Makes no sense to me but seems we have to live with it.

[engswee]

Hey @fippu82

Yes I am well aware of this happening when the IFlow is copied. The issue is that whatever you do in the Git repo is not reflected back in the tenant, which is the "source" of the problem. What I personally do is to rename the .iflw file in the Git repo and then either run sync or update to force the change back into the tenant. This is a one time change for every copied IFlow, and then subsequent sync or snapshot from the tenant would be fine.

For your two suggestions

• ignore filename changes of .iflw - the diff command is run as a Unix executable and I'm not that keen on messing on that part
• include file name correction in flashpipe - I'm also reluctant to do this as I want to avoid tampering with the content of the IFlow as much as possible. Plus as mentioned above, this does not solve the "source" of the problem which is the incorrect file name that is still in the tenant (although it is not visible through the UI).

[fippu82]

@engswee Thanks for your answer. Fair points! As long as we have a way to force the changed filenames back into the tenant, I'm fine with the approach. But could you please give me some more hints about how to achieve this? As mentioned, I run a snapshot to download the whole tenant daily, then rename the files in batch and commit/push all changes. When I look at the docs I see that 'sync' requires to specify a package ID. I'm rather looking for something that pushes the whole repo back to the tenant (hopefully without overwriting changes done there in the meantime while the job is running). Wouldn't 'snapshot restore' be the right command for this? Or do you suggest to run 'update artifact' or maybe 'update package' somehow after each renamed file?

[engswee]

@fippu82
It depends on whether you are the only developer working on the tenant, or if there is anyone else. If you are the only one, then you can make sure that nothing else has changed before running snapshot restore.

If there are others, then it's probably safer to run sync or update artifact - it will depend on whether you are responsible for everything within a single package, or there could be multiple developers working in the same package.

[fippu82]

@engswee
I'm struggling a bit... may I ask you for some more hints?
After renaming the file in the repo (and committing/pushing it), I tried to run flashpipe update artifact. That fails with an error "Integration flow project must contain source folders."

Now I'm trying to use sync. However, what this does is it downloads the artifact from the tenant again and overwrites the local version of the ZIP in which I've just renamed the file... So besides renaming the file, do I need to update something else so that flashpipe correctly detects the local change? Strangely, flashpipe thinks that the artifact doesn't exist yet in the repo, although it's there:

"Downloaded artifact unzipped to /tmp/download/IF1068
🏆 Artifact IF1068 does not exist, and will be added to Git"

How do you handle this?

Thank you!

[fippu82]

PS: Please finde below the output of the update artifact command (before this was run, the pipeline renamed the file IF1340__xxx to IF1410__xxx:

flashpipe update artifact --tmn-host m-xxx-xx.it-cpi0xx.cfapps.xxx.hana.ondemand.com --oauth-host m-cpidev-xx.authentication.ch20.hana.ondemand.com --oauth-clientid xxx --oauth-clientsecret [MASKED] --artifact-id IF1410 --dir-artifact backup/FI2013/IF1410 --package-id FI2013
30 Mar 26 09:09 UTC INF Executing update artifact Integration command
30 Mar 26 09:09 UTC INF Using package ID FI2013 as package name
30 Mar 26 09:09 UTC INF Using IF1410__xxx from Bundle-Name in MANIFEST.MF as artifact name
30 Mar 26 09:09 UTC INF Getting details of integration package FI2013
30 Mar 26 09:09 UTC INF Getting details of Integration designtime artifact IF1410
30 Mar 26 09:09 UTC INF Active version of artifact IF1410 exists
30 Mar 26 09:09 UTC INF Getting Integration designtime artifacts of package FI2013
30 Mar 26 09:09 UTC INF Checking if designtime artifact needs to be updated
30 Mar 26 09:09 UTC INF Getting content of artifact IF1410 from tenant for comparison
30 Mar 26 09:09 UTC INF Getting content of Integration designtime artifact IF1410
30 Mar 26 09:09 UTC INF Content of artifact IF1410 downloaded to /tmp/IF1410.zip
30 Mar 26 09:09 UTC INF Unzipping downloaded designtime artifact /tmp/IF1410.zip to /tmp/download
30 Mar 26 09:09 UTC INF Checking for changes in META-INF directory
30 Mar 26 09:09 UTC INF Executing command: diff --ignore-matching-lines=^Origin.* --strip-trailing-cr --recursive --ignore-all-space --ignore-blank-lines --exclude=parameters.prop --exclude=.DS_Store backup/FI2013/IF1410/META-INF /tmp/download/META-INF
30 Mar 26 09:09 UTC INF Checking for changes in src/main/resources directory
30 Mar 26 09:09 UTC INF Executing command: diff --ignore-matching-lines=^Origin.* --strip-trailing-cr --recursive --ignore-all-space --ignore-blank-lines --exclude=parameters.prop --exclude=.DS_Store backup/FI2013/IF1410/src/main/resources /tmp/download/src/main/resources
30 Mar 26 09:09 UTC INF Diff results:
Only in backup/FI2013/IF1410/src/main/resources/scenarioflows: IF1410.iflw
Only in /tmp/download/src/main/resources/scenarioflows/integrationflow: IF1340__xxx.iflw
30 Mar 26 09:09 UTC INF Checking for changes in metainfo.prop
30 Mar 26 09:09 UTC INF Executing command: diff --ignore-matching-lines=^#.* --strip-trailing-cr --ignore-all-space --ignore-blank-lines backup/FI2013/IF1410/metainfo.prop /tmp/download/metainfo.prop
30 Mar 26 09:09 UTC INF Changes found in designtime artifact. Designtime artifact will be updated in CPI tenant
30 Mar 26 09:09 UTC INF Updating Integration designtime artifact IF1410
30 Mar 26 09:09 UTC WRN Response body = {"error":{"code":"Bad Request","message":{"lang":"en","value":"Integration flow project must contain source folders."}}}
30 Mar 26 09:09 UTC FTL Update Integration designtime artifact call failed with response code = 400

[engswee]

Hi @fippu82, I've been away and so took a while to get to look into this. Did you manage to get it working in the meantime?

I've never seen this Integration flow project must contain source folders error before! And it is a bit hard to analyse this without knowing how your directory/files are organised and any other scripts you are running.

If this is still not working for you, I'd suggest trying this out in a different repo with the following steps:

1. Run flashpipe sync --target git (with all other necessary parameters) to sync a package from tenant to Git
2. Manually rename the .iflw file with no other changes, and commit/push to Git
3. Run flashpipe sync --target tenant (with all other necessary parameters) to sync the changes back to tenant

Hopefully a change with a reduced scope like this would work. If it does (or not), maybe it could give further hints into what is going on