Rolling 6.1.0 doesn't start if opensearch backend contains previously created index

[schweigisito]

Describe the bug

If OpenCloud 4.0.x was connected to opensearch, and has created an opencloud-resource index, updating to 6.1.0 fails while starting the OpenCloud. The search service throws an error and OpenCloud is not started properly.

Steps to reproduce

1. Install current stable OpenCloud 4.0.x (e.g. via the helmfile deployment with included opensearch cluster deployment
2. Change deployment to use rolling 6.1.0
3. Restart OpenCloud pods/service

Expected behavior

• Either a new index is created (and the old one gets deleted)
• Or a throwing an error message telling the administrator what to do or what exactly needs to be done manually

Actual behavior

• While starting the search service, it gets logged:

opencloud: Failed service 'search' (1.000000 failures of 5.000000), restarting: true, error: failed to create OpenSearch backend: failed to apply index template: index opencloud-resource allready exists and is different from the requested version, manual action required: mappings.properties.Content
mappings.properties.Favorites

• After the fifth error the pod gets destroyed and restarted, or, depending on the deployment, the process gets restarted

• deleting the opencloud-resource index manually via

$ curl -X DELETE localhost:9200/opencloud-resource

(on an opensearch pod or against the cluster URL) partially fixes the problem. The pod/service is starting again.

• Unfortunately the search doesn't return any results.
• Trying to re-index, doesn't work either:

$ kubectl -n opencloud exec opencloud-api-6f687b6cf-4485z -- opencloud search index --all-spaces
Defaulted container "api" out of: api, fetch-password-list (init)
failed to index space: {"id":"go.micro.client","code":500,"detail":"service eu.opencloud.api.search: not found","status":"Internal Server Error"}
Error: {"id":"go.micro.client","code":500,"detail":"service eu.opencloud.api.search: not found","status":"Internal Server Error"}
Usage:
  opencloud search index [flags]

Aliases:
  index, i

Flags:
      --all-spaces     index all spaces instead. This or --space is required.
      --force-rescan   force a rescan of all files, even if they are already indexed. This will make the indexing process much slower, but ensures that the index is up-to-date using the current search service configuration.
  -h, --help           help for index
  -s, --space string   the id of the space to travers and index the files of. This or --all-spaces is required.

{"id":"go.micro.client","code":500,"detail":"service eu.opencloud.api.search: not found","status":"Internal Server Error"}
command terminated with exit code 1

[schweigisito]

The typo within the error message has been addressed in #2701

[aduffeck]

OpenCloud 6.0 introduced a breaking change regarding the opensearch index and the changelog linked to this issue: #2514. Did you upgrade from 5.x to 6.1 directly or was that a 6.0->6.1 upgrade? Regardless, I understand that there probably should have been a clearer statement on the release notes besides the "Breaking change" link to the issue.

The service eu.opencloud.api.search: not found error indicates that the search service didn't come up properly yet after you dropped the old index. Can you try to bring it up and run the reindex command again?

[schweigisito]

Hm:

opencloud-api-6f687b6cf-8lbq9:~$ opencloud search server
{"level":"info","service":"search","time":"2026-05-06T09:11:13Z","line":"github.com/opencloud-eu/opencloud/services/search/pkg/command/server.go:211","message":"event listening disabled, not starting event service"}
Error: listen tcp 127.0.0.1:9224: bind: address already in use
Usage:
  opencloud search server [flags]

Flags:
  -h, --help   help for server

listen tcp 127.0.0.1:9224: bind: address already in use

Also the actual search service' port 9220 seems to be bound already:

opencloud-api-6f687b6cf-8lbq9:~$ netstat -tulpen |grep 9220
tcp        0      0 127.0.0.1:9220          0.0.0.0:*               LISTEN      1/opencloud
opencloud-api-6f687b6cf-8lbq9:~$ opencloud search index --all-spaces
failed to index space: {"id":"go.micro.client","code":500,"detail":"service eu.opencloud.api.search: not found","status":"Internal Server Error"}
Error: {"id":"go.micro.client","code":500,"detail":"service eu.opencloud.api.search: not found","status":"Internal Server Error"}
Usage:
  opencloud search index [flags]

Aliases:
  index, i

Flags:
      --all-spaces     index all spaces instead. This or --space is required.
      --force-rescan   force a rescan of all files, even if they are already indexed. This will make the indexing process much slower, but ensures that the index is up-to-date using the current search service configuration.
  -h, --help           help for index
  -s, --space string   the id of the space to travers and index the files of. This or --all-spaces is required.

{"id":"go.micro.client","code":500,"detail":"service eu.opencloud.api.search: not found","status":"Internal Server Error"}

Creating a new text file via WebUI, including e.g. "test search string" and finally using the search and search for "test search string" finds the new text file. So the search seems to work correctly.

[schweigisito]

Did you upgrade from 5.x to 6.1 directly or was that a 6.0->6.1 upgrade? Regardless, I understand that there probably should have been a clearer statement on the release notes besides the "Breaking change" link to the issue.

I was coming from latest stable release 4.0.x and used the latest 6.1.0 rolling image.

[schweigisito]

Maybe worth to mention again: The setup has been deployed using opencloud-charts and helmfile. Means: the following services are enabled on the two different pod types:

opencloud-api-6f687b6cf-64hdc:~$ echo $OC_RUN_SERVICES
activitylog,app-registry,auth-app,auth-basic,auth-machine,auth-service,eventhistory,frontend,gateway,graph,groups,ocdav,ocs,proxy,search,settings,sharing,sse,storage-publiclink,storage-shares,storage-system,storage-users,thumbnails,userlog,users,web,webdav,webfinger,collaboration

and

opencloud-backend-5d55dfb6dc-75l4d:~$ echo $OC_RUN_SERVICES
clientlog,postprocessing,settings,search,notifications