Rig support is broken: each shot in a riginstance overwrites the previous shot

[dkogan]

Hello. I'm trying to run a solve with a fixed-geometry rig; this doesn't work, and I'm wondering if it has ever worked.

I'm on Debian GNU/Linux. I'm buliding the bleeding-edge git tree from source: 6322e7c

I run reconstruct --algorithm triangulation. Without rig definitions in the data directory (rig_cameras.json, rig_assignments.json) this runs ok. With those definitions present I see this:

2026-03-31 18:44:58,678 INFO: Starting triangulation reconstruction
2026-03-31 18:45:05,584 INFO: Triangulation SfM. Outer iteration 0, triangulated 17628 points.
2026-03-31 18:45:05,584 INFO: Triangulation SfM. Inner iteration 0, running bundle ...
2026-03-31 18:45:05,585 INFO: Shots and/or GCPs are well-conditioned. Using naive 3D-3D alignment.
Traceback (most recent call last):
  File ".../OpenSfM/bin/opensfm_main.py", line 34, in <module>
    main()  # pragma: no cover
    ~~~~^^
  File ".../OpenSfM/bin/opensfm_main.py", line 26, in main
    commands.command_runner(
    ~~~~~~~~~~~~~~~~~~~~~~~^
        commands.opensfm_commands,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
        create_default_dataset_context,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        dataset_choices=["opensfm"],
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File ".../OpenSfM/opensfm/commands/command_runner.py", line 46, in command_runner
    command.run(data, args)
    ~~~~~~~~~~~^^^^^^^^^^^^
  File ".../OpenSfM/opensfm/commands/command.py", line 16, in run
    self.run_impl(data, args)
    ~~~~~~~~~~~~~^^^^^^^^^^^^
  File ".../OpenSfM/opensfm/commands/reconstruct.py", line 16, in run_impl
    reconstruct.run_dataset(dataset, args.algorithm)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../OpenSfM/opensfm/actions/reconstruct.py", line 18, in run_dataset
    report, reconstructions = reconstruction.triangulation_reconstruction(
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        data, tracks_manager
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File ".../OpenSfM/opensfm/reconstruction.py", line 1596, in triangulation_reconstruction
    b1rep = bundle(
        reconstruction, camera_priors, rig_camera_priors, None, config_override
    )
  File ".../OpenSfM/opensfm/reconstruction.py", line 77, in bundle
    report = pysfm.BAHelpers.bundle(
        reconstruction.map,
    ...<3 lines>...
        config,
    )
IndexError: map::at

Would be great to make this error message more friendly. In any case this happens because the

OpenSfM/opensfm/src/sfm/src/ba_helpers.cc

Line 712 in 6cbe48a

ba.AddPointProjectionObservation(shot.id_, lm_obs.first->id_, obs.point,

call tries to find a shot in the list:

OpenSfM/opensfm/src/bundle/src/bundle_adjuster.cc

Line 244 in 6cbe48a

o.shot = &shots_.at(shot);

and this shot isn't there.

It isn't there because when we loop through all our images, and add them to the relevant rig:

OpenSfM/opensfm/reconstruction_helpers.py

Line 104 in 6cbe48a

reconstruction.add_rig_instance(pymap.RigInstance(rig_instance_id))

we actually overwrite the existing images in the rig instance with each image that comes in:

OpenSfM/opensfm/types.py

Line 177 in 6cbe48a

return self.map.update_rig_instance(rig_instance)

. Note that we call create_rig_instance() and then unconditionally call update_rig_instance() to overwrite the rig instance we currently have with the brand-new empty one.

If I remove this update_rig_instance() then I get complete rig instances. There's another error here:

OpenSfM/opensfm/reconstruction_helpers.py

Line 119 in 6cbe48a

rotation = rotation_from_shot_metadata(shot)

because the shot.pose is immutable or something. If I make those errors soft, the reconstruction run completes. The rig constraint isn't obviously enforced (the reconstruction geometry doesn't do what I asked), but I'm no longer convinced that this has ever worked, so I don't want to continue to debug before hearing from yall first.

I don't want to send you my reproducing data (it is big!) but I suspect that defining any kind of rig with the included sample data would tickle this bug just fine. Let me know if that's useful.

Thanks!

[fabianschenk]

Hey @dkogan ,

As you might have guessed from the very low activity in this repo, OpenSfM is not being actively developed anymore.
If you have an idea for a fix, please create a PR and I'll try to incorporate it.
For a more active SfM pipeline, you can take a look at Open Drone Map or Colmap.

Best regards, Fabian

[dkogan]

Hi. Thanks for the note. I did wonder if this repo was abandoned or not, but the version control activity makes this ambiguous. I was planning on doing the solve in opensfm, and then handing it off to opendronemap to finish the processing, BUT the opendronemap fork of opensfm happened before the support for camera rigs, and I want that functionality. Since this repo is abandoned and the support for rigs isn't functional anyway, would you consider adding a prominent notice to the front page, saying that? It would save people time. Thanks.

[dkogan]

For the record, here's a reproducer:

1. Copy the attached rig definition files to data/lund in the opensfm tree. These define a made-up rig. The geometry is irrelevant here; I'm just demonstrating that reconstruction fails to even ingest this data
2. Try to run the reconstruction. I use this zsh script to run the stages in order:

stages=(
  extract_metadata
  detect_features
  match_features
  create_tracks
  "reconstruct --algorithm triangulation"
)
for stage ($stages) (
  cd ~/OpenSfM/;
  bin/opensfm ${=stage} ~/OpenSfM/data/lund
)

I see IndexError: map::at when I do this.

I just tried again with the default reconstruct, without --algorithm triangulation. This gets past the error, but this is fictitious: the rig definition is still corrupted when it is read in, and it presumably doesn't actually work.

rig_assignments.json
rig_cameras.json