UI Updates on the Wellplate Controller by beniroquai · Pull Request #272 · openUC2/ImSwitch

beniroquai · 2026-05-12T07:45:31Z

This pull request introduces several improvements and cleanups across the codebase, most notably adding comprehensive documentation for Opentrons-style labware integration in ImSwitch, enhancing the frontend's Axon tab with a new Overview Scan feature, and refining the overview registration workflow. Additionally, it removes obsolete or development-only files and makes small adjustments for better cross-platform support.

Documentation and Labware Integration:

Added a detailed docs/labware_opentrons.md guide describing how Opentrons-compatible labware is integrated, discovered, and used within ImSwitch, including backend structure, HTTP endpoints, frontend integration, and OME-NGFF metadata output.

Frontend Features and Improvements:

Added the OverviewScanTab component to the Axon tab's UI, updating AxonTabComponent.js to include the new tab and its contents. [1] [2] [3]
Improved the overview registration wizard (OverviewRegistrationWizard.js) so the MJPEG stream URL is built whenever the wizard is open, regardless of camera availability, and enhanced metadata persistence by storing the stage position at snapshot time for later autonomous scanning. [1] [2] [3]

Configuration and Dependency Cleanups:

Removed the .readthedocs.yaml file, indicating a possible shift away from Read the Docs for documentation builds.
Added a Windows-specific Python launch configuration to .vscode/launch.json for improved cross-platform development support.
Updated package-lock.json to add a new yaml dependency under the tailwindcss node_modules tree, likely as a transitive dependency.

Maintenance and Cleanup:

Removed the frontend/scripts/create_test_animations.sh script, which was used for generating placeholder GIFs for acceptance testing, indicating these assets or their creation are no longer needed.
Deleted the development/test server implementation in frontend/python/server.py, further cleaning up unused or obsolete files.

Refactor pixel calibration end-to-end: backend, detector managers, and frontend. Adds a pending-calibration approval flow (getPendingCalibration / applyPendingCalibration / discardPendingCalibration) and switches storage to per-detector calibration entries instead of objective-only keys. Removes the overview/AprilTag calibrators and related stream UI, simplifies PixelCalibrationController initialization so it always loads (even without config), and applies flips/pixel-size to detectors via a new _applyCalibrationToDetector helper. Backend: major controller changes, pending state, config persistence, and removal/archiving of apriltag/overview modules. Detector managers (e.g. HikCam) stop reading flipX/flipY and authoritative pixel-size from setup and default to neutral values so PixelCalibrationController is the single source of truth. Frontend: new API wrappers (get/apply/discard pending), calibrate API updated to use detectorName, PixelCalibrationTab reworked to remove overview stream and add a review/edit/apply UI for pending results. Also adds a large CLAUDE_CODE_INSTRUCTIONS_Pixelcalibration.md with refactor guidance and archives some tests/docs and old requirement files.

Introduce objective-aware pixel calibration throughout frontend and backend. Frontend: expose an Objective selector in pixel calibration tabs, pass objectiveId as a query param to PixelCalibration API calls and respect FastAPI query/body mapping. Backend: migrate PixelCalibrationController to key calibrations by "<detector>::<objective>", support objectiveId on calibrate/apply/get/discard/delete/set operations, and only apply calibrations for the currently selected objective while still recognising legacy detector-only keys. Add getPixelSizeUm API, react to objective changes to re-apply matching calibrations, and store objective metadata with pending/persisted entries. Misc: add overviewCameraName config in setup/ExperimentManager and resolve overview camera in ExperimentController; move pixel size/flip defaults out of camera managers (GXPIPY/Tucsen/Virtual) so PixelCalibrationController is the single source of truth.

Make ashlar stitching pick up pixel size from a parent *_protocol.json (falls back to 1.0 µm if not found) by scanning the parent directory and extracting post_params.pixel_size; pass that value to build_ashlar_stitched. Also switch script to run main() by default instead of the test helper. In PixelCalibrationController, remove the transitive cv2 import and simplify imports from affine_stage_calibration (remove re-export of validate_calibration), plus minor whitespace/usage example cleanups.

Introduce detector-aware pixel calibration endpoints and UI changes. Frontend: add API helpers (getAvailableDetectors, getCalibrationData, setCalibrationData), surface detector dropdowns in PixelCalibrationTab and ManualPixelCalibrationTab, send previewSubsamplingFactor and raw preview coords from manual calibration, and add an editable per-(detector,objective) save UI. Backend: PixelCalibrationController gains getAvailableDetectors, previewSubsamplingFactor handling in manual calibration, richer setCalibrationData response, and JSON sanitization to convert numpy types and replace NaN/Inf with nulls. ObjectiveController now logs a one-time warning when falling back to deprecated static pixelsizes. ArkitektManager gets a device-code hook to handle/display device codes. Also bump NanoImagingPack to 2.1.5 in pyproject.toml and setup.py.

Introduce an Opentrons-compatible labware subsystem and UI integration. Adds a new imswitch/imcontrol/model/labware package (models, loader, manager, selector, generators, schema_v2 and built-in definitions), OME plate metadata writer, and unit tests for labware loading/validation/selection. Frontend: add LabwareSelectionPanel and backend API stubs under frontend/backendapi; update WellSelectorComponent and Redux slices to integrate wellplate selection. Backend: update ExperimentController and related experiment controller modules to expose labware/well-selection functionality. Misc: add Windows launch configuration in .vscode/launch.json, remove PixelCalibration instruction doc and add WellplateOpentrons spec.

Migrate ImSwitch to an Opentrons-style labware layer: add docs (docs/labware_opentrons.md), vendored labware JSON definitions, and update labware generator/models/schema to the new format. Remove the legacy wellplate_layouts module and associated frontend API shims; ExperimentController now forwards to the LabwareManager and exposes compatibility wrappers that return the legacy dict shape. Frontend changes: add confirmation dialogs when switching layouts/labware (to avoid accidental point loss), add per-well sub-position grid generation with an SVG preview, and wire apply/replace behaviour to expand sub-positions. Small controller/fallback tweaks keep backward compatibility while encouraging use of the new endpoints and JSON definitions.

Implements autonomous overview scanning, registration persistence, overlay JPEG storage, a new Overview Scan UI tab, and freehand drawing for region-to-points conversion. Key changes: - Added CLAUDE_CODE_INSTRUCTIONS.md with implementation notes and design spec. - Backend: extended overview registration to persist stagePosition/warp, save/load registration config and overlay JPEGs; added endpoints in ExperimentController for running autonomous scans, getting/updating registration config, and serving overlay images. - Frontend: new OverviewScanTab component and API wrappers (apiGetOverviewRegistrationConfigData, apiUpdateOverviewRegistrationConfig, apiRunAutonomousOverviewScan, apiGetOverviewOverlayImage); moved overlay controls into the new tab and added tabbed UI in WellSelectorComponent. - Freehand drawing: introduced FREEHAND_DRAW mode in WellSelectorCanvas (drawing, throttling, polygon fill), added generateFreehandScanPositions and clearFreehand exposed via ref, and added convert button in WellSelectorComponent to create experiment points from the polygon. - State: updated OverviewRegistrationSlice to hold registrationConfig, autonomous scan state and progress, and new reducers used by the tab. Files added/modified include frontend components (OverviewScanTab, WellSelectorCanvas, WellSelectorComponent, OverviewRegistrationWizard), frontend backendapi wrappers, state slice updates, and backend overview_registration/ExperimentController updates to support persistence and autonomous operation.

Persist overview-camera registration configs into the setup file and load them on startup so registrations survive process restarts. ExperimentController: import config tools, load persisted registrations at init, persist registrations after register/save operations, and add helpers for setup keying. Add overviewRegistration field to SetupInfo. Frontend: expose Overview Scan tab in the main Axon UI, remove the separate tab from WellSelector, and adjust OverviewRegistrationWizard to start the MJPEG stream as soon as the wizard opens. Add areaId/groupId/wellId metadata to points (freehand and well sub-position flows) so related tiles can be grouped by downstream writers, and surface these groups in a new PerWellPointsOverview component. Also persist overview overlay preferences to localStorage and rename/refactor a labware layout helper for clarity.

…/openUC2/ImSwitch into feature/OpentronsWellplatelayout

Add editable objective metadata and related state/handlers, plus various UI and status improvements. Key changes: - Add new API: apiObjectiveControllerSetObjectiveParameters to set objective metadata on the backend. - Import and wire the new API in ObjectiveController; add per-slot editable metadata fields, a save handler (handleSaveObjectiveMeta) and UI for name/NA/magnification/pixelsize. - Populate new objective metadata in Redux via fetchObjectiveControllerGetStatus and add corresponding actions/state in ObjectiveSlice (availableObjectivesNames, availableObjectiveMagnifications, availableObjectiveNAs, availableObjectivePixelSizes). - Small behavior tweaks: make skipZ default to false in apiObjectiveControllerMoveToObjective; remove an early return in the move handler so flow continues after the positions check (previous return commented out). - UI/label improvements: rename X0/X1/Z0/Z1 labels to more descriptive Position/Focus 1/2 and adjust related button labels/placeholders. - Fix FRAMESettingsController tab indices (shift TestHoming/ObjectiveController tabs) and add polling in WizardStep4 so Z updates while user adjusts focus. These changes enable editing/saving objective parameters from the UI, improve clarity of objective/position labels, and ensure objective metadata is synchronized into application state.

Frontend: Refactor detector parameter fetching to a useCallback and add polling while auto-exposure is active; show a warning Alert when mode is auto. In the StreamControlOverlay add detector discovery, a detector selector (when multiple detectors present), include the selected detector_name in stream parameter payloads, and update labels/tooltips to use RAW (16-bit) wording. Backend: Add per-detector stream parameter storage (_perDetectorStreamParams). setStreamParams now accepts an optional detectorName and stores per-detector overrides. getStreamParams accepts detectorName (defaults to the first available detector) and merges per-detector overrides into the response. These changes enable selecting and persisting stream settings per detector.

Fetch detector exposure/gain on mount and add a button to apply current camera settings (frontend/src/axon/ExperimentComponent.js). Add z_speed to the experiment model and defaults (models.py, ExperimentSlice.js), expose setZSpeed in the Redux slice, and wire a Z Speed selector into the tiling UI (TilingDimension.js). Improve FocusMap controls to disable/annotate compute buttons in manual-map mode (FocusMapDimension.js). Simplify stream overlay UI by renaming RAW -> Binary, removing detector-name fetching/selector and excluding detector_name from stream requests, and adjust related labels (StreamControlOverlay.js). Server-side ExperimentController now respects z_speed when setting Z motion speed and tightens laser-enable logic to handle falsy values safely (ExperimentController.py). These changes align UI defaults with hardware settings and add explicit Z speed handling across model, controller, and UI.

Replace legacy PixelCalibrationController overview stream usage with LiveViewController endpoints. Components (OverviewRegistrationWizard, SetLasersTab, TestHomingTab) now build MJPEG URLs using /LiveViewController/mjpeg_stream?detectorName=ObservationCamera and call apiLiveViewControllerStartLiveView / apiLiveViewControllerStopLiveView to start/stop streams (subsampling_factor: 1). apiPixelCalibrationControllerOverviewStream was updated to start/stop the ObservationCamera via LiveViewController. Also include stream lifecycle handling in the wizard (start on open, stop on close) and small UI changes in FocusMapDimension (removed disabled on manual measure button and updated button labels).

Switch FRAMESettings tabs to reuse the central WebSocket live stream instead of creating per-component MJPEG endpoints. Added LiveStreamSlice and LiveViewSlice selectors and use apiLiveViewControllerStartLiveView / apiLiveViewControllerStopLiveView to switch the active detector to ObservationCamera (jpeg, 1x subsampling), and restore the previously active detector/protocol when stopping. Replaced component-local MJPEG URLs and img refs with the shared base64 image from liveStreamState.liveViewImage and updated UI to show "Waiting for image..." while active but no image yet. Changes applied to ObjectiveControllerTab, SetLasersTab and TestHomingTab, and include small state/prop cleanup (prevDetectorRef / prevProtocolRef) and error/status messaging updates.

Expose configurable Move Camera speeds and wire them through the UI and state. Added moveCameraSpeedXY and moveCameraSpeedZ to the well selector slice with defaults (20000 µm/s and 1000 µm/s) and validation (parseFloat fallback to defaults). Created actions and handlers to update these values from the component. The WellSelectorComponent now shows XY/Z speed inputs when mode === MOVE_CAMERA and dispatches changes. WellSelectorCanvas uses moveCameraSpeedXY when issuing X/Y positioner move calls. This enables users to adjust camera movement speeds from the UI and persists values in the slice.

beniroquai · 2026-05-12T07:46:02Z

@gokugiant the remaining issues based on yesterdays' and last Friday's session are the following:

WP-3 · Referential Z-Offset / Multi-Objective Focus

Scope: Backend-heavy, minor frontend
Complexity: Medium · ~2–3 hours
Branch suggestion: feature/z-offset-auto-apply

Analysis

The backend already implements Z-offset logic. In ObjectiveController.py, moveToObjectiveThread() (line ~218) calculates z_delta = target_z - current_z and applies a relative Z move. The values come from self._zPositions[slot] which are loaded from config and can be saved via saveCurrentZPosition().

What's broken: The saveCurrentZPosition endpoint (line ~295) works correctly — it reads the live Z from the positioner and saves it. But the frontend wizard (WizardStep4.js) never calls it because the Z display doesn't update (fixed in WP-2.5).

Tasks

3.1 Verify the Z-offset flow end-to-end: After WP-2.5 fixes the Z display, test: calibrate Z for slot 0, switch to slot 1, focus manually, save Z1, switch back to slot 0 — stage should auto-move Z.

3.2 Add per-objective exposure/gain to the config. Currently ObjectiveManager.py stores pixelsizes, NAs, magnifications, objectiveNames per slot but NOT exposure/gain. Add:

Backend: add exposures and gains lists to ObjectiveManager.__init__ (alongside the existing lists), load from ObjectiveInfo, add to setObjectiveParameters/getObjectiveParameters/_saveObjectiveConfigToFile
Frontend: add exposure/gain fields in the ObjectiveController component per slot

3.3 Add +/− Z fine-focus buttons to the frontend ObjectiveController. These should call the existing apiPositionerControllerMovePositioner with axis: "Z" and small step values (e.g. ±10, ±100 µm).

WP-8 · Hardware Connection Status & Menu Organization

Scope: Frontend + minor Backend
Complexity: Small–Medium · ~2–3 hours
Branch suggestion: feature/connection-status-improvements

Task 8.1 — ESP32 connection heartbeat

Backend: The server already has /getAvailableControllers (in ImSwitchServer.py line ~117). Add a /ping or /health endpoint that actually tests ESP32 responsiveness:

@api_router.get("/ping")
def ping():
    """Quick health check including hardware connectivity."""
    try:
        stage = master.positionersManager["ESP32Stage"]
        pos = stage.getPosition()  # actual hardware call
        return {"status": "ok", "hardware": "connected"}
    except:
        return {"status": "ok", "hardware": "disconnected"}

Frontend: In the connection status indicator (likely in frontend/src/components/navigation/TopBar.jsx or ConnectionSettings.jsx), add periodic polling:

useEffect(() => {
  const interval = setInterval(async () => {
    try {
      const resp = await fetch(`${hostIP}:${hostPort}/imswitch/api/ping`);
      const data = await resp.json();
      setHardwareConnected(data.hardware === "connected");
    } catch { setHardwareConnected(false); }
  }, 5000);
  return () => clearInterval(interval);
}, []);

Task 8.2 — Hardware menu consolidation

File: frontend/src/components/navigation/NavigationDrawer.jsx

The sidebar uses an app registry pattern (APP_REGISTRY from constants/appRegistry.js). Hardware-related widgets should be grouped under a "Hardware" or "System" category. Check appRegistry.js and add a new category or reassign scattered hardware entries.

I worked on the others in our document

Copilot

Pull request overview

This PR modernizes ImSwitch’s wellplate/overview workflow by introducing an Opentrons-compatible labware layer (with validation + selection patterns), extending acquisition metadata output for plate-aware OME-NGFF sidecars, and updating both backend pixel-calibration handling and frontend Axon/WellSelector UX to support these workflows.

Changes:

Added a new imswitch.imcontrol.model.labware module (schema validation, loaders, selection patterns, definitions) and accompanying unit tests.
Extended experiment/OME-Zarr writing to emit OME-NGFF plate/well metadata (plate_metadata.json sidecar + optional per-well attrs).
Updated multiple controllers/managers and frontend components to centralize pixel calibration (PixelCalibration as mandatory), add labware selection state, and improve overview/stream controls.

Reviewed changes

Copilot reviewed 109 out of 144 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
uv.lock	Adds `jsonschema`, `opencv-contrib-python`, updates NanoImagingPack lock entry.
setup.py	Bumps NanoImagingPack pin to `2.1.5`.
scripts/convert_experiment_tiffs.py	Ashlar mode now tries to infer pixel size from protocol JSON; switches default entrypoint to `main()`.
requirements.txt	Removed legacy requirements file.
requirements-dev.txt	Removed legacy dev requirements file.
requirements-arm64.txt	Removed legacy ARM64 requirements file.
pyproject.toml	Adds `jsonschema`, adds `opencv-contrib-python`, bumps NanoImagingPack pin to `2.1.5`.
imswitch/imcontrol/view/ImConMainView.py	Forces PixelCalibration widget to be present in enabled dock keys.
imswitch/imcontrol/model/SetupInfo.py	Updates PixelCalibrationInfo doc/shape, adds `overviewCameraName` and `overviewRegistration`, refines affine calibration keying.
imswitch/imcontrol/model/managers/ExperimentManager.py	Adds `overviewCameraName` config plumbing.
imswitch/imcontrol/model/managers/detectors/VirtualCameraManager.py	Removes per-manager flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/detectors/TucsenCamManager.py	Removes per-manager pixel size/flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/detectors/HikCamManager.py	Removes per-manager pixel size/flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/detectors/GXPIPYManager.py	Removes per-manager pixel size/flip config; uses neutral defaults pending PixelCalibration injection.
imswitch/imcontrol/model/managers/ArkitektManager.py	Adds device-code hook for auth flow.
imswitch/imcontrol/model/labware/selector.py	New: well selection pattern resolution utilities.
imswitch/imcontrol/model/labware/schema_v2.json	New: vendored (reduced) Opentrons labware schema v2 subset.
imswitch/imcontrol/model/labware/models.py	New: pydantic models for Opentrons v2 + ImSwitch in-memory labware.
imswitch/imcontrol/model/labware/manager.py	New: discovery/caching/offset API for labware definitions.
imswitch/imcontrol/model/labware/loader.py	New: validates/parses/converts Opentrons JSON mm→µm.
imswitch/imcontrol/model/labware/definitions/openuc2/slide_4x_histosample_heidstar/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/ropod_2slides_uc2/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/ibidi_8well_chambered_coverslip/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/corning_6_wellplate_16.8ml_flat/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/definitions/openuc2/corning_12_wellplate_6.9ml_flat/1.json	New: labware definition JSON.
imswitch/imcontrol/model/labware/init.py	New: labware public API exports.
imswitch/imcontrol/model/io/ome_writers/plate_metadata.py	New: OME-NGFF plate metadata sidecar writer.
imswitch/imcontrol/model/io/ome_writers/ome_writer.py	Adds optional `well_metadata` to emit OME-NGFF well attrs + ImSwitch extension block.
imswitch/imcontrol/model/io/ome_writers/init.py	Exports plate metadata helpers.
imswitch/imcontrol/model/interfaces/thorlabs_tsi_sdk/dll/thorlabs_tsi_logger.cfg	Removed SDK logger config file.
imswitch/imcontrol/model/interfaces/thorlabs_tsi_sdk/dll/Copy 64-bit native libraries here.txt	Removed placeholder text file.
imswitch/imcontrol/controller/MasterController.py	Always instantiates PixelCalibrationManager.
imswitch/imcontrol/controller/ImConMainController.py	Ensures PixelCalibrationController is created last and registered.
imswitch/imcontrol/controller/controllers/wellplate_layouts.py	Removed legacy hard-coded layout definitions.
imswitch/imcontrol/controller/controllers/SettingsController.py	Adds per-detector stream params storage and optional `detectorName` parameter.
imswitch/imcontrol/controller/controllers/pixelcalibration/_archive/init.py	Added archive package marker.
imswitch/imcontrol/controller/controllers/ObjectiveController.py	Pixel size now prefers PixelCalibrationController with legacy fallback.
imswitch/imcontrol/controller/controllers/experiment_controller/models.py	Adds labware/well metadata fields + `z_speed` to parameters.
imswitch/imcontrol/controller/controllers/experiment_controller/experiment_performance_mode.py	Writes best-effort plate metadata sidecar (performance mode).
imswitch/imcontrol/controller/controllers/experiment_controller/experiment_normal_mode.py	Adds per-well metadata into OMEWriter + writes plate metadata sidecar.
imswitch/imcontrol/_test/unit/test_labware/test_selector_patterns.py	New: selector tests.
imswitch/imcontrol/_test/unit/test_labware/test_plate_metadata.py	New: plate sidecar tests.
imswitch/imcontrol/_test/unit/test_labware/test_loader_validation.py	New: loader validation tests.
imswitch/imcontrol/_test/unit/test_labware/test_loader_unit_conversion.py	New: mm→µm conversion tests.
imswitch/imcontrol/_test/unit/test_labware/test_loader_roundtrip.py	New: shipped definitions load/roundtrip tests.
imswitch/imcontrol/_test/unit/test_labware/test_generators.py	New: generator geometry sanity tests.
imswitch/imcontrol/_test/unit/test_labware/init.py	New: test package marker.
imswitch/imcontrol/_test/unit/pixelcalibration/_archive/test_overview_calibrator.py	Adds archived pixelcalibration tests.
frontend/src/state/slices/WellSelectorSlice.js	Adds labware selection state + MOVE_CAMERA speed controls.
frontend/src/state/slices/OverviewRegistrationSlice.js	Persists overlay prefs, adds autonomous scan + editable config state.
frontend/src/state/slices/ObjectiveSlice.js	Adds arrays for per-slot objective metadata.
frontend/src/state/slices/ExperimentSlice.js	Adds `z_speed`, point grouping metadata, and batch point append/replace helpers.
frontend/src/middleware/fetchObjectiveControllerGetStatus.js	Populates new objective metadata arrays.
frontend/src/components/wizard-steps/WizardStep6.js	Tweaks wizard styling.
frontend/src/components/wizard-steps/WizardStep5.js	Tweaks wizard styling.
frontend/src/components/wizard-steps/WizardStep4.js	Adds 1s polling for Z updates.
frontend/src/components/StreamControlOverlay.js	Refreshes objective status after stream setting changes.
frontend/src/components/ObjectiveController.js	Adds editable objective metadata UI; alters objective-switch guard behavior.
frontend/src/components/FRAMESettingsController.js	Removes Track Motion tab.
frontend/src/components/FRAMESettings/TestHomingTab.js	Reuses LiveView stream instead of PixelCalibrationController MJPEG stream.
frontend/src/components/FRAMESettings/SetLasersTab.js	Reuses LiveView stream instead of PixelCalibrationController MJPEG stream.
frontend/src/components/FRAMESettings/ObjectiveControllerTab.js	Reuses LiveView stream instead of PixelCalibrationController MJPEG stream.
frontend/src/components/FRAMESettings/ManualPixelCalibrationTab.js	Adds detector selection + calibration read/write/edit; fixes subsampling scaling path.
frontend/src/components/DetectorParameters.js	Adds polling in auto-exposure mode; improves numeric parsing; adds warning alert.
frontend/src/backendapi/apiUpdateOverviewRegistrationConfig.js	New: persist overview registration config edits.
frontend/src/backendapi/apiRunAutonomousOverviewScan.js	New: triggers autonomous overview scan endpoint.
frontend/src/backendapi/apiPixelCalibrationControllerSetCalibrationData.js	New: direct calibration write endpoint helper.
frontend/src/backendapi/apiPixelCalibrationControllerOverviewStream.js	Repoints overview stream control to LiveViewController.
frontend/src/backendapi/apiPixelCalibrationControllerManualPixelSizeCalibration.js	Adds detectorName + previewSubsamplingFactor parameters.
frontend/src/backendapi/apiPixelCalibrationControllerGetPendingCalibration.js	New: pending calibration polling helper.
frontend/src/backendapi/apiPixelCalibrationControllerGetCalibrationData.js	New: persisted calibration read helper.
frontend/src/backendapi/apiPixelCalibrationControllerGetAvailableDetectors.js	New: detector list helper for pixel calibration.
frontend/src/backendapi/apiPixelCalibrationControllerDiscardPendingCalibration.js	New: discard pending calibration helper.
frontend/src/backendapi/apiPixelCalibrationControllerCalibrateStageAffine.js	Adds detectorName/objectiveId query param handling.
frontend/src/backendapi/apiPixelCalibrationControllerApplyPendingCalibration.js	New: apply pending calibration helper.
frontend/src/backendapi/apiObjectiveControllerSetObjectiveParameters.js	New: set objective metadata endpoint helper.
frontend/src/backendapi/apiObjectiveControllerMoveToObjective.js	Default `skipZ=false`.
frontend/src/backendapi/apiGetOverviewRegistrationConfigData.js	New: loads persisted overview-registration config.
frontend/src/backendapi/apiGetOverviewOverlayImage.js	New: loads a stored slot overlay image.
frontend/src/backendapi/apiExperimentControllerSelectWellsByPattern.js	New: resolves selection patterns without mutating experiment state.
frontend/src/backendapi/apiExperimentControllerGetWellplateLayouts.js	Removed legacy layouts list helper.
frontend/src/backendapi/apiExperimentControllerGetWellplateLayout.js	Removed legacy single-layout helper.
frontend/src/backendapi/apiExperimentControllerGetLabwareList.js	New: fetch labware summaries.
frontend/src/backendapi/apiExperimentControllerGetLabwareDefinition.js	New: fetch full labware definition.
frontend/src/backendapi/apiExperimentControllerGenerateCustomWellplate.js	Removed legacy custom layout generator helper.
frontend/src/backendapi/apiExperimentControllerApplyWellSelectionToExperiment.js	New: returns point dicts for experiment pointList population.
frontend/src/axon/WellSelectorUtils.js	Adds point-in-polygon helper.
frontend/src/axon/WellSelectorComponent.js	Adds labware selection panel, freehand convert UI, and layout-switch confirmation.
frontend/src/axon/WellSelectorCanvas.js	Adds FREEHAND mode + freehand rendering/scan-position generation; MOVE_CAMERA XY speed usage.
frontend/src/axon/OverviewRegistrationWizard.js	Starts/stops MJPEG via LiveViewController and persists snapshot stage position.
frontend/src/axon/LiveViewComponent.js	Improves adaptive pixel-size computation for subsampled streams.
frontend/src/axon/ExperimentComponent.js	Adds “Use Current Camera Settings” and auto-loads exposure/gain defaults.
frontend/src/axon/experiment-designer/TilingDimension.js	Syncs overlap to areaSelectOverlap; adds Z speed control.
frontend/src/axon/experiment-designer/FocusMapDimension.js	Improves button enable/labels/tooltips.
frontend/src/axon/AxonTabComponent.js	Adds Overview Scan tab.
frontend/scripts/create_test_animations.sh	Removed acceptance-test GIF generator script.
frontend/python/server.py	Removed dev/test server implementation.
frontend/package-lock.json	Adds transitive `yaml` under tailwindcss tree.
docs/labware_opentrons.md	New: detailed Opentrons labware integration documentation.
.vscode/launch.json	Adds Windows Python launch config.
.readthedocs.yaml	Removes Read the Docs build config.

Files not reviewed (1)

frontend/package-lock.json: Language not supported

Comments suppressed due to low confidence (1)

frontend/src/axon/WellSelectorComponent.js:345

style={{ backgroundColor: "primary.main" }} uses a theme token string in a plain inline style, so the browser receives an invalid CSS color and the highlight won’t apply. Use MUI sx={{ bgcolor: 'primary.main' }} (or style={{ backgroundColor: theme.palette.primary.main }} via useTheme).

          >
            {/* current layout */}
            <MenuItem
              style={{ backgroundColor: "primary.main" }}
              value={experimentState.wellLayout.name}
            >
              {experimentState.wellLayout.name}
            </MenuItem>

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+            with open(json_file, "r") as f: protocolDict = json.load(f)
+            # search for "pixel_size" in protocolDict and use it if found, otherwise default to 1.0
+            pixel_size = next(step["post_params"]["pixel_size"] for step in protocolDict["workflow_steps"] if "pixel_size" in step.get("post_params", {}))
+            print(f"Using pixel size from protocol: {pixel_size} microns")
+            if pixel_size is None:
+                pixel_size = 1.0


+    rows: List[str] = []
+    cols: List[int] = []
+    seen_rows: set[str] = set()
+    seen_cols: set[int] = set()
+    for wid in ordering_well_ids:
+        r, c = _parse_well_id(wid)
+        if r not in seen_rows:
+            seen_rows.add(r)
+            rows.append(r)
+        if c not in seen_cols:
+            seen_cols.add(c)
+            cols.append(c)
+    rows.sort()
+    cols.sort()
+


  const handleSwitchObjective = (slot, skipZ) => {
    // Warn user if the target positions have not been configured yet
    const x0 = objectiveState.posX0;
    const x1 = objectiveState.posX1;
    const positionsConfigured =
      x0 !== null && x0 !== undefined && x0 !== 0 &&
      x1 !== null && x1 !== undefined && x1 !== 0;
    if (!positionsConfigured) {
      alert(
        "Objective positions (X0 / X1) are not configured yet.\n" +
        "Please use the Calibration Wizard to set them before switching."
      );
-      return;
+      // return;
    }
    apiObjectiveControllerMoveToObjective(slot, skipZ)


+    async def _store_device_code_hook(self, device_code: str) -> None:
+        """Store the device code for user authentication."""
+        self.__logger.info(f"Received Arkitekt device code: {device_code}")
+        # Here you could implement logic to display the device code to the user
+        # or store it in a file for later retrieval. For now, we just log it.


+    generateFreehandScanPositions: (overlap = 0) => {
+      const polygon = freehandPoints;
+      if (!polygon || polygon.length < 3) return [];
+      const fovX = objectiveState?.fovX || 0;
+      const fovY = objectiveState?.fovY || 0;
+      if (fovX <= 0 || fovY <= 0) return [];
+      const stepX = fovX * (1 - overlap);
+      const stepY = fovY * (1 - overlap);
+      let minX = Infinity,
+        minY = Infinity,
+        maxX = -Infinity,
+        maxY = -Infinity;
+      polygon.forEach((p) => {
+        if (p.x < minX) minX = p.x;
+        if (p.y < minY) minY = p.y;
+        if (p.x > maxX) maxX = p.x;
+        if (p.y > maxY) maxY = p.y;
+      });
+      const positions = [];
+      for (let y = minY; y <= maxY; y += stepY) {
+        for (let x = minX; x <= maxX; x += stepX) {
+          if (wsUtils.isPointInPolygon({ x, y }, polygon)) {
+            positions.push({ x, y });
+          }
+        }
+      }
+      return positions;


+      const last = freehandPoints[freehandPoints.length - 1];
+      if (!last) {
+        setFreehandPoints([phy]);
+      } else {
+        const dx = phy.x - last.x;
+        const dy = phy.y - last.y;
+        if (Math.hypot(dx, dy) >= FREEHAND_MIN_STEP_UM) {
+          setFreehandPoints([...freehandPoints, phy]);
+        }
+      }


        # Try to use LiveViewController if available through CommunicationChannel
        print("RReceived parameters: ", compression, subsampling, throttle_ms)
        try:


+        # Try PixelCalibrationController first.
+        try:
+            pix_ctrl = self._master._controllersRegistry.get("PixelCalibration", None)
+            if pix_ctrl is not None and hasattr(pix_ctrl, "getPixelSize"):
+                detector_name = self._master.detectorsManager.getAllDeviceNames()[0]
+                sx, sy = pix_ctrl.getPixelSize(detector_name, str(self._currentObjective))
+                avg = (abs(sx) + abs(sy)) / 2.0
+                if avg > 0 and avg != 1.0:
+                    return avg


Move OverviewRegistrationWizard import and mounting from WellSelectorComponent to AxonTabComponent so the wizard is always available regardless of which tab is active. Remove the redundant import/instance from WellSelectorComponent. Adjust TestHomingTab layout/styles: switch from minHeight to a fixed 500px height, add overflow:hidden, and constrain the live preview image with maxHeight, objectFit: 'contain', and display:block to keep the image scaled and centered without overflow.

Introduce a raster-based stage-center calibration workflow and per-slot recapture support. Frontend: - Add StageOffsetCalibrationTab (ui + logic) to FRAME Settings: runs an XY raster scan, polls heatmap, renders heatmap canvas, shows brightest spot and allows manual override + storing offsets via PositionerController. - Add API clients: apiStageCenterCalibrationGetHeatmap, apiStageCenterCalibrationGetIsRunning, apiExperimentControllerRecaptureSlot. - Wire a "Retake" button into OverviewRegistrationWizard to call ExperimentController.recaptureSlot and refresh overlay. - Register the new Stage Offset Calibration tab in FRAMESettingsController. Backend: - Add ExperimentController.recaptureSlot API: computes slot center (from layout_data / labware / Heidstar fallback), moves stage to center, and reuses snapOverviewImage to replace a single slot snapshot. - Rework StageCenterCalibrationController: replace legacy spiral worker with a deterministic raster scan that collects (x,y,mean_intensity) samples, exposes performCalibration/getCalibrationHeatmap/getIsCalibrationRunning/stopCalibration, returns structured results including brightest spot and metadata, emits progress, and saves CSVs. Minor MovementController cleanup. Purpose: enable accurate stage offset calibration via a heatmap-driven raster scan and let users selectively retake individual overview slots without re-running the full registration.

Add a redesigned two-step Stage Offset Calibration flow and supporting APIs. Frontend: - New backend API wrappers: getKnownCalibrationLayouts, getKnownCalibrationPoint, getOverviewAsyncStatus, getRecommendedScanParameters. - Major rewrite of StageOffsetCalibrationTab: introduces LayoutMapMini and HeatmapCanvas, adaptive recommended scan defaults, polling-driven run state, click-to-move stage, merge of backend-known layouts with fallbacks, removal of exposure input (managed by detector), improved UX for known vs measured points, and persisting offsets via setStageOffsetAxis using known+current positions. - Integrates position state and positioner move API calls. Backend: - StageCenterCalibrationController: added getRecommendedScanParameters to compute pixel-size/FOV-based defaults; updated raster worker to use explicit keyword args for stage.move; minor cleanup. - VirtualCameraManager: change initial _running flag to False. These changes provide adaptive scan defaults, a visual layout map and heatmap for easier calibration, and the API endpoints required by the new UI.

beniroquai added 20 commits May 1, 2026 18:55

update versions of libraries (cv2)

efef36c

Merge branch 'feature/OpentronsWellplatelayout' of https://github.com…

4b0290f

…/openUC2/ImSwitch into feature/OpentronsWellplatelayout

remove binaries

4490dc2

fixing overlap converting to 0 frontend

0070b35

fix colours

b5712bc

fix pixelsize in liveview

0357d9a

Copilot AI review requested due to automatic review settings May 12, 2026 07:45

Copilot started reviewing on behalf of beniroquai May 12, 2026 07:46 View session

Copilot AI reviewed May 12, 2026

View reviewed changes

beniroquai added 3 commits May 12, 2026 13:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UI Updates on the Wellplate Controller #272

UI Updates on the Wellplate Controller #272
beniroquai wants to merge 23 commits into
masterfrom
feature/OpentronsWellplatelayout

beniroquai commented May 12, 2026

Uh oh!

beniroquai commented May 12, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

beniroquai commented May 12, 2026

Uh oh!

beniroquai commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

WP-3 · Referential Z-Offset / Multi-Objective Focus

Analysis

Tasks

WP-8 · Hardware Connection Status & Menu Organization

Task 8.1 — ESP32 connection heartbeat

Task 8.2 — Hardware menu consolidation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

beniroquai commented May 12, 2026 •

edited

Loading