feat(android): Camera Intrinsics #3806
Conversation
|
@pvedula7 is attempting to deploy a commit to the Margelo Team on Vercel. A member of the Team first needs to authorize it. |
|
Thanks this is great stuff!! I'll look into this soon, for one I don't like the two paths in |
Makes sense, agreed. Originally tried avoiding threadinging CameraInfo through every output because some modules like barcode scanner do not need it. Moved it into NativeCameraOutput.Config with a nullable default, so the createUseCase(...) method signature stays unchanged. Moved Camera2-specific logic into small extensions. |
What
Adds Android support for
Frame.cameraIntrinsicMatrixon Frame Outputs.Android now honors the existing
enableCameraMatrixDeliveryoption. When enabled, eachHybridFrameattempts to resolve camera intrinsics from Camera2 metadata and exposes them through the existingFrame.cameraIntrinsicMatrixproperty. No new JS API is added.Implementation
On Android, intrinsics are resolved in this order:
CaptureResult.LENS_INTRINSIC_CALIBRATIONCameraCharacteristics.LENS_INTRINSIC_CALIBRATIONCamera2 exposes intrinsics as five calibration values:
[fx, fy, cx, cy, s]VisionCamera converts those into the existing 3x3 matrix shape:
serialized consistently with the existing native matrix array convention.
The Android implementation also applies CameraX’s
ImageInfo.sensorToBufferTransformMatrixbefore exposing the matrix, so the returned values are in the Frame/buffer coordinate space expected by Frame.cameraIntrinsicMatrix.This PR intentionally does not use CaptureResult.STATISTICS_LENS_INTRINSICS_SAMPLES. That key is API 35+, optional, and represents multiple intra-frame lens intrinsics samples rather than a single frame-level matrix. The existing JS API exposes a single number[] intrinsic matrix on Frame, so those samples would require a different API shape to represent correctly.
Validation
Android Kotlin compile:
./gradlew :react-native-vision-camera:compileDebugKotlin :react-native-vision-camera-barcode-scanner:compileDebugKotlinTypecheck:
npm run typecheck --workspace react-native-vision-cameraManual Android validation on Pixel 6a:
enableCameraMatrixDelivery: true returns a 9-number matrix.
Returned cx/cy are transformed into frame coordinates.
Unsupported or initially unavailable metadata keeps streaming and returns undefined.