feat: capture platform views in session replay screenshots#393
feat: capture platform views in session replay screenshots#393marcorizza wants to merge 2 commits into
Conversation
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
posthog_flutter/android/src/main/kotlin/com/posthog/flutter/PosthogFlutterPlugin.kt:572-606
**Soft-canvas fallback silently misses GPU-composited platform views**
`View.draw()` renders using a software `Canvas`, which cannot read content from hardware-accelerated surfaces like `SurfaceView`, `TextureView`, or Metal/Vulkan-backed views. This is the exact content type `capturePlatformViews` is meant to capture. On pre-API 26 devices, or whenever `PixelCopy` fails, this fallback will produce blank areas for the native platform views the user opted into capturing. A brief inline comment explaining this limitation would prevent confusion when developers debug unexpected blank regions on older API levels.
### Issue 2 of 2
posthog_flutter/android/src/main/kotlin/com/posthog/flutter/PosthogFlutterPlugin.kt:608-613
**PNG compression runs on the main UI thread**
`bitmapToPng` is called directly inside the `PixelCopy` completion callback, which is dispatched to `Handler(Looper.getMainLooper())`. PNG encoding of a full-screen `ARGB_8888` bitmap (`Bitmap.CompressFormat.PNG, 100`) can take 100–300 ms on a typical device, blocking the main thread for the duration of every session-replay snapshot when `capturePlatformViews` is enabled. The same applies to the `captureNativeScreenshotFallback` path. Consider offloading the compression to a background thread (e.g. `Executors.newSingleThreadExecutor()`) and marshalling the byte-array result back to the main thread before calling `result.success()`.
Reviews (1): Last reviewed commit: "feat: capture platform views in session ..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| private fun captureNativeScreenshotFallback( | ||
| contentView: View, | ||
| cropLeft: Int, | ||
| cropTop: Int, | ||
| cropRight: Int, | ||
| cropBottom: Int, | ||
| logicalWidth: Int, | ||
| logicalHeight: Int, | ||
| result: Result, | ||
| ) { | ||
| val contentBitmap = | ||
| Bitmap.createBitmap(contentView.width, contentView.height, Bitmap.Config.ARGB_8888) | ||
| contentView.draw(android.graphics.Canvas(contentBitmap)) | ||
|
|
||
| val croppedBitmap = | ||
| Bitmap.createBitmap( | ||
| contentBitmap, | ||
| cropLeft, | ||
| cropTop, | ||
| cropRight - cropLeft, | ||
| cropBottom - cropTop, | ||
| ) | ||
| contentBitmap.recycle() | ||
|
|
||
| val outputBitmap = | ||
| if (croppedBitmap.width == logicalWidth && croppedBitmap.height == logicalHeight) { | ||
| croppedBitmap | ||
| } else { | ||
| Bitmap.createScaledBitmap(croppedBitmap, logicalWidth, logicalHeight, true).also { | ||
| croppedBitmap.recycle() | ||
| } | ||
| } | ||
|
|
||
| result.success(bitmapToPng(outputBitmap)) | ||
| } |
There was a problem hiding this comment.
View.draw() renders using a software Canvas, which cannot read content from hardware-accelerated surfaces like SurfaceView, TextureView, or Metal/Vulkan-backed views. This is the exact content type capturePlatformViews is meant to capture. On pre-API 26 devices, or whenever PixelCopy fails, this fallback will produce blank areas for the native platform views the user opted into capturing. A brief inline comment explaining this limitation would prevent confusion when developers debug unexpected blank regions on older API levels.
Prompt To Fix With AI
This is a comment left during a code review.
Path: posthog_flutter/android/src/main/kotlin/com/posthog/flutter/PosthogFlutterPlugin.kt
Line: 572-606
Comment:
**Soft-canvas fallback silently misses GPU-composited platform views**
`View.draw()` renders using a software `Canvas`, which cannot read content from hardware-accelerated surfaces like `SurfaceView`, `TextureView`, or Metal/Vulkan-backed views. This is the exact content type `capturePlatformViews` is meant to capture. On pre-API 26 devices, or whenever `PixelCopy` fails, this fallback will produce blank areas for the native platform views the user opted into capturing. A brief inline comment explaining this limitation would prevent confusion when developers debug unexpected blank regions on older API levels.
How can I resolve this? If you propose a fix, please make it concise.| private fun bitmapToPng(bitmap: Bitmap): ByteArray { | ||
| val outputStream = ByteArrayOutputStream() | ||
| bitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream) | ||
| bitmap.recycle() | ||
| return outputStream.toByteArray() | ||
| } |
There was a problem hiding this comment.
bitmapToPng is called directly inside the PixelCopy completion callback, which is dispatched to Handler(Looper.getMainLooper()). PNG encoding of a full-screen ARGB_8888 bitmap (Bitmap.CompressFormat.PNG, 100) can take 100–300 ms on a typical device, blocking the main thread for the duration of every session-replay snapshot when capturePlatformViews is enabled. The same applies to the captureNativeScreenshotFallback path. Consider offloading the compression to a background thread (e.g. Executors.newSingleThreadExecutor()) and marshalling the byte-array result back to the main thread before calling result.success().
Prompt To Fix With AI
This is a comment left during a code review.
Path: posthog_flutter/android/src/main/kotlin/com/posthog/flutter/PosthogFlutterPlugin.kt
Line: 608-613
Comment:
**PNG compression runs on the main UI thread**
`bitmapToPng` is called directly inside the `PixelCopy` completion callback, which is dispatched to `Handler(Looper.getMainLooper())`. PNG encoding of a full-screen `ARGB_8888` bitmap (`Bitmap.CompressFormat.PNG, 100`) can take 100–300 ms on a typical device, blocking the main thread for the duration of every session-replay snapshot when `capturePlatformViews` is enabled. The same applies to the `captureNativeScreenshotFallback` path. Consider offloading the compression to a background thread (e.g. `Executors.newSingleThreadExecutor()`) and marshalling the byte-array result back to the main thread before calling `result.success()`.
How can I resolve this? If you propose a fix, please make it concise.
|
Hey, thank you for the work here! I'll have a deeper look soon but at first glance my biggest concern is PII leaking from platform views when the option is enabled. Can PostHogMaskWidget be used for those cases, so at least developers can have control over opt-in masking of platform views? Also I wonder if somehow exposing findMaskableWidgets could help, since this will return masking rects from native. Or maybe just grab a screenshot directly from native sdks with masking already applied (both require refactor of posthog-ios and posthog-android) Curious to hear thoughts from @marandaneto @turnipdabeets as well |
|
i think completely replacing the flutter approach with a native approach isn't great because of masking, afaik its not possible to mask on the native side since we dont have access to the maskable widgets imo a better approach is: stitching recordings, the flutter and the native one theres no automatic way to find out if a screen is Flutter or native (via platform views or pure native) eg |
3 participants
💡 Motivation and Context
Fixes #151.
Session Replay in Flutter was only capturing the Flutter render tree, so native platform views rendered on top of Flutter content were missing from recordings. In practice, this meant native overlays such as paywalls, maps, or web views could appear blank or invisible in session replay snapshots.
This PR adds an opt-in
sessionReplayConfig.capturePlatformViewsflag for session replay. When enabled, Flutter requests a screenshot from the native view hierarchy on iOS and Android and uses that image for replay snapshots, allowing native platform views to be included in recordings.The option defaults to
falsebecause native platform view content cannot be automatically masked using Flutter widget masking rules.💚 How did you test it?
sessionReplayConfig.capturePlatformViewsis disabled by default and correctly serialized in the replay config map.sessionReplayConfig.capturePlatformViewsis enabled.📝 Checklist
If releasing new changes
pnpm changesetto generate a changeset file