Skip to content

Add Android native port for OpenLessAdd Android native port for OpenLess#278

Open
ccbili30-collab wants to merge 1 commit into
Open-Less:mainfrom
ccbili30-collab:codex/openless-android-port
Open

Add Android native port for OpenLessAdd Android native port for OpenLess#278
ccbili30-collab wants to merge 1 commit into
Open-Less:mainfrom
ccbili30-collab:codex/openless-android-port

Conversation

@ccbili30-collab
Copy link
Copy Markdown

@ccbili30-collab ccbili30-collab commented May 6, 2026

User description

Summary

This PR adds a native Android port of OpenLess under openless-android/.

The Android implementation ports the desktop dictation pipeline into Java/Android-native components, including:

  • floating trigger service replacing desktop hotkeys
  • microphone foreground service flow
  • Volcengine streaming ASR
  • Whisper-compatible ASR
  • OpenAI-compatible polish / translation / Q&A
  • IME insertion with clipboard fallback
  • history, dictionary, settings, model list, Q&A, and error/detail screens
  • Android-side build / verify / deploy scripts and release checklists

Included

  • new Android app module in openless-android/
  • native Activities for:
    • main screen
    • settings
    • dictionary
    • history detail
    • model list
    • Q&A
    • error detail
  • floating capsule service and notification actions
  • Android Keystore-backed secure storage
  • documentation for build, release, QA, and port status

Validation

Verified locally with:

  • .\build.ps1
  • .\verify.ps1

Also completed one emulator validation round for:

  • main sections
  • settings
  • dictionary
  • Q&A page
  • APK install / launch

Notes

This is a large additive PR because the Android port is introduced as a new directory rather than modifying the existing desktop implementation.

Known remaining work is mainly around:

  • real-device / real-credential end-to-end validation
  • release signing and store submission assets
  • further UI parity and final polish

PR Type

Enhancement, Documentation


Description

  • Android port with build/verify/deploy scripts and version sync

  • Main dictation flow: floating trigger, mic recording, streaming ASR

  • Polish, translation, and IME insertion with clipboard fallback

  • History, dictionary, settings, Q&A, and error detail screens


Diagram Walkthrough

flowchart LR
  A["build.ps1 / verify.ps1 / deploy.ps1"] -- "generates" --> B["APK"]
  B --> C["MainActivity"]
  C --> D["FloatingTriggerService"]
  C --> E["AudioRecorder"]
  E --> F["VolcengineStreamingSession"]
  F --> G["OpenAiPolishProvider"]
  G --> H["TextInserter (IME/Clipboard)"]
  C --> I["HistoryStore / DictionaryStore"]
  C --> J["SettingsActivity / QaPanelActivity"]
Loading

File Walkthrough

Relevant files
Configuration changes
3 files
build.ps1
PowerShell build script for APK generation                             
+126/-0 
deploy.ps1
ADB deployment script for device install                                 
+77/-0   
version.ps1
Version sync script from desktop app                                         
+50/-0   
Tests
1 files
verify.ps1
APK verification script checking signing and permissions 
+109/-0 
Enhancement
15 files
MainActivity.java
Main UI with dictation, history, settings, and tools sections
+1544/-0
QaPanelActivity.java
Q&A panel with voice question and streaming answers           
+754/-0 
ProviderDiagnostics.java
LLM and ASR configuration validation utilities                     
+163/-0 
SettingsActivity.java
Settings screen for providers and preferences                       
+1004/-0
DictionaryActivity.java
Dictionary management for hotwords                                             
+492/-0 
FloatingTriggerService.java
Foreground service for floating dictation trigger               
+389/-0 
HistoryDetailActivity.java
Detail view for individual history entries                             
+323/-0 
ModelListActivity.java
LLM model listing activity                                                             
+229/-0 
AudioRecorder.java
Audio recording and PCM buffer management                               
+138/-0 
VolcengineStreamingSession.java
Streaming ASR via Volcengine WebSocket                                     
+218/-0 
WhisperAsrProvider.java
Whisper-compatible ASR via HTTP                                                   
+53/-0   
OpenAiPolishProvider.java
Polish/translation using OpenAI-compatible API                     
+208/-0 
TextInserter.java
IME and clipboard insertion helper                                             
+63/-0   
HistoryStore.java
Persistent dictation history storage                                         
+155/-0 
SettingsStore.java
Secure settings storage with Android Keystore                       
+155/-0 
Error handling
1 files
ErrorDetailActivity.java
Error detail screen for troubleshooting                                   
+259/-0 
Additional files
36 files
AndroidManifest.xml +81/-0   
ISSUES_FOR_NEXT_SESSION.md [link]   
PORT_STATUS.md +168/-0 
QA_CHECKLIST.md +77/-0   
README.md +146/-0 
RELEASE.md +83/-0   
STORE_SUBMISSION_CHECKLIST.md +54/-0   
ic_launcher_foreground.xml +19/-0   
colors.xml +18/-0   
strings.xml +15/-0   
styles.xml +11/-0   
openless_input_method.xml +4/-0     
AndroidDictationCoordinator.java +278/-0 
AndroidPermissionDiagnostics.java +135/-0 
AsrProvider.java +5/-0     
CapsuleState.java +21/-0   
DictationPhase.java +8/-0     
DictionaryStore.java +232/-0 
InsertStatus.java +25/-0   
OpenLessClient.java +16/-0   
OpenLessHttp.java +43/-0   
OpenLessInputMethodService.java +72/-0   
OpenLessPrompts.java +177/-0 
PermissionStatus.java +23/-0   
PolishMode.java +29/-0   
PolishProvider.java +8/-0     
ProcessTextActivity.java +37/-0   
QaAnswerProvider.java +26/-0   
QaChatMessage.java +11/-0   
QaSessionStore.java +73/-0   
RawTranscript.java +11/-0   
SecureValueStore.java +103/-0 
SimpleWebSocket.java +208/-0 
VolcengineAsrProvider.java +206/-0 
VolcengineFrameCodec.java +111/-0 
WavEncoder.java +54/-0   

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 059b5067c6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +228 to +230
if (System.currentTimeMillis() - bubble.downAt >= LONG_PRESS_CANCEL_MS) {
coordinator.cancel();
} else { coordinator.toggle(); }
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Record press time before deciding cancel vs toggle

A normal tap on the floating bubble will be treated as a long-press here, so coordinator.toggle() is effectively unreachable in non-drag taps. downAt is only updated in MicBubbleView.onTouchEvent, but touch events are already consumed by the OnTouchListener (setOnTouchListener(this::onBubbleTouch) returns true), so downAt remains 0 and System.currentTimeMillis() - bubble.downAt is always greater than LONG_PRESS_CANCEL_MS. This makes the main start/stop dictation interaction fail in production.

Useful? React with 👍 / 👎.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 6, 2026

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 Security concerns

Sensitive information exposure:
build.ps1 and deploy.ps1 accept StorePass and KeyPass as command-line arguments for APK signing. Those values can end up in PowerShell history, logs, or process listings on a shared machine or CI runner, which exposes release credentials. Consider reading them from a secure prompt/secret store instead.

⚡ Recommended focus areas for review

Session leak

When a non-Whisper recording is very short, the early return happens before inlineVolcengineSession is cleared or closed. That leaves the streaming ASR session open, which can leak the WebSocket/connection and interfere with the next dictation attempt after a quick tap.

micButton.setText("按住说话");
if (recording.pcm.length < 1000) { setStatus("录音过短", OL_WARN); return; }
setStatus("正在转写...", OL_WARN);
VolcengineStreamingSession session = inlineVolcengineSession;
Multi-device install

The deploy flow checks that at least one adb device is attached, but it still runs adb install -r without selecting a target serial. If two devices or emulators are connected, adb will fail with the usual "more than one device/emulator" error instead of installing to one of them.

$deviceLines = & $AdbPath devices | Select-Object -Skip 1 | Where-Object { $_.Trim() -ne "" }
if ($deviceLines.Count -eq 0) {
    throw "No adb device attached."
}

Write-Host "Deploying $ApkPath"
foreach ($line in $deviceLines) {
    Write-Host ("  Device: " + $line.Trim())
}

& $AdbPath install -r $ApkPath
if ($LASTEXITCODE -ne 0) {

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 059b5067c6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +228 to +229
if (System.currentTimeMillis() - bubble.downAt >= LONG_PRESS_CANCEL_MS) {
coordinator.cancel();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Capture press start time in bubble touch handling

A normal tap on the floating bubble is always interpreted as a long press, so coordinator.toggle() is never reached. onBubbleTouch compares System.currentTimeMillis() - bubble.downAt, but downAt is only assigned in MicBubbleView.onTouchEvent; that method does not run here because the installed OnTouchListener consumes every event (return true). As a result, downAt stays at its default (0) and every non-drag tap triggers cancel instead of start/stop dictation.

Useful? React with 👍 / 👎.

@appergb
Copy link
Copy Markdown
Collaborator

appergb commented May 6, 2026

我们可能会把你的 PR 合并到新分支,在此之前请先解决 Codex 以及 AI agent 所提出的问题。

@HKLHaoBin
Copy link
Copy Markdown

您好,我看到您已经很久没有继续跟进这条PR了,我认为您实现的功能质量很好,因此我决定跟进您的PR。解决了此前未能解决的问题。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants