Skip to content

Add Android native port for OpenLessAdd Android native port for OpenLess#447

Open
HKLHaoBin wants to merge 3 commits into
Open-Less:betafrom
HKLHaoBin:feat/openless-android-port
Open

Add Android native port for OpenLessAdd Android native port for OpenLess#447
HKLHaoBin wants to merge 3 commits into
Open-Less:betafrom
HKLHaoBin:feat/openless-android-port

Conversation

@HKLHaoBin
Copy link
Copy Markdown

@HKLHaoBin HKLHaoBin commented May 15, 2026

User description

from #278

User description

Summary

This PR adds a native Android port of OpenLess under openless-android/.

The Android implementation ports the desktop dictation pipeline into Java/Android-native components, including:

  • floating trigger service replacing desktop hotkeys
  • microphone foreground service flow
  • Volcengine streaming ASR
  • Whisper-compatible ASR
  • OpenAI-compatible polish / translation / Q&A
  • IME insertion with clipboard fallback
  • history, dictionary, settings, model list, Q&A, and error/detail screens
  • Android-side build / verify / deploy scripts and release checklists

Included

  • new Android app module in openless-android/
  • native Activities for:
    • main screen
    • settings
    • dictionary
    • history detail
    • model list
    • Q&A
    • error detail
  • floating capsule service and notification actions
  • Android Keystore-backed secure storage
  • documentation for build, release, QA, and port status

Validation

Verified locally with:

  • .\build.ps1
  • .\verify.ps1

Also completed one emulator validation round for:

  • main sections
  • settings
  • dictionary
  • Q&A page
  • APK install / launch

Notes

This is a large additive PR because the Android port is introduced as a new directory rather than modifying the existing desktop implementation.

Known remaining work is mainly around:

  • real-device / real-credential end-to-end validation
  • release signing and store submission assets
  • further UI parity and final polish

PR Type

Enhancement, Documentation


Description

  • Android port with build/verify/deploy scripts and version sync

  • Main dictation flow: floating trigger, mic recording, streaming ASR

  • Polish, translation, and IME insertion with clipboard fallback

  • History, dictionary, settings, Q&A, and error detail screens


Diagram Walkthrough

flowchart LR
  A["build.ps1 / verify.ps1 / deploy.ps1"] -- "generates" --> B["APK"]
  B --> C["MainActivity"]
  C --> D["FloatingTriggerService"]
  C --> E["AudioRecorder"]
  E --> F["VolcengineStreamingSession"]
  F --> G["OpenAiPolishProvider"]
  G --> H["TextInserter (IME/Clipboard)"]
  C --> I["HistoryStore / DictionaryStore"]
  C --> J["SettingsActivity / QaPanelActivity"]
Loading

File Walkthrough

Relevant files
Configuration changes
3 files
build.ps1
PowerShell build script for APK generation                             
+126/-0 
deploy.ps1
ADB deployment script for device install                                 
+77/-0   
version.ps1
Version sync script from desktop app                                         
+50/-0   
Tests
1 files
verify.ps1
APK verification script checking signing and permissions 
+109/-0 
Enhancement
15 files
MainActivity.java
Main UI with dictation, history, settings, and tools sections
+1544/-0
QaPanelActivity.java
Q&A panel with voice question and streaming answers           
+754/-0 
ProviderDiagnostics.java
LLM and ASR configuration validation utilities                     
+163/-0 
SettingsActivity.java
Settings screen for providers and preferences                       
+1004/-0
DictionaryActivity.java
Dictionary management for hotwords                                             
+492/-0 
FloatingTriggerService.java
Foreground service for floating dictation trigger               
+389/-0 
HistoryDetailActivity.java
Detail view for individual history entries                             
+323/-0 
ModelListActivity.java
LLM model listing activity                                                             
+229/-0 
AudioRecorder.java
Audio recording and PCM buffer management                               
+138/-0 
VolcengineStreamingSession.java
Streaming ASR via Volcengine WebSocket                                     
+218/-0 
WhisperAsrProvider.java
Whisper-compatible ASR via HTTP                                                   
+53/-0   
OpenAiPolishProvider.java
Polish/translation using OpenAI-compatible API                     
+208/-0 
TextInserter.java
IME and clipboard insertion helper                                             
+63/-0   
HistoryStore.java
Persistent dictation history storage                                         
+155/-0 
SettingsStore.java
Secure settings storage with Android Keystore                       
+155/-0 
Error handling
1 files
ErrorDetailActivity.java
Error detail screen for troubleshooting                                   
+259/-0 
Additional files
36 files
AndroidManifest.xml +81/-0   
ISSUES_FOR_NEXT_SESSION.md [link]   
PORT_STATUS.md +168/-0 
QA_CHECKLIST.md +77/-0   
README.md +146/-0 
RELEASE.md +83/-0   
STORE_SUBMISSION_CHECKLIST.md +54/-0   
ic_launcher_foreground.xml +19/-0   
colors.xml +18/-0   
strings.xml +15/-0   
styles.xml +11/-0   
openless_input_method.xml +4/-0     
AndroidDictationCoordinator.java +278/-0 
AndroidPermissionDiagnostics.java +135/-0 
AsrProvider.java +5/-0     
CapsuleState.java +21/-0   
DictationPhase.java +8/-0     
DictionaryStore.java +232/-0 
InsertStatus.java +25/-0   
OpenLessClient.java +16/-0   
OpenLessHttp.java +43/-0   
OpenLessInputMethodService.java +72/-0   
OpenLessPrompts.java +177/-0 
PermissionStatus.java +23/-0   
PolishMode.java +29/-0   
PolishProvider.java +8/-0     
ProcessTextActivity.java +37/-0   
QaAnswerProvider.java +26/-0   
QaChatMessage.java +11/-0   
QaSessionStore.java +73/-0   
RawTranscript.java +11/-0   
SecureValueStore.java +103/-0 
SimpleWebSocket.java +208/-0 
VolcengineAsrProvider.java +206/-0 
VolcengineFrameCodec.java +111/-0 
WavEncoder.java +54/-0   


PR Type

Enhancement, Documentation


Description

  • Adds native Android OpenLess app

  • Implements dictation, QA, and IME flows

  • Adds build, verify, deploy scripts

  • Documents release, QA, port status


Diagram Walkthrough

flowchart LR
  V["version.ps1 derives Android versioning"] --> B["build.ps1 compiles and signs APK"]
  B --> P["verify.ps1 checks APK metadata"]
  B --> D["deploy.ps1 installs to device"]
  A["Android app module"] --> M["MainActivity and core flows"]
  A --> Q["QaPanelActivity voice question UI"]
  A --> S["SettingsStore and settings screens"]
  A --> F["FloatingTriggerService overlay trigger"]
  A --> I["OpenLessInputMethodService IME insertion"]
  A --> R["Docs, checklists, and manifest"]
Loading

File Walkthrough

Relevant files
Configuration changes
6 files
build.ps1
Add Android APK build pipeline                                                     
+126/-0 
verify.ps1
Validate APK metadata and permissions                                       
+109/-0 
deploy.ps1
Install and launch Android builds                                               
+77/-0   
version.ps1
Derive Android version from desktop                                           
+50/-0   
AndroidManifest.xml
Declare Android permissions and components                             
+81/-0   
release-tauri.yml
Adjust release workflow indexing                                                 
+1/-0     
Enhancement
28 files
MainActivity.java
Build Android dictation home screen                                           
+1544/-0
QaPanelActivity.java
Add conversational QA panel UI                                                     
+754/-0 
SettingsStore.java
Persist Android settings securely                                               
+155/-0 
AsrProvider.java
Define ASR provider interface                                                       
+5/-0     
SettingsActivity.java
Add Android settings screen                                                           
+1004/-0
DictionaryActivity.java
Manage dictionary entries on Android                                         
+492/-0 
FloatingTriggerService.java
Run floating trigger overlay service                                         
+390/-0 
HistoryDetailActivity.java
Show detailed history records                                                       
+323/-0 
AndroidDictationCoordinator.java
Coordinate native dictation flow                                                 
+278/-0 
ModelListActivity.java
List available LLM models                                                               
+229/-0 
SimpleWebSocket.java
Add lightweight websocket client                                                 
+208/-0 
OpenLessPrompts.java
Centralize prompt templates                                                           
+177/-0 
VolcengineAsrProvider.java
Implement Volcengine ASR client                                                   
+206/-0 
OpenAiPolishProvider.java
Implement LLM polish and translate                                             
+208/-0 
VolcengineStreamingSession.java
Stream audio into ASR sessions                                                     
+218/-0 
DictionaryStore.java
Persist and score hotwords                                                             
+232/-0 
ProviderDiagnostics.java
Add provider validation helpers                                                   
+163/-0 
HistoryStore.java
Store Android dictation history                                                   
+155/-0 
AndroidPermissionDiagnostics.java
Report Android permission readiness                                           
+135/-0 
AudioRecorder.java
Capture microphone audio streams                                                 
+138/-0 
VolcengineFrameCodec.java
Encode Volcengine websocket frames                                             
+111/-0 
SecureValueStore.java
Store secrets in Android Keystore                                               
+103/-0 
WhisperAsrProvider.java
Implement Whisper-compatible transcription                             
+53/-0   
OpenLessInputMethodService.java
Add Android IME insertion path                                                     
+72/-0   
TextInserter.java
Insert text with clipboard fallback                                           
+63/-0   
QaSessionStore.java
Track QA conversation state                                                           
+73/-0   
WavEncoder.java
Encode audio to WAV format                                                             
+54/-0   
ic_launcher_foreground.xml
Add Android launcher foreground asset                                       
+19/-0   
Documentation
5 files
PORT_STATUS.md
Document Android port progress                                                     
+168/-0 
README.md
Explain Android build and usage                                                   
+146/-0 
QA_CHECKLIST.md
Add Android QA checklist                                                                 
+77/-0   
RELEASE.md
Document Android release steps                                                     
+83/-0   
STORE_SUBMISSION_CHECKLIST.md
Prepare store submission checklist                                             
+54/-0   
Error handling
1 files
ErrorDetailActivity.java
Present structured Android errors                                               
+259/-0 
Additional files
17 files
ISSUES_FOR_NEXT_SESSION.md [link]   
colors.xml +18/-0   
strings.xml +15/-0   
styles.xml +11/-0   
openless_input_method.xml +4/-0     
CapsuleState.java +21/-0   
DictationPhase.java +8/-0     
InsertStatus.java +25/-0   
OpenLessClient.java +16/-0   
OpenLessHttp.java +43/-0   
PermissionStatus.java +23/-0   
PolishMode.java +29/-0   
PolishProvider.java +8/-0     
ProcessTextActivity.java +37/-0   
QaAnswerProvider.java +26/-0   
QaChatMessage.java +11/-0   
RawTranscript.java +11/-0   

HKLHaoBin and others added 3 commits May 15, 2026 18:17
Co-authored-by: Cursor <cursoragent@cursor.com>
…droid-port)

Co-authored-by: Cursor <cursoragent@cursor.com>
在 FloatingTriggerService 而非 MicBubbleView 中跟踪手势开始时间,以确保长按检测的准确性。此前,MicBubbleView 中的 downAt 时间戳可能会过期,导致 ACTION_UP 时协调器行为异常。同时增加了适当的 ACTION_CANCEL 处理,用于重置手势状态,防止过时的计时数据影响后续手势。
@github-actions
Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Release signing

When -Configuration release is used without all keystore parameters, the script silently falls back to the debug keystore and still emits OpenLessAndroid-release.apk. That produces a release-named APK that is not release-signed, which can break store submission and make later installs fail if it is expected to replace a properly signed build.

if ($UseCustomKeystore) {
    Invoke-Checked { & $ApkSigner sign `
        --ks $KeystorePath `
        --ks-key-alias $KeystoreAlias `
        --ks-pass ("pass:" + $StorePass) `
        --key-pass ("pass:" + $KeyPass) `
        --out $Signed `
        $Aligned }
} else {
    if (-not (Test-Path $Keystore)) {
        keytool -genkeypair `
            -keystore $Keystore `
            -storepass android `
            -keypass android `
            -alias androiddebugkey `
            -keyalg RSA `
            -keysize 2048 `
            -validity 10000 `
            -dname "CN=Android Debug,O=Android,C=US" | Out-Null
    }

    Invoke-Checked { & $ApkSigner sign `
        --ks $Keystore `
        --ks-pass pass:android `
        --key-pass pass:android `
        --out $Signed `
        $Aligned }
Stale opacity

Disabled mode buttons are dimmed, but the enabled branch never restores their alpha. If a mode is disabled in settings and later re-enabled, the button stays semi-transparent even though it is active, which makes the control look broken and harder to use.

private void refreshModeButtons() {
    if (modeRow == null) return;
    for (int i = 0; i < modeRow.getChildCount(); i++) {
        View child = modeRow.getChildAt(i);
        if (child instanceof Button && child.getTag() instanceof PolishMode) {
            Button b = (Button) child;
            PolishMode mode = (PolishMode) b.getTag();
            boolean enabled = isModeEnabled(mode);
            boolean active = mode == settings.mode;
            b.setEnabled(enabled);
            b.setTextColor(enabled && active ? Color.WHITE : OL_INK_3);
            if (enabled && active) b.setBackgroundDrawable(pillBg(OL_BLUE));
            else if (enabled) b.setBackgroundDrawable(outlineBg(OL_LINE_STRONG));
            else { b.setBackgroundDrawable(outlineBg(OL_LINE)); b.setAlpha(0.4f); }
        }
    }

@appergb
Copy link
Copy Markdown
Collaborator

appergb commented May 16, 2026

@claude 应该不会对现有代码造成破坏性更改吧?如果没有的话,就可以合并了,帮我审查一下。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants