feat: 支持控制 HarmonyOS(hdc) #203
Open
shouldnotappearcalm wants to merge 1 commit intozai-org:mainfrom
Open
Conversation
- Introduce device controller abstraction (base.py) to support multiple backends - Implement AdbDeviceController for Android devices - Implement HarmonyDeviceController for HarmonyOS devices via hdc - Add --backend and --hdc-path CLI arguments to main.py - Refactor ActionHandler to use device controller interface - Skip ADB checks when using Harmony backend
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
phone_agent引入统一的设备控制抽象层,使 PhoneAgent 可以面向接口编程,解耦具体设备实现。主要改动
设备抽象
DeviceController接口,统一封装:截屏、点击、滑动、文本输入、系统返回/Home、应用启动、当前 App 获取等能力。AdbDeviceController实现,集中承接现有 ADB 行为,便于后续扩展其它平台。HarmonyOS 支持
HarmonyDeviceController,通过hdc和uitest命令与鸿蒙设备交互:hdc截屏并拉取到本地,供模型视觉输入使用。uitest uiInput执行点击、滑动、系统按键和文本输入等操作。Agent 与 Action 解耦
PhoneAgent不再直接依赖 adb 模块,而是接受一个设备后端(ADB 或 Harmony),只面向抽象的DeviceController。ActionHandler从直接调用 adb 函数,改为调用注入的DeviceController,动作语义保持不变(Tap/Swipe/Type/Back/Home 等)。CLI 扩展
main.py新增设备相关参数:--backend {adb,harmony}:选择设备控制后端(默认adb,兼容原行为)。--hdc-path PATH:在backend=harmony时指定hdc可执行文件路径。backend=adb:保留原有 ADB 工具 / 设备 / ADB Keyboard 检查。backend=harmony:跳过 ADB 相关检查,避免在纯 Harmony 场景下被阻塞。使用说明
Android / ADB 模式(与原来保持一致)
HarmonyOS / hdc 模式
可选环境变量
之后可省略
--backend和--hdc-path参数,直接通过main.py传入模型和任务即可。兼容性与影响