mirror of
https://github.com/FoundationAgents/MetaGPT.git
synced 2026-06-08 15:05:17 +02:00
Update Readme V0.2
This commit is contained in:
didi
parent
52e709cc26
commit
5df1c9fed6
2 changed files with 20 additions and 20 deletions
|
|
@ -1,7 +1,6 @@
|
|||
# MetaGPT Android Assistant
|
||||
|
||||
The MetaGPT Android Assistant is an intelligent assistance tool driven by a multi-modal large language model based on the advanced MetaGPT framework.
|
||||
It has the ability to self-learn, mastering users' daily usage patterns through learning, and can automatically complete various application operations according to user instructions, achieving comprehensive liberation of users' hands.
|
||||
The MetaGPT Android Assistant is an intelligent assistance tool driven by a multi-modal large language model based on the advanced MetaGPT framework. It has the ability to self-learn, mastering users' daily usage patterns through learning, and can automatically complete various application operations according to user instructions, achieving comprehensive liberation of users' hands.
|
||||
Next, we will introduce the functions of the MetaGPT Android Assistant and how to use it.
|
||||
|
||||
## Features
|
||||
|
|
@ -12,16 +11,15 @@ ### Learning Stage
|
|||
|
||||
By learning from human demonstrations or exploring apps based on human instructions, the MetaGPT Android Assistant can learn the functionality of apps, generate corresponding operation documents for use in the subsequent "automatic execution" stage. Approximately 20 rounds of exploration for any given task objective can significantly improve performance.
|
||||
|
||||
By setting the stage to "learn", you can ask the Android Assistant to enter the self-learning stage. By setting the mode to auto, you can instruct the Android Assistant to learn through automatic exploration; by setting the mode to manual, you can instruct the Android Assistant to learn through human manual demonstration. In the usage section, we provide detailed explanations of the script parameters.
|
||||
You can try experimenting with automatic exploration and manual demonstration modes on the "Messenger" app with the following commands:
|
||||
By setting the `stage` to `learn`, you can ask the Android Assistant to enter the learning stage. By setting the `mode` to `auto`, you can instruct the Android Assistant to learn through automatic exploration; by setting the mode to manual, you can instruct the Android Assistant to learn through human manual demonstration. In the usage section, we provide detailed explanations of the script parameters. You can try experimenting with automatic exploration and manual demonstration modes on the "Messenger" app with the following commands:
|
||||
|
||||
```bash
|
||||
cd examples/android_assistant
|
||||
python run_assistant.py "your task description" --stage "learn" --mode "auto or manual" --app-name "Messenger"
|
||||
python run_assistant.py "Send 'When will we release this feature?' to +86 8888888" --stage "learn" --mode "auto or manual" --app-name "Messenger"
|
||||
```
|
||||
|
||||
#### Learning Based on Human Demonstration
|
||||
When asking the Android Assistant to perform self-exploration during the self-learning stage, you can free your hands. However, when instructing it to learn according to your commands, you need to follow the instructions in the terminal for the Android Assistant to accurately learn your operation methods.
|
||||
When asking the Android Assistant to perform self-exploration during the learning stage, you can free your hands. However, when instructing it to learn according to your commands, you need to follow the instructions in the terminal for the Android Assistant to accurately learn your operation methods.
|
||||
A possible example is as follows:
|
||||
|
||||
```bash
|
||||
|
|
@ -29,7 +27,9 @@ #### Learning Based on Human Demonstration
|
|||
python run_assistant.py "Send 'When will we release this feature?' to +86 8888888" --stage "learn" --mode "manual" --app-name "Messenger"
|
||||
```
|
||||
|
||||
After running this command, you will have a conversation similar to the following in the terminal, directing the Android Assistant to learn your demonstration behavior:
|
||||
After running this command, you will first see a screenshot of an Android screen that has been marked at various interactive locations, as shown in the figure below:
|
||||
##### TODO Add Image
|
||||
After remembering the location where you want to operate, a request similar to the one below will be output in the terminal. Reply to it and thereby direct the Android assistant to learn your demonstration action:
|
||||
|
||||
```bash
|
||||
| INFO | examples.android_assistant.actions.manual_record:run:96 - Which element do you want to tap? Choose a numeric tag from 1 to 11:
|
||||
|
|
@ -43,9 +43,9 @@ ### Automatic Execution Stage
|
|||
After the Android Assistant completes the learning stage, you can command it to complete tasks on the phone through text descriptions. By configuring the operation documents from the self-learning stage, the Android Assistant has richer prior knowledge, and its execution capabilities are further enhanced.
|
||||
You can instruct the Android Assistant to send messages in the "Messenger" app with the following command:
|
||||
```bash
|
||||
python run_assistant.py "your task description" --stage "act" --mode "auto or manual" --app-name "Messenger"
|
||||
python run_assistant.py "Send 'When will we release this feature?' to +86 8888888" --stage "act" --mode "auto or manual" --app-name "Messenger"
|
||||
```
|
||||
Where, selecting mode as auto will cause the Android Assistant to use the operation documents accumulated during self-exploration; selecting mode as manual will cause the Android Assistant to use the operation documents accumulated during human demonstration learning.
|
||||
Specifically, by selecting `auto` for `mode`, the Android assistant will employ the operational records compiled through self-exploration. Alternatively, if `manual` is chosen as the `mode`, the Android assistant will leverage the operation manuals accrued from learning via human demonstration.
|
||||
|
||||
## Installation
|
||||
To use the Android Assistant, you first need to meet the following conditions:
|
||||
|
|
@ -100,5 +100,5 @@ ## Usage
|
|||
```
|
||||
|
||||
## Acknowledgements
|
||||
The MetaGPT Android Assistant has referenced some ideas and code from the [Appagent](https://github.com/mnotgod96/AppAgent) project. We thank the developers of the Appagent project.
|
||||
The MetaGPT Android Assistant has referenced some ideas and code from the [AppAgent](https://github.com/mnotgod96/AppAgent) project. We thank the developers of the Appagent project.
|
||||
|
||||
|
|
|
|||
|
|
@ -10,16 +10,14 @@ ## 功能
|
|||
|
||||
### 自我学习阶段
|
||||
|
||||
通过学习人类演示或基于人类指令对app进行探索，MetaGPT
|
||||
安卓助理可以对app的功能进行学习，生成相应的操作文档，为后续的“自动执行”阶段使用。对于任何给定的任务目标，进行约20轮的探索可以显著提高性能。
|
||||
通过学习人类演示或基于人类指令对app进行探索，MetaGPT安卓助理可以对app的功能进行学习，生成相应的操作文档，为后续的“自动执行”阶段使用。对于任何给定的任务目标，进行约20轮的探索可以显著提高性能。
|
||||
|
||||
通过设定stage 为 "learn" 可要求安卓助理进入自我学习阶段。通过设定 mode 为
|
||||
auto，可要求安卓助理通过自动探索学习，通过设定 mode 为 manual，可要求安卓助理通过人类手动演示学习。在使用章节，我们对脚本的参数进行了详细的说明。
|
||||
通过设定`stage`为`learn`可要求安卓助理进入自我学习阶段。通过设定`mode`为`auto`，可要求安卓助理通过自动探索学习，通过设定`mode`为`manual`，可要求安卓助理通过人类手动演示学习。在使用章节，我们对脚本的参数进行了详细的说明。
|
||||
您可以尝试对“Messenger”应用程序进行自动探索和手动演示模式的实验，具体命令如下：
|
||||
|
||||
```bash
|
||||
cd examples/android_assistant
|
||||
python run_assistant.py "your task description" --stage "learn" --mode "auto or manual" --app-name "Messenger"
|
||||
python run_assistant.py "Send 'When will we release this feature? to +86 8888888'" --stage "learn" --mode "auto or manual" --app-name "Messenger"
|
||||
```
|
||||
|
||||
#### 基于人类演示的学习
|
||||
|
|
@ -31,7 +29,9 @@ #### 基于人类演示的学习
|
|||
python run_assistant.py "Send 'When will we release this feature? to +86 8888888'" --stage "learn" --mode "manual" --app-name "Messenger"
|
||||
```
|
||||
|
||||
在运行这一指令后，你将在终端中产生与下面类似的对话，进而指挥安卓助理学习你的演示行为：
|
||||
在运行这一指令后，你将首先看到一个在各个可交互的位置进行了标记的安卓屏幕的截图，如下图：
|
||||
###### TODO Add Image
|
||||
在记住你要操作的位置之后，终端中将会输出与下面类似的要求，回复它，进而指挥安卓助理学习你的演示行为：
|
||||
|
||||
```bash
|
||||
| INFO | examples.android_assistant.actions.manual_record:run:96 - Which element do you want to tap? Choose a numeric tag from 1 to 11:
|
||||
|
|
@ -44,9 +44,9 @@ ### 自动执行阶段
|
|||
在安卓助理完成了自我学习阶段之后，您可以通过文本描述的方式，指挥安卓助理在手机中完成任务。通过为其配置自我学习阶段的操作文档，安卓助理具备了更丰富的前置知识，执行能力进一步得到提升。
|
||||
你可以通过以下指令，指挥安卓助理在“Messenger”应用中发送信息：
|
||||
```bash
|
||||
python run_assistant.py "your task description" --stage "act" --mode "auto or manual" --app-name "Messenger"
|
||||
python run_assistant.py "Send 'When will we release this feature? to +86 8888888'" --stage "act" --mode "auto or manual" --app-name "Messenger"
|
||||
```
|
||||
其中，mode选择auto，将导致安卓助理使用自我探索中积累的操作文档；mode选择 manual，将导致安卓助理使用人类演示学习中积累的操作文档。
|
||||
其中，`mode`选择`auto`，安卓助理将使用自我探索中积累的操作文档；`mode`选择`manual`，安卓助理将使用人类演示学习中积累的操作文档。
|
||||
|
||||
## 安装
|
||||
为了使用安卓助理，你首先需要满足以下条件：
|
||||
|
|
@ -62,7 +62,7 @@ ## 安装
|
|||
adb devices
|
||||
```
|
||||
## 使用
|
||||
MetaGPT 安卓助理在MetaGPT框架中被设计为一个Role与多个Action的集合，你可以通过运行`run_assistant.py`脚本来运行它。这一脚本具体的参数说明如下：
|
||||
MetaGPT 安卓助理在MetaGPT框架中被设计为一个`Role`与多个`Action`的集合，你可以通过运行`run_assistant.py`脚本来运行它。这一脚本具体的参数说明如下：
|
||||
```text
|
||||
用法：run_assistant.py [选项] 任务描述
|
||||
|
||||
|
|
@ -95,4 +95,4 @@ ## 使用
|
|||
```
|
||||
|
||||
## 致谢
|
||||
MetaGPT 安卓助理参考了 [Appagent](https://github.com/mnotgod96/AppAgent) 项目的部分思路与代码，感谢 Appagent 项目的开发者们。
|
||||
MetaGPT 安卓助理参考了 [AppAgent](https://github.com/mnotgod96/AppAgent) 项目的部分思路与代码，感谢 Appagent 项目的开发者们。
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue