feat: merge geekan:env_refactor

This commit is contained in:
莘权 马 2023-12-13 17:57:36 +08:00
commit 4582f65cf8
32 changed files with 240 additions and 107 deletions

View file

@ -1,3 +1,4 @@
# MetaGPT: The Multi-Agent Framework
<p align="center">
@ -50,9 +51,9 @@ # Step 2: Clone the repository to your local machine for latest version, and ins
cd MetaGPT
pip3 install -e. # or pip3 install metagpt # for stable version
# Step 3: run the startup.py
# Step 3: run metagpt cli
# setup your OPENAI_API_KEY in key.yaml copy from config.yaml
python3 startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# Step 4 [Optional]: If you want to save the artifacts like diagrams such as quadrant chart, system designs, sequence flow in the workspace, you can execute the step before Step 3. By default, the framework is compatible, and the entire process can be run completely without executing this step.
# If executing, ensure that NPM is installed on your system. Then install mermaid-js. (If you don't have npm in your computer, please go to the Node.js official website to install Node.js https://nodejs.org/ and then you will have npm tool in your computer.)
@ -78,7 +79,7 @@ # Step 2: Run metagpt demo with container
-v /opt/metagpt/config/key.yaml:/app/metagpt/config/key.yaml \
-v /opt/metagpt/workspace:/app/metagpt/workspace \
metagpt/metagpt:latest \
python startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
```
detail installation please refer to [docker_install](https://docs.deepwisdom.ai/guide/get_started/installation.html#install-with-docker)
@ -117,7 +118,7 @@ ### Contact Information
If you have any questions or feedback about this project, please feel free to contact us. We highly appreciate your suggestions!
- **Email:** alexanderwu@fuzhi.ai
- **Email:** alexanderwu@deepwisdom.ai
- **GitHub Issues:** For more technical inquiries, you can also create a new issue in our [GitHub repository](https://github.com/geekan/metagpt/issues).
We will respond to all questions within 2-3 business days.

View file

@ -98,7 +98,7 @@
1. How to change the investment amount?
1. You can view all commands by typing `python startup.py --help`
1. You can view all commands by typing `metagpt --help`
1. Which version of Python is more stable?
@ -134,7 +134,7 @@
1. Configuration instructions for SD Skills: The SD interface is currently deployed based on *https://github.com/AUTOMATIC1111/stable-diffusion-webui* **For environmental configurations and model downloads, please refer to the aforementioned GitHub repository. To initiate the SD service that supports API calls, run the command specified in cmd with the parameter nowebui, i.e.,
1. > python webui.py --enable-insecure-extension-access --port xxx --no-gradio-queue --nowebui
1. > python3 webui.py --enable-insecure-extension-access --port xxx --no-gradio-queue --nowebui
1.     Once it runs without errors, the interface will be accessible after approximately 1 minute when the model finishes loading.
1. Configure SD_URL and SD_T2I_API in the config.yaml/key.yaml files.
1. ![](https://p3-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp/065295a67b0b4feea665d1372722d49d~tplv-k3u1fbpfcp-zoom-1.image)

View file

@ -47,9 +47,9 @@ # 第 2 步:克隆最新仓库到您的本地机器,并进行安装。
cd MetaGPT
pip3 install -e. # 或者 pip3 install metagpt # 安装稳定版本
# 第 3 步:执行startup.py
# 第 3 步:执行metagpt
# 拷贝config.yaml为key.yaml并设置你自己的OPENAI_API_KEY
python3 startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# 第 4 步【可选的】如果你想在执行过程中保存像象限图、系统设计、序列流程等图表这些产物可以在第3步前执行该步骤。默认的框架做了兼容在不执行该步的情况下也可以完整跑完整个流程。
# 如果执行,确保您的系统上安装了 NPM。并使用npm安装mermaid-js
@ -75,7 +75,7 @@ # 步骤2: 使用容器运行metagpt演示
-v /opt/metagpt/config/key.yaml:/app/metagpt/config/key.yaml \
-v /opt/metagpt/workspace:/app/metagpt/workspace \
metagpt/metagpt:latest \
python startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
```
详细的安装请安装 [docker_install](https://docs.deepwisdom.ai/zhcn/guide/get_started/installation.html#%E4%BD%BF%E7%94%A8docker%E5%AE%89%E8%A3%85)
@ -114,7 +114,7 @@ ### 联系信息
如果您对这个项目有任何问题或反馈,欢迎联系我们。我们非常欢迎您的建议!
- **邮箱:** alexanderwu@fuzhi.ai
- **邮箱:** alexanderwu@deepwisdom.ai
- **GitHub 问题:** 对于更技术性的问题,您也可以在我们的 [GitHub 仓库](https://github.com/geekan/metagpt/issues) 中创建一个新的问题。
我们会在2-3个工作日内回复所有问题。

View file

@ -41,7 +41,7 @@ ## MetaGPT の能力
## 例GPT-4 で完全生成)
例えば、`python startup.py "Toutiao のような RecSys をデザインする"`と入力すると、多くの出力が得られます
例えば、`metagpt "Toutiao のような RecSys をデザインする"`と入力すると、多くの出力が得られます
![Jinri Toutiao Recsys データと API デザイン](resources/workspace/content_rec_sys/resources/data_api_design.png)
@ -60,16 +60,16 @@ ### 伝統的なインストール
```bash
# ステップ 1: Python 3.9+ がシステムにインストールされていることを確認してください。これを確認するには:
python --version
python3 --version
# ステップ 2: リポジトリをローカルマシンにクローンし、インストールする。
git clone https://github.com/geekan/MetaGPT.git
cd MetaGPT
pip install -e.
# ステップ 3: startup.py を実行する
# ステップ 3: metagpt を実行する
# config.yaml を key.yaml にコピーし、独自の OPENAI_API_KEY を設定します
python3 startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# ステップ 4 [オプション]: 実行中に PRD ファイルなどのアーティファクトを保存する場合は、ステップ 3 の前にこのステップを実行できます。デフォルトでは、フレームワークには互換性があり、この手順を実行しなくてもプロセス全体を完了できます。
# NPM がシステムにインストールされていることを確認してください。次に mermaid-js をインストールします。(お使いのコンピューターに npm がない場合は、Node.js 公式サイトで Node.js https://nodejs.org/ をインストールしてください。)
@ -178,7 +178,7 @@ # ステップ 2: コンテナで metagpt デモを実行する
-v /opt/metagpt/config/key.yaml:/app/metagpt/config/key.yaml \
-v /opt/metagpt/workspace:/app/metagpt/workspace \
metagpt/metagpt:latest \
python startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# コンテナを起動し、その中でコマンドを実行することもできます
docker run --name metagpt -d \
@ -188,7 +188,7 @@ # コンテナを起動し、その中でコマンドを実行することもで
metagpt/metagpt:latest
docker exec -it metagpt /bin/bash
$ python startup.py "Write a cli snake game"
$ metagpt "Write a cli snake game"
```
コマンド `docker run ...` は以下のことを行います:
@ -196,7 +196,7 @@ # コンテナを起動し、その中でコマンドを実行することもで
- 特権モードで実行し、ブラウザの実行権限を得る
- ホスト設定ファイル `/opt/metagpt/config/key.yaml` をコンテナ `/app/metagpt/config/key.yaml` にマップします
- ホストディレクトリ `/opt/metagpt/workspace` をコンテナディレクトリ `/app/metagpt/workspace` にマップするs
- デモコマンド `python startup.py "Write a cli snake game"` を実行する
- デモコマンド `metagpt "Write a cli snake game"` を実行する
### 自分でイメージをビルドする
@ -225,11 +225,11 @@ ## チュートリアル: スタートアップの開始
```shell
# スクリプトの実行
python startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# プロジェクトの実施にエンジニアを雇わないこと
python startup.py "Write a cli snake game" --implement False
metagpt "Write a cli snake game" --no-implement
# エンジニアを雇い、コードレビューを行う
python startup.py "Write a cli snake game" --code_review True
metagpt "Write a cli snake game" --code_review
```
スクリプトを実行すると、`workspace/` ディレクトリに新しいプロジェクトが見つかります。
@ -239,17 +239,17 @@ ### プラットフォームまたはツールの設定
要件を述べるときに、どのプラットフォームまたはツールを使用するかを指定できます。
```shell
python startup.py "pygame をベースとした cli ヘビゲームを書く"
metagpt "pygame をベースとした cli ヘビゲームを書く"
```
### 使用方法
```
会社名
startup.py - 私たちは AI で構成されたソフトウェア・スタートアップです。私たちに投資することは、無限の可能性に満ちた未来に力を与えることです。
metagpt - 私たちは AI で構成されたソフトウェア・スタートアップです。私たちに投資することは、無限の可能性に満ちた未来に力を与えることです。
シノプシス
startup.py IDEA <flags>
metagpt IDEA <flags>
説明
私たちは AI で構成されたソフトウェア・スタートアップです。私たちに投資することは、無限の可能性に満ちた未来に力を与えることです。
@ -317,7 +317,7 @@ ## お問い合わせ先
このプロジェクトに関するご質問やご意見がございましたら、お気軽にお問い合わせください。皆様のご意見をお待ちしております!
- **Email:** alexanderwu@fuzhi.ai
- **Email:** alexanderwu@deepwisdom.ai
- **GitHub Issues:** 技術的なお問い合わせについては、[GitHub リポジトリ](https://github.com/geekan/metagpt/issues) に新しい issue を作成することもできます。
ご質問には 2-3 営業日以内に回答いたします。

View file

@ -16,12 +16,12 @@ ### Tasks
To reach version v0.5, approximately 70% of the following tasks need to be completed.
1. Usability
1. Release v0.01 pip package to try to solve issues like npm installation (though not necessarily successfully)
1. ~~Release v0.01 pip package to try to solve issues like npm installation (though not necessarily successfully)~~ (v0.3.0)
2. Support for overall save and recovery of software companies
3. Support human confirmation and modification during the process
3. ~~Support human confirmation and modification during the process~~ (v0.3.0) New: Support human confirmation and modification with fewer constrainsts and a more user-friendly interface
4. Support process caching: Consider carefully whether to add server caching mechanism
5. Resolve occasional failure to follow instruction under current prompts, causing code parsing errors, through stricter system prompts
6. Write documentation, describing the current features and usage at all levels
5. ~~Resolve occasional failure to follow instruction under current prompts, causing code parsing errors, through stricter system prompts~~ (v0.4.0, with function call)
6. Write documentation, describing the current features and usage at all levels (ongoing, continuously adding contents to [documentation site](https://docs.deepwisdom.ai/guide/get_started/introduction.html))
7. ~~Support Docker~~
2. Features
1. Support a more standard and stable parser (need to analyze the format that the current LLM is better at)
@ -30,31 +30,33 @@ ### Tasks
4. Complete the design and implementation of module breakdown
5. Support various modes of memory: clearly distinguish between long-term and short-term memory
6. Perfect the test role, and carry out necessary interactions with humans
7. Provide full mode instead of the current fast mode, allowing natural communication between roles
8. Implement SkillManager and the process of incremental Skill learning
7. Allowing natural communication between roles (expected v0.5.0)
8. Implement SkillManager and the process of incremental Skill learning (experimentation done with game agents)
9. Automatically get RPM and configure it by calling the corresponding openai page, so that each key does not need to be manually configured
10. IMPORTANT: Support incremental development (expected v0.5.0)
3. Strategies
1. Support ReAct strategy
2. Support CoT strategy
1. Support ReAct strategy (experimentation done with game agents)
2. Support CoT strategy (experimentation done with game agents)
3. Support ToT strategy
4. Support Reflection strategy
4. Support Reflection strategy (experimentation done with game agents)
5. Support planning
4. Actions
1. Implementation: Search
1. ~~Implementation: Search~~ (v0.2.1)
2. Implementation: Knowledge search, supporting 10+ data formats
3. Implementation: Data EDA
3. Implementation: Data EDA (expected v0.6.0)
4. Implementation: Review
5. Implementation: Add Document
6. Implementation: Delete Document
5. Implementation: Add Document (expected v0.5.0)
6. Implementation: Delete Document (expected v0.5.0)
7. Implementation: Self-training
8. Implementation: DebugError
8. ~~Implementation: DebugError~~ (v0.2.1)
9. Implementation: Generate reliable unit tests based on YAPI
10. Implementation: Self-evaluation
11. Implementation: AI Invocation
12. Implementation: Learning and using third-party standard libraries
13. Implementation: Data collection
14. Implementation: AI training
15. Implementation: Run code
16. Implementation: Web access
15. ~~Implementation: Run code~~ (v0.2.1)
16. ~~Implementation: Web access~~ (v0.2.1)
5. Plugins: Compatibility with plugin system
6. Tools
1. ~~Support SERPER api~~
@ -64,13 +66,13 @@ ### Tasks
1. Perfect the action pool/skill pool for each role
2. Red Book blogger
3. E-commerce seller
4. Data analyst
4. Data analyst (expected v0.6.0)
5. News observer
6. Institutional researcher
6. ~~Institutional researcher~~ (v0.2.1)
8. Evaluation
1. Support an evaluation on a game dataset
2. Reproduce papers, implement full skill acquisition for a single game role, achieving SOTA results
3. Support an evaluation on a math dataset
1. Support an evaluation on a game dataset (experimentation done with game agents)
2. Reproduce papers, implement full skill acquisition for a single game role, achieving SOTA results (experimentation done with game agents)
3. Support an evaluation on a math dataset (expected v0.6.0)
4. Reproduce papers, achieving SOTA results for current mathematical problem solving process
9. LLM
1. Support Claude underlying API

View file

@ -15,7 +15,7 @@ # 第 1 步:确保您的系统上安装了 NPM。并使用npm安装mermaid-js
sudo npm install -g @mermaid-js/mermaid-cli
# 第 2 步:确保您的系统上安装了 Python 3.9+。您可以使用以下命令进行检查:
python --version
python3 --version
# 第 3 步:克隆仓库到您的本地机器,并进行安装。
git clone https://github.com/geekan/MetaGPT.git

View file

@ -15,7 +15,7 @@ # Step 2: Run metagpt demo with container
-v /opt/metagpt/config/key.yaml:/app/metagpt/config/key.yaml \
-v /opt/metagpt/workspace:/app/metagpt/workspace \
metagpt/metagpt:latest \
python3 startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# You can also start a container and execute commands in it
docker run --name metagpt -d \
@ -25,7 +25,7 @@ # You can also start a container and execute commands in it
metagpt/metagpt:latest
docker exec -it metagpt /bin/bash
$ python3 startup.py "Write a cli snake game"
$ metagpt "Write a cli snake game"
```
The command `docker run ...` do the following things:
@ -33,7 +33,7 @@ # You can also start a container and execute commands in it
- Run in privileged mode to have permission to run the browser
- Map host configure file `/opt/metagpt/config/key.yaml` to container `/app/metagpt/config/key.yaml`
- Map host directory `/opt/metagpt/workspace` to container `/app/metagpt/workspace`
- Execute the demo command `python3 startup.py "Write a cli snake game"`
- Execute the demo command `metagpt "Write a cli snake game"`
### Build image by yourself

View file

@ -15,7 +15,7 @@ # 步骤2: 使用容器运行metagpt演示
-v /opt/metagpt/config/key.yaml:/app/metagpt/config/key.yaml \
-v /opt/metagpt/workspace:/app/metagpt/workspace \
metagpt/metagpt:latest \
python startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# 您也可以启动一个容器并在其中执行命令
docker run --name metagpt -d \
@ -25,7 +25,7 @@ # 您也可以启动一个容器并在其中执行命令
metagpt/metagpt:latest
docker exec -it metagpt /bin/bash
$ python startup.py "Write a cli snake game"
$ metagpt "Write a cli snake game"
```
`docker run ...`做了以下事情:
@ -33,7 +33,7 @@ # 您也可以启动一个容器并在其中执行命令
- 以特权模式运行,有权限运行浏览器
- 将主机文件 `/opt/metagpt/config/key.yaml` 映射到容器文件 `/app/metagpt/config/key.yaml`
- 将主机目录 `/opt/metagpt/workspace` 映射到容器目录 `/app/metagpt/workspace`
- 执行示例命令 `python startup.py "Write a cli snake game"`
- 执行示例命令 `metagpt "Write a cli snake game"`
### 自己构建镜像

View file

@ -19,11 +19,11 @@ ### Initiating a startup
```shell
# Run the script
python startup.py "Write a cli snake game"
metagpt "Write a cli snake game"
# Do not hire an engineer to implement the project
python startup.py "Write a cli snake game" --implement False
metagpt "Write a cli snake game" --no-implement
# Hire an engineer and perform code reviews
python startup.py "Write a cli snake game" --code_review True
metagpt "Write a cli snake game" --code_review
```
After running the script, you can find your new project in the `workspace/` directory.
@ -33,17 +33,17 @@ ### Preference of Platform or Tool
You can tell which platform or tool you want to use when stating your requirements.
```shell
python startup.py "Write a cli snake game based on pygame"
metagpt "Write a cli snake game based on pygame"
```
### Usage
```
NAME
startup.py - We are a software startup comprised of AI. By investing in us, you are empowering a future filled with limitless possibilities.
metagpt - We are a software startup comprised of AI. By investing in us, you are empowering a future filled with limitless possibilities.
SYNOPSIS
startup.py IDEA <flags>
metagpt IDEA <flags>
DESCRIPTION
We are a software startup comprised of AI. By investing in us, you are empowering a future filled with limitless possibilities.

View file

@ -18,9 +18,9 @@ # 复制配置文件并进行必要的修改
### 示例:启动一个创业公司
```shell
python startup.py "写一个命令行贪吃蛇"
metagpt "写一个命令行贪吃蛇"
# 开启code review模式会花费更多的金钱, 但是会提升代码质量和成功率
python startup.py "写一个命令行贪吃蛇" --code_review True
metagpt "写一个命令行贪吃蛇" --code_review
```
运行脚本后,您可以在 `workspace/` 目录中找到您的新项目。
@ -29,17 +29,17 @@ ### 平台或工具的倾向性
可以在阐述需求时说明想要使用的平台或工具。
例如:
```shell
python startup.py "写一个基于pygame的命令行贪吃蛇"
metagpt "写一个基于pygame的命令行贪吃蛇"
```
### 使用
```
名称
startup.py - 我们是一家AI软件创业公司。通过投资我们您将赋能一个充满无限可能的未来。
metagpt - 我们是一家AI软件创业公司。通过投资我们您将赋能一个充满无限可能的未来。
概要
startup.py IDEA <flags>
metagpt IDEA <flags>
描述
我们是一家AI软件创业公司。通过投资我们您将赋能一个充满无限可能的未来。

View file

@ -0,0 +1,14 @@
# -*- coding: utf-8 -*-
"""
@Time : 2023-12-12
@Author : mashenquan
@File : fix_bug.py
"""
from metagpt.actions import Action
class FixBug(Action):
"""Fix bug action without any implementation details"""
async def run(self, *args, **kwargs):
raise NotImplementedError

View file

@ -14,12 +14,14 @@
3. Encapsulate the input of RunCode into RunCodeContext and encapsulate the output of RunCode into
RunCodeResult to standardize and unify parameter passing between WriteCode, RunCode, and DebugError.
"""
import json
from tenacity import retry, stop_after_attempt, wait_random_exponential
from metagpt.actions.action import Action
from metagpt.config import CONFIG
from metagpt.const import CODE_SUMMARIES_FILE_REPO, TEST_OUTPUTS_FILE_REPO
from metagpt.const import CODE_SUMMARIES_FILE_REPO, TEST_OUTPUTS_FILE_REPO, TASK_FILE_REPO, BUGFIX_FILENAME, \
DOCS_FILE_REPO
from metagpt.logs import logger
from metagpt.schema import CodingContext, Document, RunCodeResult
from metagpt.utils.common import CodeParser
@ -54,6 +56,12 @@ ATTENTION: Use '##' to SPLIT SECTIONS, not '#'. Output format carefully referenc
{summary_log}
```
-----
# Bug Feedback logs
```text
{feedback}
```
-----
## Code: {filename} Write code with triple quoto, based on the following list and context.
1. Do your best to implement THIS ONLY ONE FILE. ONLY USE EXISTING API. IF NO API, IMPLEMENT IT.
@ -88,6 +96,7 @@ class WriteCode(Action):
return code
async def run(self, *args, **kwargs) -> CodingContext:
bug_feedback = await FileRepository.get_file(filename=BUGFIX_FILENAME, relative_path=DOCS_FILE_REPO)
coding_context = CodingContext.loads(self.context.content)
test_doc = await FileRepository.get_file(
filename="test_" + coding_context.filename + ".json", relative_path=TEST_OUTPUTS_FILE_REPO
@ -101,11 +110,13 @@ class WriteCode(Action):
if test_doc:
test_detail = RunCodeResult.loads(test_doc.content)
logs = test_detail.stderr
code_context = await self._get_codes(coding_context.task_doc)
prompt = PROMPT_TEMPLATE.format(
design=coding_context.design_doc.content,
tasks=coding_context.task_doc.content if coding_context.task_doc else "",
code=coding_context.code_doc.content if coding_context.code_doc else "",
code=code_context,
logs=logs,
feedback=bug_feedback.content if bug_feedback else "",
filename=self.context.filename,
summary_log=summary_doc.content if summary_doc else "",
)
@ -115,3 +126,21 @@ class WriteCode(Action):
coding_context.code_doc = Document(filename=coding_context.filename, root_path=CONFIG.src_workspace)
coding_context.code_doc.content = code
return coding_context
@staticmethod
async def _get_codes(task_doc) -> str:
if not task_doc:
return ""
if not task_doc.content:
task_doc.content = FileRepository.get_file(filename=task_doc.filename, relative_path=TASK_FILE_REPO)
m = json.loads(task_doc.content)
code_filenames = m.get("Task list", [])
codes = []
src_file_repo = CONFIG.git_repo.new_file_repository(relative_path=CONFIG.src_workspace)
for filename in code_filenames:
doc = await src_file_repo.get(filename=filename)
if not doc:
continue
codes.append(doc.content)
return "\n----------\n".join(codes)

View file

@ -16,7 +16,7 @@ Options:
Default: 'google'
Example:
python3 -m metagpt.actions.write_docstring startup.py --overwrite False --style=numpy
python3 -m metagpt.actions.write_docstring ./metagpt/startup.py --overwrite False --style=numpy
This script uses the 'fire' library to create a command-line interface. It generates docstrings for the given Python code using
the specified docstring style and adds them to the code.

View file

@ -17,6 +17,7 @@ from pathlib import Path
from typing import List
from metagpt.actions import Action, ActionOutput
from metagpt.actions.fix_bug import FixBug
from metagpt.actions.search_and_summarize import SearchAndSummarize
from metagpt.config import CONFIG
from metagpt.const import (
@ -24,10 +25,10 @@ from metagpt.const import (
DOCS_FILE_REPO,
PRD_PDF_FILE_REPO,
PRDS_FILE_REPO,
REQUIREMENT_FILENAME,
REQUIREMENT_FILENAME, BUGFIX_FILENAME,
)
from metagpt.logs import logger
from metagpt.schema import Document, Documents
from metagpt.schema import Document, Documents, Message, BugFixContext
from metagpt.utils.common import CodeParser
from metagpt.utils.file_repository import FileRepository
from metagpt.utils.get_template import get_template
@ -227,7 +228,6 @@ There are no unclear points.
},
}
OUTPUT_MAPPING = {
"Language": (str, ...),
"Original Requirements": (str, ...),
@ -305,15 +305,44 @@ output a properly formatted JSON, wrapped inside [CONTENT][/CONTENT] like "Old P
and only output the json inside this tag, nothing else
"""
IS_BUGFIX_PROMPT = """
{content}
___
You are a professional product manager; You need to determine whether the above content describes a requirement or provides feedback about a bug.
Respond with `YES` if it is a feedback about a bug, `NO` if it is not, and provide the reasons. Return the response in JSON format like below:
```json
{{
"is_bugfix": ..., # `YES` or `NO`
"reason": ..., # reason string
}}
```
"""
class WritePRD(Action):
def __init__(self, name="", context=None, llm=None):
super().__init__(name, context, llm)
async def run(self, with_messages, format=CONFIG.prompt_format, *args, **kwargs) -> ActionOutput:
async def run(self, with_messages, format=CONFIG.prompt_format, *args, **kwargs) -> ActionOutput | Message:
# Determine which requirement documents need to be rewritten: Use LLM to assess whether new requirements are
# related to the PRD. If they are related, rewrite the PRD.
requirement_doc = await FileRepository.get_file(filename=REQUIREMENT_FILENAME, relative_path=DOCS_FILE_REPO)
docs_file_repo = CONFIG.git_repo.new_file_repository(relative_path=DOCS_FILE_REPO)
requirement_doc = await docs_file_repo.get(filename=REQUIREMENT_FILENAME)
if await self._is_bugfix(requirement_doc.content):
await docs_file_repo.save(filename=BUGFIX_FILENAME, content=requirement_doc.content)
await docs_file_repo.save(filename=REQUIREMENT_FILENAME, content="")
bug_fix = BugFixContext(filename=BUGFIX_FILENAME)
return Message(content=bug_fix.json(), instruct_content=bug_fix,
role=self.profile,
cause_by=FixBug,
sent_from=self,
send_to="Alex", # the name of Engineer
)
else:
await docs_file_repo.delete(filename=BUGFIX_FILENAME)
prds_file_repo = CONFIG.git_repo.new_file_repository(PRDS_FILE_REPO)
prd_docs = await prds_file_repo.get_all()
change_files = Documents()
@ -405,7 +434,7 @@ class WritePRD(Action):
if not quadrant_chart:
return
pathname = (
CONFIG.git_repo.workdir / Path(COMPETITIVE_ANALYSIS_FILE_REPO) / Path(prd_doc.filename).with_suffix("")
CONFIG.git_repo.workdir / Path(COMPETITIVE_ANALYSIS_FILE_REPO) / Path(prd_doc.filename).with_suffix("")
)
if not pathname.parent.exists():
pathname.parent.mkdir(parents=True, exist_ok=True)
@ -430,3 +459,11 @@ class WritePRD(Action):
ws_name = CodeParser.parse_str(block="Project Name", text=prd)
CONFIG.project_name = ws_name
CONFIG.git_repo.rename_root(CONFIG.project_name)
async def _is_bugfix(self, content):
prompt = IS_BUGFIX_PROMPT.format(content=content)
res = await self._aask(prompt=prompt)
logger.info(f"IS_BUGFIX:{res}")
if "YES" in res:
return True
return False

View file

@ -74,6 +74,7 @@ MESSAGE_ROUTE_TO_ALL = "<all>"
MESSAGE_ROUTE_TO_NONE = "<none>"
REQUIREMENT_FILENAME = "requirement.txt"
BUGFIX_FILENAME = "bugfix.txt"
PACKAGE_REQUIREMENTS_FILENAME = "requirements.txt"
DOCS_FILE_REPO = "docs"

View file

@ -24,6 +24,7 @@ from pathlib import Path
from typing import Set
from metagpt.actions import Action, WriteCode, WriteCodeReview, WriteTasks
from metagpt.actions.fix_bug import FixBug
from metagpt.actions.summarize_code import SummarizeCode
from metagpt.config import CONFIG
from metagpt.const import (
@ -78,7 +79,7 @@ class Engineer(Role):
"""Initializes the Engineer role with given attributes."""
super().__init__(name, profile, goal, constraints)
self.use_code_review = use_code_review
self._watch([WriteTasks, SummarizeCode, WriteCode, WriteCodeReview])
self._watch([WriteTasks, SummarizeCode, WriteCode, WriteCodeReview, FixBug])
self.code_todos = []
self.summarize_todos = []
self.n_borg = n_borg
@ -191,14 +192,14 @@ class Engineer(Role):
async def _think(self) -> Action | None:
if not CONFIG.src_workspace:
CONFIG.src_workspace = CONFIG.git_repo.workdir / CONFIG.git_repo.workdir.name
write_code_filters = any_to_str_set([WriteTasks, SummarizeCode])
write_code_filters = any_to_str_set([WriteTasks, SummarizeCode, FixBug])
summarize_code_filters = any_to_str_set([WriteCode, WriteCodeReview])
if not self._rc.news:
return None
msg = self._rc.news[0]
if msg.cause_by in write_code_filters:
logger.info(f"TODO WriteCode:{msg.json()}")
await self._new_code_actions()
await self._new_code_actions(bug_fix=msg.cause_by == any_to_str(FixBug))
return self._rc.todo
if msg.cause_by in summarize_code_filters and msg.sent_from == any_to_str(self):
logger.info(f"TODO SummarizeCode:{msg.json()}")
@ -232,10 +233,10 @@ class Engineer(Role):
coding_doc = Document(root_path=str(src_file_repo.root_path), filename=filename, content=context.json())
return coding_doc
async def _new_code_actions(self):
async def _new_code_actions(self, bug_fix=False):
# Prepare file repos
src_file_repo = CONFIG.git_repo.new_file_repository(CONFIG.src_workspace)
changed_src_files = src_file_repo.changed_files
changed_src_files = src_file_repo.all_files if bug_fix else src_file_repo.changed_files
task_file_repo = CONFIG.git_repo.new_file_repository(TASK_FILE_REPO)
changed_task_files = task_file_repo.changed_files
design_file_repo = CONFIG.git_repo.new_file_repository(SYSTEM_DESIGN_FILE_REPO)

View file

@ -286,6 +286,8 @@ class Role:
cause_by=self._rc.todo,
sent_from=self,
)
elif isinstance(response, Message):
msg = response
else:
msg = Message(content=response, role=self.profile, cause_by=self._rc.todo, sent_from=self)
self._rc.memory.add(msg)

View file

@ -97,14 +97,14 @@ class Message(BaseModel):
send_to: Set = Field(default_factory={MESSAGE_ROUTE_TO_ALL})
def __init__(
self,
content,
instruct_content=None,
role="user",
cause_by="",
sent_from="",
send_to=MESSAGE_ROUTE_TO_ALL,
**kwargs,
self,
content,
instruct_content=None,
role="user",
cause_by="",
sent_from="",
send_to=MESSAGE_ROUTE_TO_ALL,
**kwargs,
):
"""
Parameters not listed below will be stored as meta info, including custom parameters.
@ -341,3 +341,7 @@ class CodeSummarizeContext(BaseModel):
def __hash__(self):
return hash((self.design_filename, self.task_filename))
class BugFixContext(BaseModel):
filename: str = ""

View file

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# @Date : 2023/7/19 16:28
# @Author : stellahong (stellahong@fuzhi.ai)
# @Author : stellahong (stellahong@deepwisdom.ai)
# @Desc :
import asyncio
import base64

View file

@ -8,15 +8,13 @@
"""
from __future__ import annotations
import os
from gitignore_parser import parse_gitignore, rule_from_pattern, handle_negation
import shutil
from enum import Enum
from pathlib import Path
from typing import Dict, List
from git.repo import Repo
from git.repo.fun import is_git_dir
from metagpt.const import DEFAULT_WORKSPACE_ROOT
from metagpt.logs import logger
from metagpt.utils.dependency_file import DependencyFile
@ -51,6 +49,7 @@ class GitRepository:
"""
self._repository = None
self._dependency = None
self._gitignore_rules = None
if local_path:
self.open(local_path=local_path, auto_init=auto_init)
@ -63,6 +62,7 @@ class GitRepository:
local_path = Path(local_path)
if self.is_git_dir(local_path):
self._repository = Repo(local_path)
self._gitignore_rules = parse_gitignore(full_path=str(local_path / ".gitignore"))
return
if not auto_init:
return
@ -82,6 +82,7 @@ class GitRepository:
writer.write("\n".join(ignores))
self._repository.index.add([".gitignore"])
self._repository.index.commit("Add .gitignore")
self._gitignore_rules = parse_gitignore(full_path=gitignore_filename)
def add_change(self, files: Dict):
"""Add or remove files from the staging area based on the provided changes.
@ -204,8 +205,9 @@ class GitRepository:
logger.info(f"Rename directory {str(self.workdir)} to {str(new_path)}")
self._repository = Repo(new_path)
def get_files(self, relative_path: Path | str, root_relative_path: Path | str = None) -> List:
"""Retrieve a list of files in the specified relative path.
def get_files(self, relative_path: Path | str, root_relative_path: Path | str = None, filter_ignored=True) -> List:
"""
Retrieve a list of files in the specified relative path.
The method returns a list of file paths relative to the current FileRepository.
@ -213,6 +215,8 @@ class GitRepository:
:type relative_path: Path or str
:param root_relative_path: The root relative path within the repository.
:type root_relative_path: Path or str
:param filter_ignored: Flag to indicate whether to filter files based on .gitignore rules.
:type filter_ignored: bool
:return: A list of file paths in the specified directory.
:rtype: List[str]
"""
@ -231,10 +235,35 @@ class GitRepository:
rpath = file_path.relative_to(root_relative_path)
files.append(str(rpath))
else:
subfolder_files = self.get_files(relative_path=file_path, root_relative_path=root_relative_path)
subfolder_files = self.get_files(relative_path=file_path, root_relative_path=root_relative_path,
filter_ignored=False)
files.extend(subfolder_files)
except Exception as e:
logger.error(f"Error: {e}")
if not filter_ignored:
return files
filtered_files = self.filter_gitignore(filenames=files, root_relative_path=root_relative_path)
return filtered_files
def filter_gitignore(self, filenames: List[str], root_relative_path: Path | str = None) -> List[str]:
"""
Filter a list of filenames based on .gitignore rules.
:param filenames: A list of filenames to be filtered.
:type filenames: List[str]
:param root_relative_path: The root relative path within the repository.
:type root_relative_path: Path or str
:return: A list of filenames that pass the .gitignore filtering.
:rtype: List[str]
"""
if root_relative_path is None:
root_relative_path = self.workdir
files = []
for filename in filenames:
pathname = root_relative_path / filename
if self._gitignore_rules(str(pathname)):
continue
files.append(filename)
return files
@ -244,6 +273,7 @@ if __name__ == "__main__":
repo = GitRepository()
repo.open(path, auto_init=True)
repo.filter_gitignore(filenames=["snake_game/snake_game/__pycache__", "snake_game/snake_game/game.py"])
changes = repo.changed_files
print(changes)

View file

@ -16,6 +16,7 @@ TOKEN_COSTS = {
"gpt-3.5-turbo-0613": {"prompt": 0.0015, "completion": 0.002},
"gpt-3.5-turbo-16k": {"prompt": 0.003, "completion": 0.004},
"gpt-3.5-turbo-16k-0613": {"prompt": 0.003, "completion": 0.004},
"gpt-3.5-turbo-1106": {"prompt": 0.001, "completion": 0.002},
"gpt-4-0314": {"prompt": 0.03, "completion": 0.06},
"gpt-4": {"prompt": 0.03, "completion": 0.06},
"gpt-4-32k": {"prompt": 0.06, "completion": 0.12},
@ -33,6 +34,7 @@ TOKEN_MAX = {
"gpt-3.5-turbo-0613": 4096,
"gpt-3.5-turbo-16k": 16384,
"gpt-3.5-turbo-16k-0613": 16384,
"gpt-3.5-turbo-1106": 16384,
"gpt-4-0314": 8192,
"gpt-4": 8192,
"gpt-4-32k": 32768,
@ -54,6 +56,7 @@ def count_message_tokens(messages, model="gpt-3.5-turbo-0613"):
if model in {
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k-0613",
"gpt-3.5-turbo-1106",
"gpt-4-0314",
"gpt-4-32k-0314",
"gpt-4-0613",

View file

@ -53,3 +53,4 @@ aiofiles==23.2.1
gitpython==3.1.40
zhipuai==1.0.7
socksio~=1.0.0
gitignore-parser==0.1.9

View file

@ -30,13 +30,13 @@ with open(path.join(here, "requirements.txt"), encoding="utf-8") as f:
setup(
name="metagpt",
version="0.3.0",
description="The Multi-Agent Framework",
version="0.4.0",
description="The Multi-Role Meta Programming Framework",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/geekan/MetaGPT",
author="Alexander Wu",
author_email="alexanderwu@fuzhi.ai",
author_email="alexanderwu@deepwisdom.ai",
license="MIT",
keywords="metagpt multi-role multi-agent programming gpt llm metaprogramming",
packages=find_packages(exclude=["contrib", "docs", "examples", "tests*"]),

View file

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# @Date : 2023/7/22 02:40
# @Author : stellahong (stellahong@fuzhi.ai)
# @Author : stellahong (stellahong@deepwisdom.ai)
#
from tests.metagpt.roles.ui_role import UIDesign

View file

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# @Date : 2023/7/22 02:40
# @Author : stellahong (stellahong@fuzhi.ai)
# @Author : stellahong (stellahong@deepwisdom.ai)
#
from metagpt.roles import ProductManager
from metagpt.team import Team

View file

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# @Date : 2023/7/15 16:40
# @Author : stellahong (stellahong@fuzhi.ai)
# @Author : stellahong (stellahong@deepwisdom.ai)
# @Desc :
import os
import re

View file

@ -16,6 +16,7 @@ runner = CliRunner()
@pytest.mark.asyncio
async def test_team():
# FIXME: we're now using "metagpt" cli, so the entrance should be replaced instead.
company = Team()
company.run_project("做一个基础搜索引擎,可以支持知识库")
history = await company.run(n_round=5)

View file

@ -1,6 +1,6 @@
# -*- coding: utf-8 -*-
# @Date : 2023/7/22 02:40
# @Author : stellahong (stellahong@fuzhi.ai)
# @Author : stellahong (stellahong@deepwisdom.ai)
#
import os

View file

@ -7,8 +7,8 @@ from metagpt.tools import WebBrowserEngineType, web_browser_engine
@pytest.mark.parametrize(
"browser_type, url, urls",
[
(WebBrowserEngineType.PLAYWRIGHT, "https://fuzhi.ai", ("https://fuzhi.ai",)),
(WebBrowserEngineType.SELENIUM, "https://fuzhi.ai", ("https://fuzhi.ai",)),
(WebBrowserEngineType.PLAYWRIGHT, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
(WebBrowserEngineType.SELENIUM, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
],
ids=["playwright", "selenium"],
)

View file

@ -8,9 +8,9 @@ from metagpt.tools import web_browser_engine_playwright
@pytest.mark.parametrize(
"browser_type, use_proxy, kwagrs, url, urls",
[
("chromium", {"proxy": True}, {}, "https://fuzhi.ai", ("https://fuzhi.ai",)),
("firefox", {}, {"ignore_https_errors": True}, "https://fuzhi.ai", ("https://fuzhi.ai",)),
("webkit", {}, {"ignore_https_errors": True}, "https://fuzhi.ai", ("https://fuzhi.ai",)),
("chromium", {"proxy": True}, {}, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
("firefox", {}, {"ignore_https_errors": True}, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
("webkit", {}, {"ignore_https_errors": True}, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
],
ids=["chromium-normal", "firefox-normal", "webkit-normal"],
)

View file

@ -8,9 +8,9 @@ from metagpt.tools import web_browser_engine_selenium
@pytest.mark.parametrize(
"browser_type, use_proxy, url, urls",
[
("chrome", True, "https://fuzhi.ai", ("https://fuzhi.ai",)),
("firefox", False, "https://fuzhi.ai", ("https://fuzhi.ai",)),
("edge", False, "https://fuzhi.ai", ("https://fuzhi.ai",)),
("chrome", True, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
("firefox", False, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
("edge", False, "https://deepwisdom.ai", ("https://deepwisdom.ai",)),
],
ids=["chrome-normal", "firefox-normal", "edge-normal"],
)

View file

@ -73,6 +73,13 @@ async def test_git1():
repo1 = GitRepository(local_path=local_path, auto_init=False)
assert repo1.changed_files
file_repo = repo1.new_file_repository("__pycache__")
await file_repo.save("a.pyc", content="")
all_files = repo1.get_files(relative_path=".", filter_ignored=False)
assert "__pycache__/a.pyc" in all_files
all_files = repo1.get_files(relative_path=".", filter_ignored=True)
assert "__pycache__/a.pyc" not in all_files
repo1.delete_repository()
assert not local_path.exists()