Merge branch 'fixbug_custom_development_process' into 'mgx_ops'

优化了1.所有任务交由工程师完成的问题；解决了1. teamleader 未察觉团队成员完成任务的问题。2. 团队成员完成任务时直接终止而未确认完成任务的问题。

See merge request pub/MetaGPT!266

metagpt/actions/analyze_requirements.py

@@ -4,6 +4,8 @@ ANALYZE_REQUIREMENTS = """
 # Example
 {examples}
 
+----------------
+
 # Requirements
 {requirements}
 
@@ -18,14 +20,6 @@ Follow the instructions and output format. Do not include any additional content
 
 EXAMPLES = """
 Example 1
-Requirements:
-Create 2048 game using Python. Do not write PRD.
-Outputs:
-[User Restrictions] : Do not write PRD.
-[Language Restrictions] : The response, message and instruction must be in the language specified by English.
-[Programming Language] : Python
-
-Example 2
 Requirements: 
 创建一个贪吃蛇，只需要给出设计文档和代码
 Outputs:
@@ -33,6 +27,14 @@ Outputs:
 [Language Restrictions] : The response, message and instruction must be in the language specified by Chinese.
 [Programming Language] : HTML (*.html), CSS (*.css), and JavaScript (*.js)
 
+Example 2
+Requirements:
+Create 2048 game using Python. Do not write PRD.
+Outputs:
+[User Restrictions] : Do not write PRD.
+[Language Restrictions] : The response, message and instruction must be in the language specified by English.
+[Programming Language] : Python
+
 Example 3
 Requirements:
 You must ignore create PRD and TRD. Help me write a schedule display program for the Paris Olympics. 
@@ -44,7 +46,7 @@ Outputs:
 
 INSTRUCTIONS = """
 You must output in the same language as the Requirements.
-First, This language should be consistent with the language used in the requirement description. determine the natural language you must respond in. The default language is English
+First, This language should be consistent with the language used in the requirement description. determine the natural language you must respond in. If the requirements specify a special language, follow those instructions. The default language for responses is English.
 Second, extract the restrictions in the requirements, specifically the steps. Do not include detailed demand descriptions; focus only on the restrictions.
 Third, if the requirements is a software development, extract the program language. If If no specific programming language is required, Use HTML (*.html), CSS (*.css), and JavaScript (*.js)
 

metagpt/prompts/di/data_analyst.py

@@ -10,6 +10,7 @@ EXTRA_INSTRUCTION = """
 7. When you are making plan. It is highly recommend to plan and append all the tasks in first response once time.
 8. Don't finish_current_task multiple times for the same task. 
 9. Finish current task timely, such as when the code is written and executed successfully.
+10. When using the command 'end', add the command 'finish_current_task' before it.
 """
 
 TASK_TYPE_DESC = "\n".join([f"- **{tt.type_name}**: {tt.value.desc}" for tt in TaskType])

metagpt/prompts/di/role_zero.py

@@ -56,6 +56,7 @@ In your response, include at least one command.
 Some text indicating your thoughts before JSON is required, such as what tasks have been completed, what tasks are next, how you should update the plan status, respond to inquiry, or seek for help. Then a json array of commands. You must output ONE and ONLY ONE json array. DON'T output multiple json arrays with thoughts between them.
 Output should adhere to the following format.
 {thought_guidance}
+Finally, combine your thoughts, describe what you want to do conscisely in 20 words, including which process you will taked and whether you will end, then follow your thoughts to list the commands, adhering closely to the instructions provided.
 ```json
 [
     {{
@@ -76,8 +77,8 @@ Fifth, describe if you should terminate, you should use **end** command to termi
  - You have completed the overall user requirement
  - All tasks are finished and current task is empty
  - You are repetitively replying to human
-Finally, combine your thoughts, describe what you want to do conscisely in 20 words, including whether you will end, then follow your thoughts to list the commands, adhering closely to the instructions provided.
 """.strip()
+
 REGENERATE_PROMPT = """
 Review and reflect on the history carefully, provide a different response.
 Describe if you should terminate using **end** command, or use **RoleZero.ask_human** to ask human for help, or try a different approach and output different commands. You are NOT allowed to provide the same commands again.

metagpt/prompts/di/team_leader.py

@@ -1,3 +1,5 @@
+from metagpt.prompts.di.role_zero import THOUGHT_GUIDANCE
+
 SYSTEM_PROMPT = """
 You are a team leader, and you are responsible for drafting tasks and routing tasks to your team members.
 When drafting and routing tasks, ALWAYS include necessary or important info inside the instruction, such as path, link, environment to team members, because you are their sole info source.
@@ -27,9 +29,15 @@ Note:
 9. Do not use the 'end' command when the current task remains unfinished; instead, use the 'finish_current_task' command to indicate completion before switching to the next task.
 10. Do not use escape characters in json data, particularly within file paths.
 11. Analyze the capabilities of team members and assign tasks to them based on user Requirements. If the requirements ask to ignore certain tasks, follow the requirements.
-12. If the requirement is to develop software and does not specify a programming language, use the default web technologies: HTML (`*.html`), CSS (`*.css`), and JavaScript (`*.js`). You must add this to the requirements.
+12. Add default web technologies: HTML (*.html), CSS (*.css), and JavaScript (*.js) to your requirements.If no specific programming language is required, include these technologies in the project requirements. Using instruction  to forward this information to your team members.
 """
-
+TL_THOUGHT_GUIDANCE = (
+    THOUGHT_GUIDANCE
+    + """
+Sixth, when planning, describe the requirements as they pertain to software development, data analysis, or other areas. If the requirements is a software development and no specific restrictions are mentioned, you must create a Product Requirements Document (PRD), write a System Design document, develop a project schedule, and then begin coding. List the steps you will undertake. Plan these steps in a single response.
+Seventh, describe the technologies you must use.
+"""
+)
 QUICK_THINK_SYSTEM_PROMPT = """
 {role_info}
 Your team member:

metagpt/roles/architect.py

@@ -33,7 +33,7 @@ class Architect(RoleZero):
 
     name: str = "Bob"
     profile: str = "Architect"
-    goal: str = "design a concise, usable, complete software system."
+    goal: str = "design a concise, usable, complete software system. ouput the system design or software framework."
     constraints: str = (
         "make sure the architecture is simple enough and use  appropriate open source "
         "libraries. Use same language as user requirement"

metagpt/roles/di/data_analyst.py

@@ -3,7 +3,7 @@ from __future__ import annotations
 from pydantic import Field, model_validator
 
 from metagpt.actions.di.execute_nb_code import ExecuteNbCode
-from metagpt.actions.di.write_analysis_code import WriteAnalysisCode, CheckData
+from metagpt.actions.di.write_analysis_code import CheckData, WriteAnalysisCode
 from metagpt.logs import logger
 from metagpt.prompts.di.data_analyst import (
     CODE_STATUS,
@@ -111,15 +111,11 @@ class DataAnalyst(RoleZero):
         return output
 
     async def _check_data(self):
-        if (
-            not self.planner.plan.get_finished_tasks()
-            or self.planner.plan.current_task.task_type
-            not in [
-                TaskType.DATA_PREPROCESS.type_name,
-                TaskType.FEATURE_ENGINEERING.type_name,
-                TaskType.MODEL_TRAIN.type_name,
-            ]
-        ):
+        if not self.planner.plan.get_finished_tasks() or self.planner.plan.current_task.task_type not in [
+            TaskType.DATA_PREPROCESS.type_name,
+            TaskType.FEATURE_ENGINEERING.type_name,
+            TaskType.MODEL_TRAIN.type_name,
+        ]:
             return
         logger.info("Check updated data")
         code = await CheckData().run(self.planner.plan)
@@ -130,3 +126,15 @@ class DataAnalyst(RoleZero):
             print(result)
             data_info = DATA_INFO.format(info=result)
             self.rc.working_memory.add(Message(content=data_info, role="user", cause_by=CheckData))
+
+    async def _run_special_command(self, cmd) -> str:
+        """command requiring special check or parsing."""
+        # TODO: duplicate with Engineer2._run_special_command, consider dedup
+
+        # finish current task before end.
+        command_output = ""
+        if cmd["command_name"] == "end" and not self.planner.plan.is_plan_finished():
+            self.planner.plan.finish_all_tasks()
+            command_output += "All tasks are finished.\n"
+        command_output += await super()._run_special_command(cmd)
+        return command_output

metagpt/roles/di/engineer2.py

@@ -9,7 +9,7 @@ from metagpt.strategy.experience_retriever import ENGINEER_EXAMPLE
 class Engineer2(RoleZero):
     name: str = "Alex"
     profile: str = "Engineer"
-    goal: str = "Take on game, app, and web development"
+    goal: str = "Take on game, app, and web development."
     instruction: str = ENGINEER2_INSTRUCTION
 
     tools: list[str] = ["Plan", "Editor:write,read", "RoleZero", "ReviewAndRewriteCode"]
@@ -26,3 +26,13 @@ class Engineer2(RoleZero):
 
     def _retrieve_experience(self) -> str:
         return ENGINEER_EXAMPLE
+
+    async def _run_special_command(self, cmd) -> str:
+        """command requiring special check or parsing."""
+        # finish current task before end.
+        command_output = ""
+        if cmd["command_name"] == "end" and not self.planner.plan.is_plan_finished():
+            self.planner.plan.finish_all_tasks()
+            command_output += "All tasks are finished.\n"
+        command_output += await super()._run_special_command(cmd)
+        return command_output

metagpt/roles/di/role_zero.py

@@ -9,7 +9,7 @@ from typing import Callable, Dict, List, Literal, Tuple
 from pydantic import model_validator
 
 from metagpt.actions import Action, UserRequirement
-from metagpt.actions.anaylze_requirements import AnalyzeRequirementsRestrictions
+from metagpt.actions.analyze_requirements import AnalyzeRequirementsRestrictions
 from metagpt.actions.di.run_command import RunCommand
 from metagpt.exp_pool import exp_cache
 from metagpt.exp_pool.context_builders import RoleZeroContextBuilder
@@ -47,6 +47,7 @@ class RoleZero(Role):
     goal: str = ""
     system_msg: list[str] = None  # Use None to conform to the default value at llm.aask
     cmd_prompt: str = CMD_PROMPT
+    thought_guidance: str = THOUGHT_GUIDANCE
     instruction: str = ROLE_INSTRUCTION
     task_type_desc: str = None
 
@@ -161,7 +162,7 @@ class RoleZero(Role):
             plan_status=plan_status,
             current_task=current_task,
             instruction=instruction,
-            thought_guidance=THOUGHT_GUIDANCE,
+            thought_guidance=self.thought_guidance,
             latest_observation=memory[-1].content,
             requirements_constraints=self.requirements_constraints,
         )
@@ -214,7 +215,7 @@ class RoleZero(Role):
         self.rc.memory.add(UserMessage(content=outputs))
 
         return AIMessage(
-            content=f"{self.name} has finished the task, mark it as finished. Complete run with outputs: {outputs}",
+            content=f"I have finished the task, please mark my task as finished. Outputs: {outputs}",
             sent_from=self.name,
             cause_by=RunCommand,
         )

metagpt/roles/di/team_leader.py

@@ -6,6 +6,7 @@ from metagpt.prompts.di.team_leader import (
     QUICK_THINK_SYSTEM_PROMPT,
     SYSTEM_PROMPT,
     TL_INSTRUCTION,
+    TL_THOUGHT_GUIDANCE,
 )
 from metagpt.roles.di.role_zero import RoleZero
 from metagpt.schema import AIMessage, Message, UserMessage
@@ -19,7 +20,7 @@ class TeamLeader(RoleZero):
     profile: str = "Team Leader"
     goal: str = "Manage a team to assist users"
     system_msg: list[str] = [SYSTEM_PROMPT]
-
+    thought_guidance: str = TL_THOUGHT_GUIDANCE
     # TeamLeader only reacts once each time, but may encounter errors or need to ask human, thus allowing 2 more turns
     max_react_loop: int = 3
 

metagpt/schema.py

@@ -657,6 +657,11 @@ class Plan(BaseModel):
             self.current_task.is_finished = True
             self._update_current_task()  # set to next task
 
+    def finish_all_tasks(self):
+        "Finish all tasks."
+        while self.current_task:
+            self.finish_current_task()
+
     def is_plan_finished(self) -> bool:
         """Check if all tasks are finished"""
         return all(task.is_finished for task in self.tasks)
@@ -669,14 +674,16 @@ class Plan(BaseModel):
         """
         return [task for task in self.tasks if task.is_finished]
 
-    def append_task(self, task_id: str, dependent_task_ids: list[str], instruction: str, assignee: str, task_type: str = ""):
+    def append_task(
+        self, task_id: str, dependent_task_ids: list[str], instruction: str, assignee: str, task_type: str = ""
+    ):
         """Append a new task with task_id (number) to the end of existing task sequences. If dependent_task_ids is not empty, the task will depend on the tasks with the ids in the list. Note that the assignee should be the 'name' of the role."""
         new_task = Task(
             task_id=task_id,
             dependent_task_ids=dependent_task_ids,
             instruction=instruction,
             assignee=assignee,
-            task_type=task_type
+            task_type=task_type,
         )
         return self._append_task(new_task)
 

metagpt/strategy/experience_retriever.py

@@ -479,7 +479,7 @@ Explanation: The requirement is about software development. Assign each tasks to
         "args": {
             "task_id": "1",
             "dependent_task_ids": [],
-            "instruction": "Create a product requirement document (PRD) outlining the features, user interface, and user experience of the CLI python snake game.Using Python as the programming language.",
+            "instruction": "Create a product requirement document (PRD) outlining the features, user interface, and user experience of the CLI python snake game. Using Python as the programming language.",
             "assignee": "Alice"
         }
     },