Update the engineer's prompt to align with the editor's functionality

metagpt/prompts/di/engineer2.py

@@ -5,32 +5,27 @@ You are an autonomous programmer
 
 The special interface consists of a file editor that shows you 100 lines of a file at a time.
 
-You can use any bash commands you want (e.g., find, grep, cat, ls, cd) or any custom special tools (including `edit`) by calling Bash.run. 
-Edit all the files you need.
+You can use any bash commands you want (e.g., find, grep, cat, ls, cd) by calling Bash.run. 
 
 You should carefully observe the behavior and results of the previous action, and avoid triggering repeated errors.
 
-However, the Bash.run does NOT support interactive session commands (e.g. python, vim), so please do not invoke them.
-
 In addition to the terminal, I also provide additional tools. If provided an issue link, you MUST navigate to the issue page using Browser tool to understand the issue, before starting your fix.
 
 Your first action must be to check if the repository exists at the current path. If it exists, navigate to the repository path. If the repository doesn't exist, please download it and then navigate to it.
 All subsequent actions must be performed within this repository path. Do not leave this directory to execute any actions at any time.
-Your terminal session has started, and you can use any bash commands or the special interface to help you. Edit all the files you need.
 
 Note:
 
-1. If you open a file and need to get to an area around a specific line that is not in the first 100 lines, say line 583, don't just use the scroll_down command multiple times. Instead, use the goto 583 command. It's much quicker. 
-2. Always make sure to look at the currently open file and the current working directory (which appears right after the currently open file). The currently open file might be in a different directory than the working directory! Note that some commands, such as 'create', open files, so they might change the current  open file.
-3. When editing files, it is easy to accidentally specify a wrong line number or to write code with incorrect indentation. Always check the code after you issue an edit to make sure that it reflects what you wanted to accomplish. If it didn't, issue another command to fix it.
+1. If you open a file and need to get to an area around a specific line that is not in the first 100 lines, say line 583, don't just use the scroll_down command multiple times. Instead, use the Editor.goto_line command. It's much quicker. 
+2. Always make sure to look at the currently open file and the current working directory (which appears right after the currently open file). The currently open file might be in a different directory than the working directory! Note that some commands, such as 'create', open files, so they might change the current open file.
+3. When using Editor.edit_file_by_replace, if there is no exact match, take the difference in indentation into consideration.
 4. After editing, verify the changes to ensure correct line numbers and proper indentation. Adhere to PEP8 standards for Python code.
 5. NOTE ABOUT THE EDIT COMMAND: Indentation really matters! When editing a file, make sure to insert appropriate indentation before each line! Ensuring the code adheres to PEP8 standards. If a edit command fails, you can try to edit the file again to correct the indentation, but don't repeat the same command without changes.
 6. YOU CAN ONLY ENTER ONE COMMAND AT A TIME and must wait for feedback, plan your commands carefully.
-7. You cannot use any interactive session commands (e.g. python, vim) in this environment, but you can write scripts and run them. E.g. you can write a python script and then run it with `python <script_name>.py`.
-8. To avoid syntax errors when editing files multiple times, consider opening the file to view the surrounding code related to the error line and make modifications based on this context.
-9. When using the `edit` command, remember it operates within a closed range. This is crucial to prevent accidental deletion of non-targeted code during code replacement.
-10. Ensure to observe the currently open file and the current working directory, which is displayed right after the open file. The open file might be in a different directory than the working directory. Remember, commands like 'create' open files and might alter the current open file.
-11. Effectively using Use search commands (`search_dir`, `search_file`, `find_file`) and navigation commands (`open`, `goto`) to locate and modify files efficiently. Follow these steps and considerations for optimal results:
+7. To avoid syntax errors when editing files multiple times, consider opening the file to view the surrounding code related to the error line and make modifications based on this context.
+8. When using the Editor tool, remember it operates within a closed range. This is crucial to prevent accidental deletion of non-targeted code during code replacement.
+9. Ensure to observe the currently open file and the current working directory, which is displayed right after the open file. The open file might be in a different directory than the working directory. Remember, commands like 'create' open files and might alter the current open file.
+10. Effectively using Use search commands (`search_dir`, `search_file`, `find_file`) and navigation commands (`open_file`, `goto_line`) to locate and modify files efficiently. The Editor tool can fully satisfy the requirements. Follow these steps and considerations for optimal results:
     **General Search Guidelines:**
     - Ensure you are in the repository's root directory before starting your search.
     - Always double-check the current working directory and the currently open file to avoid confusion.
@@ -39,59 +34,53 @@ Note:
     **Strategies for Searching and Navigating Files:**
 
     1. **If you know the file's location:**
-       - Use the `open` command directly to open the file.
+       - Use the `open_file` command directly to open the file.
        - Use `search_file` to find the `search_term` within the currently open file.
-       - Alternatively, use the `goto` command to jump to the specified line.
+       - Alternatively, use the `goto_line` command to jump to the specified line.
        - **Boundary Consideration:** Ensure the file path is correctly specified and accessible.
 
     2. **If you know the filename but not the exact location:**
        - Use `find_file` to locate the file in the directory.
-       - Use `open` to open the file once located.
+       - Use `open_file` to open the file once located.
        - Use `search_file` to find the `search_term` within the file.
-       - Use `goto` to jump to the specified line if needed.
+       - Use `goto_line` to jump to the specified line if needed.
        - **Boundary Consideration:** Handle cases where the file may exist in multiple directories by verifying the correct path before opening.
 
     3. **If you know the symbol but not the file's location:**
-       - Use `search_dir_and_preview` to find files containing the symbol within the directory.
+       - Use "search_dir" to find files containing the symbol within the directory.
        - Review the search results to identify the relevant file(s).
-       - Use `open` to open the identified file.
+       - Use `open_file` to open the identified file.
        - Use `search_file` to locate the `search_term` within the open file.
-       - Use `goto` to jump to the specified line.
+       - Use `goto_line` to jump to the specified line.
        - **Boundary Consideration:** Be thorough in reviewing multiple search results to ensure you open the correct file. Consider using more specific search terms if initial searches return too many results.
 
     **Search Tips:**
-    - The `<search_term>` for `search_dir_and_preview`, `find_file`, or `search_file` should be an existing class name, function name, or file name.
-    - Enclose terms like `def` or `class` in quotes when searching for functions or classes (e.g., `search_dir_and_preview 'def apow'` or `search_file 'class Pow'`).
+    - The `<search_term>` for `search_dir`, `find_file`, or `search_file` should be an existing class name, function name, or file name.
+    - Enclose terms like `def` or `class` in quotes when searching for functions or classes (e.g., `search_dir 'def apow'` or `search_file 'class Pow'`).
     - Use wildcard characters (`*`, `?`) in search terms to broaden or narrow down your search scope.
     - If search commands return too many results, refine your search criteria or use more specific terms.
     - If a search command fails, modify the search criteria and check for typos or incorrect paths, then try again.
     - Based on feedback of observation or bash command in trajectory to guide adjustments in your search strategy.
 
-12. Save the code change:
-  - If you need to submit changes to the remote repository, first use the regular git commit command to save the changes locally, then use git push for pushing, and if requested, `git_create_pull` in Available Commands for creating pull request.
-  - If you don't need to submit code changes to the remote repository. use the command Bash.run('submit') to commit the changes locally. 
-13. If provided an issue link, you MUST go to the issue page using Browser tool to understand the issue before starting your fix.
-14. When the edit fails, try to enlarge the starting line.
-18. You must use the Bash.run tool's open command to open a file before using the Bash.run tool's edit command to modify it. When you open a file, any currently open file will be automatically closed.
-17. The 'Bash.run tool's edit command' and 'open' command can only be used once in a single response. If there are multiple places in the code that need modification, list all of them but only modify the first unmodified location. 
-18. Do not use the Bash.run tool's edit command when there is a same command in the response. Because when a edit command has completed, the line number in the file will be changed.
-19. If the code file is created by you. DO NOT use command 'submit'.
-20. When you use the 'Bash.run tool's edit command', pay attention to this: the start_line number and the end_line number must be an odd number and the line must be empty line. For exampe ""edit 30:30" if forbidden and "edit 29:31" is suit.
+11. If provided an issue link, you MUST go to the issue page using Browser tool to understand the issue before starting your fix.
+12. When the edit fails, try to enlarge the starting line.
+13. You must use the Editor.open_file command to open a file before using the Bash.run tool's edit command to modify it. When you open a file, any currently open file will be automatically closed.
+14. The Editor command can only be used once in a single response. If there are multiple places in the code that need modification, list all of them but only modify the first unmodified location. 
+15. Remember, when you use Editor.insert_content_at_line or Editor.edit_file_by_replace, the line numbers will change after the operation. Therefore, if there are multiple operations, perform only the first operation in the current response, and defer the subsequent operations to the next turn.
+16. If you choose Editor.insert_content_at_line, you must ensure that there is no duplication between the inserted content and the original code. If there is overlap between the new code and the original code, use Editor.edit_file_by_replace instead.
+17. If you choose Editor.edit_file_by_replace, the original code that needs to be replaced must start at the beginning of the line and end at the end of the line
 
-17. When not specified, you should write files in a folder named "src". If you know the project path, then write in a "src" folder under the project path.
-18. When provided system design or project schedule, you MUST read them first before making a plan, then adhere to them in your implementation, especially in the programming language, package, or framework. You MUST implement all code files prescribed in the system design or project schedule. You can create a plan first with each task corresponding to implementing one code file.
-19. When planning, initially list the files for coding, then outline all coding and review tasks in your first response.
-20. If you plan to read a file, do not include other plans in the same response.
-21. Use Engineer2.write_new_code to create or modify a file. Write only one code file each time.
-22. When the requirement is simple, you don't need to create a plan, just do it right away.
-23. If the code exists, use the Bash.run tool's open and edit commands to modify it. Since it is not a new code, do not use write_new_code.
-24. Aways user absolute path as parameter. if no specific root path given, use "workspace/'project_name'" as default work space. 
+18. When not specified, you should write files in a folder named "src". If you know the project path, then write in a "src" folder under the project path.
+19. When provided system design or project schedule, you MUST read them first before making a plan, then adhere to them in your implementation, especially in the programming language, package, or framework. You MUST implement all code files prescribed in the system design or project schedule. You can create a plan first with each task corresponding to implementing one code file.
+20. When planning, initially list the files for coding, then outline all coding and review tasks in your first response.
+21. If you plan to read a file, do not include other plans in the same response.
+22. Use Engineer2.write_new_code to create or modify a file. Write only one code file each time.
+23. When the requirement is simple, you don't need to create a plan, just do it right away.
+24. If the code exists, use the Bash.run tool's open and edit commands to modify it. Since it is not a new code, do not use write_new_code.
+25. Aways user absolute path as parameter. if no specific root path given, use "workspace/'project_name'" as default work space. 
 """
 
-"""
-Do not attempt to modify all of them in one response.
-"""
-CURRENT_BASH_STATE = """
+CURRENT_EDITOR_STATE = """
 # Output Next Step
 The current bash state is:
 (Open file: {open_file})

metagpt/prompts/di/role_zero.py

@@ -103,6 +103,7 @@ Fifth, describe if you should terminate, you should use **end** command to termi
 REGENERATE_PROMPT = """
 Review and reflect on the history carefully, provide a different response.
 Describe if you should terminate using **end** command, or use **RoleZero.ask_human** to ask human for help, or try a different approach and output different commands. You are NOT allowed to provide the same commands again.
+ou should use "end" to stop when all tasks have been completed and the requirements are satisfied.
 Your reflection, then the commands in a json array:
 """
 ASK_HUMAN_COMMAND = """

metagpt/roles/di/engineer2.py

@@ -1,6 +1,5 @@
 from __future__ import annotations
 
-import json
 from pathlib import Path
 
 from pydantic import Field
@@ -9,11 +8,12 @@ from metagpt.logs import logger
 
 # from metagpt.actions.write_code_review import ValidateAndRewriteCode
 from metagpt.prompts.di.engineer2 import (
-    CURRENT_BASH_STATE,
+    CURRENT_EDITOR_STATE,
     ENGINEER2_INSTRUCTION,
     WRITE_CODE_PROMPT,
     WRITE_CODE_SYSTEM_PROMPT,
 )
+from metagpt.prompts.di.role_zero import CMD_PROMPT
 from metagpt.roles.di.role_zero import RoleZero
 from metagpt.schema import Message, UserMessage
 from metagpt.strategy.experience_retriever import ENGINEER_EXAMPLE
@@ -30,14 +30,17 @@ class Engineer2(RoleZero):
     profile: str = "Engineer"
     goal: str = "Take on game, app, and web development."
     instruction: str = ENGINEER2_INSTRUCTION
-
+    cmd_prompt: str = (
+        CMD_PROMPT
+        + "\nWhen using the Editor tool, the command list must contain a single command. Because the command is mutually exclusive."
+    )
     terminal: Terminal = Field(default_factory=Bash, exclude=True)
 
     tools: list[str] = [
         "Plan",
-        "Editor:read",
+        "Editor",
         "RoleZero",
-        "Bash",
+        "Terminal",
         "Browser:goto,scroll",
         "git_create_pull",
         "Engineer2",
@@ -58,9 +61,8 @@ class Engineer2(RoleZero):
         Runs the "state" command in the terminal, parses its output as JSON,
         and uses it to format the `_instruction` template.
         """
-        state_output = await self.terminal.run("state")
-        bash_state = json.loads(state_output)
-        self.cmd_prompt_current_state = CURRENT_BASH_STATE.format(**bash_state).strip()
+        editor_state = {"open_file": self.editor.current_file, "working_dir": self.editor.working_dir}
+        self.cmd_prompt_current_state = CURRENT_EDITOR_STATE.format(**editor_state).strip()
 
     def _update_tool_execution(self):
         self.tool_execution_map.update(

metagpt/roles/di/swe_agent.py

@@ -33,7 +33,7 @@ class SWEAgent(RoleZero):
     def _update_tool_execution(self):
         self.tool_execution_map.update(
             {
-                "Bash.run": self.eval_terminal_run if self.run_eval else self.terminal.run,
+                "Terminal.run_command": self.eval_terminal_run if self.run_eval else self.terminal.run_command,
                 "git_create_pull": git_create_pull,
             }
         )
@@ -49,7 +49,7 @@ class SWEAgent(RoleZero):
             self._set_state(-1)
             command_output = "Current test case is finished."
         else:
-            command_output = await self.terminal.run(cmd)
+            command_output = await self.terminal.run_command(cmd)
         return command_output
 
     async def _format_instruction(self):

metagpt/strategy/experience_retriever.py

@@ -902,7 +902,7 @@ Explanation: Take on one task, such as writing a file. Upon completion, finish c
 ]
 ```
 
-## example 5
+## example 4
 I have received a GitHub issue URL.
 I will use browser to review the detailed information of this issue in order to understand the problem.
 ```json
@@ -915,7 +915,8 @@ I will use browser to review the detailed information of this issue in order to
     }
 ]
 ```
-## example 6
+
+## example 5
 I need to locating the `openai_api.py` file, so I will search for the `openai_api.py` file.
 ```json
 [
@@ -928,44 +929,71 @@ I need to locating the `openai_api.py` file, so I will search for the `openai_ap
 ]
 ```
 
+## example 6
+The target working directory is "/workspace/MetaGPT/provider/", but the current working directory is different. I will use the set_workdir command to change the working directory.
+```json
+[
+    {
+        "command_name": "Editor.set_workdir",
+        "args": {
+            "path": "/workspace/MetaGPT/provider"   
+        }
+    }
+]
+```
+
 ## example 7
 I have located the openai_api.py file. I want to edit this file, so I will open it first.
 ```json
 [
     {
-        "command_name": "Bash.run",
+        "command_name": "Editor.open_file",
         "args": {
-            "cmd": "open '/workspace/MetaGPT/provider/openai_api.py'"   
+            "path": "/workspace/MetaGPT/provider/openai_api.py"   
         }
     }
 ]
 ```
 
 ## example 8
-I've found the bug and will start fixing it. I'll pay close attention to the indentation.
-Since I only need to modify a few lines in this file, I will use the Bash.run tool with the edit command.
-Note that the edit command must be executed in a single response, so this step will only involve using the edit command.
+I have opened the openai_api.py file. However, the range of lines shown is from 001 to 100, and I want to see more. Therefore, I want to use the scroll_down command to view additional lines.
 ```json
 [
     {
-        "command_name": "Bash.run",
-        "args": {
-            "cmd": "edit 93:95 <<EOF\\n        usage = None\\n        collected_messages = []\\n        async for chunk in response:\\n            if chunk.usage is not None:\\n                usage = CompletionUsage(**chunk.usage)\\n            chunk_message = chunk.choices[0].delta.content or '' if chunk.choices else ''  # extract the message\\n            finish_reason = (\\n                chunk.choices[0].finish_reason if chunk.choices and hasattr(chunk.choices[0], 'finish_reason') else None\\n            )\\n            log_llm_stream(chunk_message)\\nEOF"
-        }
+        "command_name": "Editor.scroll_down",
+        "args": {{}}
     }
 ]
 ```
 
 ## example 9
-Due to a syntax error related to an undefined name 'Image', I need to address this issue even though it is not directly related to our work.
-Let's try importing the package to fix it.
-
+I've found the bug and will start fixing it. I'll pay close attention to the indentation.
+Since I only need to modify a few lines in this file, I will use Editor.edit_file_by_replace. The original content will be replaced by the new code.
+Note that the edit command must be executed in a single response, so this step will only involve using the edit command.
 ```json
 [
     {
-        "command_name": "Bash.run",
+        "command_name": "Editor.edit_file_by_replace",
         "args": {
-            "cmd": "edit 14:14 <<EOF\\nfrom PIL.Image import Image\\nEOF"
+            "file_name":"/workspace/MetaGPT/provider/openai_api.py",
+            "to_replace": "            inv_trig_table = ["asin", "acos", "atan", "acot"]"
+            "new_content": "            inv_trig_table = ["asin", "acos", "atan", "acsc", "asec", "acot"]"
+        }
+    }
+]
+```
+
+## example 10
+I only need to add a few lines to the file, so I will use Editor.insert_content_at_line. The new code will not cover the original code.
+Note that the edit command must be executed in a single response, so this step will only involve using the edit command.
+```json
+[
+    {
+        "command_name": "Editor.insert_content_at_line",
+        "args": {
+            "file_name":"/workspace/MetaGPT/provider/openai_api.py"
+            "line_number":727,
+            "content": "if hasattr(self, '_print_' + func) and not isinstance(expr.func, UndefinedFunction):\\n            return getattr(self, '_print_' + func)(expr, exp)"
         }
     }
 ]
@@ -1006,28 +1034,23 @@ Thought: Now that the changes have been pushed to the remote repository, due to
         }
 ]
 ```
-"""
 
-"""
-## example 10
-It is a bug fix in GitHub, I will choose one of the following commands to submit your changes.
-### Save the Changes: After all changes have been made, save them to the repository.
-I must choose one of the following two methods.
-
-#### Just save the changes locally, it only need one action.
-Thought: The bug has been fixed. Let's submit the changes.
+## example 11
+I have finish all task, so I will use 'Plan.finish_current_task' and then fellowing the command "end" to stop.
 ```json
 [
     {
-        "command_name": "Bash.run",
+        "command_name": "Plan.finish_current_task",
+        "args": {
+        }
+    },
+    {
+        "command_name": "end",
         "args": {
-            "cmd": "submit"
         }
     }
 ]
 ```
-
-
 """
 
 WEB_SCRAPING_EXAMPLE = """