Merge branch 'fix_duplicate' into 'mgx_ops'

DA update: custom tool, write_and_exec_code docstring, search example

See merge request pub/MetaGPT!262

metagpt/prompts/di/data_analyst.py

@@ -5,6 +5,8 @@ EXTRA_INSTRUCTION = """
     - When no click action is required, no need to use the Browser tool to navigate to the webpage before scraping.
     - Write code to view the HTML content rather than using the Browser tool.
     - Make sure the command_name are certainly in Available Commands when you use the Browser tool.
+    - For information searching requirement, you should use the Browser tool instead of web scraping.
+    - When no link is provided, you should use the Browser tool to search for the information.
 7. When you are making plan. It is highly recommend to plan and append all the tasks in first response once time.
 8. Don't finish_current_task multiple times for the same task. 
 9. Finish current task timely, such as when the code is written and executed successfully.

metagpt/prompts/di/role_zero.py

@@ -76,7 +76,7 @@ Fifth, describe if you should terminate, you should use **end** command to termi
  - You have completed the overall user requirement
  - All tasks are finished and current task is empty
  - You are repetitively replying to human
-Finally, combine your thoughts, describe what you want to do conscisely in 20 words, then follow your thoughts to list the commands, adhering closely to the instructions provided.
+Finally, combine your thoughts, describe what you want to do conscisely in 20 words, including whether you will end, then follow your thoughts to list the commands, adhering closely to the instructions provided.
 """.strip()
 REGENERATE_PROMPT = """
 Review and reflect on the history carefully, provide a different response.

metagpt/prompts/task_type.py

@@ -58,4 +58,5 @@ The current task is about converting image into webpage code. please note the fo
 WEB_SCRAPING_PROMPT = """
 - Remember to view and print the necessary HTML content in a separate task to understand the structure first before scraping data. Such as `html_content = await view_page_element_to_scrape(...)\nprint(html_content)`.
 - Since the data required by user may not correspond directly to the actual HTML element names, you should thoroughly analyze the HTML structure and meanings of all elements in your context first. Ensure the `class_` in your code should derived from the actual HTML structure directly, not based on your knowledge. To ensure it, analyse the most suitable location of the 'class_' in the actual HTML content before code.
+- Reuse existing html object variable from previous code (if any) to extract data, do not mock or hard code a html variable yourself.
 """

metagpt/roles/di/data_analyst.py

@@ -38,7 +38,7 @@ class DataAnalyst(RoleZero):
     @model_validator(mode="after")
     def set_custom_tool(self):
         if self.custom_tools and not self.custom_tool_recommender:
-            self.custom_tool_recommender = BM25ToolRecommender(tools=self.custom_tools)
+            self.custom_tool_recommender = BM25ToolRecommender(tools=self.custom_tools, force=True)
 
     def _update_tool_execution(self):
         self.tool_execution_map.update(
@@ -48,7 +48,10 @@ class DataAnalyst(RoleZero):
         )
 
     async def write_and_exec_code(self):
-        """Write a code block for current task step and execute it in an interactive notebook environment."""
+        """Write a code block for current task and execute it in an interactive notebook environment. No argument is needed."""
+        if self.planner.plan:
+            logger.info(f"Current task {self.planner.plan.current_task}")
+
         counter = 0
         success = False
         await self.execute_code.init_code()

metagpt/strategy/experience_retriever.py

@@ -663,7 +663,7 @@ class KeywordExpRetriever(ExpRetriever):
                 return DEPLOY_EXAMPLE
             elif "issue" in context.lower():
                 return FIX_ISSUE_EXAMPLE
-            elif "https:" in context.lower() or "http:" in context.lower():
+            elif "https:" in context.lower() or "http:" in context.lower() or "search" in context.lower():
                 if "search" in context.lower() or "click" in context.lower():
                     return WEB_SCRAPING_EXAMPLE
                 return WEB_SCRAPING_EXAMPLE_SIMPLE