From bd9cb75f1adfa847cc6110182b951435d3162be4 Mon Sep 17 00:00:00 2001 From: Yizhou Chi Date: Mon, 12 Aug 2024 14:35:30 +0800 Subject: [PATCH] optimized intention recognition for ood --- metagpt/prompts/di/role_zero.py | 106 ++------------------------------ 1 file changed, 5 insertions(+), 101 deletions(-) diff --git a/metagpt/prompts/di/role_zero.py b/metagpt/prompts/di/role_zero.py index eeb86d43e..2aea566ae 100644 --- a/metagpt/prompts/di/role_zero.py +++ b/metagpt/prompts/di/role_zero.py @@ -110,69 +110,13 @@ If no issues are detected, the original json data should be returned unchanged. Output the JSON data in a format that can be loaded by the json.loads() function. """ -# QUICK_THINK_PROMPT = """ -# Decide if the latest user message previously is a quick question. -# Quick questions include common-sense, legal, logical, math, multiple-choice questions, greetings, or casual chat that you can answer directly. -# Questions about you or your team info are also quick questions. -# Software development tasks are NOT quick questions. Code execution, however trivial, is NOT a quick question. -# However, these programming-related tasks are quick questions: writing trivial code snippets (fewer than 30 lines), filling a single function or class, explaining concepts, writing tutorials and documentation. - -# If the question is a quick question, you should output QUICK to indicate the question is a quick question. -# Time- or location-sensitive questions such as wheather or news inquiry are NOT quick questions. Moreover, you should output a keyword SEARCH to indicate the need for a google search. -# If the query is ambiguous or requires more information, you should output OOD (Out of Domain) to indicate the question requires further clarification. - -# Respond with a concise thought then a QUICK if the question is a quick question, otherwise, a SEARCH, a TASK, or an OOD. Your response: -# """ - QUICK_THINK_PROMPT = """ -Decide if the latest user message previously is a quick question. -Quick questions include common-sense, legal, logical, math, multiple-choice questions, greetings, or casual chat that you can answer directly. -Questions about you or your team info are also quick questions. -Software development tasks are NOT quick questions. Code execution, however trivial, is NOT a quick question. -However, these programming-related tasks are quick questions: writing trivial code snippets (fewer than 30 lines), filling a single function or class, explaining concepts, writing tutorials and documentation. - -## QUICK -If the question is a quick question, you should output QUICK to indicate the question is a quick question. -## SEARCH -If the question is a time- or location-sensitive such as wheather or news inquiry, you should output a keyword SEARCH to indicate the need for a google search. -## TASK -If the question is a software development task, or requires multiple steps of planning an execution, you should output a keyword TASK. -## OOD -If the question is ambiguous or requires more context such as link, file path, or the task cannot be done without more user's assistance, you should output OOD (Out of Domain). - -Respond with a concise thought then a QUICK if the question is a quick question, otherwise, a SEARCH, a TASK, or an OOD. Your response: - -""" - -QUICK_THINK_PROMPT = """ -Response Categories: - -## QUICK -For straightforward questions or requests that can be answered directly. Quick questions include common-sense, legal, logical, math, short-coding, multiple-choice questions, greetings, or casual chat that you can answer directly. Questions about you or your team info are also quick questions. -## SEARCH -For queries that require up-to-date or detailed information retrieval. These include time- or location-sensitive questions, such as weather or news inquiries. However, no need to perform a search if the information is readily available. -## TASK -For complex, multi-step tasks that involve a series of actions or detailed instructions. -## AMBIGUOUS -For requests that are ambiguous, lack necessary information, or fall outside the system's capabilities. AMBIGUOUS requests have these common properties: -- Incomplete Information: Requests that mention tasks but lack critical details (e.g., no document provided for summarization). -- Vagueness: Requests that are too broad, unclear, or unspecified, making it difficult to respond effectively. -- Out of Expertise: Requests that ask for specialized advice (e.g., legal, medical) or highly technical tasks outside the model's design. -- Unrealistic Scope: The request is too extensive or unrealistic to address within a single response (e.g., “Tell me everything about…”). - - -Respond with a concise thought, then provide the appropriate response category: QUICK, SEARCH, TASK, or AMBIGUOUS. Response: - -""" - - -QUICK_THINK_PROMPT = """ -# Response Categories: +# Response Categories ## QUICK: For straightforward questions or requests that can be answered directly. This includes common-sense inquiries, legal or logical questions, basic math, short coding tasks, multiple-choice questions, greetings, casual chat, and inquiries about you or your team. ## SEARCH -For queries that require retrieving up-to-date or detailed information. This includes time-sensitive or location-specific questions like current events or weather. Use this only if the information isn’t readily available. +For queries that require retrieving up-to-date or detailed information. This includes time-sensitive or location-specific questions like current events or weather. Use this only if the information isn't readily available. ## TASK For complex requests that involve multiple steps or detailed instructions. Examples include software development, project planning, or any task that requires a sequence of actions. @@ -180,57 +124,18 @@ For complex requests that involve multiple steps or detailed instructions. Examp ## AMBIGUOUS For requests that are unclear, lack sufficient detail, or are outside the system's capabilities. Common characteristics of AMBIGUOUS requests: -- Incomplete Information: Lacking critical details needed to perform the task (e.g., fail to provide dependent files, links, or context for a task). +- Incomplete Information: Requests that imply complex tasks but lack critical details (e.g., "Redesign this logo" without providing the original logo or specifying design requirements). - Vagueness: Broad, unspecified, or unclear requests that make it difficult to provide a precise answer. - Out of Expertise: Requests for specialized advice (e.g., medical or legal advice) or highly technical tasks beyond the model's scope. - Unrealistic Scope: Overly broad requests that are impossible to address meaningfully in a single response (e.g., "Tell me everything about..."). +**Note:** Before categorizing a request as TASK, consider whether the user has provided sufficient information to proceed with the task. If the request is complex but lacks essential details, it may fall under AMBIGUOUS. + {examples} Respond with a concise thought, then provide the appropriate response category: QUICK, SEARCH, TASK, or AMBIGUOUS. Your response: """ -# QUICK_THINK_EXAMPLES =""" -# # Example - -# 1. Given the request: "How to design an online document editing platform that supports real-time collaboration? Please answer me directly.", We can get the response: (This requires an direct answer) should be answered with YES. -# 2. Given the request: "Help me find some of the latest research papers on deep learning.", We can get the response: (This is a time-sensitive question) should be answered with SEARCH. -# 3. Given the request: "Tell me the difference between supervised learning and unsupervised learning in machine learning.", We can get the response: (This is a general knowledge question) should be answered with YES. -# 4. Given the request: "Recommend some programming practice websites suitable for beginners.", We can get the response: (This is a general knowledge question) should be answered with YES. -# 5. Given the request: "Make a personal website that runs Game of Life.", We can get the response: (This is a software development task) should be answered with NO. -# 6. Given the request: "Summarize the document for me.", We can get the response: (Nothing is provided by the user, requires further information) should be answered with OOD. - -# # Instruction -# """ - -# QUICK_THINK_EXAMPLES =""" -# # Example - -# 1. Request: "How to design an online document editing platform that supports real-time collaboration? Please answer me directly.", Response: (This requires an direct answer) should be answered with QUICK. -# 2. Request: "Help me find some of the latest research papers on deep learning.", Response: (This is a time-sensitive question) should be answered with SEARCH. -# 3. Request: "Tell me the difference between supervised learning and unsupervised learning in machine learning.", Response: (This is a general knowledge question) should be answered with QUICK. -# 4. Request: "Recommend some programming practice websites suitable for beginners.", Response: (This is a general knowledge question) should be answered with QUICK. -# 5. Request: "Make a personal website that runs Game of Life.", Response: (This is a software development task) should be answered with TASK. -# 6. Request: "Summarize the document for me.", Response: (The user needs to provide a link or file path to the document) should be answered with OOD. -# 7. Request: "Optimize our process.", Response: (Clarification needed: Which specific process? What does "optimize" mean in this context?) should be answered with OOD. - -# # Instruction -# """ - - -QUICK_THINK_EXAMPLES =""" -# Example - -1. Request: "How to design an online document editing platform that supports real-time collaboration? Please answer me directly.", Response: The user is asking for a general approach to design a platform, should be answered with QUICK. -2. Request: "Help me find some of the latest research papers on deep learning.", Response: The user is asking for the latest research papers, which is a time-sensitive question, should be answered with SEARCH. -3. Request: "Tell me the difference between supervised learning and unsupervised learning in machine learning.", Response: The user is asking for a general knowledge question, should be answered with QUICK. -4. Request: "Help me develop a one week healthy eating plan.", Response: The user is asking for advice on developing a healthy eating plan. The plan can be provided directly, should be answered with QUICK. -5. Request: "Make a personal website that runs Game of Life.", Response: The user is asking for a software development task with multiple steps, should be answered with TASK. -6. Request: "Summarize the document for me.", Response: The user doesn't provide a link or file path to the document, should be answered with OOD. -7. Request: "Optimize our process.", Response: Optimizing a process is a vague request, and the user needs to clarify what process it is and what is meant by 'optimize', should be answered with OOD. - -# Instruction -""" QUICK_THINK_EXAMPLES =""" # Example @@ -270,5 +175,4 @@ Response Category: AMBIGUOUS. # Instruction """ -# QUICK_THINK_PROMPT = QUICK_THINK_EXAMPLES + QUICK_THINK_PROMPT QUICK_THINK_PROMPT = QUICK_THINK_PROMPT.format(examples=QUICK_THINK_EXAMPLES) \ No newline at end of file