Improved the prompt for correcting llm output to be more strict

HarrySu123 · HarrySu123 · commit 1c3fa128dc02 · 2025-07-17T14:16:21.000+01:00
diff --git a/conversion2025/mathpix_to_llm_to_in2lambda_to_JSON.ipynb b/conversion2025/mathpix_to_llm_to_in2lambda_to_JSON.ipynb
@@ -308,7 +308,7 @@
     "# Uses gpt-4o-mini:\n",
     "#    - more intelligent\n",
     "llm_mini = ChatOpenAI(\n",
-    "            model=\"gpt-4o-mini\",\n",
+    "            model=\"gpt-4.1-mini\",\n",
     "            api_key=os.environ[\"OPENAI_API_KEY\"],\n",
     "        )"
    ]
@@ -407,8 +407,9 @@
     "        -   Do NOT include any explanations, comments, or markdown code blocks (like ```json).\n",
     "\n",
     "    3.  **JSON Formatting Rules:**\n",
-    "        -   **Escape Backslashes:** All LaTeX backslashes (`\\`) MUST be escaped as double backslashes (`\\\\`). For example, `\\cup` must become `\\\\cup`. This is the most important rule.\n",
-    "        -   **Newlines:** Use `\\n` for newlines within the JSON string values.\n",
+    "        -   **Backslash Escaping (Very Important):**\n",
+    "            -   For **LaTeX commands**, a single backslash `\\` MUST be escaped as a double backslash `\\\\`. For example, `\\frac` must become `\\\\frac`.\n",
+    "            -   For **newlines**, you MUST use a single `\\n`. Do NOT escape it as `\\\\n`.\n",
     "        -   **Content Integrity:** Preserve all text, LaTeX (`$...$`, `$$...$$`), and image tags perfectly. Do not alter or summarize content.\n",
     "        -   **Strict Schema:** Ensure the final JSON has no trailing commas and includes all fields, even if they are empty.\n",
     "    \"\"\"\n",
@@ -545,22 +546,22 @@
     "\n",
     "    1.  **Content Splitting:**\n",
     "        -   From the input `question_content`, identify the main introductory text (the stem) and place it in the `content` field.\n",
-    "        -   Identify all sub-questions (e.g., \"(a)\", \"(b)\", \"i.\", \"ii.\") and place their text into the `parts` list.\n",
-    "        -   Parts may also be implied.\n",
-    "        -   All Question Must have at least one part.\n",
+    "        -   Identify all sub-questions (e.g., \"(a)\", \"(b)\", \"i.\", \"ii.\") and place their text into the `parts` list. Sub-questions may also be implied.\n",
+    "        -   Questions with no sub-questions should have a single part in the `parts` list, which is the entire question text.\n",
     "        -   Ensure that images references are correctly placed with their respective parts.\n",
     "        -   Preserve all content perfectly, including text, LaTeX, and image tags like `![pictureTag](filename.jpg)`.\n",
     "        -   Ensure no solution content is included in the `content` or `parts` fields.\n",
-    "        -   The `title` should be a concise summary of the question.\n",
+    "        -  You may choose what the title of the question should be.\n",
     "        -   The `images` list should be copied exactly from the input.\n",
     "\n",
     "    2.  **Output Format (Crucial):**\n",
     "        -   You MUST output ONLY a single, raw, valid JSON string.\n",
     "        -   Do NOT include any explanations, comments, or markdown code blocks (like ```json).\n",
     "\n",
     "    3.  **JSON Formatting Rules:**\n",
-    "        -   **Escape Backslashes:** All LaTeX backslashes (`\\`) MUST be escaped as double backslashes (`\\\\`). For example, `\\cup` must become `\\\\cup`. This is the most important rule.\n",
-    "        -   **Newlines:** Use `\\n` for newlines within the JSON string values.\n",
+    "        -   **Backslash Escaping (Very Important):**\n",
+    "            -   For **LaTeX commands**, a single backslash `\\` MUST be escaped as a double backslash `\\\\`. For example, `\\frac` must become `\\\\frac`.\n",
+    "            -   For **newlines**, you MUST use a single `\\n`. Do NOT escape it as `\\\\n`.\n",
     "        -   **Content Integrity:** Preserve all text, LaTeX (`$...$`, `$$...$$`), and image tags (`![pictureTag](...)`) perfectly. Do not alter or summarize content.\n",
     "    \"\"\"\n",
     "\n",
@@ -581,7 +582,8 @@
     "        -   Do NOT include any explanations, comments, or markdown code blocks (like ```json).\n",
     "\n",
     "    3.  **JSON Formatting Rules:**\n",
-    "        -   **Escape Backslashes:** All LaTeX backslashes (`\\`) MUST be escaped as double backslashes (`\\\\`). For example, `\\cup` must become `\\\\cup` in the JSON string. This is the most important rule.\n",
+    "        -   **Backslash Escaping (Very Important):**\n",
+    "            -   For **LaTeX commands**, a single backslash `\\` MUST be escaped as a double backslash `\\\\`. For example, `\\frac` must become `\\\\frac` in the JSON string. This is the most important rule.\n",
     "        -   **Newlines:** Use `\\n` for newlines within the JSON string values.\n",
     "        -   **Math Delimiters:** Ensure all math delimiters (`$...$` and `$$...$$`) are correctly balanced and preserved.\n",
     "    \"\"\"\n",
@@ -713,16 +715,23 @@
     "    content: str = Field(..., description=\"The main content of the question\")\n",
     "\n",
     "llm_task_text_check = r\"\"\"\n",
-    "    Your task is to validate and correct the content within the `part_text` field of the provided JSON input.\n",
+    "    Your task is to validate and correct the content of the provided JSON fields to ensure it is clean, well-formatted, and valid Texdown (Markdown with LaTeX).\n",
     "    You MUST return ONLY a single, raw, valid JSON string that strictly follows the original schema. Do NOT add any explanations, comments, or markdown code blocks.\n",
     "\n",
     "    Apply these correction rules to the content inside the JSON fields:\n",
-    "    1.  **JSON Escaping:** All LaTeX backslashes (`\\`) MUST be escaped as double backslashes (`\\\\`). For example, `\\cup` must be written as `\\\\cup`. Never escape backslashes for newlines (`\\n`), as they should remain as is.\n",
-    "    2.  **Math Delimiters:** All mathematical content must be enclosed in `$...$` for inline math or `$$...$$` for display math. Ensure all delimiters are correctly balanced and closed. '$' and '$$' should not be used for any other purpose. Move all `\\n` outside the math delimiters.\n",
-    "    3.  **Display Math:** `$$` delimiters must be on their own separate lines.\n",
-    "    4.  **Image Tags:** Preserve image tags like `![pictureTag](filename.jpg)` exactly as they are.\n",
-    "    5.  **Content Integrity:** Do not change, paraphrase, or summarize any text, formulas, or image links. Only fix formatting errors according to these rules.\n",
-    "    6.  **Newlines:** Use `\\n` for newlines within the JSON string values.\n",
+    "    1.  **JSON Escaping:** All LaTeX backslashes (`\\`) MUST be escaped as double backslashes (`\\\\`). For example, `\\frac` must become `\\\\frac`. Do not escape backslashes for newlines (`\\n`).\n",
+    "    2.  **Enforce Math Delimiters:** This is the most important rule. Any text containing LaTeX commands (e.g., `\\lim`, `\\frac`, `\\sin`, `\\alpha`, `^`, `_`) or mathematical structures that is NOT already enclosed in `$..$` or `$$..$$` MUST be wrapped.\n",
+    "        -   Use `$$...$$` for standalone equations and `$...$` for inline math.\n",
+    "        -   Be careful to not wrap text that is already correctly formatted with LaTeX math delimiters.\n",
+    "    3.  **Display Math Formatting:** This rule is critical. Display math blocks MUST be formatted strictly as follows: a blank line, the opening `$$` on its own line, the LaTeX content, the closing `$$` on its own line, and a blank line.\n",
+    "        -   **Incorrect:** `...text $$x=y$$ more text...`\n",
+    "        -   **Incorrect:** `...text\\n$$\\nx=y\\n\\n$$\\nmore text...`\n",
+    "        -   **Correct:** `...text\\n\\n$$\\nx=y\\n$$\\n\\nmore text...`\n",
+    "    4.  **LaTeX Environments:** Environments like `aligned`, `cases`, `matrix`, `gathered`, etc., must be entirely contained within a single display math block (`$$...$$`). Ensure that every `\\begin{...}` has a matching `\\end{...}`.\n",
+    "    7.  **Markdown Lists:** Ensure that markdown lists (e.g., using `*`, `-`, or `1.`) are correctly formatted with proper indentation and spacing.\n",
+    "    8.  **Spacing and Readability:** Ensure there is a single blank line between paragraphs, lists, and other block elements to improve readability. Remove any excessive blank lines.\n",
+    "    9.  **Cleanup Redundancy:** Correct or remove any repetitive or nonsensical phrases that may be artifacts from OCR (e.g., \"is monotone. is monotone\" should be corrected to \"is monotone\").\n",
+    "    10. **Content Integrity:** Do not change, paraphrase, or summarize any text, formulas, or image links. Only fix formatting, spacing, and structural errors according to these rules.\n",
     "    \"\"\"\n",
     "\n",
     "def validate_part_text(part_text_data):\n",
@@ -736,7 +745,7 @@
     "        }\n",
     "        \n",
     "        validation_prompt = f\"\"\"\n",
-    "            Your task is to extract a JSON with the following structure exactly:\n",
+    "            Your task is to extract a JSON with the following structure exactly, to be parsed by a pydantic model:\n",
     "            {part_text_parser.get_format_instructions()}\n",
     "\n",
     "            Your task is to validate and correct the content within the `part_text` field of the provided JSON input.\n",
@@ -777,7 +786,7 @@
     "        }\n",
     "        \n",
     "        validation_prompt = f\"\"\"\n",
-    "            Your task is to extract a JSON with the following structure exactly:\n",
+    "            Your task is to extract a JSON with the following structure exactly, to be parsed by a pydantic model:\n",
     "            {part_solution_parser.get_format_instructions()}\n",
     "\n",
     "            Your task is to validate and correct the content within the `part_solution` field of the provided JSON input.\n",
@@ -819,7 +828,7 @@
     "        }\n",
     "        \n",
     "        validation_prompt = f\"\"\"\n",
-    "            Your task is to extract a JSON with the following structure exactly:\n",
+    "            Your task is to extract a JSON with the following structure exactly, to be parsed by a pydantic model:\n",
     "            {content_parser.get_format_instructions()}\n",
     "\n",
     "            Your task is to validate and correct the content within the `title` and `content` fields of the provided JSON input.\n",
@@ -937,12 +946,14 @@
     "    print(json.dumps(extracted_dict, indent=2))\n",
     "    print(\"Now validating the content...\")\n",
     "\n",
-    "    # content_validated_dict = content_texdown_check(extracted_dict)\n",
-    "    # print(\"successfully validated the content.\")\n",
-    "    # print(json.dumps(content_validated_dict, indent=2))\n",
-    "    # print(\"successfully converted markdown to JSON.\")\n",
-    "    \n",
-    "    return extracted_dict"
+    "    # return extracted_dict\n",
+    "\n",
+    "    content_validated_dict = content_texdown_check(extracted_dict)\n",
+    "    print(\"successfully validated the content.\")\n",
+    "    print(json.dumps(content_validated_dict, indent=2))\n",
+    "    print(\"successfully converted markdown to JSON.\")\n",
+    "\n",
+    "    return content_validated_dict"
    ]
   },
   {
