ReActAgent.run aborts the investigation on the first zero-result tool return at iteration >= 2 — single transient web failure or one narrow query produces an ungrounded answer

[Fr3ya]

What happened?

ReActAgent.run at packages/leann-core/src/leann/react_agent.py#L286-L298 terminates the entire investigation the first time any tool returns 0 results on iteration 2 or later, and falls through to "ask the LLM for its best answer based on what it knows" — i.e. an ungrounded answer with no source citation. The guard does not require N consecutive zeros and does not distinguish transient failures (web search 502, rate-limit, network blip) from legitimately empty results.

# react_agent.py:286-298 — buggy early-exit
if results_count == 0 and iteration >= 2:
    logger.warning("No results found, asking LLM for final answer...")
    final_prompt = f"""Based on the previous searches, provide your best answer to the question.

Question: {question}

Previous searches and results:
{chr(10).join(all_context)}

Since no new results were found, provide your final answer based on what you know.
"""
    final_answer = self.llm.ask(final_prompt)
    return final_answer.strip()

results_count == 0 is set in three different places, all of which trigger this premature exit:

• legitimately empty local results from self.search(...) at line 264 (a too-narrow first query that the model would naturally rephrase on the next iteration);
• Serper API structured error at lines 233-239 (results_count = 0 after the error string is folded into the observation);
• empty Serper response at lines 240-242 (results_count = 0 for no web results found).

All three are normal intermediate states in a multi-turn ReAct loop. The agent's whole purpose is to route between leann_search, web_search, and visit_page based on prior observations — "no results" is a signal to switch tool / rephrase / broaden, not to abandon the investigation.

The bug is shaped by the same family as the classic "cycle exit guard fires too eagerly" anti-pattern, but here the guard fires too eagerly outward (abandons the loop) rather than too eagerly inward (never exits).

How to reproduce

from dataclasses import dataclass, field


@dataclass
class FakeLLM:
    scripted_responses: list[str]
    final_answer: str = "I do not know — no grounded source available."
    _call_index: int = 0

    def ask(self, prompt: str) -> str:
        if self._call_index >= len(self.scripted_responses):
            return f"Final Answer: {self.final_answer}"
        resp = self.scripted_responses[self._call_index]
        self._call_index += 1
        return resp


@dataclass
class SearchToolStub:
    scripted_result_counts: list[int]
    _call_index: int = 0

    def search(self, query):
        if self._call_index >= len(self.scripted_result_counts):
            return 0
        n = self.scripted_result_counts[self._call_index]
        self._call_index += 1
        return n


def _parse(response):
    if "Final Answer:" in response:
        return None, response.split("Final Answer:")[1].strip()
    if "Action:" in response:
        return response.split("Action:")[1].strip(), None
    return None, response.strip()


def react_agent_run(*, question, llm, tool, max_iterations=5):
    """Faithful port of react_agent.py:182-311."""
    previous_observations, all_context = [], []
    for iteration in range(1, max_iterations + 1):
        prompt = f"Q: {question} / iter: {iteration} / prev: {chr(10).join(previous_observations) or 'none'}"
        response = llm.ask(prompt)
        action, final_answer = _parse(response)
        if action is None and final_answer is not None:
            return final_answer

        results_count = tool.search(action or "")
        previous_observations.append(f"<{results_count} results>" if results_count else "No results found.")
        all_context.append(f"Action: {action}")

        # react_agent.py:286-298 — THE BUG
        if results_count == 0 and iteration >= 2:
            return llm.ask(f"Based on previous searches, best answer to {question!r} (no new results)").strip()

    return llm.ask(f"Based on all searches, final answer to {question!r}").strip()


# Scenario A: narrow query miss
# Iter 1: leann_search → 3 hits (partial)
# Iter 2: refined leann_search → 0 hits  ← BUG TRIGGERS HERE
# Iter 3: would have switched to web_search and found 5 hits (NEVER REACHED)
llm_a = FakeLLM(scripted_responses=[
    'Action: leann_search("LEANN benchmark")',
    'Action: leann_search("LEANN recall numbers vs faiss")',
    'Action: web_search("LEANN paper recall comparison FAISS")',
    'Final Answer: LEANN achieves recall@10 = 0.95 vs FAISS 0.97 per the paper.',
])
ans_a = react_agent_run(
    question="What is LEANN's recall vs FAISS at recall@10?",
    llm=llm_a,
    tool=SearchToolStub([3, 0, 5]),
)
print(f"Scenario A: {ans_a!r}")
assert "0.95" not in ans_a, "If iter 3 had been reached, the grounded recall would be present"


# Scenario B: transient web_search failure
# Iter 1: leann_search → 2 hits
# Iter 2: web_search → 0 (transient 502 / empty Serper)  ← BUG TRIGGERS HERE
# Iter 3: would have retried web_search with different query and got 4 hits (NEVER REACHED)
llm_b = FakeLLM(scripted_responses=[
    'Action: leann_search("project setup")',
    'Action: web_search("latest setup guide")',
    'Action: web_search("setup guide tutorial")',
    'Final Answer: per the latest docs, run `make install`.',
])
ans_b = react_agent_run(
    question="How do I set up the project?",
    llm=llm_b,
    tool=SearchToolStub([2, 0, 4]),
)
print(f"Scenario B: {ans_b!r}")
assert "make install" not in ans_b, "If iter 3 had retried, the grounded setup would be present"

Save as repro.py, run python repro.py. Both scenarios return the LLM's mid-reasoning text (or the configured final_answer fallback) instead of the grounded answer the agent's own plan would have produced on iteration 3.

Error message

No error is raised. The bug is silent — the run finishes "successfully" with an ungrounded answer, and the only signal is the `logger.warning("No results found, asking LLM for final answer...")`

LEANN Version

Reproduced against main branch at commit 55ff236dc4b5 (pushed 2026-06-22). The bug is in the ReActAgent class which is invoked via the CLI leann ask command (resolved via create_react_agent at cli.py#L3624-L3631).

Operating System

Linux