Conversation
Reasoning models (magistral) return the assistant content as a list of chunks, a thinking chunk followed by the text of the answer. The response extractor set that list straight on message.content, which OpenTelemetry rejects as an attribute value, so the output message lost both the reasoning and the answer. A string is emitted as before. A list of chunks now goes out as message.contents blocks: each text inside a thinking chunk as a reasoning block, and each text chunk as a text block, the shape the other instrumentors use for reasoning.
feiiiiii5
left a comment
There was a problem hiding this comment.
Ran this locally before commenting. Environment: mistralai + respx + OTel SDK in a fresh venv, packages taken from this PR's head tree via PYTHONPATH, -p no:randomly.
What I verified works
tests/.../test_reasoning.py→ 2 passed. The magistral list-content shape maps tomessage.contents.0(type=reasoning) andmessage.contents.1(type=text), and the plain-string guard test confirmsmessage.contentis still used and nomessage.contents.*appears for a string reply.- Emitting content blocks rather than a raw list is the right target shape:
message.contentsis already what the OpenAI request extractor (_request_attributes_extractor.py:131) and the LangChain tracer (_tracer.py:885,889) use, so this brings the mistralai output side in line instead of inventing a new layout. _flatten_content_chunksyields one block per inner thinking text, which matches how a multi-chunk thinking payload arrives.- No regression on this machine. The whole
tests/directory for this package gives the identical 7 pre-existing failures at the base commit (5da0a966) and at this head — same node IDs:test_entrypoint_for_opentelemetry_instrumentplus the six*_streaming_*cases, which look like cassette/network artifacts in my environment rather than code failures — while the pass count goes 18 → 20. So the two new tests add coverage without breaking anything. (Both runs took ~7.5 min, same command,-p no:randomly.)
Question about scope, because the streamed path looks unaffected
The change is in _response_attributes_extractor.py, i.e. the non-streamed response object. For client.chat.stream(...), attributes come from _ChatCompletionAccumulator, and there:
_response_accumulator.py:68declares the message field ascontent=_StringAccumulator();- the merge at
_response_accumulator.py:147-149iselif isinstance(self_value, _StringAccumulator): if isinstance(value, str): self_value += value, so a list-valuedcontentdelta is not appended and not replaced — it is dropped; __iter__at:131-133only yields the key whenstr(value)is non-empty, so the accumulated message ends up with nocontentat all, and this extractor change never sees a list on that path.
So if magistral streams the same chunked content shape it returns in the non-streamed response, a streamed call would still export neither the reasoning nor the answer text. I did not verify what magistral actually puts in delta.content on a live stream, hence the question rather than a claim:
- Does magistral stream chunked
content, or only plain strings per event? - If it streams chunks, should this PR also make the accumulator list-aware (mirroring how
tool_callsuses_IndexedAccumulator), or is the stream case deliberately left to a follow-up? A line in the PR description would be enough for reviewers to know they are not looking at a half-finished fix. - If chunked content can reach the accumulator at all, a
chat.stream()regression test alongside the two non-streamed ones would pin it.
Related, in case it is useful for deciding the shape once: I filed #3785 for the OpenAI instrumentor, where string fields that are simply absent from the declared schema (refusal, reasoning_content) survive as a plain str and are then overwritten per chunk, so only the final fragment is exported. Same underlying question — the declared schema is the single source of truth for what a stream can reconstruct — and fixing it per field in each package will keep drifting.
The streaming accumulator kept the message content as a string accumulator, which ignored any delta that was a list, so a streamed reasoning answer reached the extractor with no content at all. A content accumulator now takes text or chunk lists, folding a delta into the trailing chunk of the same type, so the accumulated message has the same blocks a non-streamed response has. One streamed test served as server-sent events.
|
Thanks for the careful run. To your questions: magistral streams the same chunk shape it returns non-streamed (thinking chunks in |
Description
Mistral's reasoning models (
magistral-*) return the assistant message content as a list of chunks rather than a string: athinkingchunk, which itself holds a list of text chunks, followed by atextchunk with the answer (AssistantMessageContent = Union[str, List[ContentChunk]]in the SDK)._get_attributes_from_chat_completion_messageyielded that list straight intomessage.content. OpenTelemetry refuses it:so the output message on the span carried neither the reasoning nor the answer.
This keeps the string case exactly as it was and, for a list, emits
message.contents.Nblocks: each text inside a thinking chunk as areasoningblock and each text chunk as atextblock, the same shape the LiteLLM, OpenAI and (as of #3754) Groq instrumentors use for reasoning. Chunk types with no text (images, references) are skipped, as the request side already does by serialising them.Two tests with a mocked
/v1/chat/completionsresponse in the style of the existing ones: the magistral shape, and a plain string response pinned as unchanged. The first fails onmainwith the warning above and aKeyErroron the reasoning block.Checked with
tox run -e test-mistralai,test-mistralai-latestandruff-mypy-mistralai, all clean.Streaming
The streaming accumulator kept the message content as a
_StringAccumulator, which ignored any list-valued delta, so a streamed magistral answer reached the extractor with no content at all (as the review below found). A_ContentAccumulatornow takes either text or chunk lists; a delta that continues the trailing chunk of the same type extends it, so token-by-token thinking and text fold into one reasoning block and one text block, the same shape the non-streamed response gives. A third test serves a magistral stream as server-sent events throughrespxand reads the blocks back; it fails without the accumulator change with aKeyErroron the reasoning block.