When using chat.sendMessage() from both Javascript and Python, I noticed that for some models and with minimal thinking level the thoughts return instantly, but contain some kind of seemingly almost hard-coded texts that match verbatim between different calls.
Surprisingly, no such issue for chat.sendMessageStream() - it returns either empty thoughts for minimal level or relevant thoughts when level is not set.
It caused some hair pulling trying to figure out what's going on. I'm wondering if these fixed texts count as tokens or not? It would be bad if they do.
Here are my all test cases. If you need, I can provide the example code, but it's almost copy-paste from Google's Javascript examples.
includeThoughts: true for all the tests below.
Thoughts are extracted from candidates.content[0].parts that have part.thought=true
a\ Non-streaming tests with function: chat.sendMessage
a1
Model: gemini-3.1-flash-lite
Thinking level: not set
Thoughts: Generic hard-coded text that may repeat verbatim between requests. !!!
a2
Model: gemini-3.1-flash-lite
Thinking level: ThinkingLevel.MINIMAL
Thoughts: Generic hard-coded text that may repeat verbatim between requests. !!!
a3
Model: gemini-3.5-flash
Thinking level: not set
Thoughts: Unique, relevant to the prompt
a4
Model: gemini-3.5-flash
Thinking level: ThinkingLevel.MINIMAL
Thoughts: Generic hard-coded text that may repeat verbatim between requests. !!!
b\ Streaming tests with function: chat.sendMessageStream
b1
Model: gemini-3.1-flash-lite
Thinking level: not set
Thoughts: empty
b2
Model: gemini-3.1-flash-lite
Thinking level: ThinkingLevel.MINIMAL
Thoughts: empty
b3
Model: gemini-3.5-flash
Thinking level: not set
Thoughts: Unique, relevant to the prompt
b4
Model: gemini-3.5-flash
Thinking level: ThinkingLevel.MINIMAL
Thoughts: empty
And here are examples of two "fake thinking" texts that I've noticed thus far:
1
Okay, I'm ready. Here's how I'll summarize those thoughts, channeling my expert understanding:
Analyzing the Core of the Matter
Alright, let's break this down. My initial reaction is to sift through the data – what are the key components presented? What assumptions are being made, explicitly or implicitly? I need to quickly identify the central argument or concept. From there, I'll assess its validity and potential flaws, drawing upon my years of experience and specialized knowledge. I’m thinking about the established body of work in this area; is this novel, a refinement, or a complete departure? What are the implications if this proves to be correct, or incorrect? The value here is in critically evaluating this material, connecting it to the broader landscape, and formulating my own informed viewpoint. This will let me assess this from an expert perspective, allowing me to provide a robust response. Ultimately, I aim to provide a concise and insightful commentary.
2
Okay, I'm ready. Here's how I'll summarize the text, assuming I'm an expert in the field and providing a fulsome thought process:
My Analysis of the Text's Core Concepts
Alright, let's break this down. My initial reaction is to consider the nuances. What's the real thrust of this? Is it a novel application, a re-framing of existing principles, or perhaps a criticism of current methodology? I need to look for the key verbs and nouns, the relationships between them. Where are the connections to pre-existing literature? Does this build on prior research, or is it a complete departure?
Immediately, I'm considering the potential audience for this work. Who are they targeting? Are they addressing a specific sub-field, or casting a wider net? If it's a specialized group, I'll need to assess the level of assumed knowledge. What assumptions are they making about prior understanding? If this is targeted at a general audience, are they watering down the concepts, or simply approaching them from a different angle? That's an important distinction to make.
I'm thinking about the methodology. What's the approach? Is it quantitative, qualitative, or perhaps a mixed-methods design? What are the limitations of the chosen method? And, crucially, do the methods actually support the conclusions being drawn? I'm already mentally checking for any logical fallacies or unsupported claims. Are they relying on correlation as causation? Is there any confirmation bias apparent?
Finally, what's the potential impact of this work? Even if it's a small piece of research, is there a clear potential for theoretical contribution or practical application? Could this potentially shift the field in some way? Or is it simply an incremental step forward? That will inform how I weigh the merits and shortcomings. I'm going to look for any hidden gems that might spark future research or challenge me to change my work.
When using chat.sendMessage() from both Javascript and Python, I noticed that for some models and with minimal thinking level the thoughts return instantly, but contain some kind of seemingly almost hard-coded texts that match verbatim between different calls.
Surprisingly, no such issue for chat.sendMessageStream() - it returns either empty thoughts for minimal level or relevant thoughts when level is not set.
It caused some hair pulling trying to figure out what's going on. I'm wondering if these fixed texts count as tokens or not? It would be bad if they do.
Here are my all test cases. If you need, I can provide the example code, but it's almost copy-paste from Google's Javascript examples.
includeThoughts: true for all the tests below.
Thoughts are extracted from candidates.content[0].parts that have part.thought=true
a\ Non-streaming tests with function: chat.sendMessage
a1
Model: gemini-3.1-flash-lite
Thinking level: not set
Thoughts: Generic hard-coded text that may repeat verbatim between requests. !!!
a2
Model: gemini-3.1-flash-lite
Thinking level: ThinkingLevel.MINIMAL
Thoughts: Generic hard-coded text that may repeat verbatim between requests. !!!
a3
Model: gemini-3.5-flash
Thinking level: not set
Thoughts: Unique, relevant to the prompt
a4
Model: gemini-3.5-flash
Thinking level: ThinkingLevel.MINIMAL
Thoughts: Generic hard-coded text that may repeat verbatim between requests. !!!
b\ Streaming tests with function: chat.sendMessageStream
b1
Model: gemini-3.1-flash-lite
Thinking level: not set
Thoughts: empty
b2
Model: gemini-3.1-flash-lite
Thinking level: ThinkingLevel.MINIMAL
Thoughts: empty
b3
Model: gemini-3.5-flash
Thinking level: not set
Thoughts: Unique, relevant to the prompt
b4
Model: gemini-3.5-flash
Thinking level: ThinkingLevel.MINIMAL
Thoughts: empty
And here are examples of two "fake thinking" texts that I've noticed thus far:
1
Okay, I'm ready. Here's how I'll summarize those thoughts, channeling my expert understanding:
Analyzing the Core of the Matter
Alright, let's break this down. My initial reaction is to sift through the data – what are the key components presented? What assumptions are being made, explicitly or implicitly? I need to quickly identify the central argument or concept. From there, I'll assess its validity and potential flaws, drawing upon my years of experience and specialized knowledge. I’m thinking about the established body of work in this area; is this novel, a refinement, or a complete departure? What are the implications if this proves to be correct, or incorrect? The value here is in critically evaluating this material, connecting it to the broader landscape, and formulating my own informed viewpoint. This will let me assess this from an expert perspective, allowing me to provide a robust response. Ultimately, I aim to provide a concise and insightful commentary.
2
Okay, I'm ready. Here's how I'll summarize the text, assuming I'm an expert in the field and providing a fulsome thought process:
My Analysis of the Text's Core Concepts
Alright, let's break this down. My initial reaction is to consider the nuances. What's the real thrust of this? Is it a novel application, a re-framing of existing principles, or perhaps a criticism of current methodology? I need to look for the key verbs and nouns, the relationships between them. Where are the connections to pre-existing literature? Does this build on prior research, or is it a complete departure?
Immediately, I'm considering the potential audience for this work. Who are they targeting? Are they addressing a specific sub-field, or casting a wider net? If it's a specialized group, I'll need to assess the level of assumed knowledge. What assumptions are they making about prior understanding? If this is targeted at a general audience, are they watering down the concepts, or simply approaching them from a different angle? That's an important distinction to make.
I'm thinking about the methodology. What's the approach? Is it quantitative, qualitative, or perhaps a mixed-methods design? What are the limitations of the chosen method? And, crucially, do the methods actually support the conclusions being drawn? I'm already mentally checking for any logical fallacies or unsupported claims. Are they relying on correlation as causation? Is there any confirmation bias apparent?
Finally, what's the potential impact of this work? Even if it's a small piece of research, is there a clear potential for theoretical contribution or practical application? Could this potentially shift the field in some way? Or is it simply an incremental step forward? That will inform how I weigh the merits and shortcomings. I'm going to look for any hidden gems that might spark future research or challenge me to change my work.