The model is definitely over-thinking, can you fix this ?

#3
by bipinm - opened

Let me start by saying - a big THANK YOU to RecursiveSI. You have hit the sweet spot of running reliability at speed on a local machine, If you can fix this over-thinking problem i can say this could be the go-to model for many. Thank you!

This is the first model i'm able to run reliably on my local machine - M4 Pro 48 GB. I'hv tried several models in the past, but all of them failed one way or the other - going into loops, tools not working as expected, being too slow to use for any meaningful tasks etc. I'm running it with rapid-mlx + opencode.

One issue, i would definitely like to be fixed in this model is over-thinking. I have tried several code generation/fixing tasks and every time it seems to over-think. Even after generating a proper result, it keeps iterating over and over to verify and re-verify results. Often i see it generate python script to verify results, then use node.js to re-verify and then again it tried to verify code itself etc.. I found this issue mentioned by other people who has tested the model too, perhaps on youtube.

An example ...

Let me verify the complete state of the file to make sure everything is coherent and there are no remaining issues.
Let me check the full current state of the key functions and structure.
Let me now review the full generation flow once more to ensure it's all consistent and the user's issue is fully resolved.
Let me read through the complete file once more to verify everything is in order.
Let me check the remaining parts of the file I haven't reviewed after the edits.
Explored: 1 read
Let me verify the complete current state of the file to make sure everything is coherent and works correctly.
Let me check a few remaining areas - the HTML structure around the probChart, the vocab usage, and the overall flow.
Let me verify there are no duplicate function definitions or issues.
Let me check the full file once more for any remaining problems.
$ cd /Users/I038437/Temp/opencode/llm-sim 
... [REMOVED FOR BREVITY]
Let me verify the complete state of the file to ensure everything is coherent and works correctly.
Let me check the remaining parts of the file I haven't reviewed after the edits.
Explored: 1 read
<system-reminder>
The user wants you to continue from where you left off, without restarting, without summarizing, without wrapping up, without concluding, without starting over, without switching modes, and without summarizing or responding with acknowledgements or questions. Just continue the main text directly.
Assistant: Let me continue from where I left off and verify the file state is fully coherent.
Let me check the remaining issues. The file is 752 lines. Let me verify the complete state, particularly the sections I haven't fully reviewed after all the edits.
Let me check a few key areas - the beginning of the file, the HTML body, and the CSS.
Let me now verify the complete file state to ensure everything is coherent and working correctly.
Let me check the current state of the file structure and any remaining issues.
Let me look at the full HTML body section and the beginning of the file.
Explored: 1 read
The user wants me to continue from where I left off without restarting, without summarizing, without wrapping up, or concluding. I was in the middle of verifying the file state and improving the clarity of the generated outputs regarding top-p, top-k, and temperature.
Let me continue by checking the current state of the file and making final fixes.
I need to:
Verify the complete file is coherent
Make sure the generation output is clear regarding top-p, top-k, temperature
Possibly add explanations for what the numbers mean
Let me continue reading the file to check the current state.
Explored: 1 read
Let me continue verifying the file state and addressing the user's core concern about clarity of generated outputs regarding top-p, top-k, temperature.
Let me check the remaining sections of the file to make sure everything is coherent.
Let me read the rest of the file.
Explored: 1 read

Another run example

Now let me verify the changes look right in context.
→Explored: 1 read
Let me verify the full generation flow once more (the end of runGeneration and the output container markup), and check for any issues.
→Explored: 1 read, 1 search
Let me verify the page renders without JS errors by checking the full structure of the generation flow once more.

The HTML structure is valid. Let me also run a quick JS syntax check on the inline script.

JS syntax is valid.
Summary of changes made to llm-sim/index.html
1. Fixed the scrolling issue in the "Generated Output" section

Let me look at the current state of the generation-related elements to make the right fix.
→
The user wants: under the "Generate Text (temperature & sampling)" card there's a readout showing "Waiting for input…" / "Generating…". They want the generated text/tokens/etc. to show up there (in the readout area) and not below the tokens, and the "Generation complete — X tokens generated" text should remain there.

... [REMOVED FOR BREVITY]

What if instead of removing, I make the Generated Output card not exist and move its stepLog to... no.
ENOUGH. Implementation:
...
Let me write the edits.I understand the issue now. 
...
Let me make the JavaScript write the tokens and completion message into the #readout element instead:
...
The edit already applied correctly. Now let me update the remaining out references in the JS.
...
Good — the first edit applied cleanly and the redundant second attempt was unnecessary. Now let me fix the remaining out references in the JS.
← Edit
No changes to apply: oldString and newString are identical.
Let me view the current state of the key sections to make precise edits.

.. and it keeps doing this forever and i need to cancel the session to make it stop after 10-15 minutes. All i asked it was a simple edit which already worked on first try within 2-3 minutes

After testing it for few days and comparing against rapid-mlx/Qwen3.8-27B-4bit-MTP-MLX, this model definitely needs a lot of fixing

  • Qwen 3.8 is much slower but generates decent results
  • Qwen 3.8 does no over-thinking and able to generate full output as expected
  • Tested both models with such a prompt "create a modern space shooter game in html" and flint created "something" which didn't work, there were floating polygons in the HTML page and nothing else. Qwen on the other hand created a full decent working game

Flint
Context 17,641 tokens
Steps 25
Input 17,263
Output 7,529
Reasoning 10,724
Cache Read 293,072
Cache Rate 94.4%
Total 328,588
TPS 57.8 tok/s
TTFT 2.2s
Run Time 5m 17s

Qwen
Context 16,938 tokens
Steps 5
Input 16,676
Output 11,366
Reasoning 0
Cache Read 54,470
Cache Rate 76.6%
Total 82,512
TPS 14.5 tok/s
TTFT 3.8s
Run Time 13m 02s

Sign up or log in to comment