Repository navigation
feat(sim): let the persona see iframes and tabs, press keys and see colours - #77
Merged
Merged
Conversation
Claude Code spilled any MCP result over its default size to a file and gave the model its path, with no tool to read it. A portals run's form description (~85k characters) never reached the model. Each bridged tool now declares anthropic/maxResultSizeChars, which keeps a 179k-character result inline (checked live). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ee colours Judged portals runs failed for want of what a person has: - an outline that stopped at iframes, so each host added a root per frame; - no keyboard, so the persona reported click errors as screen content; - no new tabs, so « Voir sur le portail » looked like a dead link; - no colours, so a theme change could not be checked. The persona now has these tools: - a root can set `frames: true` to see and act inside its iframes, nested ones included (AI-mode outline, normalized back to the plain shape); - `press` a key, in a named field or wherever the focus is; - a link opening a new tab takes the persona there, and `switch_tab` brings it back; - `screenshot` shows the screen as an image; the record keeps only that it was taken, and the judge is told such claims are unverifiable. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…any frame A judged run had the persona click « Barre de navigation »: the chat frame's reply contained those words, the editor's tab of that name sat in a later frame, and the click landed on the chat's text. Every control role (tabs, menu items, options, checkboxes…) is now tried across all frames before the plain-text fallback. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Judges kept reporting that observation turns run past the person's messages. Turn N is what the person did while writing their Nth reply, and the final DONE reply is not sent. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A judged run had the persona click « Type de lien ». The label text was hit, the drop-down of that name never opened, and the person reported the product as broken. Comboboxes are now tried with the other controls, before any text. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vuetify keeps a select's input under the field's div. A judged run's click on « Type de lien » waited out the 15 s timeout on « intercepts pointer events », and the persona reported the product as broken. When a short check shows that the control cannot take the click, the persona now clicks at its position, on whatever covers it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…the person's turns The bridge replayed the history as « user: … / assistant: … » lines and asked for the next assistant turn. Continuing that script, judged haiku runs wrote the person's next « user: » line themselves, along with copies of the application's host-event blocks, then answered them. A real provider receives structured messages. Each message now sits in its own tag, and the model is told to write only the assistant's reply. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tly first A long drop-down renders only the options in view: a judged run read the first 9 options of « Type de page » and told the assistant the catalogue it named was not there. The outline now says when an open list holds more than it shows, and clicking an option out of sight scrolls the list to it, as a person scans for the option they were told to pick. Clicks matched a role's name as a substring, links before drop-downs: a click on the « Page » field hit the « Pages de portails » link and threw away a half-configured menu item. Exact names are now tried first. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… results right A judged run's persona clicked « Page blanche – Commencer avec une page vide » with a dash the card's name lacks: the click fell back to the chat's text of that name, nothing happened, and the person and the assistant concluded the wizard was broken. Controls are now matched on their words before text is. The « more options » note fired on a list of 6 options all in view, a few pixels short of its content; it now needs half an option hidden. And the persona is told what a click on plain text and a list holding more options mean, which it misreported as product failures. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The sentence about lists holding more options was wrapped in the middle of its key phrase. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…hows A judged run listed the 9 options in view of « Type de page » and told the assistant « Catalogue d'événements » was missing, without clicking it by its name as the look's note and the persona's instructions both said. A look now scrolls such a list through, names all its options, and puts the list back where it was. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A judged portal simulation: the result of wait_for_user_action listed « draft-validated » first, then the older « unpublished changes: the person must press « Valider le brouillon » » state, which reads as the state that followed the validation. The event that resolved the wait and its followers are now listed in time order. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The settings page also has a « Usage » activity tab. When it rendered before
the card's title, getByText('Usage') matched both and strict mode failed: it
passed alone and failed in the full suite, depending on which came first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Give the simulated persona a more complete and more reliable view of the page and control over it, and fix bridge framing problems that judged runs exposed.
Why: in judged simulations the persona reported missing options, dead links and controls that "did nothing", and those reports came from blind spots in the harness, not from the product.
screenshot(colours and layout, recorded as taken and without the image),press(a key in a named field or wherever the focus is) andswitch_tab(atarget="_blank"link moves the persona to the new tab).frames: truesees and acts inside its iframes, nested ones included.lookscrolls through an open virtual list and names every option. Clicking an option that is out of view scrolls the list to it.role:lines, so the model no longer writes the person's turns or host events itself.lib-sim0.10.0. It exportsPerceptionContentand addscallContenttoPagePerception, both additive.Heads-up:
renderTranscriptapplies to every bridged conversation, including the dev workspace onnpm run dev-bridge, not only to simulations.looknow scrolls open listboxes (and restores them) on every root. With a long list this can add a few seconds per look.