Saved Beam Teams preserve reinforcement prefixes and suffixes causing mismatch when attempting to use different AI Persona

[Slava-256]

What's happening?

When Beam Team is saved, it appears that reinforcement prefixes and suffixes that were used at the time are getting saved as well.

This causes a mismatch when trying to use the same beam team with a different AI persona.

Since reinforcement prefixes and suffixes always come from AI Persona anyway, I don't see why they need to be saved with the team.

Where does this happen?

Big-AGI Pro (big-agi.com)

Impact on your workflow

Medium - Workaround exists

Environment (if applicable)

No response

Additional context

No response

[enricoros]

@Slava-256 HI, I've looked into this but I'm not sure I can reproduce. When I reload a different Beam team, it uses the reinforcements (prefix/suffix) from the current persona that the Beam was launched with -- which means every team gets the same reinforcements. I could make persona switchable on Beam (it's not yet, and it's a simple change but requires Fable to look over my shoulder as I do that to make sure all the corner cases as covered).

Please let me know more about how this manifests (right now the prefixes are not saved)

[Slava-256]

@enricoros I think the issue is that the reinforcements are used from the 1st persona used to create the chat.

I just started a new chat with persona A, and beamed into 1 beam (I did not even use beam team loading), and I saw it correctly used reinforcements from persona A.

I then cancelled the beam mode and changed the persona to Persona B in the main chat window, and when I went into the beam mode, I saw it was still using reinforcement from persona A.

[enricoros]

@Slava-256 This makes sense, and I just reproduced it with the steps above! When a Beam Team is reloaded, the persona used in each Merge gets persisted rather than the current one - and the UI doesn't show that.

The persona ID is saved because Merge prompts are pulled live from that persona at merge-time (edit the persona's merge prompts -> the next Merge updates too) - so the Team holds a pointer, not a copy.

Should the merge prompts come from the saved "Team" persona and the reinforcements from the current one (today's behavior), should both follow the current persona, or both stay pinned to the saved one? It's a genuinely complex feature and I want to land on the right model - looking forward to your direction there.

[Slava-256]

@enricoros So if I understand correctly, there is a per-chat implicit/unnamed persistent Beam Team, independent of named saved teams?

And the first time I invoke Beam Mode in a new chat, persona ID gets saved for this chat's persistent Beam Team, and it can never be changed thereafter?

I feel like it is best to consider this and #1141 issues together.

Conceptually, I think of reinforcements as a user message counterpart to the system message of the AI persona. This is very similar to Merge instruction that also consists of system and user message parts.

From this point of view, the relationship between reinforcement and persona system prompt are more close than the relationship between persona's custom merge instruction and persona's main system message.

So, if you have a conversation that is compatible with multiple personas (let's call them Persona A and Persona B), then I would sometimes like to be able to use merge instruction from Persona B, after the beams have been generated by Persona A.

But I would not want to use reinforcements from Persona B when the system message is from Persona A. This is similar to how Merge instruction is always an inseparable pair of system and user message, and you do not mix and match system message from one merge instruction and user message from another merge instruction.

As an a aside, a couple of examples of reinforcement instructions to illustrate how they are tied to the system message:

For a web search optimized persona, the reinforcement counteracts models training to cut the web search short.
For deep analysis optimized persona, the reinforcement overrides models pre-trained judgement of how much thinking a given problem needs.
(experiments had shown that while system message can provide an algorithm for how to handle the problem, the cases where LLM makes pre-trained judgements about the specific question (such as complexity of the question) are best engaged and corrected by the user message.)

As you can see, each of these reinforcements makes sense only in the context of their counterpart system message.

Now, merge instructions, consisting of system and user message are more independent of the persona, because merge system message completely replaces persona system message.
(Custom merge system message may contain parts, or the entirety of the related persona's system message, so even though it is closely related to a given persona, it can work completely independent of it.)

So I think there should not be a global persona ID saved for the beam team. Reinforcements should always be used from the persona currently selected for the chat and used to generate the beams.

And if a merge is a persona's custom merge, and it is saved by reference, then persona ID from which a given merge is referenced should be saved per-merge.

This way #1141 will get addressed, and it will even handle the case where you may load a beam team that has merges from a different persona than is currently selected for the chat, and be able to add merges from the current persona.

Per-merge saving of persona ID would allow having merges from multiple personas in the same team. While I don't think easy access to multiple personas' custom merges is needed in the UI, it is still useful to be able to work correctly if you end up with multiple persona's merges in a given team by way of saving and loading beam teams across persona changes in a chat.