[Architecture exploration] Utopia + Jev for embodied intelligence: temporal knowledge and closed-loop decisions

[UNSW-Student-Lab]

Background and motivation

Hi Utopia team — I am a student at a leading AI research lab, currently exploring architectures for embodied intelligence. I would like to discuss whether Utopia's ontology and temporal knowledge layer could be combined with Jev's structured decision capabilities to support robots performing tasks in changing environments.

This is an early research idea, not a completed integration or a claim of demonstrated performance. I am interested in the architectural fit, a small experimental starting point, and whether this direction aligns with Utopia's roadmap.

The problem I would like to explore

A robot needs to distinguish current observations, historical knowledge, and inferred beliefs. When an object moves or an action fails, updating memory is only part of the problem: the system must also reconsider decisions that depended on the outdated information.

For example, a robot previously observed an object on a table. Someone moves it. On the next task, the robot should preserve the earlier observation as history, stop treating it as sufficient evidence of the current location, and gather fresh observations instead of repeatedly executing a stale plan.

Utopia's emphasis on how knowledge changes over time seems particularly relevant here. The research question is: can explicit belief revision, linked to plan dependencies, improve task recovery under environmental change?

Proposed division of responsibilities

This table describes a proposed integration, not capabilities I assume are already implemented:

Component	Proposed role
Perception and state estimation	Produce observations with object identities, timestamps, coordinate frames, and uncertainty
Utopia	Maintain persistent semantic knowledge, evidence, temporal revisions, and task constraints; supply relevant knowledge snapshots
Task planner	Decompose goals, track plan dependencies, and handle longer-horizon reasoning or replanning
Jev	Make bounded judgments over supplied state and candidate skills, including whether to reobserve, pause, or escalate to the planner
Skill executor and local controllers	Check physical preconditions, execute available skills, and verify outcomes using sensor feedback
Feedback adapter	Compare expected and observed outcomes, then trigger state revision and reconsideration of affected plan steps

The intended loop is:

Observe → update state → retrieve relevant knowledge → plan/select → validate and execute → verify outcome → revise.

Jev's documentation describes typed judgments and probability distributions over supplied state. I would investigate it as a skill-selection and supervision component, rather than assume it is a robot policy or long-horizon planner. Its confidence would need evaluation on robotic observations; structured output does not guarantee a correct decision.

High-frequency state and real-time control would remain local to the robot. Utopia would receive meaningful events and revisions asynchronously. Geometric mapping, dynamics, balance, collision avoidance, and emergency stopping would remain the responsibility of dedicated robotics components.

Mechanisms worth investigating

1. Observation versus accepted knowledge. Preserve source, uncertainty, freshness, and unknown states. “Not observed” should not automatically become “absent.”
2. Revision-driven plan invalidation. Associate decisions with the fact versions and preconditions they used, so changes can trigger targeted checks or replanning.
3. Constrained skill selection. Filter candidates using physical preconditions and explicit rules before asking Jev to choose; revalidate immediately before execution.
4. Traceable feedback. Link observations, knowledge snapshots, candidate skills, decisions, and outcomes to distinguish perception failures, stale knowledge, selection errors, and execution failures.

The possible contribution would be this interaction between temporal knowledge, bounded decisions, and execution feedback. I would not claim architectural novelty merely from connecting the two systems.

A small initial experiment

Start with a limited “find–approach–confirm” task, a few objects, and existing robot skills. Introduce object relocation, occlusion, stale observations, and failed actions.

With perception, skills, and task conditions held constant, compare:

• A baseline planner with ordinary memory.
• The same system with Utopia-backed temporal knowledge and revision handling.
• The same system with Jev-based bounded selection and escalation.

Measure completion rate, repeated ineffective actions, recovery time, decisions based on stale facts, end-to-end latency, and inference cost. Ablations should distinguish the effects of additional memory, temporal revision, and the decision component.

Questions for the maintainers

• Would an external robotics adapter be the right starting point, or does this connect directly to planned decision capabilities?
• How would you recommend separating transient observations, uncertain beliefs, and persistent facts? Which existing API/MCP write and review semantics could be reused, and where would an extension be needed?
• Which minimal interface would be most useful to validate first: event ingestion, relevant knowledge snapshots, fact-change notifications, or links between decisions and their factual premises?

I would appreciate feedback on feasibility, relevant prior work, and the appropriate boundaries for Utopia. If Discussions is a better home for this exploration, please feel free to move it there.

Hardware context

The humanoid and hexapod platforms below provide hardware context for this exploration. These photos do not imply a completed Utopia–Jev integration.

[WaylandYang]

Thanks for writing this up carefully. "Not observed is not absent" is already how the
ledger thinks: a missing reading means nobody wrote it down, not that the value is false.

1. Adapter or planned capabilities. An external adapter, and I would not wait for
us. The "actions" work on our side is about the base calling out to other systems,
hands rather than a decision loop. The roadmap's decision
intelligence is the piece that would link a decision to the fact versions it used,
and it is not built. An adapter that has to do that linking itself is the best input we
could get for designing it.

2. Observations, beliefs, facts, and the write paths you can reuse. Until this week
everything entered as text with a time and a model read it back; your issue is what
made that a gap worth closing. #884 adds a statements source: you push
the extraction contract itself, {e, s, n} with external_id and doc_time, no model
reads it, and the item lands as an open statement with the payload as its evidence. For
your loop that is the write path: one push per observation, same external_id per
object so a later observation updates it and records a version. The two older paths
still apply where prose is what you have:

• POST /sources/{id}/ingest with a source key: a text item with an external_id;
  same id with new content updates in place and records a version. One item per
  object or event, doc_time set to observation time. Extraction turns it into
  statements in the item's own words with provenance.
• MCP remember, gated on a write-scoped token and editor role: proposes memory facts
  that wait in pending_facts for a human nod.

The three-way split is the two clocks. World time (valid_from / valid_to, with unknown
bounds allowed) is when something held; record time (attested_at /
invalidated_at) is when the base came to believe it and stopped. A stale
observation is a fact whose world interval closed; the history stays. Confidence is a
column. What is still missing is the typed layer that lets you query "location of X" as a
property rather than a phrase: that is the alignment cut, in progress. A pushed statement
reaches it the same way a document's does, through alignment; a second push under the
same identity marks the earlier statement stale rather than closing it, and the
record says why.

Where Jev fits. Jev takes a text or JSON state of up to 32k tokens and answers a
map of typed questions against it in one call, each with a distribution and a
confidence; it does not generate, call tools, loop, remember or plan, and its docs
say date comparison and temporal logic are unreliable. So: Utopia resolves time, Jev
never sees a timestamp. Hand it an as_of snapshot (entity_facts, neighbors and
timeline over MCP are the right grain for one object) in which "still
holds" versus "closed" is already a field. Then Noul "the last observation of X is
still current" reads that field, and Choice "reobserve / proceed / escalate" reads
the same state. Do not give Jev two dates and ask which is newer; you would be
measuring its weakest behaviour. Its confidence with per-action thresholds in your
code maps onto our side too: execution_gate holds a decision by impact, and the nod
queue is where "escalate to a human" lands.

3. Which interface to validate first. The premise link, and you can do it without
a robot or Jev. Ingestion exists, as_of is your snapshot, and the SSE stream on
/kbs/{id}/events carries no fact data, only "something changed". Derived facts
already carry their premises, are reproved when a premise changes, and are retired
when no longer supported. Write each plan step's precondition as a business
rule whose conclusion is "step S is executable", replay a logged observation
stream through ingest with a relocation and an occlusion in it, and check that the
affected step's fact is retracted at the right moment and the others are not. If it
holds, the adapter subscribes to those retractions and hands Jev the resulting
snapshot. If it does not, you will have found exactly where a decision-to-premise
link has to be richer than a rule.

One boundary you drew and I agree with: Utopia's latency is job-queue shaped, an LLM
call per chunk, so it belongs at event granularity; Jev's 70 to 500 ms sits closer to
the executor.

This thread is a fine place to report what the replay shows.