You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
PRD: Ontology Lifecycle Management (Requests & Comments for Concepts and Collections)
Problem Statement
The Ontology Generator produces curated Turtle output that a steward can save into a Knowledge Collection, but once that save completes the produced concepts and the collection that holds them live in a governance vacuum. Stewards and business users coming from Data Products and Data Contracts find a stark ergonomics gap: those entities have a mature Request... dropdown (access, review, publish, certify, status change), an Activity Timeline with Comments, an Ownership panel, and a real workflow trigger system that fans out approvals to the right reviewers. Ontology Concepts and Collections have none of this.
Concretely, the gaps that block this feature from being usable in a real governance program are:
No personal-draft workspace. A user who runs the Generator and saves to a Collection drops their output directly into an Enterprise-scoped collection by default. There is no notion of "my drafts" that they can iterate in privately before sharing. The first save is the first publish.
Collection ownership is not real. Collections carry a created_by user email and a scope_level label (enterprise, domain, department, team, project, external), but the scope is cosmetic — setting scope_level='team' does not actually link the collection to a Team row. There is no team or domain foreign key, no enforceable visibility, and nothing for "Promote to a team / LOB collection" to actually target.
Concept status enums drift across layers. Pydantic models declare draft / under_review / approved / active / deprecated / retired, the runtime writes draft / under_review / approved / published / certified / deprecated / archived, and the frontend types match the Pydantic version. The same concept can be simultaneously active (Pydantic-readable), published (RDF-readable), and approved (workflow-readable) depending on which surface is asking. The introduction of prd-unified-lifecycle-tracking.md already split Status, Certification, and Publication into three orthogonal dimensions for Data Products and Data Contracts; ontology concepts were explicitly deferred and now need that alignment.
Publishing is broken at the system level, not just the UX level. Concept status='published' writes RDF triples but the Concept Browser does not filter by status, the global concept search does not enforce it, and most critically the Term Mapping bulk suggester reads only urn:semantic-model:* contexts. Generator output lands in urn:ontology:* contexts and is therefore invisible to the very suggester the platform exists to feed. Publishing a generated concept today does not make it available for semantic assignment, which is the entire promise.
Promotion exists in code but not in product.promote_concept and migrate_concept functions are wired in the manager with provenance triples, but they have no approval gate, no audit batching, no surfacing in the UI, and no entry point from the Generator save dialog. Nothing in the user experience makes "move this term from my drafts to the company glossary" a real motion.
Concepts and Collections have no Comments and no Activity Timeline. The entity panel matrix marks both as — for Comments. Stewards and business users cannot discuss a term proposal in the system. Change-log entries fire from update_concept_status but there is no panel to read them through. Coordination happens in Slack and gets lost.
No Collection detail page exists. Collections are edited via a dialog from the list view. Concept-level Requests can be added to the existing concept-detail.tsx, but collection-level requests, ownership, comments, and change-log have nowhere to live.
Without lifecycle management, the Ontology Generator remains a useful prototyping tool whose output cannot enter the governed catalog without one-off manual effort. Customer ontologies cannot reach "available for semantic assignment and global search" — they sit in collections that the suggester cannot read, with statuses no surface enforces, owned (in name only) by labels that do not link to real teams. The platform's downstream value (impact analysis across business terms, semantic search, ontology-driven compliance) is gated on this work being done.
Solution
Extend the existing polymorphic Request / Workflow / Comments infrastructure to Ontology Concepts and Collections; give Collections real team and domain ownership backed by foreign keys; align Concept lifecycle with the unified Status / Certification / Publication dimensional model already in flight for Products and Contracts; and unify "Publish" into a single milestone that simultaneously sets concept publication state, registers the collection as a Term Mapping concept source, and gates global Concept Browser visibility.
The Generator becomes the natural entry point to this lifecycle: its save dialog defaults to the current user's "My drafts" collection (a personal scope that is the absence of team and domain assignment), creates concepts in status='draft', and offers two optional shortcuts — Submit for Review on save and Save & Request Promote to.... Once saved, the user navigates to the concept or collection and uses the same Request... dropdown they already know from Data Products and Data Contracts.
Three reusable backend pieces from the existing infrastructure carry most of the weight:
The TriggerRegistry / WorkflowExecutor stack handles request submission, approval routing via role aliases (data_stewards, admins, owner), pause-and-resume for approval steps, notifications, and audit log entries. New trigger types are added only where the semantics genuinely differ (promote, reassign-ownership); review / approve / publish / certify / status-change reuse existing triggers.
The polymorphic Comments + Activity Timeline is enabled for entity_type='ontology_concept' and entity_type='ontology_collection'. Audience scoping, change-log merge, and the existing CommentSidebar work unchanged. Mention notifications stay out of scope — they ship through prd-comments-audience-and-mentions.md.
The business_owners polymorphic table + team / domain assignment pattern from Data Products is reused for collections, in line with the consolidation direction set in prd-consolidate-team-and-ownership.md. A collection's "owner" is the same shape as a product's owner; visibility filters use the same predicates.
A new "Promote Concept" request type is the single new flow that does not have a Product/Contract analogue. It is a copy-with-provenance operation: on approval, the suggester-source concept is duplicated into the target collection with a fresh IRI, an ONTOS.sourceConceptIri provenance triple, and the source is auto-marked deprecated with an ONTOS.supersededBy pointer to the new IRI. This avoids breaking inbound references in entity_semantic_links, agreements, and search history that already point at the original IRI.
The new Collection detail page hosts everything that has no home today: ownership (team / domain / business owners), comments and change-log, pending requests, and direct buttons for collection-level requests. The existing collection editor dialog stays for quick metadata edits from the list view.
This PRD also delivers the alignment of Ontology Concept lifecycle with the unified model previously deferred in prd-unified-lifecycle-tracking.md. Concepts get the same three orthogonal dimensions (Status, Certification, Publication) as Data Products and Data Contracts, with the same enum values, the same certification audit metadata, and the same publication scopes. Once aligned, the Term Mapping suggester reads concepts whose publication scope is organization (the closest analogue to "available for global functions").
User Stories
Personal drafts and Generator handoff
As a steward running the Ontology Generator for the first time, I want my saved concepts to land in a personal-draft collection of mine (rather than directly in an Enterprise-scoped one), so that I can iterate before others see my output.
As a steward, I want the Generator's Save to Collection dialog to default to my personal-draft collection (auto-created on first use), so that I do not have to think about scope before I have content.
As a steward, I want to override the default target in the Save dialog and explicitly pick a team or LOB collection if I am confident, so that experienced users can skip the personal-draft step.
As a steward saving a generation run, I want an optional Submit for review on save checkbox, so that I can enqueue review requests on every concept in one click.
As a steward saving a generation run, I want a Save & Request Promote to... shortcut that opens the promote-request dialog pre-filled with the saved concepts, so that I can move directly from generation to promotion.
As a steward, I want every newly-generated and saved concept to land with status='draft', so that the lifecycle starts at the unambiguous beginning regardless of how the Generator inferred the term.
Collection ownership and visibility
As a steward, I want a Collection to have a real Team assignment (a foreign key to a Team row), so that "team-owned" is enforceable and not just a label.
As a steward, I want a Collection to have a real Domain assignment, so that LOB-level ontologies have a first-class home.
As a steward, I want a Collection with no Team and no Domain to be considered "personal" — visible only to its created_by user and admins, so that my drafts stay private without me configuring an ACL.
As a steward, I want a Collection's Business Owners list (using the same business_owners table as Data Products) to be the authoritative list of named owners, so that ownership concepts are uniform across the platform.
As a Data Consumer browsing concepts, I want to see only concepts whose collection is visible to me (mine, my team's, my domain's, or enterprise-wide), so that personal drafts of other users do not clutter my view.
As an admin, I want to see all collections regardless of ownership, so that governance oversight is preserved.
As a steward, I want a Concept's lifecycle to use the same three orthogonal dimensions (Status, Certification, Publication) as Data Products and Data Contracts, so that I learn the model once.
As a steward, I want Status on a concept to follow the shared EntityStatus enum (draft / proposed / under_review / approved / active / deprecated / retired), so that the existing transition machinery works without entity-specific branches.
As a steward, I want Certified to be a separate dimension with admin-configurable levels (Bronze / Silver / Gold etc.), an audit trail, and an expiration, so that high-trust concepts can be recognised as such.
As a steward, I want Publication on a concept to be a scoped value (none / team / domain / organization / external), so that I can publish a term within my domain before promoting it organisation-wide.
As a steward, I want a Concept's effective certification to inherit from its containing Collection where the Concept itself is uncertified, so that a Gold-certified glossary uplifts all its terms by default.
As a Data Consumer, I want to see Status, Certification, and Publication as three separate inline indicators on a Concept (badge / shield / icon), so that I can scan quickly.
Concept-level Requests
As a steward on a Concept detail page, I want a Request... dropdown identical in shape to the one on Data Products and Data Contracts, so that I learn one pattern.
As a steward, I want to Request Review on a draft concept, so that I can hand off curation to a peer.
As a reviewer, I want to receive a notification with a deep link to the concept under review, so that I do not need to hunt through the queue.
As a steward, I want to Request Approval on a reviewed concept, so that the approver role explicitly signs off rather than the reviewer implicitly approving.
As a steward, I want to Request Publish on an approved concept to set a target publication scope (team / domain / organization / external), so that publication is intentional.
As a steward, I want Request Certify to ask for a target certification level, an optional rationale, and an optional expiration, so that the certification request carries the metadata that will appear on the certified concept.
As a steward, I want to Request Promote on a concept to pick a target collection, choose Move-with-provenance vs Copy-with-provenance (default Copy), and submit, so that I can move my concept up to a team or LOB collection.
As a steward, I want to Request Status Change to specify a target status from the allowed transitions for the concept's current state, so that the workflow knows what is being requested.
As a steward, I want each request submission to be tied to a default workflow that ships out of the box, so that organisations can run this feature without configuring anything first.
As an admin, I want each default workflow to be editable in Settings (same surface as the existing Product/Contract workflows), so that I can adapt the approval routing to my organisation.
Collection-level Requests
As a steward on a Collection detail page, I want a Request... dropdown that exposes Request Publish, Request Reassign Ownership, and Request Status Change, so that collection-level moves use the same vocabulary as concept-level ones.
As a steward, I want Request Publish on a Collection to publish all approved concepts in the collection in one shot at a chosen scope, so that "release my ontology" is a single action.
As a steward, I want Request Reassign Ownership on a Collection to propose a new Team and / or Domain, so that promoting an ontology up the ownership hierarchy follows the same approval path as other governance changes.
As a steward, I want Request Status Change on a Collection to support transitions between active and archived, so that retired ontologies can be archived without losing their content.
Approval, notification, and audit
As an approver of a Concept-level request, I want to receive an in-app notification with a deep link to the concept, the request type, the requester, and any reason text, so that I have one-click access to the decision context.
As an approver, I want to approve or reject directly from the notification handler dialog (same pattern as WorkflowApprovalResponseDialog for products), so that I do not have to navigate.
As an approver of a Promote request, I want to be the owner of the target collection (or domain owner if the target is domain-scoped), so that the receiving collection's governance has the final say.
As an approver of a Reassign Ownership request, I want admin permission to be required (not just stewardship), so that ownership changes are admin-grade events.
As an auditor, I want every request submission, approval, rejection, and apply event recorded in the user-action audit log under a ontology-lifecycle feature key, so that compliance reviews are traceable.
As an auditor, I want per-concept and per-collection events to appear in the same entity_change_log infrastructure used by Products and Contracts, so that per-entity history is consistent.
Publish unification (Term Mapping and search visibility)
As a steward publishing a concept at organization scope, I want the concept's collection to be automatically registered as a Term Mapping concept source from that moment on, so that publication actually makes the concept available for semantic assignment.
As a steward, I want the Term Mapping run-config dialog to list published-at-organization collections alongside existing semantic_models sources, so that generator-produced collections become first-class citizens of the suggester.
As a Data Consumer in the Concept Browser, I want to see only concepts whose Status is at least approved and whose Publication scope reaches me (team / domain / organization), so that draft and under-review concepts do not pollute the search results.
As a steward, I want a "Show all (including drafts)" toggle in the Concept Browser when I have steward role, so that I can find my own and my team's drafts when I need to.
As an admin, I want to override the Term Mapping inclusion behaviour with a feature flag, so that the unification can be rolled out gradually to large customers.
Comments and Activity Timeline
As a steward on a Concept detail page, I want an Activity Timeline panel that shows comments and change-log entries interleaved, so that the discussion and history of the concept live in one place.
As a steward on a Collection detail page, I want the same Activity Timeline, so that collection-level discussion has a home.
As a comment author, I want to use the same audience picker as on Products and Contracts (teams, roles, owner — once prd-comments-audience-and-mentions.md ships), so that I do not learn a new tool.
As a steward, I want concept status changes, certification events, publication events, and ownership changes to appear in the Activity Timeline as system entries, so that I can see the history of governance moves alongside the human discussion.
As an admin, I want the Comments feature permission to gate comment creation on Concepts and Collections in the same way it gates other entity types, so that role-based access is uniform.
Collection detail page
As a steward, I want a Collection detail page reachable from the Collections list (and deep-linkable), so that I have a stable URL to share when I am working on a specific ontology.
As a steward on the Collection detail page, I want tabs for Overview (metadata + ownership), Concepts (the concept list), Activity (comments + change-log), and Pending Requests (open requests targeting this collection), so that I can manage the whole lifecycle from one place.
As a steward, I want the existing collection editor dialog to remain available for quick metadata edits from the list view, so that the detail page is not required for trivial changes.
As a steward on a Collection detail page, I want a "Pending Requests" panel listing open requests on the collection or its concepts, with my-action-needed entries highlighted, so that I know what is waiting on me.
Permissions
As an admin, I want a new feature key (ontology-lifecycle) registered with standard Read-Only / Read-Write / Admin levels, so that role-based access can be configured per organisation.
As an admin, I want request submission gated to Read-Write, approval / rejection gated to the configured workflow approver roles, and the "Show all (including drafts)" Concept Browser toggle gated to Read-Write or higher, so that the trust tiers are explicit.
Demo and onboarding
As an evaluator running the retail demo preset, I want at least one pre-seeded personal-draft collection (for the demo user) and one team-owned collection, so that the lifecycle UI is non-empty on first load.
As an evaluator, I want at least one open Request Publish request on a demo concept, so that the approval handler dialog can be demonstrated without setting up a workflow first.
Implementation Decisions
Scope and adjacencies
In scope for v1: Concept-level and Collection-level Requests (catalog defined below); Comments + Activity Timeline on Concepts and Collections; real Team / Domain / Owner ownership on Collections; unified Status / Certification / Publication dimensions on Concepts (deferred alignment from prd-unified-lifecycle-tracking.md); atomic Publish milestone that includes Term Mapping source registration and Concept Browser visibility enforcement; Generator save flow change to default-personal-draft; new Collection detail page.
Out of scope for v1 but documented for follow-ups: @-mention notifications in comments (covered by prd-comments-audience-and-mentions.md), comment-create notification fan-out beyond mentions, advanced collection-level ACLs (per-user audience tokens on a collection), inline reviewer suggestions on the Activity Timeline.
Lifecycle alignment with the unified model
Ontology Concepts adopt the three orthogonal dimensions defined in prd-unified-lifecycle-tracking.md: Status, Certification, Publication. The current single ConceptStatus enum is decomposed.
Status uses the shared EntityStatus enum (draft / proposed / under_review / approved / active / deprecated / retired). Concept-specific transitions are configured in the same transition-map mechanism Products and Contracts already use.
Certification is a separate dimension with admin-configurable levels (Bronze / Silver / Gold etc.) and audit metadata (certified_at, certified_by, certification_level, expires_at). Effective certification inherits from the containing Collection where the Concept itself is uncertified, mirroring the Product→Contract→Dataset inheritance described in the unified PRD.
Publication is a scoped enum (none / team / domain / organization / external). The existing published_at / published_by triples become the audit fields for the Publication dimension; the published boolean is removed in the migration.
The runtime values currently being written under ConceptStatus (published, certified, archived) are remapped: published becomes Status=active + Publication scope set; certified becomes Certification level set; archived becomes Status=retired.
Collection ownership and visibility model
Collections gain a real Team assignment (FK to the Team entity introduced in prd-consolidate-team-and-ownership.md) and a real Domain assignment (FK to the Data Domain entity), both nullable.
Collections gain Business Owners through the existing polymorphic business_owners table (same table Products use), with the same role taxonomy and lifecycle tracking.
"Personal" is defined by the absence of Team and Domain assignment. A collection with neither is visible only to its created_by user and admins.
Visibility filters are enforced server-side at the manager / repository layer (not just cosmetically in the UI). The same predicate is applied to: the Collections list, the Concept Browser default scope, the Term Mapping concept-source picker, the global concept search.
Request type catalog and trigger reuse
Concept: Submit for Review, Request Approval, Request Publish, Request Certify, Request Promote, Request Status Change.
Collection: Request Publish, Request Reassign Ownership, Request Status Change.
The existing for_request_review, for_request_publish, for_request_certify, for_request_status_change wizard triggers and their on_request_* process counterparts are reused as-is. The wizards are entity-agnostic; the entity type flows through trigger_context.
Two new trigger pairs are added: for_request_promote / on_request_promote (concept-only) and for_request_reassign_ownership / on_request_reassign_ownership (collection-only). They follow the exact shape of the existing trigger handlers.
Two new EntityType values are added: ontology_concept and ontology_collection. They are accepted by all polymorphic routes (comments, change-log, business_owners, agreements, business_lineage where applicable).
No Request Access or Subscribe types are added — concepts and collections are not access-controlled like data products; visibility is governed by the ownership model above.
Approval routing
Submit for Review (Concept) → owners of the containing collection, falling back to data_stewards.
Request Approval (Concept) → data_stewards.
Request Publish (Concept or Collection) → data_stewards + admins. Publication is a global-visibility act that warrants two-tier approval by default.
Request Certify (Concept) → admins (matches the certification approver pattern from the unified lifecycle PRD).
Request Promote (Concept) → owners of the target collection, escalating to domain owners or admins if the target is team or domain-scoped and has no business owners.
Request Status Change → owners of the affected entity, plus data_stewards for transitions involving deprecated or retired.
All approver lists are encoded as default workflow templates in the existing default_workflows.yaml. Organisations edit them in Settings using the existing workflow editor — no new editor surface is needed.
Promote semantics: copy with provenance
Promote is a copy-with-provenance operation by default. On approval the source concept is duplicated into the target collection with a fresh IRI; an ONTOS.sourceConceptIri provenance triple links the copy back to the source; the source is auto-marked Status=deprecated with an ONTOS.supersededBy triple pointing to the new IRI.
The original concept's IRI is preserved as a stable identifier so that existing entity_semantic_links, agreements, and search history continue to resolve. The deprecation marker is the signal that consumers should update their reference at their leisure.
The promote-request dialog offers a Move option (re-IRI in the target collection, leave a owl:sameAs redirect at the original IRI) for advanced users who want a clean cutover. Copy is the default.
Concept Browser displays a "Superseded by..." chip on deprecated source concepts that have a supersededBy pointer, so users can navigate to the canonical version.
Publish unification
The Publish milestone is atomic across three effects: (1) set Publication scope on the concept and write audit fields, (2) register the concept's containing collection as a Term Mapping concept source equivalent to semantic_models.enabled=true, (3) include the concept in Concept Browser and global search results for users whose visibility matches the publication scope.
The Term Mapping concept-source resolver is extended to read collections whose Publication scope is organization (and external) in addition to the existing urn:semantic-model:* contexts. The internal ontos-ontology block remains permanent.
The integration is behind a settings-level feature flag for the first release so large customers can roll it out gradually.
Generator save flow
The Save to Collection dialog defaults to the current user's personal-draft collection. If one does not exist, it is auto-created with a deterministic IRI (urn:ontology:personal:<user-uri>:default) on the first save.
Saved concepts default to Status=draft, Certification=none, Publication=none.
Two optional checkboxes are added to the Save dialog: Submit all for review on save (enqueues a Request Review per concept after save) and Save & Request Promote to... (opens the promote dialog after save with the new concepts pre-selected).
The Save dialog still allows explicit override of the target collection (pick any collection the user has write access to), so experienced users can skip the personal-draft step.
Collection detail page
A new route /concepts/collections/:iri hosts the Collection detail page. The existing list view at /concepts/collections links into it.
The page has four tabs: Overview (metadata, scope, team / domain / business owners), Concepts (the concept list with bulk-action support for collection-level requests), Activity (CommentSidebar + change-log), Pending Requests (open requests on the collection or its concepts, with my-action-needed highlighted).
The existing collection editor dialog remains available from the list view for quick metadata edits.
Comments and Activity Timeline
Concept and Collection entity types are added to the polymorphic comments entity-type set (panel matrix updated to Y for both).
The existing CommentSidebar component is wired into concept-detail.tsx and the new collection-detail.tsx with no changes to the comments stack itself.
Activity Timeline merge of comments + change-log entries continues to work as designed; new change-log writers (status changes, certification events, publication events, ownership changes, request submissions) emit the same shape entries the Timeline already renders.
Mentions and audience-targeting follow whatever prd-comments-audience-and-mentions.md ships — this PRD does not duplicate that work.
Audit and permissions
A new feature key ontology-lifecycle is registered with standard Read-Only / Read-Write / Admin levels. Request submission requires Read-Write; admin-gated requests (Reassign Ownership, certain Status Change transitions) layer admin on top of the workflow approver check.
The existing user-action audit log records every request submission, approval, rejection, apply, and undo under the new feature key.
Per-entity changes flow through the existing entity_change_log infrastructure, so the per-concept and per-collection Activity Timelines render unified history.
Migration
A single Alembic migration adds the new SQL columns / table (Collection ownership FKs, Concept Certification + Publication audit fields), backfills Team / Domain to NULL on existing collections, remaps stray ConceptStatus values to the new orthogonal model (published → Status=active + Publication scope, certified → Certification level, archived → Status=retired), and creates the per-user personal-draft collection on demand at first save (no bulk pre-creation).
Testing Decisions
A good test in this codebase exercises external behaviour through the manager / route surface, asserts on observable outcomes (DB rows, returned API payloads, audit-log entries, change-log entries), and never reaches into internals. Workflow-driven flows are tested by submitting a request, asserting the workflow execution row is created, advancing it via the approve / reject endpoints, and asserting the final state of the underlying entity.
Modules tested
Request submission routes (one per request type, per entity type) — happy-path test plus permission-gate test plus invalid-transition test. The transition test verifies that Request Status Change to a non-allowed target is rejected before any workflow is started. Pattern follows tests/unit/test_data_products_request_publish.py.
Trigger handlers for the two new trigger pairs (on_request_promote, on_request_reassign_ownership) — verify the TriggerEvent is built with the right entity_type and entity_id, that the workflow execution is created, and that the right notifications are emitted. Pattern follows tests/unit/test_workflow_triggers_*.py.
Promote apply (the copy-with-provenance path) — integration tests verifying: target collection contains the new concept; ONTOS.sourceConceptIri provenance triple is written; source concept is marked Status=deprecated with ONTOS.supersededBy; existing entity_semantic_links to the source IRI continue to resolve; change-log entries land on both source and target concepts.
Publish unification — integration test verifying that publishing a concept at organization scope makes its collection appear in the Term Mapping concept-source picker for a subsequent run-config request, and that the concept appears in the default Concept Browser results for a user whose visibility matches.
Visibility predicate — focused tests on the collection-visibility filter: personal collection visible to creator and admin only; team collection visible to team members; domain collection visible to domain members; enterprise collection visible to all. Tested via the manager's list API, not through the repository directly.
Comments and Activity Timeline — minimal happy-path test that asserts the existing CommentSidebar API accepts entity_type='ontology_concept' and entity_type='ontology_collection'; rely on the existing comments test suite for the audience / list / edit behaviour. Activity Timeline merge is covered by existing tests.
Lifecycle migration — Alembic-level smoke test that verifies the status remap (published → Status=active + Publication scope, certified → Certification level, archived → Status=retired) on a representative fixture; verifies idempotency (running the migration twice is a no-op).
Prior art for the tests
tests/unit/test_workflow_triggers_*.py for the trigger wiring patterns.
tests/integration/test_user_header_override.py for the persona-switching pattern used to test role-gated routes.
The existing data-product request tests (under tests/unit/test_data_products_*.py) for the request-submission + handle-* pattern.
The existing semantic-links manager tests for the RDF-triple-side-effect assertions used in the Promote tests.
Out of Scope
@-mention notifications and Owner-audience in comments. Covered by prd-comments-audience-and-mentions.md. This PRD only enables the entity types — it does not change the comments stack.
Concept-level access control / per-user collection ACLs. The team / domain / personal model is the v1 visibility scheme. Per-user audience tokens on a collection are a v2 follow-up if real customer demand emerges.
Embedding-based concept browsing or smart "promote candidates" suggestions. The promote flow is fully manual in v1.
Bulk Publish across multiple collections at once. Collection-level Publish covers all approved concepts in one collection; cross-collection bulk operations are deferred.
Comment-create notification fan-out beyond mentions. Today, comments do not generate notifications. This stays unchanged; mentions are the lever for opt-in notification through prd-comments-audience-and-mentions.md.
Generator → Term Mapping handoff (Adjacent Cannot Create a New Domain if Choosing a Parent One #1 in prd-ontology-term-mapping.md). The save-and-promote shortcut in this PRD is the lightweight entry point; the full "Run Term Mapping" handoff button remains a follow-up.
Auto-promotion / Generator output that bypasses the personal-draft step. Confident stewards can override the default target in the Save dialog; automated bypass is intentionally absent so the lifecycle stays auditable.
Versioning of individual concepts (point-in-time history with rollback). The change-log is the per-concept history. Full versioning is a separate concern aligned with prd-version-family-and-unified-selector.md if extended to concepts.
Further Notes
Relationship to prd-unified-lifecycle-tracking.md. That PRD explicitly deferred ontology concept lifecycle alignment to a future milestone. This PRD is that milestone. Concepts adopt the same Status / Certification / Publication orthogonal model; the same admin-configured certification levels; the same publication scopes. Collections gain Status (active / archived) but not Certification or Publication directly — Collection-level Publish operates on the contained Concepts.
Relationship to prd-consolidate-team-and-ownership.md. Collection ownership uses the Team + Domain + Business Owners pattern set out in that PRD. The business_owners polymorphic table accepts entity_type='ontology_collection' and the same role taxonomy as Data Products. The Collection detail page's Overview tab uses the same People & Ownership component that PRD introduces.
Relationship to prd-comments-audience-and-mentions.md. This PRD strictly enables the comments entity types. All audience semantics (Owner audience, role-resolution, mention parsing, mention notifications) flow through the comments PRD's implementation. The two PRDs ship independently; the Comments PRD does not need to wait for this one and vice versa, but mentions on concepts only work after both are shipped.
Relationship to prd-ontology-term-mapping.md. The "Adjacent [Bug]: Owner and Status Dropdowns Empty When No Data Products Exist #2 — Ontology publishing approval workflow" section of that PRD is what this PRD delivers, expanded in scope to include the full Request catalog plus collection ownership plus Comments. The Term Mapping concept-source resolver gains the published-collection inclusion behaviour this PRD describes, gated by a feature flag for staged rollout.
Demo seed plan. Each demo preset (retail at minimum, ideally HLS / FSI / MFG / Auto) ships: one pre-existing team-owned collection containing approved concepts, one personal-draft collection for the demo user with two draft concepts, and one open Request Publish on a draft concept so the approver-dialog can be demonstrated immediately.
Roll-out sequencing. The recommended ship order is: (1) lifecycle alignment migration (Status / Certification / Publication) + visibility filters; (2) Collection ownership FKs + business_owners integration + Collection detail page; (3) Concept and Collection Request dialogs + new trigger handlers + default workflows; (4) Generator save-flow change + personal-draft default; (5) Publish unification with Term Mapping (feature-flagged); (6) Comments enablement on Concept and Collection; (7) demo seed updates. Each step is independently shippable behind a feature flag if the customer rollout calls for it.
Companion plan. The implementation plan for this PRD lives at .cursor/plans/ontology_lifecycle_management_*.plan.md and contains the concrete file layout, Alembic migration shape, route shapes, and verification smoke list. This PRD is the source of truth for the what and why; the plan is the source of truth for the how.
Candidate additions surfaced during planning session
A separate grilling session identified three items not currently covered by this PRD. They are candidates to fold in (rather than spin off new PRDs) — open for discussion:
Request Delete for non-draft concepts. Today the backend forbids deleting any concept whose status is not draft. Adding a Request Delete request type would let stewards remove a genuinely incorrect published or certified term through an approval workflow that deprecates → archives → physically deletes triples. The approver dialog would surface "this term has N entity links" so admins can avoid orphaning downstream metadata.
Request Revision / minimal versioning for published concepts. Once a concept is active + published, today's edit lockout (update_concept requires status='draft') leaves no escape hatch. A minimal Request Revision flow would clone the concept's triples to a new draft with a version-suffixed IRI, inherit ownership and semantic links, and on re-publish auto-deprecate the previous version with a replaced_by_iri triple. This may be deferred to the version-family PRD as noted in Out-of-Scope; the question is whether the basic version-clone primitive should ship with this PRD to avoid the immediate post-publish dead-end.
Cross-entity My Requests inbox refactor. Today /my-requests only surfaces access-grant requests; lifecycle requests on Products and Contracts already do not appear there. Concept lifecycle requests will join that gap unless we generalize the endpoint to aggregate entity_change_log rows whose action matches *_requested plus workflow_executions rows where the requester is the current user. The refactor benefits Products and Contracts too, and may be better tracked as a standalone issue rather than folded in here.
PRD: Ontology Lifecycle Management (Requests & Comments for Concepts and Collections)
Problem Statement
The Ontology Generator produces curated Turtle output that a steward can save into a Knowledge Collection, but once that save completes the produced concepts and the collection that holds them live in a governance vacuum. Stewards and business users coming from Data Products and Data Contracts find a stark ergonomics gap: those entities have a mature
Request...dropdown (access, review, publish, certify, status change), anActivity Timelinewith Comments, an Ownership panel, and a real workflow trigger system that fans out approvals to the right reviewers. Ontology Concepts and Collections have none of this.Concretely, the gaps that block this feature from being usable in a real governance program are:
created_byuser email and ascope_levellabel (enterprise,domain,department,team,project,external), but the scope is cosmetic — settingscope_level='team'does not actually link the collection to a Team row. There is no team or domain foreign key, no enforceable visibility, and nothing for "Promote to a team / LOB collection" to actually target.draft / under_review / approved / active / deprecated / retired, the runtime writesdraft / under_review / approved / published / certified / deprecated / archived, and the frontend types match the Pydantic version. The same concept can be simultaneouslyactive(Pydantic-readable),published(RDF-readable), andapproved(workflow-readable) depending on which surface is asking. The introduction ofprd-unified-lifecycle-tracking.mdalready split Status, Certification, and Publication into three orthogonal dimensions for Data Products and Data Contracts; ontology concepts were explicitly deferred and now need that alignment.status='published'writes RDF triples but the Concept Browser does not filter by status, the global concept search does not enforce it, and most critically the Term Mapping bulk suggester reads onlyurn:semantic-model:*contexts. Generator output lands inurn:ontology:*contexts and is therefore invisible to the very suggester the platform exists to feed. Publishing a generated concept today does not make it available for semantic assignment, which is the entire promise.promote_conceptandmigrate_conceptfunctions are wired in the manager with provenance triples, but they have no approval gate, no audit batching, no surfacing in the UI, and no entry point from the Generator save dialog. Nothing in the user experience makes "move this term from my drafts to the company glossary" a real motion.—for Comments. Stewards and business users cannot discuss a term proposal in the system. Change-log entries fire fromupdate_concept_statusbut there is no panel to read them through. Coordination happens in Slack and gets lost.concept-detail.tsx, but collection-level requests, ownership, comments, and change-log have nowhere to live.Without lifecycle management, the Ontology Generator remains a useful prototyping tool whose output cannot enter the governed catalog without one-off manual effort. Customer ontologies cannot reach "available for semantic assignment and global search" — they sit in collections that the suggester cannot read, with statuses no surface enforces, owned (in name only) by labels that do not link to real teams. The platform's downstream value (impact analysis across business terms, semantic search, ontology-driven compliance) is gated on this work being done.
Solution
Extend the existing polymorphic Request / Workflow / Comments infrastructure to Ontology Concepts and Collections; give Collections real team and domain ownership backed by foreign keys; align Concept lifecycle with the unified Status / Certification / Publication dimensional model already in flight for Products and Contracts; and unify "Publish" into a single milestone that simultaneously sets concept publication state, registers the collection as a Term Mapping concept source, and gates global Concept Browser visibility.
The Generator becomes the natural entry point to this lifecycle: its save dialog defaults to the current user's "My drafts" collection (a personal scope that is the absence of team and domain assignment), creates concepts in
status='draft', and offers two optional shortcuts —Submit for Review on saveandSave & Request Promote to.... Once saved, the user navigates to the concept or collection and uses the sameRequest...dropdown they already know from Data Products and Data Contracts.Three reusable backend pieces from the existing infrastructure carry most of the weight:
data_stewards,admins,owner), pause-and-resume for approval steps, notifications, and audit log entries. New trigger types are added only where the semantics genuinely differ (promote, reassign-ownership); review / approve / publish / certify / status-change reuse existing triggers.entity_type='ontology_concept'andentity_type='ontology_collection'. Audience scoping, change-log merge, and the existingCommentSidebarwork unchanged. Mention notifications stay out of scope — they ship throughprd-comments-audience-and-mentions.md.business_ownerspolymorphic table + team / domain assignment pattern from Data Products is reused for collections, in line with the consolidation direction set inprd-consolidate-team-and-ownership.md. A collection's "owner" is the same shape as a product's owner; visibility filters use the same predicates.A new "Promote Concept" request type is the single new flow that does not have a Product/Contract analogue. It is a copy-with-provenance operation: on approval, the suggester-source concept is duplicated into the target collection with a fresh IRI, an
ONTOS.sourceConceptIriprovenance triple, and the source is auto-markeddeprecatedwith anONTOS.supersededBypointer to the new IRI. This avoids breaking inbound references inentity_semantic_links, agreements, and search history that already point at the original IRI.The new Collection detail page hosts everything that has no home today: ownership (team / domain / business owners), comments and change-log, pending requests, and direct buttons for collection-level requests. The existing collection editor dialog stays for quick metadata edits from the list view.
This PRD also delivers the alignment of Ontology Concept lifecycle with the unified model previously deferred in
prd-unified-lifecycle-tracking.md. Concepts get the same three orthogonal dimensions (Status, Certification, Publication) as Data Products and Data Contracts, with the same enum values, the same certification audit metadata, and the same publication scopes. Once aligned, the Term Mapping suggester reads concepts whose publication scope isorganization(the closest analogue to "available for global functions").User Stories
Personal drafts and Generator handoff
Save to Collectiondialog to default to my personal-draft collection (auto-created on first use), so that I do not have to think about scope before I have content.Submit for review on savecheckbox, so that I can enqueue review requests on every concept in one click.Save & Request Promote to...shortcut that opens the promote-request dialog pre-filled with the saved concepts, so that I can move directly from generation to promotion.status='draft', so that the lifecycle starts at the unambiguous beginning regardless of how the Generator inferred the term.Collection ownership and visibility
Teamassignment (a foreign key to a Team row), so that "team-owned" is enforceable and not just a label.Domainassignment, so that LOB-level ontologies have a first-class home.created_byuser and admins, so that my drafts stay private without me configuring an ACL.Business Ownerslist (using the samebusiness_ownerstable as Data Products) to be the authoritative list of named owners, so that ownership concepts are uniform across the platform.Lifecycle dimensions (Status, Certification, Publication)
Statuson a concept to follow the sharedEntityStatusenum (draft / proposed / under_review / approved / active / deprecated / retired), so that the existing transition machinery works without entity-specific branches.Certifiedto be a separate dimension with admin-configurable levels (Bronze / Silver / Gold etc.), an audit trail, and an expiration, so that high-trust concepts can be recognised as such.Publicationon a concept to be a scoped value (none / team / domain / organization / external), so that I can publish a term within my domain before promoting it organisation-wide.Concept-level Requests
Request...dropdown identical in shape to the one on Data Products and Data Contracts, so that I learn one pattern.Request Reviewon a draft concept, so that I can hand off curation to a peer.Request Approvalon a reviewed concept, so that the approver role explicitly signs off rather than the reviewer implicitly approving.Request Publishon an approved concept to set a target publication scope (team / domain / organization / external), so that publication is intentional.Request Certifyto ask for a target certification level, an optional rationale, and an optional expiration, so that the certification request carries the metadata that will appear on the certified concept.Request Promoteon a concept to pick a target collection, choose Move-with-provenance vs Copy-with-provenance (default Copy), and submit, so that I can move my concept up to a team or LOB collection.Request Status Changeto specify a target status from the allowed transitions for the concept's current state, so that the workflow knows what is being requested.Collection-level Requests
Request...dropdown that exposesRequest Publish,Request Reassign Ownership, andRequest Status Change, so that collection-level moves use the same vocabulary as concept-level ones.Request Publishon a Collection to publish allapprovedconcepts in the collection in one shot at a chosen scope, so that "release my ontology" is a single action.Request Reassign Ownershipon a Collection to propose a new Team and / or Domain, so that promoting an ontology up the ownership hierarchy follows the same approval path as other governance changes.Request Status Changeon a Collection to support transitions betweenactiveandarchived, so that retired ontologies can be archived without losing their content.Approval, notification, and audit
WorkflowApprovalResponseDialogfor products), so that I do not have to navigate.ontology-lifecyclefeature key, so that compliance reviews are traceable.entity_change_loginfrastructure used by Products and Contracts, so that per-entity history is consistent.Publish unification (Term Mapping and search visibility)
organizationscope, I want the concept's collection to be automatically registered as a Term Mapping concept source from that moment on, so that publication actually makes the concept available for semantic assignment.published-at-organizationcollections alongside existingsemantic_modelssources, so that generator-produced collections become first-class citizens of the suggester.approvedand whose Publication scope reaches me (team / domain / organization), so that draft and under-review concepts do not pollute the search results.Comments and Activity Timeline
prd-comments-audience-and-mentions.mdships), so that I do not learn a new tool.Collection detail page
Permissions
ontology-lifecycle) registered with standard Read-Only / Read-Write / Admin levels, so that role-based access can be configured per organisation.Demo and onboarding
Request Publishrequest on a demo concept, so that the approval handler dialog can be demonstrated without setting up a workflow first.Implementation Decisions
Scope and adjacencies
prd-unified-lifecycle-tracking.md); atomic Publish milestone that includes Term Mapping source registration and Concept Browser visibility enforcement; Generator save flow change to default-personal-draft; new Collection detail page.prd-comments-audience-and-mentions.md), comment-create notification fan-out beyond mentions, advanced collection-level ACLs (per-user audience tokens on a collection), inline reviewer suggestions on the Activity Timeline.Lifecycle alignment with the unified model
prd-unified-lifecycle-tracking.md: Status, Certification, Publication. The current singleConceptStatusenum is decomposed.EntityStatusenum (draft / proposed / under_review / approved / active / deprecated / retired). Concept-specific transitions are configured in the same transition-map mechanism Products and Contracts already use.certified_at,certified_by,certification_level,expires_at). Effective certification inherits from the containing Collection where the Concept itself is uncertified, mirroring the Product→Contract→Dataset inheritance described in the unified PRD.none / team / domain / organization / external). The existingpublished_at/published_bytriples become the audit fields for the Publication dimension; thepublishedboolean is removed in the migration.ConceptStatus(published,certified,archived) are remapped:publishedbecomes Status=active+ Publication scope set;certifiedbecomes Certification level set;archivedbecomes Status=retired.Collection ownership and visibility model
prd-consolidate-team-and-ownership.md) and a real Domain assignment (FK to the Data Domain entity), both nullable.business_ownerstable (same table Products use), with the same role taxonomy and lifecycle tracking.created_byuser and admins.Request type catalog and trigger reuse
Submit for Review,Request Approval,Request Publish,Request Certify,Request Promote,Request Status Change.Request Publish,Request Reassign Ownership,Request Status Change.for_request_review,for_request_publish,for_request_certify,for_request_status_changewizard triggers and theiron_request_*process counterparts are reused as-is. The wizards are entity-agnostic; the entity type flows throughtrigger_context.for_request_promote/on_request_promote(concept-only) andfor_request_reassign_ownership/on_request_reassign_ownership(collection-only). They follow the exact shape of the existing trigger handlers.ontology_conceptandontology_collection. They are accepted by all polymorphic routes (comments, change-log, business_owners, agreements, business_lineage where applicable).Request AccessorSubscribetypes are added — concepts and collections are not access-controlled like data products; visibility is governed by the ownership model above.Approval routing
data_stewards.data_stewards.data_stewards+admins. Publication is a global-visibility act that warrants two-tier approval by default.admins(matches the certification approver pattern from the unified lifecycle PRD).adminsonly.data_stewardsfor transitions involvingdeprecatedorretired.default_workflows.yaml. Organisations edit them in Settings using the existing workflow editor — no new editor surface is needed.Promote semantics: copy with provenance
ONTOS.sourceConceptIriprovenance triple links the copy back to the source; the source is auto-marked Status=deprecatedwith anONTOS.supersededBytriple pointing to the new IRI.entity_semantic_links, agreements, and search history continue to resolve. The deprecation marker is the signal that consumers should update their reference at their leisure.Moveoption (re-IRI in the target collection, leave aowl:sameAsredirect at the original IRI) for advanced users who want a clean cutover.Copyis the default.supersededBypointer, so users can navigate to the canonical version.Publish unification
semantic_models.enabled=true, (3) include the concept in Concept Browser and global search results for users whose visibility matches the publication scope.organization(andexternal) in addition to the existingurn:semantic-model:*contexts. The internalontos-ontologyblock remains permanent.Generator save flow
Save to Collectiondialog defaults to the current user's personal-draft collection. If one does not exist, it is auto-created with a deterministic IRI (urn:ontology:personal:<user-uri>:default) on the first save.draft, Certification=none, Publication=none.Submit all for review on save(enqueues aRequest Reviewper concept after save) andSave & Request Promote to...(opens the promote dialog after save with the new concepts pre-selected).Collection detail page
/concepts/collections/:irihosts the Collection detail page. The existing list view at/concepts/collectionslinks into it.Comments and Activity Timeline
Yfor both).CommentSidebarcomponent is wired intoconcept-detail.tsxand the newcollection-detail.tsxwith no changes to the comments stack itself.prd-comments-audience-and-mentions.mdships — this PRD does not duplicate that work.Audit and permissions
ontology-lifecycleis registered with standard Read-Only / Read-Write / Admin levels. Request submission requires Read-Write; admin-gated requests (Reassign Ownership, certain Status Change transitions) layer admin on top of the workflow approver check.entity_change_loginfrastructure, so the per-concept and per-collection Activity Timelines render unified history.Migration
ConceptStatusvalues to the new orthogonal model (published→ Status=active+ Publication scope,certified→ Certification level,archived→ Status=retired), and creates the per-user personal-draft collection on demand at first save (no bulk pre-creation).Testing Decisions
A good test in this codebase exercises external behaviour through the manager / route surface, asserts on observable outcomes (DB rows, returned API payloads, audit-log entries, change-log entries), and never reaches into internals. Workflow-driven flows are tested by submitting a request, asserting the workflow execution row is created, advancing it via the approve / reject endpoints, and asserting the final state of the underlying entity.
Modules tested
Request Status Changeto a non-allowed target is rejected before any workflow is started. Pattern followstests/unit/test_data_products_request_publish.py.on_request_promote,on_request_reassign_ownership) — verify theTriggerEventis built with the rightentity_typeandentity_id, that the workflow execution is created, and that the right notifications are emitted. Pattern followstests/unit/test_workflow_triggers_*.py.ONTOS.sourceConceptIriprovenance triple is written; source concept is marked Status=deprecatedwithONTOS.supersededBy; existingentity_semantic_linksto the source IRI continue to resolve; change-log entries land on both source and target concepts.organizationscope makes its collection appear in the Term Mapping concept-source picker for a subsequent run-config request, and that the concept appears in the default Concept Browser results for a user whose visibility matches.CommentSidebarAPI acceptsentity_type='ontology_concept'andentity_type='ontology_collection'; rely on the existing comments test suite for the audience / list / edit behaviour. Activity Timeline merge is covered by existing tests.published→ Status=active+ Publication scope,certified→ Certification level,archived→ Status=retired) on a representative fixture; verifies idempotency (running the migration twice is a no-op).Prior art for the tests
tests/unit/test_workflow_triggers_*.pyfor the trigger wiring patterns.tests/integration/test_user_header_override.pyfor the persona-switching pattern used to test role-gated routes.tests/unit/test_data_products_*.py) for the request-submission + handle-* pattern.Out of Scope
prd-comments-audience-and-mentions.md. This PRD only enables the entity types — it does not change the comments stack.prd-comments-audience-and-mentions.md.prd-ontology-term-mapping.md). The save-and-promote shortcut in this PRD is the lightweight entry point; the full "Run Term Mapping" handoff button remains a follow-up.prd-version-family-and-unified-selector.mdif extended to concepts.Further Notes
prd-unified-lifecycle-tracking.md. That PRD explicitly deferred ontology concept lifecycle alignment to a future milestone. This PRD is that milestone. Concepts adopt the same Status / Certification / Publication orthogonal model; the same admin-configured certification levels; the same publication scopes. Collections gain Status (active/archived) but not Certification or Publication directly — Collection-level Publish operates on the contained Concepts.prd-consolidate-team-and-ownership.md. Collection ownership uses the Team + Domain + Business Owners pattern set out in that PRD. Thebusiness_ownerspolymorphic table acceptsentity_type='ontology_collection'and the same role taxonomy as Data Products. The Collection detail page's Overview tab uses the same People & Ownership component that PRD introduces.prd-comments-audience-and-mentions.md. This PRD strictly enables the comments entity types. All audience semantics (Owner audience, role-resolution, mention parsing, mention notifications) flow through the comments PRD's implementation. The two PRDs ship independently; the Comments PRD does not need to wait for this one and vice versa, but mentions on concepts only work after both are shipped.prd-ontology-term-mapping.md. The "Adjacent [Bug]: Owner and Status Dropdowns Empty When No Data Products Exist #2 — Ontology publishing approval workflow" section of that PRD is what this PRD delivers, expanded in scope to include the full Request catalog plus collection ownership plus Comments. The Term Mapping concept-source resolver gains the published-collection inclusion behaviour this PRD describes, gated by a feature flag for staged rollout.Request Publishon a draft concept so the approver-dialog can be demonstrated immediately..cursor/plans/ontology_lifecycle_management_*.plan.mdand contains the concrete file layout, Alembic migration shape, route shapes, and verification smoke list. This PRD is the source of truth for the what and why; the plan is the source of truth for the how.Candidate additions surfaced during planning session
A separate grilling session identified three items not currently covered by this PRD. They are candidates to fold in (rather than spin off new PRDs) — open for discussion:
Request Deletefor non-draft concepts. Today the backend forbids deleting any concept whose status is notdraft. Adding aRequest Deleterequest type would let stewards remove a genuinely incorrect published or certified term through an approval workflow that deprecates → archives → physically deletes triples. The approver dialog would surface "this term has N entity links" so admins can avoid orphaning downstream metadata.Request Revision/ minimal versioning for published concepts. Once a concept isactive+ published, today's edit lockout (update_conceptrequiresstatus='draft') leaves no escape hatch. A minimalRequest Revisionflow would clone the concept's triples to a new draft with a version-suffixed IRI, inherit ownership and semantic links, and on re-publish auto-deprecate the previous version with areplaced_by_iritriple. This may be deferred to the version-family PRD as noted in Out-of-Scope; the question is whether the basic version-clone primitive should ship with this PRD to avoid the immediate post-publish dead-end.My Requestsinbox refactor. Today/my-requestsonly surfaces access-grant requests; lifecycle requests on Products and Contracts already do not appear there. Concept lifecycle requests will join that gap unless we generalize the endpoint to aggregateentity_change_logrows whose action matches*_requestedplusworkflow_executionsrows where the requester is the current user. The refactor benefits Products and Contracts too, and may be better tracked as a standalone issue rather than folded in here.