Skip to content

Add executable OKF conformance check against reference tooling #82

Description

@ajdelaguila

Summary

README.md, SPEC.md, and WHY.md all claim data-olympus is an "OKF-compatible profile" ("any OKF consumer can read a data-olympus bundle unchanged"). Today that claim has zero executable backing: there is no test that runs a real OKF reference consumer/producer against a data-olympus bundle, and no dependency on OKF reference tooling anywhere in this repo.

As part of WP4c (0.3.0 spec/example-bundle/CI work), the strongest wording of this claim was softened in README.md, SPEC.md, and WHY.md to "designed to be readable by OKF consumers (formal conformance testing tracked in this issue)" rather than removing the claim outright, since directional intent (build on OKF's baseline rather than around it) is accurate and the two formats do genuinely share directory structure, frontmatter conventions, reserved filenames, and link model by construction.

What's missing, concretely:

  • No dependency on or vendoring of the OKF reference implementation (https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf).
  • No test that feeds a data-olympus bundle (e.g. example-bundle/) through an actual OKF consumer and asserts it parses without error.
  • No test that feeds an OKF-producer-generated bundle through kb lint / the indexer and asserts it is at least readable (even if not immediately conformant to data-olympus's stricter governance schema).
  • The only structural floor that exists today (added alongside this issue) is a cheap, non-behavioral check that every example-bundle concept doc has non-empty id/type, and that the bundle-root index.md declares okf_version — this proves frontmatter shape, not that any OKF tool can actually read the bundle. See tests/test_okf_minimal_fields.py for the exact scope and its documented "does not prove" list.

Proposed follow-up

  1. Vendor or dependency-pin a minimal OKF reference consumer (whatever the OKF repo ships, if anything runnable) or hand-write a strict-baseline OKF parser matching the published spec.
  2. Add a CI job that runs that consumer against example-bundle/ and fails on any parse error.
  3. If OKF ships a reference producer, generate a bundle with it and confirm data-olympus lint/data-olympus index handle it without crashing (governance-field warnings are expected and fine; crashes are not).
  4. Update SPEC.md section 11 and WHY.md to cite the new conformance test directly once it exists, and restore the stronger compatibility wording only once it is backed by that test.

Context

Filed as part of data-olympus 0.3.0 work package WP4c (format differentiators: specified, demonstrated, CI-guarded).

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority: highHigh priority; pull from the queue ASAP

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions