cardea

Cardea is a web application for extracting domain-specific subgraphs from massive knowledge graphs and applying graph-based semantic similarity measures. It uses predicate metrics to guide Derzis, a semantic web crawler, to crawl only the relevant portions of a knowledge graph — requiring no prior knowledge of the graph's ontology or vocabulary.

sequenceDiagram
    Actor u as User
    participant c as Cardea
    participant d as Derzis

    u->>+c: newSubgraph(seeds)
    c->>+d: newProcess(seeds)

    d->>d: crawl(MPL=2,MPP=1)

    d->>-c: CSG1 + predMetrics1
    c->>u: showResults(CSG1, predMetrics1)

    loop

        u->>c: addStep(predLimitsN)
        c->>+d: addStep(predLimitsN)
        d->>d: crawl(predLimitsN)

        d->>-c: CSG2 + predMetricsN
        c->>u: showResults(CSGN, predMetricsN)
    end
    deactivate c

Loading

Features

• Subgraph extraction — crawl a knowledge graph via its linked data interface, guided by predicate metrics (seed coverage, branching factor, component analysis, cycle detection, etc.) to extract a domain-relevant subgraph.
• Scored predicate pairs — at each crawl step, pairs of entry and hierarchical predicates are scored using a function based on average component height, predicate-pair seed interaction, and entry predicate relative frequency.
• Semantic similarity measures — 15 graph-based measures (path-based, feature-based, and IC-based) are applied to the extracted subgraph.
• LLM evaluation — compare graph-based measure results against ratings from 8 large language models (Anthropic Claude, OpenAI GPT, Mistral, Google Gemini).
• Human dataset correlation — evaluate against gold-standard datasets (RG65, MESH2, MiniMSRS, GERESID, Zie55, SemEval17) when available.

Architecture

Cardea is a SvelteKit application (Node.js adapter) that orchestrates several services:

Service	Role
Cardea (this app)	Web UI + HTTP API, orchestrates crawling, calculates metrics, applies SMs
Derzis	Semantic web crawler — performs the actual graph traversal
MongoDB	Primary data store: subgraphs, triples, metrics, evaluations
Apache Jena Fuseki	SPARQL endpoint for predicate-pair seed interaction queries
Redis	Pub/sub between Derzis manager and workers

See docker-compose.yml for the full service topology.

Prerequisites

• Node.js >= 22
• MongoDB (local or Docker)
• Apache Jena Fuseki (local or Docker)
• Derzis (manager + worker)
• Redis (for Derzis pub/sub)

Quick Start (Local Development)

# Install dependencies
npm install

# Copy environment file and adjust as needed
cp .env.example .env

# Start the dev server
npm run dev

The server starts on http://localhost:5173 by default (configured via Vite). The app expects MongoDB, Fuseki, and Derzis to be running and reachable at the hosts/ports specified in .env.

Docker Setup

A complete Docker Compose environment is provided in docker-compose.yml:

# Start all services (Cardea, Derzis, MongoDB, Fuseki, Redis)
docker compose up

By default, services use the DC_* environment variables (ports 60001–60005).

Configuration

Copy .env.example to .env and configure:

Variable	Description
MONGO_HOST / MONGO_PORT	MongoDB connection
DERZIS_MNG_HOST / DERZIS_MNG_PORT	Derzis manager API
FUSEKI_HOST / FUSEKI_PORT	Apache Jena Fuseki
PUBLIC_MODE_TAG	Display tag for the deployment mode
LOG_LEVEL	Pino log level (default: debug)
ANTHROPIC_API_KEY / OPENAI_API_KEY / MISTRAL_API_KEY / GOOGLE_API_KEY	LLM API keys for evaluation
IAEDU_API_URL / IAEDU_API_KEY / IAEDU_CHANNEL_ID	IAedu service (free GPT-4o access for Portuguese researchers)
MAILTRAP_API_TOKEN	Email testing
PRED_BF_NEUTRAL_MIN / PRED_BF_NEUTRAL_MAX	Branching factor neutral zone for predicate direction

Development

npm run dev          # Start Vite dev server (with 8 GB heap)
npm run build        # Production build
npm run preview      # Preview production build
npm run check        # Type checking (svelte-check)
npm run format       # Format with Prettier
npm run lint         # Lint with ESLint + Prettier check
npm run storybook    # Component browser (Storybook on port 6006)

Database Scripts

npm run db:seed:dev      # Seed development data
npm run db:drop:dev      # Drop development database
npm run db:seed:dc       # Seed Docker environment
npm run db:drop:dc       # Drop Docker environment database
npm run db:backup:dev    # Backup dev database
npm run db:restore:dev   # Restore dev database

Testing

npm run test:unit          # Vitest unit tests (watch mode)
npm run test:unit:run      # Vitest unit tests (single run)
npm run test:e2e           # Playwright end-to-end tests
npm run test               # Full test suite (unit + e2e)

Project Structure

src/
├── lib/
│   ├── api/               # API route handlers
│   ├── components/        # Svelte components (UI, subgraph, FASM)
│   ├── correlation/       # Correlation analysis
│   ├── derzis/            # Derzis API client
│   ├── fasm/              # FASM normalization
│   ├── fuseki/            # Fuseki SPARQL client
│   ├── graph/             # Graph analysis utilities
│   ├── models/            # Mongoose models (SubGraph, FASM, Triple, FasmEval)
│   ├── predicates/        # Predicate pair metrics & scoring
│   ├── server/            # Server-side utilities
│   ├── stores/            # Svelte stores
│   ├── styles/            # Global styles
│   ├── triples/           # Triple processing
│   ├── utils/             # Shared utilities
│   ├── workers/           # Background worker logic
│   ├── webhook/           # Webhook handler
│   ├── cardea-info.ts     # Cardea server info
│   ├── connect-db.ts      # MongoDB connection
│   ├── logger.ts          # Pino logger
│   ├── model-config.ts    # LLM model configuration
│   └── types.ts           # Shared TypeScript types
├── routes/
│   ├── api/               # REST API endpoints
│   ├── docs/              # API documentation
│   ├── fasms/             # FASM management
│   ├── info/              # System info
│   └── subgraphs/         # Subgraph creation & exploration
└── scripts/               # CLI scripts (DB seed, migration, etc.)

The Cardea Process

1. Submit seeds — provide a set of resource IRIs relevant to your domain.
2. Initial crawl — Derzis dereferences the seeds with default parameters (MPL=2, MPP=1).
3. Predicate analysis — Cardea calculates metrics for all predicate pairs and scores them.
4. Iterative refinement — add steps with predicate limitations to follow only the most promising predicates.
5. Subgraph extraction — once a common ancestor is found (or an artificial root is added), build the final subgraph.
6. Evaluation — apply semantic measures, query LLMs, and compare against human datasets.

Related Projects

• Derzis — Semantic web crawler used by Cardea.
• sparql-to-ld — Translates HTTP requests into SPARQL queries to serve linked data from a local Fuseki instance.
• cardea-eval-vis — Interactive visualization of evaluation results.

License

The MIT License (MIT)

Copyright (c) 2024 André Santos andrefs@andrefs.com

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.