Skip to content

SWE-Agent: Implement the hallucinations metric from the main branch in the workflow branch #298

Description

@naddeoa

A lot of things have changed between the main branch and the workflow branch. It reimplements all of the metrics from the main branch with a different Workflow, Metric based interface. One of the metrics that have not been ported over yet is Hallucination:

This needs to be ported to the workflow branch and implemented like the other metrics in the workflow brach. Some examples of how that looks:

The metric python module is full of other metrics too.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions