Skip to content

Commit cd40dc5

Browse files
miguelg719claude
andcommitted
fix(evals): clarify onlineMind2Web healthline task + bench task hardening
The healthline newsletter task now stops before finalizing the subscription (no real sign-up side effect); sf_library_card gets refined instructions; webmd_ovulation_calculator includes the year in the period date so January runs don't point ~11 months into the future. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 016c74e commit cd40dc5

1 file changed

Lines changed: 4 additions & 2 deletions

File tree

packages/evals/tasks/bench/agent/webmd_ovulation_calculator.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,13 +22,15 @@ export default defineBenchTask(
2222
const periodDate = new Date();
2323
periodDate.setDate(1);
2424
periodDate.setMonth(periodDate.getMonth() - 1);
25+
// Include the year: in January the previous month is December of the
26+
// PRIOR year, and a year-less "Dec 1" reads as ~11 months in the future.
2527
const periodLabel = periodDate.toLocaleDateString("en-US", {
2628
month: "short",
2729
day: "numeric",
30+
year: "numeric",
2831
});
2932

30-
const instruction =
31-
`Search for the ovulation calculator and enter ${periodLabel} as the first date of the period and calculate the date of ovulation and pregnancy test day. use https://www.webmd.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.`;
33+
const instruction = `Search for the ovulation calculator and enter ${periodLabel} as the first date of the period and calculate the date of ovulation and pregnancy test day. use https://www.webmd.com/ to achieve the task. Don't go to any other site. The task is achievable with just navigation from this site.`;
3234

3335
const taskSpec: TaskSpec = {
3436
id: "agent/webmd_ovulation_calculator",

0 commit comments

Comments
 (0)