Skip to content

Commit 44d9963

Browse files
daveshenaldaveshenal
authored
Dev (#73)
* Add transition and remove chat list styles Update dummy-ui/src/index.css: add a 0.3s transition to the body::before rule and remove the now-redundant .message transition block. Also remove specific .chat-messages ul/ol and li styles and the blockquote padding-left to simplify and avoid duplicate/conflicting message animations and list formatting. This is a CSS cleanup to consolidate animations and rely on global/list styles elsewhere. * Add typing for job status and request bodies Introduce stronger TypeScript typings across several components and the API service to improve type safety. Changes include: - Add a JobStatus interface and use it for jobStatus state in DocPlanGenerator and EduflowGenerator; type docs items in map callbacks. - Default prompt template types in PromptsEditor by casting to PromptTemplate['type'] and falling back to 'main' when missing. - Add EduflowJobBody, BaselineJobBody, and MemoryJobBody interfaces in services/api.ts and replace loose any bodies with these concrete types. - Remove an unused Promise reject parameter in an upload helper (change to only use resolve) to avoid an unused variable. These updates are intended to reduce runtime errors and make the codebase easier to maintain by clarifying expected shapes of job payloads and statuses. * Remove index_manager and refine UI components Remove standalone FastAPI index_manager.py and apply small UI component refactors in dummy-ui: loosen doc item typing to `any` in DocPlanGenerator and EduflowGenerator, rename unused map iterator `index` to `_index` in Dropdown to satisfy linter, and change `let updated` to `const updated` in PromptsEditor to enforce immutability. These are minor type/lint cleanups and removal of an unused standalone app. * Quiet wget, disable uvicorn entries, fix Dropdown Add -q to the Dockerfile wget command to suppress download output; comment out the standalone uvicorn startup blocks in app/main.py, evaluator_service/main.py, and extra/user_backend.py to avoid running servers on import (expecting an external runner/process manager); and remove the unused _index parameter from the options.map in dummy-ui/src/components/Dropdown.tsx to clean up unused variable warnings. * Use mkdir -p for /app/certs in Dockerfile Replace mkdir -q with mkdir -p when creating /app/certs to make the step idempotent and avoid errors if the directory already exists. This improves build stability across Docker layers; wget command and other lines remain unchanged. * fixed Unexpected any. Specify a different type error * Add useDocJobPolling and update doc generators Introduce a new useDocJobPolling hook to centralize polling logic for document job status. DocMemGenerator.tsx and DocPlanGenerator.tsx were updated to use the new hook and simplify their job status handling. package.json was updated to add node-forge and include dependency version bumps to support the changes. * Split semantic_match into focused private helpers in Hybrid concept matching script * Use ephemeral cache and add reports & extras Set cache_control={"type": "ephemeral"} on Claude client message calls across several services (content, curriculum plan, voice script, and chat pipeline) to avoid caching responses. Add security and linting artifacts: new docs/bandit_report.html, move pylint_report.txt into docs/, and include pylint_report.txt in the deploy workflow file watch list. Add extra utilities (extra/load-test.js, extra/print_pylint.py) and update the evaluator notebook (execution counts and a new stats cell) for analysis convenience. * Add faithfulness evaluator and analysis Introduce faithfulness evaluation features: add a new evaluator (evaluator_service/evaluators/faithfulness_evaluator.py) and an API route (evaluator_service/routes/faithfulness.py), plus a notebook for analysis (evaluator_service/evaluation/faithfulness_analysis.ipynb). Update existing evaluation notebooks (automated_llm_evals.ipynb, downlaod_blos.ipynb) to include additional jobs and captured outputs, and refresh evaluation_results.xlsx. Update Dockerfile to COPY app so the faithfulness code and its imports are available in the image. Minor housekeeping: fix evaluator_service/evaluation/downloaded_jobs path in .gitignore and update main.py and requirements.txt to wire up the new evaluator/route. * Update anthropic dependency version in requirements.txt to allow for newer releases (>=0.64.0).
1 parent 9e84875 commit 44d9963

11 files changed

Lines changed: 1176 additions & 17 deletions

‎.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,5 +20,5 @@ certs/DigiCertGlobalRootCA.crt.pem
2020
/.vscode
2121

2222
# Downloaded ai-saves
23-
/evaluator_service/evaluation/downloaded_jobs
23+
/evaluator_service/evaluation/downloaded_jobs/
2424
evaluator_service/evaluation/~$evaluation_results.xlsx

‎evaluator_service/Dockerfile‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,10 @@ ENV PYTHONDONTWRITEBYTECODE=1 \
1111
COPY evaluator_service/requirements.txt ./requirements.txt
1212
RUN pip install --no-cache-dir -r requirements.txt
1313

14-
# Copy the evaluator service and shared config it imports.
14+
# Copy the evaluator service, shared config, and app retriever code used by faithfulness.
1515
COPY evaluator_service ./evaluator_service
1616
COPY config ./config
17+
COPY app ./app
1718

1819
EXPOSE 8002
1920

‎evaluator_service/evaluation/automated_llm_evals.ipynb‎

Lines changed: 197 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -463,11 +463,206 @@
463463
"wb.save(EXCEL_PATH)\n",
464464
"print(\"\\Done! All pairwise comparisons complete.\")"
465465
]
466+
},
467+
{
468+
"cell_type": "code",
469+
"execution_count": 1,
470+
"id": "848d2fae",
471+
"metadata": {},
472+
"outputs": [
473+
{
474+
"name": "stdout",
475+
"output_type": "stream",
476+
"text": [
477+
"\n",
478+
"[1/18] EduFlow — job-1774534470161\n",
479+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774534470161\\eval_results\\faithfulness.json\n",
480+
" Score: 3.75\n",
481+
" Sleeping 60s...\n",
482+
"\n",
483+
"[2/18] EduFlow — job-1774545286253\n",
484+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774545286253\\eval_results\\faithfulness.json\n",
485+
" Score: 3.5\n",
486+
" Sleeping 60s...\n",
487+
"\n",
488+
"[3/18] EduFlow — job-1774548177351\n",
489+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774548177351\\eval_results\\faithfulness.json\n",
490+
" Score: 3.5\n",
491+
" Sleeping 60s...\n",
492+
"\n",
493+
"[4/18] EduFlow — job-1774574045744\n",
494+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774574045744\\eval_results\\faithfulness.json\n",
495+
" Score: 3.25\n",
496+
" Sleeping 60s...\n",
497+
"\n",
498+
"[5/18] EduFlow — job-1774590551422\n",
499+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774590551422\\eval_results\\faithfulness.json\n",
500+
" Score: 4.0\n",
501+
" Sleeping 60s...\n",
502+
"\n",
503+
"[6/18] EduFlow — job-1774596210500\n",
504+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774596210500\\eval_results\\faithfulness.json\n",
505+
" Score: 4.25\n",
506+
" Sleeping 60s...\n",
507+
"\n",
508+
"[7/18] Baseline — job-1774534999912\n",
509+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774534999912\\eval_results\\faithfulness.json\n",
510+
" Score: 4.5\n",
511+
" Sleeping 60s...\n",
512+
"\n",
513+
"[8/18] Baseline — job-1774544649734\n",
514+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774544649734\\eval_results\\faithfulness.json\n",
515+
" Score: 4.75\n",
516+
" Sleeping 60s...\n",
517+
"\n",
518+
"[9/18] Baseline — job-1774545754239\n",
519+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774545754239\\eval_results\\faithfulness.json\n",
520+
" Score: 4.0\n",
521+
" Sleeping 60s...\n",
522+
"\n",
523+
"[10/18] Baseline — job-1774547498894\n",
524+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774547498894\\eval_results\\faithfulness.json\n",
525+
" Score: 4.0\n",
526+
" Sleeping 60s...\n",
527+
"\n",
528+
"[11/18] Baseline — job-1774548651030\n",
529+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774548651030\\eval_results\\faithfulness.json\n",
530+
" Score: 3.75\n",
531+
" Sleeping 60s...\n",
532+
"\n",
533+
"[12/18] Baseline — job-1774549026730\n",
534+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774549026730\\eval_results\\faithfulness.json\n",
535+
" Score: 3.75\n",
536+
" Sleeping 60s...\n",
537+
"\n",
538+
"[13/18] Baseline — job-1774574325545\n",
539+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774574325545\\eval_results\\faithfulness.json\n",
540+
" Score: 3.75\n",
541+
" Sleeping 60s...\n",
542+
"\n",
543+
"[14/18] Baseline — job-1774574626655\n",
544+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774574626655\\eval_results\\faithfulness.json\n",
545+
" Score: 3.5\n",
546+
" Sleeping 60s...\n",
547+
"\n",
548+
"[15/18] Baseline — job-1774584736262\n",
549+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774584736262\\eval_results\\faithfulness.json\n",
550+
" Score: 4.0\n",
551+
" Sleeping 60s...\n",
552+
"\n",
553+
"[16/18] Baseline — job-1774584969330\n",
554+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774584969330\\eval_results\\faithfulness.json\n",
555+
" Score: 4.0\n",
556+
" Sleeping 60s...\n",
557+
"\n",
558+
"[17/18] Baseline — job-1774595522117\n",
559+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774595522117\\eval_results\\faithfulness.json\n",
560+
" Score: 4.25\n",
561+
" Sleeping 60s...\n",
562+
"\n",
563+
"[18/18] Baseline — job-1774596683553\n",
564+
" Saved → C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\\job-1774596683553\\eval_results\\faithfulness.json\n",
565+
" Score: 4.0\n",
566+
"\n",
567+
"All faithfulness evaluations complete.\n"
568+
]
569+
}
570+
],
571+
"source": [
572+
"import requests\n",
573+
"import time\n",
574+
"import os\n",
575+
"import json\n",
576+
"\n",
577+
"# ==== CONFIG ====\n",
578+
"BASE_URL = \"http://localhost:8002/evaluate/faithfulness\"\n",
579+
"BASE_PATH = r\"C:/Users/daves/Desktop/EduFlow/eduflow/evaluator_service/evaluation/downloaded_jobs\"\n",
580+
"\n",
581+
"# EduFlow jobs → use_input_param_queries = False\n",
582+
"EDUFLOW_JOBS = [\n",
583+
" (\"job-1774534470161\", \"0001\"),\n",
584+
" (\"job-1774545286253\", \"0001\"),\n",
585+
" (\"job-1774548177351\", \"0001\"),\n",
586+
" (\"job-1774574045744\", \"0001\"),\n",
587+
" (\"job-1774590551422\", \"0001\"),\n",
588+
" (\"job-1774596210500\", \"0001\"),\n",
589+
"]\n",
590+
"\n",
591+
"# Baseline jobs → use_input_param_queries = True\n",
592+
"BASELINE_JOBS = [\n",
593+
" (\"job-1774534999912\", \"0001\"),\n",
594+
" (\"job-1774544649734\", \"0001\"),\n",
595+
" (\"job-1774545754239\", \"0001\"),\n",
596+
" (\"job-1774547498894\", \"0001\"),\n",
597+
" (\"job-1774548651030\", \"0001\"),\n",
598+
" (\"job-1774549026730\", \"0001\"),\n",
599+
" (\"job-1774574325545\", \"0001\"),\n",
600+
" (\"job-1774574626655\", \"0001\"),\n",
601+
" (\"job-1774584736262\", \"0001\"),\n",
602+
" (\"job-1774584969330\", \"0001\"),\n",
603+
" (\"job-1774595522117\", \"0001\"),\n",
604+
" (\"job-1774596683553\", \"0001\"),\n",
605+
"]\n",
606+
"\n",
607+
"SLEEP_SECONDS = 60\n",
608+
"\n",
609+
"\n",
610+
"def save_result(job_id, result):\n",
611+
" out_dir = os.path.join(BASE_PATH, job_id, \"eval_results\")\n",
612+
" os.makedirs(out_dir, exist_ok=True)\n",
613+
" out_path = os.path.join(out_dir, \"faithfulness.json\")\n",
614+
" with open(out_path, \"w\") as f:\n",
615+
" json.dump(result, f, indent=2)\n",
616+
" print(f\" Saved → {out_path}\")\n",
617+
"\n",
618+
"\n",
619+
"def run_faithfulness(job_id, index_id, use_input_param_queries):\n",
620+
" payload = {\n",
621+
" \"job_id\": job_id,\n",
622+
" \"index_id\": index_id,\n",
623+
" \"use_input_param_queries\": use_input_param_queries\n",
624+
" }\n",
625+
" try:\n",
626+
" response = requests.post(BASE_URL, json=payload, timeout=300)\n",
627+
" if response.status_code == 200:\n",
628+
" result = response.json()\n",
629+
" save_result(job_id, result)\n",
630+
" score = result.get(\"overall_faithfulness_score\", \"N/A\")\n",
631+
" print(f\" Score: {score}\")\n",
632+
" else:\n",
633+
" print(f\" Failed: HTTP {response.status_code}\")\n",
634+
" print(f\" {response.text}\")\n",
635+
" except Exception as e:\n",
636+
" print(f\" Error: {str(e)}\")\n",
637+
"\n",
638+
"\n",
639+
"# ==== BUILD FULL JOB LIST ====\n",
640+
"all_jobs = [\n",
641+
" (job_id, index_id, False) for job_id, index_id in EDUFLOW_JOBS\n",
642+
"] + [\n",
643+
" (job_id, index_id, True) for job_id, index_id in BASELINE_JOBS\n",
644+
"]\n",
645+
"\n",
646+
"total = len(all_jobs)\n",
647+
"\n",
648+
"# ==== MAIN LOOP ====\n",
649+
"for i, (job_id, index_id, use_input_param_queries) in enumerate(all_jobs):\n",
650+
" arch = \"EduFlow\" if not use_input_param_queries else \"Baseline\"\n",
651+
" print(f\"\\n[{i+1}/{total}] {arch} — {job_id}\")\n",
652+
"\n",
653+
" run_faithfulness(job_id, index_id, use_input_param_queries)\n",
654+
"\n",
655+
" if i < total - 1:\n",
656+
" print(f\" Sleeping {SLEEP_SECONDS}s...\")\n",
657+
" time.sleep(SLEEP_SECONDS)\n",
658+
"\n",
659+
"print(\"\\nAll faithfulness evaluations complete.\")"
660+
]
466661
}
467662
],
468663
"metadata": {
469664
"kernelspec": {
470-
"display_name": "tf_py310",
665+
"display_name": "pt_py311",
471666
"language": "python",
472667
"name": "python3"
473668
},
@@ -481,7 +676,7 @@
481676
"name": "python",
482677
"nbconvert_exporter": "python",
483678
"pygments_lexer": "ipython3",
484-
"version": "3.10.14"
679+
"version": "3.11.11"
485680
}
486681
},
487682
"nbformat": 4,

‎evaluator_service/evaluation/downlaod_blos.ipynb‎

Lines changed: 112 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,24 @@
22
"cells": [
33
{
44
"cell_type": "code",
5-
"execution_count": null,
5+
"execution_count": 1,
66
"id": "160961d0",
77
"metadata": {},
88
"outputs": [
99
{
1010
"name": "stdout",
1111
"output_type": "stream",
1212
"text": [
13+
"\n",
14+
"Processing job-1774534470161...\n",
15+
"Downloaded: downloaded_jobs\\job-1774534470161/logs/input_params.json\n",
16+
"Downloaded: downloaded_jobs\\job-1774534470161/logs/plan.json\n",
17+
"Downloaded: downloaded_jobs\\job-1774534470161/logs/response.json\n",
18+
"Downloaded: downloaded_jobs\\job-1774534470161/logs/usage.json\n",
19+
"Downloaded: downloaded_jobs\\job-1774534470161/pdf/doc-1.pdf\n",
20+
"Downloaded: downloaded_jobs\\job-1774534470161/pdf/doc-2.pdf\n",
21+
"Downloaded: downloaded_jobs\\job-1774534470161/pdf/doc-3.pdf\n",
22+
"Downloaded: downloaded_jobs\\job-1774534470161/pdf/doc-4.pdf\n",
1323
"\n",
1424
"Processing job-1774534999912...\n",
1525
"Downloaded: downloaded_jobs\\job-1774534999912/logs/input_params.json\n",
@@ -89,7 +99,97 @@
8999
"Downloaded: downloaded_jobs\\job-1774549026730/pdf/doc-1.pdf\n",
90100
"Downloaded: downloaded_jobs\\job-1774549026730/pdf/doc-2.pdf\n",
91101
"Downloaded: downloaded_jobs\\job-1774549026730/pdf/doc-3.pdf\n",
92-
"Downloaded: downloaded_jobs\\job-1774549026730/pdf/doc-4.pdf\n"
102+
"Downloaded: downloaded_jobs\\job-1774549026730/pdf/doc-4.pdf\n",
103+
"\n",
104+
"Processing job-1774574045744...\n",
105+
"Downloaded: downloaded_jobs\\job-1774574045744/logs/input_params.json\n",
106+
"Downloaded: downloaded_jobs\\job-1774574045744/logs/plan.json\n",
107+
"Downloaded: downloaded_jobs\\job-1774574045744/logs/response.json\n",
108+
"Downloaded: downloaded_jobs\\job-1774574045744/logs/usage.json\n",
109+
"Downloaded: downloaded_jobs\\job-1774574045744/pdf/doc-1.pdf\n",
110+
"Downloaded: downloaded_jobs\\job-1774574045744/pdf/doc-2.pdf\n",
111+
"Downloaded: downloaded_jobs\\job-1774574045744/pdf/doc-3.pdf\n",
112+
"Downloaded: downloaded_jobs\\job-1774574045744/pdf/doc-4.pdf\n",
113+
"\n",
114+
"Processing job-1774574325545...\n",
115+
"Downloaded: downloaded_jobs\\job-1774574325545/logs/input_params.json\n",
116+
"Downloaded: downloaded_jobs\\job-1774574325545/logs/plan.json\n",
117+
"Downloaded: downloaded_jobs\\job-1774574325545/logs/response.json\n",
118+
"Downloaded: downloaded_jobs\\job-1774574325545/logs/usage.json\n",
119+
"Downloaded: downloaded_jobs\\job-1774574325545/pdf/doc-1.pdf\n",
120+
"Downloaded: downloaded_jobs\\job-1774574325545/pdf/doc-2.pdf\n",
121+
"Downloaded: downloaded_jobs\\job-1774574325545/pdf/doc-3.pdf\n",
122+
"Downloaded: downloaded_jobs\\job-1774574325545/pdf/doc-4.pdf\n",
123+
"\n",
124+
"Processing job-1774574626655...\n",
125+
"Downloaded: downloaded_jobs\\job-1774574626655/logs/input_params.json\n",
126+
"Downloaded: downloaded_jobs\\job-1774574626655/logs/plan.json\n",
127+
"Downloaded: downloaded_jobs\\job-1774574626655/logs/response.json\n",
128+
"Downloaded: downloaded_jobs\\job-1774574626655/logs/usage.json\n",
129+
"Downloaded: downloaded_jobs\\job-1774574626655/pdf/doc-1.pdf\n",
130+
"Downloaded: downloaded_jobs\\job-1774574626655/pdf/doc-2.pdf\n",
131+
"Downloaded: downloaded_jobs\\job-1774574626655/pdf/doc-3.pdf\n",
132+
"Downloaded: downloaded_jobs\\job-1774574626655/pdf/doc-4.pdf\n",
133+
"\n",
134+
"Processing job-1774590551422...\n",
135+
"Downloaded: downloaded_jobs\\job-1774590551422/logs/input_params.json\n",
136+
"Downloaded: downloaded_jobs\\job-1774590551422/logs/plan.json\n",
137+
"Downloaded: downloaded_jobs\\job-1774590551422/logs/response.json\n",
138+
"Downloaded: downloaded_jobs\\job-1774590551422/logs/usage.json\n",
139+
"Downloaded: downloaded_jobs\\job-1774590551422/pdf/doc-1.pdf\n",
140+
"Downloaded: downloaded_jobs\\job-1774590551422/pdf/doc-2.pdf\n",
141+
"Downloaded: downloaded_jobs\\job-1774590551422/pdf/doc-3.pdf\n",
142+
"Downloaded: downloaded_jobs\\job-1774590551422/pdf/doc-4.pdf\n",
143+
"\n",
144+
"Processing job-1774584736262...\n",
145+
"Downloaded: downloaded_jobs\\job-1774584736262/logs/input_params.json\n",
146+
"Downloaded: downloaded_jobs\\job-1774584736262/logs/plan.json\n",
147+
"Downloaded: downloaded_jobs\\job-1774584736262/logs/response.json\n",
148+
"Downloaded: downloaded_jobs\\job-1774584736262/logs/usage.json\n",
149+
"Downloaded: downloaded_jobs\\job-1774584736262/pdf/doc-1.pdf\n",
150+
"Downloaded: downloaded_jobs\\job-1774584736262/pdf/doc-2.pdf\n",
151+
"Downloaded: downloaded_jobs\\job-1774584736262/pdf/doc-3.pdf\n",
152+
"Downloaded: downloaded_jobs\\job-1774584736262/pdf/doc-4.pdf\n",
153+
"\n",
154+
"Processing job-1774584969330...\n",
155+
"Downloaded: downloaded_jobs\\job-1774584969330/logs/input_params.json\n",
156+
"Downloaded: downloaded_jobs\\job-1774584969330/logs/plan.json\n",
157+
"Downloaded: downloaded_jobs\\job-1774584969330/logs/response.json\n",
158+
"Downloaded: downloaded_jobs\\job-1774584969330/logs/usage.json\n",
159+
"Downloaded: downloaded_jobs\\job-1774584969330/pdf/doc-1.pdf\n",
160+
"Downloaded: downloaded_jobs\\job-1774584969330/pdf/doc-2.pdf\n",
161+
"Downloaded: downloaded_jobs\\job-1774584969330/pdf/doc-3.pdf\n",
162+
"Downloaded: downloaded_jobs\\job-1774584969330/pdf/doc-4.pdf\n",
163+
"\n",
164+
"Processing job-1774596210500...\n",
165+
"Downloaded: downloaded_jobs\\job-1774596210500/logs/input_params.json\n",
166+
"Downloaded: downloaded_jobs\\job-1774596210500/logs/plan.json\n",
167+
"Downloaded: downloaded_jobs\\job-1774596210500/logs/response.json\n",
168+
"Downloaded: downloaded_jobs\\job-1774596210500/logs/usage.json\n",
169+
"Downloaded: downloaded_jobs\\job-1774596210500/pdf/doc-1.pdf\n",
170+
"Downloaded: downloaded_jobs\\job-1774596210500/pdf/doc-2.pdf\n",
171+
"Downloaded: downloaded_jobs\\job-1774596210500/pdf/doc-3.pdf\n",
172+
"Downloaded: downloaded_jobs\\job-1774596210500/pdf/doc-4.pdf\n",
173+
"\n",
174+
"Processing job-1774595522117...\n",
175+
"Downloaded: downloaded_jobs\\job-1774595522117/logs/input_params.json\n",
176+
"Downloaded: downloaded_jobs\\job-1774595522117/logs/plan.json\n",
177+
"Downloaded: downloaded_jobs\\job-1774595522117/logs/response.json\n",
178+
"Downloaded: downloaded_jobs\\job-1774595522117/logs/usage.json\n",
179+
"Downloaded: downloaded_jobs\\job-1774595522117/pdf/doc-1.pdf\n",
180+
"Downloaded: downloaded_jobs\\job-1774595522117/pdf/doc-2.pdf\n",
181+
"Downloaded: downloaded_jobs\\job-1774595522117/pdf/doc-3.pdf\n",
182+
"Downloaded: downloaded_jobs\\job-1774595522117/pdf/doc-4.pdf\n",
183+
"\n",
184+
"Processing job-1774596683553...\n",
185+
"Downloaded: downloaded_jobs\\job-1774596683553/logs/input_params.json\n",
186+
"Downloaded: downloaded_jobs\\job-1774596683553/logs/plan.json\n",
187+
"Downloaded: downloaded_jobs\\job-1774596683553/logs/response.json\n",
188+
"Downloaded: downloaded_jobs\\job-1774596683553/logs/usage.json\n",
189+
"Downloaded: downloaded_jobs\\job-1774596683553/pdf/doc-1.pdf\n",
190+
"Downloaded: downloaded_jobs\\job-1774596683553/pdf/doc-2.pdf\n",
191+
"Downloaded: downloaded_jobs\\job-1774596683553/pdf/doc-3.pdf\n",
192+
"Downloaded: downloaded_jobs\\job-1774596683553/pdf/doc-4.pdf\n"
93193
]
94194
}
95195
],
@@ -115,7 +215,16 @@
115215
" \"job-1774547498894\",\n",
116216
" \"job-1774548177351\",\n",
117217
" \"job-1774548651030\",\n",
118-
" \"job-1774549026730\"\n",
218+
" \"job-1774549026730\",\n",
219+
" \"job-1774574045744\",\n",
220+
" \"job-1774574325545\",\n",
221+
" \"job-1774574626655\",\n",
222+
" \"job-1774590551422\",\n",
223+
" \"job-1774584736262\",\n",
224+
" \"job-1774584969330\",\n",
225+
" \"job-1774596210500\",\n",
226+
" \"job-1774595522117\",\n",
227+
" \"job-1774596683553\"\n",
119228
"]\n",
120229
"\n",
121230
"# Local base directory\n",

‎evaluator_service/evaluation/evaluation_results.xlsx‎

-39 Bytes
Binary file not shown.

0 commit comments

Comments
 (0)