Results view update#239
Conversation
…n template for status display Implement download_results view for evaluation results in JSON format Update .gitignore to include VS Code specific files
for more information, see https://pre-commit.ci
|
This adds:
|
|
Dependent on janelia-cellmap/cellmap-segmentation-challenge#158 |
Now complete. Can be tested in staging. |
jnywong
force-pushed
the
results_view_update
branch
from
e5f1d88 to
f1c986b
Compare
|
Hey @rhoadesScholar ! I have deployed this to the staging website and the evaluation has failed with the error concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.You can see the full evaluator logs at https://staging.cellmapchallenge.2i2c.cloud/submissions/40. It could be due to the limited resources I have allocated to the staging site, see 2i2c-org/frx-challenges-deploy#84 |
@jnywong |
|
@jnywong Just ran with a small submission and it was fine: https://staging.cellmapchallenge.2i2c.cloud/versions/63 |
|
@jnywong I just pushed an update to try to fix the download link not showing up. Can you try pushing it to the staging? |
|
Okay nice work! I have merged your updates into this PR where the CI/CD is able to build an image. This is so that we can run a container with the website updates from your PR in the staging Kubernetes cluster. I would like to find a way for you to do this without my intervention! |
That would be awesome! |
|
439ca6f Just reverted a change that seemed to break things: https://staging.cellmapchallenge.2i2c.cloud/versions/43 |
| @property | ||
| def percent_complete(self) -> str: | ||
| """ | ||
| Return the percentage of completion of this evaluation | ||
| """ | ||
| if self.status == Evaluation.Status.EVALUATED: | ||
| return "100%" | ||
| elif self.status == Evaluation.Status.EVALUATING: | ||
| if self.result: | ||
| percent_complete = self.result.get( | ||
| "num_evals_done", 0 | ||
| ) / self.result.get("total_evals", 1) | ||
| return f"{int(percent_complete * 100)}%" | ||
| else: | ||
| return "0%" | ||
| else: | ||
| return "0%" |
There was a problem hiding this comment.
This won't work unfortunately. If you look at https://github.com/2i2c-org/frx-challenges/blob/main/frx_challenges/web/management/commands/evaluator.py#L199, you see that the evaluation.result is not saved until the evaluation is 100% complete.
evaluation.status = Evaluation.Status.EVALUATED
evaluation.result = resultThis will take a lot more logic in the evaluator to implement. I would suggest removing this component from this PR for now.
There was a problem hiding this comment.
Hmmm. I see.
Looks like if we just need to make sure to return even partial results by trying to load the file even if the evaluator hasn't exited and update the evaluation with it even if the evaluator is still running.
I'll move it to a separate PR.
There was a problem hiding this comment.
Moved this to #245 with fixes to the evaluation object updates to gather results even if evaluation is still ongoing.
Update frx_challenges/web/views/versions.py
…ersion.html template structure
for more information, see https://pre-commit.ci
2 participants
Need to try testing this on the staging site - waiting for documentation of how to do that.