Skip to content

feat: create a script to obtain the status of the simulation#117

Open
Namyalg wants to merge 2 commits into
moja-global:masterfrom
Namyalg:simulation-status
Open

feat: create a script to obtain the status of the simulation#117
Namyalg wants to merge 2 commits into
moja-global:masterfrom
Namyalg:simulation-status

Conversation

@Namyalg

@Namyalg Namyalg commented May 25, 2022

Copy link
Copy Markdown
Member

Pull Request Template

Description

The existing endpoints for the gcbm example do not report a failure in the simulation.
The script simulation-status.py returns the logs of the docker container (if it is running).
This can be used in the process of automated testing

To obtain the logs of the simulation :

  1. Start the simulation
  2. Run the script simulation-status.py , a text file containing the logs will be created

Fixes #93

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

aornugent
aornugent previously approved these changes May 29, 2022
@aornugent

Copy link
Copy Markdown
Contributor

I think in order to make this an integration test, we need to publish a reference to compare against. Can you add the scaffolding to save the results and compare with another file of the same schema? Then we check the reference into version control and monitor change from there.

This does not tell us whether the results are numerically identical, which is considered the holy grail in integration tests.

@Namyalg

Namyalg commented Jun 7, 2022

Copy link
Copy Markdown
Member Author

A quick demonstration on how the script can be used

logs.mp4

@Namyalg

Namyalg commented Jun 8, 2022

Copy link
Copy Markdown
Member Author

Logs from a successful simulation :

docker_logs.txt

@aornugent aornugent requested a review from radistoubalidis June 8, 2022 04:08
@aornugent

aornugent commented Jun 8, 2022

Copy link
Copy Markdown
Contributor

@radistoubalidis - could you use this script in your CML Action?

@Namyalg - if it weren't for the timestamps, we would be able to use the hash of these logs as a reference. Are timestamps included in the console output when you run the simulation inside the container interactively? Or are they part of th Docker output?

@radistoubalidis

Copy link
Copy Markdown
Contributor

@radistoubalidis - could you use this script in your CML Action?

@Namyalg - if it weren't for the timestamps, we would be able to use the hash of these logs as a reference. Are timestamps included in the console output when you run the simulation inside the container interactively? Or are they part of th Docker output?

I just added it to my action here , the only issue is a haven't yet managed to run the whole workflow inside actions so for now, I made it use the results from a sim I run locally.

@Namyalg

Namyalg commented Jun 8, 2022

Copy link
Copy Markdown
Member Author

The timestamp can be removed, I'll process it such that only

@radistoubalidis - could you use this script in your CML Action?

@Namyalg - if it weren't for the timestamps, we would be able to use the hash of these logs as a reference. Are timestamps included in the console output when you run the simulation inside the container interactively? Or are they part of th Docker output?

Yes, this can be removed, I'll process it in such a way that <2022-06-08 00:16:24.906554> (info) - Config has files: 7 will be stored as Config has files: 7

To identify the simulation uniquely, how about appending the end of the file with a randomly generated string?

@radistoubalidis radistoubalidis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I managed to add the python script to my simulation results and retrieve the log file

@Namyalg Namyalg marked this pull request as ready for review July 3, 2022 04:20
@aornugent

Copy link
Copy Markdown
Contributor

@Namyalg - can you please remind me if this is ready to merge? Do you have permissions to do so?

@Namyalg

Namyalg commented Dec 24, 2022

Copy link
Copy Markdown
Member Author

Yes @aornugent, I tested the script again, it works as expected -> retrieve the logs of a container running the simulation
It is ready for merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GCBM Test suite does not report simulation failure

3 participants