CI fails when other jobs are running because of time out #832
Description
See https://git.rwth-aachen.de/acs/public/villas/node/-/jobs/5510111 for details.
Relevant output:
[TIME] pipe-loopback-zeromq (ran for more then 1m)
[SKIP] pipe-python-protobuf: Test is broken
[SKIP] relay: Test is broken
[PASS] signal (ran for .100993659s)
[PASS] test-config (ran for 7.858488891s)
Summary: 1 of 61 tests failed.
Timedout: 1
Skipped: 18Maybe we can remove the time out?
Activity
stv0g commentedon Nov 29, 2024
I guess the timeout is probably more of an indicator for a broken test?
Shouldn't we attempt to fix the test, or disable it?
n-eiling commentedon Dec 2, 2024
The test does work if it I rerun it. So it seems to work, but is just too slow if run in parallel with everything else.
I haven't investigated thoroughly though and I don't know if a similar functionality is tested in another test.
stv0g commentedon Feb 10, 2025
Is this still happening? I've seen it not myself yet..
n-eiling commentedon Feb 13, 2025
Haven't seen it in a while - but that doesn't mean it fixed itself. Don't you think it's bad practice to rely on timing in the CI? The runner might be really slow sometimes.
stv0g commentedon Feb 13, 2025
Yeah, its a valid consern.
I have added those timeout checks to avoid clogging the CI with jobs which are dead-locked.
Maybe we should raise the timeout to something like 1h?
n-eiling commentedon Feb 13, 2025
sounds good to me
IgnoreWarnings commentedon Feb 21, 2025
I got the timeout yesterday, it is still there.
IgnoreWarnings commentedon Feb 24, 2025
I get it consistently now on every single commit.
fix VILLASframework#832: Set CI timeout to 15 min
IgnoreWarnings commentedon Feb 24, 2025
Increasing the timeout didnt help, the integrationtest seems to get stuck somehow.
IgnoreWarnings commentedon Feb 24, 2025
It looks like @stv0g is right that the test is broken.It looks like the Test breaks under some external condition paralell Test/Ports in use or something else.I increased the timeout and the CI hangs on integration test.
IgnoreWarnings commentedon Feb 24, 2025
"[TIME] pipe-loopback-nanomsg (ran for more then 15m)"
IgnoreWarnings commentedon Feb 24, 2025
But it looks like before it was on another test: [TIME] pipe-loopback-zeromq (ran for more then 1m)
stv0g commentedon Feb 25, 2025
Okay, if the tests are broken, we need to figure out what broke them as they used to work in previous commits.
Maybe git bisect could help..
n-eiling commentedon Feb 25, 2025
I suspect this is more due to docker / strange interactions between the tests.
n-eiling commentedon May 23, 2025
This is still happening...
IgnoreWarnings commentedon May 23, 2025
I can look into this with my current task (broken stats hook on arm).
[-]CI fails when other jobs are running because of time out.[/-][+]CI fails when other jobs are running because of time out[/+]