Add Connector check to liveness probe

[gpongelli]

Last week I faced an issue with JFrog Artifactory, it was unreachable via UI and via CLI even if all pods/statefulset were ready .

Looking at all logs, I've found that Artifactory pod reported Connector 'http-nio-8081' is not ready every 15 seconds with an increasing fail count (over 400) .

The workaround was to rollout restart the statefulset, but it should be better to add a check for that on the artifactory's liveness probe, so at least is clear which pod is failing and kubernetes can restart that pod.

Thanks.

[kfaan11]

Hi,

The same issue occurs with helm chart version jfrog-platform-11.3.1. The connector fails after the artifactory container receives a SIGTERM command. The SIGTERM command is generated by the container artifactory OOM. The SIGTERM shutdowns the artifactory process. The shutdown goes well but the container is still ready.
As @gpongelli said, the liveness does not check that the port 8081. The current liveness check the maintenance port 8091 but it is not enough to check whether the service on port 8081 is available.

curl -s -k --fail --max-time 5 http://localhost:8091/artifactory/api/v1/system/liveness

My advice is to to improve the current liveness probe to check both ports.
This is the current liveness probe in the v11.3.1.

curl -s -k --fail --max-time {{ .Values.probes.timeoutSeconds }} http://localhost:{{ .Values.artifactory.tomcat.maintenanceConnector.port }}/artifactory/api/v1/system/liveness

This is my advice:

exec:
  command:
    - sh
    - -c
    - >
      nc -z localhost 8081 &&
      curl -s -k --fail --max-time {{ .Values.probes.timeoutSeconds }}
      http://localhost:8081/artifactory/api/v1/system/liveness