Connection error because users not included in backup

[simcod]

Description

After restoring metal-api from a s3 backend, backup-restore-sidecar successfully restored the database. However, the metal-api was not able to connect to the database with following error:

...
{"level":"ERROR","msg":"db connection error","datastore":{"db":"metalapi","hosts":["metal-db"],"error":"cannot connect to DB: rethinkdb: Unknown user"}
...

It was necessary to redeploy the metal-stack control plane. The initdb job successfully created the metal user in the database again.
However, we should consider backing up and restoring the users as well.

[majst01]

This behaviour is documented here: https://rethinkdb.com/docs/backup/

[Gerrit91]

I still do not see why these should not be included in the backup? In the examples they even backup user tables explicitly, but it does not look like they are excluded by default?

[majst01]

from that site:

Note: The dump command saves database and table contents and metadata, but does not save cluster configuration data.

[Gerrit91]

The users should be part of a table, it's not configuration data?