Puppet certificates don't work with the version of go that prometheus servers use, so it fails to scrape metrics from them. Update the certificate so that they have the fqdn in SAN, not only on CN, which apparently is deprecated.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T262668 WMF media storage must be adequately backed up | |||
| Resolved | jcrespo | T276442 Puppetize media backups infrastructure | |||
| Resolved | fgiunchedi | T222113 prometheus: upgrade to >= 2.12 | |||
| Resolved | jcrespo | T288195 Update media backup TLS certificates |
Event Timeline
Comment Actions
High because it is causing metrics to fail to be collected for minio and creating alerts that are not easily acknowledgeable.
Comment Actions
My two cents: not necessarily for this task but IMHO would be worth exploring if this is fixed (or has been, or it will be) on the puppet side, IOW asking for certs with fqdn in SAN too at enrollment time
Comment Actions
I had an IRC discussion with @fgiunchedi , profile::pki seems the proper way to fix this, which may require changes on cert generation on backup hosts and ca configuration on prometheus ones- I need to research a bit about PKI support, as I don't have experience implementing it. CC @jbond
Comment Actions
This was fixed in https://gerrit.wikimedia.org/r/710491 thus optimistically resolving