PT-2237 - fixed PostgreSQL DB logs collection #1051
Conversation
This fix ensures `pt-k8s-debug-collector` collects postgreSql databse logs which is stored in `/pgdata/<cluster_name>/pg_log`. A test `TestIndividualFiles` was refactored, and modified for a new case with new feature.
|
|
| switch resourceType(resource) { | ||
| case "pg": | ||
| dirPaths = append(dirPaths, | ||
| "$PGBACKREST_DB_PATH/pg_log", |
There was a problem hiding this comment.
Where does the variable PGBACKREST_DB_PATH come from? It may not be set on the client machine where pt-k8s-debug-collector is running. Or could be set to a completely irrelevant value regarding the cluster it collects the data from.
There was a problem hiding this comment.
This comes from pod manifest.
Environment:
MODE: postgres
PGHA_PG_PORT: 5432
PGHA_USER: postgres
PATRONI_POSTGRESQL_DATA_DIR: /pgdata/cluster1-repl1
PGBACKREST_STANZA: db
PGBACKREST_REPO1_HOST: cluster1-backrest-shared-repo
BACKREST_SKIP_CREATE_STANZA: true
PGHA_PGBACKREST: true
PGBACKREST_REPO1_PATH: /backrestrepo/cluster1-backrest-shared-repo
PGBACKREST_DB_PATH: /pgdata/cluster1-repl1
^
| }() | ||
|
|
||
| time.Sleep(3 * time.Second) // wait for port-forward command | ||
| time.Sleep(10 * time.Second) // wait for port-forward command |
There was a problem hiding this comment.
Why sleep time increase?
There was a problem hiding this comment.
It was too little on my machine. Maybe we should implement some tcp checker to confirm that port-forward is working?
| // runCmd run command (Dumper.cmd) with given args, return it output | ||
| func (d *Dumper) runCmd(args ...string) ([]byte, error) { | ||
| baseArgs := []string{"--kubeconfig", d.kubeconfig} | ||
| baseArgs = append(baseArgs, args...) |
There was a problem hiding this comment.
If you prefix arguments with --kubeconfig and d.kubeconfig is empty, the following argument will be treated as a path to --kubeconfig. This is wrong.
There was a problem hiding this comment.
Added an if condition for that. We need this to be prefixed because exec command will be wrong.
This fix ensures
pt-k8s-debug-collectorcollects PostgreSQL database logs.What was added:
To keep changes minimal, the existing test
TestIndividualFileswas modified to cover the new case.The test was refactored because the previous implementation couldn't handle files with variable names.
In
/pgdata/<cluster_name>/pg_log, logs are named according to the day/week, so filenames are not known at runtime.A new function
getAllFilesFromDirectorywas added todumper.go.Its purpose is the same as the existing
getIndividualFiles, but it supports extracting multiple files from a directory. It is useful because pg log filenames can vary, and there can be multiple log files.A new function
parseEnvswas added todumper.go.Its purpose is to load environment variables present in the pod into a string.
It is used inside
getAllFilesFromDirectoryto load$PGBACKREST_DB_PATH, because there is no simpler way to determine the path/pgdata/<cluster_name>/pg_log, which can vary.(
/libhas not been changed)