diff --git a/docs/admin/external_services/postgres.mdx b/docs/admin/external_services/postgres.mdx index a48e298d8..5d99b5b53 100644 --- a/docs/admin/external_services/postgres.mdx +++ b/docs/admin/external_services/postgres.mdx @@ -2,380 +2,214 @@ import { CURRENT_VERSION_STRING_NO_V, CURRENT_VERSION_STRING } from 'src/compone # Using your own PostgreSQL server -You can use your own PostgreSQL v16+ server with Sourcegraph if you wish. For example, you may prefer this if you already have existing backup infrastructure around your own PostgreSQL server, wish to use Amazon RDS, etc. +As of Sourcegraph v6.0.0, the minimum supported version of Postgres is v16 -Please review the [PostgreSQL](/admin/postgres) documentation for a complete list of requirements. +Please review the [PostgreSQL](/admin/postgres) documentation for a complete list of requirements -As of version 6.0.0 The Sourcegraph application will require PostgreSQL v16+, earlier versions worked with 12 or greater. +## Recommendations -## General recommendations +- Use a managed Postgres service, as it provides you with much greater capabilities to manage your databases, ex. snapshots / restores, scale, etc. -If you choose to set up your own PostgreSQL server, please note **we strongly recommend each database to be set up in different servers and/or hosts**. We suggest either: +- Use a separate server / host for each of the 3 Postgres databases, to isolate and manage performance load and storage growth separately -1. Deploy _codeintel-db_ alongside the other Sourcegraph containers, i.e. not as a managed PostgreSQL instance. -2. Deploy a separate PostgreSQL instance. The primary reason to not use the same Postgres instance for this data is because code graph data can take up a significant of space (given the amount of indexed repositories is large) and the performance of the database may impact the performance of the general application database. You'll most likely want to be able to scale their resources independently. - -We also recommend having backups for the _codeintel-db_ as a best practice. The reason behind this recommendation is that _codeintel-db_ data is uploaded via CI systems. If data is lost, Sourcegraph cannot automatically rebuild it from the repositories, which means you'd have to wait until it is re-uploaded from your CI systems. +- Take regular backups of all the Postgres databases ## Instructions -The addition of `PG*` environment variables to your Sourcegraph deployment files will instruct Sourcegraph to target an external PostgreSQL server. To externalize the _frontend database_, use the following standard `PG*` variables: - -- `PGHOST` -- `PGPORT` -- `PGUSER` -- `PGPASSWORD` -- `PGDATABASE` -- `PGSSLMODE` - -To externalize the _code navigation database_, use the following prefixed `CODEINTEL_PG*` variables: - -- `CODEINTEL_PGHOST` -- `CODEINTEL_PGPORT` -- `CODEINTEL_PGUSER` -- `CODEINTEL_PGPASSWORD` -- `CODEINTEL_PGDATABASE` -- `CODEINTEL_PGSSLMODE` - -If you have configured both the frontend (pgsql) and code navigation (codeintel-db) databases with the same values, the Sourcegraph instance will refuse to start. Each database should either be configured to point to distinct hosts (recommended), or configured to point to distinct databases on the same host. - -### sourcegraph/server - -Add the following to your `docker run` command: - -```shell -docker run [...] --e PGHOST=psql1.mycompany.org --e PGUSER=sourcegraph --e PGPASSWORD=secret --e PGDATABASE=sourcegraph --e PGSSLMODE=require --e CODEINTEL_PGHOST=psql2.mycompany.org --e CODEINTEL_PGUSER=sourcegraph --e CODEINTEL_PGPASSWORD=secret --e CODEINTEL_PGDATABASE=sourcegraph-codeintel --e CODEINTEL_PGSSLMODE=require -sourcegraph/server:{CURRENT_VERSION_NO_V} -``` - ### Docker Compose -1. Add/modify the following environment variables to all of the `sourcegraph-frontend-*` services, the `sourcegraph-frontend-internal` service, and the `migrator` service (for Sourcegraph versions 3.37+) in [docker-compose.yaml](https://github.com/sourcegraph/deploy-sourcegraph-docker/blob/3.37/docker-compose/docker-compose.yaml): - -```yaml -sourcegraph-frontend-0: - environment: - - 'PGHOST=psql1.mycompany.org' - - 'PGUSER=sourcegraph' - - 'PGPASSWORD=secret' - - 'PGDATABASE=sourcegraph' - - 'PGSSLMODE=require' - - 'CODEINTEL_PGHOST=psql2.mycompany.org' - - 'CODEINTEL_PGUSER=sourcegraph' - - 'CODEINTEL_PGPASSWORD=secret' - - 'CODEINTEL_PGDATABASE=sourcegraph-codeintel' - - 'CODEINTEL_PGSSLMODE=require' -``` - -See [Environment variables in Compose](https://docs.docker.com/compose/environment-variables/) for other ways to pass these environment variables to the relevant services (including from the command line, a `.env` file, etc.). - -1. Comment out / remove the internal `pgsql` and `codeintel-db` services in [docker-compose.yaml](https://github.com/sourcegraph/deploy-sourcegraph-docker/blob/3.37/docker-compose/docker-compose.yaml) since Sourcegraph is using the external one now. - -### Kubernetes - -Update the `PG*` and `CODEINTEL_PG*` environment variables in the `sourcegraph-frontend` deployment YAML file to point to the external frontend (`pgsql`) and code navigation (`codeintel-db`) PostgreSQL instances, respectively. Again, these must not point to the same database or the Sourcegraph instance will refuse to start. +- Configure your database connection parameters as environment variables, as shown in the [External Databases example docker-compose.override.yaml](https://github.com/sourcegraph/deploy-sourcegraph-docker/tree/main/docker-compose/examples/external-databases/docker-compose.override.yaml) -You are then free to remove the now unused `pgsql` and `codeintel-db` services and deployments from your cluster. +- See Docker's documentation [Environment variables in Compose](https://docs.docker.com/compose/environment-variables/) for secure methods to pass in sensitive values -### Version requirements +### Helm -Please refer to our [Postgres](/admin/postgres) documentation to learn about version requirements. +- Configure your database connection parameters as Kubernetes secrets, as shown in the [External Databases example Helm override.yaml](https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/charts/sourcegraph/examples/external-databases/override.yaml) -### Caveats +### Notes -If your PostgreSQL server does not support SSL, set `PGSSLMODE=disable` instead of `PGSSLMODE=require`. Note that this is potentially insecure. +If your PostgreSQL server does not support SSL, set `PGSSLMODE: disable` instead of `PGSSLMODE: require`. Note that this is less secure. -Most standard PostgreSQL environment variables may be specified (`PGPORT`, etc). See http://www.postgresql.org/docs/current/static/libpq-envars.html for a full list. - -On Mac/Windows, if trying to connect to a PostgreSQL server on the same host machine, remember that Sourcegraph is running inside a Docker container inside of the Docker virtual machine. You may need to specify your actual machine IP address and not `localhost` or `127.0.0.1` as that refers to the Docker VM itself. - ----- +Most standard PostgreSQL environment variables may be specified (`PGPORT`, etc). See the [Postgres documentation](http://www.postgresql.org/docs/current/static/libpq-envars.html) for details. ## Usage with AWS RDS IAM Auth -For AWS RDS for Postgres, you have the option to use IAM database authentication to avoid using static database credentials. Learn more from [AWS documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html). - -In order to enable IAM Auth, you first need to: - -- enabled [IAM authentication on the RDS instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Enabling.html) -- created [the database account using IAM authentication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.DBAccounts.html#UsingWithRDS.IAMDBAuth.DBAccounts.PostgreSQL) -- created [IAM policy for IAM database access](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.IAMPolicy.html) -- created IAM roles for your compute workload and grant the role with the above policy -- ensured your compute resources can assume those IAM roles - - For EKS (k8s deployment), use [IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) - - For EC2 (docker-compose deployment), use [IAM roles for Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) - -For [every services that require postgres database connection](https://github.com/sourcegraph/sourcegraph-public-snapshot/blob/main/lib/servicecatalog/service-catalog.yaml), ensure below environment variables are configured: - -- `PG_CONNECTION_UPDATER=EC2_ROLE_CREDENTIALS` -- `PGSSLMODE=require` -- `PGHOST=<>` -- `PGPORT=<>` -- `PGUSER=<>` - this should be the database accounts created above -- `PGDATABASE=<>` -- `CODEINTEL_PGSSLMODE=require` -- `CODEINTEL_PGPORT=<>` -- `CODEINTEL_PGUSER=<>` this should be the database accounts created above -- `CODEINTEL_PGDATABASE=<>` -- `CODEINSIGHTS_PGSSLMODE=require` -- `CODEINSIGHTS_PGHOST=<>` -- `CODEINSIGHTS_PGPORT=<>` -- `CODEINSIGHTS_PGUSER=<>` this should be the database accounts created above -- `CODEINSIGHTS_PGDATABASE=<>` +For AWS RDS for Postgres, you have the option to use IAM database authentication to avoid using static database credentials. See the [AWS documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html) for details. -## Usage with PgBouncer +To enable IAM Auth: -[PgBouncer] is a lightweight connections pooler for PostgreSQL. It allows more clients to connect with the PostgreSQL database without running into connection limits. +- Enable [IAM authentication on the RDS instance](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Enabling.html) -When [PgBouncer] is used, we need to include `statement_cache_mode=describe` in the PostgreSQL connection url. This can be done by configuring the `PGDATASOURCE` and `CODEINSIGHTS_PGDATASOURCE` environment variables to `postgres://username:password@pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe` +- Create [the database user account using IAM authentication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.DBAccounts.html#UsingWithRDS.IAMDBAuth.DBAccounts.PostgreSQL) -### sourcegraph/server +- Create [IAM policy for IAM database access](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.IAMPolicy.html) -Add the following to your `docker run` command: +- Create an IAM role for your compute workload and grant the role with the above policy -
docker run [...] -e PGDATASOURCE="postgres://username:password@sourcegraph-pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe" -e CODEINSIGHTS_PGDATASOURCE="postgres://username:password@sourcegraph-codeintel-pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe" sourcegraph/server:{CURRENT_VERSION_STRING_NO_V()}
+- Have your compute instances assume the IAM role -### Docker Compose + - For EKS, see [IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) -1. Add/modify the following environment variables to all of the `sourcegraph-frontend-*` services, the `sourcegraph-frontend-internal` service, and the `migrator` service (for Sourcegraph versions 3.37+) in [docker-compose.yaml](https://github.com/sourcegraph/deploy-sourcegraph-docker/blob/3.37/docker-compose/docker-compose.yaml): + - For EC2 (docker-compose), see [IAM roles for Amazon EC2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) -```yaml -sourcegraph-frontend-0: - environment: - - 'PGDATASOURCE=postgres://username:password@sourcegraph-pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe' - - 'CODEINSIGHTS_PGDATASOURCE=postgres://username:password@sourcegraph-codeintel-pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe' -``` +- Configure the `PG_CONNECTION_UPDATER: EC2_ROLE_CREDENTIALS` environment variable -See ["Environment variables in Compose"](https://docs.docker.com/compose/environment-variables/) for other ways to pass these environment variables to the relevant services (including from the command line, a `.env` file, etc.). + - In Docker Compose, add it to `migrator`, and all `sourcegraph-frontend-*` services, as shown in the [External Databases example docker-compose.override.yaml](https://github.com/sourcegraph/deploy-sourcegraph-docker/tree/main/docker-compose/examples/external-databases/docker-compose.override.yaml) -1. Comment out / remove the internal `pgsql` and `codeintel-db` services in [docker-compose.yaml](https://github.com/sourcegraph/deploy-sourcegraph-docker/blob/3.37/docker-compose/docker-compose.yaml) since Sourcegraph is using the external one now. + - In Helm, add it to the `migrator` and `frontend` values, as shown in the [AWS example Helm override.yaml](https://github.com/sourcegraph/deploy-sourcegraph-helm/blob/main/charts/sourcegraph/examples/aws/override.yaml) -### Kubernetes - -Create a new [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) to store the [PgBouncer] credentials. - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: sourcegraph-pgbouncer-credentials -data: - password: "" # note: secrets data has to be base64-encoded -``` - -```yaml -apiVersion: v1 -kind: Secret -metadata: - name: sourcegraph-codeintel-pgbouncer-credentials -data: - password: "" # note: secrets data has to be base64-encoded -``` - -Update the environment variables in the `sourcegraph-frontend` deployment YAML. - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: sourcegraph-frontend -spec: - template: - spec: - containers: - - name: frontend - env: - - name: PGDATABASE - value: sg - - name: PGHOST - value: sourcegraph-pgbouncer - - name: PGPORT - value: "5432" - - name: PGSSLMODE - value: disable - - name: PGUSER - value: sg - - name: PGPASSWORD - valueFrom: - secretKeyRef: - name: sourcegraph-pgbouncer-credentials - key: password - - name: PGDATASOURCE - value: postgres://$(PGUSER):$(PGPASSWORD)@$(PGHOST):$(PGPORT)/$(PGDATABASE)?statement_cache_mode=describe - - name: CODEINTEL_PGDATABASE - value: sg-codeintel - - name: CODEINTEL_PGHOST - value: sourcegraph-codeintel-pgbouncer.mycompany.com - - name: CODEINTEL_PGPORT - value: "5432" - - name: CODEINTEL_PGSSLMODE - value: disable - - name: CODEINTEL_PGUSER - value: sg - - name: CODEINTEL_PGPASSWORD - valueFrom: - secretKeyRef: - name: sourcegraph-codeintel-pgbouncer-credentials - key: password - - name: CODEINSIGHTS_PGDATASOURCE - value: postgres://$(CODEINTEL_PGUSER):$(CODEINTEL_PGPASSWORD)@$(CODEINTEL_PGHOST):$(CODEINTEL_PGPORT)/$(CODEINTEL_PGDATABASE)?statement_cache_mode=describe -``` +- **Optional:** If you prefer to use regional STS endpoints for improved latency, also set the `AWS_STS_REGIONAL_ENDPOINTS: regional` environment variable. See [AWS documentation](https://docs.aws.amazon.com/sdkref/latest/guide/feature-sts-regionalized-endpoints.html) for details. ---- -## Postgres Permissions and Database Migrations - -There is a tight coupling between the respective database service accounts for the Frontend DB, CodeIntel DB and Sourcegraph [database migrations](https://docs-legacy.sourcegraph.com/dev/background-information/sql/migrations). - -By default, the migrations that Sourcegraph runs expect `SUPERUSER` permissions. Sourcegraph migrations contain SQL that enable extensions and modify roles. +## Postgres Permissions -On AWS RDS, you will need to perform the operations below using the `rds_superuser` role because RDS does not grant SUPERUSER privileges to user database accounts. +The Postgres user/role your Sourcegraph instance uses should have `SUPERUSER` permissions to complete the first migration query in each database, when initializing fresh new databases, as it installs extensions. This may be prohibited in your environment, so we have workarounds available below. -This may not be acceptable in all environments. At minimum we expect that the `PGUSER` and `CODEINTEL_PGUSER` have the `ALL` permissions on `PGDATABASE` and `CODEINTEL_PGDATABASE` respectively. +At minimum, we expect that the user has the `ALL` permission on the [Database objects](https://www.postgresql.org/docs/current/sql-grant.html), which includes: +``` +CONNECT +CREATE +DELETE +EXECUTE +INSERT +REFERENCES +SELECT +TEMPORARY +TRIGGER +TRUNCATE +UPDATE +USAGE +``` -`ALL` privileges on the [Database object](https://www.postgresql.org/docs/current/sql-grant.html) include: - * `SELECT` - * `INSERT` - * `UPDATE` - * `DELETE` - * `TRUNCATE` - * `REFERENCES` - * `TRIGGER` - * `CREATE` - * `CONNECT` - * `TEMPORARY` - * `EXECUTE` - * `USAGE` +### Using restricted permissions -### Using restricted permissions for pgsql (frontend DB) +When initializing fresh new databases, the first migration query in each database installs extensions, so it must be performed by a user with `SUPERUSER` permissions. Find your `SUPERUSER` username and password, to connect to your databases and execute the below queries. -For AWS RDS, refer to the note from this [section](#postgres-permissions-and-database-migrations). +On AWS RDS, you will need to perform these operations using the `rds_superuser` role, because RDS does not grant SUPERUSER privileges to other database users. -Sourcegraph requires some initial setup that requires `SUPERUSER` permissions. A database administrator needs to perform the necessary actions on behalf of Sourcegraph migrations as `SUPERUSER`. +#### Pgsql (frontend) database -Update these variables to match your deployment of the Sourcegraph _frontend_ database following [the guidance from the instructions section](#instructions). This database is called `pgsql` in the Docker Compose and Kubernetes deployments. +- Update these variables to match your deployment of the Sourcegraph pgsql (frontend) database following [the guidance from the instructions section](#instructions) ```bash -PGHOST=psql -PGUSER=sourcegraph -PGPASSWORD=secret -PGDATABASE=sourcegraph -``` +# Export the environment variables to connect to the database +export PGHOST=psql +export PGPASSWORD=rds_superuser_password +export PGPORT=5432 +export PGSSLMODE=require +export PGUSER=rds_superuser -The SQL script below is intended to be run from by a database administrator with `SUPERUSER` priviledges against the Frontend Database. It creates a database, user, and configures necesasry permissions for use by the Sourcegraph _frontend_ services. +# Export the environment variables to create the database and service account +export PGDATABASE=sg +export SERVICE_ACCOUNT_PASSWORD=sg_service_account_password +export SERVICE_ACCOUNT_USERNAME=sg_service_account_username + +# Connect to the database and execute the query in one command +psql -d postgres -f - << EOF -```sql --- Create the application database CREATE DATABASE $PGDATABASE; --- Create the application service user -CREATE USER $PGUSER with encrypted password '$PGPASSWORD'; +CREATE USER $SERVICE_ACCOUNT_USERNAME with encrypted password $SERVICE_ACCOUNT_PASSWORD; --- Give the application service permissions to the application database -GRANT ALL PRIVILEGES ON DATABASE $PGDATABASE to $PGUSER; +GRANT ALL PRIVILEGES ON DATABASE $PGDATABASE to $SERVICE_ACCOUNT_USERNAME; --- Select the application database \c $PGDATABASE; --- Install necessary extensions CREATE extension citext; CREATE extension hstore; +CREATE extension intarray; CREATE extension pg_stat_statements; CREATE extension pg_trgm; CREATE extension pgcrypto; -CREATE extension intarray; -``` - -After the database is configured, Sourcegraph will attempt to run migrations. There are a few migrations that may fail as they attempt to run actions that require `SUPERUSER` permissions. - -These failures must be interpreted by the database administrator and resolved using guidance from [How to Troubleshoot a Dirty Database](/admin/how-to/dirty_database). Generally-speaking this will involve looking up the migration source code and manually applying the necessary SQL code. -**Initial Schema Creation** - -The first migration fails since it attempts to add `COMMENT`s to installed extensions. You may see the following error message: - -``` -failed to run migration for schema "frontend": failed upgrade migration 1528395834: ERROR: current transaction is aborted, commands ignored until end of transaction block (SQLSTATE 25P02) +EOF ``` -In this case, locate the UP migration 1528395834 and apply all SQL after the final `COMMENT ON EXTENSION` command following the [dirty database procedure](/admin/how-to/dirty_database). +- After the databases are configured, start up migrator, it will attempt to run migrations, and will fail on the first privileged migration; manually mark it as completed, then start up migrator normally, and it should complete the rest of the migrations. For details, see [How to Troubleshoot a Dirty Database](/admin/how-to/dirty_database). -**Dropping the `sg_service` role** +#### Codeintel database -The `sg_service` database role is a legacy role that should be removed from all Sourcegraph installations at this time. Migration `remove_sg_service_role` attempts to enforce this with a `DROP ROLE` command. The `PGUSER` does not have permissions to perform this action, therefore the migration fails. You can safely skip this migration. +- Repeat the above, with the correct environment variables for the codeintel database ----- - -### Using restricted permissions for CodeIntel DB +#### Codeinsights database -For AWS RDS, refer to the note from this [section](#postgres-permissions-and-database-migrations). +- Repeat the above, with the correct environment variables for the codeinsights database -CodeIntel requires some initial setup that requires `SUPERUSER` permissions. A database administrator needs to perform the necessary actions on behalf of Sourcegraph migrations as `SUPERUSER`. +---- -```bash -CODEINTEL_PGHOST=psql2 -CODEINTEL_PGUSER=sourcegraph -CODEINTEL_PGPASSWORD=secret -CODEINTEL_PGDATABASE=sourcegraph-codeintel -CODEINTEL_PGSSLMODE=require -``` +## Usage with PgBouncer -The SQL script below is intended to be run from by a database administrator with `SUPERUSER` priviledges against the CodeIntel Database. It creates a database, user, and configures necesasry permissions for use by the Sourcegraph _frontend_ services. + -```sql --- Create the CodeIntel database -CREATE DATABASE $CODEINTEL_PGDATABASE; +PgBouncer is a connection pooler for PostgreSQL. It allows more clients to connect with the PostgreSQL databases without running into connection limits. --- Create the CodeIntel service user -CREATE USER $CODEINTEL_PGUSER with encrypted password '$CODEINTEL_PGPASSWORD'; +When PgBouncer is used, we need to include `statement_cache_mode=describe` in the PostgreSQL connection URI. This can be done by configuring the `PGDATASOURCE` environment variables. --- Give the CodeIntel permissions to the application database -GRANT ALL PRIVILEGES ON DATABASE $CODEINTEL_PGDATABASE to $CODEINTEL_PGUSER; +### Docker Compose --- Select the application database -\c $CODEINTEL_PGDATABASE; +Add/modify the following environment variables on `migrator`, and all `sourcegraph-frontend-*` services: --- Install necessary extensions -CREATE extension citext; -CREATE extension hstore; -CREATE extension pg_stat_statements; -CREATE extension pg_trgm; -CREATE extension pgcrypto; -CREATE extension intarray; +```yaml +sourcegraph-frontend-0: + environment: + CODEINSIGHTS_PGDATASOURCE: "postgres://user:password@codeinsights-pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe" + CODEINTEL_PGDATASOURCE: "postgres://user:password@codeintel-pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe" + PGDATASOURCE: "postgres://user:password@pgsql-pgbouncer.mycompany.com:5432/sg?statement_cache_mode=describe" ``` -After the database is configured, Sourcegraph will attempt to run migrations, this time using the CodeIntel DB. There are a few migrations that may fail as they attempt to run actions that require `SUPERUSER` permissions. - -These failures must be intepreted by the database administrator and resolved using guidance from [How to Troubleshoot a Dirty Database](/admin/how-to/dirty_database). Generally-speaking this will involve looking up the migration source code and manually applying the necessary SQL code. The `codeintel_schema_migrations` table should be consulted for dirty migrations in this case. +### Helm -**Initial CodeIntel schema creation** - -Like the failure in the Sourcegraph DB (pgsql) migrations, the CodeIntel initial migration attempts to `COMMENT` on an extension. Resolve this in a similar manner by executing the SQL in the `1000000015_squashed_migrations.up` migration after the `COMMENT` SQL statement. - -The following error is a nudge to check the `codeintel_schema_migrations` table in `$CODEINTEL_PGDATABASE`. +Update the [Secret](https://kubernetes.io/docs/concepts/configuration/secret/) to store the PgBouncer credentials, for each database +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: [database]-pgbouncer-credentials +data: + password: "" # note: secrets data has to be base64-encoded ``` -Failed to connect to codeintel database: 1 error occurred: - * dirty database: schema is marked as dirty but no migrator instance appears to be running -The target schema is marked as dirty and no other migration operation is seen running on this schema. The last migration operation over this schema has failed (or, at least, the migrator instance issuing that migration has died). Please contact support@sourcegraph.com for further assistance. -``` +Add the `PGDATASOURCE` environment variables for frontend in your Helm values override.yaml file: -[pgbouncer]: https://www.pgbouncer.org/ +```yaml +frontend: + env: + + - name: CODEINSIGHTS_PGPASSWORD + valueFrom: + secretKeyRef: + name: codeintel-pgbouncer-credentials + key: password + - name: CODEINSIGHTS_PGDATASOURCE + value: postgres://$(CODEINSIGHTS_PGUSER):$(CODEINSIGHTS_PGPASSWORD)@$(CODEINSIGHTS_PGHOST):$(CODEINSIGHTS_PGPORT)/$(CODEINSIGHTS_PGDATABASE)?statement_cache_mode=describe + + - name: CODEINTEL_PGPASSWORD + valueFrom: + secretKeyRef: + name: codeintel-pgbouncer-credentials + key: password + - name: CODEINTEL_PGDATASOURCE + value: postgres://$(CODEINTEL_PGUSER):$(CODEINTEL_PGPASSWORD)@$(CODEINTEL_PGHOST):$(CODEINTEL_PGPORT)/$(CODEINTEL_PGDATABASE)?statement_cache_mode=describe + + - name: PGPASSWORD + valueFrom: + secretKeyRef: + name: pgsql-pgbouncer-credentials + key: password + - name: PGDATASOURCE + value: postgres://$(PGUSER):$(PGPASSWORD)@$(PGHOST):$(PGPORT)/$(PGDATABASE)?statement_cache_mode=describe +```