Auto IAM Auth not working as intended?

[JaimeLennox]

Hello!

I've been trying to get the operator to work with my application using a config similar to:

apiVersion: cloudsql.cloud.google.com/v1
kind: AuthProxyWorkload
metadata:
  name: my-proxy
spec:
  authProxyContainer:
    authentication:
      impersonationChain:
        - <service_account>@<project>.iam.gserviceaccount.com
  workloadSelector:
    kind: "Deployment"
    selector:
      matchExpressions:
        - key: "component"
          operator: In
          values:
            - my-app
  instances:
    - connectionString: "<my_instance_connection>"
      port: 5432
      privateIP: true
      autoIAMAuthN: true

This generates an init container like so (some parts removed for brevity) - so far so good:

  initContainers:
  - args:
    - <my_instance_connection>?auto-iam-authn=true&port=5432&private-ip=true
    env:
    - name: CSQL_PROXY_HTTP_PORT
      value: "9801"
    - name: CSQL_PROXY_HTTP_ADDRESS
      value: 0.0.0.0
    - name: CSQL_PROXY_HEALTH_CHECK
      value: "true"
    - name: CSQL_PROXY_EXIT_ZERO_ON_SIGTERM
      value: "true"
    - name: CSQL_PROXY_QUITQUITQUIT
      value: "true"
    - name: CSQL_PROXY_ADMIN_PORT
      value: "9091"
    - name: CSQL_PROXY_IMPERSONATE_SERVICE_ACCOUNT
      value: <service_account>@<project>.iam.gserviceaccount.com
    - name: CSQL_PROXY_USER_AGENT
      value: cloud-sql-proxy-operator/1.7.2
    - name: CSQL_PROXY_STRUCTURED_LOGS
      value: "true"
    - name: CSQL_PROXY_QUIT_URLS
      value: http://localhost:9091/quitquitquit
    image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.19.0
    name: csql-dagster-dagster

No issues appear in the logs for the init container, but my application receives an error:

FATAL:  Cloud SQL IAM service account authentication failed for user "<service_account>@<project>.iam"

After checking I've configured everything right a few times, I tried using the sidecar method instead of the operator using the example from the docs:

    - name: cloud-sql-proxy
      image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.19.0
      args:
        - "--private-ip"
        - "--auto-iam-authn"
        - "--structured-logs"
        - "--impersonate-service-account=<service_account>@<project>.iam.gserviceaccount.com"
        - "--port=5432"
        - "<my_instance_connection>"

This worked straight away.

After playing around with some of the arguments, I found using the instance configuration parameter of auto-iam-authn=true instead of the flag or environment variable is what causes the authentication to fail.

Works:

- "--auto-iam-authn"
- "<my_instance_connection>"

or

env:
  - name: CSQL_PROXY_AUTO_IAM_AUTHN
    value: 'true'
args:
  - "<my_instance_connection>"

Doesn't work:

- "<my_instance_connection>?auto-iam-authn=true"

I'm therefore curious:

1. Why would this behave differently - does the instance config do something separate from the flag/envvar ?
2. Should the operator be using one of those other methods and not the instance configuration for IAM Auth?

[dirsigler]

Got a similar issue. Using Impersonation and autoIAMAuthN: true kills here the whole flow. Means our Apps ServiceAccount (which is part of an IAM Group we added to CloudSQL to support many individual GSAs of many Apps) is not able to login to CloudSQL.

If I remove autoIAMAuthN: true it works perfectly fine with the impersonation.
Else I can add autoIAMAuthN: true and remove Impersonation (which is not suitable in our case).