Disable auto_minor_version_upgrade for Aurora RDS cluster

When deploying a PoC of Astronomer Enterprise I left it running over the weekend and the next week the maintenance window took place and the cluster was upgraded from PostgreSQL 10.6 to 10.7.

When deploying a change on min_cluster_size/max_cluster_size that applies for the EKS workers, I realized on the output that terraform wanted to delete the RDS cluster since it was downgrading it from PostgreSQL 10.7 back to 10.6 that is on the terraform code.

Can we please set auto_minor_version_upgrade to false?, this might help to solve the issue.


References:


Extra info:

$ terraform show

	...

# module.astronomer-enterprise.module.aws.module.aurora.aws_rds_cluster.this:
resource "aws_rds_cluster" "this" {
	...
    engine                              = "aurora-postgresql"
    engine_version                      = "10.6"
    preferred_maintenance_window        = "sun:05:00-sun:06:00"
	...
}

# module.astronomer-enterprise.module.aws.module.aurora.aws_rds_cluster_instance.this[0]:
resource "aws_rds_cluster_instance" "this" {
	...
    auto_minor_version_upgrade   = true
    engine                       = "aurora-postgresql"
    engine_version               = "10.6"
    preferred_maintenance_window = "sun:05:00-sun:06:00"
	...
}

	...

I’m still having issues with this, when I do terraform plan the set of changes to be applied show that the PostgreSQL version on the cluster will be downgraded from 10.7 to 10.6!

$ terraform plan

        ...

  # module.astronomer-enterprise.module.aws.module.aurora.aws_rds_cluster_instance.this[0] must be replaced
-/+ resource "aws_rds_cluster_instance" "this" {
        apply_immediately               = true
      ~ arn                             = "arn:aws:rds:us-east-1:XXXXXXXXXXXX:db:astrodb-XXXXXXXXXXXXXXXX-X" -> (known after apply)
        auto_minor_version_upgrade      = true
      ~ availability_zone               = "us-east-1b" -> (known after apply)
        cluster_identifier              = "astrodb-XXXXXXXXXXXXXXXX"
        copy_tags_to_snapshot           = false
        db_parameter_group_name         = "astronomer-aurora-db-postgres-parameter-group-XXXXXXXXXXXXXXXX"
        db_subnet_group_name            = "astrodb-XXXXXXXXXXXXXXXX"
      ~ dbi_resource_id                 = "db-XXXXXXXXXXXXXXXXXXXXXXXXXX" -> (known after apply)
      ~ endpoint                        = "astrodb-XXXXXXXXXXXXXXXX-X.XXXXXXXXXXXX.us-east-1.rds.amazonaws.com" -> (known after apply)
        engine                          = "aurora-postgresql"
      ~ engine_version                  = "10.7" -> "10.6" # forces replacement
      ~ id                              = "astrodb-XXXXXXXXXXXXXXXX-X" -> (known after apply)
        identifier                      = "astrodb-XXXXXXXXXXXXXXXX-X"
      + identifier_prefix               = (known after apply)
        instance_class                  = "db.r4.large"
      ~ kms_key_id                      = "arn:aws:kms:us-east-1:XXXXXXXXXXXX:key/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" -> (known after apply)
        monitoring_interval             = 0
      + monitoring_role_arn             = (known after apply)
        performance_insights_enabled    = false
      + performance_insights_kms_key_id = (known after apply)
      ~ port                            = 5432 -> (known after apply)
      ~ preferred_backup_window         = "02:00-03:00" -> (known after apply)
        preferred_maintenance_window    = "sun:05:00-sun:06:00"
        promotion_tier                  = 1
        publicly_accessible             = false
      ~ storage_encrypted               = true -> (known after apply)
        tags                            = {
            "Billing"       = "astronomer-airflow"
            "Deployment ID" = "astronomer"
            "Terraform"     = "true"
        }
      ~ writer                          = true -> (known after apply)
    }

        ...

I dig into the terraform modules from Astronomer and found the following:

  • I use astronomer/astronomer-enterprise/aws version 0.0.182 on my main.tf

  • terraform-aws-astronomer-enterprise uses astronomer/astronomer-aws/aws version 1.1.101 [1]. It doesn’t specify the PostgreSQL version to use for Aurora

[1] https://github.com/astronomer/terraform-aws-astronomer-enterprise/blob/0.0.182/main.tf#L2

  • terraform-aws-astronomer-aws uses terraform-aws-modules/rds-aurora/aws version 2.11.0 [2]. It specifies that postgresql-aurora should be version 10.6, doesn’t say anything about auto_minor_version_upgrade

[2] https://github.com/astronomer/terraform-aws-astronomer-aws/blob/1.1.101/db.tf#L15

  • terraform-aws-modules/terraform-aws-rds-aurora specifies the aws_rds_cluster_instance resource [3] where the auto_minor_version_upgrade parameter is set from the variables.tf file, it is set to true by default.

[3] https://github.com/terraform-aws-modules/terraform-aws-rds-aurora/blob/v2.11.0/main.tf#L95
[4] https://github.com/terraform-aws-modules/terraform-aws-rds-aurora/blob/v2.11.0/variables.tf#L122-L126


Can Astronomer please:

  1. Override the auto_minor_version_upgrade parameter of the aws_rds_cluster_instance resource on terraform-aws-astronomer-aws [2] with the value false so our cluster doesn’t upgrade minor version and we don’t have inconsistencies between terraform code and the actual infrastructure.

  2. Bump the PostgreSQL engine_version from 10.6 to 10.7 on terraform-aws-modules/rds-aurora so I can apply my changes without downgrading the instance from 10.7 to 10.6 because I’m not sure what would happen if I accept the downgrade on terraform. I assumme the snapshot will fail to import (if any) and I’m afraid of having data loss on our prod cluster db :frowning:

I’ve already set a support ticket, I’m documenting this in case anyone faces the same issue :expressionless:

I created a PR so you guys can merge :slight_smile: