Streamlined Cluster Bootstrapping with Versioned Packages and Context-Specific IAM Policies

In our previous article we’ve explored practical challenges when managing IAM policies and provided examples. In this article, we'll examine how Appvia Wayfinder indirectly addresses some of these challenges through Wayfinder packages, which feature built-in templating to support context-specific IAM policies.

To explore how Wayfinder addresses these challenges through its versioned packages, let's first understand the fundamentals of creating a package in Wayfinder.

‍

Creating a Package in Wayfinder

Wayfinder provides versioned packages as a secure and consistent delivery method for bootstrapping clusters with software using cluster plans.

‍

In Wayfinder, packages provide a delivery method for bootstrapping multiple clusters with software. Wayfinder offers versioned packages that include generic configuration that can be automatically tailored for all the specific environments  that development teams can create.

‍

To ensure that your versioned package is created correctly, you need to:

‍

1. Provide General Information: This section requires you to specify essential package details such as name, description, package version, and Helm chart details like repository URL and Helm chart version. This information is important for identifying and understanding the package and its contents when it is used in other parts of Wayfinder, such as cluster plans.
2. Configure Workload Identity Settings: This section allows you to configure workload identity settings for each cloud provider. Setting up a workload identity ensures that your package, when released, can securely interact with cloud resources by providing the necessary authentication and authorisation.
3. Define Policy Definitions: When using a workload identity, you need to outline the actions that the Kubernetes service account is permitted to undertake in the cloud. This includes interactions with cloud services like DNS zones or cloud storage, ensuring that your package operates with the required permissions.
4. Define the Helm Chart Configuration: A Helm chart packages pre-configured Kubernetes resources, including templates, configuration files, and metadata. Defining this configuration ensures that your application or service can be correctly deployed in a Kubernetes cluster.
5. Define Supporting Manifests: Your package might need additional configurations not included in its Helm Chart but required for its functionality. For instance, if your package deploys cert-manager, it may need to configure solvers using the cert-manager ClusterIssuer resource. Providing these supporting manifests ensures the package operates correctly.
6. Validation: You need to validate the package's configuration by deploying it to a test cluster. Successful validation confirms that the package works as intended. 
7. Deployment: Once validated, you can add the package to one or more cluster plans, ensuring that it will be deployed consistently to all clusters built from those plans.

Cluster plans in Wayfinder serve as versioned blueprints for creating one or more clusters. Each cluster plan version outlines the cluster's configuration which includes the cluster's network and the versioned packages that'll be deployed to each cluster created with a particular version of the cluster plan.

‍

Next, we'll explore how Wayfinder’s versioned packages address the IAM challenges highlighted in our previous blog article.

‍

IAM Challenges and Corresponding Solutions

Appvia Wayfinder indirectly addresses IAM policy management challenges by using versioned packages to ensure predictable releases, central management, efficient change propagation, dependency management, auditing, compliance, context adaptation, validation coverage, interoperability, and team coordination, thereby streamlining secure and consistent deployments across multiple clusters and environments.

‍

In our exploration of IAM policy management challenges, we've highlighted several complexities faced by organisations managing policies across multiple tenants. Appvia Wayfinder addresses these challenges indirectly through its innovative use of versioned packages:

‍

‍

Predictable Releases

• Challenge: Maintaining consistent version control across multiple clusters and tenants can be complex, leading to potential inconsistencies and security vulnerabilities.
• Solution: Wayfinder pre-validates package configurations against test clusters, ensuring deployments succeed reliably in targeted environments.

Central Management 

• Challenge: Centrally managing IAM policies for various clusters and tenants is intricate and prone to deployment errors.
• Solution: Wayfinder offers a web interface that provides centralises visibility into where versioned cluster plans and packages are utilised, ensuring consistent policy enforcement and minimising operational risks.

Change Management 

• Challenge: Verifying changes across different environments and ensuring compatibility is challenging.
• Solution: Wayfinder supports updates to packages and cluster plans through semantic versioning, allowing tailored configurations to be applied efficiently, ensuring isolation and correct functioning.

Change Propagation 

• Challenge: Efficiently propagating policy changes across clusters is crucial to maintain up-to-date configurations.
• Solution:  Wayfinder encapsulates validated, versioned packages into cluster plans, ensuring reliable propagation through isolated deployment stages (e.g., ”PlatformTesting”, “Development” and then “Production”).

Dependency Management 

• Challenge: Managing dependencies like Kubernetes service accounts and cloud IAM roles requires precise tracking to prevent disruptions.
• Solution: Wayfinder's versioned packages specify dependencies and include templated configurations, ensuring seamless integration and functionality upon deployment.

Auditing and Compliance 

• Challenge: Ensuring consistent compliance across environments demands automated checks and regular audits.
• Solution: Wayfinder allows tailored access permissions per package version, mitigating security risks and ensuring compliance across all clusters.

Context Adaptation 

• Challenge: Adapting policies to different cluster configurations without compromising security requires flexibility and configurability.
• Solution: Wayfinder supports versioned packages with configurable templates, enabling context-specific deployments while maintaining security standards.

Validation Coverage

• Challenge: Regular validation is essential to ensure policy integrity over time.
• Solution: Wayfinder's comprehensive validation criteria cover template structures, resolved values, and workload identity policies, ensuring deployments align with evolving application needs.

Interoperability

• Challenge: Testing policy changes alongside application deployments is critical to prevent vulnerabilities.
• Solution: Wayfinder facilitates integrated testing of cluster plans and packages, ensuring configurations are correct and secure upon deployment.

Team Coordination

• Challenge: Effective communication among policy management teams is critical to avoid conflicting changes and misconfigurations.
• Solution: Wayfinder empowers teams with self-service provisioning within defined guardrails, using validated cluster plans and packages to enhance coordination and operational efficiency.

From these insights, it's clear that while Wayfinder doesn’t directly manage all IAM Policy Management challenges, it does offer robust solutions to address related challenges when deploying packages with the context-specific IAM policy required to bootstrap clusters.

‍

Let's illustrate this in practice with ExternalDNS as an example, demonstrating how it can be used to bootstrap a cluster using the ExternalDNS package.

‍

Use case: Bootstrapping a cluster with the ExternalDNS package

‍

Background

An organisation hosting multiple tenants on a multi-cloud environment (AWS, Azure, and GCP) faces challenges in granting the workload identity the proper permissions needed for ExternalDNS to function. ExternalDNS manages DNS records for Kubernetes resources by dynamically updating DNS providers (like Route 53, Azure DNS, and Cloud DNS) based on changes in the Kubernetes cluster. Ensuring security and isolation while managing these permissions across different cloud providers is essential.

‍

Challenge

The primary challenge is to provide tenants with cloud-specific DNS resolution for their Kubernetes clusters while maintaining strict security and isolation. The platform engineering team requires a centralised approach to manage ExternalDNS configurations across multiple clouds, minimising the overhead of managing permissions to the workload identity. Key challenges include:

‍

• Granting Permissions: Ensuring the workload identity has necessary permissions to manage DNS records across AWS, Azure, and GCP environments, maintaining security and compliance.
• Central Management: Synchronising ExternalDNS configurations across multiple clusters and cloud environments centrally to prevent inconsistencies and errors.
• Change Propagation: Efficiently propagating policy changes across all clusters to ensure up-to-date configurations and minimise the risk of outdated policies.
• Version Control and Validation: Implementing version control for the ExternalDNS package and validating configurations before deployment to prevent issues in production environments.

By addressing these challenges, the organisation aims to simplify the management of ExternalDNS permissions and configurations across its multi-cloud infrastructure, ensuring secure and efficient DNS resolution for all tenants.

‍

Solution

Wayfinder offers a centralised platform for maintaining the entire lifecycle of a package like ExternalDNS. It manages multiple versions of packages, enabling you to centrally update the latest version and manage package updates without disruption. Workload identities are configured on a per-package basis and across clouds as needed. Wayfinder uses familiar tools like the Go templating language, allowing you to utilise a comprehensive set of Wayfinder variables to resolve values based on the context, such as cloud provider and cluster name.

‍

Steps to Bootstrap a Cluster with the ExternalDNS Package:

1. Create a Wayfinder Package for ExternalDNS using the CLI or Web Interface.
2. Specify required cloud access for the workload identity.
3. Define access policies using Wayfinder’s templating language and variables to grant permissions based on the cloud provider and accessible DNSZone resources.
4. Configure Helm chart settings and include any necessary configurations.
5. Validate the package configuration on a test cluster using Wayfinder’s `wf validate package` command.
6. Include the validated ExternalDNS package in a Wayfinder cluster plan.
7. Use the cluster plan to bootstrap new clusters with the pre-configured ExternalDNS package.

By following these steps, you can efficiently manage and deploy the ExternalDNS package across multiple clouds while ensuring proper permissions, security, and compliance.

‍

In the next section, we provide a practical example of how to bootstrap cross-cloud clusters with the ExternalDNS package.

‍

Practical Example: Bootstrapping a cluster with the ExternalDNS package using Wayfinder

This section illustrates how to bootstrap cross-cloud clusters with the ExternalDNS package using Wayfinder. Wayfinder allows you to create versioned packages using its CLI or Web Interface. For newcomers, starting with the Web Interface helps in understanding Wayfinder concepts and the structure of Wayfinder package YAML files.

‍

Follow the steps below to create a Wayfinder Package for ExternalDNS using Wayfinder's Web Interface and CLI.

‍

Step 1 - Specify General Information

• Specify general information like, package name, package version, Helm Repo URL and Helm Chart version.

Step 2 - Define Access Policies 

• Specify that the workload identity requires cloud access and indicate which clouds it needs access to.
• AWS templated policy example:

Version: "2012-10-17"
Statement:
{{- if .Cluster.DNSZones }}
- Sid: "AllowRoute53ChangeResourceRecordSets"
  Effect: "Allow"
  Action:
  - "route53:ChangeResourceRecordSets"
  Resource:
  {{- range .Cluster.DNSZones }}
  {{- if .AWS }}
  - "arn:aws:route53:::{{ trimPrefix "/" .ZoneID }}"
  {{- end }}
  {{- end }}
{{- end }}
# must always have a valid role for the workload identity to work
- Sid: "AllowRoute53List"
  Effect: "Allow"
  Action:
  - "route53:GetChange"
  - "route53:ListHostedZones"
  - "route53:ListResourceRecordSets"
  - "route53:ListHostedZonesByName"
  Resource:
  - "*"
  

‍

• Azure templated policy example:

# must always have a valid role for the workload identity to work
- scope: /subscriptions/{{ .CloudAccessConfig.Azure.Subscription }}
  roleDefinitionName: Reader
{{- if .Cluster.DNSZones }}
  {{- range .Cluster.DNSZones }}
    {{- if not .Private }}
- scope: {{ .ZoneID }}
  roleDefinitionName: Contributor
    {{- end }}
  {{- end }}
{{- end }}

‍

• GCP templated policy example:

# must always have a valid role for the workload identity to 
work
- role: roles/dns.reader
  scopeType: Project
{{- if .Cluster.DNSZones }}
  {{- range .Cluster.DNSZones }}
    {{- if .GCP }}
- role: roles/dns.admin
  scopeType: Project
    {{- end }}
  {{- end }}
{{- end }}

‍

‍

Step 3 - Add Helm Chart Configuration and Supporting Manifests

Include any necessary Helm chart configurations. 

• Use static configuration to override default Helm chart settings for the EcternalDNS package. These settings will be applicable to all clusters, such as resource limits.
• Use dynamic configuration to create a workload identity per cloud if it doesn’t already exist. 
• Note: The service account must be created by either the Helm chart or the manifests. Configuring the correct name and annotations for a service account requires the cloud context using the correct value keys specific to the Helm Chart. Each chart has unique values.
• Note: In this example ExternalDNS does not require any supporting manifests.

domainFilters:
{{- range .Cluster.DNSZones }}
- {{ .Domain }}
{{- end }}
fullnameOverride: external-dns
logFormat: json
policy: sync
txtOwnerId: {{ .Cluster.Name }}
replicas: 2
crd:
  create: false
metrics:
  enabled: true
  serviceMonitor:
    enabled: false
rbac:
  apiVersion: v1
resources:
  limits:
    cpu: 20m
    memory: 50Mi
  requests:
    cpu: 10m
    memory: 50Mi
serviceAccount:
  create: true
  name: {{ .Package.WorkloadIdentity.ServiceAccountName }}
  annotations:
{{ .Package.WorkloadIdentity.ServiceAccountAnnotations | toYaml | indent 4 }}
podLabels:
{{ .Package.WorkloadIdentity.PodLabels | toYaml | indent 2 }}

{{- if .Cluster.AWS }}
provider: aws
aws:
  region: {{ .Cluster.Region }}
{{- end }}
{{- $resGrp := "" }}
{{- range .Cluster.DNSZones }}
  {{- if not .Private }}
    {{- $resGrp = .Azure.ResourceGroup }}
  {{- end }}
{{- end }}
{{- if $resGrp }}
provider: azure
azure:
  tenantId: {{ .CloudAccessConfig.Azure.Tenant }}
  subscriptionId: {{ .CloudAccessConfig.Azure.Subscription }}
  useWorkloadIdentityExtension: true
  resourceGroup: {{ $resGrp }}
{{- end }}
{{- if .Cluster.GCP }}
provider: google
{{- end }}

‍

‍

Step 4 - Test the Package Configuration

Validate the package configuration on a test cluster using Wayfinder’s `wf validate package` command. In the following example the validation fails as Wayfinder detected that the Service Account specified in the template doesn’t exist in the cloud and that you haven’t specified that it should be created.

‍

‍

Step 5 - Include the Package in a Wayfinder Cluster Plan

• Once validated, include the ExternalDNS package in a Wayfinder cluster plan.

Following these steps enables efficient management and deployment of the ExternalDNS package across multiple clouds using Wayfinder, effectively addressing IAM challenges discussed earlier in this article.

‍

Conclusion

Appvia Wayfinder provides a robust and efficient solution for managing IAM-related challenges through the use of versioned packages. By offering a centralised platform for package creation, configuration, and management, Wayfinder helps streamline the process of bootstrapping clusters with essential services like ExternalDNS. It addresses critical IAM challenges such as central management, change propagation, and dependency management. 

‍

Through its use of familiar tools like the Go templating language and a focus on validation and managed versions, Wayfinder ensures secure, compliant, and consistent deployment of packages across multi-cloud environments. This approach not only simplifies IAM management but also enhances overall operational efficiency and security.