Cuestomize Introduction

Cuestomize is a Kubernetes Package Manager using CUE-lang and integrated in Kustomize.

It is implemented as a Kustomize KRM function that reads a CUE model, and optionally some input resources from the Kustomize stream, generates some manifests, and passes them back to the Kustomize stream.

It provides the type-safety of CUE and the flexibility of kustomize, combined in a single tool.

Moreover, it allows your CUE model to consume resources from the Kustomize stream, which can be used to feed the CUE model with additonal contextual data.
This means with Cuestomize you have two ways to pass input data to your CUE model:

• the input section of the KRM function's specification (similar to Helm values)
• resources from the Kustomize stream.

The CUE model can then use the input values and resources to generate the output manifests.

The CUE model can either be pulled from an OCI registry, or be local to the KRM function (in which case you need to build and image that bundles both the CUE model and the Cuestomize binary).

Features

• Type-safety: CUE is a strongly typed language, so you are guaranteed that the generated resources are valid from a schema point of view.
• Flexibility: Kustomize is a very flexible tool, and Cuestomize lets you leverage that flexibility by being integrated in Kustomize, while still having the benefits of CUE.
• Modularity: CUE models can be composed together, so you can build complex models by combining simpler ones.
• OCI support: CUE models can be pulled from OCI registries, making it easy to share and reuse them.
• Validation: Cuestomize, leveraging CUE, gives you the power to validate your manifest generation process from end to end. You can validate both the input data and the generated resources against their respective schemas.
  With Cuestomize, you won't suffer from YAML indentation issues or misspelled fields anymore. If an unexpected field is found, or a required field is missing, CUE will raise an error during the evaluation of the model, preventing the generation of invalid manifests.

How it Works

Cuestomize is implemented as a Kustomize KRM function. It reads its configuration (and optionally other manifests) from the Kustomize input stream, unifies them with a CUE model of your choice, collects the model's outputs, and passes them back to Kustomize.

Visual Example Representation

A practical example of how data are passed and manifests generated is shown below.

Say that you have a CUE model that generates a ConfigMap with data taken from the input values, and a Deployment and a Service that are expected to be present in the Kustomize input stream.

Input Stream

On the left side of Figure 2, you can see the Kustomize input stream, which contains:

• the Cuestomize KRM function configuration, which specifies the CUE model to use, the input values to pass to the model, and which resources from the stream to forward to the model
• the other manifests from the Kustomize input stream.

CUE Model Unification

In Figure 2, on the center-top, you can see the CUE model that the function will use to generate the manifests.

In the center, you can see how the unified CUE model – i.e. the resulting CUE configuration after inputs and includes are forwarded to the model – would look like:

• the input field contains the input values forwarded from the function configuration
• the includes field contains the resources forwarded from the Kustomize input stream, in a map for ease of access
• the outputs field contains the generated resources, the ConfigMap in this case, which will be collected and passed back to Kustomize by Cuestomize.

Output Stream

On the right side of Figure 2, you can see the manifests that are collected by the function, only the ConfigMap in this case. The outputs manifests are then passed back to Kustomize, which can further process them.

Getting Started

This section will guide you through the steps to get started with Cuestomize.

01. The CUE Model

To get started with Cuestomize, you need to create a CUE module that defines your manifest generation logic.

You can either create your own CUE model from scratch, or use an existing one.

How to create CUE models that are compatible with Cuestomize is explained in different sections of this book, so we will use one of the existing models for this example.

02. The Kustomization

You need to have a Kustomization directory that you can use to run Kustomize. In this directory, you need to create a kustomization.yaml file that defines the resources you want to manage with Kustomize.

How to create a Kustomization project is out of the scope of this book, so we will assume you already have one, and the next steps will assume you are using the one under examples/simple/kustomize.

In your kustomization directory, you need to add a file that holds the configuration of the KRM function that will run Cuestomize. The name you give to this file is not important, as long as you reference it in the transformers section of your kustomization.yaml file.

Create the KRM Function Configuration File

Change directory, and create a file named krm-func.yaml in the kustomization directory:

# We assume you have git cloned the repo locally and are in the root directory of the repo
cd examples/simple/kustomize

touch krm-func.yaml

Note: the name you give to this file is not important, as long as you reference it in the transformers section of your kustomization.yaml file.

Update the Kustomization File

Edit the kustomization.yaml file to add the krm-func.yaml file to the transformers section:

kind: Kustomization

# ... other sections ...

transformers:
- krm-func.yaml

03. Configuring the KRM Function

Edit the krm-func.yaml file to configure the KRM function that will run Cuestomize.

Here is an example configuration:

apiVersion: cuestomize.dev/v1alpha1
kind: Cuestomization
metadata:
  name: example
  annotations:
    config.kubernetes.io/local-config: "true"
    config.kubernetes.io/function: |
      container:
        image: ghcr.io/workday/cuestomize:latest
        network: true
input:
  configMapName: example-configmap
includes:
- group: apps
  version: v1
  kind: Deployment
  name: example-deployment
  namespace: example-namespace
- version: v1
  kind: Service
  name: example-service
  namespace: example-namespace
remoteModule:
  registry: ghcr.io
  repo: workday/cuestomize/cuemodules/cuestomize-examples-simple
  tag: latest

Note: Cuestomize does not constrain the apiVersion and kind fields of the KRM function configuration, so you can use whatever values you want, as long as they are valid Kubernetes resource names. In your CUE model, on the other hand, you can constrain these to specific values in order to ensure compatibility between the model and the function's configuration.

04. Running Kustomize to Generate the Manifests

Now that you have everything set up, you can run Kustomize to generate the manifests.

Since Cuestomize is a KRM function, you'll need a few extra flags in order for kustomize build to work properly:

• --enable-alpha-plugins to enable the KRM function
• --network if your CUE model is pulled from a registry (can be omitted if the model is local to the function's image).

Build the Manifests

kustomize build . --enable-alpha-plugins --network

You should see the build to be successful, and the CUE-generated manifests within the printed manifests.

Configuration Reference

This section documents all configurable fields for a Cuestomize KRM function configuration.

KRM Function Configuration

Metadata

The metadata field of the configuration must contain some annotations in order for kustomize to recognise it as a KRM function.
On top of that, Cuestomize offers some configurations options through the .metadata field.
All these options are documented below.

Annotations

.metadata.annotations

Annotation – config.kubernetes.io/function

The annotation config.kubernetes.io/function is the one used by kustomize to configure a KRM function (kustomize docs).

Its value must contains the configuration for the container that runs the KRM function.

metadata:
  name: my-config
  annotations:
    config.kubernetes.io/function: |
      container:
        # the Cuestomize image you want to use
        image: ghcr.io/workday/cuestomize:latest

        # this is required to pull the CUE module from a registry
        network: true

⚠️ Passing environment variables to KRM functions is a discouraged practice (and may be removed in future kustomize versions), but is documented here for completeness. It also may be useful when developing to quickly iterate, without having to change the configuration.

The KRM function configuration also accepts environment variables to be passed to the container running the function, although that is discouraged and may be removed in future kustomize versions.

Cuestomize allows you to configure the logging level and pass the credentials for private registries through environment variables.

Annotation – config.cuestomize.io/validator

Setting config.cuestomize.io/validator: "true" in the configuration annotations tells Cuestomize to use the CUE module as a validator only: it will unify the inputs and includes with the module, but it won't collect the outputs.

This is useful if you want to validate a set of manifests with some CUE constraints, e.g. ensuring that all Deployments use a particular securityContext, or that resources in certain namespaces has a particular label, etc.

When used in validator mode, CUE will be used to validate, instead of to generate, and the behaviour you can expect is the same as running cue eval command.

Example

apiVersion: cuestomize.io/v1
kind: CuestomizeConfig
metadata:
  name: my-config
  annotations:
    config.kubernetes.io/function: |
      container:
        image: ghcr.io/workday/cuestomize:latest
        network: true
remoteModule:
  oci: docker.io/wackoninja/cuemodules:latest
includes:
- version: "v1"
  kind: ConfigMap
  name: "test-configmap"
  namespace: "test-namespace"

Pull From OCI Registry

Cuestomize can fetch CUE modules directly from an OCI registry (such as Docker Hub or GitHub Container Registry). This allows you to share and reuse CUE logic across projects and teams, and makes distributing your CUE models easier.

To pull a CUE module from an OCI registry, specify the remoteModule field in your Cuestomize KRM configuration.

Public Registries

ⓘ No auth is required for public modules.

To pull from a public registry, you don't need to specify the .remoteModule.auth field to pass the credentials, you just need to instruct the function on where the CUE module to pull is stored, and which tag you want to pull.

apiVersion: cuestomize.dev/v1alpha1
kind: Cuestomization
metadata:
  name: example
  annotations:
    config.kubernetes.io/local-config: "true"
    config.kubernetes.io/function: |
      container:
        image: ghcr.io/workday/cuestomize:latest
        network: true
input:
  configMapName: example-configmap
remoteModule:
  registry: ghcr.io
  repo: workday/cuestomize/cuemodules/cuestomize-examples-simple
  tag: latest

Private Registries (With Auth)

For private registries or repositories, you need to provide credentials. The recommended way is to use a Kubernetes Secret and reference it in your configuration.

You need to select a Kubernetes Secret through the remoteModule.auth field:

remoteModule:
  registry: ghcr.io
  repo: workday/cuestomize/cuemodules/cuestomize-examples-simple
  tag: latest
  auth:
    kind: Secret
    name: oci-auth

This tells Cuestomize to use the oci-auth Secret for authenticating to the registry.
The secret must be in the kustomize input stream to the function in order for it to be found and used by it.

💡 You can use Kustomize’s secretGenerator to create a Secret from environment variables:

.env file

username=<username>
password=<password>

`kustomization.yaml

secretGenerator:
- name: oci-auth
  envs:
  - .env
  options:
    disableNameSuffixHash: true
    annotations:
      config.kubernetes.io/local-config: "true"

This will generate a Secret named oci-auth with your credentials.

Glossary

• Cuestomize: both the KRM function and the binary that gets executed in the container.
• CUE model: it is a CUE-lang module that bundles the manifest generation logic. It is expected to have an outputs field which is a slice of KRM resources, and optionally an input field and an includes field (see CUE Model Integration for more details).
• Unified CUE model: the result of merging the CUE model with the input values and resources forwarded from the Kustomize input stream.
• Inputs: the data that is passed to the CUE model via the input field of the function's specification.
• Includes: resources that are passed to the CUE model via the includes field of the function's specification. These resources are taken from the Kustomize input stream.
• Outputs: the resources generated by the CUE model and passed back to Kustomize.