Modes

Airways are custom resources that allow you to create custom resource definitions bound to a flight implementation. This enables us to manage our packages as Kubernetes resources. Airway modes refer to how the subresources of our packages deployed by the ATC are managed.

Airways support three modes:

• standard
• static
• dynamic
• subscription

An airway’s mode can be set as part of that Airway’s specification:

apiVersion: yoke.cd/v1alpha1
kind: Airway
metadata:
  name: modes.examples.com
spec:
  mode: dynamic
  # ... other fields ...

Let’s explore what each of these modes do.

Standard

Standard mode is the default and is equivalent to leaving the mode property unset.

In Standard mode, subresources of your packages are left unmanaged. This means third-party actors (users with API access, other controllers, and so on) can make changes to subresources created by the ATC.

These changes will persist until the package’s custom resource is modified and the ATC computes a new desired state for the package — at which point any conflicting changes made by other actors will be overwritten via server-side apply.

The only way to rectify drift in this mode without updating the package’s custom resource is to set the fixDriftInterval in the airway’s specification.

This mode is useful when you want the flexibility to make manual changes to your cluster.

Static

Static mode, as its name implies, aims to prevent drift from the desired state from happening at all.

If a subresource of a package is modified in a way that conflicts with the desired state, the update is rejected at admission, and you will see an error similar to the following:

admission webhook “resources.yoke.cd” denied the request: cannot modify flight sub-resources.

This mode is useful when you want to lock down changes to your cluster, keeping drift to a minimum.

In this mode, users must modify the custom resource representing the package to make changes, and are disallowed from modifying its subresources directly.

Although this mode is powerful and generally prevents drift, the admission webhook is configured to be ignored on failure. This means that in rare, unexpected cases, drift can still occur. Therefore, it is still recommended to set the fixDriftInterval property.

Dynamic

In Dynamic mode, changes to any subresources of packages managed by the ATC will requeue the package for evaluation.
Unlike Static mode, where modifications are compared to the current desired state and rejected if they conflict,
Dynamic mode always allows modifications to pass admission.

However, even if a conflicting state is admitted, the flight will be re-evaluated and the desired state reapplied.
This mode is most similar to ArgoCD’s self-healing behavior. It’s like using fixDriftInterval, except drift detection is triggered not on some interval duration, but on every change to a subresource.

Dynamic mode has one particularly interesting property: since the flight is being re-evaluated, it doesn’t need to return the old desired state—it can compute a new one.
This, coupled with cluster-access, means that flight implementations can react to changes made by users or controllers to their subresources.

Let’s take a look at how this solves a classic Kubernetes problem: redeploying a Deployment when a Secret changes.

Note:
We’ll be discussing this example at a high level. A fully working code example will be added to Yoke’s examples repository in the future.

Suppose we have an Airway called backends.examples.com, and the flight associated with it produces three resources:

• A Deployment
• A Secret
• An ExternalSecret

An ExternalSecret is a custom resource from the External Secrets Operator project that allows you to sync secrets from external sources like Vault, AWS Secrets Manager, Google Secret Manager, and more.

Our flight would have the following properties:

• Cluster-access is enabled at the Airway level.
• The Secret computes itself by reading its own data from the cluster.
• The ExternalSecret references the Secret.
• The Secret’s data is hashed and used as a label on the Deployment.

And that’s it. When a Secret changes in an external source, the following occurs:

1. The External Secrets Operator updates the Secret.
2. The package is requeued for evaluation because the Secret was modified.
3. The flight reads the Secret’s data, hashes it, and updates the Deployment label.
4. The Deployment is redeployed.

The same effect would occur if we updated the Secret directly via kubectl, but this example offers a more realistic, real-world use case for how this might work.

Instance status

A dynamic Airway instance serves as a control-loop for desired package state. On events to our instance resources (and external ones too) the instance is re-evaluated and a new desired state computed.

In this scenario it is useful to report back the state of the package to the top-level instance custom resource’s status.

To do so, as a special case you can optionally include the top-level instance resource in the output of the flight. When doing so you assume control of the status of your top-level instance.

If omitted, the ATC defaults to setting the status ready condition when all resources deployed by the instance are ready. This is best effort, as yoke can only know the ready conditions for a set of known APIs.

A note on external-resources

Prior to version 0.15.8 of the AirTrafficController, updates to external resources, Kubernetes resources outside of your current instance package but referenced via resource matchers, did not trigger a re-evaluation of the flight.

As of version 0.15.8, any external resource that is looked up will now trigger a re-evaluation of the instance on update or deletion. This allows flights to react to changes in external state and keep in sync with resources elsewhere in the cluster.

Subscription

Subscription mode is very similar to Dynamic mode, as it allows you to make decisions based on state changes within your cluster by dynamically reacting to events.

The key difference is what triggers a re-evaluation. While Dynamic mode requeues the package on any change to a subresource, Subscription mode is only triggered by changes to subresources and external resources that the flight has explicitly subscribed to. This is achieved by looking up the resource via the wasi interface.

This mode acts as a halfway point between Standard mode and Dynamic mode. State that is not subscribed to can be modified by external actors (similar to Standard mode), but changes to subscribed state trigger a complete re-evaluation.

The main advantage is that Subscription mode is less resource intensive than pure Dynamic mode, making it a good fit for flights that contain many unstable resources (i.e., resources that receive frequent updates). It also allows more flexibility for performing manual edits. As with other modes, using fixDriftInterval can still serve as a basic self-healing mechanism for any unsubscribed state.

Overrides

Modes are described at the Airway level. However sometimes we may want to change the mode for a specific package without affecting all resources associated with our Airway.

To achieve this the ATC will use the mode specified by the annotation: overrides.yoke.cd/mode. Given that this is an annotation, if the value is not exactly one of standard, static or dynamic it will be ignored.

If we use our Backends example from the previous section, we could override the mode for a specific package like so:

apiVersion: examples.com/v1
kind: Backend
metadata:
  annotations:
    # This sets the subresources of this backend to be managed by the atc given this specific mode.
    # This overrides the mode set at the Airway level defining Backends.
    overrides.yoke.cd/mode: standard
  name: demo
spec:
  image: nginx:latest
  replicas: 2