GitOps Action

GitOps action allows you to make commits and push to a remote repository.

edit-kubernetes-manifests-gitops.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: edit-kubernetes-manifests-gitops
spec:
  title: 'Edit Kustomize Resource'
  icon: 'flux'
  parameters:
    - default: 'chore: update $(.config.type)/$(.config.name)'
      label: 'Commit Message'
      name: 'commit_message'
    - default: '$(.config.config | toJSON | neat | json | toYAML)'
      label: 'Changes'
      name: 'yamlInput'
      properties:
        size: 'large'
      type: 'code'
  configs:
    - labelSelector: 'kustomize.toolkit.fluxcd.io/name'
  actions:
    - name: 'Create Pull Request With Changes'
      gitops:
        repo:
          url: 'https://github.com/flanksource/flux'
          connection: 'connection://default/github'
          base: 'main'
          branch: 'edit-manifest-$(random.Alpha 8)'
        commit:
          author: '$(.user.name)'
          email: '$(.user.email)'
          message: '$(.params.commit_message)'
        pr:
          title: '$(.params.commit_message)'
        patches:
          - path: 'prod/kustomization.yaml'
            yq: |
              select(
                .kind=="$(.config.config | jq `.kind`)" and
                .metadata.name=="$(.config.config | jq `.metadata.name`)"
              ) |= $(.params.yamlInput | yaml | toJSON)

Field	Description	Scheme
commit*	Commit details	Commit
name*	Step Name	string
repo*	The repository details	Repository
files	List of files to create	[]File
patches	List of JQ/YQ patches to apply	[]Patch
pr	Create a Pull Request instead of a commit (Only available for github and gitlab)	PullRequest
delay	A delay before running the action e.g. 8h	Duration or CEL with Playbook Context
filter	Conditionally run an action	CEL with Playbook Context
runsOn	Which runner (agent) to run the action on	[]Agent
templatesOn	Where templating (and secret management) of actions should occur	host or agent
timeout	Timeout on this action.	Duration

You must specify patches and/or files

Repository

Field	Description	Scheme
base	Branch to clone. Defaults to main	string
branch	The branch to update. Defaults to base	string
connection	Connection name to use the credentials for the git repo	string
type	Git repository type	github, gitlab, azure_devops, http, git
url	URL of the git repository	string

You must specify connection or url but not both

Commit Detail

Field	Description	Scheme	Required	Templatable
author	Author name	string	true	true
email	Author email	string	true
message	Commit message	string	true

Pull Request

Field	Description	Scheme	Required	Templatable
title	Title of the Pull request	string	true	true
tags	Tags to add to the PR	[]string		true

Patch

Patches modify existing files on the git repo. You can either use yq to modify yaml files or jq to modify json files.

Field	Description	Scheme	Required	Templatable
path	Path to patch	string	true	true
yq	yq query	string	false	true
jq	jq query	string	false	true

File

Files create or delete existing files on the git repo.

Field	Description	Scheme	Required	Templatable
path	Path to file	string	true	true
content	Content of the file. Use the $delete keyword to delete an existing file	string	true	true

Custom Origin Annotation

In the example above, you saw a playbook that edited Flux-created manifests. However, there was a slight issue - it had some variables hardcoded, specifically the git repository, git branch, and the kustomization file path. When you have configurations from multiple clusters, you'll need to duplicate the playbook for each cluster, as the git repository and kustomization file path will differ. Even if you have multiple Flux kustomizations on a single cluster, you'll need to duplicate the playbook for each of those kustomizations.

There's a better way to build a single generic GitOps playbook that works for configurations with any Flux source and kustomization.

When a GitOps action is run against a config resource that has a config.kubernetes.io/origin annotation, the following env vars are automatically available:

Field	Description	Scheme
git.branch	The primary branch	string
git.file	Name of the file that created the config resource the playbook is run against.	string
git.url	URL of the git repository	string
kustomize.path	Path of the kustomization file that created the config resource the playbook is run against. This comes from the spec of the kustomization.	string

These env vars are extracted by traversing up the Flux Kustomization and Git Repository that created the config resource the playbook is running against. We'll see this in detail shortly.

To tag all the resources with that annotation, you'll need to add originAnnotations to the buildMetadata field as shown below:

Setting origin annotation

kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
buildMetadata: [originAnnotations]
resources:
- monitoring

The above kustomization would add the following annotation to the config resource:

monitoring/alertmanager.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    config.kubernetes.io/origin: |
      path: monitoring/alertmanager.yaml
  ...

Example

Wihout origin annotation

edit-kubernetes-manifests-gitops.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: edit-kubernetes-manifests-gitops
spec:
  title: 'Edit Kustomize Resource'
  icon: flux
  parameters:
    - default: 'chore: update $(.config.type)/$(.config.name)'
      label: Commit Message
      name: commit_message
    - default: $(.config.config | toJSON | neat | json | toYAML)
      label: ""
      name: yamlInput
      properties:
        size: large
      type: code
  configs:
    - labelSelector: 'kustomize.toolkit.fluxcd.io/name'
  env:
    - name: file_path
      value: {{ .config.config | jq `.metadata.annotations["config.kubernetes.io/origin"]` | yaml).path }}
    - name: kustomization_path
      value:  {{ (catalog_traverse .config.id  "Kubernetes::Kustomization").Config | json | jq `.spec.path` }}
    - name: git_url
      value:  {{ (catalog_traverse .config.id  "Kubernetes::Kustomization/Kubernetes::GitRepository").Config | json | jq `.spec.url` }}
    - name: git_branch
      value:  {{ (catalog_traverse .config.id  "Kubernetes::Kustomization/Kubernetes::GitRepository").Config | json | jq `.spec.ref.branch` }}
  actions:
    - name: Create Pull Request With Changes
      gitops:
        repo:
          url: '$(.env.git_url)'
          connection: 'connection://default/github'
          base: '$(.env.git_branch)'
          branch: edit-manifest-$(random.Alpha 8)
        commit:
          author: '$(.user.name)'
          email: '$(.user.email)'
          message: $(.params.commit_message)
        pr:
          title: $(.params.commit_message)
        patches:
          - path: '$(filepath.Join .env.kustomization_path .env.file_path)'
            yq: |
              select(
                .kind=="$(.config.config | jq `.kind`)" and
                .metadata.name=="$(.config.config | jq `.metadata.name`)"
              ) |= $(.params.yamlInput | yaml | toJSON)

With origin annotation

edit-kubernetes-manifests-gitops.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: edit-kubernetes-manifests-gitops
spec:
  title: 'Edit Kustomize Resource'
  icon: flux
  parameters:
    - default: 'chore: update $(.config.type)/$(.config.name)'
      label: Commit Message
      name: commit_message
    - default: $(.config.config | toJSON | neat | json | toYAML)
      label: ''
      name: yamlInput
      properties:
        size: large
      type: code
  configs:
    - labelSelector: 'kustomize.toolkit.fluxcd.io/name'
  actions:
    - name: Create Pull Request With Changes
      gitops:
        repo:
          url: '$(.env.git.url)'
          connection: 'connection://default/github'
          base: '$(.env.git.branch)'
          branch: edit-manifest-$(random.Alpha 8)
        commit:
          author: '$(.user.name)'
          email: '$(.user.email)'
          message: $(.params.commit_message)
        pr:
          title: $(.params.commit_message)
        patches:
          - path: '$(filepath.Join .env.kustomization.path .env.git.file)'
            yq: |
              select(
                .kind=="$(.config.config | jq `.kind`)" and
                .metadata.name=="$(.config.config | jq `.metadata.name`)"
              ) |= $(.params.yamlInput | yaml | toJSON)

Templating

CEL Expressions

The following variables can be used within the CEL expressions of filter, if, delays and parameters.default:

Field	Description	Schema
config	Config passed to the playbook	ConfigItem
component	Component passed to the playbook	Component
check	Canary Check passed to the playbook	Check
playbook	Playbook passed to the playbook	Playbook
run	Current run	Run
params	User provided parameters to the playbook	map[string]any
request	Webhook request	Webhook Request
env	Environment variables defined on the playbook	map[string]any
user.name	Name of the user who invoked the action	string
user.email	Email of the user who invoked the action	string
agent.id	ID of the agent the resource belongs to.	string
agent.name	Name of the agent the resource belongs to.	string

Conditionally Running Actions

Playbook actions can be selectively executed based on CEL expressions. These expressions must either return

• a boolean value (true indicating run the action & skip the action otherwise)
• or a special function among the ones listed below

Function	Description
always()	run no matter what; even if the playbook is cancelled/fails
failure()	run if any of the previous actions failed
skip()	skip running this action
success()	run only if all previous actions succeeded (default)
timeout()	run only if any of the previous actions timed out

delete-kubernetes-pod.yaml
---
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: notify-send-with-filter
spec:
  parameters:
    - name: message
      label: The message for notification
      default: '{{.config.name}}'
  configs:
    - types: 
        - Kubernetes::Pod
  actions:
    - name: Send notification
      exec:
        script: notify-send "{{.config.name}} was created"
    - name: Bad script
      exec:
        script: deltaforce
    - name: Send all success notification
      if: success() # this filter practically skips this action as the second action above always fails
      exec:
        script: notify-send "Everything went successfully"
    - name: Send notification regardless
      if: always()
      exec:
        script: notify-send "a Pod config was created"

Defaulting Parameters

delete-kubernetes-pod.yaml
apiVersion:
  mission-control.flanksource.com/v1
  kind: Playbook
  metadata:
    name: edit
  spec:
    title: 'Edit Kustomize Resource'
    icon: flux
    parameters:
    - default: 'chore: update $(.config.type)/$(.config.name)'
      name: commit_message

Go Templating

When templating actions with Go Templates, the context variables are available as fields of the template's context object . eg .config, .user.email

Templating Actions

delete-kubernetes-pod.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: scale-deployment
spec:
  description: Scale Deployment
  configs:
    - types:
        - Kubernetes::Deployment
  parameters:
    - name: replicas
      label: The new desired number of replicas.
  actions:
    - name: kubectl scale
      exec:
        script: |
          kubectl scale --replicas={{.params.replicas}} \
            --namespace={{.config.tags.namespace}} \
            deployment {{.config.name}}

Functions

Function	Description	Return
getLastAction()	Returns the result of the action that just run	Action Specific
getAction({action})	Return the result of a specific action	Action Specific

Reusing Action Results

action-results.yaml
apiVersion: mission-control.flanksource.com/v1
kind: Playbook
metadata:
  name: use-previous-action-result
spec:
  description: Creates a file with the content of the config
  configs:
    - types:
        - Kubernetes::Pod
  actions:
    - name: Fetch all changes
      sql:
        query: SELECT id FROM config_changes WHERE config_id = '{{.config.id}}'
        driver: postgres
        connection: connection://postgres/local
    - name: Send notification
      if: 'last_result().count > 0'
      notification:
        title: 'Changes summary for {{.config.name}}'
        connection: connection://slack/flanksource
        message: |
          {{$rows:=index last_result "count"}}
          Found {{$rows}} changes