Domains

• 1: API Domain
• 2: File Domain
• 3: Kubernetes Domain

# ... Rest of Lula Validation
domain:
    type: kubernetes   # kubernetes or api accepted
    kubernetes-spec:
        # ... Rest of kubernetes-spec
# ... Rest of Lula Validation

1 - API Domain

The API Domain allows for collection of data (via HTTP Get or Post Requests) generically from API endpoints.

[!Important] This domain supports both read and write operations (Get and Post operations), so use with care. If you configure validations with API calls that mutate resources, add the executable flag to the request so that Lula will ask for verification before making the API call.

Specification

The API domain Specification (api-spec) accepts a list of requests and an options block. options can be configured at the top-level and will apply to all requests except those which have embedded options. request-level options will override top-level options.

domain: 
  type: api
  api-spec:
    # options (optional): Options specified at this level will apply to all requests except those with an embedded options block.
    options:
      # timeout (optional, default 30s): configures the request timeout. The default timeout is 30 seconds (30s). The timeout string is a number followed by a unit suffix (ms, s, m, h, d), such as 30s or 1m.
      timeout: 30s
      # proxy (optional): Specifies a proxy server for all requests.
      proxy: "https://my.proxy"
      # headers (optional):  a map of key value pairs to send with all requests.
      headers: 
        key: "value"
        my-customer-header: "my-custom-value"
    # Requests is a list of URLs to query. The request name is the map key used when referencing the resources returned by the API.
    requests:
      # name (required): A descriptive name for the request.
      - name: "healthcheck" 
        # url (required): The URL for the request. The API domain supports any rfc3986-formatted URI. Lula also supports URL parameters as a separate argument.
        url: "https://example.com/health/ready"
        # method (optional, default get): The HTTP Method to use for the API call. "get" and "post" are supported. Default is "get".
        method: "get"
        # parameters (optional): parameters to append to the URL. Lula also supports full URIs in the URL.
        parameters: 
          key: "value"
        # Body (optional): a json-compatible string to pass into the request as the request body.
        body: |
stringjsondata
        # executable (optional, default false): Lula will request user verification before performing API actions if *any* API request is flagged "executable".
        executable: true
        # options (optional): Request-level options have the same specification as the api-spec-level options at the top. These options apply only to this request.
        options:
          # timeout (optional, default 30s): configures the request timeout. The default timeout is 30 seconds (30s). The timeout string is a number followed by a unit suffix (ms, s, m, h, d), such as 30s or 1m.
          timeout: 30s
          # proxy (optional): Specifies a proxy server for this request.
          proxy: "https://my.proxy"
          # headers (optional): a map of key value pairs to send with this request.
          headers: 
            key: "value"
            my-customer-header: "my-custom-value"
      - name: "readycheck"
      # etc ...

API Domain Resources

The API response body is serialized into a json object with the request name as the top-level key. The API status code is included in the output domain resources under status. raw contains the entire API repsonse in an unmarshalled (json.RawMessage) format.

Example output:

"healthcheck": {
  "status": 200,
  "response": {
    "healthy": true,
  },
  "raw": {"healthy": true}
}

The following example validation verifies that the request named “healthcheck” returns "healthy": true

provider: 
  type: opa
  opa-spec:
    rego: |
      package validate

      validate {
        input.healthcheck.response.healthy == true
      }

You can define a variable to simplify referencing the response:

provider: 
  type: opa
  opa-spec:
    rego: |
      package validate

      resp := input.healthcheck.response

      validate {
        resp == true
      }

2 - File Domain

The File domain allows for validation of arbitrary file contents from a list of supported file types. The file domain can evaluate local files and network files. Files are copied to a temporary directory for evaluation and deleted afterwards.

Specification

The File domain specification accepts a descriptive name for the file as well as its path. The filenames and descriptive names must be unique.

domain:
  type: file
  file-spec:
    filepaths:
    - name: config
      path: grafana.ini
      parser: ini         # optionally specify which parser to use for the file type

Supported File Types

The file domain uses OPA’s conftest to parse files into a json-compatible format for validations. Both OPA and Kyverno (using kyverno-json) can validate files parsed by the file domain.

The file domain includes the following file parsers:

• cue
• cyclonedx
• dockerfile
• dotenv
• edn
• hcl1
• hcl2
• hocon
• ignore
• ini
• json
• jsonc
• jsonnet
• properties
• spdx
• string
• textproto
• toml
• vcl
• xml
• yaml

The file domain can also parse arbitrary file types as strings. The entire file contents will be represented as a single string.

The file parser can usually be inferred from the file extension. However, if the file extension does not match the filetype you are parsing (for example, if you have a json file that does not have a .json extension), or if you wish to parse an arbitrary file type as a string, use the parser field in the FileSpec to specify which parser to use. The list above contains all the available parses.

Validations

When writing validations against files, the filepath name must be included as the top-level key in the validation. The placement varies between providers.

Given the following ini file:

[server]
# Protocol (http, https, socket)
protocol = http

The below Kyverno policy validates the protocol is https by including Grafana as the top-level key under check:

metadata:
  name: check-grafana-protocol
  uuid: ad38ef57-99f6-4ac6-862e-e0bc9f55eebe
domain:
  type: file
  file-spec:
    filepaths:
    - name: 'grafana'
      path: 'custom.ini'
provider:
  type: kyverno
  kyverno-spec:
    policy:
      apiVersion: json.kyverno.io/v1alpha1
      kind: ValidatingPolicy
      metadata:
        name: grafana-config
      spec:
        rules:
        - name: protocol-is-https
          assert:
            all:
            - check:
                grafana:
                  server:
                    protocol: https

While in an OPA policy, the filepath Name is the input key to access the config:

metadata:
  name: validate-grafana-config
  uuid: ad38ef57-99f6-4ac6-862e-e0bc9f55eebe
domain:
  type: file
  file-spec:
    filepaths:
    - name: 'grafana'
      path: 'custom.ini'
provider:
  type: opa
  opa-spec:
    rego: |
      package validate
      import rego.v1

      # Default values
      default validate := false
      default msg := "Not evaluated"
      
      validate if {
       check_grafana_config.result
      }
      msg = check_grafana_config.msg

      config := input["grafana"]
      protocol := config.server.protocol

      check_grafana_config = {"result": true, "msg": msg} if {
        protocol == "https"
        msg := "Server protocol is set to https"
      } else = {"result": false, "msg": msg} if {
        protocol == "http"
        msg := "Grafana protocol is insecure"
      }

    output:
      validation: validate.validate
      observations:
        - validate.msg

Parsing files as arbitrary strings

Files that are parsed as strings are represented as a key-value pair where the key is the user-supplied file name and the value is a string representation of the file contexts, including special characters, for e.g. newlines (\n).

As an example, let’s parse a similar file as before as an arbitrary string.

When reading the following multiline file contents as a string:

server = https
port = 3000

The resources for validation will be formatted as a single string with newline characters:

{"config": "server = https\nport = 3000"}

And the following validation will confirm if the server is configured for https:

  domain:
    type: file
    file-spec:
      filepaths:
      - name: 'config'
        path: 'server.txt'
        parser: string
  provider:
    type: opa
    opa-spec:
      rego: |
        package validate
        import rego.v1

        # Default values
        default validate := false
        default msg := "Not evaluated"
        
        validate if {
          check_server_protocol.result
        }
        msg = check_server_protocol.msg
        
        config := input["config"]
        
        check_server_protocol = {"result": true, "msg": msg} if {
          regex.match(
            `server = https\n`,
            config
          )
          msg := "Server protocol is set to https"
        } else = {"result": false, "msg": msg} if {
          regex.match(
            `server = http\n`,
            config
          )
          msg := "Server Protocol must be https - http is disallowed"
        }

      output:
        validation: validate.validate
        observations:
          - validate.msg

Note on Compose

While the file domain is capable of referencing relative file paths in the file-spec, Lula does not de-reference those paths during composition. If you are composing multiple files together, you must either use absolute filepaths (including network filepaths), or ensure that all referenced filepaths are relative to the output directory of the compose command.

Evidence Collection

The use of lula dev get-resources and lula validate --save-resources will produce evidence in the form of json files. These files provide point-in-time evidence for auditing and review purposes.

Evidence collection occurs for each file specified and - in association with any error will produce an empty representation of the target file(s) data to be collected.

3 - Kubernetes Domain

The Kubernetes domain provides Lula access to data collection of Kubernetes resources.

[!Important] This domain supports both read and write operations on the Kubernetes cluster in the current context, so use with care

Specification

The Kubernetes specification can be used to return Kubernetes resources as JSON data to the providers. The specification allows for both manifest and resource field data to be retrieved.

The following sample kubernetes-spec yields a list of all pods in the validation-test namespace.

domain:
  type: kubernetes
  kubernetes-spec:
    resources:                          # Optional - Group of resources to read from Kubernetes
    - name: podsvt                      # Required - Identifier to the list or set read by the policy
      resource-rule:                    # Required - Resource selection criteria, at least one resource rule is required
        name:                           # Optional - Used to retrieve a specific resource in a single namespace
        group:                          # Optional - empty or "" for core group
        version: v1                     # Required - Version of resource
        resource: pods                  # Required - Resource type (API-recognized type, not Kind)
        namespaces: [validation-test]   # Optional - Namespaces to validate the above resources in. Empty or "" for all namespace or non-namespaced resources
        field:                          # Optional - Field to grab in a resource if it is in an unusable type, e.g., string json data. Must specify named resource to use.
          jsonpath:                     # Required - Jsonpath specifier of where to find the field from the top level object
          type:                         # Optional - Accepts "json" or "yaml". Default is "json".
          base64:                       # Optional - Boolean whether field is base64 encoded

Lula supports eventual-consistency through use of an optional wait field in the kubernetes-spec. This parameter supports waiting for a specified resource to be Ready in the cluster. This may be particularly useful if evaluating the status of a selected resource or evaluating the children of a specified resource.

domain:
  type: kubernetes
  kubernetes-spec:
    wait:                               # Optional - Group of resources to read from Kubernetes
      group:                            # Optional - Empty or "" for core group
      version: v1                       # Required - Version of resource
      resource: pods                    # Required - Resource type (API-recognized type, not Kind)Required - Resource type (API-recognized type, not Kind)
      name: test-pod-wait               # Required - Name of the resource to wait for
      namespace: validation-test        # Optional - For namespaced resources
      timeout: 30s                      # Optional - Defaults to 30s
    resources:
    - name: podsvt
      resource-rule:
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]

[!Tip] Both resources and wait use the Group, Version, Resource constructs to identify the resource to be evaluated. To identify those using kubectl, executing kubectl explain <resource/kind/short name> will provide the Group and Version, the resource field is the API-recognized type and can be confirmed by consulting the list provided by kubectl api-resources.

Resource Creation

The Kubernetes domain also supports creating, reading, and destroying test resources in the cluster. This feature should be used with caution since it’s writing to the cluster and ideally should be implemented on separate namespaces to make any erroneous outcomes easier to mitigate.

domain:
  type: kubernetes
  kubernetes-spec:
    create-resources:                   # Optional - Group of resources to be created/read/destroyed in Kubernetes
      - name: testPod                   # Required - Identifier to be read by the policy
        namespace: validation-test      # Optional - Namespace to be created if applicable (no need to specify if ns exists OR resource is non-namespaced)
        manifest: |                     # Optional - Manifest string for resource(s) to create; Only optional if file is not specified
          <some manifest(s)>
        file: '<some url>'              # Optional - File name where resource(s) to create are stored; Only optional if manifest is not specified. Currently does not support relative paths.

In addition to simply creating and reading individual resources, you can create a resource, wait for it to be ready, then read the possible children resources that should be created. For example the following kubernetes-spec will create a deployment, wait for it to be ready, and then read the pods that should be children of that deployment:

domain: 
  type: kubernetes
  kubernetes-spec:
    create-resources:
    - name: testDeploy
      manifest: |
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          name: test-deployment
          namespace: validation-test
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: test-app
          template:
            metadata:
              labels:
                app: test-app
            spec:
              containers:
              - name: test-container
                image: nginx
    wait:
      group: apps
      version: v1
      resource: deployments
      name: test-deployment
      namespace: validation-test
    resources:
    - name: validationTestPods
      resource-rule:
        version: v1
        resource: pods
        namespaces: [validation-test]

[!NOTE] The create-resources is evaluated prior to the wait, and wait is evaluated prior to the resources.

Lists vs Named Resource

When Lula retrieves all targeted resources (bounded by namespace when applicable), the payload is a list of resources. When a resource Name is specified - the payload will be a single object.

Example

Let’s get all pods in the validation-test namespace and evaluate them with the OPA provider:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: podsvt
      resource-rule:
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]
provider: 
  type: opa
  opa-spec:
    rego: |
      package validate

      import future.keywords.every

      validate {
        every pod in input.podsvt {
          podLabel := pod.metadata.labels.foo
          podLabel == "bar"
        }
      }

[!IMPORTANT] Note how the rego evaluates a list of items that can be iterated over. The podsvt field is the name of the field in the kubernetes-spec.resources that contains the list of items.

Now let’s retrieve a single pod from the validation-test namespace:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: podvt
      resource-rule:
        name: test-pod-label
        group:
        version: v1
        resource: pods
        namespaces: [validation-test]
provider: 
  type: opa
  opa-spec:  
    rego: |
      package validate

      validate {
        podLabel := input.podvt.metadata.labels.foo
        podLabel == "bar"
      }

[!IMPORTANT] Note how the rego now evaluates a single object called podvt. This is the name of the resource that is being validated.

We can also retrieve a single cluster-scoped resource as follows, where the rego evaluates a single object called namespaceVt.

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: namespaceVt
      resource-rule:
        name: validation-test
        version: v1
        resource: namespaces
provider: 
  type: opa
  opa-spec:  
    rego: |
      package validate

      validate {
        input.namespaceVt.metadata.name == "validation-test"
      }

Extracting Resource Field Data

Many of the tool-specific configuration data is stored as json or yaml text inside configmaps and secrets. Some valuable data may also be stored in json or yaml strings in other resource locations, such as annotations. The field parameter of the resource-rule allows this data to be extracted and used by the Rego.

Here’s an example of extracting config.yaml from a test configmap:

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: configdata
      resource-rule:
        name: test-configmap
        group:
        version: v1
        resource: configmaps
        namespaces: [validation-test]
        field:
          jsonpath: .data.my-config.yaml
          type: yaml
provider: 
  type: opa
  opa-spec:
    rego: |
      package validate

      validate {
        configdata.configuration.foo == "bar"
      }

Where the raw ConfigMap data would look as follows:

configuration:
  foo: bar
  anotherValue:
    subValue: ba

This field also supports grabbing secret data using the optional base64 field as follows

domain: 
  type: kubernetes
  kubernetes-spec:
    resources:
    - name: secretdata
      resource-rule:
        name: test-secret
        group:
        version: v1
        resource: secrets
        namespaces: [validation-test]
        field: 
          jsonpath: .data.secret-value.json
          type: json
          base64: true

Evidence Collection

The use of lula dev get-resources and lula validate --save-resources will produce evidence in the form of json files. These files provide point-in-time evidence for auditing and review purposes.

The Kubernetes domain requires connectivity to a cluster in order to perform data collection. The inability to connect to a cluster during the evaluation of a validation with --save-resources will result in an empty payload in the associated observation evidence file.

Evidence collection occurs for each resource specified and - in association with any error will produce an empty representation of the target resource(s) to be collected.