Testing

Charms should have tests to verify that they are functioning correctly. This page describes the types of testing that you should consider.

See also: How to set up continuous integration for a charm

Unit testing

Unit tests isolate and validate individual code units (functions, methods, and so on) by mocking Juju APIs and workloads without external interactions. Unit tests are intended to be isolated and fast to complete. These are the tests you would run before committing any code changes.

Every unit test involves a mocked event context, as charms only execute in response to events. A charm doesn’t do anything unless it’s being run, and it is only run when an event occurs. So there is always an event context to be mocked, and the starting point of a unit test is typically an event.

A charm acts like a function, taking event context (always present), configuration, relation data, and stored state as inputs. It then performs operations affecting its workload or other charms:

  • System operations such as writing files.

  • Cloud operations such as launching virtual machines.

  • Workload operations, using Pebble in the case of a Kubernetes charm.

  • Juju operations such as sharing data with related charms.

Unit tests focus on mapping these inputs to expected outputs. For example, a unit test could verify a system call, the contents of a file, or the contents of a relation databag.

See also: How to write unit tests for a charm

Coverage

Unit testing a charm should cover at least:

  • How relation data is modified as a result of an event.

  • What pebble services are running as a result of an event.

  • Which configuration files are written and their contents, as a result of an event.

Tools

The ops.testing framework provides State, which mocks inputs and outputs. The framework also provides Context and Container, which offer mock filesystems. Tests involve:

  1. Setting up the charm, metadata, context, output mocks, and Juju state.

  2. Simulating events using Context.run. For example, config_changed, relation_changed, storage_attached, pebble_ready, and so on.

  3. Retrieving and asserting the output.

Context and State are instantiated before the charm. This enables you to prepare the state of config, relations, and storage before simulating an event.

Examples

Integration testing

Integration tests verify the interaction of multiple software components. In the context of a charm, they ensure the charm functions correctly when deployed in a test model in a real controller, checking for “blocked” or “error” states during typical operations. The goal of integration testing is to ensure the charm’s operational logic performs as expected under diverse conditions.

Integration tests should be focused on a single charm. Sometimes an integration test requires multiple charms to be deployed for adequate testing, but ideally integration tests should not become end-to-end tests.

Integration tests typically take significantly longer to run than unit tests.

See also: How to write integration tests for a charm

Coverage

  • Packing and deploying the charm

  • Charm actions

  • Charm relations

  • Charm configuration

  • That the workload is up and running, and responsive

  • Upgrade sequence

    • Regression test: upgrade stable/candidate/beta/edge from charmhub with the locally-built charm.

Caution

When writing an integration test, it is not sufficient to simply check that Juju reports that running the action was successful; rather, additional checks need to be executed to ensure that whatever the action was intended to achieve worked.

Tools

Integration tests and unit tests should run using the minor version of Python that is shipped with the OS specified in charmcraft.yaml (the base.run-on key). For example, if Ubuntu 22.04 is specified in charmcraft.yaml, you can use the following tox configuration:

[testenv]
basepython = python3.10

Examples