Skip to main content

Matching test data

By default, the data used in an interaction is strictly matched - if the contract says the userId should be "foo", then it must be "foo" during the verification too.

However, it's not always convenient to have the exact same test data between the consumer test and the provider verification. Sometimes, there are even technical constraints preventing the test data from being known ahead of time (for example, where user IDs are autogenerated on insertion into a database).

Test Equivalence Matchers

To address this, we use Test Equivalence Matchers. These tell the framework "this test is equivalent to any other test where the data passes this matcher".

In the examples in the earlier sections, we used john smith as the example username. During provider verification, it might be that the test user was set up to be called steve jenkins. This would still be a valid username - we don't want to fail the test just because the specific test data happens to be different. However, we do want to check that the name still conforms to any expectations that the consumer had. For example, the consumer might expect that the user name was a valid, non-empty string.

We can express the non-empty string state with the anyString(example?: string) Test Equivalence Matcher:

body: {
    userId: 'foo',
    // You can read this as
    //   "this test covers any name property that is a string (eg 'john smith').
    name: anyString('john smith'),
},

This means that the test will pass if any non-empty string is returned as the name. The test would fail if a complex object or other data type was returned there.

The example passed in to the matcher here is used as the specific data during contract definition - when the provider returns this object, it will reify it into:

{
  userId: 'foo',
  name: 'john smith',
}

Do I need to use matchers?​

Test-equivalence matchers are just a mechanism for making it convenient to do contract tests in an environment where it's inconvenient to keep the test data identical.

If it's convenient for you to keep the test data in sync between the provider and the consumer, there's no need to use test-equivalence matchers at all. However, our experience is that most teams find test-equivalence matchers very convenient.

Parametrising the test with state variables​

Sometimes, there are constraints on the provider team that are hard to know from the consumer side. For example, perhaps their test data is constrained in some way, and so it is hard to set up a user with the ID 'foo'. To allow them to specify some of the state, state variables can be used. Instead of:

              inState('A user with id "foo" exists'),

you can provide a state variable (and a default value) by providing an object keyed by the variable name:

              inState('A user exists', { userId: 'foo' }),

This can later be accessed using the stateVariable(name: string) Test Equivalence Matcher. You can also access the variables using the config object passed to the trigger and testResponse functions:

      await contract.runInteraction(
        {
          states: [
            inState('Server is up'),
            // Here we tell ContractCase that there's
            // a userId variable returned by this state.
            // We give the example "foo", to be used
            // during the contract definition.
            //
            // During contract verification, the value of userId
            // value will be provided by the state setup function
            // in the provider test.
            inState('A user exists', { userId: 'foo' }),
          ],
          definition: willSendHttpRequest({
            request: {
              method: 'GET',
              path: '/users',
              // This matcher tells ContractCase that
              // the id in the query will be whatever
              // the userId is
              query: { id: stateVariable('userId') },
            },
            response: {
              status: 200,
              body: {
                // In the response body, we expect the
                // userId to be present
                userId: stateVariable('userId'),
                // and the name may be any non-empty string
                name: anyString('John Smith'),
              },
            },
          }),
        },
        {
          trigger: (interactionSetup: HttpRequestConfig) =>
            new YourApi(interactionSetup.mock.baseUrl).getUser(
              interactionSetup.getStateVariable('userId'),
            ),

          testResponse: (user, interactionSetup: HttpRequestConfig) => {
            expect(user).toEqual({
              userId: interactionSetup.getStateVariable('userId'),
              name: 'John Smith',
            });
          },
        },
      );

The default value passed to the state variable can be a Test Equivalence Matcher, so you could add additional constraints:

inState('a user has a score that is an integer', { userScore: anyInteger(10) });

This would prevent the state handler from returning any userScore value that doesn't pass the anyInteger constraint.

Note: Remember that the goal here is not to describe the API, it's to describe the constraints of the service defining the contract. So, if the userScore field is an integer, but the client would actually be happy with any number, you don't need to constrain it.

As a convenience, all default values passed to state variables are automatically wrapped in a shapedLike() constraint. This means that userId: 'foo' will accept any string value, and not just the exact string 'foo'.

Anti-pattern: Describing the type system with matchers​

Sometimes, first time users try to use matchers to describe the whole type system. For example, a User payload might describe either an admin or a member, where they may have slightly different fields in the two response payloads. In this situation, these are probably separate interactions:

  • When user 123 is an admin, the get user request should return a user payload describing an admin
  • When user 123 is a member, the get user request should return a user payload describing a member

It's important to keep these tests separate, because if a service erroneously could never produce an admin payload, then a test that passed if either response was returned would give false confidence.

Be careful not to use matchers to fully realise your type system - you should use matchers only to widen the definition of responses that would also be covered by the current test.

Anti-pattern: Optional fields and empty arrays​

Similarly, a test which allows optional fields would still pass even if the field was never present. You wouldn't be sure that your provider could ever generate that missing field.

For a situation where you have an optional field that your consumer relies on, write tests for two interactions - one with the field present, and one without.

tip

To provide the best deployment confidence, a contract test needs specific examples.

Matcher overview​

In the reference section, you can find all Test Equivalence Matchers documented.