Double-sided Contract Tests

on Testing, Contracts

As systems get more complex, one way to simplify them is to break the problem down into smaller pieces. A common approach is to use micro-services. Rather than having one or two larger components that expose a large number of endpoints, you would have many smaller services that expose a smaller number of endpoints. There are a number of advantages to this approach, such as the ability to scale components independently, and being able to quickly react to problems in the components.

The downside to this approach is the large number of integrations between the services. The web of interconnections between these services may be hard to manage, and it may be difficult to know when a breaking change has occurred. Enter Consumer-driven Contracts. The idea is that the consumer of a service “tells” the producer of the service what of the service it is using, via providing a test. The producer can then run that test as part of the service’s test suite.

This is best described with an example. Suppose I have a service that publishes information about a person, as JSON. The schema for that service may look something like this:

{
  "id": 405,
  "first_name": "John",
  "last_name": "Doe",
  "address": {
    "street": "Main Street",
    "city": "Mainville",
    "state": "NY",
    "zip": "10001"
  }
  "birthdate": "1/1/1980",
  "gender": "M",
  "height": 72,
  "weight": 180,
  "eye_color": "Blue"
}

Then I have a client “ZipCodeClient” that consumes this service, but really only cares about the name and zipcode of the user. The client might consume the JSON using this bit of java:

ClientUser clientUser = new ClientUser(
                   json.get("first_name").getAsString(),
                   json.get("last_name").getAsString(),
                   json.get("address").get("zip").getAsString());

Therefore, the “contract” between ZipCodeClient and PersonService is these three fields. A Consumer Contract Test could then be:

public class PersonServiceTest
    @Test
    public void shouldProvideFieldsNeededByZipCodeClient(){
        String response = makeRequest();  //Actually call the service

        JsonParser parser = new JsonParser();
        JsonObject jsonObject = (JsonObject)parser.parse(response);

        assertThat(jsonObject.get("first_name")).isInstanceOf(JsonElement.class);
        assertThat(jsonObject.get("last_name")).isInstanceOf(JsonElement.class);
        assertThat(jsonObject.get("address")).isInstanceOf(JsonElement.class);
        assertThat(jsonObject.get("address").get("zip").isInstanceOf(JsonElement.class);
    }
}

This test would then run as an integration test against the PersonService. If at any time, the PersonService changes its interface, by, for example, renaming first_name to firstName, this test would break, and an early warning would be given that either PersonService can’t make that change, or the ZipCodeClient should make a compensating change.

One-way Only?

This contract, however, is only one-way so far. It specifies, “I, the Producer of the PersonService, will always provide at least the first_name, last_name, and zip fields”. What’s the contract from the consumer’s perspective? I believe it’s the following: “I, the Consumer of the PersonService, will consume only the first_name, last_name, and zip fields”. Contrasting the two statements shows a subtle difference:

  • The producer provides at least the specified fields
  • The consumer consumes only the specified fields

Our contract test thus far explicitly tests the producer side of the contract, but not the consumer. Why is this important? Suppose the consumer, looking at the full response from the PersonService, decides to consume the gender field. However, it does not update the consumer contract test. What could happen? The producer may make a decision to change the gender field’s format, or remove it entirely. It will run the contract tests, and they will all pass. Assuming that the contract tests are the basis for the contract, the producer finalizes the change, and deploys it. The consumer then breaks, because it is relying on the producer to produce the gender field in a certain way. (Note: Hopefully, this is caught in integration testing and not in production, but even then it is later than it should have been.)

There are a number of ways to solve this problem. Likely the simplest is via social engineering. The development team knows that when they change what they consume from a service, they must update the contract tests. This approach is preferred. However, some teams may want some systemic assurances. An approach that can be used is Double-sided Contract Tests.

The idea behind Double-sided Contract Test is that by specifying a “contract”, you are specifying two things:

  • A test to ensure the producer produces at least the fields specified in the contract.
  • A Mock that provides only the fields in the contract. The consumer will use this mock in unit tests.

What follows is a relatively naive implementation. Let’s specify our contract as the minimal JSON:

public static final String THE_CONTRACT="{first_name:\"Rick\", " +
                                          "last_name:\"Carragher\"," + 
                                            "address: { zip: \"10001\"}}";

Our producer-side contract test, then, can iterate through this JSON, and ensure that when calling the “real” service, all the keys specified in the contract are present in the “real” response. This code is a bit hairy.

    @Test
    public void shouldIncludeContractFields(){
        //Call the service
        String response = makeRequest();  

        JsonParser parser = new JsonParser();
        JsonElement responseJson = parser.parse(response);

        //The producer PRODUCES at least the elements in the contract
        assertItIsThere(responseJson,contractElements);

    }
    private void assertItIsThere(JsonElement responseJson,JsonElement contractElements){
        assertThat(responseJson.isJsonObject()).isTrue();
        JsonObject responseJsonAsObject = responseJson.getAsJsonObject();
        for(Map.Entry<String,JsonElement> element: contractElements.getAsJsonObject().entrySet()){
            //Ensure the response has the fields we're expecting.
            assertThat(responseJsonAsObject.get(element.getKey())).isInstanceOf(JsonElement.class);
            //Ensure that children of this object are also there.
            if (element.getValue().isJsonObject()){
                assertItIsThere(responseJsonAsObject.get(element.getKey()),element.getValue());
            }
        }

    }

The nice thing about this somewhat ugly code is that the “assertItIsThere” can take any arbitrary contract and a JSON response, and ensure they’re aligned.

Now, how can this contract be used on the client side? It’s actually even easier. Mock the service response with the contract, and run your tests. Any attempt to consume a field that’s NOT in the contract (or has a different format) will fail, because it’s not in the contract!

    @Test
    public void shouldGetValidResult() throws Exception {
        ApiService service = mock(ApiService.class);

        //The client CONSUMES at most first_name, last_name, zip
        when(service.getServiceResponse()).thenReturn(ApiContractTest.THE_CONTRACT);

        ApiClient client = new ApiClient(service);

        ApiClientResponse response = client.consumeEndpoint();
        assertThat(response.last_name).isEqualTo("Carragher");
        assertThat(response.first_name).isEqualTo("Rick");
        assertThat(response.zip).isEqualTo("10001");
    }

The key thing to note is that this mock uses ApiContractTest.THE_CONTRACT. This constant is used in BOTH tests (thus the term “double-sided”). ANY change to the Contract will ensure that both tests must work.

As noted, this is a fairly naive implementation. Colleague Prem quite correctly points out that this test doesn’t really validate a schema, or do semantic validations. Quite true. I’d like to spend a bit more time seeing if I can incorporate those elements into this kind of test. Have ideas on how? Leave a comment!

Happy Double-sided contracting!

Notes:

  • Colleague Giles Alexander has worked on a tool to support this called Janus.
  • Sample code used to develop this article is available at my github account.

© 2024. All rights reserved.