Automated tests play a very important role in our games. In our daily business we add new game features, improve existing ones or do refactoring based on technical debt. Every single change could lead to a bug that breaks our game. Even simple changes like updating a version of an external dependency could lead to unexpected behaviour. Our tests ensure that the games are always in a stable condition and that the quality meets our expectations.
There are several types of automated tests that we use in our games. In this article I will focus on Backend System Integration Tests. In these tests, we test the entire game backend as a whole, including a real database. Usually they consist of three stages:
- Preparation: Prepare a scenario in which we want to run the test action. This means setting up a test player with all conditions and requirements needed for the test action.
- Execution: Run the action that we want to test. In most cases we call an API endpoint here.
- Assertion: Validate that the outcome is the one that we expect.
Unlike in end-to-end tests, in which the preparation stage also uses API endpoints, the System Integration Tests directly manipulate the database. This has the advantage that it’s more flexible and leads to faster test executions. The drawback is that you have to basically mirror the business logic in your tests.
One problem we always faced was that the preparation stage could already make the tests unreadable, even in very simple test scenarios. Let’s take a city builder game as example. We want to write a test where we collect a finished production from a building that produces coins.
The preparation stage contains several steps:
- Create a new player
- Create a city for this player (imagine we can have multiple cities per player)
- Place a building into the city that can produce coins
- Create a production that is finished and can be collected
- Set the initial coin amount of the player to do proper validation
Think a moment how you would write this test. A common way to go would be to create reusable test helper classes that we can use for the preparation steps. In the best case we have one line per step. It could look like this:
But based on our experience it’s not always that easy. At some point you need to be more flexible. Either you end up with more parameters in a helper method or you have to create more and more helper methods to keep the test readable. It can happen really fast that the preparation code is bigger than the rest of the test. Looking at this test in a month, you need some moments to understand what happens.
Making the test setup readable
I would like to immediately understand what a test is doing, even when I come back to it after a year. To accomplish that, we introduced a builder-like API that allows us to define the whole test scenario by just configuring a scenario object:
As you can see, the test setup is much easier to read. The Indentation makes it clear which configuration belongs to which entity and which entity belongs to which parent entity. The builder-like approach along with autocompletion makes it also fun to write such tests. We don’t have to think about test helper classes that need to be injected and if there’s a method that provides the functionality I need. We simply have to configure the scenario object by using the provided with-methods. The actual creation of entities happens in the background in the
But where do we get the generated IDs from? We created a building entity in the background, but we need to know the generated ID for the “collect production” action. For this case we can make use of reference objects:
For the reference, we can use any simple reference holder. Java already comes with an AtomicReference class that provides the functionality we need, so we’re using this one for convenience reasons. We just create a reference object before building the scenario (of course this object is empty at that point). When
buildScenario() creates the entities, the reference objects are also filled with the created entities. That means after
buildScenario() was called, we can access the entities via
That’s nice! But the full power of reference objects will be clear in the next example. Let’s say we have workers that we can send to buildings in order to boost it. How can we assign a building to a worker when the building is not yet created? Just use the reference object:
You can see that we used the same reference object in two places. In one place it will be filled with the entity and in another place we use the created entity. Of course you have to make sure that the building is created first before creating the workers. But in the test itself we don’t have to care about it.
There are two main components. You already saw the scenario object in action. This and all its children are called Configuration classes (all of them having a “Given” prefix). In addition we have the Setup classes (With a “Setup” suffix) that take care of the actual entity creation.
There should be a Configuration class per domain logic or entity that can exist as a test requirement. The root class is the GivenScenario class. An instance of this class, the scenario object, is created and can be configured by calling the
buildScenario() method which you saw in the examples earlier.
In our case the GivenScenario object contains a list of GivenPlayer objects, a GivenPlayer contains the player resources and cities, and so on. This should reflect your business logic. So, if a building belongs to a city, the GivenCity object should contain GivenBuilding objects.
A Configuration class typically …
- contains “with”-methods in order to configure it. A with-method is a kind of setter method, but should
- always return its own instance in order to provide a fluent API
- accept a Consumer parameter in order to configure the child object
- contains getter methods for all configurations which should be used by the Setup classes
- contains a reference to the Entity that is created by the Setup class. We abstracted this part to a GivenEntity class that you can find in the following example.
A Setup class is responsible for building a specific part of the test scenario. In most cases there is a Setup class per domain logic or entity.
For example the CitySetup class is responsible for creating all cities defined in the scenario. The BuildingSetup class creates the buildings and also takes care of starting its productions.
Here’s an example of the CitySetup class:
setUp() method receives the complete scenario object, it should just pick the information it needs to create the entities. After an entity was created, it should be passed to the configuration object (e.g. givenCity.setEntity(city)) which then fills the reference object to make it accessible in following Setup classes and tests.
As some Setup classes are dependent on others, the order is important. For instance the CitySetup class accesses the Player Entity which was created by the PlayerSetup before. The order can be ensured by an Order-Annotation on each Setup class, as you can see in the example above. We also created a ScenarioSetupPartOrder class to easily maintain the order. The Order-Annotation is part of the Spring Framework. If you inject a list of dependencies (e.g.
List<ScenarioSetupPart>) it makes sure that this list is sorted based on the defined order.
Putting it together
Now that we have the Configuration and Setup classes defined, we can look at the
buildScenario() method that should be placed in the base test class:
It creates an object of GivenScenario each time it is called. The Consumer parameter allows the caller to configure the object. Afterwards the
scenarioSetup.setUp() method is called which just delegates to the specific Setup classes in the correct order:
That’s it! When we create a new feature, we just have to create a Configuration and the related Setup class. Afterwards it can be used in any test.
This test setup architecture improved the readability of our tests a lot. Writing new tests or extending scenarios is very easy and the general maintenance effort is lower. If we’re doing a small refactoring, it might be enough to just update the Setup class and leave the actual tests untouched.
However, one downside is that we always have to keep some things in mind when we create new Configuration and Setup classes. For example setting the reference entities or having the correct order of Setup classes. Because the setup happens in the background, the reason for a test failure may not always be obvious.
But if you think further, this system will open up new paths for us. The preparation step is just a matter of configuring an object now. The configuration does not only have to take place in the test. We also created an API endpoint for our Frontend automation tests and in-game cheats. They can now make use of the same system to set up a player without any additional effort. But this is something for the next blog post