Introducing Eventide Fixtures: Testing So Easy It Feels Like Cheating

The Eventide Project is thrilled to announce the release of a comprehensive set of test fixtures covering all major aspects of testing Eventide solutions.

Eventide fixtures are a massive leap forward in testing and testability for pub/sub, event-sourced, service-oriented, and microservice systems. They provide an enormous economy of effort, and they lower the barrier for one of the most daunting aspects of building systems with evented, autonomous services patterns.

As one user said recently: Testing with these fixtures is so easy, it feels like cheating.

What’s a Fixture?

A fixture is a test abstraction that greatly reduces the effort required to write tests, and provides guidance and standards to the developers who use them.

An Eventide fixture allows a part of a system to be tested in a purpose-built software jig that simulates live, operational scenarios, but without causing any permanent side effects.

The fixtures provide a set of specialized assertions that dramatically improve the ease of verifying the operation of all aspects of Eventide systems, as well as exacting automated controls that execute the most common test scenarios via a dramatically-simplified API.

The set of fixtures includes:

Scenarios Supported

With the support of the test fixtures, a developer building Eventide services can:

Simulate the operation of systems and services in isolation without needing to use the message store at all
Handle an input message and test the resulting messages, including the messages’ attribute values and metadata attribute values
Control a handler’s entity projection so that event streams don’t have to be populated in order to return a specific entity to a handler
Simulate idempotence and concurrency scenarios to easily and thoroughly test handlers for fussy distributed systems conditions
Verify input message preconditions to fail fast when the inputs aren’t exactly right
Ensure that messages are written to the correct streams
Prove that the correct optimistic concurrency sequence number is used when a message is written to its stream
Ensure that no unintended messages were written by a handler due to some logic bug
Control the system clock and UUID generators to make their returned values entirely deterministic
Compare two different data objects with each other, including message objects and entity objects
Verify that event attribute values are correctly projected onto entities, including when using attribute name mappings to accommodate schemas that don’t use the same attribute names
Verify that message attribute values are correctly copied from one message to another, including when using attribute name mappings
Assert that all of an output message’s attributes have been muted as expected before writing the message
Assert that messages that follow each other in the sequences of messaging workflows have had their workflow metadata attributes set correctly
…amongst numerous other scenarios

An Example

While the documentation provides detailed examples for each kind of fixture, here’s an overview of testing the handling of a message using the handler fixture:

context "Handle SomeMessage" do
  processed_time = Time.now

  some_message = SomeMessage.build(some_data, some_metadata)

  some_entity = SomeEntity.build(some_entity_data)
  entity_version = 1111

  event_class = SomeEvent
  output_stream_name = "some_entity-#{some_id}"

  handler = SomeHandler.new

  fixture(
    Messsaging::Fixtures::Handler,
    handler,
    some_message,
    some_entity,
    entity_version,
    clock_time: processed_time
  ) do |handler|

    handler.assert_input_message do |some_message|
      some_message.assert_all_attributes_assigned

      some_message.assert_metadata do |metadata|
        metadata.assert_source_attributes_assigned
      end
    end

    event = handler.assert_write(event_class) do |write|
      write.assert_stream_name(output_stream_name)
      write.assert_expected_version(entity_version)
    end

    handler.assert_written_message(event) do |written_message|
      written_message.assert_attributes_copied([
        :some_id,
        { :amount => :quantity },
        :time,
      ])

      written_message.assert_all_attributes_assigned

      written_message.assert_attribute_value(:processed_time,
        Clock.iso8601(processed_time))

      written_message.assert_follows

      written_message.assert_metadata do |metadata|
        metadata.assert_workflow_attributes
      end
    end

    handler.refute_write(SomeOtherEvent)
  end
end

Output

The output from the fixture use above would be:

Handler: SomeHandler
  Input Message: SomeMessage
    Attributes Assigned
      something_id
      amount
      time
    Metadata
      Source Attributes Assigned
        stream_name
        position
        global_position
  Write: SomeEvent
    Written
    Stream name
    Expected version
  Written Message: SomeEvent
    Follows
    Attributes Copied: SomeMessage => SomeEvent
      something_id
      amount => quantity
      time
    Attribute Value
      processed_time
    Attributes Assigned
      something_id
      quantity
      time
      processed_time
    Metadata
      Workflow Attributes Assigned
        causation_message_stream_name
        causation_message_position
        causation_message_global_position
        correlation_stream_name
        reply_stream_name

In addition, fixtures can optionally output more far more detailed data, including the values of all data attributes and the detailed findings of all assertions. When a fixture has an error, all detailed extra data is printed out by default to provide more context for troubleshooting and debugging.

More Detailed Examples

The account component example that ships with the Eventide project has been updated to include versions of some of its tests that have been re-written using fixtures.

While the two different styles of implementations can be compared side-by-side, it should be noted the each is not implemented with intention of acting as a test style comparison, and there are differences that exist that are not due to either the use or the absence of fixtures. Nonetheless, a side-by-side comparison of the two different styles of implementation will indeed illustrate the power of the fixtures.

Open Command Handler Tests

Opened Event Projection Tests

Built Using TestBench

Eventide fixtures are built using the fixtures capability of the TestBench testing framework for Ruby.

Fixtures are a unique feature of TestBench and are not available in any other testing framework. In order to take advantage of the fixtures, the use of TestBench as a test framework is required.

While TestBench is the most commonly-used Ruby test framework chosen by Eventide developers, other test frameworks can still certainly be used, but the fixtures cannot.

A Long Time Coming

Test fixtures are an important feature that the team gave as much thought to as possible, and resisted every temptation to leap into prematurely. The team has been sketching and prototyping various ideas for fixtures for the past four years, and different bespoke implementations have been crafted and dissected along the way by users on their own projects.

Fixtures Blackboard Sketch 2020-02

The full-time, focused development push to ship “official” Eventide Project fixtures finally started on Monday, July 6th, and proceeded for 40 straight days, including weekends and some nights. The effort entailed adding new features to core Eventide libraries, and adding three new libraries to the stack. And not to be outdone, it required an almost complete re-write of TestBench. Indeed, not only did the delivery of Eventide’s fixtures require designing and building the fixtures themselves, but it also required building a new test framework from scratch.

The announcement of the test fixtures release is a significant step for the Eventide Project, but we also hope it provides an opportunity for frameworks and framework developers to reconsider where we collectively set the bar for testing, testability, and the design principles and practices that enable them and that enable the community and culture that are empowered by them. Frameworks have an important role to play in setting that higher bar, and in setting expectations higher than falling back on brute force tactics like ad hoc mock objects. We’re gratified if the Eventide Project has played even a small part in such a movement.

The release also marks one of the signficant milestones in the Eventide v2 timeline. Our focus in the coming development horizon returns to features that support using Eventide side-by-side with ActiveRecord inside Rails apps. With more users exploring this scenario now, we have rich examples of solutions in the wild to drawn upon, extract, and standardize.

We’re pleased and gratified that we’re finally at launch day for Eventide’s test fixtures. As gratifying as it is to celebrate the release of something we’re been imagining for years, we’re much more excited to bring these tools to bear in our own projects. The goal, after all, is to improve the developer experience and the efficiency of building evented services.

With that, we wish you smooth sailing ahead in all your evented services, event sourcing, and pub/sub endeavors. As always, reach out on the Eventide Slack with any questions or comments, or just to say hi.

Happy testing!