Testing GraphQL Subscriptions

In case you haven't heard, the Speckle Server is going through a complete overhaul for 2.0 -- a big part of which is moving the API over to GraphQL! The latest thing I've been working on is implementing subscriptions which will allow for real time updates for UIs and notifications. After a few days of happily tip-tapping away in VS code, a troubling thought started to dawn on me: I have no idea how I'm going to test these 😨

A lot of furious Googling and cryptic error messages later (plus a tonne of help from my pal Dimitrie), we cracked the code! In the hopes of saving you from a similar struggle, we're documenting the process for you here. Let's get cracking ✨

Setting Up the Plumbing

There were several problems we found in setting up the plumbing. It's all a combination of how our application is architected, port availability, etc. Websockets are quite painful to test. Note: we're using Mocha and Chai here for our tests, but you could of course adapt this to work with whatever test framework you are using.

Overall, our subscription test file looks like this:

describe( 'GraphQL API Subscriptions', ( ) => {
before( async function() {
// Server startup
} )

after( async function() {
// Server shutdown
} )

// Actual tests follow
it( 'Should test a subscription', async ( ) => {
// ...
} )

})


Starting the server

The before() hook is used to start our HTTP GraphQL server. By trial and error, we've found that the best way is to literally spawn it into a separate process and make sure we're giving things enough time to boot! Here's how that looks:

before( async function ( ) {
// Ensures that this function doesn't timeout too fast. Some testing environments can be a bit slow.
this.timeout( 10000 )

// Then... start the server (& check if running on win)
const childProcess = require( 'child_process' )
serverProcess = childProcess.spawn( /^win/.test( process.platform ) ? 'npm.cmd' : 'npm', [ "run", "dev:server:test" ], { cwd: `${appRoot}` } )

// Now we need to wait a bit to make sure the server properly started.
await sleep( 5000 )
} )

after( async ( ) => {
// Tidy up
serverProcess.kill( )
} )


Here's how the sleep function looks like. Nothing special; you can add it at the end of your test file.

function sleep( ms ) {
return new Promise( ( resolve ) => {
setTimeout( resolve, ms )
} )
}


Setting up a subscription client

Since subscriptions require a persistent connection to the server, we'll be setting up a WebSocket client to handle this. We'll also need to create a subscription observable which will send our subscription query and listen for events. The beginning of this setup is totally borrowed from this boilerplate, so check out that repo if you'd like!

The first setup function we'll need is for creating a WebSocket transport for handling subscriptions. This is done using SubscriptionClient from subscription-transport-ws which you can read more about here. Our function getWsClient takes your WebSocket url and an authorisation token as arguments and returns the client.

const { SubscriptionClient } = require('subscriptions-transport-ws');
const ws = require('ws');

const getWsClient = ( wsurl, authToken ) => {
const client = new SubscriptionClient( wsurl, {
reconnect: true,
connectionParams: {
headers: { Authorization: authToken },
// any other params you need
}
}, ws )
return client
}


The next function we'll need is for creating a subscription observable through which to send your subscription queries and listen for a response. This is achieved using apollo-link which creates the WebSocket connection to the client you returned in the getWsClient function. The documentation for this can be found here.

const { execute } = require( 'apollo-link' )
const { WebSocketLink } = require( 'apollo-link-ws' )

const createSubscriptionObservable = ( wsurl, authToken, query, variables ) => {
const link = new WebSocketLink( getWsClient( wsurl, authToken ) )
return execute( link, { query: query, variables: variables } )
}


In our case, all of this plumbing gets chucked into describe(). You can put it outside as well, but mind your scopes and variables. Here's how everything should look now:

// imports for the testing
const chai = require( 'chai' )
const chaiHttp = require( 'chai-http' )
// imports for the subscription connection
const { execute } = require( 'apollo-link' )
const { WebSocketLink } = require( 'apollo-link-ws' )
const { SubscriptionClient } = require( 'subscriptions-transport-ws' )
const ws = require( 'ws' )

const expect = chai.expect
chai.use( chaiHttp )

// ...whatever else you need

describe( 'GraphQL API Subscriptions', ( ) => {
let userA = { '...' } // set up some test users
let userB = { '...' }
let userC = { '...' }
let serverProcess // instantiate the serverProcess which we'll use later

const getWsClient = ( wsurl, authToken ) => {
const client = new SubscriptionClient( wsurl, {
reconnect: true,
connectionParams: {
headers: { Authorization: authToken },
// any other params you need
}
},
ws )
return client
}

const createSubscriptionObservable = ( wsurl, authToken, query, variables ) => {
const link = new WebSocketLink( getWsClient( wsurl, authToken ) )
return execute( link, { query: query, variables: variables } )
}

// ...
} )


Writing the Tests

Anatomy of Our Subscription

Before we get into it, let's quickly go over the subscription we'll be testing. In Speckle, user data is saved into streams. The idea is similar to a git repository that can have multiple branches each with its own history of commits. We have implemented subscriptions for created, updated, and deleted events for streams, branches, and commits. To take the simplest example, let's test Speckle's userStreamCreated subscription.

The userStreamCreated subscription is pinged by the streamCreate mutation whenever the subscribing user creates a stream. The subscription query is super simple as it gets the user ID in question from the authorisation token and thus doesn't require any arguments:

subscription {
userStreamCreated
}


The mutation that triggers it looks like this:

mutation {
streamCreate(stream: {
name: "A Cool Stream 🌊",
description: "isn't this nifty?",
isPublic: false
})
}


The data received by the subscription would look like this:

{
"data": {
"streamCreate": "6753b35369" // the generated stream id
}
}


Testing the Subscription

Our test for the userStreamCreated subscription needs to do four things:

1. Set up the subscription observable using the WebSocket link
2. Subscribe to the event and confirm we're receiving the expected data
3. Use the streamCreate mutation to create some streams that our subscriber should be notified of
4. Confirm that the correct data was received by the subscriber

Let's take a look at the code to see how we're doing this:

describe( 'Streams', ( ) => {
it( 'Should be notified when a stream is created', async ( ) => {

// 1. SET UP
let eventNum = 0 // initialise a count of the events received by the subscription
const query = gql `subscription { userStreamCreated }` // the subscription query
const client = createSubscriptionObservable( wsAddr, userA.token, query )

// 2. SUBSCRIBE
const consumer = client.subscribe( eventData => {
// whatever you need to do with the received data
// for us, that's just checking that data is received and incrementing `eventNum`
expect( eventData.data.userStreamCreated ).to.exist
eventNum++
} )

// 3. SEND MUTATIONS: CREATE TWO STREAMS FOR USERA (and one for userB)
await sleep( 500 ) // we need to wait a hot second here for the sub to connect

let sc1 = await sendRequest( userA.token, { // will receive eventData in subscription
query: `mutation {
streamCreate(stream: { name: "Subs Test (u A) Private", description: "Hello World", isPublic:false } )
}`} )
expect( sc1.body.errors ).to.not.exist // just making sure the mutations are executing

let sc2 = await sendRequest( userA.token, { // will receive eventData in subscription
query: `mutation {
streamCreate(stream: { name: "Subs Test (u A) Private", description: "Hello World", isPublic:false } )
}`} )
expect( sc2.body.errors ).to.not.exist

let sc3 = await sendRequest( userB.token, { // will *not* receive eventData in subscription
query: `mutation {
streamCreate(stream: { name: "Subs Test (u B) Private", description: "Hello World", isPublic:false } )
}`} )
expect( sc3.body.errors ).to.not.exist

await sleep( 1000 ) // and also wait a sec here for the mutations to go through and for the sub to receive data

// 4. CONFIRM WE RECEIVED BOTH STREAM CREATE EVENTS
expect( eventNum ).to.equal( 2 ) // make sure we were pinged about both streams
consumer.unsubscribe( )
} )
} )


As you can see, we're using createSubscriptionObservable with our WebSocket address, userA's authorisation token, and the subscription query to create the observable WebSocket connection. We're then using .subscribe() to listen for the event data received by the subscriber. We then use the streamCreate mutation to create three streams, two of which belong to userA and should notify the subscriber. Finally, we confirm that the subscriber received two streamCreated events before unsubscribing.

You might have noticed the sendRequest() function for executing the streamCreated mutations. This is just a convenience wrapper of chai.request() which takes the auth token, the mutation/query, and the http address as arguments.

function sendRequest( auth, obj, address = addr ) {
return chai.request( address ).post( '/graphql' ).set( 'Authorization', auth ).send( obj )
}


We now have a full working test for the userStreamCreated subscription 🥳

Conclusions

Testing isn't always the most exciting thing on your to-do list, but hopefully this post will help make it a bit easier. You can see all of Speckle's subscription tests here (and maybe drop us a 🌟 if it was helpful 😉). If you're up for some unit testing adventures in Revit, we've got you covered there as well!

Now get out there and test your code! 🛠️