From unit testing to integration testing: setup, findings and limits

Introduction

Some of the frontend projects KNP works on are business applications that require a rich and responsive interface. The development of an SPA generally matches these needs, despite a potentially high complexity, and the cost raise it implies.

In such codebase, we commonly find the following three layers: a view layer, for the presentation logic; a layer for side effects, to isolate the interactions of the application with the outside world; a global state layer, mainly used to connect the first two layers. The latter can also possibly store data that cannot be held in the internal state of views.

This architecture is agnostic of the technologies used to implement the different layers. With a few syntactic variations, it is therefore quite simple for a team to move from such a codebase to another. The separation of responsibilities also significantly increases the application testability.

From the user point of view, a business functionality generally uses at least one component (class, function, etc.) of each layer. These components are, at the very least, unit tested. However, because these tests run in isolation, the level of confidence they provide may be insufficient. This is especially true given we also wish to guarantee that components of the different layers correctly interact with each other. As a matter of fact, these interactions are most often much more representative of how the application really works and how the end user experiences it.

This article illustrates the problem through a practical use case. It presents the generally retained solution on the most recent projects developed at KNP, along with its limitations.

Prerequisites

The basic concepts of the React / Redux / React Testing Library ecosystem are assumed to be acquired to comfortably read this article.

Use case

Here we present the simple case of a connection form, whose functional specifications are as follows: A user authenticates by entering its email and password; when authentication is in progress, the send button must be disabled; if the authentication is successful, the application is displayed; otherwise, an error message is displayed. The following model shows the form as it could be integrated:

The form is rendered by a functional React component, which uses different hooks to interact with the application store. The form is an aggregation of styled components, which are not detailed here for the sake of brevity.

const Firewall: React.FC = () => {
  const dispatch = useDispatch()
  const isAuthenticating = useSelector(selectIsAuthenticating)
  const isAuthenticated = useSelector(selectIsAuthenticated)
  const hasError = useSelector(selectHasError)
  
  if (isAuthenticated) {
    return <>{ children }</>
  }

  const onSubmit = (e: React.FormEvent<HTMLFormElement>): void => {
    e.preventDefault()

    dispatch(slice.actions.authenticateWithCredentials({
      username: e.currentTarget.username?.value || '',
      password: e.currentTarget.password?.value || '',
    }))
  }

  return <Form onSubmit={ onSubmit }>
    <Label htmlFor="username">Email :</Label>
    <Input
      type="email"
      autoComplete="username"
      id="username"
      name="username"
      icon="fa fa-user"
      required
    />
    <Label htmlFor="password">Mot de passe :</Label>
    <Input
      type="password"
      autoComplete="current-password"
      id="password"
      name="password"
      icon="fa fa-lock"
      required
    />
    <SubmitButton loading={ isAuthenticating } $backgroundColor="blue" $iconLayout="right">
      Se connecter
      <i className="fa fa-arrow-right" />
    </SubmitButton>

    { hasError &&
      <Message type={ MessageType.ERROR } content="Une erreur s'est produite..." />
    }
  </Form>
}

Different calls to the API are necessary to authenticate the user. Those belong to the outside world of the front app and are therefore isolated using a middleware. This example shows the implementation proposed by redux-saga.

export function* authenticateWithCredentials({ payload }: { payload: AuthenticatePayload }): Generator {
  try {
    const post = (yield getContext(Context.Post)) as ApiPost
    const container = (yield call(post, '/auth-tokens', payload)) as LdapToken

    const storage = (yield getContext(Context.Storage)) as Storage
    yield call([ storage, 'setItem' ], 'token', container.token)

    yield put(slice.actions.successfullyRetrievedToken())
  } catch (e) {
    error(e)
    yield put(slice.actions.error())
  }
}

export function* getAuthenticatedUser(): Generator {
  try {
    const get = (yield getContext(Context.Get)) as ApiGet
    const user = yield call(get, '/me')

    yield put(slice.actions.authenticated(user))
  } catch (e) {
    error(e)
    yield put(slice.actions.error())
  }
}

export default function* rootSaga(): Generator {
  yield takeLeading(slice.actions.authenticateWithCredentials, authenticateWithCredentials)
  yield takeEvery(slice.actions.successfullyRetrievedToken, getAuthenticatedUser)
}

Note that the authentication is not direct. A first call to the API is made with the user's authentication information (email and password) to retrieve a token. This token is then stored in the local storage of the browser. That way, we can later re-authenticate the user without asking him for credentials again. This part is not covered in the article. A second call to the API retrieves a [User] business object which contains user data: first name, last name, etc. The user is considered authenticated if and only if the sequence of these two actions occurs without error. This point is important for the rest of the article.

Finally, we use redux to handle the form’s life cycle and its different states (unauthenticated / being authenticated / authenticated / error), depending on the result returned to us by the API.

export enum AuthStatus {
  NeedsAuthentication = 'NeedsAuthentication',
  Authenticating = 'Authenticating',
  Authenticated = 'Authenticated',
  UnknownError = 'UnknownError',
}

type AuthenticationState = {
  status: AuthStatus
}

export const initialState: AuthenticationState = ({
  status: AuthStatus.NeedsAuthentication,
})

export interface AuthenticatePayload {
  username: string
  password: string
}

const slice = createSlice({
  name: 'me/authentication',
  initialState,
  reducers: {
    authenticateWithCredentials: (state, _action: PayloadAction<AuthenticatePayload>) => ({
      ...state,
      status: AuthStatus.Authenticating,
    }),
    
    successfullyRetrievedToken: state => state,
    
    authenticated: (state, _action: PayloadAction<User>) => ({
      ...state,
      status: AuthStatus.Authenticated,
    }),
    
    error: state => ({
      ...state,
      status: AuthStatus.UnknownError,
    }),
  },
})

export const selectIsAuthenticating: Selector<boolean> = state =>
  state.authentication.status === AuthStatus.Authenticating

export const selectIsAuthenticated: Selector<boolean> = state =>
  state.authentication.status === AuthStatus.Authenticated

export const selectHasError: Selector<boolean> = state =>
  state.authentication.status === AuthStatus.UnknownError

export default slice

Unitary approach

As highlighted in the introduction, the separation of responsibilities of these three components greatly facilitates their coverage by unit tests. A project whose testing strategy relies exclusively on this type of testing will therefore verify the following assertions.

For the redux part [slice.ts], there is a test file covering all reducers and selectors. This part is trivial and its implementation is immediate. For the sagas part [effects.ts], there is at least one test per execution scenario of each saga (nominal case and error case), i.e. a total of four tests if we consider the two sagas of our use case. Note that unit tests of sagas which truly cover their behavior and not their implementation can be challenging. Please refer to the excellent work of Phil Herbert (ThoughtWorks, 2018) on that topic.

For the presentation part [Firewall.tsx], we would probably find an implementation close to that presented below.

describe('features/Me/Authentication', () => {
  test('authenticates', async () => {
    const { userEvent } = setup(<Firewall>app</Firewall>)
    const submit = screen.getByRole('button', { name: 'Se connecter' })
    expect(submit).toBeEnabled()
    
    await userEvent.type(screen.getByLabelText('Email'), 'hibous.forestis@knplabs.forest')
    await userEvent.type(screen.getByLabelText('Mot de passe'), 'owl')

    await userEvent.click(submit)
    expect(submit).toBeDisabled()
    
    act(() => {
      store.dispatch(slice.actions.authenticated({
        firstname: 'Hibous',
        lastname: 'Forestis',
        company: 'KNP Labs',
        // ...
      }))
    })    
    expect(await screen.findByText(/app/)).toBeInTheDocument()
  })

  test('cannot authenticate', async () => {
    const { userEvent } = setup(<Firewall>app</Firewall>)
    const submit = screen.getByRole('button', { name: 'Se connecter' })
    expect(submit).toBeEnabled()

    await userEvent.type(screen.getByLabelText('Email'), 'hibous.forestis@knplabs.forest')
    await userEvent.type(screen.getByLabelText('Mot de passe'), 'owl')

    await userEvent.click(submit)
    expect(submit).toBeDisabled()

    act(() => {
      store.dispatch(slice.actions.error())
    })   
    expect(screen.getByText(trans('error'))).toBeInTheDocument()
  })
})

This file contains a single suite of two unit tests. The tested component is rendered by a [setup] utility, which decorates the component with providers (test store, router, etc). Note that the test store differs from the one used by the real application by not running the redux-saga middleware. That is, redux actions are not listened to by any saga in test context, and API calls will remain unanswered. This is not unusual when the testing strategy, presentation components included, solely relies on unit testing. With such a strategy indeed, we only assert on the rendering resulting from human events (click on a button, action with the keyboard, etc.) or system events (API response for example) triggered during the test.

The [setup] utility returns an [UserEvent] object, which is used to simulate a user's interactions as closely as possible to what would happen in a browser. Those are the best practices recommended by React Testing Library developers, which can be found in their official documentation online.

[authenticated] and [error] actions are system ones, emitted by sagas. As the test store does not run any middleware, those actions are manually triggered. This is the only way, in the chosen test environment, to set the tested component in the specific state it would be in after authentication passes or fails. In real-life conditions, those actions would be emitted as a result of observing the input [authenticateWithCredentials] action emitted when the form is submitted by the user. Given there are unit tests to back this behavior, we can confidently say that functional specifications of the form are covered.

Problem

Confidence provided by unit testing a component ends where interactions with other components begins. With this idea in mind, manually triggering system actions as practiced in the previous example obviously poses two major problems.

First of all, assuming that sagas have changed and are no longer tested (everything happens), the application may be down while the tests described above will still pass. In this situation, these tests produce false positives results. As an immediate corollary, if the tests are modified so that [authenticated] and [error] actions are no longer triggered, both tests fail while the application still works. In this case tests are producing false negative results.

There is a significant difference between the reality tested and the real execution of the application. Therefore, the return on investment of testing declines, along with the confidence tests delivered.

Apart from reusable interface components operating in pure isolation (form field, for example), presentation components rarely meet the conditions enabling them to be efficiently unit tested. When an interface component is coupled with a store, which can itself potentially be connected to one or more middlewares, additional efforts in setting up the test environment are necessary, for a moderate benefit.

This leads to interface components being much more efficiently covered using integration tests. Moreover, those tests make some unit tests redundant in other layers of the application, which can be greatly simplified.

Integration approach

MSW is an API mocking tool which remains agnostic of the client consuming it. Requests interception happens at the lowest level. All components involved from the moment the request is issued (for example using the JavaScript Fetch API) to the moment the response is received (for example in a saga) are therefore executed. We’ve chosen to set up MSW as described in official recommendations. First, the server must be declared as follows:

import { setupServer } from 'msw/node'
import { handlers } from 'test/mocks/handlers'

export const server = setupServer(...handlers)

This server must be running before starting any test suite. With Jest, used here as a testing framework, the implementation is as follows:

import { server } from 'test/mocks/server'

beforeAll(() => {
  server.listen({
    onUnhandledRequest(req) {
      console.log('%s request to %s unhandled by tests', req.method, req.url)
    },
  })
})

Finally, handlers are declared for each API endpoint called throughout the application's tests. We grouped these handlers in a single file. It is possible to separate them into several files, or even to declare the necessary handlers in each test file.

import { HttpResponse, http } from 'msw'
import { users } from 'test/fixtures'

export const ok = (body:any = null) => new HttpResponse(JSON.stringify(body), {
  status: 200,
  headers: { 'Content-Type': ContentType.Json },
})

export const handlers = [
  http.post('/auth-tokens', () => ok({ token: 'my-token' })),
  http.get('/me', () => ok(users[0])),
]

Once this new environment is set up, the form presentation component tests can be modified as follows:

describe('features/Me/Authentication', () => {
  test('authenticates', async () => {
    const { userEvent } = setup(<Firewall>app</Firewall>)
    const submit = screen.getByRole('button', { name: 'Se connecter' })
    expect(submit).toBeEnabled()
    
    await userEvent.type(screen.getByLabelText('Email', 'hibous.forestins@knp.forest')
    await userEvent.type(screen.getByLabelText('Mot de passe', 'owl')
    await userEvent.click(submit)

    expect(submit).toBeDisabled()
    expect(await screen.findByText(/app/)).toBeInTheDocument()
  })

  test('cannot authenticate', async () => {
    const { userEvent } = setup(<Firewall />)
    const submit = screen.getByRole('button', { name: 'Se connecter' })
    
    server.use(
      http.get('/me', async () => HttpResponse.error())
    )

    await userEvent.type(screen.getByLabelText('Email', 'hibous.forestins@knp.forest')
    await userEvent.type(screen.getByLabelText('Mot de passe', 'owl')
    
    await userEvent.click(submit)
    await waitFor(() => expect(submit).toBeDisabled())
    await waitFor(() => expect(submit).toBeEnabled())

    expect(screen.getByText(trans('error'))).toBeInTheDocument()
  })
})

With this approach, the functional specifications remain perfectly covered. However, there are important differences with the unitary approach. First of all, the component is now tested in integration with the components of other layers. It is tested the same way it is used in the real application, because there is no longer any need to decorate it with a specific test store. Still, the real application interacts with a real API, whereas here the calls are intercepted by MSW.

First consequence, direct interactions with the store disappeared from the tests. The chain of actions [authenticateWithCredentials] -> [authenticated | error] runs in real conditions and no longer appears in the test file: we removed implementation details from the test, which yielded false positives/negatives risks, to concentrate on pure functionality.

Second consequence, sagas used for authentication are also entirely covered by the integration tests of the presentation layer. A unit test could nevertheless remain justified to cover the interaction of the [authenticateWithCredentials] saga with the local storage. In most cases though, sagas no longer require to be directly tested. The same goes for selector tests: these are now redundant, because the presentation layer tests already cover all possible states the authentication status can take.

This is why, for most functionalities tested with this environment, we can significantly reduce the amount of unit testing without losing coverage. Selectors which only access a portion of the state, sagas which only interact with an API, those are good examples of verbose, redundant tests which provide little confidence. The same remark applies for reducers: aren't they already tested in integration with the presentation layer? They are, but we nevertheless choose to keep a unit test for each reducer. Recall that the global application store is potentially shared with other components: a successful test of a component using it does not guarantee other components will act the same. The same logic applies to complex, reusable selectors.

Limits

Integration tests slowly run compared to unit tests, because some layers of the app potentially perform asynchronous calls. For applications with hundreds of use cases, the continuous integration (CI) execution time can significantly increase and cause the project budget to skyrocket. Writing asynchronous tests can also be confusing in the beginning. Extra vigilance should be taken to properly wait for the various elements of the interface to be in the desired state throughout each test step. Inconsistencies may otherwise appear, such as tests failing randomly from one environment to another, or from one execution to another. Such tests can be detected by running tests with high CPU load.

Conclusion

Despite these limitations, the testing strategy developed in this article has been adopted on several projects of different scales at KNP. It has proven to be effective and maintainable, on projects with at least two developers and relatively long life cycles (in years).