Skip to main content

Unit testing Rx methods Timeout & Retry with moq

Earlier this week I was trying to unit test an asynchronous service (Foo) which used another asynchronous service (Bar) internally and ran into an issue trying to mock out the Bar service so that it would cause the retry & timeout schedules to fire.

Bar is defined as follows, the implementation is irrelevant as it being mocked for the tests:

   1:  public interface IBarService
   2:  {
   3:      IObservable<Unit> Generate();
   4:  }

Foo is similarly defined:

   1:  public interface IFooService
   2:  {
   3:      IObservable<Unit> Generate();
   4:  }

The implementation of the Foo service is the important part, it uses the Boo service to generate a value, it's expected to generate the value or Timeout, if it fails to generate a value (for what ever reason) it's expected to to Retry:

   1:  public class FooService : IFooService
   2:  {
   3:      private readonly IBarService _barService;
   4:      private readonly IScheduler _scheduler;
   5:   
   6:      public FooService(IBarService barService, IScheduler scheduler)
   7:      {
   8:          _barService = barService;
   9:          _scheduler = scheduler;
  10:      }
  11:   
  12:      public IObservable<Unit> Generate()
  13:      {
  14:          return _barService.Generate()
  15:                            .Timeout(TimeSpan.FromSeconds(10), _scheduler)
  16:                            .Retry(5)
  17:                            .Select(bar => Unit.Default);
  18:      }
  19:  }

You can see I've set the time out to be 10 seconds and a maximum of 5 attempts...

So I wanted to put Timeout & Retry under test, to do this we use moq for mocking out our dependencies, we also use nCrunch for continuous testing, you'll see this in the screenshots below as the red\green icons on the left-hand side of the code - these icons tell me where the code is failing etc..

   1:  [Test]
   2:  public void should_timeout()
   3:  {
   4:      // ARRANGE
   5:      Exception exception = null;
   6:      var fooService = new FooService(_barService.Object, _testScheduler);
   7:   
   8:      // ACT
   9:      fooService.Generate()
  10:                .ObserveOn(_testScheduler)
  11:                .Subscribe(_ => { }, exn => exception = exn);
  12:   
  13:      _testScheduler.AdvanceBy(TimeSpan.FromHours(1).Ticks);
  14:   
  15:      // ASSERT
  16:      Assert.That(exception, Is.Not.Null);
  17:  }
  18:          
  19:  [Test]
  20:  public void should_retry()
  21:  {
  22:      // ARRANGE
  23:      Exception exception = null;
  24:      var fooService = new FooService(_barService.Object, _testScheduler);
  25:   
  26:      // ACT
  27:      fooService.Generate()
  28:                .ObserveOn(_testScheduler)
  29:                .Subscribe(_ => { }, exn => exception = exn);
  30:   
  31:      _testScheduler.AdvanceBy(TimeSpan.FromHours(1).Ticks);
  32:   
  33:      // ASSERT
  34:      Assert.That(_retryCount, Is.EqualTo(5));
  35:  }

As you can see two identical tests, asserting on different behaviours of the FooService, you'll notice the constructor of the FooService is being injected with an instance of the BarService, this is being created as Mock<T> of the IBarService interface.

You'll also notice the use of the MS TestScheduler for Rx, essential for unit testing anything in Rx.

So my first attempt at mocking this out looked like this:

   1:  [SetUp]
   2:  public void SetUp()
   3:  {
   4:      _testScheduler = new TestScheduler();
   5:   
   6:      _retryCount = 0;
   7:      _barService = new Mock<IBarService>();
   8:      _barService.Setup(x => x.Generate()).Returns(
   9:          () =>
  10:              {
  11:                  Debug.WriteLine("Retry {0}", ++_retryCount);
  12:                  return Observable.Never<Unit>();
  13:              });
  14:  }

When the tests were run I didn't expected either of the tests to fail, but what I got was one successful & one failed test! The Timeout had worked but the Retry schedule hadn't!
My initial thought was I hadn't passed the scheduler to the Rx Retry method in the FooService implementation, but after checking I realised it doesn't need take the scheduler so the problem must be with the test setup. To be more precise the problem must have been with what was being returned for the mocked Generate method on the IBarService.

This was indeed the problem, the mock should have been returning a Func which created an observable sequence which never pumps instead of what I initially had, a Func returning a sequence which never pumped:

   1:  [SetUp]
   2:  public void SetUp()
   3:  {
   4:      _testScheduler = new TestScheduler();
   5:   
   6:      _retryCount = 0;
   7:      _barService = new Mock<IBarService>();
   8:      _barService.Setup(x => x.Generate()).Returns(
   9:          () =>
  10:              {
  11:                  return Observable.Create<Unit>(o =>
  12:                  {
  13:                      Debug.WriteLine("Retry {0}", ++_retryCount);
  14:                      return Observable.Never<Unit>().Subscribe(o);                
  15:                  });
  16:              });
  17:  }

 The change is subtle but makes a big difference when testing:
Looking at the decompiled code it tells me why, the Retry extension method is using the Catch extension method to replace the faulted source stream with another, it just so happens that happens to be the source stream, so therefore from a mocking point of view you need to make sure that every time the Return Func is executed it creates a valid non-faulted stream.


Labels:
Location: London Borough of Hackney, London N16 1AZ, UK

Comments

  1. Stelrad5 January 2013 at 08:15

    "This was indeed the problem, the mock should have been returning a Func which created an observable sequence which never pumps instead of what I initially had, a Func returning a sequence which never pumped" - I don't follow that explaination, they are both observable sequences which never pump.

    A simpler explanation is; Generate is called once, the thing that it returns is subscribed to 5 times.

    ReplyDelete
  2. awkwardcoder5 January 2013 at 13:46

    Generate is called multiple times and you need to make sure what's returned is non faulted stream - the first attempt doesn't do that

    ReplyDelete
  3. Stelrad5 January 2013 at 15:47

    Dunno how well this will format but:


    [SetUp]
    public void SetUp()
    {
    _testScheduler = new TestScheduler();

    var called = 0;
    _retryCount = 0;
    _barService = new Mock();
    _barService.Setup(x => x.Generate()).Returns(
    () => Observable.Create(o =>
    {
    Debug.WriteLine("Retry {0}", ++_retryCount);
    return Observable.Never().Subscribe(o);
    })).Callback(() =>
    {
    ++called;
    if(called > 1)
    Console.WriteLine("oh");
    });
    }

    ReplyDelete
  4. awkwardcoder5 January 2013 at 18:31

    The point is, the first attempt at testing the retry does not use Observable.Create for the moq setup...

    ReplyDelete
  5. Stelrad5 January 2013 at 21:43

    Hey Ollie,
    Just wanted to clear this up, sorry i wasn't clearer from the start.
    Yeah, you are right you need the Create to verify the Retry operator is resubscribing to the stream - its important to note that the method itself is only called once though. For this reason I find that it is better to expose IObservabe streams as properties (unless they are paramterised). In this case maybe

    IObservabe Bars { get; }

    The faulted stream thing is also interesting - Observabe.Empty is a do-nothing cold-observable, it can carry no state so cannot be 'faulted'. Indeed as far as I am aware it is the responsibility of the Observer to ignore OnNext's after an OnError so you could say that only a subscription could be errored. Using Observabe.Create and the lambda-based subscribe extensions methods shield you from behaving badly on both sides.

    HTHs
    Cheers

    ReplyDelete
  6. Stelrad6 January 2013 at 04:51

    "only a subscription could be errored" sorry for the obviuous bullshit - whilst well behaved observers do ignore OnNexts after OnErrors, well behaved IObservables do not call OnNext after an OnError - so indeed the stream is faulted.

    ReplyDelete

Post a Comment

Popular posts from this blog

Implementing a busy indicator using a visual overlay in MVVM

This is a technique we use at work to lock the UI whilst some long running process is happening - preventing the user clicking on stuff whilst it's retrieving or rendering data. Now we could have done this by launching a child dialog window but that feels rather out of date and clumsy, we wanted a more modern pattern similar to the way <div> overlays are done on the web. Imagine we have the following simple WPF app and when 'Click' is pressed a busy waiting overlay is shown for the duration entered into the text box. What I'm interested in here is not the actual UI element of the busy indicator but how I go about getting this to show & hide from when using MVVM. The actual UI elements are the standard Busy Indicator coming from the WPF Toolkit : The XAML behind this window is very simple, the important part is the ViewHost. As you can see the ViewHost uses a ContentPresenter element which is bound to the view model, IMainViewModel, it contains 3 child v...
Post a Comment
Read more

Showing a message box from a ViewModel in MVVM

I was doing a code review with a client last week for a WPF app using MVVM and they asked ' How can I show a message from the ViewModel? '. What follows is how I would (and have) solved the problem in the past. When I hear the words ' show a message... ' I instantly think you mean show a transient modal message box that requires the user input before continuing ' with something else ' - once the user has interacted with the message box it will disappear. The following solution only applies to this scenario. The first solution is the easiest but is very wrong from a separation perspective. It violates the ideas behind the Model-View-Controller pattern because it places View concerns inside the ViewModel - the ViewModel now knows about the type of the View and specifically it knows how to show a message box window: The second approach addresses this concern by introducing the idea of messaging\events between the ViewModel and the View. In the example ...
5 comments
Read more

WPF tips & tricks: Dispatcher thread performance

Not blogged for an age, and I received an email last week which provoked me back to life. It was a job spec for a WPF contract where they want help sorting out the performance of their app especially around grids and tabular data. I thought I'd shared some tips & tricks I've picked up along the way, these aren't probably going to solve any issues you might be having directly, but they might point you in the right direction when trying to find and resolve performance issues with a WPF app. First off, performance is something you shouldn't try and improve without evidence, and this means having evidence proving you've improved the performance - before & after metrics for example. Without this you're basically pissing into the wind, which can be fun from a developer point of view but bad for a project :) So, what do I mean by ' Dispatcher thread performance '? The 'dispatcher thread' or the 'UI thread' is probably the most ...
Post a Comment
Read more