Skip to main content

Celebrity Async Deathmatch - round 1

I'm working on an app with lots of asynchronous stream processing done using Rx (Reactive Extensions) - IObservable<T> method. We were discussing the other day whether to replace the implementations which only return a single data item via the Rx stream with the more standard TPL (Task Parallel Library) Task<T> method.

 We came to the conclusion we wouldn't make the switch for a couple reasons, firstly keeping the code consistency, we're using Rx everywhere for async so why change; and secondly with a possibly more important reason because performance isn't an issue (at the moment).

 Then I thought....

 What's the performance difference between IObservable<T> and Task<T> for a single async invocation?

A simple console app should do, running IObservable<T> vs Task<T> and the quickest wins - reminds me of Celebrity Deathmatch...

Firstly we need something to test, calculating the first 100 primes:
All I need now is a couple of methods for each async implementation...

firstly IObservable<T>:
secondly Task<T>:
All I need now is a test program:

 So which test method is quicker?

The answer is Task<T>, in facts it quicker by a factor of greater than 10:
Even if I swap the order Task<T> out performs:


Labels:
Location: London, Greater London N16, UK

Comments

  1. Bart De Smet6 September 2012 at 20:21

    I've given this a try with the latest Rx v2.0 RTM binaries and couldn't repro the issue when making a few tweaks:

    1. The comparison above uses await in one case (Rx) but not in the other (Task with Wait instead). This makes the comparison biased due to the use of the async method machinery in the former case, but not in the latter. So, I've used await in both cases.

    2. Using a bigger sample size for the number of primes computed to get out of the noise of tens of milliseconds (which is close to the magical 15.6ms anyway), and put the whole thing in a loop to repeat and compute average run time.

    3. Have a warm-up phase, eliminating the skew that may be due to mscorlib and other BCL assemblies being loaded already which Rx has to come in fresh. Also, the former assemblies are NGEN'd while Rx isn't, so giving both warm up time will eliminate JIT overheads.

    When doing all of this, the result I'm seeing are vastly different:

    - First iteration: Task wins with several 10s of percents (but not factors of 10 as shown above).
    - Subsequent iterations: IO and Task are in the same ballpark, give or take a percent on either side (mostly on the Rx side, which could be explained by our awaiter type being a class rather than a struct).

    The code used boils down to:

    sw.Start();
    {
    for (int i = 0; i < N; i++)
    await /* Task or IO */ () => CalcPrimes(M);
    }
    sw.Stop();

    with N and M being sufficiently large. When M is small, Task may take advantage of the fast path of the await code more aggressively (but then again, IObservable is optimized under the assumption event streams typically are lengthy), though Rx will do so as well (but thresholds may be different).

    In general, I'm very wary of micro-benchmarks like these. Set a performance goal for a bigger system, measure for pieces of code with relevant latencies and compute sizes, and - if goals aren't met - find the bottleneck using profilers such as the one in Visual Studio.

    ReplyDelete
  2. awkwardcoder6 September 2012 at 20:28

    Bart thanks for detailed reply and I totally agree about micro-benchmarks and yes this was a micro-benchmark come to think about it.

    Keep up the good work, the 'team' must be very busy :)


    ReplyDelete

Post a Comment

Popular posts from this blog

Implementing a busy indicator using a visual overlay in MVVM

This is a technique we use at work to lock the UI whilst some long running process is happening - preventing the user clicking on stuff whilst it's retrieving or rendering data. Now we could have done this by launching a child dialog window but that feels rather out of date and clumsy, we wanted a more modern pattern similar to the way <div> overlays are done on the web. Imagine we have the following simple WPF app and when 'Click' is pressed a busy waiting overlay is shown for the duration entered into the text box. What I'm interested in here is not the actual UI element of the busy indicator but how I go about getting this to show & hide from when using MVVM. The actual UI elements are the standard Busy Indicator coming from the WPF Toolkit : The XAML behind this window is very simple, the important part is the ViewHost. As you can see the ViewHost uses a ContentPresenter element which is bound to the view model, IMainViewModel, it contains 3 child v
Post a Comment
Read more

Showing a message box from a ViewModel in MVVM

I was doing a code review with a client last week for a WPF app using MVVM and they asked ' How can I show a message from the ViewModel? '. What follows is how I would (and have) solved the problem in the past. When I hear the words ' show a message... ' I instantly think you mean show a transient modal message box that requires the user input before continuing ' with something else ' - once the user has interacted with the message box it will disappear. The following solution only applies to this scenario. The first solution is the easiest but is very wrong from a separation perspective. It violates the ideas behind the Model-View-Controller pattern because it places View concerns inside the ViewModel - the ViewModel now knows about the type of the View and specifically it knows how to show a message box window: The second approach addresses this concern by introducing the idea of messaging\events between the ViewModel and the View. In the example below
5 comments
Read more

Custom AuthorizationHandler for SignalR Hubs

How to implement IAuthorizationRequirement for SignalR in Asp.Net Core v5.0 Been battling this for a couple of days, and eventually ended up raising an issue on Asp.Net Core gitHub  to find the answer. Wanting to do some custom authorization on a SignalR Hub when the client makes a connection (Hub is created) and when an endpoint (Hub method) is called:  I was assuming I could use the same Policy for both class & method attributes, but it ain't so - not because you can't, because you need the signatures to be different. Method implementation has a resource type of HubInnovationContext: I assumed class implementation would have a resource type of HubConnectionContext - client connects etc... This isn't the case, it's infact of type DefaultHttpContext . For me I don't even need that, it can be removed completely  from the inheritence signature and override implementation. Only other thing to note, and this could be a biggy, is the ordering of the statements in th
Post a Comment
Read more