Friday, January 16, 2015

Synchronizing multithreaded incoming messages and unit testing

I'm currently developing a Windows service application that receives data from multiple sources. It isn't a highly concurrent application, but the incoming messages are come fairly fast after one another.

Incoming messages enter the system via WCF and that part is multithreaded. But these messages must be handled sequentially. What's more, certain pieces of our application must run at regular intervals.

We can experience problems if component A is running and changing objects in memory, and component B is triggered and starts using these same objects.

That's why we introduced a simple command queue:

public interface ICommandQueue : IEnumerable<Action>
{
    void Add(Action action);
    void CompleteAdding();
}


The implementation uses a BlockingCollection:

public class ThreadSafeCommandQueue : ICommandQueue
{
    private readonly BlockingCollection<Action> _actions = new BlockingCollection<Action>();

    public void Add(Action action)
    {
        _actions.Add(action);
    }

    public IEnumerator<Action> GetEnumerator()
    {
        return _actions.GetConsumingEnumerable().GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    public void CompleteAdding()
    {
        _actions.CompleteAdding();
    }
}

This means another class can loop over it and execute every action that is added. If the queue is empty, it will wait until a new item is added. You can read more on BlockingCollections for the ins and outs.

This is our implementation:

Task.Factory.StartNew(() =>
{
    foreach (var action in _queue)
    {
        action();
    }
}, TaskCreationOptions.LongRunning);

So, what happens is we add stuff that needs to be executed safely to this command queue:

_commandQueue.Add(() =>
{
    _something.Do(_input);
});

As you can see, you have access to private fields, as you're just passing in the entire Action to be performed.

Now the cool thing is this is entirely unit-testable. Because we're using a DI-container (Autofac in our case), we can inject a different implementation when we're testing. Because we don't want our Assert methods to start asserting if the Action hasn't been executed yet, we can inject the following when we're testing:

public class ImmediateCommandQueue : ICommandQueue
{
    public IEnumerator<action> GetEnumerator()
    {
        throw new NotImplementedException();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    public void Add(Action action)
    {
        action();
    }

    public void CompleteAdding()
    {
        throw new NotImplementedException();
    }
}

When an Action is added, it is immediately executed. So when we've done the 'Act'-part of our tests (see AAA), we can be sure there aren't any Actions still waiting to be executed.

But what this also allows us to do is control when the Actions are executed. Take this method:

public void Process()
{
    _a = new List<int> { 1, 2, 3 };
    _commandQueue.Add(() => { _innerProcessor.Process(_list); });
    _list.Clear();
}

This is of course a silly method, but bear with me. The point is that, at runtime, the innerProcessor might receive en empty list because we're clearing it immediately after we added the Action. But in our tests, this isn't very clear because we're immediately executing the Action.

So we can introduce yet another implementation of ICommandQueue, purely for testing purposes:

public class ManuallyTriggeredCommandQueue : ICommandQueue
{
    private readonly IList<Action> _actions = new List<Action>();

    public IEnumerator<Action> GetEnumerator()
    {
        throw new NotImplementedException();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    public void Add(Action action)
    {
        _actions.Add(action);
    }

    public void CompleteAdding()
    {
    }

    public void ExecuteNextAction()
    {
        var action = _actions[0];
        _actions.RemoveAt(0);
        action();
    }
}

Now, in our tests, we can call the Process method first, and only then execute the Action by calling ExecuteNextAction. This way we can get a failing test (because _innerProcessor.Process was called with an empty list instead of a list with 3 integers). And after a failing test, we can turn it into a green test:

public void Process()
{
    _a = new List<int> { 1, 2, 3 };
    _commandQueue.Add(() => 
    { 
        _innerProcessor.Process(_list);
        _list.Clear();
    });
}

I'm very happy with this solution. It's geared towards our domain, as we're not going to flood our queue because we won't be getting thousands of messages every millisecond. But it allows us to avoid threads getting in each others way, and our code remains testable.

Wednesday, December 31, 2014

2014 is so 2014

Another year has past, and I suspect many bloggers will be making lists of what happened in 2014 and what will happen in 2015. And so will I, but I'll keep it short.

These lists are actually quite handy, because they allow you to keep track of what your goals are and whether or not you met them. Depending on your situation, you might decide you slacked too much, or maybe you had too many goals. Or unrealistic ones. Or maybe you met all your goals and in that case, good for you!

Revisiting my post from the previous year, I can list what I achieved and what I didn't.

Didn't achieve:
  • Acquire 2 Microsoft certifications: this is a subject for a future post, but those who know me, know I'm in the non-believer-camp regarding certifications.
  • Finetune pwa.js, FlitsLogo: these are still on the mental to-do-list, but aren't very high in priority.
  • Migrate my blog: still very much want to do this.
  • Finish my course on beer: I've actually read and studied the material, but I still need to plan the exam.
  • Kitesurfing: oh well :)

Achieved:
  • Find and follow a good training: I attended ngEurope this year where I really learnt a lot. Also Techorama, but ngEurope was more interesting for me because it's outside of the traditional Microsoft-world where I'm in most of the time. (Though I will most likely be attending Techorama again this year)
  • Write a Drupal module: I did write a Drupal module for my frisbee team. Actually, I wrote several custom modules. There's still a lot of work to be done there, but I'm happy with what I have already. Also, it's already being used on a site with 140+ users.
  • Write a SPA: I wrote a web application to have a small contest with some friends during the World Cup. The code isn't pretty, but it worked and wasn't meant to be a long-term project anyway. But I did learn Angular that way.
Didn't plan, but achieved:
  • Start my own company: I am now a freelance developer, so that's exciting. More on that later.
  • Learnt Cordova: but I still regard myself as a beginner.
 And finally, plans for 2015:
  • Finish the Cordova mobile app I'm writing for my frisbee team
  • Push out some changes I've made to my 8cam Windows Phone app.
  • Maybe: rewrite FlitsLogo in HTML5 and javascript (using Cordova) so I can put it on multiple platforms
This list is definitely shorter than my 2014-list. But with starting my own company I suspect there might be a little less time, plus there are also personal/private goals I have set. And I tend to avoid putting too many private stuff on the internet.

So there is my list for 2015. Let's hope I can achieve these things.

And with that, I wish you a great 2015 and good luck with your goals!

Wednesday, December 3, 2014

ngEurope: recap

I've finished all my posts on ngEurope, so here's a small recap.

The conference itself was very decent, with some minor hiccups. Kudos to the organizers, and I hope to be able to attend it again. The sessions and speakers were very good, and it was really interesting (dare I say inspiring?) to see all the things that are happening in the Angular, javascript, and web world.

Here is an overview of my posts, which is an overview of what I found most interesting:
I think it's safe to say the web (both 'normal' and mobile) has a bright future. There are many promising projects being run by smart and enthusiastic people.

Of course, ngEurope focussed on Angular, and there are so many other frameworks and libraries out there. Will Angular stick around long enough for that long-term project? Or should you try another option? Or will all this javascript-nonsense fade away sooner or later?

Nobody knows the answer to this, but these questions can be asked about any technology stack. I'd be willing to bet the web will be around for quite some time. If Angular will too? I don't know. But it's been around for some time now (released in 2009) and has a growing community around it. At the moment, it seems a pretty safe bet to me. But of course, it depends on the specifics of your project, which might be something I discuss in a separate post in the future.

Tuesday, December 2, 2014

ngEurope: $q and promises

If you've dabbled with AngularJS, it's very likely you've encountered $q. At ngEurope, there was a session devoted to $q, which gave a good overview of what it can do.

The 'old way' of doing asynchronous programming in javascript, is to use callbacks. But when your application becomes more complex, this leads to nested callbacks, which are ugly, unreadable, hard to maintain,... But they're also not parallelizable, composable or dynamic.

With $q, you can stop the callback-madness. Instead of passing callbacks as parameters, you can start returning promises.

Using $q can seem a little confusing at first, but it is actually quite simple. Start by creating a deferred object with
$q.defer();
Then call
deffered.resolve()
when you're done.
A user of your API (which could be yourself) can then call:
getSomething().then(function() {});
But $q can do more than just that. It can run functions in parallel:
$q.all([getA(), getB()]).then(function(responses) {});
The all-function returns a new promise and will only resolve when all functions are done. The responses-argument contains the responses of both functions. Or, if you prefer to have the different responses as separate arguments:
$q.all([getA(), getB()]).then(spread(function(a, b) {}));
Promises can be composed:
var promise1 = foo();
var promise2 = promise1.then(...);
var promise3 = promise2.then(...);
The then-function returns a new promise every time.

Because promises don't force you to chain at write-time, you can make all this dynamic. For example, based on if-else statements, different arrays of promises could be built and executed.

Promises can also notify when something went wrong. The 'notifier' doesn't call deferred.resolve, but deferred.reject() instead. The receiver then has its second argument called:
getSomething().then(function() {
    alert('success');
}, function() {
    alert('error');
});
There is also a third arguments to indicate progress. This is a function that can be called multiple times, while resolve and reject can only be called once:
deferred.notify();
The when-function allows you to wrap functions/objects in a promise and return it. For example:
getMovies: function() {
    return $q.when(cachedMovies || fetchFromServer());
}
This allows for easy client-side caching. Even better would be to use this piece of code:
getMovies: function() {
    return $q.when(cachedMovies || p || fetchFromServer());
}
The 'p' variable should be set in the fetchFromServer-function and it should get the value of deferred.promise. This will avoid multiple http requests when we try to get the movies but haven't yet returned from the previous call.

$q is not the same thing as Q, although it is based on it. $q is aware of the digest loop and is used throughout Angular.

I encourage you to start using promises as they are used everywhere, not just in Angular. Plus, native promises are coming soon.

Friday, November 28, 2014

ngEurope: Software Patterns and Design

This session went into some of the patterns you see out there. A quick summary:
  • Services: these are injected singletons. They're an ideal place to cache application-level data, as a facade for the browser and third-party API's, or factories for instantiating other objects.
  • Databinding: this is just an implementation of the observer pattern.
  • Inheritance for services & controllers: Standard prototypal inheritance can be used with services and controllers, but it's recommended to avoid inheritance for page controllers.
  • Thick models: for example, instead of having a method isActive(item) on the controller, put the isActive method on the item itself. Prefer thick models with methods and behavior over having this in your controllers. This way, you can encapsulate your domain logic.
  • Directive proxy object: how can you communicate with a directive from a controller? We want to avoid coupling the directive tightly to the controller. With this pattern, the controller creates and owns a proxy. It doesn't need to know the directive. The directive can then bind to the proxy object.
  • Polymorphism and directives: move responsibilities that normally are in the directive to the model. The model can be switched for another, as long as it has the same interface.
Then some anti-patterns:
  • War & Peace controllers: very large controllers that do too much. Split it up in more controllers, services,...
  • Link Function of Doom: all code of a directive in its link function
  • Forgot About Dialogs Global state: services storing per page state and clearing upon route change. This is not what services are for.
  • Magical Prototype Chain Dependency: relying on properties up the chain, making refactorings/changes harder/more scary.
Don't ask yourself if it's "the Angular way". Ask yourself if it's good software (refactorable, testable, readable,...).

Thursday, November 27, 2014

ngEurope: Angular and accessibility

At ngEurope, Marcy Sutton's session on accessibility was one of the sessions I looked forward to most. Accessibility is something we often forget. While screenreaders and other tools are becoming increasingly sophisticated, so are our websites and applications. And it's worth it to invest some time in accessibility, because everyone should be able to use the web.

So here's a quick recap of what was covered.

Accessibility (or a11y, with 11 referring to the 11 omitted letters) starts with using semantic elements. With HTML5, we have lots of new elements that communicate something meaningful about their contents: article, aside, etc.

If you're using javascript, and have elements that are only there for your javascript code, add the attribute tabindex="-1" to make it accessible for javascript, but not for the keyboard.

The ARIA-attributes are for screenreaders, magnifiers, etc. Agnular provides the ng-aria directive for this. There also decent documentation on accessibility and AngularJS. ngAria hooks into ngModel and dynamically adds ARIA attributes where needed.

But there's more:

Roles communicate what an element does:

<div role="img"...>

States:
<md-input-group aria-disabled="true">

This can be used because this is a custom element and the disabled property is only available on certain elements.

The application role disables a screenreader's virtual cursor. Don't do this on the entire body, only on things that mimic for example desktop widgets. A user won't be able to navigate in this element anymore. This is an example of the application role:
<div role="application">

Think about enabling keyboard access.
<div ng-click="sorryKeyboard()"></div> // this won't work
<button ng-click="better()"></button> // this is better
<md-button ng-click="better()" ng-keypress="better()"></md-button> // if you must use a custom element

Be sure to test your site with the keyboard. Does it work? Are all elements accessible via the keyboard and can everything be controlled/clicked/toggled/...? Does it work pleasantly or do you find yourself having to press tab too much to navigate?

Don't forget ':focus' to let the user know where the focus is at the moment.

Marcy contributed accessibility improvements to Angular Material. For example, for Material Checkbox, it will add a role, but also a label, based on the text it finds. What's more, there's code to warn the developer that it can't find a label.

Some tools for accessibility:
This was most definitely the most interesting talk of ngEurope to me. Just because it connects our technical world with something we don't often come into contact with. What's more, it's something of an overlap with my wife's world (speech therapy/remedial teaching), where dyslexic people could also greatly benefit from. If we're serious about the web and connecting all people in the world, accessibility is something we shouldn't forget.

Monday, November 24, 2014

ngEurope: Famo.us and some other interesting things

In my series of posts on ngEurope, I've covered quite a bit by now. It's testament to the volume covered at ngEurope. There are some smaller things I'll cover in this one post. This doesn't mean they are less significant. Just that they resonated less with me, but they are interesting enough to mention.

After this one, there are still some posts coming where I focus on a single subject. At the end, I'll post an overview of everything I covered.

Famo.us
Users are spoilt when it comes to animations and interactivity on mobile devices. This points to a reason why the web hasn't taken over the mobile world. Javascript sacrifices performance, while CSS doesn't offer all possibilities.

Yes, CSS now has animations, but they are complex to build, and even harder to refactor. This is what's called the declarative ceiling.

You could use javascript, and it certainly has come a long way. But its performance is still not up to par with native code, especially on mobile devices.

This is what Famo.us tries to solve: the performance of javascript animations. Famo.us doesn't use the DOM. Instead, it has its own render tree, and outputs the result to CSS Matrix3D properties.

To use this in an existing Angular app, check out Famo.us/Angular.

Sheets
Up until now, you had to put your Famo.us attributes inline, much like inline CSS (because the were Angular bindings). So what if you could put javascript in your CSS files? Kind of ugly, no?

Sheets tries to solve this. You can define your own fields and the logic for handling that field for a given element. It separates the template from its logic, with selectors as the link. It also allows you to use media queries.

Other interesting stuff
  • Restangular: A cleaner API to talk to REST services, although a bit Active-Record-like (objects know of their URL and can get their child objects, whether you like this or not is up to you).
  • Traceur: Compiles ES6 (and more) to ES5 (javascript of tomorrow to javascript of today). Most modern browsers are working hard on supporting ES6, but there's still a lot of work. Traceur allows you to code in ES6, but run your code in ES5. The result works in all modern browsers. It is highly modular and extensible.
  • ngAnimate: Enables animations in Angular in an easy way.
  • Firebase: Backend-as-a-service. Definitely worth looking at if you want to get up and running quickly. To learn more about BaaS, I recommend episode 129 of Javascript Jabber.
  • JSON Web Tokens: Cookies don't flow between apps or servers. JSON Web Tokens can. Also checkout jwt.io to encode/decode a JSON Web Token.