Distributed Agile: The Why and How of Remote Scrum Teams

In my first real developer job I was working in a very small team in a startup. We were building a new front-end application from scratch. The team at the time consisted of only me, a junior front-end developer, and one more senior colleague.

There was one particular challenge that gave us a hard time in getting the project off the ground: My senior colleague was based in Sydney while I was based at the company’s main office in Berlin. I was surrounded by other developers in that office but the person I was working closely with – and the only other front-end developer in the team – was a couple of thousand kilometers away.

Making design decisions together and getting help from my remote colleague was a big challenge. Most of the time I had to wait until the next day to get an answer. Some written conversations dragged on for days. Because of the time difference it was also extremely hard to stay in contact over the phone. When I was just starting my day in Berlin, he would be finishing his day in Sydney.

After my initial experience with that mini remote team I’m now facing a similar challenge in my current role at AND Digital: I’m working with a Scrum team which is distributed between the UK and India. This team is much bigger and altogether we are spread between four offices.

This raised a question in me: Can you do agile in a distributed team? Does it make sense or should a Scrum team always be in the same location together?

After doing some research it turns out that in the world of software engineering it’s actually considered highly controversial whether or not having a distributed Scrum team is a good idea or not…

Why it’s a good idea:

  • Hiring offshore or remote developers can help closing the gap of finding enough tech talent in the main country you are operating in.

  • Per-head cost can be cheaper for remote developers.

  • When the decision to go with remote developers has been made, mixing them in a team together with on-site colleagues facilitates a greater exchange of knowledge in comparison to separating the teams per location. This makes it also more likely that both sides are working in the same direction.

  • With workers in different timezones a product can be worked on almost 24 hours a day and so the time to reach the market for new products and features can be significantly shorter (with excellent management).

  • Being able to work remotely from home can help better the work/life balance of team members and so can boost morale.

Why it’s a bad idea:

  • Working in different locations and timezones adds a lot of communication overhead when working on the same product. Looking at the agile manifesto it says: ‘Individuals and Interactions over processes and tools’. The interaction part is hard to put into practise over long-distance when you can’t just go over to someone’s desk and start chatting.

  • Related to the latter point the 12 principles of the agile manifesto also mention: ‘The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.’ How efficient can a Scrum master remove impediments if part of the team is not in the same place? Similarly how effective can a tech-lead give advice to people in another place and time zone?

  • The 12 principles also say: ‘Business people and developers must work together daily throughout the project.’ With business people often onshore while most or all of the developers could be remote this constitutes a definite challenge. How well does an offshore team understand the product and the requirements when there is no analyst or product owner on site with them?

  • Another one from the 12 principles: ‘Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.’ How do you keep a remote team motivated in the long run? How do you build trust and personal relationships over a long distance?

  • It’s debatable if hiring offshore developers actually decreases costs because of the added process and management overhead. My opinion is that with good management it can save costs, but the danger that it just adds frustration and effectively costs the same because of lower productivity is very high.

  • There might also be a cultural challenge e.g. if you work with an offshore team based in India. The people working for you might expect clear instructions from a project manager instead of the shared ownership and responsibility that agile propagates. How do you transfer the agile/Scrum culture and approach onto a team that might have been trained to agree with everything they are asked to do?

If you are in position to decide if your Scrum team is going to be distributed or based in one location you might consider the above mentioned points and make your decision. Happy days!

But what if you don’t have that power or if your organisation already has distributed teams? What can be done to improve the agile ways of working between continents or countries if you have to work with distributed teams?

What can be done to improve team feel and productivity across a distributed team?

  • Both sides could consider sending rotating ‘ambassadors’ to the team on the other end. When they are on site those ambassadors can then build personal relationships with the other team. In the future this can help facilitating communication much more easily because they know the ‘remote personalities’ and their surrounding work conditions much better. People are also much more likely to raise issues when the have a good personal relationship.

  • Having informal meetings that are focused on how the remote teams work together and that should contain a good amount of small talk.

  • Consider installing a Scrum Master who speaks the languages of all team members across the different locations. Being able to translate will foster a good understanding of the project requirements on both sides. Understanding the cultural differences better should also help making the communication between both sides much smoother.

  • Celebrate successes together to stay motivated. It can’t be that the offshore team members don’t get any of the (metaphorical or literal) cake when a release went well. They need to be included so that they feel part of the team and know what they are working for.

  • In terms of core working times you should try to agree to a window of time overlap that works for everyone. If the agreed time is an inconvenience for anyone try rotating the time to make sure that the inconvenience is shared between everyone.

  • Being mindful of holidays onshore/offshore for scheduling releases and workload.

What can be done to improve technical collaboration in a distributed team?

  • When kicking off a new project in a distributed team the senior developers should work on the base architecture together in the same location for a while to make sure that the project is built on stable ground.

  • Developing your software in a modular way allows teams in different locations to work autonomously on their modules. The technical communication overhead and potential for misunderstandings will decrease when only APIs and integrations need to be discussed.

  • If on the contrary the remote team members work on the same codebase then remote pair programming using screen sharing can accelerate knowledge transfer and a coherent coding style. Make sure that code reviews are shared between locations for the same reason.

  • Using CI and small increments and reviews can help immensely for onshore and offshore to keep in sync.

  • Written documentation that both parties can turn to when the other party is off due to the time difference can be very useful.

  • Consider transferring knowledge to remote developers by transferring analysis capacity to the remote location. If this can’t be done on a permanent basis then analysts from the main location should travel between locations regularly to give the remote developers the business context. Understanding why something is build will influence many small decisions that need to be made every day and eventually result in a product that is more appropriate for the purpose it was intended for.

For tips on specific tools and techniques that can help improve collaboration between remote teams please also see this article by my AND Digital colleague Dan Parkes.

Mutation Testing with Stryker in an Angularjs 1.6 App

In my current project I’m working on a large and messy Angular front-end. We are frequently making small changes and bugs keep creeping into the site. This had me wonder about some questions lately:

How do we make sure that your unit tests are testing the right things? And how do we find out if there is untestable code in the application or if there are tests that are always passing no matter what?

If you think the answer is measuring code coverage then I have to disappoint you: Code coverage only counts which lines of code are being executed while running the unit tests. It does not count which lines are actually tested let alone how thorough the tests are.

So how do we solve this problem? One of the potential answers is called mutation testing. What’s behind the interesting name? A mutation testing library takes the code that you are trying to test… and literally screws it up. It will for example remove function bodies and switch logical or binary operators for their counterpart. This is called mutation.

It will then run your tests against those destroyed versions of the code. Why does it do that? And what should happen? Each of the mutated pieces of code should at least fail one test. If all the tests still pass although a significant part of the code was altered then you know something is wrong with your tests. It’s actually kind of like a test suite for your test suite.

To illustrate the concept I’ve created a diagram: stryker_mutation_testing_principle To summarise: A tree of different so called mutations of our code is generated. Borrowing terms from Genetics, if all tests fail for a particular mutant, we say the mutant is “killed” by the test suite; on the opposite, if the tests passes, we say the mutant survived. Then our task has to be to “kill it” by writing a better or an additional test.

The concept of mutation testing itself has been around for decades but it used to be too computationally expensive to make practical use of it. Now with fast and parallel computing power at our hands we can give it another try… In Javascript among several mutation testing libraries there is one which is quite far developed: Stryker.

Stryker actually incorporates the technique of measuring code coverage to determine which unit tests should be executed against which tests.

Examples please!

Of course, right ahead… I wanted to give mutation testing a try but I couldn’t find any running example Angular applications with Stryker online, so I created one myself.

With a couple of hiccups due to the still thin documentation on the web Stryker was easy to install: adding a couple of dependencies to package.json and adding a stryker.conf.js to the project. In stryker.conf.js we define that we want to use Jasmine as a test framework and karma as a test runner.

Stryker will then import karma.conf.js to determine the test settings. An important thing to highlight is that the list of files to mutate is defined in  stryker.conf.js, but the list of files needed to execute the tests is being imported from  karma.conf.js.

So what does it look like when Stryker is run and how can we make its magic visible? Let’s run

node_modules/.bin/stryker run --logLevel debug stryker.conf.js

which will give us an actual output of what is mutated and how:

[DEBUG] ClearTextReporter - Mutant killed!
[DEBUG] ClearTextReporter - /Users/daten/Sites/AngularStrykerDemo/app.js: line 9:40
[DEBUG] ClearTextReporter - Mutator: BlockStatement
[DEBUG] ClearTextReporter - -               $ctrl.add = function (a, b) {
[DEBUG] ClearTextReporter - -                 var result = a + b;
[DEBUG] ClearTextReporter - -                 return result;
[DEBUG] ClearTextReporter - -               };
[DEBUG] ClearTextReporter - +               $ctrl.add = function (a, b) {
[DEBUG] ClearTextReporter - +   };
[DEBUG] ClearTextReporter - Ran all tests for this mutant.
[DEBUG] ClearTextReporter - Mutant killed!
[DEBUG] ClearTextReporter - /Users/daten/Sites/AngularStrykerDemo/app.js: line 10:27
[DEBUG] ClearTextReporter - Mutator: BinaryOperator
[DEBUG] ClearTextReporter - -                 var result = a + b;
[DEBUG] ClearTextReporter - +                 var result = a - b;
[DEBUG] ClearTextReporter -
[DEBUG] ClearTextReporter - Ran all tests for this mutant.
[DEBUG] ClearTextReporter - Mutant survived!
[DEBUG] ClearTextReporter -   /Users/daten/Sites/AngularStrykerDemo/app.js: line 15:27
[DEBUG] ClearTextReporter - Mutator: LogicalOperator
[DEBUG] ClearTextReporter - -                 var result = a && b;
[DEBUG] ClearTextReporter - +                 var result = a || b;

The first mutation that was created deleted the contents of our whole add method. The second mutation exchanges a plus operator for a minus in that same method. Luckily for both of these mutants at least one test failed. Hence the Mutant killed! output.

The third mutation – switching a logical and for an or – unfortunately was not caught by our unit tests which is indicated by the Mutant survived! output. We can fix this by adding the following test in app.spec.js (which is currently commented out):

it('should return false', function () {
  expect($ctrl.and(false, true)).toEqual(false);

Next time we run the mutation Stryker will recognise that the second test fails for this particular modification and so all will be good. No survivors.

To help you analyse the results of the mutation Stryker also gives you a handy summary in the form of an html page via the html reporter. And by the way a full list of the mutators that Stryker has built in can be found here.


I hope that I have convinced you that mutation testing is a great idea. But how practical is it to use mutation testing in a production app? Unfortunately I don’t have the answer to that. I myself have only tried it in a sample app so far. To integrate it in a productive way into a live app would be a much bigger effort and personally I am skeptical about the usefulness of doing so.

One problem that I see with using mutation testing in a large live application it is the increased time needed to execute all the tests. From the diagram above we can see that the time of execution for the test suite is multiplied by the amount of mutations made on the code. Every unit test is now not only run once but run once against every mutated version of the code.

Another problem that I see is the amount of time you’d have to spend to fix all your surviving mutations. Ideally this would result in great test quality and that’s what we are after. But in my own experience it is hard enough to get a team of developers to write a tests for every feature anyway. Getting them to also execute mutations and fix another layer of breaking tests might encounter a lot of resistance. But of course every project and team is different so I can’t speak for everyone.

However I think the concept is super interesting and can teach us a lot about how good tests are written. I really enjoyed my excursion into mutation testing and hope to someday use it in a way that goes beyond simple experiments. I’d be interested to hear any experiences using mutation testing in production if there are people out there!

How does code coverage report generation with Instanbul work?

Recently I started working on a project that uses Istanbul to measure and report on code coverage of the unit tests. I thought the tool was genius but I asked myself: How does Istanbul actually track code coverage? It seemed like magic. Today I found the time to take a high level  look at the source code of the project and I was able to shed some light on how it actually works.

Please note that this blog post is not to give you instructions about how to use Istanbul. I think there are plenty out there. Please consider this blog post as a work in progress, the information in here might be up to 50 percent wrong (my own estimate). So far my inspection of the source code has been quite high level and here I document how I THINK it works. I will try to update this blog post once I spend more time looking at some of the details of the code. Also I hope that some people will find this (googling “How do Javascript code coverage reports with Istanbul work” didn’t find any relevant results so far) and let me know if there is something wrong with my understanding.

Let’s start with a diagram of some of the core modules within Istanbul:

Diagram of Istanbul modules.

Now this diagram is certainly an outrageous oversimplification, but it will serve us as a base for further discussion. Let’s take a closer look at those modules:

  • instrumenter: enriches the code that is to be tested with extra statements that log the coverage. Such as if you had a function and you wanted to test how many times the function is called you would insert a counter that gets incremented in the first line of the function.
  • walker: ‘walks’ through the code in the form of a tree.
  • esprima parser: an external plugin that is used to parse the code which is to be tested. Esprima spits out a syntax tree that can be walked by the walker.
  • code: well this is the code that is to be covered.
  • coverage object: at some point the code is executed and through the instrumentation the execution of the code is logged. The output of this process is the coverage object. It is simply a json file with information about how many times the statements, functions and branches in the code have been executed.
  • collector: there might be several coverage object generated for different parts of modules of the code. The collector takes these coverage objects and merges them into one coverage object that contains all the information.
  • reporter: takes the final coverage object and turns it into a human readable and condensed coverage report.

In my opinion the instrumenter is at the heart of the whole plugin. Understanding the concept of instrumentation was the entry point for me to understand how code coverage can be measured. This contains the ‘magic’ that I could not get my head around before.

Important to say is that the coverage reporting is virtually independent of the actual unit tests. The only important thing is that the code execution is monitored while the unit tests are executed. This means that the unit tests need to execute the instrumented code in order for the coverage to be recorded. Before diving into the Istanbul source code I thought that the unit tests needed to deeply integrate with the coverage plugin in order for everything to work. Luckily this is not the case which makes the whole thing less error-prone, because it’s less interdependent.

Tadaaa. This was my short overview over how code coverage with Istanbul works. As mentioned before this is a work in progress and I hope to update it as soon as I have more time to dive into the internals of Istanbul more. In the meantime feel free to let me know if you find anything wrong with my explanation or if you know more about a certain area and can help me understand.

Modeling the Formation of Opinion Clusters using Cellular Automata

I recently finished a very interesting MOOC by the University of Groningen entitled Decision making in a complex and uncertain World. The course running on the platform Futurelearn was an interdisciplinary introduction to different aspects of complexity theory. Among other topics it introduced the concept of agent based modeling and cellular automata.

One example of how to use these concepts was a model that tried to predict how different opinions are spreading in a society. I was fascinated by how this simple model could explain and model a complex phenomenon like the distribution and formation of opinions in a society. Since I’m a programmer I felt like implementing this model as a simulation in the browser so that it is possible to see what is actually happening when you run it.

In this blog post I will explain what the simulation is about and explain a couple of words about the background. I will conclude with some decisions I had to make for my implementation and finally suggest some takeaways.

The Problem

The simulation models opinion dynamics in a society trying to answer the question:

Why do opinion clusters form?

Or a bit longer:

„What are the mechanisms that lead to the exchange of opinions between individuals in a society?“

The model defines the dynamics of a number of so called cells or agents that simulate humans in a society, but interact more predictably only defined by a restricted set of rules. Some elements of chance are applied as well to be able to more realistically reflect the behavior of a real society.

In 1997 the American political scientist Robert Axelrod proposed a model of “cultural dissemination” to simulate clustering and self-organization of opinions in a society through the interplay of social influence and homophily (both explained below). Because this model is quite simple and well defined I chose it for my implementation.

Cellular Automata

The simulation runs in a “world” represented by a two-dimensional grid of square cells. Each cell has an opinion about each one of a number of so called opinion dimensions ‒ also called features. The amount of opinions each feature can take is restricted ‒ they are called traits.

An example: A feature could be the party that an individual is voting for in the next election and the traits would be the parties available for election. Another feature could be the individual’s preferred hair product ‒ then the traits would be the available brands. Obviously the more features and the more traits there are, the more complex the resulting “opinion space” will be.

Cells can interact with other cells, but only with its neighbors. The model by Robert Axelrod uses so called Von Neumann neighborhoods. Here every cell can only interact with their direct orthogonal neighbors, not with their diagonal neighbors and not with neighbors of neighbors:

sketch of a Von Neumann Neighborhood

For the cells at the borders of the grid we imagine an “invisible bridge” to the cell at the opposite border of the grid, effectively making our world a torus rather than a rectangular plane.

Social Influence and Homophily

How does one cell influence the opinion of another? In the model we use two mechanisms:

1. Social Influence

This concept simply refers to the fact that when individuals interact with each other, their opinions tend to become more similar. Due to the neighborhood we are using only direct neighbor cells will ever directly interact and influence each other.

2. Homophily

Homophily is the second phenomenon that affects when social influence takes effect and when not. It refers to the fact that individuals who are more similar to each other are more likely to interact. In the model this is implemented in a way that the likelihood of one cell influencing each other is determined by the amount of traits they have in common. E.g. if two cells agree on 2 traits out of 3 features, their likelihood to influence each other would be 2/3.

The Simulation

The simulation starts with a random distribution of opinions. That does not necessarily correspond to reality but there needs to be a starting point.

In each step of the simulation two neighbors A and B are randomly selected out of the grid. The selected neighbors then get the chance to interact. Here the homophily principle explained above is applied and this will determine if neighbor A influences neighbor B. A random factor is used here, but the chance of interaction increases with the amount of traits the two neighbors have in common.

If the cells interact the social influence principle is applied and B receives a random trait from A out of the traits that they do not have in common yet. After this step the two cells are more similar than before.

When running the simulation we see the formation of opinion clusters indicated by color clusters. What we are dealing with here is a case of social self-organization. Although the cells are very simple and only allow a couple of different states, their interaction with each other has a state of its own.

The Implementation

I wanted to keep the implementation and the design of the user interface as simple as possible. It is all about understanding the model and its implications here. At the same time I wanted it to be possible for the user to set different parameters because this can change the path the simulation takes quite a lot. So the grid size as well as the amount of features and traits are mutable.

One challenge in the implementation was the visual representation. How can I indicate which cells have the same traits and which ones are different? I chose to implement two different modes of viewing the “world”: one that shows each cells trait with a different color for every trait. This means you can switch between color-coded visual representations of the world for every feature.

In the second way of viewing the world I wanted the user to be able to see all the information at once as far as that is possible without having to switch between the different features. So I implemented a ‘similarity’ view where the similarity of each cell to its neighbors is indicated by color. The more black a cell is displayed the less similar it is to its neighbors, the whiter it is the more traits it has in common with its neighbors. In this viewing mode the overall result of the simulation can be more easily investigated and the clustering over all the features is made visible.

Trying it out will make this more clear. You can find the live version here: projects.peterfessel.com/axelrod-modeling-formation-of-opinion-clusters

The code of the implementation lives here.


I like how this simple model can yield such a complex behavior and outcome. Although the rules of the model are very simple, it is not easy at all to predict the outcome of what happens when the cells interact with each other. The system of cells is literally more than the sum of its parts. That’s why seeing it live in action in form of a computer simulation can help us develop an intuition for these kinds of phenomena in society.

The amount of insight we get that can be applied to reality is vastly determined by the model that is chosen though. One problem with Axelrod’s model is that it represents actual developments in a very small society quite well, but that it does not reflect the development in a big society with a big opinion space. We can see this by increasing the amount of features and traits. The result after running the simulation is quite counterintuitive: there is less diversity in opinions (“cultural diversity”) than when running the simulation with a smaller opinion space. This results out of the fact that when there are more features, the likelihood of two cells agreeing on at least one feature is higher and thus the likelihood of two neighbors interacting at all is generally higher. In a world with less features there is quite a big chance that two neighbors don’t agree on a single feature and thus will never have the chance to interact. The model could be extended e.g. by introducing noise as suggested by Flache & Macy in an attempt to have an outcome closer to reality.

In summary we can for sure say that models like this can help us understand social phenomena better. But we should not make the mistake to confuse them with reality and expect exact predictions of the future. Models like this are more like a crutch trying to bring us closer to understanding what happens in reality.

EmberCamp 2015 notes

I was lucky enough to be attending the EmberCamp conference in London on the 29th of October 2015. It was the first Ember conference I ever went to and featuring talks by core contributors like Yehuda Katz and Matthew Beale. Here are some notes I took about the conference in the form of bullet points for easy consumption (and because I wrote them while listening to the talks…). I tried to highlight the key messages in bold.

Keynote / Yehuda Katz

  • video link
  • Don’t use Ember if you’re building an app that makes the server render new HTML when you click on a link
  • the previously mentioned type of app is probably going to be dominant on the web forever, but that’s not what Ember is built for
  • Ember release cycle works, incremental updates work
  • coming in 2016: Gilmmer 2, Fastboot updates, Engines

Composable Components / Miguel Camba

  • video link
  • no component works for all use cases
  • Coefficient of reusability: cost of construction > cost of reuse
  • how to build components that are reusable?
  • think about the community first
    • value existing solutions
    • share your pain with others
    • look for help, „single heroes“ die soon
  • think about the API second
    • throw away existing biases about how the API should look
    • start with the minimum
    • DDAU: prefer actions over bindings
  • reduce the amount of options (you can’t cover all use cases; but the end user can if you let him reuse your component)
  • favor composition over inheritance
  • create simple components with one focus that can be composed into more complex ones

ghost & Ember / Hannah Wolfe

  • video link
  • ghost is an open source blogging platform, non-profit
  • built with node.js & Ember = fullstack Javascript application
  • what drove the decision to rewrite the admin UI in Ember? (it was a messy Backbone.js application before)
    • the need to move faster & break less things
    • the lack of direction on how to solve problems
    • the Ember community (is very nice and helpful)
  • just migrating to Ember did not help to achieve all goals (yet), there are still a lot of challenges…
    • there is a tendency in Javascript client side applications to build overly complex things due to the use of global scope
    • the further you go into the development of an app, the more tightly coupled it gets and the harder it is to add things, the complexity curve gets steeper and steeper
    • Ember helps building a loosely coupled app; the complexity lies more in the beginning due to the steep learning curve of Ember, but then the complexity curve might actually become more shallow
  • problems with testing…
    • the things that kept breaking were not covered by the testing suite, they were due to state which got lost
    • unit testing helped, but still didn’t do the job
    • end to end testing suite was causing more and more problems and the testing suite wasn’t testing the right things
    • the majority of these tests were failing (after running for an hour and taking way too long), so eventually they were all removed
    • high quality testing suite needed
  • Ember moves really fast
    • it’s great that the framework evolves so fast, but it also leaves behind a lot of deprecated legacy code
    • it raises the question: what could have been achieved if the code didn’t have to be updated so much?
  • if you build your project with the power of open source, you’re standing „on the shoulders of giants“

The hidden Power of HTMLbars (on Scope in Ember templates) / Matthew Beale

  • video link
  • Javascript uses static/lexical scoping
  • Ember templates were moved from using dynamic scope in version 1.x to use static scope in 2.x
    • it’s easier to read and reason about and
    • makes it easier to implement with good performance (Glimmer has this incorporated)
  • as a user of templates it is hard to determine what changed the scope of a variable
  • yet it is crucial to understand how Ember templates uses scope
  • some tricks are possible:
  • partial application of arguments to actions and components
  • recursively rendering components
  • Ember templates tries to include the tools for scoping that you are using anyway, so you don’t have to apply hacks

Observations on Ember’s vibrant add-on community / Katie Gengler

  • video link
  • an add-on is something that extends ember-cli
  • there are currently 1920 add ons, 1325 of them have documentation and are meant to be used by others (not only for a specific app)
  • motivation to use an add-on: „Someone must have done this before and I bet they it it better than I would“
  • the EmberObserver.com score is made up of: sustainability (number of contributors), popularity, interest, maintained (have had a release in recent time)
  • maintainers are the lifeblood of the community
  • if you are an add-on maintainer you can test your add-on against various Ember version with Ember-try

Routing / Alex Speller

  • video link
  • the router is complex and often misunderstood
  • simple things are simple, hard things are possible
  • index routes are just an initial state of a certain route, they are not just for lists! (as e.g. in rails)
  • using renderTemplate is usually a mistake, except when it’s not, e.g. when rendering a toolbar into the application template or when rendering a modal
  • authentication can be done with an extra sign-in/logged-in route which doesn’t display anything and has the path ‘/’
  • route nesting == template nesting, route nesting === UI nesting (your routes should match your UI, otherwise you will probably end up with bugs)
  • for visualizing routes you can use ember-diagonal

Ember at Intercom / Gavin Joyce

  • video link
  • fast growing company building a customer communication platform
  • huge Ember app with high number of Controllers, Templated etc., page load takes several seconds
  • did a three week experiment to relaunch the app in a faster and smaller version (success: 8 times less code!)
  • did a lot of small performance improvements over time
  • still on Ember version 1.11 and Ember CLI 0.1.2
  • recommended watches:

Ember CLI deploy / Aaron Chambers & Luke Melia

  • video link
  • Version 0.5 just released!
  • its architecture follows the pipeline metaphor
  • the pipeline consists of hooks which get implemented by plugins
  • major hooks are configure, build, prepare, upload
  • e.g. ember-cli-deploy-s3 plugin can be used to upload to Amazon S3 and will be called by each of the hooks at several points during the build
  • there is a huge plugin ecosystem already for ember-cli-deploy 0.5
  • there is another plugin plugin pack, which packages up a number of plugins that belong to a build process for the ease of install

Closing Keynote: JavaScript Infrastructure @ Facebook / Sebastian McKenzie

  • video link
  • Ember was one of the first communities to adopt Babel, a JavaScript compiler that transpires JavaScript code into code that works on all browsers and minifies it
  • Babel is now a part of Ember CLI
  • Babel is implementing new ECMAScript features and participating in the discussions about new features
  • version 6.0 is more modular and lets the user take more decisions about what kind of code transformations should be applied (not opinionated)
  • version 6.0 was deployed live during the talk!

How to compile less on the fly inside a Single Page Ember app

My current Ember project is part of a so-called white-labelled product. We had to find a way to dynamically apply a different design to our app for different customers. The differences in design consist of different colors and a different logo for each customer.

Since our frontend was completely decoupled from the backend we had some discussions as to change that or not. We could have the Ember app’s index.html and app.css rendered by the backend, but we needed quick results and decided against that for now. Instead we created an endpoint in the backend where the frontend could request the properties of the custom design.

For our styles we use the preprocessor less. That way we can make use of variables for colors and it is possible for many css classes to incorporate the color values given by only a few top-level color variables. Since we deliver an uncompiled less file to the client now, we need to compile the less code on-the-fly with the custom values we get from the backend before starting the app.

How to programmatically compile less code on the Client

The less documentation on this matter is very sparse and I had to try out a lot and tweak the code until it finally worked. That’s why I want to share my solution here.

After some experimenting I ended up with the following chain of promises:

var customDesignResourceURI = "api/customdesign",
  lessFileURI = "assets/app.less",
  defaultCssFileURI = "assets/app.css";


// fire requests for the less file and the design data simulaneously
  lessFile: ajax(lessFileURI),
  designData: ajax(customDesignResourceURI)
}).then(function (results) {
  return window.less.render(results.lessFile, {
    env: "production", // in this mode, less seems to automatically cache the output css in localStorage (or the input? this is unclear)
    errorReporting: "console",
    useFileCache: true,
    sourceMap: false,
    modifyVars: { // pass the custom design vars into the less compiler
      '@color1': results.designData.color1 ? results.designData.color1 : "",
      '@color2': results.designData.color2 ? results.designData.color2 : "",
      '@logo-uri': results.designData.logo_uri ? "'" + results.designData.logo_uri + "'" : ""
}).then(function (output) {
  Ember.Logger.debug("Less source code successfully compiled into css.");
  return output.css;
}).catch(function (error) {
  Ember.Logger.error("There was an error receiving or compiling the source less file or receiving the company's custom design data.");
  Ember.Logger.debug("Loading default css.");
  return ajax(defaultCssFileURI);
}).then(function (css) {
  Ember.Logger.debug("Appending css to DOM.");
}).catch(function (error) {
  Ember.Logger.error("Could not load default css. This means loading any styles failed.");
  throw error;

What happens here is that we first request the uncustomized less file and the custom design data from the api endpoint. Both requests are done with ic-ajax and return promises, which we combine with the RSVP.hash method. On success we pass the custom design variables into the less compiler and then return the less.render method, which is again a promise. Should the compilation or the requests before the compilation fail, we catch the error and instead request a default version of the css, which is compiled in the build step of the frontend app. The last then takes the result, be it the compiled custom css or the default css, and passes it to a method which appends it to the DOM.

Appending the compiled CSS to the DOM

The last missing piece to make it all work is the code to append the ready css to the DOM. I made this a util on its own:

 * This method takes a string of CSS and appends it to the DOM.
 * It does this by creating a new <style> element and appending it to
 * the <header> of the page.
 * The code was pretty much copied one to one from:
 * http://stackoverflow.com/questions/10274260/programmatically-editing-less-css-code-with-jquery-like-selector-syntax/31836785#31836785

export appendCssToDom function (cssString) {
  var css = document.createElement('style');
  css.type = 'text/css';

  if (css.styleSheet) { // IE
    try {
      css.styleSheet.cssText = cssString;
    } catch (e) {
      Ember.Logger.error("Couldn't reassign styleSheet.cssText.");
  } else {
    (function (node) {
      if (css.childNodes.length > 0) {
        if (css.firstChild.nodeValue !== node.nodeValue) {
          css.replaceChild(node, css.firstChild);
      } else {

Where to execute the code

All the above code is in our app currently executed within an initializer. This makes sure that the css is available before anything in the app is being displayed. I’m planning to move it soon into the model hook of the application route though, so that we can benefit from a loading screen being displayed while the less is still being requested and compiled. In that case we don’t need to defer the readiness of the application. Instead we just return the promise chain from the hook.

This approach requires injecting some additional css though, probably directly into index.html, to keep the loading screen in place while the final css is not available yet. So both approaches have their advantages.

Another thing to pay attention to here is to what happens when we are testing the app. I decided to skip the whole process of less compilation when a TESTING environment variable is set to true and instead in this case I’m appending the default version of the css to index.html.


The reason why examples on compiling less code on the client are so sparse is probably that it is not actually recommended to do it. At least not in production. Mostly for the obvious reason that it is not very efficient and increases startup time a lot (think that all this might have to be executed on a mobile device). I want to make clear that also for us this is only a short term solution and will eventually be replaced by a properly designed solution where the backend will probably deliver the compiled and customized css code to the client. Until then I’m pretty happy with my solution though. It was an interesting thing to work on.

Backburner.js – A Dive into the Implementation of the Ember Run Loop

Recently in my Ember project I was stuck with the same bug for quite a while. The stack told me that the error was occurring somewhere in a module called Backburner. I had seen this namespace before in the stack and always wondered what it actually was. After doing my research it turned out that this is the namespace which contains the implementation of the Ember Run Loop. The backburner.js module is the Ember Run Loop in form of a generic library that can also be used as a plugin in non-Ember projects.

I decided to take a closer look at the implementation of Backburner, so I could better understand my error stack. The code is not very thoroughly commented, but on the other hand very readable. Documenting my findings here will help me gain a deeper understanding of the Run Loop and I hope it’ll help somebody else, too.


So let’s dive into the Backburner code first with a diagram to get an overview over the most important namespaces and functions.


Here I tried to display the methods calling each other as a flow diagram, which does not exactly match reality but makes it easier to understand. Also you’ll notice that I left out most methods that are being used to add actions to the queue(s) for simplicity. Instead I’m focusing here on the execution of the queued actions.

The Backburner namespace

When you call Ember.run the first thing which is called inside Backburner is Backburner.run. This method fires a new Run Loop. It then calls the begin method which instantiates a new set of Queues in the form of a new DeferredActionQueues object. After that setup work is done, control is given back to the run method which after checking for errors promptly calls the end method, which then triggers the flush method of the DeferredActionQueues object.

The DeferredActionQueues namespace

As said before a DeferredActionQueues object holds one set of queues. Now as we know in Ember there are six queues: actions, render, afterRender, routerTransitions, sync and destroy, which are executed one after another. When Backburner is used as a plugin in other applications, of course it can be instantiated with different queues as well.

The DeferredActionQueues.flush method, which is called by the Backburner namespace when the set of queues is meant to be executed, iterates over all queues and calls again the flush method of each queue.

The Queue namespace

A Queue object represents a single queue. This queue contains an array with the actions to be executed. When DeferredActionQueues iterates over the queues in its flush method, it calls the respective flush methods of each queue. This method then iterates over all the actions stored in the queue and invokes them one after another using the invoke method of the namespace.


One addition to make is that Backburner does not only ever contain one set of queues, but can as well have several instances of DeferredActionQueues on the stack. In that case the sets of queues will be executed one after another.

As we see the whole architecture is not very complicated to understand: We have a stack of sets of queues at the top level. When execution is triggered it goes one level down and the set of queues flushes its own queues one after another. Then at the level of a single queue all actions are executed one after another.

That’s it for the most important parts. Let me know if you think I forgot something important. For a full overview over the Run Loop also check out my other blog post on this topic.

Appendix: Logging RSVP errors

This is kind of unrelated to the general topic of this blog post, but I found out later that my bug was triggered by a failing ajax request which was wrapped via ic-ajax in an RSVP promise and executed in a Run Loop. So the whole thing was less mysterious than expected. The first step towards finding out about this was to add the following to my application code:

Ember.RSVP.on('error', function(reason) {
  console.assert(false, reason);

Only if you add this to your app.js or to an initializer errors occurring in RSVP promises will be logged in the console and you can inspect the full stack. That was my first gotcha and if you have errors involving promises and the Run Loop, maybe this will make life easier for you, too.

How to structure translations

In my current project, it was clear from the beginning that our customers would need the application in several languages. So it was smart to integrate internationalization from the beginning to avoid having to refactor all the templates later.

One question that pops up immediately when putting together a first localized version of a frontend application is: How do I organize the translations file(s) best, so that it doesn’t start to get really messy after a short time?

In my opinion there are two main possibilities to do this (let me know if you have different approaches!):

First option: group the translations per page

This might look like this:

home: {
  key1: "translation",
  key2: "translation"
page1: { ... },
page2: { ... },
contact: { ... }

Advantages of this are that it can be started right away without overthinking. If you don’t have an idea yet how your app is going to be structured when it’s finished, this might be a good choice because you can start with one page, then add another and so on. That’s why I would call this approach more agile. You might end up repeating the same translations over and over again though, because the same translations might appear in several pages.

Second option: group the translations logically


titles: { ... },
actions: {
  edit: "edit",
  delete: "delete"
forms: {
  labels: { ... },
  placeholder: { ... },
  validationWarnings: { ... }
runningText: { ... }

Advantages of this approach are that it makes maintenance easier IF the wording and structure of the page is already well defined. If you start implementing this, but the structure of your application is still subject to a lot of change, this might cause more work than it saves you because on every change you might end up having to refactor a large part of your translation file. Also you might be tempted to always reuse the same translation keys for the same translations because you think that they represent the same translations logically but then it turns out that they don’t and you have to refactor to be able to use two different ones.

Combining the two approaches

Why don’t we go for a combination of the two? That way we get the best of the two worlds. We have the flexibility of being able to apply bigger changes quickly by adding or removing pages, but at the same time we have a semantic way to organize our translations and we always know where to put everything. That means we might need to duplicate the same translations here and there for different pages, but we also stay very flexible. A translation file now could look like this:

page1: {
  runningText: { ... },
  titles: { ... },
  actions: { ... }
page2: {
  runningText: { ... },
  titles: { ... },
  actions: { ... }
footer: { ... }

... or we do it the other way round ...

titles: {
  page1: { ... },
  page2: { ... }
forms: {
  page1: { ... },
  page2: { ... }
actions: {



Well, there is not really a conclusion. The best choice differs from project to project. Since I mostly work in startup environments I would always begin with the per page approach, because usually in the beginning you don't even know what you are building. Once the project is more established and well defined, and maybe before actually translating the application to another language, it could make sense to reorganize the translations into a logical structure. That way the application stays easier to maintain and it might be also more efficient to translate because you don't have so much repetition. A combination of the two approaches might also always make sense because you always stay flexible.

Of course all of this is also applicable if you have several translation files for different modules of your application, but so far my projects were small enough so that one file was always sufficient.

What is the difference between adapters and serializers in Ember Data?

… a question that I asked myself several times and each time I was confronted with it again after not touching any adapter or serializer for weeks — finding myself again struggling to clearly define the boundaries between those two. It’s also hard to find a concise answer to this on the internet. The Ember Guides cover adapters, but not serializers.


Adapters define how your app talks to your API. They tell your application how to request resources from your API, especially how to build the URIs to request them and save them to.

Quote from the Ember Data docs:

[...] one thing you should know is that Ember Data uses an object called an adapter to know how to talk to your server.

An adapter is just an object that knows how to translate requests from Ember Data into requests on your server. For example, if I ask the Ember Data store for a record of type person with an ID of 123, the adapter translates that into an XHR request to (for example) api.example.com/v3/person/123.json.

Adapters have methods like find and findMany that are called by Ember in order to send an ajax request to the server to get one or several pieces of data. The methods will call the method buildURL internally, which builds the URI of the corresponding resource on the server. The adapter also takes care of belongsTo relationships and translates them into nested URIs.

Same goes for the opposite direction, methods like createRecord translate your record.save() into an ajax request to POST to the server. Right before sending the request, the createRecord method calls the serializer…


Serializers process the payload your application receives from the server. When the payload hits Ember for the first time, it first goes through the extract method before it touches anything else. Through customizing this method you can make the data look more like what your application expects. Also serializers can tell your application how to handle nested payloads.

When sending something to the server, serializers can also reverse the changes made when the data was received by implementing the serializeIntoHash method.


As a metaphor you can see it like this: Adapters are the dictionaries that tell your application how to talk to your API. Serializers are molds that can shape the data received from the API, so it fits exactly to what your models and the rest of your application expect.

A flow diagram could look like this:

Hope that helped. Correct me if you find any mistakes (-;

A Brief History of Time: The Ember Run Loop in short

What is the Ember run loop and how can I use it? I needed my time to understand what the Ember run loop does and how to work with it. Partly because its name is very confusing… There is still a lot of room for me to dig deeper but I felt like writing a short article about “the run loop” and give some usage examples for those who are completely new to the subject.

What does the run loop do?

The run loop is used to batch and order the execution of tasks. Through ordering and reordering, the tasks can be executed in a way that is most efficient. This is essential for Ember to run effectively, for example to batch DOM updates, which are always costly.

The batching is done in six different queues, with each one having its own responsibility:

  • actions: the general work queue, contains scheduled tasks like promises
  • render: rendering, DOM updates
  • afterRender: tasks that are executed after all primary render tasks; good for all 3rd-party-library DOM manipulation work
  • routerTransitions: the router’s transition jobs
  • sync: binding & synchronization jobs
  • destroy: jobs for finishing teardown of objects

Contrary to how its name run loop sounds, the run loop is actually not running at all times. Neither does one run loop exist as a singleton. Run loops are started or fired each time either in response to a user’s action or a timer event. When all tasks in one run loop are done, the current run loop finishes and control is returned to the system.

When a user or a timer event triggers the execution of some code, Ember works its way through the code and collects all tasks in the above mentioned set of queues. All tasks which are part of the code are registered as jobs on the queues. Jobs are basically just javascript (callback) functions. When Ember gets to the end of a piece of code that is executed in response to an event, the run loop for this event is closed and Ember starts executing those jobs one after one. Only those jobs themselves can then add more jobs on the queues at that point, while it is closed to any other code. If a new event triggers the execution of the next piece of code, a new run loop collects those jobs and is executed after the current run loop has finished.

How can I make use of this?

We can actively manipulate the run loop using the methods provided by the Ember.run namespace (Ember.run itself can also be called as a function additionally to its purpose as a namespace, which I find rather confusing). Here are some use cases I collected throughout the last months where the manipulation and/or control of the run loop was useful or necessary.


If your app is written in Ember and you want to schedule an event after a certain time interval in the future, you should not use the window.setTimeout function. Use Ember.run.later instead, that way the execution lies in the hand of Ember and events to be scheduled at the same time can actually be executed in the same run loop.

Ember.run.later(context, function() {
  // code here will execute within a run loop in about 500ms
}, 500);


As mentioned above, if you do DOM manipulation via a 3rd party library like jQuery, you should make use of the afterRender queue.

Ember.run.scheduleOnce('afterRender', this, function(){
  // jQuery or other 3rd-party-library DOM manipulation logic goes here

For details on this, also check out this blog post.


Used to correctly integrate 3rd-party-library callbacks into Ember. That way event handlers can be wrapped and batched in run loops, which makes their execution more efficient:

jQuery(window).on('resize', Ember.run.bind(this, this.handleResize));

... is a short version for:

var that = this;
jQuery(window).on('resize', function(){


When observing the @each property of a collection I noticed that on loading the app, the observing function was executed each time one item of the collection was loaded. This seems like too much extra work since one execution when all the items are loaded should be sufficient when the app is started. Also I noticed that the execution had a weird "off by one" problem, meaning that the observing function would be executed the first time when the first item of the collection was actually "not filled with data" yet. This results in the problem that the last execution of the observer function, triggered when the last collection item is supposed to have loaded, is also fired when the data of that item is not available yet... So for example when you want to update a view with the data of a collection and you need to do some calculations in order to do that, the last item is left out, because the data was not available at the moment when the observing function was fired. Admittedly the problem may lie deeper within my Ember app and I don't know if this is a general problem, but...

The below pattern batches the execution of the observer function, so that it only gets executed once within one loop. This reduces the amount of times the function is executed when first loading the data of a collection, although it does not guarantee that it only gets executed once. But the biggest optimization is that it also resolves my "off by one" problem:

_updateMyView: function () {
  // perform updates here...

updateMyView: function () {
  Ember.run.once(this, '_updateMyView');

To gain a quick overview over what all the methods offered by the Ember.run namespace do, I recommend taking a look at this high level overview.

Ember run loop and tests

This is an important subject. Technically you should not have to call any Ember.run methods directly from your tests. In practice, this may be different. You should be well aware of the side effects you can cause though when doing so (for example calling Ember.run.later will break the andThen method and using run.schedule, run.scheduleOnce or run.once at a time when currently no run loop is running will throw an error...).

On asynchronous side-effects in testing also check out this guide.

The End

That's it already. I recommend taking a look at the Ember run loop FAQs. If you want to do some further reading, check out the Ember runloop handbook and my blog post about the implementation of the Ember Run Loop.