Polymorphism in Elixir: Protocols vs. Behaviours

Ah, behaviours and protocols - those mysterious beasts. As a newcomer to Elixir you don’t really need to deal with them, and you’ll be far too busy wrapping your mind around syntax, functional programming and processes to care much anyway.

Once you’ve got the basics down though, it’s important to understand how these two approaches work together, since they are the foundation of polymorphism in Elixir and datatype polymorphism is one of the major advantages Elixir holds over its cousin, Erlang.

Polymorphism: the condition of occurring in several different forms. (Polymorphism as defined by the magic Google dictionary)

Here are the major differences between protocols and behaviours in Elixir:

Protocols Behaviours
Apply to data structures Apply to modules
Specify new implementations of existing functions for new datatypes Specify a public spec/contract for modules to implement
“Here’s my datatype, it can X” “Here’s my module, it implements Y”
Exclusive to Elixir Provided by the Erlang runtime

Let’s dive into some examples.

Protocols

We’re going to create a new protocol called Nameable. This protocol defines some useful methods dealing with things that have one or more names.

defprotocol Nameable do
  # The typespec is optional but useful for implementers of the protocol.
  @type name :: String.t

  @spec first_name(t) :: name
  def first_name(nameable)

  @spec last_name(t) :: name
  def last_name(nameable)

  @spec full_name(t) :: name
  def full_name(nameable)

  @spec number_of_names(t) :: integer
  def number_of_names(nameable)
end

Now we’re going to create a struct called %Person that implements Nameable. The beauty of protocols is that we didn’t need to know anything about %Person when we defined the protocol. The protocol is extensible. So you can define a protocol once and any number of unknown future data structures can implement it.

defmodule Person do
  defstruct [:names]
end

defimpl Nameable, for: Person do
  def first_name(person), do: List.first(person.names)
  def last_name(person),  do: List.last(person.names)
  def full_name(person),  do: Enum.join(person.names, " ")
  def number_of_names(person), do: Enum.count(person.names)
end

We can call these methods on the Nameable protocol just like any other ordinary module. We can supply as an argument anything that implements Nameable.

iex > linus = %Person{names: ~w(Linus Torvalds)}
=> %Person{names: ["Linus", "Torvalds"]}
iex > Nameable.full_name(linus)
=> "Linus Torvalds"
iex > eric = %Person{names: ~w(Eric S Raymond)}
=> %Person{names: ["Eric", "S", "Raymond"]}
iex > Nameable.number_of_names(eric)
=> 3

Now let’s say we want to include dogs in our application. Dogs can have a name too, but they only have one.

defmodule Dog do
  defstruct [:name]
end

What happens if we try to pass this struct to our Nameable module?

iex > my_dog = %Dog{name: "Max"}
=> %Dog{name: "Max"}
iex > Nameable.full_name(my_dog)
=> (Protocol.UndefinedError) protocol Nameable not implemented for %Dog{name: "Max"}

Kaboom! Each struct requires it’s own implementation of a protocol. So before we can use it, we have to implement the Nameable protocol for our %Dog struct.

defimpl Nameable, for: Dog do
  def first_name(dog), do: dog.name
  def last_name(_dog), do: "" # Dogs don't have a last name
  def full_name(dog),  do: dog.name
  def number_of_names(_dog), do: 1 # Dogs always only have one name
end

Let’s see if that works.

iex > my_dog = %Dog{name: "Max"}
=> %Dog{name: "Max"}
iex > Nameable.full_name(my_dog)
"Max"

Hooray! Now we can pass either a %Dog or a %Person to Nameable.full_name and it will work. We can go on to create as many other structs as we like and write implementations for those to work with Nameable, or write implementations for existing Elixir data structures like lists or maps.

You might wonder at this point what happens if you specify an implementation but don’t actually write all (or any) of the required functions. In this case Elixir will emit a compiler warning, and if you try to call one of the missing functions you’ll see an undefined function error.

So much for protocols. Now what about behaviours?

Behaviours

Where protocols allow for us to mix and match new data structures, behaviours let us mix and match modules. In other words, behaviours are pluggable backends, or interfaces.

Let’s imagine we want to be able to serialize some Elixir data into various formats. We might want to serialize it into JSON, or XML, native code literals or some other unknown format. It would be nice if the code that wants to serialize the data didn’t need to know the implementations of every serializer, and instead could deal with some idealized abstract version of it instead. This way we can hide the implementation detail from the calling code and make our backends pluggable.

# we want to be able to call all serializers in the same way
SomeArbitrarySerializer.serialize(my_data)

For this to work, both the caller of this abstract interface and the implementer of specific backends both need to agree on a “contract” that defines a clear interface so that certain function calls (callbacks) are always guaranteed to work. That’s where behaviours come in.

Here’s an example:

# Any implementation of this behaviour _must_ define the `serialize` callback.
defmodule Serializer do
  @callback serialize(any) :: String.t
end

It’s important to understand that this has not yet defined any function called Serializer.serialize/1. Instead, @callback is a special macro reserved by the Elixir compiler. Behind the scenes it calls Kernel.Typespec.defcallback/1 to register the callback type, then sets the @callback module attribute in a way that the Erlang virtual machine can understand.

Any modules that implement the Serializer behaviour must do two things:

  • Explicitly state they wish to do so by using: @behaviour Serializer.
  • Implement serialize/1 which takes any term and returns a string.

With this guarantee, we can write any module we like that implements the Serializer behaviour and we know it will work with all code that expects a Serializer.

# ❌ incorrect implementation of Serializer
defmodule JSONSerializer do
  @behaviour Serializer
end

# => warning: undefined behaviour function serialize/1 (for behaviour Serializer)

Oops! We forgot to implement the required serialize/1 function. Luckily the Elixir compiler has our back - if we try to compile the code above, we see a warning message.

Let’s try that again:

# ✅ correct implementation of Serializer
defmodule JSONSerializer do
  @behaviour Serializer

  def serialize(term) do
    do_some_magic(term)
  end
end

Great. It worked!

@behaviour might seem like a magic incantation, but it’s really just another ordinary module attribute that’s passed through to Erlang, which has its own callbacks/behaviour implementation.

This pattern can be used in combination with defoverridable to create default implementations with customisable behaviour. Let’s take a look at how this works in the case of GenServer.

defmodule GenServer do
  defmacro __using__(_) do
    quote do
      @behaviour GenServer
      def init(...) do ... end
      def terminate(..., ...) do ... end
      def code_change(..., ..., ...) do ... end
      defoverridable init: 1, terminate: 2, code_change: 3
    end
  end
end

defmodule MyStatefulModule do
  use GenServer

  # We can optionally override the default implementations of init/1,
  # terminate/2 etc here
end

When we create a new module and use GenServer, our module conforms to the behaviour of GenServer and also gets default implementations to boot. Kind of like inheriting from an abstract class in Ruby, but more explicit.

So there you have it! Protocols and Behaviours in Elixir.

BONUS: Proposed extension to behaviours in Elixir 1.5

The pattern shown in the GenServer example above is so common that a new syntax has been proposed to make it easier to use:

defmodule GenServer do
  defmacro __using__(_) do
    quote do
      @behaviour GenServer
      def init(...) do ... end
      def terminate(..., ...) do ... end
      def code_change(..., ..., ...) do ... end
      defoverridable GenServer ### PROPOSED SYNTAX FOR ELIXIR 1.5 ###
    end
  end
end

defmodule MyStatefulModule do
  use GenServer

  @impl GenServer ### PROPOSED SYNTAX FOR ELIXIR 1.5 ###
  # This declaration causes the Elixir compiler to check that the handle_call/3
  # function is a part of the GenServer behaviour, and warns if not. It's a
  # helpful tool to aid developers in correctly implementing behaviours.
  def handle_call(message, from, state) do
    ...
  end
end

I’m currently working on implementing this new syntax for Elixir 1.5, if you’re interested you can follow my progress here.

How to load a file into IEx

I’m making this little post because I’ve googled for this at least 5 times now and none of the results are ever what I’m looking for.

If you want to LOAD a file into IEx, do it like this:

iex> import_file("relative/path/to/file.ex")

If you want to RELOAD a module after it’s already been loaded from a file, do it like this:

iex> r MyModule

That’s it!

Will Elixir/Phoenix make Ruby on Rails obsolete?

In this article I want to compare Ruby vs. Elixir to see how they stack up, and explore how this new language might affect the Ruby ecosystem. But to start with, a little background on Ruby and why it’s so great.

In 2005 DHH released a video entitled “How to build a blog engine in 15 minutes”.

It showed off a little-known framework called Rails, which allowed for unheard-of speed in developing the type of CRUD web applications common throughout the 2000s.

Rails was almost magical in the how it could do so much with such a tiny amount of code, and even today Rails is still probably the best choice for fast iteration in web development.

Most of this magic was built on the back of Ruby’s extremely powerful metaprogramming abstractions. Ruby was my first love and will always occupy a special place in my heart. It is one of the most metaprogrammable languages ever designed and that shines through as it’s core strength.

This metaprogramming power allows DSLs to be constructed in a way that complex operations can be written in a way that reads almost like english:

# RSpec
login_as learner
visit course_path(course)
find('.join').click
expect(page).to have_content('Thanks for joining!')

Even in 2017 Ruby is still one of the best options as a general purpose scripting language and for many types of web application. There are a lot of developers around who are fluent in Ruby and they tend to be of high quality.

However, Ruby does have a number of shortcomings when applied to modern web developent:

Runtime speed - The Ruby runtime has never been a speed demon (although it’s slowly getting better). There are design choices in the language that make high performance difficult (ahem mutable strings) and as heroic as the efforts of Matz and his team are, they simply don’t have the developer muscle that goes into optimising something like the JVM or GoLang.

The practical upshot of this is a floor of around ~100ms for a typical web request using Rails, and test suites that can become painfully slow for large applications.

Boot speed/memory consumption - Primarily a Rails problem, as the number of gems goes up the boot speed of a Rails app gets slower and slower. It is not uncommon to see a single Rails process using 300-400MB of memory, notwithstanding any leaks - which become more and more likely as more third party gems are included in a project.

Concurrency (not parallelism!) - Ruby is pretty bad at being concurrent. The native tool for handling concurrency is the Ruby thread, which is one of the hardest concurrency primitives to reason about and requires painful messing about with mutexes and semaphores to handle safely.

Scalability (parallelism) - While it does have threads, native Ruby code is always limited by the GIL so that true parallelism within a process is not possible. The rule is, if it’s Ruby code then only one line can execute at a time within a given process. The way to scale Ruby is to throw a lot of processes at it, and a lot of servers. But processes are costly and this is an expensive way to scale.

Despite these issues, Ruby has remained popular for the primary reason that it is beautiful to work with, and is so malleable that it allows for fast iteration especially for startups who don’t quite know what they are building yet.

There are many languages that solve the performance problems listed above, but none so far have matched the beauty and expressiveness of Ruby. Elixir promises to change that.

Elixir promises to deliver the elegance and power of Ruby, built on the reliability, speed and scalability of the Erlang virtual machine. The stated design goals are compatibility (with Erlang), productivity and extensibility.

As it’s designers are keen to point out, Elixir is not Ruby for the EVM. The syntax may appear to have a passing familiarity, but many core concepts in the language are quite different.

At its core, Elixir is secretly a LISP pretending to be something else. This allows for a particularly powerful form of metaprogramming via macros, that allows you to rewrite the AST before it is evaluated. It is also a functional language with no mutable state. This promises to greatly improve long term maintainability as compared to OO languages like Ruby where data structures are mutable by default.

Many of Elixir’s features are not strictly part of the language, but rather are built using the same macro system that is also available to you as a developer. It is highly extensible in a similar way to Clojure.

Elixir’s greatest strength is the Erlang Virtual Machine, which along with Erlang’s OTP framework provides a base for building systems that run forever, self-heal and scale as large as you need them.

Among other things the Erlang platform also promises:

And I suppose this article wouldn’t be complete without some arbitrary benchmark numbers comparing Elixir/Phoenix to Ruby/Rails:

Phoenix - 43,063 req/s, 2.82ms avg latency Rails - 3,274 req/s, 17.25ms avg latency

That’s a 13.5x performance increase, which means you need 13.5x LESS machines to do the same job, and everybody gets faster page loading to boot.

So will Elixir make Ruby obsolete?

Yes, and no.

Ruby isn’t going anywhere and I will continue to use it myself for many client projects. I worked on a small ecommerce store recently handling a couple of hundred users a day. Ruby/Rails was a great choice for this, the established ecosystem allowed me to leverage existing libraries to get up and running really fast. It will never need to run on more than one server and will be easy to maintain for other developers.

Ruby is great for projects that don’t need to scale or handle realtime updates and for hacking on prototypes to get something up really fast. It has a vibrant community and a fantastic ecosystem of tools. Even the UK government is using Ruby for it’s projects which just goes to show how mainstream it has become.

However, for apps requiring high scalability and reliability as well as malleability that allows for fast iteration, I think we will start to see many successful startups and some of the more adventurous Rubyists switching to Elixir as their secret weapon.

Perhaps a better question would be, what is the best framework for a scalable app allowing for fast iteration, receiving high traffic and sending realtime updates?

As of 2017, that looks like Elixir/Phoenix.

A Simple, Tiny Flashcard App In Swift

I had to read out my postcode to someone on the phone recently and the conversation went something like this:

Me: FO0 8AR

Her: Was that F for ferret or S for sierra?

Me: Ferret

Her: OK, so that’s FO0 8ER

Me: No, that’s AR

Her: AR0 ?

Me: facepalm

So I decided that it might be a good idea to finally learn the NATO phonetic alphabet. That’s Alfa Bravo Charlie all that jazz.

Instead of actually learning it like a normal person, I thought hang on maybe I can turn this into an opportunity. Maybe I can write a flashcard app for the iPhone to learn it.

And that longwinded story is how I build my first Swift app:

Flashcard App screenshot

I’ve had my eye on Swift for some time. I like it mostly because it means I don’t have to write Objective C, which is so ugly it makes me throw up a little bit in my mouth every time I have to look at it. Those square brackets… shudders.

Syntax jokes aside, I’m also a big fan of anything that allows me to write more code and less boilerplate, and Swift seems to fit the bill.

This app turned out to be super simple, the source code is here if you feel like checking it out.

It can even read the correct pronunciation out for you. And it worked - next time I have to read out a postcode or email address over the phone I’ll be prepared.

How to implement token-based authentication between Ember.js and Rails using Devise backend

This post is more as a helpful reminder to myself, because this is the third time around I’ve had to implement this.

Here is the Rails code necessary in order to get seamless authentication working using ember-simple-auth’s Devise backend.

RAILS

# app/controllers/api_controller.rb
class ApiController < ActionController::Base
  attr_reader :current_user
  respond_to :json
  before_action :authenticate_user_from_token!
  protect_from_forgery with: :null_session

  private

  def authenticate_user_from_token!
    authenticated = authenticate_with_http_token do |user_token, options|
      user_email = options[:email].presence
      user       = user_email && User.find_by_email(user_email)

      if user && Devise.secure_compare(user.authentication_token, user_token)
        @current_user = user
        sign_in user, store: false
      else
        render json: { errors: ['Invalid authorization.'] }, status: :unauthorized
      end
    end

    unless authenticated
      render json: { errors: ['No authorization provided.'] }, status: :unauthorized
    end
  end
end
# app/controllers/api/sessions_controller.rb
class Api::SessionsController < Devise::SessionsController
  respond_to :json

  # POST /api/users/sign_in
  def create
    respond_to do |format|
      format.json do
        self.resource = warden.authenticate!(auth_options)
        data = {
          user_id: resource.id,
          token: resource.authentication_token,
          email: resource.email
        }
        render json: data, status: :created
      end
    end
  end
end
# app/models/user.rb
class User < ActiveRecord::Base
  before_save :ensure_authentication_token

  devise :database_authenticatable, :recoverable, :trackable, :validatable

  # Generate a token for this user if one does not already exist
  def ensure_authentication_token
    if authentication_token.blank?
      self.authentication_token = generate_authentication_token
    end
  end

  # Identical to above except it saves the user
  def ensure_authentication_token!
    ensure_authentication_token
    save
  end

  # Forces a new authentication token to be generated for this user and saves it
  # to the database
  def reset_authentication_token!
    self.authentication_token = generate_authentication_token
    save
  end

  private

  def generate_authentication_token
    loop do
      token = Devise.friendly_token
      break token unless User.find_by(authentication_token: token)
    end
  end
end

# config/routes.rb
Rails.application.routes.draw do
  namespace 'api', constraints: { format: 'json' } do
    devise_for :users, singular: 'user', module: 'api', controllers: { sessions: 'api/sessions' }
  end
end

And here is the Ember code counterpart:

EMBER

// app/adapters/application.js (if using Active Model Adapter)
import ActiveModelAdapter from 'active-model-adapter';
import DataAdapterMixin from 'ember-simple-auth/mixins/data-adapter-mixin';

export default ActiveModelAdapter.extend(DataAdapterMixin, {
  namespace: 'api',
  authorizer: 'authorizer:devise'
});

// app/authenticators/devise.js
import DeviseAuthenticator from 'ember-simple-auth/authenticators/devise';

export default DeviseAuthenticator.extend({
  serverTokenEndpoint: '/api/users/sign_in'
});
// app/authorizors/devise.js
import DeviseAuthorizer from 'ember-simple-auth/authorizers/devise';

export default DeviseAuthorizer.extend();

// app/controllers/application.js
import Ember from 'ember';

export default Ember.Controller.extend({
  session: Ember.inject.service('session'),

  actions: {
    invalidateSession() {
      this.get('session').invalidate();
    }
  }
});

// app/controllers/login.js
import Ember from 'ember';

export default Ember.Controller.extend({
  session: Ember.inject.service('session'),

  actions: {
    authenticate() {
      this.get('session').authenticate('authenticator:devise', EMAIL, PASSWORD).catch((reason) => {
        this.set('errorMessage', reason.error);
      });
    }
  }
});
// app/routes/some_authenticated_route
import Ember from 'ember';
import AuthenticatedRouteMixin from 'ember-simple-auth/mixins/authenticated-route-mixin';

export default Ember.Route.extend(AuthenticatedRouteMixin);
// app/routes/some_unauthenticated_route
import Ember from 'ember';
import UnauthenticatedRouteMixin from 'ember-simple-auth/mixins/unauthenticated-route-mixin';

export default Ember.Route.extend(UnauthenticatedRouteMixin);
// config/environment.js
module.exports = function(environment) {
  var ENV = {};

  ENV['ember-simple-auth'] = {
    authorizer: 'authorizer:devise',
    crossOriginWhitelist: ['*']
  }

  return ENV;
};