Getting to grips with Elixir’s GenServer

I plunged bravely into the world of Elixir. And there she was, waiting for me to open my eyes.. the GenServer. Her spindly process tentacles enclosed around me at lightning speed and pulled me deeper into the abyss all the while calling back into the darkness, recruiting her army of helper processes.

I know you’re envisioning some form of monster, but I promise it’s not that alarming ;). Instead of evoking a monster when visualising the mighty GenServer, an easier alternative is imagining Jennifer Aniston circa Friends 1994 on the phone acting as the server relaying snappy sassiness to Ross, the inquiring client. This classic Rachel-Ross relationship forms our JenServer.

My descriptions of GenServer’s intricacies are directly adapted from the getting started with Elixir documentation: https://elixir-lang.org/getting-started/mix-otp/genserver.html

What is GenServer and why do I need it?

GenServer, short for generic server can receive and reply to requests the same as any other server. The GenServer behaviour consists of:

  • an interface, which is a set of functions
  • an implementation, which is the application-specific code, and
  • the container, which is a BEAM process

In plain language, it’s an out-of-the-box protocol in Elixir which forms the bones of the server in a client-server relationship. It makes our life easy by abstracting Erlang’s underlying process handling to a series of user-defined callback functions. But why do we need it?

State

To understand the essence and necessity of GenServer we first need to talk about state.

Every software application needs to maintain state. An e-commerce website needs to know how many items a user has in their cart, whether the payment has been processed, the cost of the items and so on.

Coming from an object-oriented programming (OOP) world , state was something I didn’t give too much thought. State was stored inside my objects, their behaviour was defined and encapsulated within my classes, and every object was an instance of a class.

In Elixir, everything is a process. Processes communicate via message passing. Each process runs a function which in addition to performing a calculation or operation sends messages to processes or waits on messages to arrive in their mailbox.

Built on top of Erlang’s VM, this functional world of message passing is what makes fault-tolerant apps with high concurrency possible in Elixir. When Facebook bought Whatsapp, the Erlang infrastructure was accommodating millions of concurrent processes running on a single server. https://stackoverflow.com/questions/22090229/how-did-whatsapp-achieve-2-million-connections-per-server

GenServer provides a way to manage this two-way message passing between processes. In addition to the communication protocol it defines, it allows monitoring of these processes through supervisors (processes that manage child processes and restart them if they crash).

Implementation

The idea is to define two functions:

1. A client function accessible to GenServer clients which will call…

2. An internal server function to handle the request.

In our GenServer scenario, the client is any process that invokes the client function while the server is always the process identifier (PID) or process name that we will explicitly pass as an argument to the client function.

There are 2 types of requests a GenServer handles:

  1. One that expects a response which is known as a call and is defined by the server function – handle_call. handle_call is synchronous meaning the server must reply to the request.
  2. Requests that don’t expect a response, known as casts are defined by the server function handle_cast. Handle_cast‘s are asynchronous meaning the server does not send a reply and as such the client will not wait for one.

These pair of functions handle GenServer’s functionality by message passing through pattern matching. Pattern matching is a unique and powerful feature of Elixir that took some time for me to grasp. An excellent presentation on the concept can be seen here https://www.youtube.com/watch?v=nEUHb7RJspQ .

Storing key-value pairs in a GenServer’s state

We’re going to store key-value pairs in the state of a GenServer to demonstrate the basic concepts of:

  1. Server starting and state
  2. Synchronous messages
  3. Asynchronous messages

1. Creating a mix project

mix new <projectname> --sup # sup generates a supervision tree in project structure

Inside the /lib folder is where you will write your GenServer. Make a folder called server inside /lib

file:///home/linda/Pictures/Screenshot_20200202_190734.png

Inside your server folder create a file database.ex

We are going to wrap our code inside a module named Server.Database (representing the path to our file.)

defmodule Server.Database do
  use GenServer

# Our Genserver functions 

end

2. Starting our server

GenServers must implement an init/1 function to set the server’s initial state

To start our Server.Database process we need a start_link/1 and init/1 function (the ‘/1’ is an Erlang convention specifying the “arity” or number of parameters a function permits).

def start_link(init_args) do
         GenServer.start_link(__MODULE__, :ok, init_args)
end

@impl true
def init(:ok) do
    {:ok, %{}}
end

start_link/1 starts a GenServer process linked to the current process by calling GenServer.start_link/3, which takes the arguments:

  1. __MODULE__ – the module where the callbacks will take place. __MODULE__ is syntactic sugar for the current module (Server.Database in our case)
  2. The initialization arguments – the atom :ok (passed from GenServer.start_link/3 to init/1)
  3. The initial start_link/1 parameter passed to GenServer.start_link/3 (init_args)
  4. You can also specify a number of other options, such as the :name of the server, or a :timeout parameter, giving the server a given number of milliseconds to initiate or else be terminated.

The init/1 function sets the initial state of the server as an empty map, returning a tuple with the message {:ok, %{}}. The @impl true decorator informs the compiler that our function is a callback.

To test the initialization of our GenServer, open an IEx shell using iex -S mix in the root folder of your project.

{:ok, server} = Genserver.start_link([])
{:ok, #PID<0.146.0>}

Nice! We’ve started the server by passing in an empty list as an argument to our start_link/1 function and in return we’ve received the :ok atom message from our init/1 function as well as the process ID assigned to our server. Let’s make a client and server function to check the server’s state.

3. Checking server state

# Client 
def get_state(server) do
    GenServer.call(server, {:get_state, nil})
end

# Server callback
@impl true
def handle_call({:get_state, _}, _from, state) do
    {:reply, state, state}
    end

handle_call/3 takes the parameters :get_state – an atom which pattern matches on our client get_state function, the second parameter “_” within the tuple signifies to handle_call/3 that it can pattern match on any given input, _from doesn’t care where the call comes from (denoted by a similar “_” underscore symbol. It does care about the current state of the process so it can potentially modify it and continue to relay the state between the server and client.

The handle_call/3 function must return a reply in the form of a tuple {:reply, reply, state}. The atom :reply pattern matches on an underlying GenServer reply function responsible for message delivery to the client, the actual reply in the form of state, as well as the new state (unchanged in this example) state which continues the loop of state message passing.

GenServer.call({:get_state, nil})
{:ok, {}}

Our state is empty. Time to add some data.

Adding key-value pairs

def put(server, key, value) do
GenServer.call(server, {:put, key, value})
end

@impl true
def handle_call({:put, key, value}, _from, state) do
new_state = Map.put(state, key, value)
{:reply, new_state, new_state}
end

Run r Server.Database to recompile our Server.Database module in an IEx session after tweaking the code.

iex(1)> {:ok, server} = Server.Database.start_link([])
{:ok, #PID<0.120.0>}
iex(2)> Server.Database.put(server, "norwegian_forest","morty")
%{"norwegian_forest" => "morty"}
iex(3)> Server.Database.put(server, "tabby", "mushu")           
%{"tabby" => "mushu", "norwegian_forest" => "morty"}
iex(4)> Server.Database.put(server, "british_shorthair", "mulan")
iex(5) %{
  "british-shorthair" => "mulan",
  "norwegian_forest" => "morty",
  "tabby" => "mushu"
}

A quick tip to dissolve confusion about the parameters a function takes is to use IEx’s helper function e.g.

h Map.put which outputs:

def put(map, key, val)

Puts the given value under key.

Examples
┃ iex> Map.put(%{a: 1}, :b, 2)
┃ %{a: 1, b: 2}
┃ iex> Map.put(%{a: 1, b: 2}, :a, 3)
┃ %{a: 3, b: 2} 

Here we can see that a key is passed into the map function as an :atom.

Deleting values from our map

Exciting stuff. Since we can now :put data into our GenServer’s state, the next step is to delete data from our map.

def delete(server, key) do
    GenServer.call(server, {:delete, key})
end

@impl true
def handle_call({:delete, key}, _from, state) do
    new_state = Map.delete(state, key)
    {:reply, new_state, new_state}
end
Server.Database.delete(server, "british_shorthair")
%{"norwegian_forest" => "morty", "tabby" => "mushu"}

Replacing values in our map

Let’s make this operation a cast. The server will replace the values in our map and not return the state of the map.

# Alters the value stored under key to value, but only if the entry key already exists in map.
def replace(server, key, value) do
    GenServer.call(server, {:replace, key, value})
end

@impl true
def handle_cast({:replace, key, value}, _from, state) do
    new_state = Map.replace!(state, key, value) 
    {:noreply, new_state, new_state}
end
Server.Database.replace(server, "norwegian_forest", "minnie")
%{"norwegian_forest" => "minnie", "tabby" => "mushu"}

Getting the value of a key

def get(server, key) do
    GenServer.call(server, {:get, key})
end

@impl true
def handle_call({:get, key}, _from, state) do
    {:reply, Map.get(state, key), state}
end
Server.Database.get(server, "tabby")                    
"mushu"

Our GenServer so far

defmodule Server.Database do

use GenServer

# Client API

def start_link(init_args) do
         GenServer.start_link(__MODULE__, :ok, init_args)
end

def get_state(server) do
    GenServer.call(server, {:get_state, nil})
end
    
def put(server, key, value) do
    GenServer.call(server, {:put, key, value})
end

def delete(server, key) do
    GenServer.call(server, {:delete, key})
end

def get(server, key) do
    GenServer.call(server, {:get, key})
end

def replace(server, key, value) do
    GenServer.call(server, {:replace, key, value})
end

# Server callbacks

@impl true
def init(:ok) do
    {:ok, %{}}
end

@impl true
def handle_call({:get_state, _}, _from, state) do
    {:reply, state, state}
    end

@impl true
def handle_call({:put, key, value}, _from, state) do
    new_state = Map.put(state, key, value)
    {:reply, new_state, new_state}
end

@impl true
def handle_call({:delete, key}, _from, state) do
    new_state = Map.delete(state, key)
    {:reply, new_state, new_state}
end

@impl true
def handle_call({:get, key}, _from, state) do
    {:reply, Map.get(state, key), state}
end

@impl true
def handle_call({:replace, key, value}, _from, state) do
    new_state = Map.replace!(state, key, value)
    {:reply, new_state, new_state}
end

end

We now have the bones of a GenServer and are storing our state directly inside it. This is great but could quickly become a bottleneck in the event of multiple concurrent processes trying to access our server’s state. A more effective way to store state would be in an ETS (Erlang Term Storage). An ETS is an in-memory table allowing constant access time to its data. Data is stored as tuples in dynamic ETS tables with each table being created by a process. The table exists only as long as the parent process is alive. Luckily we have supervisors to preserve our processes.

The first supervisor

Defining supervisors to monitor your GenServer processes is extremely important to ensure if your server crashes or process dies a new process will be spawned immediately. The example below is of a module-based supervisor.

defmodule Server.FirstSupervisor do
    use Supervisor

    def start_link(opts \\ []) do
        Supervisor.start_link(__MODULE__, :ok, opts)
    end

    @impl true
    def init(:ok) do
        children = [
            {Server.Database, name: :melon}
        ]
        Supervisor.init(children, strategy: :one_for_one)
    end
end

Instead of interacting with our GenServer through saving the process id in a variable named server, a better practice is to uniquely name our GenServer process in our Supervisor so we can access it anywhere in our app. I’ve given the name :melon to the process storing map in the GenServer’s state.

By calling Server.FirstSupervisor.start_link/1 the supervisor will automatically start our GenServer melon process and keep it alive. Server.FirstSupervisor.start_link/1 directly calls Server.Database.start_link(Server.Database). With supervisors you can define a number of monitoring strategies. The :one_for_one strategy only restarts the child process if it dies.

Storing GenServer state in ETS

defmodule Server.ETSserver do
  use GenServer

  @name :melon

  # start link
  def start_link(arg \\[]) do
    GenServer.start_link(__MODULE__, :ok, arg ++ [name: :melon])
  end

  def get_state(server) do
    GenServer.call(server, {:get_state, nil})
  end

  def create(server, name) do
    GenServer.cast(server, {:create, name})
  end

  def put(key, value) do
    GenServer.call(@name, {:put, key, value})
  end

  def get(key) do
    GenServer.call(@name, {:get, key})
  end

  def delete(key) do
    GenServer.call(@name, {:delete, key})
  end


  # Callbacks
  def init(:ok) do
    state = :ets.new(:ets_table, [:set, :protected, :named_table]) # set and protected are default values and can be excluded
    {:ok, state}
  end

  def handle_call({:get_state, _}, _from, state) do
    {:reply, state, state}
  end

  def handle_call({:put, key, value}, _from, state) do
    :ets.insert(:ets_table, {key, value})
    |> case do
      true ->
        {:reply, "Key=#{key} Value=#{value} inserted", state}
      false ->
        {:reply, "Key already exists", state}
    end
  end

  def handle_call({:get, key}, _from, state) do
    :ets.lookup(:ets_table, key)
    |> case do
      [] ->
        {:reply, "#{key} not found", state}
      found ->
        [term |_] = found
        {:reply, term, state}
    end
  end

  def handle_call({:delete, key}, _from, state) do
    :ets.delete(:ets_table, key)
    |> case do
      true ->
        {:reply, "#{key} was deleted", state}
      false ->
        {:reply, "#{key} not found", state}
    end
  end


defmodule GenericSupervisor do
  use Supervisor

  def start_link(opts \\[]) do
    Supervisor.start_link(__MODULE__, :ok, opts)
  end

  # name of a process can be any atom
  @impl true
  def init(:ok) do
    children = [ 
      {Server.ETSserver, name: :melon} # Supervisor calls ETSserver.start_link(ETSserver)
    ]
    Supervisor.init(children, strategy: :one_for_one)
  end
end


defmodule Server.TestETS do
  def start() do
    {:ok, pid} = GenericSupervisor.start_link([])
    GenericSupervisor.child_spec(pid)
    Server.ETSserver.put("cat", "mushu") 
    Server.ETSserver.get_state(:melon) |> IO.inspect()
    Server.ETSserver.put("dog", "dalmation") 
    Server.ETSserver.get("dog")
    Server.ETSserver.get("potato")
    Server.ETSserver.delete("dog")
  end
end

Above is an example of a slightly more evolved GenServer storing state in an ETS.

Elixir is a young project with huge potential. (Young enough to not have built-in syntax highlighting in wordpress so apologies for the haphazard code snippets!). It’s functional flow and syntax idiosyncrasies can be odd at first but it quickly becomes a joy to work with. I look forward to how this language and community evolve in the future.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.