Learn implicits: Scala Futures

Exploring implicit parameters using Scala Futures

Posted by Jorge Montero on June 15, 2015

Implicits are difficult to understand because they have many different uses. In an earlier post, we looked at implicit parameters and type tags. Now, we’ll take a look at another usage pattern every Scala programmer sees: implicits as an alternative to passing the same argument over and over. Scala Futures use implicit parameters in this way.

There is much said about futures elsewhere. The gist of it is that a Future contains a value that may or may not have been computed yet. Futures let us spin off work into other threads, add more operations that should be performed on the result, define what should happen after failure, and (if we really must) wait for the operation to complete.

Everything we do asynchronously happens on some other thread. Creating a future, adding operations after success, adding failure handling – in each case, we need to tell it what thread to run on. The futures library lets us specify this using implicit parameters.

For illustration, let’s define some data types and a fake Data Access Object with the following operations:

case class Employee(id:Int, name:String)
case class Role(name:String, department :String)
case class EmployeeWithRole(id :Int, name :String, role :Role)

trait EmployeeGrabberBabber {
  def rawEmployee(id :Int) :Employee
  def rawRole(e :Employee) :Role
  def employee(id: Int)(implicit e :ExecutionContext) :Future[Employee]
  def role(employee :Employee)(implicit e :ExecutionContext) :Future[Role]
}

I have an implementation for that trait, but it’s not that important.

The first two methods do synchronous IO: Whenever we call them, our thread will patiently wait until we get the requested information, leaving our thread blocked. The second pair uses Futures: employee returns a Future[Employee], which will eventually provide an Employee, or error out. We do not wait for the operation to complete before returning; the caller gets the power of deciding whether to wait, whether to attach more actions, whether to handle errors.

With the first set of methods, if we wanted to get an Employee, and then get their Role, and then combine that into an EmployeeWithRole:

val employee = grabber.rawEmployee(100)
val role = grabber.rawRole(employee)
val bigEmployee = EmployeeWithRole(employee.id,employee.name,role)

This is imperative programming. It holds up the calling thread until the entire calculation is made. You probably don’t want to do this in a web application or in an event thread in a native UI toolkit.

In contrast, the asynchronous methods return instantly. We can keep right on defining what to do with the value – inside the context of the Future.

val bigEmployee: Future[EmployeeWithRole] =
  grabber.employee(100).flatMap { e =>
    grabber.role(e).map { r =>
      EmployeeWithRole(e.id,e.name,r)
    }
  }

This code puts both operations together without blocking.

Except, the code above does not work on its own. Remember those implicit parameters defined above in EmployeeGrabberBabber?

def employee(id :Int)(implicit e :ExecutionContext) :Future[Employee]
def role(employee :Employee)(implicit e: ExecutionContext) : Future[Role]

We did not define them, as the compiler helpfully reminds us.

Error: Cannot find an implicit ExecutionContext. You might pass
an (implicit ec: ExecutionContext) parameter to your method
or import scala.concurrent.ExecutionContext.Implicits.global.
    grabber.employee(100).flatMap { e =>
                    ^

That’s a useful error message! While we could just add the import, we’d not learn much from doing that, so let’s dig deeper.

Creating a Future starts an asynchronous operation on another thread. The ExecutionContext provides the thread pool that Future will use. Different execution contexts wrap different thread pools, with different properties. The one that the errors suggest, Scala’s global execution context, suits us for now.

The Future-creating methods declare two parameter lists. We can be perfectly clear about which ExecutionContext each should use by passing it explicitly:

val ec =  scala.concurrent.ExecutionContext.Implicits.global
val bigEmployee: Future[EmployeeWithRole] =
  grabber.employee(100)(ec).flatMap { e =>
    grabber.role(e)(ec).map { r =>
      EmployeeWithRole(e.id,e.name,r)
    }
  }

But that doesn’t work either!


Error: Cannot find an implicit ExecutionContext. You might pass
an (implicit ec: ExecutionContext) parameter to your method
or import scala.concurrent.ExecutionContext.Implicits.global.
      grabber.employee(100)(ec).flatMap { e =>
                                      ^

The flatmap method on Future also wants an ExecutionContext! We gave that Future another operation to perform, and it needs a thread pool to run that on. Future.map has the same problem, so pass the ExecutionContext there, too. This is getting tedious.

val ec =  scala.concurrent.ExecutionContext.Implicits.global
val bigEmployee: Future[EmployeeWithRole] =
  grabber.employee(100)(ec).flatMap { e =>
    grabber.role(e)(ec).map { r =>
      EmployeeWithRole(e.id,e.name,r)
    }(ec)
  }(ec)

So now it’s happy, and it’s very clear which ExecutionContext every operation runs in. But I’m not happy, because it’s repetitive and cluttered. It gets even more cluttered, the more things we call on a Future. Just look at these signatures from the Future API:

    def onSuccess[U](pf : PartialFunction[T, U])(implicit executor : ExecutionContext)
    def onFailure[U](callback : PartialFunction[Throwable, U])(implicit executor : ExecutionContext) 
    def onComplete[U](func : Try[T] => U)(implicit executor : ExecutionContext)
    def foreach[U](f : T => U)(implicit executor : ExecutionContext) 
    def transform[S](s : T => S, f : Throwable => Throwable)(implicit executor : ExecutionContext) : Future[S] 
    def map[S](f : T => S)(implicit executor : ExecutionContext) : Future[S]
    def flatMap[S](f : T => Future[S])(implicit executor : ExecutionContext) : Future[S] 
    def filter(pred : T => Boolean)(implicit executor : ExecutionContext) : Future[T] 
    def withFilter(p : T => Boolean)(implicit executor : ExecutionContext) : Future[T]
    def collect[S](pf : T => S)(implicit executor : ExecutionContext) : Future[S] 
    def recover[U >: T](pf : PartialFunction[Throwable, U])(implicit executor : ExecutionContext) : Future[U] 
    def recoverWith[U >: T](pf : PartialFunction[Throwable, Future[U]])(implicit executor : ExecutionContext) : Future[U]
    def andThen[U](pf : PartialFunction[Try[T], U])(implicit executor : ExecutionContext) : Future[T]

ExecutionContexts everywhere. They’re important, and sometimes we need to be specific about where each operation should run, but the common case is that they can all run in the same pool. It is tedious, cluttered, and error-prone to repeat that same bit of information over and over.

When a function has multiple parameter lists, Scala permits the implicit keyword at the beginning of the last parameter list.

  def employee(id:Int)(implicit e:ExecutionContext) :Future[Employee]

This instructs the Scala compiler to pull those arguments out of its magic hat, instead of requiring them to be passed each time.

Trixie's magic hat

If it’s ok to use all those threads in the same pool, then we can supply the execution context implicitly, which puts it in the magic hat:

implicit val ec: ExecutionContext = scala.concurrent.ExecutionContext.Implicits.global
val bigEmployee: Future[EmployeeWithRole] =
  grabber.employee(100).flatMap { e =>
    grabber.role(e).map { r =>
      EmployeeWithRole(e.id,e.name,r) 
    }
  }

Here, the implicit keyword is serving a different (but related) purpose than it did in the parameter list. The implicit val goes into the compiler’s magic hat for as long as that value is in scope. The compiler can use it anywhere it needs to supply an implicit parameter of type ExecutionContext, over and over.

This lets us use for-comprehensions too, which take the place of flatmap and map:

implicit val ec =  scala.concurrent.ExecutionContext.Implicits.global
val employeeWithRole = for { employee <- grabber.employee(200L)
                            role <- grabber.role(employee) } 
                            yield EmployeeWithRole(employee.id,employee.name,role)   

Much cleaner.

This implicit-parameter-supplying feature only works if there is exactly one value of the needed type in the compiler’s magic hat when the method that declares the implicit parameter is called. If none are available, you get that “Cannot find an implicit” compile error. If more than one are available, you get an “ambiguous implicit” error

Of the many ways to get values into the magic hat, three make sense for the ExecutionContext. The simplest is to declare an implicit val, as above. It stays in the magic hat as long as the val is in scope. This is common inside an Akka actor:

class SomeActor extends Actor {
  implicit val ec: ExecutionContext = context.dispatcher
}

Another is to declare an implicit parameter: that value is in the magic hat inside the method. This is good practice, because it lets the caller decide what to use. For example, this e is available to the Future constructor:

  def employee(id:Int)(implicit e :ExecutionContext) :Future[Employee] = Future{...}

Finally, the most common way to punt on the selection of the execution context is to import an implicit val in file scope:

import scala.concurrent.ExecutionContext.Implicits.global

Inside scala.concurrent.ExecutionContext.Implicits, global is an implicit val, so into the magic hat it goes. Adding this to the top of the file chooses the default execution context for asynchronous operations.

Any way you do it, one implicit value declaration saves repetition, providing a default per scope while allowing an override at each function call. The authors of the Future library put the ExecutionContext into an implicit parameter for this reason: it’s common to repeat the same value, common to pass it down through various methods, and essential that it be explicitly passed sometimes, at the caller’s discretion. In this way, Scala lets library designers keep the interface clean and flexible at the same time.

posted on June 15, 2015 by
Jorge Montero