Learn implicits: Type classes

Spray-json as an example of how type classes work

Posted by Jorge Montero, Jessica Kerr on September 23, 2015

In this series on Scala implicits, we have seen some, everyday, uses of implicits. One crucial pattern combines these techniques: Type classes.

Caution: please excuse the name. “Type classes” resembles neither types nor classes in a way useful for understanding. The pattern is called “type classes” for historical reasons.

Type classes extend functionality of classes without actually changing them, and with full type safety. This pattern is often used to add functionality to classes that we do not control. It’s also useful for cross-cutting concerns. Serialization is a cross-cutting concern, and spray-json provides a practical example of the type class pattern. Let’s see how it implements JSON serialization/deserialization for built-in classes.

The documentation says that any object can have the .toJson method if we add a couple of imports.

import spray.json._
import DefaultJsonProtocol._
The two imports add some implicits to the compiler’s magic hat.

the magic hat: implicit declarations go in, implicit parameter values come out

import spray.json._ brings in everything in the spray json package object, including:

implicit def pimpAny[T](any: T) = new PimpedAny(any) 
private[json] class PimpedAny[T](any: T) {
    def toJson(implicit writer: JsonWriter[T]): JsValue = writer.write(any)
}

After the last few articles, we are ready for this. The first line is a view that turns anything into a PimpedAny. Now every object in the world implements toJson. We warned against this kind of breadth in views, but here we are safe from surprises: the only way to activate this view is by calling .toJson. The intermediate class is invisible; you never even have to see its terrible name. The useful type is the return value of .toJson.

Calling toJson transforms an object into a JsValue, a representation of JSON data. Two methods on JsValue, prettyPrint and compactPrint, return a String we can transmit or save.

Can we now serialize every single object, just like that? No. Not that easy. Here’s the declaration again:

def toJson(implicit writer: JsonWriter[T]): JsValue
This toJson method takes an implicit parameter, a JsonWriter of T. So for any type T we want to convert to Json, there must be a JsonWriter[T], and it must be in the magic hat, in scope where toJson is called.

What is a JsonWriter[T], and where would the compiler find one?

JsonWriter is a trait with a single method, write.

trait JsonWriter[T] {
  def write(obj: T): JsValue
}
spray-json defines this trait, along with JsonReader for deserialization, and JsonFormat for both together. JsonFormat is the one we create, typically. Spray-json has built-in JsonFormat implementations for many common types; these lurk in DefaultJsonProtocol. We bring all of them into implicit scope when we import DefaultJsonProtocol._. It’s those JsonFormats that know how to serialize and deserialize JSON.

For instance, there is an implicit JsonFormat[String]. In type class parlance, “There is an instance of the JsonFormat type class for String.” We can use it like this:

import spray.json._
import DefaultJsonProtocol._ 
val pony = "Fluttershy"
val json = pony.toJson
The implicits resolve to:
 
val pony = "Fluttershy"
val json = new PimpedAny[String](pony).toJson(DefaultJsonProtocol.StringJsonFormat)

This desugared syntax looks like serialization in a language without implicits.

This use of the type class pattern adds a whole feature (serialization) to any class we want, in a generic way, without changing the classes. All the usual types (String, Int, Seq, Map, Option, etc) have serialization code in DefaultJsonFormat.

the magic hat: importing DefaultJsonProtocol puts a JsonFormat of String in the hat

For our own class T, we make .toJson work when we define an implicit val of type JsonFormat[T]. This is called “providing an instance of the JsonFormat type class for T.” We can write these by hand, or use helper methods from spray-json. There’s even a project that makes the compiler generate it all for case classes; the details are way outside the scope of this post.

Here’s the kicker: when we make a JsonFormat[MyClass], we get more than serialization/deserialization for MyClass. We can now call toJson on MyClass, Seq[MyClass], on Map[String,MyClass], on Option[Map[MyClass,List[MyClass]]] – without writing any extra code!

This is the killer feature of the type class pattern: it composes. One generic definition of JsonFormat[List[T]] means a List of any JsonFormat-able T is also JsonFormat-able. T could be String, Int, Long, MyClass – you name it, if we can format it, we can also format Lists of it. Here’s the trick: instead of an implicit val for JsonFormat of List, there is an implicit def in DefaultJsonFormat:

    implicit def listFormat[T : JsonFormat] = new RootJsonFormat[List[T]] {
      def write(list: List[T]) = ..
      def read(value: JsValue): List[T] = ..
    }

What is this doing? First, we have to understand some new syntax: inside the type parameter, there is a colon, followed by a type class. This is called Context Bounds (good luck finding the documentation without knowing this special name). This is shorthand for “a type T such that there exists in the magic hat a JsonFormat[T]”. The context-bounds notation above expands to:

    implicit def listFormat[T](implicit _ : JsonFormat[T]) = new RootJsonFormat[List[T]] {
      def write(list: List[T]) = ..
      def read(value: JsValue): List[T] = ..
    }

The implicit parameter ensures that the write function inside listFormat will be able to call .toJson on the elements in the List.

This implicit def does not work the same way as a view, which converts a single type to another. Instead, it is a supplier of implicit values. It can give the compiler a JsonFormat[List[T]],as long as the compiler supplies a JsonFormat[T].

the magic hat: JsonDefaultProtocol puts in a function that turns an implicit JsonFormat of T into a JsonFormat of Seq of T

One definition composes with any other JsonFormats in the magic hat. The compiler calls as many of these implicit functions, as many times as needed, to produce the implicit parameter it desperately desires.

the magic hat: to satisfy the implicit parameter of type JsonFormat of Seq of T, the magic hat uses both these values

This works on types as complicated as we want. Let’s say we want to serialize an Option[Map[String,List[Int]]]:

import spray.json._
import DefaultJsonProtocol._
val a:Option[Map[String,List[Int]]] = Some(Map("Applejack" -> List(1,2,3,4),
                                               "Fluttershy" -> List(2,4,6,8))) 
val json = a.toJson
println(json.prettyPrint)

The compiler uses implicit functions for Option, Map, and List, along with implicit vals for String and Int, to compose a JsonFormat[Option[Map[String,List[Int]]]]. That gets passed into .toJson, and only then does serialization occur. If we use the JsonFormats explicitly, the code above becomes:

import spray.json._
import spray.json.{DefaultJsonProtocol => Djp}
val a:Option[Map[String,List[Int]]] = Some(Map("Applejack" -> List(1,2,3,4),
                                               "Fluttershy" -> List(2,4,6,8)))
val json = new PimpedAny[Option[Map[String,List[Int]]]](a).toJson(Djp.optionFormat(
            Djp.mapFormat(
                Djp.StringJsonFormat,
                Djp.listFormat(
                    Djp.IntJsonFormat
                )
            )))
println(json.prettyPrint)

Whew, that’s a lot of magic. The compiler does all that composition for us. This property of implicit parameters makes the type class pattern very useful.

That much magic also means it’s hard to understand. While you’ll rarely need to create your own types in the style of JsonFormat, you’ll often want to create new type class instances, such as JsonFormat[MyClass]. Other times you need to find the right ones to import. Either way, familiarity with the pattern is essential when using spray-json and many other libraries.

Spray-routing, a library for writing RESTful services, uses this pattern for a lot of things, including returning data, and to avoid some pitfalls of method overloading. They call it the ‘magnet pattern’ and try to get you to read a post much, much longer than this one. Ultimately it’s the same pattern, used for different properties.

In some ways, the type class pattern is the culmination of Scala’s implicit feature. If this post makes sense, then you’re well on your way to Scala mastery.

posted on September 23, 2015 by
Jorge Montero
Jessica Kerr