Scala: Making implicit conversion A->B work for Option[A] -> Option[B]

Question

I'm trying to write a function which re-uses the implicit conversions which I have for Object A -> Object B when they are wrapped in an Option in a generic way so that Option[A] -> Option[B] conversions also work.

What I've come up with is:

implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] = from.map(conversion(_))

This works when I assign a Some(..) to a value but not when I assign an Option val; see the following console output:

scala> trait T
defined trait T

scala> case class Foo(i: Int) extends T
defined class Foo

scala> case class Bar(i: Int) extends T
defined class Bar

scala> implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
fromFooToBar: (f: Foo)Bar

scala> implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
fromBarToFoo: (b: Bar)Foo

scala> implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] = from.map(conversion(_))
fromOptionToOption: [A, B](from: Option[A])(implicit conversion: (A) => B)Option[B]

scala> val foo: Option[Foo] = Some(Bar(1))
foo: Option[Foo] = Some(Foo(1))
// THIS WORKS as expected

scala> val fooOpt = Some(Foo(4))
fooOpt: Some[Foo] = Some(Foo(4))

scala> val barOpt2: Option[Bar] = fooOpt
<console>:16: error: type mismatch;
 found   : Some[Foo]
 required: Option[Bar]
       val barOpt2: Option[Bar] = fooOpt
                                  ^
//THIS FAILS.

I don't really see the difference between the first and second conversion. Somehow it doesn't invoke the implicit conversion in the latter. I guess it has something to do with the type system, but I can't see how just yet. Any ideas? -Albert (I'm on scala 2.9.1)

Answer 1

Here's clue:

scala> val fooOpt: Option[Bar] = Option(Foo(1))
fooOpt: Option[Bar] = Some(Bar(1))

And another:

scala> implicit def foobar(x: String): Int = augmentString(x).toInt
foobar: (x: String)Int

scala> val y: Option[String] = Option(1)
y: Option[String] = Some(1)

scala> val y: Option[Int] = Option("1")
y: Option[Int] = Some(1)

Looks like a legitimately odd bug. I'd pop open a smaller test case and open an issue (or search for one in JIRA).

As an aside:

You could use some category theory to handle lots of different types of "Option-ish" things.

package object fun {
  trait Functor[Container[_]] {
    def fmap[A,B](x: Container[A], f: A => B): Container[B]
  }
  object Functor {
     implicit object optionFunctor extends Functor[Option] {
       override def fmap[A,B](x: Option[A], f: A => B): Option[B] = x map f
     }
     // Note: With some CanBuildFrom magic, we can support Traversables here.
  }
  implicit def liftConversion[F[_], A, B](x: F[A])(implicit f: A => B, functor: Functor[F]): F[B] = 
    functor.fmap(x,f)

}

That's a bit more advanced, as you're mapping some category theory FP onto the problem, but it's a more general solution to lift implicit conversations into containers as needed. Notice how they chain by using one implicit conversation method that takes a more limited implicit argument.

ALSO, this should make the examples work:

scala> val tmp = Option(Foo(1))
tmp: Option[Foo] = Some(Foo(1))

scala> val y: Option[Bar] = tmp
y: Option[Bar] = Some(Bar(1))

And make your usage of Some more dangerous:

scala> val tmp = Some(Foo(1))
tmp: Some[Foo] = Some(Foo(1))

scala> val y: Option[Bar] = tmp
<console>:25: error: could not find implicit value for parameter functor: fun.Functor[Some]
       val y: Option[Bar] = tmp
                            ^

That's telling you that variance is critical, and interacts with implicits. My guess is you ran into a very rare, probably hard to fix bug that can be avoided using other techniques.

Answer 2

You might not be aware of it, but there's a flag for that: -Xlog-implicits. And this is what it says:

scala> val barOpt2: Option[Bar] = fooOpt
fromOptionToOption is not a valid implicit value for Some[Foo] => Option[Bar] because:
incompatible: (from: Option[Foo])(implicit conversion: Foo => B)Option[B] does not match expected type Some[Foo] => Option[Bar]
<console>:16: error: type mismatch;
 found   : Some[Foo]
 required: Option[Bar]
       val barOpt2: Option[Bar] = fooOpt
                                  ^

And there you go -- it doesn't know what type B must be. 0__ mentioned that this problem doesn't happen with invariant collections, and that makes some sense. In invariant collections, B must be exactly Bar, while for covariant collections it could be any subtype of Bar.

So, why does val foo: Option[Foo] = Some(Bar(1)) work? Well, there's a flag for that too... -Ytyper-debug. Not for the weak, however, given the extreme verbosity.

I waddled through anyway, comparing what happens in both cases, and the answer is rather simple... it's not the Option that is being converted in that case, but Bar! Remember, you declared an implicit conversion from Bar => Foo, so it is applying that conversion before passing the result to Some!

Answer 3

It doesn't work because the Scala Language Specification defines view as follows:

Implicit parameters and methods can also define implicit conversions called views. A view from type S to type T is defined by an implicit value which has function type S=>T or (=>S)=>T or by a method convertible to a value of that type.

fromOptionToOption doesn't conform to the three categories since it takes an implicit parameter. Compiler doesn't seem to find converter with both destination and source having generic type.

Defining a view from Option[Foo] to Option[Bar] works as expected.

trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T

object Main {
  implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
  implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
  // implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] =
  //  from.map(conversion(_))
  implicit def fromOptionFooToOptionBar(o: Option[Foo]): Option[Bar] = o map { foo => foo } 

  def test(): Option[Bar] = {
    val fooOpt = Some(Foo(4))
    val barOpt2: Option[Bar] = fooOpt

    barOpt2
  }
}

println(Main.test)

Running this prints out:

$ scala so.scala
Some(Bar(4))

However, all is not lost. It's not as nice as general Option to Option, but we can do something like anything that can turn into Bar to Option[Bar] by view bound.

trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T

object Main {
  implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
  implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
  implicit def fromOptionToOptionBar[A <% Bar](from: Option[A]): Option[Bar] =
    from map { foo => foo }

  def test(): Option[Bar] = {
    val fooOpt = Some(Foo(4))
    val barOpt2: Option[Bar] = fooOpt

    barOpt2
  }
}

println(Main.test)

Here's another workaround that can be used for general Option to Option but requires extra .convert call:

trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T

case class Converter[A](x: Option[A]) {
  def convert[B](implicit ev: Function1[A, B]): Option[B] = x map { a: A => ev(a) }
}

object Main {
  implicit def optionToConverter[A](x: Option[A]) = Converter(x)
  implicit def fooToBar(x: Foo) = Bar(x.i)

  def test(): Option[Bar] = {
    val fooOpt = Some(Foo(4))
    val barOpt: Option[Bar] = fooOpt.convert
    barOpt
  }
}

println(Main.test)

Answer 4

Indeed it's a very strange problem. I tried to use another type than Option, and it turns out that the problem is that Option is covariant in its type parameter. This works all:

case class A[B](value: B)  // invariant in B

case class X()
case class Y()

implicit def xtoy(x: X): Y = Y()
implicit def ytox(x: Y): X = X()
implicit def movea[U, V](from: A[U])(implicit view: U => V): A[V] =  A[V](from.value)

def test(a: A[Y]) = "ok"
test(A(X()))   // (1)
val f = A(X())
test(f)        // (2)

But if instead I define A as

case class A[+B](value: B)  // covariant in B

The case (2) fails. Case (1) always succeeds, because Scala already converts X to Y before wrapping it in an A.

Now that we know the problem source, you need to wait for a type guru to explain why this is actually a problem... The conversion is still valid, you see:

askForY(movea(f))  // succeeds, even with A[+B]

Answer 5

I improved @jseureth answer and added support for Traversable:

trait Mappable[A, B, C[_]] {
  def apply(f: A => B): C[B]
}

package object app {

  implicit class OptionMappable[A, B, C[X] <: Option[X]](option: C[A]) extends Mappable[A, B, Option] {
    override def apply(f: A => B): Option[B] = option.map(f)
  }

  implicit class TraversableMappable[A, B, C[X] <: Traversable[X]](traversable: C[A])
    (implicit cbf: CanBuildFrom[C[A], B, C[B]]) extends Mappable[A, B, C] {
    override def apply(f: A => B): C[B] = {
      val builder = cbf(traversable)
      builder.sizeHint(traversable)
      builder ++= traversable.map(f)
      builder.result()
    }
  }

  implicit def liftConversion[C[_], A, B](x: C[A])
    (implicit f: A => B, m: C[A] => Mappable[A, B, C]): C[B] = m(x)(f)

}

Now you can implicitly convert options and traversables:

implicit def f(i: Int): String = s"$i"

val a: Option[String] = Some(1)
val b: Seq[String] = Seq(1, 2, 3)