It seems the implicit conversion from Enumeration#Value is missing, so you could define it yourself:
import cascading.tuple.Fields
implicit def valueToFields(v: Enumeration#Value): Fields = v.toString
object Schema extends Enumeration {
val first, last, phone, age, country = Value // arbitrary number of fields
}
import Schema._
var current = Int.MaxValue
Csv("tutorial/data/phones.txt", separator = " ", fields = Schema)
.read
.map(age -> ('current, 'previous)) { a: String =>
val previous = current
current = a.toInt
current -> previous
}
.filter('current, 'previous) { age: (Int, Int) => age._1 > age._2 }
.project(first, age)
.write(Tsv("tutorial/data/output6.tsv"))
In the end, we expect the result to be the same as that of:
Csv("tutorial/data/phones.txt", separator = " ", fields = Schema)
.read
.map((new Fields("age"), (new Fields("current", "previous"))) { a: String =>
val previous = current
current = a.toInt
current -> previous
}
.filter(new Fields("current", "previous")) { age: (Int, Int) =>
age._1 > age._2
}
.project(new Fields("first", "age"))
.write(Tsv("tutorial/data/output6.tsv"))
The implicit conversions provided by scalding allow you to write shorter versions of these new Fields(...)
.
An inplicit conversion is just a view, which will get used by the compiler when you pass arguments which are not of the expected type, but can be converted to the appropriate type by this view. For example, because map()
expects a pair of Fields
while you're passing it a pair of Symbols, Scala will search for an implicit conversion from Symbol -> Symbol
to Fields -> Fields
. A short explanation on views can be found here.
Scalding 0.8.5 introduced conversions from a product of Eumeration#Value
to a Fields
, but was missing conversions from a pair of values. The develop
branch now also provides the latter.