Fwd: Collect of user defined classes doesn't work

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: Collect of user defined classes doesn't work

Felix Neutatz
Hi,

I want to run the following example:

import org.apache.flink.api.scala._

case class EdgeType(src: Int, target: Int)

object Test {
   def main(args: Array[String]) {
      implicit val env = ExecutionEnvironment.getExecutionEnvironment

      val graphEdges = readEdges("edges.csv")

      graphEdges.collect()
   }
   def readEdges(file: String)(implicit env: ExecutionEnvironment) = {
      env.readCsvFile[EdgeType](file, "\n", "\t")
   }
}

But IntelliJ doesn't compile it and gives me the following explanation:


Error:(31, 21) not enough arguments for method apply: (n: Int)EdgeType
in trait BufferLike.
Unspecified value parameter n.
                graphEdges.collect()
                                  ^

Can anyone help me out here?

Thanks,

Felix
Reply | Threaded
Open this post in threaded view
|

Re: Collect of user defined classes doesn't work

till.rohrmann
Hi,

I assume that your graphEdges is of type DataSet[EdgeType], right?

Depending on the version you're using the reason might be that the collect
method in the old version does not expect parameters. Thus your parentheses
actually call the apply method on the returned Scala buffer.

In the latest master, this is changed. Thus, your program should work
without an error.

Cheers,

Till

On Tue, Apr 14, 2015 at 3:36 PM, Felix Neutatz <[hidden email]>
wrote:

> Hi,
>
> I want to run the following example:
>
> import org.apache.flink.api.scala._
>
> case class EdgeType(src: Int, target: Int)
>
> object Test {
>    def main(args: Array[String]) {
>       implicit val env = ExecutionEnvironment.getExecutionEnvironment
>
>       val graphEdges = readEdges("edges.csv")
>
>       graphEdges.collect()
>    }
>    def readEdges(file: String)(implicit env: ExecutionEnvironment) = {
>       env.readCsvFile[EdgeType](file, "\n", "\t")
>    }
> }
>
> But IntelliJ doesn't compile it and gives me the following explanation:
>
>
> Error:(31, 21) not enough arguments for method apply: (n: Int)EdgeType
> in trait BufferLike.
> Unspecified value parameter n.
>                 graphEdges.collect()
>                                   ^
>
> Can anyone help me out here?
>
> Thanks,
>
> Felix
>
Reply | Threaded
Open this post in threaded view
|

Re: Collect of user defined classes doesn't work

Robert Metzger
In reply to this post by Felix Neutatz
Hi,

which version of Flink are you using?

On Tue, Apr 14, 2015 at 3:36 PM, Felix Neutatz <[hidden email]>
wrote:

> Hi,
>
> I want to run the following example:
>
> import org.apache.flink.api.scala._
>
> case class EdgeType(src: Int, target: Int)
>
> object Test {
>    def main(args: Array[String]) {
>       implicit val env = ExecutionEnvironment.getExecutionEnvironment
>
>       val graphEdges = readEdges("edges.csv")
>
>       graphEdges.collect()
>    }
>    def readEdges(file: String)(implicit env: ExecutionEnvironment) = {
>       env.readCsvFile[EdgeType](file, "\n", "\t")
>    }
> }
>
> But IntelliJ doesn't compile it and gives me the following explanation:
>
>
> Error:(31, 21) not enough arguments for method apply: (n: Int)EdgeType
> in trait BufferLike.
> Unspecified value parameter n.
>                 graphEdges.collect()
>                                   ^
>
> Can anyone help me out here?
>
> Thanks,
>
> Felix
>
Reply | Threaded
Open this post in threaded view
|

Re: Collect of user defined classes doesn't work

Felix Neutatz
I use the latest maven snapshot:

<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-scala</artifactId>
  <version>0.9-SNAPSHOT</version>
</dependency>
<dependency>
  <groupId>org.apache.flink</groupId>
  <artifactId>flink-clients</artifactId>
  <version>0.9-SNAPSHOT</version>
</dependency>


2015-04-14 15:45 GMT+02:00 Robert Metzger <[hidden email]>:

> Hi,
>
> which version of Flink are you using?
>
> On Tue, Apr 14, 2015 at 3:36 PM, Felix Neutatz <[hidden email]>
> wrote:
>
> > Hi,
> >
> > I want to run the following example:
> >
> > import org.apache.flink.api.scala._
> >
> > case class EdgeType(src: Int, target: Int)
> >
> > object Test {
> >    def main(args: Array[String]) {
> >       implicit val env = ExecutionEnvironment.getExecutionEnvironment
> >
> >       val graphEdges = readEdges("edges.csv")
> >
> >       graphEdges.collect()
> >    }
> >    def readEdges(file: String)(implicit env: ExecutionEnvironment) = {
> >       env.readCsvFile[EdgeType](file, "\n", "\t")
> >    }
> > }
> >
> > But IntelliJ doesn't compile it and gives me the following explanation:
> >
> >
> > Error:(31, 21) not enough arguments for method apply: (n: Int)EdgeType
> > in trait BufferLike.
> > Unspecified value parameter n.
> >                 graphEdges.collect()
> >                                   ^
> >
> > Can anyone help me out here?
> >
> > Thanks,
> >
> > Felix
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Collect of user defined classes doesn't work

till.rohrmann
Could you check the definition of the collect method in the DataSet.scala
file? Does it contain parentheses or not?

On Tue, Apr 14, 2015 at 3:48 PM, Felix Neutatz <[hidden email]>
wrote:

> I use the latest maven snapshot:
>
> <dependency>
>   <groupId>org.apache.flink</groupId>
>   <artifactId>flink-scala</artifactId>
>   <version>0.9-SNAPSHOT</version>
> </dependency>
> <dependency>
>   <groupId>org.apache.flink</groupId>
>   <artifactId>flink-clients</artifactId>
>   <version>0.9-SNAPSHOT</version>
> </dependency>
>
>
> 2015-04-14 15:45 GMT+02:00 Robert Metzger <[hidden email]>:
>
> > Hi,
> >
> > which version of Flink are you using?
> >
> > On Tue, Apr 14, 2015 at 3:36 PM, Felix Neutatz <[hidden email]>
> > wrote:
> >
> > > Hi,
> > >
> > > I want to run the following example:
> > >
> > > import org.apache.flink.api.scala._
> > >
> > > case class EdgeType(src: Int, target: Int)
> > >
> > > object Test {
> > >    def main(args: Array[String]) {
> > >       implicit val env = ExecutionEnvironment.getExecutionEnvironment
> > >
> > >       val graphEdges = readEdges("edges.csv")
> > >
> > >       graphEdges.collect()
> > >    }
> > >    def readEdges(file: String)(implicit env: ExecutionEnvironment) = {
> > >       env.readCsvFile[EdgeType](file, "\n", "\t")
> > >    }
> > > }
> > >
> > > But IntelliJ doesn't compile it and gives me the following explanation:
> > >
> > >
> > > Error:(31, 21) not enough arguments for method apply: (n: Int)EdgeType
> > > in trait BufferLike.
> > > Unspecified value parameter n.
> > >                 graphEdges.collect()
> > >                                   ^
> > >
> > > Can anyone help me out here?
> > >
> > > Thanks,
> > >
> > > Felix
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Collect of user defined classes doesn't work

Felix Neutatz
I don't know. I can only see the following:

def collect : scala.collection.mutable.Buffer[T] = { /* compiled code */ }

When do they update the latest snapshot on Maven?


2015-04-14 15:52 GMT+02:00 Till Rohrmann <[hidden email]>:

> Could you check the definition of the collect method in the DataSet.scala
> file? Does it contain parentheses or not?
>
> On Tue, Apr 14, 2015 at 3:48 PM, Felix Neutatz <[hidden email]>
> wrote:
>
> > I use the latest maven snapshot:
> >
> > <dependency>
> >   <groupId>org.apache.flink</groupId>
> >   <artifactId>flink-scala</artifactId>
> >   <version>0.9-SNAPSHOT</version>
> > </dependency>
> > <dependency>
> >   <groupId>org.apache.flink</groupId>
> >   <artifactId>flink-clients</artifactId>
> >   <version>0.9-SNAPSHOT</version>
> > </dependency>
> >
> >
> > 2015-04-14 15:45 GMT+02:00 Robert Metzger <[hidden email]>:
> >
> > > Hi,
> > >
> > > which version of Flink are you using?
> > >
> > > On Tue, Apr 14, 2015 at 3:36 PM, Felix Neutatz <[hidden email]
> >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I want to run the following example:
> > > >
> > > > import org.apache.flink.api.scala._
> > > >
> > > > case class EdgeType(src: Int, target: Int)
> > > >
> > > > object Test {
> > > >    def main(args: Array[String]) {
> > > >       implicit val env = ExecutionEnvironment.getExecutionEnvironment
> > > >
> > > >       val graphEdges = readEdges("edges.csv")
> > > >
> > > >       graphEdges.collect()
> > > >    }
> > > >    def readEdges(file: String)(implicit env: ExecutionEnvironment) =
> {
> > > >       env.readCsvFile[EdgeType](file, "\n", "\t")
> > > >    }
> > > > }
> > > >
> > > > But IntelliJ doesn't compile it and gives me the following
> explanation:
> > > >
> > > >
> > > > Error:(31, 21) not enough arguments for method apply: (n:
> Int)EdgeType
> > > > in trait BufferLike.
> > > > Unspecified value parameter n.
> > > >                 graphEdges.collect()
> > > >                                   ^
> > > >
> > > > Can anyone help me out here?
> > > >
> > > > Thanks,
> > > >
> > > > Felix
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Collect of user defined classes doesn't work

Felix Neutatz
@Till: Yes, it works without the parentheses :) Thanks :)

2015-04-14 16:52 GMT+02:00 Felix Neutatz <[hidden email]>:

> I don't know. I can only see the following:
>
> def collect : scala.collection.mutable.Buffer[T] = { /* compiled code */ }
>
> When do they update the latest snapshot on Maven?
>
>
> 2015-04-14 15:52 GMT+02:00 Till Rohrmann <[hidden email]>:
>
>> Could you check the definition of the collect method in the DataSet.scala
>> file? Does it contain parentheses or not?
>>
>> On Tue, Apr 14, 2015 at 3:48 PM, Felix Neutatz <[hidden email]>
>> wrote:
>>
>> > I use the latest maven snapshot:
>> >
>> > <dependency>
>> >   <groupId>org.apache.flink</groupId>
>> >   <artifactId>flink-scala</artifactId>
>> >   <version>0.9-SNAPSHOT</version>
>> > </dependency>
>> > <dependency>
>> >   <groupId>org.apache.flink</groupId>
>> >   <artifactId>flink-clients</artifactId>
>> >   <version>0.9-SNAPSHOT</version>
>> > </dependency>
>> >
>> >
>> > 2015-04-14 15:45 GMT+02:00 Robert Metzger <[hidden email]>:
>> >
>> > > Hi,
>> > >
>> > > which version of Flink are you using?
>> > >
>> > > On Tue, Apr 14, 2015 at 3:36 PM, Felix Neutatz <
>> [hidden email]>
>> > > wrote:
>> > >
>> > > > Hi,
>> > > >
>> > > > I want to run the following example:
>> > > >
>> > > > import org.apache.flink.api.scala._
>> > > >
>> > > > case class EdgeType(src: Int, target: Int)
>> > > >
>> > > > object Test {
>> > > >    def main(args: Array[String]) {
>> > > >       implicit val env =
>> ExecutionEnvironment.getExecutionEnvironment
>> > > >
>> > > >       val graphEdges = readEdges("edges.csv")
>> > > >
>> > > >       graphEdges.collect()
>> > > >    }
>> > > >    def readEdges(file: String)(implicit env: ExecutionEnvironment)
>> = {
>> > > >       env.readCsvFile[EdgeType](file, "\n", "\t")
>> > > >    }
>> > > > }
>> > > >
>> > > > But IntelliJ doesn't compile it and gives me the following
>> explanation:
>> > > >
>> > > >
>> > > > Error:(31, 21) not enough arguments for method apply: (n:
>> Int)EdgeType
>> > > > in trait BufferLike.
>> > > > Unspecified value parameter n.
>> > > >                 graphEdges.collect()
>> > > >                                   ^
>> > > >
>> > > > Can anyone help me out here?
>> > > >
>> > > > Thanks,
>> > > >
>> > > > Felix
>> > > >
>> > >
>> >
>>
>
>