A soft reminder

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

A soft reminder

Andra Lungu
Hi guys,

I opened these JIRAs a while ago and, unfortunately, no one had time to
look at them :|.

[1] https://issues.apache.org/jira/browse/FLINK-2361
[2] https://issues.apache.org/jira/browse/FLINK-2360

[1] could probably be hacked, but [2] is a huge problem that was
encountered by other people outside of Gelly.

Thanks for the help!
Andra
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Stephan Ewen
Hey!

It is pretty much as you said: No one had the time to fix this so far. They
are on the list, though,

Do you want to pick up one of them?

Greetings,
Stephan


On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]> wrote:

> Hi guys,
>
> I opened these JIRAs a while ago and, unfortunately, no one had time to
> look at them :|.
>
> [1] https://issues.apache.org/jira/browse/FLINK-2361
> [2] https://issues.apache.org/jira/browse/FLINK-2360
>
> [1] could probably be hacked, but [2] is a huge problem that was
> encountered by other people outside of Gelly.
>
> Thanks for the help!
> Andra
>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Andra Lungu
Hi Stephan,

I tried to debug a bit around the EOF Exception. It seems that I am pretty
useless on my own. I have no idea how to fix it. And it occurs in quite a
lot of experiments... I believe it's something behind a delta iteration
[this is the common point between my experiments and the others' code].

Some pointers would be greatly appreciated :)
Thanks!
Andra

On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote:

> Hey!
>
> It is pretty much as you said: No one had the time to fix this so far. They
> are on the list, though,
>
> Do you want to pick up one of them?
>
> Greetings,
> Stephan
>
>
> On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]>
> wrote:
>
> > Hi guys,
> >
> > I opened these JIRAs a while ago and, unfortunately, no one had time to
> > look at them :|.
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-2361
> > [2] https://issues.apache.org/jira/browse/FLINK-2360
> >
> > [1] could probably be hacked, but [2] is a huge problem that was
> > encountered by other people outside of Gelly.
> >
> > Thanks for the help!
> > Andra
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Felix Neutatz
Hi,

I also encountered the EOF exception for a delta iteration with "more
data". With less data it works ...

Best regards,
Felix
Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>:

> Hi Stephan,
>
> I tried to debug a bit around the EOF Exception. It seems that I am pretty
> useless on my own. I have no idea how to fix it. And it occurs in quite a
> lot of experiments... I believe it's something behind a delta iteration
> [this is the common point between my experiments and the others' code].
>
> Some pointers would be greatly appreciated :)
> Thanks!
> Andra
>
> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote:
>
> > Hey!
> >
> > It is pretty much as you said: No one had the time to fix this so far.
> They
> > are on the list, though,
> >
> > Do you want to pick up one of them?
> >
> > Greetings,
> > Stephan
> >
> >
> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]>
> > wrote:
> >
> > > Hi guys,
> > >
> > > I opened these JIRAs a while ago and, unfortunately, no one had time to
> > > look at them :|.
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
> > >
> > > [1] could probably be hacked, but [2] is a huge problem that was
> > > encountered by other people outside of Gelly.
> > >
> > > Thanks for the help!
> > > Andra
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Gábor Gévay
Hi,

I have also run into this problem just now. It only happens with much data.

Best regards,
Gabor



2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>:

> Hi,
>
> I also encountered the EOF exception for a delta iteration with "more
> data". With less data it works ...
>
> Best regards,
> Felix
> Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>:
>
>> Hi Stephan,
>>
>> I tried to debug a bit around the EOF Exception. It seems that I am pretty
>> useless on my own. I have no idea how to fix it. And it occurs in quite a
>> lot of experiments... I believe it's something behind a delta iteration
>> [this is the common point between my experiments and the others' code].
>>
>> Some pointers would be greatly appreciated :)
>> Thanks!
>> Andra
>>
>> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote:
>>
>> > Hey!
>> >
>> > It is pretty much as you said: No one had the time to fix this so far.
>> They
>> > are on the list, though,
>> >
>> > Do you want to pick up one of them?
>> >
>> > Greetings,
>> > Stephan
>> >
>> >
>> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]>
>> > wrote:
>> >
>> > > Hi guys,
>> > >
>> > > I opened these JIRAs a while ago and, unfortunately, no one had time to
>> > > look at them :|.
>> > >
>> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
>> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
>> > >
>> > > [1] could probably be hacked, but [2] is a huge problem that was
>> > > encountered by other people outside of Gelly.
>> > >
>> > > Thanks for the help!
>> > > Andra
>> > >
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Andra Lungu
Hi Gabor,

Within a delta iteration right?

On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote:

> Hi,
>
> I have also run into this problem just now. It only happens with much data.
>
> Best regards,
> Gabor
>
>
>
> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>:
> > Hi,
> >
> > I also encountered the EOF exception for a delta iteration with "more
> > data". With less data it works ...
> >
> > Best regards,
> > Felix
> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>:
> >
> >> Hi Stephan,
> >>
> >> I tried to debug a bit around the EOF Exception. It seems that I am
> pretty
> >> useless on my own. I have no idea how to fix it. And it occurs in quite
> a
> >> lot of experiments... I believe it's something behind a delta iteration
> >> [this is the common point between my experiments and the others' code].
> >>
> >> Some pointers would be greatly appreciated :)
> >> Thanks!
> >> Andra
> >>
> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote:
> >>
> >> > Hey!
> >> >
> >> > It is pretty much as you said: No one had the time to fix this so far.
> >> They
> >> > are on the list, though,
> >> >
> >> > Do you want to pick up one of them?
> >> >
> >> > Greetings,
> >> > Stephan
> >> >
> >> >
> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]>
> >> > wrote:
> >> >
> >> > > Hi guys,
> >> > >
> >> > > I opened these JIRAs a while ago and, unfortunately, no one had
> time to
> >> > > look at them :|.
> >> > >
> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
> >> > >
> >> > > [1] could probably be hacked, but [2] is a huge problem that was
> >> > > encountered by other people outside of Gelly.
> >> > >
> >> > > Thanks for the help!
> >> > > Andra
> >> > >
> >> >
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Gábor Gévay
Yes, in a VertexCentricIteration with a few million nodes, running
locally on my laptop with about 10 GB of memory given to java.

Best,
Gabor



2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>:

> Hi Gabor,
>
> Within a delta iteration right?
>
> On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote:
>
>> Hi,
>>
>> I have also run into this problem just now. It only happens with much data.
>>
>> Best regards,
>> Gabor
>>
>>
>>
>> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>:
>> > Hi,
>> >
>> > I also encountered the EOF exception for a delta iteration with "more
>> > data". With less data it works ...
>> >
>> > Best regards,
>> > Felix
>> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>:
>> >
>> >> Hi Stephan,
>> >>
>> >> I tried to debug a bit around the EOF Exception. It seems that I am
>> pretty
>> >> useless on my own. I have no idea how to fix it. And it occurs in quite
>> a
>> >> lot of experiments... I believe it's something behind a delta iteration
>> >> [this is the common point between my experiments and the others' code].
>> >>
>> >> Some pointers would be greatly appreciated :)
>> >> Thanks!
>> >> Andra
>> >>
>> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote:
>> >>
>> >> > Hey!
>> >> >
>> >> > It is pretty much as you said: No one had the time to fix this so far.
>> >> They
>> >> > are on the list, though,
>> >> >
>> >> > Do you want to pick up one of them?
>> >> >
>> >> > Greetings,
>> >> > Stephan
>> >> >
>> >> >
>> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]>
>> >> > wrote:
>> >> >
>> >> > > Hi guys,
>> >> > >
>> >> > > I opened these JIRAs a while ago and, unfortunately, no one had
>> time to
>> >> > > look at them :|.
>> >> > >
>> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
>> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
>> >> > >
>> >> > > [1] could probably be hacked, but [2] is a huge problem that was
>> >> > > encountered by other people outside of Gelly.
>> >> > >
>> >> > > Thanks for the help!
>> >> > > Andra
>> >> > >
>> >> >
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Andra Lungu
Could you try adding the following lines to your code?

VertexCentricConfiguration parameters = new VertexCentricConfiguration();
parameters.setSolutionSetUnmanagedMemory(true);

return graphWithScoredVertices.runVertexCentricIteration(new
VertexLabelUpdater(delta),
      new LabelMessenger(), maxIterations, parameters);

Change the functions appropriately in the runVertexCentricIteration call.
The error seems to be caused by this CompactingHashTable and this parameter
deactivates its usage. I am very curious what happens in your case. If you
could tell us the outcome, I'd greately appreciate it.


On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay <[hidden email]> wrote:

> Yes, in a VertexCentricIteration with a few million nodes, running
> locally on my laptop with about 10 GB of memory given to java.
>
> Best,
> Gabor
>
>
>
> 2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>:
> > Hi Gabor,
> >
> > Within a delta iteration right?
> >
> > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote:
> >
> >> Hi,
> >>
> >> I have also run into this problem just now. It only happens with much
> data.
> >>
> >> Best regards,
> >> Gabor
> >>
> >>
> >>
> >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>:
> >> > Hi,
> >> >
> >> > I also encountered the EOF exception for a delta iteration with "more
> >> > data". With less data it works ...
> >> >
> >> > Best regards,
> >> > Felix
> >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <
> [hidden email]>:
> >> >
> >> >> Hi Stephan,
> >> >>
> >> >> I tried to debug a bit around the EOF Exception. It seems that I am
> >> pretty
> >> >> useless on my own. I have no idea how to fix it. And it occurs in
> quite
> >> a
> >> >> lot of experiments... I believe it's something behind a delta
> iteration
> >> >> [this is the common point between my experiments and the others'
> code].
> >> >>
> >> >> Some pointers would be greatly appreciated :)
> >> >> Thanks!
> >> >> Andra
> >> >>
> >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]>
> wrote:
> >> >>
> >> >> > Hey!
> >> >> >
> >> >> > It is pretty much as you said: No one had the time to fix this so
> far.
> >> >> They
> >> >> > are on the list, though,
> >> >> >
> >> >> > Do you want to pick up one of them?
> >> >> >
> >> >> > Greetings,
> >> >> > Stephan
> >> >> >
> >> >> >
> >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <
> [hidden email]>
> >> >> > wrote:
> >> >> >
> >> >> > > Hi guys,
> >> >> > >
> >> >> > > I opened these JIRAs a while ago and, unfortunately, no one had
> >> time to
> >> >> > > look at them :|.
> >> >> > >
> >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
> >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
> >> >> > >
> >> >> > > [1] could probably be hacked, but [2] is a huge problem that was
> >> >> > > encountered by other people outside of Gelly.
> >> >> > >
> >> >> > > Thanks for the help!
> >> >> > > Andra
> >> >> > >
> >> >> >
> >> >>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Gábor Gévay
It is working with setSolutionSetUnmanagedMemory(true), thanks!

Gabor



2015-07-30 19:23 GMT+02:00 Andra Lungu <[hidden email]>:

> Could you try adding the following lines to your code?
>
> VertexCentricConfiguration parameters = new VertexCentricConfiguration();
> parameters.setSolutionSetUnmanagedMemory(true);
>
> return graphWithScoredVertices.runVertexCentricIteration(new
> VertexLabelUpdater(delta),
>       new LabelMessenger(), maxIterations, parameters);
>
> Change the functions appropriately in the runVertexCentricIteration call.
> The error seems to be caused by this CompactingHashTable and this parameter
> deactivates its usage. I am very curious what happens in your case. If you
> could tell us the outcome, I'd greately appreciate it.
>
>
> On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay <[hidden email]> wrote:
>
>> Yes, in a VertexCentricIteration with a few million nodes, running
>> locally on my laptop with about 10 GB of memory given to java.
>>
>> Best,
>> Gabor
>>
>>
>>
>> 2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>:
>> > Hi Gabor,
>> >
>> > Within a delta iteration right?
>> >
>> > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote:
>> >
>> >> Hi,
>> >>
>> >> I have also run into this problem just now. It only happens with much
>> data.
>> >>
>> >> Best regards,
>> >> Gabor
>> >>
>> >>
>> >>
>> >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>:
>> >> > Hi,
>> >> >
>> >> > I also encountered the EOF exception for a delta iteration with "more
>> >> > data". With less data it works ...
>> >> >
>> >> > Best regards,
>> >> > Felix
>> >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <
>> [hidden email]>:
>> >> >
>> >> >> Hi Stephan,
>> >> >>
>> >> >> I tried to debug a bit around the EOF Exception. It seems that I am
>> >> pretty
>> >> >> useless on my own. I have no idea how to fix it. And it occurs in
>> quite
>> >> a
>> >> >> lot of experiments... I believe it's something behind a delta
>> iteration
>> >> >> [this is the common point between my experiments and the others'
>> code].
>> >> >>
>> >> >> Some pointers would be greatly appreciated :)
>> >> >> Thanks!
>> >> >> Andra
>> >> >>
>> >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]>
>> wrote:
>> >> >>
>> >> >> > Hey!
>> >> >> >
>> >> >> > It is pretty much as you said: No one had the time to fix this so
>> far.
>> >> >> They
>> >> >> > are on the list, though,
>> >> >> >
>> >> >> > Do you want to pick up one of them?
>> >> >> >
>> >> >> > Greetings,
>> >> >> > Stephan
>> >> >> >
>> >> >> >
>> >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <
>> [hidden email]>
>> >> >> > wrote:
>> >> >> >
>> >> >> > > Hi guys,
>> >> >> > >
>> >> >> > > I opened these JIRAs a while ago and, unfortunately, no one had
>> >> time to
>> >> >> > > look at them :|.
>> >> >> > >
>> >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
>> >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
>> >> >> > >
>> >> >> > > [1] could probably be hacked, but [2] is a huge problem that was
>> >> >> > > encountered by other people outside of Gelly.
>> >> >> > >
>> >> >> > > Thanks for the help!
>> >> >> > > Andra
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: A soft reminder

Stephan Ewen
I think I have found two problems in the hash table. One may be the loss of
records, the other one the EOF.

Give me a bit to confirm, test, and prepare a patch.

On Thu, Jul 30, 2015 at 7:46 PM, Gábor Gévay <[hidden email]> wrote:

> It is working with setSolutionSetUnmanagedMemory(true), thanks!
>
> Gabor
>
>
>
> 2015-07-30 19:23 GMT+02:00 Andra Lungu <[hidden email]>:
> > Could you try adding the following lines to your code?
> >
> > VertexCentricConfiguration parameters = new VertexCentricConfiguration();
> > parameters.setSolutionSetUnmanagedMemory(true);
> >
> > return graphWithScoredVertices.runVertexCentricIteration(new
> > VertexLabelUpdater(delta),
> >       new LabelMessenger(), maxIterations, parameters);
> >
> > Change the functions appropriately in the runVertexCentricIteration call.
> > The error seems to be caused by this CompactingHashTable and this
> parameter
> > deactivates its usage. I am very curious what happens in your case. If
> you
> > could tell us the outcome, I'd greately appreciate it.
> >
> >
> > On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay <[hidden email]> wrote:
> >
> >> Yes, in a VertexCentricIteration with a few million nodes, running
> >> locally on my laptop with about 10 GB of memory given to java.
> >>
> >> Best,
> >> Gabor
> >>
> >>
> >>
> >> 2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>:
> >> > Hi Gabor,
> >> >
> >> > Within a delta iteration right?
> >> >
> >> > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]>
> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> I have also run into this problem just now. It only happens with much
> >> data.
> >> >>
> >> >> Best regards,
> >> >> Gabor
> >> >>
> >> >>
> >> >>
> >> >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>:
> >> >> > Hi,
> >> >> >
> >> >> > I also encountered the EOF exception for a delta iteration with
> "more
> >> >> > data". With less data it works ...
> >> >> >
> >> >> > Best regards,
> >> >> > Felix
> >> >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <
> >> [hidden email]>:
> >> >> >
> >> >> >> Hi Stephan,
> >> >> >>
> >> >> >> I tried to debug a bit around the EOF Exception. It seems that I
> am
> >> >> pretty
> >> >> >> useless on my own. I have no idea how to fix it. And it occurs in
> >> quite
> >> >> a
> >> >> >> lot of experiments... I believe it's something behind a delta
> >> iteration
> >> >> >> [this is the common point between my experiments and the others'
> >> code].
> >> >> >>
> >> >> >> Some pointers would be greatly appreciated :)
> >> >> >> Thanks!
> >> >> >> Andra
> >> >> >>
> >> >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]>
> >> wrote:
> >> >> >>
> >> >> >> > Hey!
> >> >> >> >
> >> >> >> > It is pretty much as you said: No one had the time to fix this
> so
> >> far.
> >> >> >> They
> >> >> >> > are on the list, though,
> >> >> >> >
> >> >> >> > Do you want to pick up one of them?
> >> >> >> >
> >> >> >> > Greetings,
> >> >> >> > Stephan
> >> >> >> >
> >> >> >> >
> >> >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <
> >> [hidden email]>
> >> >> >> > wrote:
> >> >> >> >
> >> >> >> > > Hi guys,
> >> >> >> > >
> >> >> >> > > I opened these JIRAs a while ago and, unfortunately, no one
> had
> >> >> time to
> >> >> >> > > look at them :|.
> >> >> >> > >
> >> >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
> >> >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
> >> >> >> > >
> >> >> >> > > [1] could probably be hacked, but [2] is a huge problem that
> was
> >> >> >> > > encountered by other people outside of Gelly.
> >> >> >> > >
> >> >> >> > > Thanks for the help!
> >> >> >> > > Andra
> >> >> >> > >
> >> >> >> >
> >> >> >>
> >> >>
> >>
>