Hi guys,
I opened these JIRAs a while ago and, unfortunately, no one had time to look at them :|. [1] https://issues.apache.org/jira/browse/FLINK-2361 [2] https://issues.apache.org/jira/browse/FLINK-2360 [1] could probably be hacked, but [2] is a huge problem that was encountered by other people outside of Gelly. Thanks for the help! Andra |
Hey!
It is pretty much as you said: No one had the time to fix this so far. They are on the list, though, Do you want to pick up one of them? Greetings, Stephan On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]> wrote: > Hi guys, > > I opened these JIRAs a while ago and, unfortunately, no one had time to > look at them :|. > > [1] https://issues.apache.org/jira/browse/FLINK-2361 > [2] https://issues.apache.org/jira/browse/FLINK-2360 > > [1] could probably be hacked, but [2] is a huge problem that was > encountered by other people outside of Gelly. > > Thanks for the help! > Andra > |
Hi Stephan,
I tried to debug a bit around the EOF Exception. It seems that I am pretty useless on my own. I have no idea how to fix it. And it occurs in quite a lot of experiments... I believe it's something behind a delta iteration [this is the common point between my experiments and the others' code]. Some pointers would be greatly appreciated :) Thanks! Andra On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote: > Hey! > > It is pretty much as you said: No one had the time to fix this so far. They > are on the list, though, > > Do you want to pick up one of them? > > Greetings, > Stephan > > > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]> > wrote: > > > Hi guys, > > > > I opened these JIRAs a while ago and, unfortunately, no one had time to > > look at them :|. > > > > [1] https://issues.apache.org/jira/browse/FLINK-2361 > > [2] https://issues.apache.org/jira/browse/FLINK-2360 > > > > [1] could probably be hacked, but [2] is a huge problem that was > > encountered by other people outside of Gelly. > > > > Thanks for the help! > > Andra > > > |
Hi,
I also encountered the EOF exception for a delta iteration with "more data". With less data it works ... Best regards, Felix Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>: > Hi Stephan, > > I tried to debug a bit around the EOF Exception. It seems that I am pretty > useless on my own. I have no idea how to fix it. And it occurs in quite a > lot of experiments... I believe it's something behind a delta iteration > [this is the common point between my experiments and the others' code]. > > Some pointers would be greatly appreciated :) > Thanks! > Andra > > On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote: > > > Hey! > > > > It is pretty much as you said: No one had the time to fix this so far. > They > > are on the list, though, > > > > Do you want to pick up one of them? > > > > Greetings, > > Stephan > > > > > > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]> > > wrote: > > > > > Hi guys, > > > > > > I opened these JIRAs a while ago and, unfortunately, no one had time to > > > look at them :|. > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-2361 > > > [2] https://issues.apache.org/jira/browse/FLINK-2360 > > > > > > [1] could probably be hacked, but [2] is a huge problem that was > > > encountered by other people outside of Gelly. > > > > > > Thanks for the help! > > > Andra > > > > > > |
Hi,
I have also run into this problem just now. It only happens with much data. Best regards, Gabor 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>: > Hi, > > I also encountered the EOF exception for a delta iteration with "more > data". With less data it works ... > > Best regards, > Felix > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>: > >> Hi Stephan, >> >> I tried to debug a bit around the EOF Exception. It seems that I am pretty >> useless on my own. I have no idea how to fix it. And it occurs in quite a >> lot of experiments... I believe it's something behind a delta iteration >> [this is the common point between my experiments and the others' code]. >> >> Some pointers would be greatly appreciated :) >> Thanks! >> Andra >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote: >> >> > Hey! >> > >> > It is pretty much as you said: No one had the time to fix this so far. >> They >> > are on the list, though, >> > >> > Do you want to pick up one of them? >> > >> > Greetings, >> > Stephan >> > >> > >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]> >> > wrote: >> > >> > > Hi guys, >> > > >> > > I opened these JIRAs a while ago and, unfortunately, no one had time to >> > > look at them :|. >> > > >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361 >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360 >> > > >> > > [1] could probably be hacked, but [2] is a huge problem that was >> > > encountered by other people outside of Gelly. >> > > >> > > Thanks for the help! >> > > Andra >> > > >> > >> |
Hi Gabor,
Within a delta iteration right? On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote: > Hi, > > I have also run into this problem just now. It only happens with much data. > > Best regards, > Gabor > > > > 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>: > > Hi, > > > > I also encountered the EOF exception for a delta iteration with "more > > data". With less data it works ... > > > > Best regards, > > Felix > > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>: > > > >> Hi Stephan, > >> > >> I tried to debug a bit around the EOF Exception. It seems that I am > pretty > >> useless on my own. I have no idea how to fix it. And it occurs in quite > a > >> lot of experiments... I believe it's something behind a delta iteration > >> [this is the common point between my experiments and the others' code]. > >> > >> Some pointers would be greatly appreciated :) > >> Thanks! > >> Andra > >> > >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote: > >> > >> > Hey! > >> > > >> > It is pretty much as you said: No one had the time to fix this so far. > >> They > >> > are on the list, though, > >> > > >> > Do you want to pick up one of them? > >> > > >> > Greetings, > >> > Stephan > >> > > >> > > >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]> > >> > wrote: > >> > > >> > > Hi guys, > >> > > > >> > > I opened these JIRAs a while ago and, unfortunately, no one had > time to > >> > > look at them :|. > >> > > > >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361 > >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360 > >> > > > >> > > [1] could probably be hacked, but [2] is a huge problem that was > >> > > encountered by other people outside of Gelly. > >> > > > >> > > Thanks for the help! > >> > > Andra > >> > > > >> > > >> > |
Yes, in a VertexCentricIteration with a few million nodes, running
locally on my laptop with about 10 GB of memory given to java. Best, Gabor 2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>: > Hi Gabor, > > Within a delta iteration right? > > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote: > >> Hi, >> >> I have also run into this problem just now. It only happens with much data. >> >> Best regards, >> Gabor >> >> >> >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>: >> > Hi, >> > >> > I also encountered the EOF exception for a delta iteration with "more >> > data". With less data it works ... >> > >> > Best regards, >> > Felix >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <[hidden email]>: >> > >> >> Hi Stephan, >> >> >> >> I tried to debug a bit around the EOF Exception. It seems that I am >> pretty >> >> useless on my own. I have no idea how to fix it. And it occurs in quite >> a >> >> lot of experiments... I believe it's something behind a delta iteration >> >> [this is the common point between my experiments and the others' code]. >> >> >> >> Some pointers would be greatly appreciated :) >> >> Thanks! >> >> Andra >> >> >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> wrote: >> >> >> >> > Hey! >> >> > >> >> > It is pretty much as you said: No one had the time to fix this so far. >> >> They >> >> > are on the list, though, >> >> > >> >> > Do you want to pick up one of them? >> >> > >> >> > Greetings, >> >> > Stephan >> >> > >> >> > >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <[hidden email]> >> >> > wrote: >> >> > >> >> > > Hi guys, >> >> > > >> >> > > I opened these JIRAs a while ago and, unfortunately, no one had >> time to >> >> > > look at them :|. >> >> > > >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361 >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360 >> >> > > >> >> > > [1] could probably be hacked, but [2] is a huge problem that was >> >> > > encountered by other people outside of Gelly. >> >> > > >> >> > > Thanks for the help! >> >> > > Andra >> >> > > >> >> > >> >> >> |
Could you try adding the following lines to your code?
VertexCentricConfiguration parameters = new VertexCentricConfiguration(); parameters.setSolutionSetUnmanagedMemory(true); return graphWithScoredVertices.runVertexCentricIteration(new VertexLabelUpdater(delta), new LabelMessenger(), maxIterations, parameters); Change the functions appropriately in the runVertexCentricIteration call. The error seems to be caused by this CompactingHashTable and this parameter deactivates its usage. I am very curious what happens in your case. If you could tell us the outcome, I'd greately appreciate it. On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay <[hidden email]> wrote: > Yes, in a VertexCentricIteration with a few million nodes, running > locally on my laptop with about 10 GB of memory given to java. > > Best, > Gabor > > > > 2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>: > > Hi Gabor, > > > > Within a delta iteration right? > > > > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote: > > > >> Hi, > >> > >> I have also run into this problem just now. It only happens with much > data. > >> > >> Best regards, > >> Gabor > >> > >> > >> > >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>: > >> > Hi, > >> > > >> > I also encountered the EOF exception for a delta iteration with "more > >> > data". With less data it works ... > >> > > >> > Best regards, > >> > Felix > >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" < > [hidden email]>: > >> > > >> >> Hi Stephan, > >> >> > >> >> I tried to debug a bit around the EOF Exception. It seems that I am > >> pretty > >> >> useless on my own. I have no idea how to fix it. And it occurs in > quite > >> a > >> >> lot of experiments... I believe it's something behind a delta > iteration > >> >> [this is the common point between my experiments and the others' > code]. > >> >> > >> >> Some pointers would be greatly appreciated :) > >> >> Thanks! > >> >> Andra > >> >> > >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> > wrote: > >> >> > >> >> > Hey! > >> >> > > >> >> > It is pretty much as you said: No one had the time to fix this so > far. > >> >> They > >> >> > are on the list, though, > >> >> > > >> >> > Do you want to pick up one of them? > >> >> > > >> >> > Greetings, > >> >> > Stephan > >> >> > > >> >> > > >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu < > [hidden email]> > >> >> > wrote: > >> >> > > >> >> > > Hi guys, > >> >> > > > >> >> > > I opened these JIRAs a while ago and, unfortunately, no one had > >> time to > >> >> > > look at them :|. > >> >> > > > >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361 > >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360 > >> >> > > > >> >> > > [1] could probably be hacked, but [2] is a huge problem that was > >> >> > > encountered by other people outside of Gelly. > >> >> > > > >> >> > > Thanks for the help! > >> >> > > Andra > >> >> > > > >> >> > > >> >> > >> > |
It is working with setSolutionSetUnmanagedMemory(true), thanks!
Gabor 2015-07-30 19:23 GMT+02:00 Andra Lungu <[hidden email]>: > Could you try adding the following lines to your code? > > VertexCentricConfiguration parameters = new VertexCentricConfiguration(); > parameters.setSolutionSetUnmanagedMemory(true); > > return graphWithScoredVertices.runVertexCentricIteration(new > VertexLabelUpdater(delta), > new LabelMessenger(), maxIterations, parameters); > > Change the functions appropriately in the runVertexCentricIteration call. > The error seems to be caused by this CompactingHashTable and this parameter > deactivates its usage. I am very curious what happens in your case. If you > could tell us the outcome, I'd greately appreciate it. > > > On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay <[hidden email]> wrote: > >> Yes, in a VertexCentricIteration with a few million nodes, running >> locally on my laptop with about 10 GB of memory given to java. >> >> Best, >> Gabor >> >> >> >> 2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>: >> > Hi Gabor, >> > >> > Within a delta iteration right? >> > >> > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> wrote: >> > >> >> Hi, >> >> >> >> I have also run into this problem just now. It only happens with much >> data. >> >> >> >> Best regards, >> >> Gabor >> >> >> >> >> >> >> >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>: >> >> > Hi, >> >> > >> >> > I also encountered the EOF exception for a delta iteration with "more >> >> > data". With less data it works ... >> >> > >> >> > Best regards, >> >> > Felix >> >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" < >> [hidden email]>: >> >> > >> >> >> Hi Stephan, >> >> >> >> >> >> I tried to debug a bit around the EOF Exception. It seems that I am >> >> pretty >> >> >> useless on my own. I have no idea how to fix it. And it occurs in >> quite >> >> a >> >> >> lot of experiments... I believe it's something behind a delta >> iteration >> >> >> [this is the common point between my experiments and the others' >> code]. >> >> >> >> >> >> Some pointers would be greatly appreciated :) >> >> >> Thanks! >> >> >> Andra >> >> >> >> >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> >> wrote: >> >> >> >> >> >> > Hey! >> >> >> > >> >> >> > It is pretty much as you said: No one had the time to fix this so >> far. >> >> >> They >> >> >> > are on the list, though, >> >> >> > >> >> >> > Do you want to pick up one of them? >> >> >> > >> >> >> > Greetings, >> >> >> > Stephan >> >> >> > >> >> >> > >> >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu < >> [hidden email]> >> >> >> > wrote: >> >> >> > >> >> >> > > Hi guys, >> >> >> > > >> >> >> > > I opened these JIRAs a while ago and, unfortunately, no one had >> >> time to >> >> >> > > look at them :|. >> >> >> > > >> >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361 >> >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360 >> >> >> > > >> >> >> > > [1] could probably be hacked, but [2] is a huge problem that was >> >> >> > > encountered by other people outside of Gelly. >> >> >> > > >> >> >> > > Thanks for the help! >> >> >> > > Andra >> >> >> > > >> >> >> > >> >> >> >> >> >> |
I think I have found two problems in the hash table. One may be the loss of
records, the other one the EOF. Give me a bit to confirm, test, and prepare a patch. On Thu, Jul 30, 2015 at 7:46 PM, Gábor Gévay <[hidden email]> wrote: > It is working with setSolutionSetUnmanagedMemory(true), thanks! > > Gabor > > > > 2015-07-30 19:23 GMT+02:00 Andra Lungu <[hidden email]>: > > Could you try adding the following lines to your code? > > > > VertexCentricConfiguration parameters = new VertexCentricConfiguration(); > > parameters.setSolutionSetUnmanagedMemory(true); > > > > return graphWithScoredVertices.runVertexCentricIteration(new > > VertexLabelUpdater(delta), > > new LabelMessenger(), maxIterations, parameters); > > > > Change the functions appropriately in the runVertexCentricIteration call. > > The error seems to be caused by this CompactingHashTable and this > parameter > > deactivates its usage. I am very curious what happens in your case. If > you > > could tell us the outcome, I'd greately appreciate it. > > > > > > On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay <[hidden email]> wrote: > > > >> Yes, in a VertexCentricIteration with a few million nodes, running > >> locally on my laptop with about 10 GB of memory given to java. > >> > >> Best, > >> Gabor > >> > >> > >> > >> 2015-07-30 18:32 GMT+02:00 Andra Lungu <[hidden email]>: > >> > Hi Gabor, > >> > > >> > Within a delta iteration right? > >> > > >> > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <[hidden email]> > wrote: > >> > > >> >> Hi, > >> >> > >> >> I have also run into this problem just now. It only happens with much > >> data. > >> >> > >> >> Best regards, > >> >> Gabor > >> >> > >> >> > >> >> > >> >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <[hidden email]>: > >> >> > Hi, > >> >> > > >> >> > I also encountered the EOF exception for a delta iteration with > "more > >> >> > data". With less data it works ... > >> >> > > >> >> > Best regards, > >> >> > Felix > >> >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" < > >> [hidden email]>: > >> >> > > >> >> >> Hi Stephan, > >> >> >> > >> >> >> I tried to debug a bit around the EOF Exception. It seems that I > am > >> >> pretty > >> >> >> useless on my own. I have no idea how to fix it. And it occurs in > >> quite > >> >> a > >> >> >> lot of experiments... I believe it's something behind a delta > >> iteration > >> >> >> [this is the common point between my experiments and the others' > >> code]. > >> >> >> > >> >> >> Some pointers would be greatly appreciated :) > >> >> >> Thanks! > >> >> >> Andra > >> >> >> > >> >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <[hidden email]> > >> wrote: > >> >> >> > >> >> >> > Hey! > >> >> >> > > >> >> >> > It is pretty much as you said: No one had the time to fix this > so > >> far. > >> >> >> They > >> >> >> > are on the list, though, > >> >> >> > > >> >> >> > Do you want to pick up one of them? > >> >> >> > > >> >> >> > Greetings, > >> >> >> > Stephan > >> >> >> > > >> >> >> > > >> >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu < > >> [hidden email]> > >> >> >> > wrote: > >> >> >> > > >> >> >> > > Hi guys, > >> >> >> > > > >> >> >> > > I opened these JIRAs a while ago and, unfortunately, no one > had > >> >> time to > >> >> >> > > look at them :|. > >> >> >> > > > >> >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361 > >> >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360 > >> >> >> > > > >> >> >> > > [1] could probably be hacked, but [2] is a huge problem that > was > >> >> >> > > encountered by other people outside of Gelly. > >> >> >> > > > >> >> >> > > Thanks for the help! > >> >> >> > > Andra > >> >> >> > > > >> >> >> > > >> >> >> > >> >> > >> > |
Free forum by Nabble | Edit this page |