[jira] [Resolved] (FLINK-256) Nephele "loses" data packets

[Shang Yuanchun (Jira)]

     [ https://issues.apache.org/jira/browse/FLINK-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stephan Ewen resolved FLINK-256.
--------------------------------

    Resolution: Fixed
      Assignee: Ufuk Celebi

Fixed with the introduction of the Netty library to replace custom NIO code.

> Nephele "loses" data packets
> ----------------------------
>
>                 Key: FLINK-256
>                 URL: https://issues.apache.org/jira/browse/FLINK-256
>             Project: Flink
>          Issue Type: Bug
>            Reporter: GitHub Import
>            Assignee: Ufuk Celebi
>              Labels: github-import
>             Fix For: pre-apache
>
>
> A pact job dies with the following message: "An error occurred in the channel: Expected data packet 25 but received 27"
> The error occurred both in version 0.21 and with the current 0.4-snapshot. Apparently only with bigger data sizes. 15GB Input data worked fine, 22GB crashed. I sent the job, that caused the error to Stephan Ewen.
> Full stack trace:
> 15:32:11,639 ERROR eu.stratosphere.pact.runtime.task.RegularPactTask             - Error in PACT code: Join tweets and dates (2/4)
> 15:32:11,640 ERROR eu.stratosphere.pact.runtime.task.RegularPactTask             - java.io.IOException: An error occurred in the channel: Expected data packet 25 but received 27
> java.io.IOException: An error occurred in the channel: Expected data packet 25 but received 27
>         at eu.stratosphere.nephele.io.channels.bytebuffered.AbstractByteBufferedInputChannel.isClosed(AbstractByteBufferedInputChannel.java:144)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.isClosed(RuntimeInputGate.java:261)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.readRecord(RuntimeInputGate.java:182)
>         at eu.stratosphere.nephele.io.MutableRecordReader.next(MutableRecordReader.java:80)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:62)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:27)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable$ProbeIterator.next(MutableHashTable.java:1525)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.processProbeIter(MutableHashTable.java:450)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.nextRecord(MutableHashTable.java:536)
>         at eu.stratosphere.pact.runtime.hash.BuildFirstHashMatchIterator.callWithNextKey(BuildFirstHashMatchIterator.java:116)
>         at eu.stratosphere.pact.runtime.task.MatchDriver.run(MatchDriver.java:164)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.run(RegularPactTask.java:372)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.invoke(RegularPactTask.java:291)
>         at eu.stratosphere.nephele.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:344)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Expected data packet 25 but received 27
>         at eu.stratosphere.nephele.taskmanager.runtime.RuntimeInputChannelContext.queueTransferEnvelope(RuntimeInputChannelContext.java:148)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeWithBuffer(ByteBufferedChannelManager.java:365)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelope(ByteBufferedChannelManager.java:331)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeFromNetwork(ByteBufferedChannelManager.java:644)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnection.read(IncomingConnection.java:100)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.doRead(IncomingConnectionThread.java:187)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.run(IncomingConnectionThread.java:126)
> 15:32:12,362 WARN eu.stratosphere.pact.runtime.task.RegularPactTask             - Cancelling PACT code: Join tweets and dates (2/4)
> 15:32:12,362 INFO eu.stratosphere.nephele.execution.ExecutionStateTransition    - TM: ExecutionState set from RUNNING to FAILED for task Join tweets and dates (2/4)
> 15:32:12,362 ERROR eu.stratosphere.nephele.taskmanager.runtime.RuntimeTask       - java.io.IOException: An error occurred in the channel: Expected data packet 25 but received 27
>         at eu.stratosphere.nephele.io.channels.bytebuffered.AbstractByteBufferedInputChannel.isClosed(AbstractByteBufferedInputChannel.java:144)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.isClosed(RuntimeInputGate.java:261)
>         at eu.stratosphere.nephele.io.RuntimeInputGate.readRecord(RuntimeInputGate.java:182)
>         at eu.stratosphere.nephele.io.MutableRecordReader.next(MutableRecordReader.java:80)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:62)
>         at eu.stratosphere.pact.runtime.task.util.PactRecordNepheleReaderIterator.next(PactRecordNepheleReaderIterator.java:27)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable$ProbeIterator.next(MutableHashTable.java:1525)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.processProbeIter(MutableHashTable.java:450)
>         at eu.stratosphere.pact.runtime.hash.MutableHashTable.nextRecord(MutableHashTable.java:536)
>         at eu.stratosphere.pact.runtime.hash.BuildFirstHashMatchIterator.callWithNextKey(BuildFirstHashMatchIterator.java:116)
>         at eu.stratosphere.pact.runtime.task.MatchDriver.run(MatchDriver.java:164)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.run(RegularPactTask.java:372)
>         at eu.stratosphere.pact.runtime.task.RegularPactTask.invoke(RegularPactTask.java:291)
>         at eu.stratosphere.nephele.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:344)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.IOException: Expected data packet 25 but received 27
>         at eu.stratosphere.nephele.taskmanager.runtime.RuntimeInputChannelContext.queueTransferEnvelope(RuntimeInputChannelContext.java:148)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeWithBuffer(ByteBufferedChannelManager.java:365)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelope(ByteBufferedChannelManager.java:331)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.ByteBufferedChannelManager.processEnvelopeFromNetwork(ByteBufferedChannelManager.java:644)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnection.read(IncomingConnection.java:100)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.doRead(IncomingConnectionThread.java:187)
>         at eu.stratosphere.nephele.taskmanager.bytebuffered.IncomingConnectionThread.run(IncomingConnectionThread.java:126)
> 15:32:12,394 INFO eu.stratosphere.nephele.taskmanager.runtime.RuntimeTask       - Canceling Hashtag Polarity Match (2/4)
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/256
> Created by: [matpeters|https://github.com/matpeters]
> Labels: bug, runtime, user satisfaction,
> Created at: Fri Nov 08 13:57:57 CET 2013
> State: open

--
This message was sent by Atlassian JIRA
(v6.2#6252)