[QUESTION] about FsCheckpointStateOutputStream#createStream max 10 times create outStream

[未来阳光]

Hi all,

 I have a question about FsCheckpointStateOutputStream#createStream method, is it necessary to attempt max 10 times to create outStream?
 
 here's the FsCheckpointStateOutputStream#createStream link
 https://github.com/apache/flink/blob/322e479d540a74e45a9b5e38eb17b4eb3a4c6e9e/flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FsCheckpointStreamFactory.java#L344

Best,
lamber-ken

[Stephan Ewen]

The reason it tries 10 times is that
  - it is mainly meant to catch collisions on paths
  - if it fails for 10 times, there is a very high chance something more
fundamental is broken
  - we did not want to repeat an infinite number of times, to avoid
infinite loops and system hanging

On Thu, Jan 17, 2019 at 12:25 PM 未来阳光 <[hidden email]> wrote:

> Hi all,
>
>  I have a question about FsCheckpointStateOutputStream#createStream
> method, is it necessary to attempt max 10 times to create outStream?
>
>  here's the FsCheckpointStateOutputStream#createStream link
>
> https://github.com/apache/flink/blob/322e479d540a74e45a9b5e38eb17b4eb3a4c6e9e/flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FsCheckpointStreamFactory.java#L344
>
> Best,
> lamber-ken

[未来阳光]

Ok, I understand it. thanks

 

 ------------------ 原始邮件 ------------------
  发件人: "sewen"<[hidden email]>;
 发送时间: 2019年1月17日(星期四) 晚上8:28
 收件人: "dev"<[hidden email]>;
 
 主题: Re: [QUESTION] about FsCheckpointStateOutputStream#createStream max10 times create outStream

 

The reason it tries 10 times is that
  - it is mainly meant to catch collisions on paths
  - if it fails for 10 times, there is a very high chance something more
fundamental is broken
  - we did not want to repeat an infinite number of times, to avoid
infinite loops and system hanging

On Thu, Jan 17, 2019 at 12:25 PM 未来阳光 <[hidden email]> wrote:

> Hi all,
>
>  I have a question about FsCheckpointStateOutputStream#createStream
> method, is it necessary to attempt max 10 times to create outStream?
>
>  here's the FsCheckpointStateOutputStream#createStream link
>
> https://github.com/apache/flink/blob/322e479d540a74e45a9b5e38eb17b4eb3a4c6e9e/flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FsCheckpointStreamFactory.java#L344
>
> Best,
> lamber-ken