QueryableState, unable to register KvStateRegistryListener

[Vishnu Viswanath]

Hi,

I was trying the QueryableState from the pull request
https://github.com/apache/flink/pull/2051
I am doing the following:

1. Make the stream queryable by calling
myKeyedStreamd.asQueryableState("my-state",myStateDescriptor)

2. Create a client that takes a job id, conf, query-name and key,
num_of_retries. Similar to the test case written in
QueryableStateITCase.java

My QueryClient is able to talk to the job manager, i.e., it is sending the
KvStateMessage and JobManager is able to get the job graph and registry.
But it is not able to retrieve the location from the registry, because it
is not able to find the registrationName in the lookupTable.

After digging a little deep, I found that the lookupTable is not having the
registrationName because NetworkEnvironment is unable to register the
KvStateRegisryListener, since Option<NettyConfig> isDefined() is returning
false in the NetworkEnvironment.java. I am not sure how to proceed from
here, do I need to set some configs for QueryableState to work?

PS : I know this feature is not merged yet, but I was trying this out as
part of my POC, any help is appreciated.

Thanks,
Vishnu Viswanath

[Ufuk Celebi-2]

Hey Vishnu,

thanks for trying out the PR. :-) Would be great to move future
questions to the PR.

How are you starting your cluster? My guess is that you are running
the cluster in local mode, which is not starting up the network
components. Is that the case?

– Ufuk


On Thu, Jul 21, 2016 at 1:56 AM, Vishnu Viswanath
<[hidden email]> wrote:

> Hi,
>
> I was trying the QueryableState from the pull request
> https://github.com/apache/flink/pull/2051
> I am doing the following:
>
> 1. Make the stream queryable by calling
> myKeyedStreamd.asQueryableState("my-state",myStateDescriptor)
>
> 2. Create a client that takes a job id, conf, query-name and key,
> num_of_retries. Similar to the test case written in
> QueryableStateITCase.java
>
> My QueryClient is able to talk to the job manager, i.e., it is sending the
> KvStateMessage and JobManager is able to get the job graph and registry.
> But it is not able to retrieve the location from the registry, because it
> is not able to find the registrationName in the lookupTable.
>
> After digging a little deep, I found that the lookupTable is not having the
> registrationName because NetworkEnvironment is unable to register the
> KvStateRegisryListener, since Option<NettyConfig> isDefined() is returning
> false in the NetworkEnvironment.java. I am not sure how to proceed from
> here, do I need to set some configs for QueryableState to work?
>
> PS : I know this feature is not merged yet, but I was trying this out as
> part of my POC, any help is appreciated.
>
> Thanks,
> Vishnu Viswanath

[Vishnu Viswanath]

Hi Ufuk,

Thanks for the reply. Yes I am running in local mode (and hope it can be
used in local mode)

Thanks,
Vishnu

On Thu, Jul 21, 2016 at 4:29 AM, Ufuk Celebi <[hidden email]> wrote:

> Hey Vishnu,
>
> thanks for trying out the PR. :-) Would be great to move future
> questions to the PR.
>
> How are you starting your cluster? My guess is that you are running
> the cluster in local mode, which is not starting up the network
> components. Is that the case?
>
> – Ufuk
>
>
> On Thu, Jul 21, 2016 at 1:56 AM, Vishnu Viswanath
> <[hidden email]> wrote:
> > Hi,
> >
> > I was trying the QueryableState from the pull request
> > https://github.com/apache/flink/pull/2051
> > I am doing the following:
> >
> > 1. Make the stream queryable by calling
> > myKeyedStreamd.asQueryableState("my-state",myStateDescriptor)
> >
> > 2. Create a client that takes a job id, conf, query-name and key,
> > num_of_retries. Similar to the test case written in
> > QueryableStateITCase.java
> >
> > My QueryClient is able to talk to the job manager, i.e., it is sending
> the
> > KvStateMessage and JobManager is able to get the job graph and registry.
> > But it is not able to retrieve the location from the registry, because it
> > is not able to find the registrationName in the lookupTable.
> >
> > After digging a little deep, I found that the lookupTable is not having
> the
> > registrationName because NetworkEnvironment is unable to register the
> > KvStateRegisryListener, since Option<NettyConfig> isDefined() is
> returning
> > false in the NetworkEnvironment.java. I am not sure how to proceed from
> > here, do I need to set some configs for QueryableState to work?
> >
> > PS : I know this feature is not merged yet, but I was trying this out as
> > part of my POC, any help is appreciated.
> >
> > Thanks,
> > Vishnu Viswanath
>