Hey,
Is it currently possible to obtain the state that was created by SQL query via the State Processor API? I am able to load the checkpoint via the State Processor API, but I wasn't able to think of a way to access the internal state of my JOIN Query. Best Regards, Dom. |
Hi Dom,
+ user mail list Once you got to know the state descriptor, I think you could query the join state. The state name is easy to get via [1], it should be "left-records" and "right-records", and you could check what kind of join and whether has unique key to decide what kind of state (value state or map state). The last part is to find what the rowData type is in your join, and maybe other SQL guys could answer this or you might find it by yourself by dumping the memory of your taskmanager. [1] https://github.com/apache/flink/blob/7a7c87096ab76f416cd7c393240faa8454db36f0/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/operators/join/stream/StreamingJoinOperator.java#L83 Best Yun Tang ________________________________ From: Dominik Wosiński <[hidden email]> Sent: Tuesday, December 1, 2020 21:05 To: dev <[hidden email]> Subject: State Processor API SQL State Hey, Is it currently possible to obtain the state that was created by SQL query via the State Processor API? I am able to load the checkpoint via the State Processor API, but I wasn't able to think of a way to access the internal state of my JOIN Query. Best Regards, Dom. |
Hey,
Thanks a lot for that. Best Regards, Dom. śr., 2 gru 2020 o 03:45 Yun Tang <[hidden email]> napisał(a): > Hi Dom, > > + user mail list > > Once you got to know the state descriptor, I think you could query the > join state. The state name is easy to get via [1], it should be > "left-records" and "right-records", and you could check what kind of join > and whether has unique key to decide what kind of state (value state or map > state). The last part is to find what the rowData type is in your join, and > maybe other SQL guys could answer this or you might find it by yourself by > dumping the memory of your taskmanager. > > [1] > https://github.com/apache/flink/blob/7a7c87096ab76f416cd7c393240faa8454db36f0/flink-table/flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/operators/join/stream/StreamingJoinOperator.java#L83 > > Best > Yun Tang > > > > ________________________________ > From: Dominik Wosiński <[hidden email]> > Sent: Tuesday, December 1, 2020 21:05 > To: dev <[hidden email]> > Subject: State Processor API SQL State > > Hey, > Is it currently possible to obtain the state that was created by SQL query > via the State Processor API? I am able to load the checkpoint via the State > Processor API, but I wasn't able to think of a way to access the internal > state of my JOIN Query. > > Best Regards, > Dom. > |
Free forum by Nabble | Edit this page |