[jira] [Created] (FLINK-14243) flink hiveudf need some check whether it is using cache

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-14243) flink hiveudf need some check whether it is using cache

Shang Yuanchun (Jira)
jackylau created FLINK-14243:
--------------------------------

             Summary: flink hiveudf need some check whether it is using cache
                 Key: FLINK-14243
                 URL: https://issues.apache.org/jira/browse/FLINK-14243
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Hive
    Affects Versions: 1.9.0
            Reporter: jackylau
             Fix For: 1.10.0


Flink1.9 bring in hive connector, but it will have some problem when the original hive udf using cache. We konw that hive isĀ  processed level parallel based on jvm, while flink/spark is task level parallel. If flink just calls the hive udf, it wll exists thread-safe problem when using cache.

So it may need check the hive udf code and if it is not thread-fase, and set the flink parallize=1



--
This message was sent by Atlassian Jira
(v8.3.4#803005)