[jira] [Created] (FLINK-19322) Specified parallelism in create table operation

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-19322) Specified parallelism in create table operation

Shang Yuanchun (Jira)
YufeiLiu created FLINK-19322:
--------------------------------

             Summary: Specified parallelism in create table operation
                 Key: FLINK-19322
                 URL: https://issues.apache.org/jira/browse/FLINK-19322
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / Common, Table SQL / API
    Affects Versions: 1.12.0
            Reporter: YufeiLiu


Flink SQL job only has global parallelism, and all operators share the same parallelism. But in many cases like using kafka connector, I want to set table parallelism separately to control resource usage.
* Source & Sink topic has different partition size.
* Consume multi topic with different throughput.
* Increase sink parallelism to insure write speed.

I think it would be better if we can specified parallelism property in create table operation, and the actual implementation depends on different connectors.

The create table operation may like this:
{code:sql}
CREATE TABLE source (
  ...
) WITH (
  'connector' = 'kafka',
  'parallelism' = '2'
)
{code}





--
This message was sent by Atlassian Jira
(v8.3.4#803005)