[jira] [Created] (FLINK-17360) Support custom partitioners in JDBCReadOptions

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-17360) Support custom partitioners in JDBCReadOptions

Shang Yuanchun (Jira)
Flavio Pompermaier created FLINK-17360:
------------------------------------------

             Summary: Support custom partitioners in JDBCReadOptions
                 Key: FLINK-17360
                 URL: https://issues.apache.org/jira/browse/FLINK-17360
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / API
            Reporter: Flavio Pompermaier


Suport custom ParameterValuesProvider. At the moment only NumericBetweenParametersProvider is handled if a partition column and min/max values are specified.

In a discussion in the mailing list some discussion about this was made.

Me ([~f.pompermaier]):
Then we can add a *scan.parametervalues.provider.class* in order to customize the splitting of the query (we can also add a check that the query contains at least 1 question mark).
If we introduce a custom parameters provider we need also to specify parameters, using something like:
'scan.parametervalues.0.name' = 'minDate',
'scan.parametervalues.0.value'= '12/10/2019'
'scan.parametervalues.1.name' = 'maxDate',
'scan.parametervalues.1.value'= '01/01/2020'

[~lzljs3620320]
Maybe we need add something like "scan.parametervalues.provider.type", it can be "bound, specify, custom":
- when *bound*, using old p*artitionLowerBound* and *partitionUpperBound*, *numPartitions*
- when *specify*, using specify parameters like your proposal
- when *custom*, need *scan.parametervalues.provider.class*

Actually I don't know if specify and custom can be separated..but this can be further discussed



--
This message was sent by Atlassian Jira
(v8.3.4#803005)