[jira] [Created] (FLINK-19007) Automatically set presto staging directory to io.tmp.dirs

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (FLINK-19007) Automatically set presto staging directory to io.tmp.dirs

Shang Yuanchun (Jira)
Chesnay Schepler created FLINK-19007:
----------------------------------------

             Summary: Automatically set presto staging directory to io.tmp.dirs
                 Key: FLINK-19007
                 URL: https://issues.apache.org/jira/browse/FLINK-19007
             Project: Flink
          Issue Type: Improvement
          Components: Connectors / FileSystem, Runtime / Configuration
            Reporter: Chesnay Schepler
             Fix For: 1.12.0


The presto S3 filesystem uses a staging directory on the local disk (for buffering data or smth). This directory by default points to {{/tmp}}.

We often see users configuring a special directory for temporary files via {{io.tmp.dirs}}, and it would make sense to automatically set the {{staging-directory}} to the same path, iff it was not explicitly configured.

The corresponding property is {{presto.s3.staging-directory}}, found at {{com.facebook.presto.hive.s3.PrestoS3FileSystem.S3_STAGING_DIRECTORY}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)