Hi all,
Is there any possibility to query Elastic search using flink? Thanks and Regards, Santosh |
Hi Santosh
There is an Elastic search connector under streaming connectors. Can you check if that satisfies your needs? Aljoscha will be able to provide more details in case you need them. Cheers! Sachin -- Sachin Goel Computer Science, IIT Delhi m. +91-9871457685 On Tue, Oct 13, 2015 at 6:38 PM, santosh_rajaguru <[hidden email]> wrote: > Hi all, > > Is there any possibility to query Elastic search using flink? > > > Thanks and Regards, > Santosh > > > > -- > View this message in context: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-flink-with-Elastic-Search-tp8435.html > Sent from the Apache Flink Mailing List archive. mailing list archive at > Nabble.com. > |
Hi,
the Elasticsearch connector can only be used for writing right now. If there is need we could also think about adding a connector for reading, though. Cheers, Aljoscha > On 13 Oct 2015, at 16:01, Sachin Goel <[hidden email]> wrote: > > Hi Santosh > There is an Elastic search connector under streaming connectors. Can you > check if that satisfies your needs? > Aljoscha will be able to provide more details in case you need them. > > Cheers! > Sachin > > -- Sachin Goel > Computer Science, IIT Delhi > m. +91-9871457685 > > On Tue, Oct 13, 2015 at 6:38 PM, santosh_rajaguru <[hidden email]> wrote: > >> Hi all, >> >> Is there any possibility to query Elastic search using flink? >> >> >> Thanks and Regards, >> Santosh >> >> >> >> -- >> View this message in context: >> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-flink-with-Elastic-Search-tp8435.html >> Sent from the Apache Flink Mailing List archive. mailing list archive at >> Nabble.com. >> |
You can always use the Hadoop connector of Elasticsearch ;)
On 14 Oct 2015 08:32, "Aljoscha Krettek" <[hidden email]> wrote: > Hi, > the Elasticsearch connector can only be used for writing right now. If > there is need we could also think about adding a connector for reading, > though. > > Cheers, > Aljoscha > > On 13 Oct 2015, at 16:01, Sachin Goel <[hidden email]> wrote: > > > > Hi Santosh > > There is an Elastic search connector under streaming connectors. Can you > > check if that satisfies your needs? > > Aljoscha will be able to provide more details in case you need them. > > > > Cheers! > > Sachin > > > > -- Sachin Goel > > Computer Science, IIT Delhi > > m. +91-9871457685 > > > > On Tue, Oct 13, 2015 at 6:38 PM, santosh_rajaguru <[hidden email]> > wrote: > > > >> Hi all, > >> > >> Is there any possibility to query Elastic search using flink? > >> > >> > >> Thanks and Regards, > >> Santosh > >> > >> > >> > >> -- > >> View this message in context: > >> > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-flink-with-Elastic-Search-tp8435.html > >> Sent from the Apache Flink Mailing List archive. mailing list archive at > >> Nabble.com. > >> > > |
Thanks flavio.
|
I think the Hadoop connector of ElasticSearch only works for batch jobs.
In my understanding, elasticsearch also allows "streaming" standing search queries. On Wed, Oct 14, 2015 at 10:41 AM, santosh_rajaguru <[hidden email]> wrote: > Thanks flavio. > > > > -- > View this message in context: > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-flink-with-Elastic-Search-tp8435p8445.html > Sent from the Apache Flink Mailing List archive. mailing list archive at > Nabble.com. > |
Elasticsearch does not support streaming in the sense, that you subscribe
to a search query and as new documents come in, you will get any matching your query. There is a rather long discussion in one of the issues in the ES repository on GitHub concerning this (active for 4 years now) [1] It does however provide you with an API to scroll through a large set of results, so you can retrieve them in chunks which fit your application. [2] That is supported by the hadoop connector via [3] which is used as far as I can tell in almost all of the different connectors (hive, cascading, mr, spark) Best regards, Martin [1] https://github.com/elastic/elasticsearch/issues/1242 [2] https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html [3] https://github.com/elastic/elasticsearch-hadoop/blob/03c056142a5ab7422b81bb1f519fd67a9581405f/mr/src/main/java/org/elasticsearch/hadoop/rest/ScrollQuery.java Robert Metzger <[hidden email]> schrieb am Mi., 14. Okt. 2015 um 17:49 Uhr: > I think the Hadoop connector of ElasticSearch only works for batch jobs. > In my understanding, elasticsearch also allows "streaming" standing search > queries. > > On Wed, Oct 14, 2015 at 10:41 AM, santosh_rajaguru <[hidden email]> > wrote: > > > Thanks flavio. > > > > > > > > -- > > View this message in context: > > > http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Apache-flink-with-Elastic-Search-tp8435p8445.html > > Sent from the Apache Flink Mailing List archive. mailing list archive at > > Nabble.com. > > > |
Thanks Martin for the input.
|
Free forum by Nabble | Edit this page |