Re: Integrate IGFS and Impala

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Integrate IGFS and Impala

dmagda
Cross-posting to the dev list.

IGFS gurus, are you aware of any existing and workable plug-n-play integration of Impala and IGFS? Looks this is unattainable unless Impala is supported directly by our in-memory file system.


Denis

> On Mar 24, 2017, at 10:23 PM, Masayuki Takahashi <[hidden email]> wrote:
>
> Hi,
>
> I want to use Impala with IGFS. But it failed.
> The reason why is that Impala depends on
> org.apache.hadoop.hdfs.DistributedFileSystem.
>
> https://github.com/cloudera/Impala/blob/2717f7378c406195eab241b34b184eee5f574f91/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L719
>
> IgniteHadoopFileSystem does not extend it.
>
> https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v1/IgniteHadoopFileSystem.html
> https://ignite.apache.org/releases/mobile/org/apache/ignite/hadoop/fs/v2/IgniteHadoopFileSystem.html
>
> Have anyone tried it?
>
> thanks.
>
> --
> Masayuki Takahashi

Reply | Threaded
Open this post in threaded view
|

Re: Integrate IGFS and Impala

Vladimir Ozerov
It looks like Impala expect some specific file system types. We can do
nothing with it. IGFS will never extend "org.apache.hadoop.hdfs.
DistributedFileSystem" because it is completely different implementation.

Question should be addressed to Impala developers.

On Sun, Mar 26, 2017 at 9:19 PM, Denis Magda <[hidden email]> wrote:

> Cross-posting to the dev list.
>
> IGFS gurus, are you aware of any existing and workable plug-n-play
> integration of Impala and IGFS? Looks this is unattainable unless Impala is
> supported directly by our in-memory file system.
>
> —
> Denis
>
> > On Mar 24, 2017, at 10:23 PM, Masayuki Takahashi <[hidden email]>
> wrote:
> >
> > Hi,
> >
> > I want to use Impala with IGFS. But it failed.
> > The reason why is that Impala depends on
> > org.apache.hadoop.hdfs.DistributedFileSystem.
> >
> > https://github.com/cloudera/Impala/blob/2717f7378c406195eab241b34b184e
> ee5f574f91/fe/src/main/java/org/apache/impala/service/
> JniFrontend.java#L719
> >
> > IgniteHadoopFileSystem does not extend it.
> >
> > https://ignite.apache.org/releases/mobile/org/apache/
> ignite/hadoop/fs/v1/IgniteHadoopFileSystem.html
> > https://ignite.apache.org/releases/mobile/org/apache/
> ignite/hadoop/fs/v2/IgniteHadoopFileSystem.html
> >
> > Have anyone tried it?
> >
> > thanks.
> >
> > --
> > Masayuki Takahashi
>
>