Geo spatial index

[Sergi]

Guys,

Looks like we have to move our geospatial indexes to the new approach with
BPlusTree. Right now it stores data in Java heap. This is especially
important because we are going to have a persistence layer donated by
GridGain and obviously geo spatial indexes will not work with it at all.

Sergi

[dmagda]

+1

This looks natural considering that we switched to the new memory architecture. Sergi, how difficult is to support this?

—
Denis

> On May 23, 2017, at 4:25 AM, Sergi Vladykin <[hidden email]> wrote:
>
> Guys,
>
> Looks like we have to move our geospatial indexes to the new approach with
> BPlusTree. Right now it stores data in Java heap. This is especially
> important because we are going to have a persistence layer donated by
> GridGain and obviously geo spatial indexes will not work with it at all.
>
> Sergi

[Sergi]

It must not be too hard to implement kd-tree over b+tree [1]. Depending on
level we have to compare either X or Y coordinate.

I think we will even have a performance boost for spatial indexes after
this change.

[1] https://en.wikipedia.org/wiki/K-d_tree

Sergi

2017-05-23 18:59 GMT+03:00 Denis Magda <[hidden email]>:

> +1
>
> This looks natural considering that we switched to the new memory
> architecture. Sergi, how difficult is to support this?
>
> —
> Denis
>
> > On May 23, 2017, at 4:25 AM, Sergi Vladykin <[hidden email]>
> wrote:
> >
> > Guys,
> >
> > Looks like we have to move our geospatial indexes to the new approach
> with
> > BPlusTree. Right now it stores data in Java heap. This is especially
> > important because we are going to have a persistence layer donated by
> > GridGain and obviously geo spatial indexes will not work with it at all.
> >
> > Sergi
>
>

[Sergi]

Though this may require some changes in BPlusTree. Let me think.

Sergi

2017-05-24 8:58 GMT+03:00 Sergi Vladykin <[hidden email]>:

> It must not be too hard to implement kd-tree over b+tree [1]. Depending on
> level we have to compare either X or Y coordinate.
>
> I think we will even have a performance boost for spatial indexes after
> this change.
>
> [1] https://en.wikipedia.org/wiki/K-d_tree
>
> Sergi
>
> 2017-05-23 18:59 GMT+03:00 Denis Magda <[hidden email]>:
>
>> +1
>>
>> This looks natural considering that we switched to the new memory
>> architecture. Sergi, how difficult is to support this?
>>
>> —
>> Denis
>>
>> > On May 23, 2017, at 4:25 AM, Sergi Vladykin <[hidden email]>
>> wrote:
>> >
>> > Guys,
>> >
>> > Looks like we have to move our geospatial indexes to the new approach
>> with
>> > BPlusTree. Right now it stores data in Java heap. This is especially
>> > important because we are going to have a persistence layer donated by
>> > GridGain and obviously geo spatial indexes will not work with it at all.
>> >
>> > Sergi
>>
>>
>

[dsetrakyan]

Sergi,

While we are figuring this out, what happens to the GeoSpatial
functionality in the mean time? Is it going to work at all? If not, should
we throw some sort of exception?

D.

On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin <[hidden email]>
wrote:

> Though this may require some changes in BPlusTree. Let me think.
>
> Sergi
>
> 2017-05-24 8:58 GMT+03:00 Sergi Vladykin <[hidden email]>:
>
> > It must not be too hard to implement kd-tree over b+tree [1]. Depending
> on
> > level we have to compare either X or Y coordinate.
> >
> > I think we will even have a performance boost for spatial indexes after
> > this change.
> >
> > [1] https://en.wikipedia.org/wiki/K-d_tree
> >
> > Sergi
> >
> > 2017-05-23 18:59 GMT+03:00 Denis Magda <[hidden email]>:
> >
> >> +1
> >>
> >> This looks natural considering that we switched to the new memory
> >> architecture. Sergi, how difficult is to support this?
> >>
> >> —
> >> Denis
> >>
> >> > On May 23, 2017, at 4:25 AM, Sergi Vladykin <[hidden email]
> >
> >> wrote:
> >> >
> >> > Guys,
> >> >
> >> > Looks like we have to move our geospatial indexes to the new approach
> >> with
> >> > BPlusTree. Right now it stores data in Java heap. This is especially
> >> > important because we are going to have a persistence layer donated by
> >> > GridGain and obviously geo spatial indexes will not work with it at
> all.
> >> >
> >> > Sergi
> >>
> >>
> >
>

[Alexey Goncharuk]

Same think stays for the full-text indexes which are currently stored in
Lucene.

2017-05-24 21:56 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:

> Sergi,
>
> While we are figuring this out, what happens to the GeoSpatial
> functionality in the mean time? Is it going to work at all? If not, should
> we throw some sort of exception?
>
> D.
>
> On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin <[hidden email]>
> wrote:
>
> > Though this may require some changes in BPlusTree. Let me think.
> >
> > Sergi
> >
> > 2017-05-24 8:58 GMT+03:00 Sergi Vladykin <[hidden email]>:
> >
> > > It must not be too hard to implement kd-tree over b+tree [1]. Depending
> > on
> > > level we have to compare either X or Y coordinate.
> > >
> > > I think we will even have a performance boost for spatial indexes after
> > > this change.
> > >
> > > [1] https://en.wikipedia.org/wiki/K-d_tree
> > >
> > > Sergi
> > >
> > > 2017-05-23 18:59 GMT+03:00 Denis Magda <[hidden email]>:
> > >
> > >> +1
> > >>
> > >> This looks natural considering that we switched to the new memory
> > >> architecture. Sergi, how difficult is to support this?
> > >>
> > >> —
> > >> Denis
> > >>
> > >> > On May 23, 2017, at 4:25 AM, Sergi Vladykin <
> [hidden email]
> > >
> > >> wrote:
> > >> >
> > >> > Guys,
> > >> >
> > >> > Looks like we have to move our geospatial indexes to the new
> approach
> > >> with
> > >> > BPlusTree. Right now it stores data in Java heap. This is especially
> > >> > important because we are going to have a persistence layer donated
> by
> > >> > GridGain and obviously geo spatial indexes will not work with it at
> > all.
> > >> >
> > >> > Sergi
> > >>
> > >>
> > >
> >
>

[dmagda]

Good catch. I found the ticket that’s aim is to integrate the full-text search indexes with the virtual page memory architecture.
https://issues.apache.org/jira/browse/IGNITE-5371 <https://issues.apache.org/jira/browse/IGNITE-5371>

Sergi, do we have a similar one for the geo-spatial?

—
Denis
 

> On Jun 1, 2017, at 2:12 AM, Alexey Goncharuk <[hidden email]> wrote:
>
> Same think stays for the full-text indexes which are currently stored in
> Lucene.
>
> 2017-05-24 21:56 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
>
>> Sergi,
>>
>> While we are figuring this out, what happens to the GeoSpatial
>> functionality in the mean time? Is it going to work at all? If not, should
>> we throw some sort of exception?
>>
>> D.
>>
>> On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin <[hidden email]>
>> wrote:
>>
>>> Though this may require some changes in BPlusTree. Let me think.
>>>
>>> Sergi
>>>
>>> 2017-05-24 8:58 GMT+03:00 Sergi Vladykin <[hidden email]>:
>>>
>>>> It must not be too hard to implement kd-tree over b+tree [1]. Depending
>>> on
>>>> level we have to compare either X or Y coordinate.
>>>>
>>>> I think we will even have a performance boost for spatial indexes after
>>>> this change.
>>>>
>>>> [1] https://en.wikipedia.org/wiki/K-d_tree
>>>>
>>>> Sergi
>>>>
>>>> 2017-05-23 18:59 GMT+03:00 Denis Magda <[hidden email]>:
>>>>
>>>>> +1
>>>>>
>>>>> This looks natural considering that we switched to the new memory
>>>>> architecture. Sergi, how difficult is to support this?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>>> On May 23, 2017, at 4:25 AM, Sergi Vladykin <
>> [hidden email]
>>>>
>>>>> wrote:
>>>>>>
>>>>>> Guys,
>>>>>>
>>>>>> Looks like we have to move our geospatial indexes to the new
>> approach
>>>>> with
>>>>>> BPlusTree. Right now it stores data in Java heap. This is especially
>>>>>> important because we are going to have a persistence layer donated
>> by
>>>>>> GridGain and obviously geo spatial indexes will not work with it at
>>> all.
>>>>>>
>>>>>> Sergi
>>>>>
>>>>>
>>>>
>>>
>>

[Sergi]

No, feel free to create one.

Sergi

2017-06-01 21:00 GMT+03:00 Denis Magda <[hidden email]>:

> Good catch. I found the ticket that’s aim is to integrate the full-text
> search indexes with the virtual page memory architecture.
> https://issues.apache.org/jira/browse/IGNITE-5371 <
> https://issues.apache.org/jira/browse/IGNITE-5371>
>
> Sergi, do we have a similar one for the geo-spatial?
>
> —
> Denis
>
> > On Jun 1, 2017, at 2:12 AM, Alexey Goncharuk <[hidden email]>
> wrote:
> >
> > Same think stays for the full-text indexes which are currently stored in
> > Lucene.
> >
> > 2017-05-24 21:56 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
> >
> >> Sergi,
> >>
> >> While we are figuring this out, what happens to the GeoSpatial
> >> functionality in the mean time? Is it going to work at all? If not,
> should
> >> we throw some sort of exception?
> >>
> >> D.
> >>
> >> On Wed, May 24, 2017 at 1:44 AM, Sergi Vladykin <
> [hidden email]>
> >> wrote:
> >>
> >>> Though this may require some changes in BPlusTree. Let me think.
> >>>
> >>> Sergi
> >>>
> >>> 2017-05-24 8:58 GMT+03:00 Sergi Vladykin <[hidden email]>:
> >>>
> >>>> It must not be too hard to implement kd-tree over b+tree [1].
> Depending
> >>> on
> >>>> level we have to compare either X or Y coordinate.
> >>>>
> >>>> I think we will even have a performance boost for spatial indexes
> after
> >>>> this change.
> >>>>
> >>>> [1] https://en.wikipedia.org/wiki/K-d_tree
> >>>>
> >>>> Sergi
> >>>>
> >>>> 2017-05-23 18:59 GMT+03:00 Denis Magda <[hidden email]>:
> >>>>
> >>>>> +1
> >>>>>
> >>>>> This looks natural considering that we switched to the new memory
> >>>>> architecture. Sergi, how difficult is to support this?
> >>>>>
> >>>>> —
> >>>>> Denis
> >>>>>
> >>>>>> On May 23, 2017, at 4:25 AM, Sergi Vladykin <
> >> [hidden email]
> >>>>
> >>>>> wrote:
> >>>>>>
> >>>>>> Guys,
> >>>>>>
> >>>>>> Looks like we have to move our geospatial indexes to the new
> >> approach
> >>>>> with
> >>>>>> BPlusTree. Right now it stores data in Java heap. This is especially
> >>>>>> important because we are going to have a persistence layer donated
> >> by
> >>>>>> GridGain and obviously geo spatial indexes will not work with it at
> >>> all.
> >>>>>>
> >>>>>> Sergi
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>