ScanQuery shows poor performance.

[Andrew Mashenkov]

Hi Igniters,

  I've found that ScanQuery iterates over partitions in non-optimal way. It
uses KeyIterator which gets value for each key via call to cache map.
I made a try to replace KeyIterator with EntryIterator to avoid cache map
lookups and got significant speed up, about 30%.

Is there some reason to use KeyIterator for this or some other use cases?

I've created a ticket [1] with patch and test attached.


[1] https://issues.apache.org/jira/browse/IGNITE-4624

Regards.

[Alexey Goncharuk]

Andrey,

From the top of my head I would guess that this is done so because
keyIterator handles on-heap, off-heap and swap, while entrySet() return
only on-heap entries. Please check that your change does not break
iteration with off-heap and swap enabled (if it does, it just means that we
need to implement a correct entry iterator for off-heap as well).

--AG

2017-01-27 16:06 GMT+03:00 Andrey Mashenkov <[hidden email]>:

> Hi Igniters,
>
>   I've found that ScanQuery iterates over partitions in non-optimal way. It
> uses KeyIterator which gets value for each key via call to cache map.
> I made a try to replace KeyIterator with EntryIterator to avoid cache map
> lookups and got significant speed up, about 30%.
>
> Is there some reason to use KeyIterator for this or some other use cases?
>
> I've created a ticket [1] with patch and test attached.
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-4624
>
> Regards.
>

[dsetrakyan]

On Fri, Jan 27, 2017 at 7:03 AM, Alexey Goncharuk <
[hidden email]> wrote:

> Andrey,
>
> From the top of my head I would guess that this is done so because
> keyIterator handles on-heap, off-heap and swap, while entrySet() return
> only on-heap entries. Please check that your change does not break
> iteration with off-heap and swap enabled (if it does, it just means that we
> need to implement a correct entry iterator for off-heap as well).
>

Well, in this case we should definitely use entry iterator whenever
off-heap is not configured.

[Andrew Mashenkov]

Alexey,

Would you please look at PR [1];
Can it be fixed the way implemented at PR or I've missed smth?

[1] https://issues.apache.org/jira/browse/IGNITE-4624

On Fri, Jan 27, 2017 at 7:57 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Fri, Jan 27, 2017 at 7:03 AM, Alexey Goncharuk <
> [hidden email]> wrote:
>
> > Andrey,
> >
> > From the top of my head I would guess that this is done so because
> > keyIterator handles on-heap, off-heap and swap, while entrySet() return
> > only on-heap entries. Please check that your change does not break
> > iteration with off-heap and swap enabled (if it does, it just means that
> we
> > need to implement a correct entry iterator for off-heap as well).
> >
>
> Well, in this case we should definitely use entry iterator whenever
> off-heap is not configured.
>

--
С уважением,
Машенков Андрей Владимирович
Тел. [phone]

Best regards,
Andrey V. Mashenkov
Cerr: [phone]