Off-Heap SQL Indexes
Locked
 
|
Engineers,
I was going through documentation for Off-Heap SQL Indexes <https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes> and found two contradicting statements- According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is the only property to enable or disable off-heap indexing. While CacheConfiguration.setMemoryMode is used to configure off-heap memory <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does not have any effect on indexing." According to [2] - "Note that when off-heap memory is configured, Ignite will store query indexes off-heap as well." Which one is true? [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes [2] - http://apacheignite.gridgain.org/docs/off-heap-memory Thanks, -Prachi |
|
|
Prachi,
In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory property, so [1] is correct. If setOffHeapMaxMemory is disabled (set to -1), then indexes will never be stored offheap, regardless of setMemoryMode setting. Probably we have to change [2] to make it clear, that it is about setOffHeapMaxMemory but not about setMemoryMode. Sergi 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: > Engineers, > > I was going through documentation for Off-Heap SQL Indexes > <https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes> and > found two contradicting statements- > > According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is the only > property to enable or disable off-heap indexing. While > CacheConfiguration.setMemoryMode is used to configure off-heap memory > <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does not > have any effect on indexing." > > According to [2] - "Note that when off-heap memory is configured, Ignite > will store query indexes off-heap as well." > > > Which one is true? > > [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes > > [2] - http://apacheignite.gridgain.org/docs/off-heap-memory > > > Thanks, > > -Prachi > |
|
|
Sergi,
Referring to your response and the source code I should say that, in fact, the documentation has to state the following: “To place both data and indexes in off heap memory CacheConfiguration.setMemoryMode has to be set to one of off heap modes and CacheConfiguration.setOffHeapMaxMemory has to be set to a value different from -1 (disabled). If at least one of these conditions is not met then both data and indexes will be located in on heap memory” Is this statement correct? — Denis > On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email]> wrote: > > Prachi, > > In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory > property, so [1] is correct. If setOffHeapMaxMemory is disabled (set to > -1), then indexes will never be stored offheap, regardless of setMemoryMode > setting. > > Probably we have to change [2] to make it clear, that it is about > setOffHeapMaxMemory > but not about setMemoryMode. > > Sergi > > > 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: > >> Engineers, >> >> I was going through documentation for Off-Heap SQL Indexes >> <https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes> and >> found two contradicting statements- >> >> According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is the only >> property to enable or disable off-heap indexing. While >> CacheConfiguration.setMemoryMode is used to configure off-heap memory >> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does not >> have any effect on indexing." >> >> According to [2] - "Note that when off-heap memory is configured, Ignite >> will store query indexes off-heap as well." >> >> >> Which one is true? >> >> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes >> >> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory >> >> >> Thanks, >> >> -Prachi >> |
|
Do we currently throw an exception if memory mode is set to off-heap by the
max-offheap-memory is set to "-1"? Otherwise, it can get very confusing to users. On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <[hidden email]> wrote: > Sergi, > > Referring to your response and the source code I should say that, in fact, > the documentation has to state the following: > > “To place both data and indexes in off heap memory CacheConfiguration.setMemoryMode > has to be set to one of off heap modes and CacheConfiguration.setOffHeapMaxMemory > has to be set to a value different from -1 (disabled). If at least one of > these conditions is not met then both data and indexes will be located in > on heap memory” > > Is this statement correct? > > — > Denis > > > On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email]> > wrote: > > > > Prachi, > > > > In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory > > property, so [1] is correct. If setOffHeapMaxMemory is disabled (set to > > -1), then indexes will never be stored offheap, regardless of > setMemoryMode > > setting. > > > > Probably we have to change [2] to make it clear, that it is about > > setOffHeapMaxMemory > > but not about setMemoryMode. > > > > Sergi > > > > > > 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: > > > >> Engineers, > >> > >> I was going through documentation for Off-Heap SQL Indexes > >> <https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes> > and > >> found two contradicting statements- > >> > >> According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is the > only > >> property to enable or disable off-heap indexing. While > >> CacheConfiguration.setMemoryMode is used to configure off-heap memory > >> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does not > >> have any effect on indexing." > >> > >> According to [2] - "Note that when off-heap memory is configured, > Ignite > >> will store query indexes off-heap as well." > >> > >> > >> Which one is true? > >> > >> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap- > sql-indexes > >> > >> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory > >> > >> > >> Thanks, > >> > >> -Prachi > >> > > |
|
|
Denis,
I checked the code now as well. I actually was wrong, but your statement is wrong either. If we are talking about indexes then the exact condition when index will be off-heap is the following: either setOffHeapMaxMemory >= 0 or setMemoryMode is OFFHEAP_TIERED. But if setMemoryMode is set to one of off-heap modes and setOffHeapMaxMemory is -1, then setOffHeapMaxMemory is silently getting switched to 0. Thus from the user standpoint we can say, that if either setOffHeapMaxMemory >= 0 or setMemoryMode is set to any of off-heap modes, then indexes will go off-heap. Sergi 2016-11-04 6:30 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > Do we currently throw an exception if memory mode is set to off-heap by the > max-offheap-memory is set to "-1"? Otherwise, it can get very confusing to > users. > > On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <[hidden email]> wrote: > > > Sergi, > > > > Referring to your response and the source code I should say that, in > fact, > > the documentation has to state the following: > > > > “To place both data and indexes in off heap memory CacheConfiguration. > setMemoryMode > > has to be set to one of off heap modes and CacheConfiguration. > setOffHeapMaxMemory > > has to be set to a value different from -1 (disabled). If at least one of > > these conditions is not met then both data and indexes will be located in > > on heap memory” > > > > Is this statement correct? > > > > — > > Denis > > > > > On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email]> > > wrote: > > > > > > Prachi, > > > > > > In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory > > > property, so [1] is correct. If setOffHeapMaxMemory is disabled (set to > > > -1), then indexes will never be stored offheap, regardless of > > setMemoryMode > > > setting. > > > > > > Probably we have to change [2] to make it clear, that it is about > > > setOffHeapMaxMemory > > > but not about setMemoryMode. > > > > > > Sergi > > > > > > > > > 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: > > > > > >> Engineers, > > >> > > >> I was going through documentation for Off-Heap SQL Indexes > > >> <https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes > > > > and > > >> found two contradicting statements- > > >> > > >> According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is the > > only > > >> property to enable or disable off-heap indexing. While > > >> CacheConfiguration.setMemoryMode is used to configure off-heap memory > > >> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does > not > > >> have any effect on indexing." > > >> > > >> According to [2] - "Note that when off-heap memory is configured, > > Ignite > > >> will store query indexes off-heap as well." > > >> > > >> > > >> Which one is true? > > >> > > >> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap- > > sql-indexes > > >> > > >> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory > > >> > > >> > > >> Thanks, > > >> > > >> -Prachi > > >> > > > > > |
|
|
Sergi,
Thanks, now I found the exact conditions that switch the indexes from on-heap to off-heap. But looks like you did a misprint in your conclusion saying that setOffHeapMaxMemory should be 0. > Thus from the user standpoint we can say, that if either setOffHeapMaxMemory >> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will go > off-heap. In fact, as you elaborated earlier, it has to be >= 0 which leads to the following final statement: SQL engine will store indexes in the off-heap region if setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 automatically if it’s value is less than -1 and one of off-heap memory modes is used. Are we on the same page now? — Denis > On Nov 4, 2016, at 1:51 AM, Sergi Vladykin <[hidden email]> wrote: > > Denis, > > I checked the code now as well. I actually was wrong, but your statement is > wrong either. > > If we are talking about indexes then the exact condition when index will be > off-heap is the following: either setOffHeapMaxMemory >= 0 or setMemoryMode > is OFFHEAP_TIERED. > > But if setMemoryMode is set to one of off-heap modes and setOffHeapMaxMemory is > -1, then setOffHeapMaxMemory is silently getting switched to 0. > > Thus from the user standpoint we can say, that if either setOffHeapMaxMemory >> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will go > off-heap. > > Sergi > > 2016-11-04 6:30 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > >> Do we currently throw an exception if memory mode is set to off-heap by the >> max-offheap-memory is set to "-1"? Otherwise, it can get very confusing to >> users. >> >> On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <[hidden email]> wrote: >> >>> Sergi, >>> >>> Referring to your response and the source code I should say that, in >> fact, >>> the documentation has to state the following: >>> >>> “To place both data and indexes in off heap memory CacheConfiguration. >> setMemoryMode >>> has to be set to one of off heap modes and CacheConfiguration. >> setOffHeapMaxMemory >>> has to be set to a value different from -1 (disabled). If at least one of >>> these conditions is not met then both data and indexes will be located in >>> on heap memory” >>> >>> Is this statement correct? >>> >>> — >>> Denis >>> >>>> On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email]> >>> wrote: >>>> >>>> Prachi, >>>> >>>> In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory >>>> property, so [1] is correct. If setOffHeapMaxMemory is disabled (set to >>>> -1), then indexes will never be stored offheap, regardless of >>> setMemoryMode >>>> setting. >>>> >>>> Probably we have to change [2] to make it clear, that it is about >>>> setOffHeapMaxMemory >>>> but not about setMemoryMode. >>>> >>>> Sergi >>>> >>>> >>>> 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: >>>> >>>>> Engineers, >>>>> >>>>> I was going through documentation for Off-Heap SQL Indexes >>>>> <https://apacheignite.readme.io/docs/sql-queries#off-heap-sql-indexes >>> >>> and >>>>> found two contradicting statements- >>>>> >>>>> According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is the >>> only >>>>> property to enable or disable off-heap indexing. While >>>>> CacheConfiguration.setMemoryMode is used to configure off-heap memory >>>>> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does >> not >>>>> have any effect on indexing." >>>>> >>>>> According to [2] - "Note that when off-heap memory is configured, >>> Ignite >>>>> will store query indexes off-heap as well." >>>>> >>>>> >>>>> Which one is true? >>>>> >>>>> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap- >>> sql-indexes >>>>> >>>>> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> -Prachi >>>>> >>> >>> >> |
|
|
Your final statement seems correct to me, but I said exactly that, I don't
know where did you get that setOffHeapMaxMemory must be 0. May be you use some weird email client which renders >> signs wrong? Sergi 2016-11-04 22:25 GMT+03:00 Denis Magda <[hidden email]>: > Sergi, > > Thanks, now I found the exact conditions that switch the indexes from > on-heap to off-heap. > > But looks like you did a misprint in your conclusion saying that > setOffHeapMaxMemory should be 0. > > Thus from the user standpoint we can say, that if either > setOffHeapMaxMemory > >> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will > go > > off-heap. > > In fact, as you elaborated earlier, it has to be >= 0 which leads to the > following final statement: > > SQL engine will store indexes in the off-heap region if > setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported > off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 > automatically if it’s value is less than -1 and one of off-heap memory > modes is used. > > Are we on the same page now? > > — > Denis > > > On Nov 4, 2016, at 1:51 AM, Sergi Vladykin <[hidden email]> > wrote: > > > > Denis, > > > > I checked the code now as well. I actually was wrong, but your statement > is > > wrong either. > > > > If we are talking about indexes then the exact condition when index will > be > > off-heap is the following: either setOffHeapMaxMemory >= 0 or > setMemoryMode > > is OFFHEAP_TIERED. > > > > But if setMemoryMode is set to one of off-heap modes and > setOffHeapMaxMemory is > > -1, then setOffHeapMaxMemory is silently getting switched to 0. > > > > Thus from the user standpoint we can say, that if either > setOffHeapMaxMemory > >> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will > go > > off-heap. > > > > Sergi > > > > 2016-11-04 6:30 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > > >> Do we currently throw an exception if memory mode is set to off-heap by > the > >> max-offheap-memory is set to "-1"? Otherwise, it can get very confusing > to > >> users. > >> > >> On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <[hidden email]> > wrote: > >> > >>> Sergi, > >>> > >>> Referring to your response and the source code I should say that, in > >> fact, > >>> the documentation has to state the following: > >>> > >>> “To place both data and indexes in off heap memory CacheConfiguration. > >> setMemoryMode > >>> has to be set to one of off heap modes and CacheConfiguration. > >> setOffHeapMaxMemory > >>> has to be set to a value different from -1 (disabled). If at least one > of > >>> these conditions is not met then both data and indexes will be located > in > >>> on heap memory” > >>> > >>> Is this statement correct? > >>> > >>> — > >>> Denis > >>> > >>>> On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email]> > >>> wrote: > >>>> > >>>> Prachi, > >>>> > >>>> In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory > >>>> property, so [1] is correct. If setOffHeapMaxMemory is disabled (set > to > >>>> -1), then indexes will never be stored offheap, regardless of > >>> setMemoryMode > >>>> setting. > >>>> > >>>> Probably we have to change [2] to make it clear, that it is about > >>>> setOffHeapMaxMemory > >>>> but not about setMemoryMode. > >>>> > >>>> Sergi > >>>> > >>>> > >>>> 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: > >>>> > >>>>> Engineers, > >>>>> > >>>>> I was going through documentation for Off-Heap SQL Indexes > >>>>> <https://apacheignite.readme.io/docs/sql-queries#off-heap- > sql-indexes > >>> > >>> and > >>>>> found two contradicting statements- > >>>>> > >>>>> According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is > the > >>> only > >>>>> property to enable or disable off-heap indexing. While > >>>>> CacheConfiguration.setMemoryMode is used to configure off-heap > memory > >>>>> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does > >> not > >>>>> have any effect on indexing." > >>>>> > >>>>> According to [2] - "Note that when off-heap memory is configured, > >>> Ignite > >>>>> will store query indexes off-heap as well." > >>>>> > >>>>> > >>>>> Which one is true? > >>>>> > >>>>> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap- > >>> sql-indexes > >>>>> > >>>>> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory > >>>>> > >>>>> > >>>>> Thanks, > >>>>> > >>>>> -Prachi > >>>>> > >>> > >>> > >> > > |
|
|
Sergi,
> May be you use > some weird email client which renders >> signs wrong? Right, my Apple’s Mail agent omitted ‘>’ character from your original statement applying some HTML formatting. Prachi, please update all the docs keeping in my that this sentence is technically correct >> SQL engine will store indexes in the off-heap region if >> setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported >> off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 >> automatically if it’s value is less than -1 and one of off-heap memory >> modes is used. — Denis > On Nov 5, 2016, at 6:23 AM, Sergi Vladykin <[hidden email]> wrote: > > Your final statement seems correct to me, but I said exactly that, I don't > know where did you get that setOffHeapMaxMemory must be 0. May be you use > some weird email client which renders >> signs wrong? > > Sergi > > 2016-11-04 22:25 GMT+03:00 Denis Magda <[hidden email]>: > >> Sergi, >> >> Thanks, now I found the exact conditions that switch the indexes from >> on-heap to off-heap. >> >> But looks like you did a misprint in your conclusion saying that >> setOffHeapMaxMemory should be 0. >>> Thus from the user standpoint we can say, that if either >> setOffHeapMaxMemory >>>> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will >> go >>> off-heap. >> >> In fact, as you elaborated earlier, it has to be >= 0 which leads to the >> following final statement: >> >> SQL engine will store indexes in the off-heap region if >> setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported >> off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 >> automatically if it’s value is less than -1 and one of off-heap memory >> modes is used. >> >> Are we on the same page now? >> >> — >> Denis >> >>> On Nov 4, 2016, at 1:51 AM, Sergi Vladykin <[hidden email]> >> wrote: >>> >>> Denis, >>> >>> I checked the code now as well. I actually was wrong, but your statement >> is >>> wrong either. >>> >>> If we are talking about indexes then the exact condition when index will >> be >>> off-heap is the following: either setOffHeapMaxMemory >= 0 or >> setMemoryMode >>> is OFFHEAP_TIERED. >>> >>> But if setMemoryMode is set to one of off-heap modes and >> setOffHeapMaxMemory is >>> -1, then setOffHeapMaxMemory is silently getting switched to 0. >>> >>> Thus from the user standpoint we can say, that if either >> setOffHeapMaxMemory >>>> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will >> go >>> off-heap. >>> >>> Sergi >>> >>> 2016-11-04 6:30 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: >>> >>>> Do we currently throw an exception if memory mode is set to off-heap by >> the >>>> max-offheap-memory is set to "-1"? Otherwise, it can get very confusing >> to >>>> users. >>>> >>>> On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <[hidden email]> >> wrote: >>>> >>>>> Sergi, >>>>> >>>>> Referring to your response and the source code I should say that, in >>>> fact, >>>>> the documentation has to state the following: >>>>> >>>>> “To place both data and indexes in off heap memory CacheConfiguration. >>>> setMemoryMode >>>>> has to be set to one of off heap modes and CacheConfiguration. >>>> setOffHeapMaxMemory >>>>> has to be set to a value different from -1 (disabled). If at least one >> of >>>>> these conditions is not met then both data and indexes will be located >> in >>>>> on heap memory” >>>>> >>>>> Is this statement correct? >>>>> >>>>> — >>>>> Denis >>>>> >>>>>> On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email]> >>>>> wrote: >>>>>> >>>>>> Prachi, >>>>>> >>>>>> In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory >>>>>> property, so [1] is correct. If setOffHeapMaxMemory is disabled (set >> to >>>>>> -1), then indexes will never be stored offheap, regardless of >>>>> setMemoryMode >>>>>> setting. >>>>>> >>>>>> Probably we have to change [2] to make it clear, that it is about >>>>>> setOffHeapMaxMemory >>>>>> but not about setMemoryMode. >>>>>> >>>>>> Sergi >>>>>> >>>>>> >>>>>> 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: >>>>>> >>>>>>> Engineers, >>>>>>> >>>>>>> I was going through documentation for Off-Heap SQL Indexes >>>>>>> <https://apacheignite.readme.io/docs/sql-queries#off-heap- >> sql-indexes >>>>> >>>>> and >>>>>>> found two contradicting statements- >>>>>>> >>>>>>> According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is >> the >>>>> only >>>>>>> property to enable or disable off-heap indexing. While >>>>>>> CacheConfiguration.setMemoryMode is used to configure off-heap >> memory >>>>>>> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does >>>> not >>>>>>> have any effect on indexing." >>>>>>> >>>>>>> According to [2] - "Note that when off-heap memory is configured, >>>>> Ignite >>>>>>> will store query indexes off-heap as well." >>>>>>> >>>>>>> >>>>>>> Which one is true? >>>>>>> >>>>>>> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap- >>>>> sql-indexes >>>>>>> >>>>>>> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> -Prachi >>>>>>> >>>>> >>>>> >>>> >> >> |
|
|
Fixed. Please review -
http://apacheignite.gridgain.org/v1.7/docs/sql-queries#off-heap-sql-indexes -P On Mon, Nov 7, 2016 at 9:36 AM, Denis Magda <[hidden email]> wrote: > Sergi, > > May be you use > some weird email client which renders >> signs wrong? > > > Right, my Apple’s Mail agent omitted ‘>’ character from your original > statement applying some HTML formatting. > > *Prachi*, please update all the docs keeping in my that this sentence is > technically correct > > SQL engine will store indexes in the off-heap region if > setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported > off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 > automatically if it’s value is less than -1 and one of off-heap memory > modes is used. > > > — > Denis > > On Nov 5, 2016, at 6:23 AM, Sergi Vladykin <[hidden email]> > wrote: > > Your final statement seems correct to me, but I said exactly that, I don't > know where did you get that setOffHeapMaxMemory must be 0. May be you use > some weird email client which renders >> signs wrong? > > Sergi > > 2016-11-04 22:25 GMT+03:00 Denis Magda <[hidden email]>: > > Sergi, > > Thanks, now I found the exact conditions that switch the indexes from > on-heap to off-heap. > > But looks like you did a misprint in your conclusion saying that > setOffHeapMaxMemory should be 0. > > Thus from the user standpoint we can say, that if either > > setOffHeapMaxMemory > > = 0 or setMemoryMode is set to any of off-heap modes, then indexes will > > go > > off-heap. > > > In fact, as you elaborated earlier, it has to be >= 0 which leads to the > following final statement: > > SQL engine will store indexes in the off-heap region if > setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported > off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 > automatically if it’s value is less than -1 and one of off-heap memory > modes is used. > > Are we on the same page now? > > — > Denis > > On Nov 4, 2016, at 1:51 AM, Sergi Vladykin <[hidden email]> > > wrote: > > > Denis, > > I checked the code now as well. I actually was wrong, but your statement > > is > > wrong either. > > If we are talking about indexes then the exact condition when index will > > be > > off-heap is the following: either setOffHeapMaxMemory >= 0 or > > setMemoryMode > > is OFFHEAP_TIERED. > > But if setMemoryMode is set to one of off-heap modes and > > setOffHeapMaxMemory is > > -1, then setOffHeapMaxMemory is silently getting switched to 0. > > Thus from the user standpoint we can say, that if either > > setOffHeapMaxMemory > > = 0 or setMemoryMode is set to any of off-heap modes, then indexes will > > go > > off-heap. > > Sergi > > 2016-11-04 6:30 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > Do we currently throw an exception if memory mode is set to off-heap by > > the > > max-offheap-memory is set to "-1"? Otherwise, it can get very confusing > > to > > users. > > On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <[hidden email]> > > wrote: > > > Sergi, > > Referring to your response and the source code I should say that, in > > fact, > > the documentation has to state the following: > > “To place both data and indexes in off heap memory CacheConfiguration. > > setMemoryMode > > has to be set to one of off heap modes and CacheConfiguration. > > setOffHeapMaxMemory > > has to be set to a value different from -1 (disabled). If at least one > > of > > these conditions is not met then both data and indexes will be located > > in > > on heap memory” > > Is this statement correct? > > — > Denis > > On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email]> > > wrote: > > > Prachi, > > In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory > property, so [1] is correct. If setOffHeapMaxMemory is disabled (set > > to > > -1), then indexes will never be stored offheap, regardless of > > setMemoryMode > > setting. > > Probably we have to change [2] to make it clear, that it is about > setOffHeapMaxMemory > but not about setMemoryMode. > > Sergi > > > 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email]>: > > Engineers, > > I was going through documentation for Off-Heap SQL Indexes > <https://apacheignite.readme.io/docs/sql-queries#off-heap- > > sql-indexes > > > and > > found two contradicting statements- > > According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is > > the > > only > > property to enable or disable off-heap indexing. While > CacheConfiguration.setMemoryMode is used to configure off-heap > > memory > > <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>, it does > > not > > have any effect on indexing." > > According to [2] - "Note that when off-heap memory is configured, > > Ignite > > will store query indexes off-heap as well." > > > Which one is true? > > [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap- > > sql-indexes > > > [2] - http://apacheignite.gridgain.org/docs/off-heap-memory > > > Thanks, > > -Prachi > > > > > > > > |
|
|
Prachi, thanks, looks good to be.
BTW, I’ve started refining the overall SQL documentation. At the moment I’ve created a new section named “SQL” located right below “Data Grid” and that sections contains a dedicated page about the indexes. So the new link to the page you’re referring to is the following: https://apacheignite.readme.io/docs/indexes#off-heap-sql-indexes <https://apacheignite.readme.io/docs/indexes#off-heap-sql-indexes> — Denis > On Nov 21, 2016, at 12:32 PM, Prachi Garg <[hidden email]> wrote: > > Fixed. Please review - http://apacheignite.gridgain.org/v1.7/docs/sql-queries#off-heap-sql-indexes <http://apacheignite.gridgain.org/v1.7/docs/sql-queries#off-heap-sql-indexes> > > -P > > On Mon, Nov 7, 2016 at 9:36 AM, Denis Magda <[hidden email] <mailto:[hidden email]>> wrote: > Sergi, > >> May be you use >> some weird email client which renders >> signs wrong? > > Right, my Apple’s Mail agent omitted ‘>’ character from your original statement applying some HTML formatting. > > Prachi, please update all the docs keeping in my that this sentence is technically correct > >>> SQL engine will store indexes in the off-heap region if >>> setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported >>> off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 >>> automatically if it’s value is less than -1 and one of off-heap memory >>> modes is used. > > — > Denis > >> On Nov 5, 2016, at 6:23 AM, Sergi Vladykin <[hidden email] <mailto:[hidden email]>> wrote: >> >> Your final statement seems correct to me, but I said exactly that, I don't >> know where did you get that setOffHeapMaxMemory must be 0. May be you use >> some weird email client which renders >> signs wrong? >> >> Sergi >> >> 2016-11-04 22:25 GMT+03:00 Denis Magda <[hidden email] <mailto:[hidden email]>>: >> >>> Sergi, >>> >>> Thanks, now I found the exact conditions that switch the indexes from >>> on-heap to off-heap. >>> >>> But looks like you did a misprint in your conclusion saying that >>> setOffHeapMaxMemory should be 0. >>>> Thus from the user standpoint we can say, that if either >>> setOffHeapMaxMemory >>>>> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will >>> go >>>> off-heap. >>> >>> In fact, as you elaborated earlier, it has to be >= 0 which leads to the >>> following final statement: >>> >>> SQL engine will store indexes in the off-heap region if >>> setOffHeapMaxMemory parameter >= 0 or setMemoryMode is set to any supported >>> off-heap mode. It worth to mention that setOffHeapMaxMemory is set to 0 >>> automatically if it’s value is less than -1 and one of off-heap memory >>> modes is used. >>> >>> Are we on the same page now? >>> >>> — >>> Denis >>> >>>> On Nov 4, 2016, at 1:51 AM, Sergi Vladykin <[hidden email] <mailto:[hidden email]>> >>> wrote: >>>> >>>> Denis, >>>> >>>> I checked the code now as well. I actually was wrong, but your statement >>> is >>>> wrong either. >>>> >>>> If we are talking about indexes then the exact condition when index will >>> be >>>> off-heap is the following: either setOffHeapMaxMemory >= 0 or >>> setMemoryMode >>>> is OFFHEAP_TIERED. >>>> >>>> But if setMemoryMode is set to one of off-heap modes and >>> setOffHeapMaxMemory is >>>> -1, then setOffHeapMaxMemory is silently getting switched to 0. >>>> >>>> Thus from the user standpoint we can say, that if either >>> setOffHeapMaxMemory >>>>> = 0 or setMemoryMode is set to any of off-heap modes, then indexes will >>> go >>>> off-heap. >>>> >>>> Sergi >>>> >>>> 2016-11-04 6:30 GMT+03:00 Dmitriy Setrakyan <[hidden email] <mailto:[hidden email]>>: >>>> >>>>> Do we currently throw an exception if memory mode is set to off-heap by >>> the >>>>> max-offheap-memory is set to "-1"? Otherwise, it can get very confusing >>> to >>>>> users. >>>>> >>>>> On Thu, Nov 3, 2016 at 10:22 AM, Denis Magda <[hidden email] <mailto:[hidden email]>> >>> wrote: >>>>> >>>>>> Sergi, >>>>>> >>>>>> Referring to your response and the source code I should say that, in >>>>> fact, >>>>>> the documentation has to state the following: >>>>>> >>>>>> “To place both data and indexes in off heap memory CacheConfiguration. >>>>> setMemoryMode >>>>>> has to be set to one of off heap modes and CacheConfiguration. >>>>> setOffHeapMaxMemory >>>>>> has to be set to a value different from -1 (disabled). If at least one >>> of >>>>>> these conditions is not met then both data and indexes will be located >>> in >>>>>> on heap memory” >>>>>> >>>>>> Is this statement correct? >>>>>> >>>>>> — >>>>>> Denis >>>>>> >>>>>>> On Nov 3, 2016, at 2:53 AM, Sergi Vladykin <[hidden email] <mailto:[hidden email]>> >>>>>> wrote: >>>>>>> >>>>>>> Prachi, >>>>>>> >>>>>>> In [2] we refer to the same CacheConfiguration.setOffHeapMaxMemory >>>>>>> property, so [1] is correct. If setOffHeapMaxMemory is disabled (set >>> to >>>>>>> -1), then indexes will never be stored offheap, regardless of >>>>>> setMemoryMode >>>>>>> setting. >>>>>>> >>>>>>> Probably we have to change [2] to make it clear, that it is about >>>>>>> setOffHeapMaxMemory >>>>>>> but not about setMemoryMode. >>>>>>> >>>>>>> Sergi >>>>>>> >>>>>>> >>>>>>> 2016-11-03 2:25 GMT+03:00 Prachi Garg <[hidden email] <mailto:[hidden email]>>: >>>>>>> >>>>>>>> Engineers, >>>>>>>> >>>>>>>> I was going through documentation for Off-Heap SQL Indexes >>>>>>>> <https://apacheignite.readme.io/docs/sql-queries#off-heap- <https://apacheignite.readme.io/docs/sql-queries#off-heap-> >>> sql-indexes >>>>>> >>>>>> and >>>>>>>> found two contradicting statements- >>>>>>>> >>>>>>>> According to [1] - " `CacheConfiguration.setOffHeapMaxMemory` is >>> the >>>>>> only >>>>>>>> property to enable or disable off-heap indexing. While >>>>>>>> CacheConfiguration.setMemoryMode is used to configure off-heap >>> memory >>>>>>>> <https://apacheignite.readme.io/v1.7/docs/off-heap-memory <https://apacheignite.readme.io/v1.7/docs/off-heap-memory>>, it does >>>>> not >>>>>>>> have any effect on indexing." >>>>>>>> >>>>>>>> According to [2] - "Note that when off-heap memory is configured, >>>>>> Ignite >>>>>>>> will store query indexes off-heap as well." >>>>>>>> >>>>>>>> >>>>>>>> Which one is true? >>>>>>>> >>>>>>>> [1] - https://apacheignite.readme.io/docs/sql-queries#off-heap- <https://apacheignite.readme.io/docs/sql-queries#off-heap-> >>>>>> sql-indexes >>>>>>>> >>>>>>>> [2] - http://apacheignite.gridgain.org/docs/off-heap-memory <http://apacheignite.gridgain.org/docs/off-heap-memory> >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> -Prachi >>>>>>>> >>>>>> >>>>>> >>>>> >>> >>> > > |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |