Hello Igniters,
I want to propose one more method for memory metrics - segmentFillVariance, float value from 0.0 to 1.0. Motivation: when persistence is enabled, we split memory policy into K equal segments (K = concurrency level). We calculate hash of pageId to determine segment where page will be stored. Pages eviction to disk starts when segment is full. If hash function is bad enough, one segment can overflow when there are lots of free space, and evictions can start too early. We want to be able to distinguish such situations. I propose to calculate segmentFillVariance as difference between maximum and minimum percentage of segment fill. Greater variance signals about worse hash function. Thoughts? -- Best Regards, Ivan Rakov |
Ivan,
How about renaming it to hashFunctionScore? This will be more digestable to users. "segmentFillVariance" sounds too scientific even to me. D. On Tue, Jul 18, 2017 at 10:39 AM, Ivan Rakov <[hidden email]> wrote: > Hello Igniters, > > I want to propose one more method for memory metrics - > segmentFillVariance, float value from 0.0 to 1.0. > > Motivation: when persistence is enabled, we split memory policy into K > equal segments (K = concurrency level). We calculate hash of pageId to > determine segment where page will be stored. > Pages eviction to disk starts when segment is full. If hash function is > bad enough, one segment can overflow when there are lots of free space, and > evictions can start too early. We want to be able to distinguish such > situations. > > I propose to calculate segmentFillVariance as difference between maximum > and minimum percentage of segment fill. Greater variance signals about > worse hash function. > > Thoughts? > > -- > Best Regards, > Ivan Rakov > > |
Dmitriy,
I agree, hashFunctionScore is more comprehensible. I'd correct it to "pageHashFunctionScore" to make it clear that it's about function that distributes pages, not about hashCode() of user-provided keys. -- Best Regards, Ivan Rakov On 18.07.2017 19:34, Dmitriy Setrakyan wrote: > Ivan, > > How about renaming it to hashFunctionScore? This will be more digestable to > users. "segmentFillVariance" sounds too scientific even to me. > > D. > > On Tue, Jul 18, 2017 at 10:39 AM, Ivan Rakov <[hidden email]> wrote: > >> Hello Igniters, >> >> I want to propose one more method for memory metrics - >> segmentFillVariance, float value from 0.0 to 1.0. >> >> Motivation: when persistence is enabled, we split memory policy into K >> equal segments (K = concurrency level). We calculate hash of pageId to >> determine segment where page will be stored. >> Pages eviction to disk starts when segment is full. If hash function is >> bad enough, one segment can overflow when there are lots of free space, and >> evictions can start too early. We want to be able to distinguish such >> situations. >> >> I propose to calculate segmentFillVariance as difference between maximum >> and minimum percentage of segment fill. Greater variance signals about >> worse hash function. >> >> Thoughts? >> >> -- >> Best Regards, >> Ivan Rakov >> >> |
Sounds good!
D. On Jul 19, 2017, 1:54 PM, at 1:54 PM, Ivan Rakov <[hidden email]> wrote: >Dmitriy, > >I agree, hashFunctionScore is more comprehensible. >I'd correct it to "pageHashFunctionScore" to make it clear that it's >about function that distributes pages, not about hashCode() of >user-provided keys. > >-- >Best Regards, >Ivan Rakov > >On 18.07.2017 19:34, Dmitriy Setrakyan wrote: >> Ivan, >> >> How about renaming it to hashFunctionScore? This will be more >digestable to >> users. "segmentFillVariance" sounds too scientific even to me. >> >> D. >> >> On Tue, Jul 18, 2017 at 10:39 AM, Ivan Rakov <[hidden email]> >wrote: >> >>> Hello Igniters, >>> >>> I want to propose one more method for memory metrics - >>> segmentFillVariance, float value from 0.0 to 1.0. >>> >>> Motivation: when persistence is enabled, we split memory policy into >K >>> equal segments (K = concurrency level). We calculate hash of pageId >to >>> determine segment where page will be stored. >>> Pages eviction to disk starts when segment is full. If hash function >is >>> bad enough, one segment can overflow when there are lots of free >space, and >>> evictions can start too early. We want to be able to distinguish >such >>> situations. >>> >>> I propose to calculate segmentFillVariance as difference between >maximum >>> and minimum percentage of segment fill. Greater variance signals >about >>> worse hash function. >>> >>> Thoughts? >>> >>> -- >>> Best Regards, >>> Ivan Rakov >>> >>> |
Free forum by Nabble | Edit this page |