Guys,
I notices we do not have support for packed ints and longs in raw binary API [1] [2] Such methods are essential for implementing efficient custom compression schemes. Their addition can simplify implementing custom serializers for the cases then default binary marshaller is not enough, without additional library dependencies. Proposed API extension for rawReader/rawWriter: org.apache.ignite.binary.BinaryRawWriter#writePackedInt org.apache.ignite.binary.BinaryRawWriter#writePackedLong org.apache.ignite.binary.BinaryRawReader#readPackedInt org.apache.ignite.binary.BinaryRawReader#readPackedLong JIRA ticket: [3] Thoughs ? [1] org.apache.ignite.binary.BinaryRawReader [2] org.apache.ignite.binary.BinaryRawWriter [3] https://issues.apache.org/jira/browse/IGNITE-6426 -- Best regards, Alexei Scherbakov |
Alexey, are you talking about arrays of ints and longs?
On Fri, Sep 29, 2017 at 3:29 AM, Alexei Scherbakov < [hidden email]> wrote: > Guys, > > I notices we do not have support for packed ints and longs in raw binary > API [1] [2] > > Such methods are essential for implementing efficient custom compression > schemes. > > Their addition can simplify implementing custom serializers for the cases > then default binary marshaller is not enough, without additional library > dependencies. > > Proposed API extension for rawReader/rawWriter: > > org.apache.ignite.binary.BinaryRawWriter#writePackedInt > > org.apache.ignite.binary.BinaryRawWriter#writePackedLong > > org.apache.ignite.binary.BinaryRawReader#readPackedInt > > org.apache.ignite.binary.BinaryRawReader#readPackedLong > > JIRA ticket: [3] > > Thoughs ? > > [1] org.apache.ignite.binary.BinaryRawReader > > [2] org.apache.ignite.binary.BinaryRawWriter > > [3] https://issues.apache.org/jira/browse/IGNITE-6426 > > -- > > Best regards, > Alexei Scherbakov > |
Dmitriy,
Not arrays, just primitives. Using special binary representation, ints and longs can be represented by 1-9 bytes, depending on their cardinality. I want to add such implementation in raw binary API to help implementing custom serializers. 2017-09-29 15:12 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > Alexey, are you talking about arrays of ints and longs? > > On Fri, Sep 29, 2017 at 3:29 AM, Alexei Scherbakov < > [hidden email]> wrote: > > > Guys, > > > > I notices we do not have support for packed ints and longs in raw binary > > API [1] [2] > > > > Such methods are essential for implementing efficient custom compression > > schemes. > > > > Their addition can simplify implementing custom serializers for the cases > > then default binary marshaller is not enough, without additional library > > dependencies. > > > > Proposed API extension for rawReader/rawWriter: > > > > org.apache.ignite.binary.BinaryRawWriter#writePackedInt > > > > org.apache.ignite.binary.BinaryRawWriter#writePackedLong > > > > org.apache.ignite.binary.BinaryRawReader#readPackedInt > > > > org.apache.ignite.binary.BinaryRawReader#readPackedLong > > > > JIRA ticket: [3] > > > > Thoughs ? > > > > [1] org.apache.ignite.binary.BinaryRawReader > > > > [2] org.apache.ignite.binary.BinaryRawWriter > > > > [3] https://issues.apache.org/jira/browse/IGNITE-6426 > > > > -- > > > > Best regards, > > Alexei Scherbakov > > > -- Best regards, Alexei Scherbakov |
Alexey,
Any reason why you propose this only for raw data? I think this should be an option for binary marshaller (probably per type). If we also add special methods to Binarylizable, then they should be applied to both raw and non-raw data. BTW, there are couple other tickets regarding this, and looks like there is already some progress: - https://issues.apache.org/jira/browse/IGNITE-5097 - https://issues.apache.org/jira/browse/IGNITE-6418 -Val On Fri, Sep 29, 2017 at 5:37 AM, Alexei Scherbakov < [hidden email]> wrote: > Dmitriy, > > Not arrays, just primitives. > > Using special binary representation, ints and longs can be represented by > 1-9 bytes, depending on their cardinality. > > I want to add such implementation in raw binary API to help implementing > custom serializers. > > > > 2017-09-29 15:12 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > > > Alexey, are you talking about arrays of ints and longs? > > > > On Fri, Sep 29, 2017 at 3:29 AM, Alexei Scherbakov < > > [hidden email]> wrote: > > > > > Guys, > > > > > > I notices we do not have support for packed ints and longs in raw > binary > > > API [1] [2] > > > > > > Such methods are essential for implementing efficient custom > compression > > > schemes. > > > > > > Their addition can simplify implementing custom serializers for the > cases > > > then default binary marshaller is not enough, without additional > library > > > dependencies. > > > > > > Proposed API extension for rawReader/rawWriter: > > > > > > org.apache.ignite.binary.BinaryRawWriter#writePackedInt > > > > > > org.apache.ignite.binary.BinaryRawWriter#writePackedLong > > > > > > org.apache.ignite.binary.BinaryRawReader#readPackedInt > > > > > > org.apache.ignite.binary.BinaryRawReader#readPackedLong > > > > > > JIRA ticket: [3] > > > > > > Thoughs ? > > > > > > [1] org.apache.ignite.binary.BinaryRawReader > > > > > > [2] org.apache.ignite.binary.BinaryRawWriter > > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-6426 > > > > > > -- > > > > > > Best regards, > > > Alexei Scherbakov > > > > > > > > > -- > > Best regards, > Alexei Scherbakov > |
Free forum by Nabble | Edit this page |