Folks,
We have some very strange logic for collections in BinaryMarshaller: https://issues.apache.org/jira/browse/IGNITE-2099 Instead of writing them as any other objects, we treat them specially and write in unified form: [collection_flag] + [collection_type_flag] + [size] + [items]. I remember we introduced this in good old GridGain days to handle interoperability issues between Java and .NET. This design is badly wrong: 1) Users cannot use their custom collections without following some ridiculous requirements. 2) We loose some important collection-specific data. E.g., comparators in sorted collections, load factor, etc.. I think we must remove all this weird logic and always write collections as any other objects. The only exception are BinaryReader.readCollection/readMap and BinaryWriter.writeCollection/writeMap methods. In this case we write collections in unified format and it is user responsibility to explain how exactly to deserialize such fields using either some factory or explicitly provided class name. If user needs interoperability - sorry, Java TreeMap is not interoperable with .NET SortedDictionary - these are different types. But user can always create a *wrapper over collection* and make it interoperable. Thoughts? Vladimir. |
I like the idea, however it has obvious downsides. First, if a user class
contains a collection, we force user to implement additional interface, even if the collection is a simple ArrayList. Second, I do not see how this plain collection can be the value for the cache - user will always need to write a wrapper/containing class around it. I think we should provide minimum support for basic types - HashMap, LinkedHashMap, ArrayList and treat other classes the way Vladimir described. |
Alex,
What interface do you mean? If user has collection field in class and explicitly call BinaryWriter.writeCollection(), we can leave current interoperability support - it is not a problem. As per your second point - user could pass collections e.g. as argument to Java task started from .NET. This is where we will loose interoperabiltiy and will force user to create some wrappers. But these are very specific use cases. BTW, proposed solution is almost exactly how we work with collections in .NET. On Tue, Dec 8, 2015 at 4:57 PM, Alexey Goncharuk <[hidden email] > wrote: > I like the idea, however it has obvious downsides. First, if a user class > contains a collection, we force user to implement additional interface, > even if the collection is a simple ArrayList. Second, I do not see how this > plain collection can be the value for the cache - user will always need to > write a wrapper/containing class around it. > > I think we should provide minimum support for basic types - HashMap, > LinkedHashMap, ArrayList and treat other classes the way Vladimir > described. > |
Vladimir,
I believe the default collections in Java and .NET should be supported out of the box. Moreover, if we know the collection type, e.g. HashMap, we can always provide a more efficient way of serializing it ourselves, in the Binary marshaller. Is this something you had in mind, or were you proposing something different. D. On Tue, Dec 8, 2015 at 6:29 AM, Vladimir Ozerov <[hidden email]> wrote: > Alex, > > What interface do you mean? If user has collection field in class and > explicitly call BinaryWriter.writeCollection(), we can leave current > interoperability support - it is not a problem. > As per your second point - user could pass collections e.g. as argument to > Java task started from .NET. This is where we will loose interoperabiltiy > and will force user to create some wrappers. But these are very specific > use cases. > > BTW, proposed solution is almost exactly how we work with collections in > .NET. > > On Tue, Dec 8, 2015 at 4:57 PM, Alexey Goncharuk < > [hidden email] > > wrote: > > > I like the idea, however it has obvious downsides. First, if a user class > > contains a collection, we force user to implement additional interface, > > even if the collection is a simple ArrayList. Second, I do not see how > this > > plain collection can be the value for the cache - user will always need > to > > write a wrapper/containing class around it. > > > > I think we should provide minimum support for basic types - HashMap, > > LinkedHashMap, ArrayList and treat other classes the way Vladimir > > described. > > > |
Dmitriy,
Correct. This is what I meant. On Tue, Dec 8, 2015 at 10:26 PM, Dmitriy Setrakyan <[hidden email]> wrote: > Vladimir, > > I believe the default collections in Java and .NET should be supported out > of the box. Moreover, if we know the collection type, e.g. HashMap, we can > always provide a more efficient way of serializing it ourselves, in the > Binary marshaller. > > Is this something you had in mind, or were you proposing something > different. > > D. > > On Tue, Dec 8, 2015 at 6:29 AM, Vladimir Ozerov <[hidden email]> > wrote: > > > Alex, > > > > What interface do you mean? If user has collection field in class and > > explicitly call BinaryWriter.writeCollection(), we can leave current > > interoperability support - it is not a problem. > > As per your second point - user could pass collections e.g. as argument > to > > Java task started from .NET. This is where we will loose interoperabiltiy > > and will force user to create some wrappers. But these are very specific > > use cases. > > > > BTW, proposed solution is almost exactly how we work with collections in > > .NET. > > > > On Tue, Dec 8, 2015 at 4:57 PM, Alexey Goncharuk < > > [hidden email] > > > wrote: > > > > > I like the idea, however it has obvious downsides. First, if a user > class > > > contains a collection, we force user to implement additional interface, > > > even if the collection is a simple ArrayList. Second, I do not see how > > this > > > plain collection can be the value for the cache - user will always need > > to > > > write a wrapper/containing class around it. > > > > > > I think we should provide minimum support for basic types - HashMap, > > > LinkedHashMap, ArrayList and treat other classes the way Vladimir > > > described. > > > > > > |
I am going to drop support of the following classes. All these classes are
currently written in a special unified way. After my changes they will be written *as any other usual object*. 1) Sorted collections: - TreeSet - TreeMap - ConcurrentSkipListSet Reason: currently we do not write Comparator which might lead to exceptions on the other side. 2) Special collections: - ConcurrentLinkedQueue - ConcurrentHashMap - PropertiesMap Reason: too specific use cases. 3) Map.Entry Reason: this is an interface with billion implemenations, we cannot unmarshal it correctly on the other side without full serialization of concrete class. On Wed, Dec 9, 2015 at 10:57 AM, Vladimir Ozerov <[hidden email]> wrote: > Dmitriy, > Correct. This is what I meant. > > On Tue, Dec 8, 2015 at 10:26 PM, Dmitriy Setrakyan <[hidden email]> > wrote: > >> Vladimir, >> >> I believe the default collections in Java and .NET should be supported out >> of the box. Moreover, if we know the collection type, e.g. HashMap, we can >> always provide a more efficient way of serializing it ourselves, in the >> Binary marshaller. >> >> Is this something you had in mind, or were you proposing something >> different. >> >> D. >> >> On Tue, Dec 8, 2015 at 6:29 AM, Vladimir Ozerov <[hidden email]> >> wrote: >> >> > Alex, >> > >> > What interface do you mean? If user has collection field in class and >> > explicitly call BinaryWriter.writeCollection(), we can leave current >> > interoperability support - it is not a problem. >> > As per your second point - user could pass collections e.g. as argument >> to >> > Java task started from .NET. This is where we will loose >> interoperabiltiy >> > and will force user to create some wrappers. But these are very specific >> > use cases. >> > >> > BTW, proposed solution is almost exactly how we work with collections in >> > .NET. >> > >> > On Tue, Dec 8, 2015 at 4:57 PM, Alexey Goncharuk < >> > [hidden email] >> > > wrote: >> > >> > > I like the idea, however it has obvious downsides. First, if a user >> class >> > > contains a collection, we force user to implement additional >> interface, >> > > even if the collection is a simple ArrayList. Second, I do not see how >> > this >> > > plain collection can be the value for the cache - user will always >> need >> > to >> > > write a wrapper/containing class around it. >> > > >> > > I think we should provide minimum support for basic types - HashMap, >> > > LinkedHashMap, ArrayList and treat other classes the way Vladimir >> > > described. >> > > >> > >> > > |
Free forum by Nabble | Edit this page |