More efficient writes of messages in direct mode.

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

More efficient writes of messages in direct mode.

Vladimir Ozerov
Igniters,

Several observations on our "direct" message writes:

1) For some reason we write null arrays, maps and ints as "-1" integer,
while it can be encoded in a single byte. And our cache messages have
looooots of containers, which are commonly null.

2) Some frequently used data types could benefit significantly if integers
are written in compacted form, while others could not. Several examples:

GridCacheVersion:
    topVer - fits in 2 bytes instead of 4;
    nodeOrderDrId -  1 instead of 4;
    globalTime -     6 instead of 8;
    order -          6 instead of 8.
Result: shrunk from 24 to 15 bytes with minimal overhead.

AffinityTopologyVersion:
    topVer -      1-2 bytes instead of 8.
    minorTopVer - 1 byte instead of 4;
Result: shrunked from 12 to 2-3 bytes in 99% cases.

IgniteUuid - takes 24 bytes, but can be shrunk only to 22 bytes => bad
candidate for optimziation.

I picked random test and set breakpoint into random cache message -
GridDhtTxPrepareRequest. It was 444 bytes long with a single int-int pair
as a payload. At least 90 bytes (~20%) could be saved easily with described
optimziations.

Looks like we should pay more attention to it.

Vladimir.
Reply | Threaded
Open this post in threaded view
|

Re: More efficient writes of messages in direct mode.

dsetrakyan
Excellent points, and does not seem hard to do.

On Fri, Sep 25, 2015 at 7:06 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Igniters,
>
> Several observations on our "direct" message writes:
>
> 1) For some reason we write null arrays, maps and ints as "-1" integer,
> while it can be encoded in a single byte. And our cache messages have
> looooots of containers, which are commonly null.
>
> 2) Some frequently used data types could benefit significantly if integers
> are written in compacted form, while others could not. Several examples:
>
> GridCacheVersion:
>     topVer - fits in 2 bytes instead of 4;
>     nodeOrderDrId -  1 instead of 4;
>     globalTime -     6 instead of 8;
>     order -          6 instead of 8.
> Result: shrunk from 24 to 15 bytes with minimal overhead.
>
> AffinityTopologyVersion:
>     topVer -      1-2 bytes instead of 8.
>     minorTopVer - 1 byte instead of 4;
> Result: shrunked from 12 to 2-3 bytes in 99% cases.
>
> IgniteUuid - takes 24 bytes, but can be shrunk only to 22 bytes => bad
> candidate for optimziation.
>
> I picked random test and set breakpoint into random cache message -
> GridDhtTxPrepareRequest. It was 444 bytes long with a single int-int pair
> as a payload. At least 90 bytes (~20%) could be saved easily with described
> optimziations.
>
> Looks like we should pay more attention to it.
>
> Vladimir.
>