Storage Class Memory and Persistent Collections

Hello,

 

Clearly Storage Class Memory represents a breakthrough for "in memory" grids
and some people already tried it on Ignite :
http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.ht
ml

I would like to know what is the position of the community towards  the
Persistent Collections (
<https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=I
STV_2017>
https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=IS
TV_2017,  <https://pcollections.org/> https://pcollections.org/)?
According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
massively reduce the marshalling/unmarshalling time. `

1)   Do you guys plan to test it and to support SCM such as 3DXPoint
natively?

2)   Would it mean that ignite would not serialize the objects off heap
anymore?

 

Hope it makes sense.

 

Best Regards

[dmagda]

Hello Steve,

Intel folks are already contributing Intel Optane Persistent Memory support
to Ignite:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
http://apache-ignite-developers.2346864.n4.nabble.com/Adding-experimental-support-for-Intel-Optane-DC-Persistent-Memory-td33041.html

Hopefully, the contribution will be reviewed and accepted soon.

--
Denis

On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
[hidden email]> wrote:

> Hello,
>
>
>
> Clearly Storage Class Memory represents a breakthrough for "in memory"
> grids
> and some people already tried it on Ignite :
>
> http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.ht
> ml
> <http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.html>
>
> I would like to know what is the position of the community towards  the
> Persistent Collections (
> <
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=I
> STV_2017
> <https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017>
> >
>
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=IS
> TV_2017
> <https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017>,
> <https://pcollections.org/> https://pcollections.org/)?
> According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> massively reduce the marshalling/unmarshalling time. `
>
> 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> natively?
>
> 2)   Would it mean that ignite would not serialize the objects off heap
> anymore?
>
>
>
> Hope it makes sense.
>
>
>
> Best Regards
>
>

That's great but would you mind explaining to me how we can get rid of the
marshalling/unmarshalling? Because that would significantly speed up the
processes that run on the grid.

On Wed, Aug 22, 2018 at 3:36 AM Denis Magda <[hidden email]> wrote:

> Hello Steve,
>
> Intel folks are already contributing Intel Optane Persistent Memory support
> to Ignite:
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-experimental-support-for-Intel-Optane-DC-Persistent-Memory-td33041.html
>
> Hopefully, the contribution will be reviewed and accepted soon.
>
> --
> Denis
>
> On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
> [hidden email]> wrote:
>
> > Hello,
> >
> >
> >
> > Clearly Storage Class Memory represents a breakthrough for "in memory"
> > grids
> > and some people already tried it on Ignite :
> >
> >
> http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.ht
> > ml
> > <
> http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.html
> >
> >
> > I would like to know what is the position of the community towards  the
> > Persistent Collections (
> > <
> >
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=I
> > STV_2017
> > <
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017
> >
> > >
> >
> >
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=IS
> > TV_2017
> > <
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017
> >,
> > <https://pcollections.org/> https://pcollections.org/)?
> > According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> > massively reduce the marshalling/unmarshalling time. `
> >
> > 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> > natively?
> >
> > 2)   Would it mean that ignite would not serialize the objects off heap
> > anymore?
> >
> >
> >
> > Hope it makes sense.
> >
> >
> >
> > Best Regards
> >
> >
>

--
===========
Steve Hostettler

Université de Genève
CUI - Battelle bat. A
Route de Drize, 7
CH-1227 Carouge
Tel. [phone]
Fax [phone]

[Pavel Kovalenko]

Hello Steve,

I've looked at Persistent Collectors library, but don't see any big
advantages to use it inside Ignite right now.
Why do you think, that marshalling/unmarshalling process will be speeded up
with this library? As I can see there are no explicit changes in
serialization process, just added persistent ability to regular java
collections.

2018-08-23 9:53 GMT+03:00 Steve Hostettler <[hidden email]>:

> That's great but would you mind explaining to me how we can get rid of the
> marshalling/unmarshalling? Because that would significantly speed up the
> processes that run on the grid.
>
> On Wed, Aug 22, 2018 at 3:36 AM Denis Magda <[hidden email]> wrote:
>
> > Hello Steve,
> >
> > Intel folks are already contributing Intel Optane Persistent Memory
> support
> > to Ignite:
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
> >
> > http://apache-ignite-developers.2346864.n4.nabble.
> com/Adding-experimental-support-for-Intel-Optane-DC-
> Persistent-Memory-td33041.html
> >
> > Hopefully, the contribution will be reviewed and accepted soon.
> >
> > --
> > Denis
> >
> > On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
> > [hidden email]> wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > > Clearly Storage Class Memory represents a breakthrough for "in memory"
> > > grids
> > > and some people already tried it on Ignite :
> > >
> > >
> > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> ssds-verified-on.ht
> > > ml
> > > <
> > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> ssds-verified-on.html
> > >
> > >
> > > I would like to know what is the position of the community towards  the
> > > Persistent Collections (
> > > <
> > >
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=I
> > > STV_2017
> > > <
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=ISTV_2017
> > >
> > > >
> > >
> > >
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=IS
> > > TV_2017
> > > <
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=ISTV_2017
> > >,
> > > <https://pcollections.org/> https://pcollections.org/)?
> > > According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> > > massively reduce the marshalling/unmarshalling time. `
> > >
> > > 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> > > natively?
> > >
> > > 2)   Would it mean that ignite would not serialize the objects off heap
> > > anymore?
> > >
> > >
> > >
> > > Hope it makes sense.
> > >
> > >
> > >
> > > Best Regards
> > >
> > >
> >
>
>
> --
> ===========
> Steve Hostettler
>
> Université de Genève
> CUI - Battelle bat. A
> Route de Drize, 7
> CH-1227 Carouge
> Tel. [phone]
> Fax [phone]
>

Hello Pavel,

well this is what Intel claims in the above video. I try to understand
exactly how but the only think I can figure out is that there is some sort
of equivalence between main memory and the 3dx point hardware.
I really really would like to understand this statemetn.

Best Regards

On Thu, Aug 23, 2018 at 2:16 PM Pavel Kovalenko <[hidden email]> wrote:

> Hello Steve,
>
> I've looked at Persistent Collectors library, but don't see any big
> advantages to use it inside Ignite right now.
> Why do you think, that marshalling/unmarshalling process will be speeded up
> with this library? As I can see there are no explicit changes in
> serialization process, just added persistent ability to regular java
> collections.
>
> 2018-08-23 9:53 GMT+03:00 Steve Hostettler <[hidden email]>:
>
> > That's great but would you mind explaining to me how we can get rid of
> the
> > marshalling/unmarshalling? Because that would significantly speed up the
> > processes that run on the grid.
> >
> > On Wed, Aug 22, 2018 at 3:36 AM Denis Magda <[hidden email]> wrote:
> >
> > > Hello Steve,
> > >
> > > Intel folks are already contributing Intel Optane Persistent Memory
> > support
> > > to Ignite:
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
> > >
> > > http://apache-ignite-developers.2346864.n4.nabble.
> > com/Adding-experimental-support-for-Intel-Optane-DC-
> > Persistent-Memory-td33041.html
> > >
> > > Hopefully, the contribution will be reviewed and accepted soon.
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
> > > [hidden email]> wrote:
> > >
> > > > Hello,
> > > >
> > > >
> > > >
> > > > Clearly Storage Class Memory represents a breakthrough for "in
> memory"
> > > > grids
> > > > and some people already tried it on Ignite :
> > > >
> > > >
> > > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> > ssds-verified-on.ht
> > > > ml
> > > > <
> > > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> > ssds-verified-on.html
> > > >
> > > >
> > > > I would like to know what is the position of the community towards
> the
> > > > Persistent Collections (
> > > > <
> > > >
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=I
> > > > STV_2017
> > > > <
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=ISTV_2017
> > > >
> > > > >
> > > >
> > > >
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=IS
> > > > TV_2017
> > > > <
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=ISTV_2017
> > > >,
> > > > <https://pcollections.org/> https://pcollections.org/)?
> > > > According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> > > > massively reduce the marshalling/unmarshalling time. `
> > > >
> > > > 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> > > > natively?
> > > >
> > > > 2)   Would it mean that ignite would not serialize the objects off
> heap
> > > > anymore?
> > > >
> > > >
> > > >
> > > > Hope it makes sense.
> > > >
> > > >
> > > >
> > > > Best Regards
> > > >
> > > >
> > >
> >
> >
> > --
> > ===========
> > Steve Hostettler
> >
> > Université de Genève
> > CUI - Battelle bat. A
> > Route de Drize, 7
> > CH-1227 Carouge
> > Tel. [phone]
> > Fax [phone]
> >
>

--
===========
Steve Hostettler

Université de Genève
CUI - Battelle bat. A
Route de Drize, 7
CH-1227 Carouge
Tel. [phone]
Fax [phone]