[DISCUSS] Ignite memory -> virtual memory

[dsetrakyan]

Igniters,

With the newly donated persistence functionality in Ignite, I have been
struggling a bit on how to fit the notion of persistence into the current
Ignite interfaces, that are almost completely memory oriented. For example,
abstractions like MemoryConfiguration or MemoryMetrics will now have to
include the persistence context, given that pages will be seamlessly mapped
to disk, whenever the memory fills up (e.g. providing the number of pages
on disk on MemoryMetrics interface).

After looking around, I have noticed that our architecture is increasingly
beginning to look like the Virtual Memory concept in operating systems [1],
if you consider Ignite off-heap memory to be the physical memory, and disk
to be the secondary memory space. Just like virtual memory, our
architecture is based on memory pages and memory segments. The total set of
all pages constitutes the total virtual memory space.

If we document our memory interfaces as virtual memory, then we won't have
to do any renaming and can comfortably add disk-based methods to these
interfaces, as it becomes consistent with the virtual memory concept.

Thoughts?

[1] - https://en.wikipedia.org/wiki/Virtual_memory

[Alexey Goncharuk]

I am fine with this javadoc change as long as there is no confusion between
Ignite page memory buffers and the OS Virtual Memory concept.

2017-06-01 2:07 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:

> Igniters,
>
> With the newly donated persistence functionality in Ignite, I have been
> struggling a bit on how to fit the notion of persistence into the current
> Ignite interfaces, that are almost completely memory oriented. For example,
> abstractions like MemoryConfiguration or MemoryMetrics will now have to
> include the persistence context, given that pages will be seamlessly mapped
> to disk, whenever the memory fills up (e.g. providing the number of pages
> on disk on MemoryMetrics interface).
>
> After looking around, I have noticed that our architecture is increasingly
> beginning to look like the Virtual Memory concept in operating systems [1],
> if you consider Ignite off-heap memory to be the physical memory, and disk
> to be the secondary memory space. Just like virtual memory, our
> architecture is based on memory pages and memory segments. The total set of
> all pages constitutes the total virtual memory space.
>
> If we document our memory interfaces as virtual memory, then we won't have
> to do any renaming and can comfortably add disk-based methods to these
> interfaces, as it becomes consistent with the virtual memory concept.
>
> Thoughts?
>
> [1] - https://en.wikipedia.org/wiki/Virtual_memory
>

[dsetrakyan]

Agreed then. Let's update the javadoc and documentation.

On Thu, Jun 1, 2017 at 1:33 AM, Alexey Goncharuk <[hidden email]
> wrote:

> I am fine with this javadoc change as long as there is no confusion between
> Ignite page memory buffers and the OS Virtual Memory concept.
>
> 2017-06-01 2:07 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
>
> > Igniters,
> >
> > With the newly donated persistence functionality in Ignite, I have been
> > struggling a bit on how to fit the notion of persistence into the current
> > Ignite interfaces, that are almost completely memory oriented. For
> example,
> > abstractions like MemoryConfiguration or MemoryMetrics will now have to
> > include the persistence context, given that pages will be seamlessly
> mapped
> > to disk, whenever the memory fills up (e.g. providing the number of pages
> > on disk on MemoryMetrics interface).
> >
> > After looking around, I have noticed that our architecture is
> increasingly
> > beginning to look like the Virtual Memory concept in operating systems
> [1],
> > if you consider Ignite off-heap memory to be the physical memory, and
> disk
> > to be the secondary memory space. Just like virtual memory, our
> > architecture is based on memory pages and memory segments. The total set
> of
> > all pages constitutes the total virtual memory space.
> >
> > If we document our memory interfaces as virtual memory, then we won't
> have
> > to do any renaming and can comfortably add disk-based methods to these
> > interfaces, as it becomes consistent with the virtual memory concept.
> >
> > Thoughts?
> >
> > [1] - https://en.wikipedia.org/wiki/Virtual_memory
> >
>

[dmagda]

Updated the javadoc and existing 2.0 documentation (decided to leave the url of the doc unchanged - there are many references to it):
https://apacheignite.readme.io/v2.0/docs/page-memory <https://apacheignite.readme.io/v2.0/docs/page-memory>

-
Denis
 

> On Jun 2, 2017, at 9:51 AM, Dmitriy Setrakyan <[hidden email]> wrote:
>
> Agreed then. Let's update the javadoc and documentation.
>
> On Thu, Jun 1, 2017 at 1:33 AM, Alexey Goncharuk <[hidden email]
>> wrote:
>
>> I am fine with this javadoc change as long as there is no confusion between
>> Ignite page memory buffers and the OS Virtual Memory concept.
>>
>> 2017-06-01 2:07 GMT+03:00 Dmitriy Setrakyan <[hidden email]>:
>>
>>> Igniters,
>>>
>>> With the newly donated persistence functionality in Ignite, I have been
>>> struggling a bit on how to fit the notion of persistence into the current
>>> Ignite interfaces, that are almost completely memory oriented. For
>> example,
>>> abstractions like MemoryConfiguration or MemoryMetrics will now have to
>>> include the persistence context, given that pages will be seamlessly
>> mapped
>>> to disk, whenever the memory fills up (e.g. providing the number of pages
>>> on disk on MemoryMetrics interface).
>>>
>>> After looking around, I have noticed that our architecture is
>> increasingly
>>> beginning to look like the Virtual Memory concept in operating systems
>> [1],
>>> if you consider Ignite off-heap memory to be the physical memory, and
>> disk
>>> to be the secondary memory space. Just like virtual memory, our
>>> architecture is based on memory pages and memory segments. The total set
>> of
>>> all pages constitutes the total virtual memory space.
>>>
>>> If we document our memory interfaces as virtual memory, then we won't
>> have
>>> to do any renaming and can comfortably add disk-based methods to these
>>> interfaces, as it becomes consistent with the virtual memory concept.
>>>
>>> Thoughts?
>>>
>>> [1] - https://en.wikipedia.org/wiki/Virtual_memory
>>>
>>