IGNITE-708

[Gianfranco Murador-2]

Hi Igniters,
I 'm starting to implement this patch:
Can you tell me if there is already a convenient method to see if the local
cache was updated last time interval ?
Regards, Gianfranco


--
Gianfranco Murador
Igniter and Software Engineer.

[Alexey Goncharuk]

Gianfranco,

What do you mean by 'local cache' here?

If you are talking about the local partition map, I do not think we have
such a method. The background exchange that is described in the ticket is
handled in controlled by the ResendTimeoutObject inner class in
GridCachePartitionExchangeManager. I cannot recall any cases when this
exchange would be needed from the top of my head, but it looks like you
need to do some investigation and code digging to check whether the
background exchange can be indeed safely removed :)

2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]>:

> Hi Igniters,
> I 'm starting to implement this patch:
> Can you tell me if there is already a convenient method to see if the local
> cache was updated last time interval ?
> Regards, Gianfranco
>
>
> --
> Gianfranco Murador
> Igniter and Software Engineer.
>

[dsetrakyan]

On Mon, Aug 31, 2015 at 4:58 PM, Alexey Goncharuk <
[hidden email]> wrote:

> Gianfranco,
>
> What do you mean by 'local cache' here?
>
> If you are talking about the local partition map, I do not think we have
> such a method. The background exchange that is described in the ticket is
> handled in controlled by the ResendTimeoutObject inner class in
> GridCachePartitionExchangeManager. I cannot recall any cases when this
> exchange would be needed from the top of my head, but it looks like you
> need to do some investigation and code digging to check whether the
> background exchange can be indeed safely removed :)
>

Alexey, I actually think that this ticket is named wrongly. After looking
at the description, it seems that Yakov is suggesting that we do not send
the exchange message if there are no changes to the exchange. Perhaps, we
should be still sending something indicating that no change happened,
otherwise, other nodes will hang forever waiting for the exchange to
complete.

Am I wrong in my understanding?

>
> 2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]>:
>
> > Hi Igniters,
> > I 'm starting to implement this patch:
> > Can you tell me if there is already a convenient method to see if the
> local
> > cache was updated last time interval ?
> > Regards, Gianfranco
> >
> >
> > --
> > Gianfranco Murador
> > Igniter and Software Engineer.
> >
>

[Alexey Goncharuk]

As far as I understood Yakov's point, even this message indicating that no
change happened is redundant because we have message delivery guarantees on
communication level and no messages can be lost. If a node is waiting for a
message and receives a message indicating that no change had happened, I am
not even sure how this node should react: it means that the message with an
important update somehow was not received (a bug in the code?) and the next
message indicates that no updates after the lost message were made.

2015-08-31 17:33 GMT-07:00 Dmitriy Setrakyan <[hidden email]>:

> On Mon, Aug 31, 2015 at 4:58 PM, Alexey Goncharuk <
> [hidden email]> wrote:
>
> > Gianfranco,
> >
> > What do you mean by 'local cache' here?
> >
> > If you are talking about the local partition map, I do not think we have
> > such a method. The background exchange that is described in the ticket is
> > handled in controlled by the ResendTimeoutObject inner class in
> > GridCachePartitionExchangeManager. I cannot recall any cases when this
> > exchange would be needed from the top of my head, but it looks like you
> > need to do some investigation and code digging to check whether the
> > background exchange can be indeed safely removed :)
> >
>
> Alexey, I actually think that this ticket is named wrongly. After looking
> at the description, it seems that Yakov is suggesting that we do not send
> the exchange message if there are no changes to the exchange. Perhaps, we
> should be still sending something indicating that no change happened,
> otherwise, other nodes will hang forever waiting for the exchange to
> complete.
>
> Am I wrong in my understanding?
>
>
>
> >
> > 2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]>:
> >
> > > Hi Igniters,
> > > I 'm starting to implement this patch:
> > > Can you tell me if there is already a convenient method to see if the
> > local
> > > cache was updated last time interval ?
> > > Regards, Gianfranco
> > >
> > >
> > > --
> > > Gianfranco Murador
> > > Igniter and Software Engineer.
> > >
> >
>

[dsetrakyan]

On Mon, Aug 31, 2015 at 5:57 PM, Alexey Goncharuk <
[hidden email]> wrote:

> As far as I understood Yakov's point, even this message indicating that no
> change happened is redundant because we have message delivery guarantees on
> communication level and no messages can be lost. If a node is waiting for a
> message and receives a message indicating that no change had happened, I am
> not even sure how this node should react: it means that the message with an
> important update somehow was not received (a bug in the code?) and the next
> message indicates that no updates after the lost message were made.
>

I still would wait for a No-Change empty partition exchange message, rather
than have no message at all (and wait for a timeout?).

Yakov, can you please chime in and let us all know what you meant by that
ticket?

>
> 2015-08-31 17:33 GMT-07:00 Dmitriy Setrakyan <[hidden email]>:
>
> > On Mon, Aug 31, 2015 at 4:58 PM, Alexey Goncharuk <
> > [hidden email]> wrote:
> >
> > > Gianfranco,
> > >
> > > What do you mean by 'local cache' here?
> > >
> > > If you are talking about the local partition map, I do not think we
> have
> > > such a method. The background exchange that is described in the ticket
> is
> > > handled in controlled by the ResendTimeoutObject inner class in
> > > GridCachePartitionExchangeManager. I cannot recall any cases when this
> > > exchange would be needed from the top of my head, but it looks like you
> > > need to do some investigation and code digging to check whether the
> > > background exchange can be indeed safely removed :)
> > >
> >
> > Alexey, I actually think that this ticket is named wrongly. After looking
> > at the description, it seems that Yakov is suggesting that we do not send
> > the exchange message if there are no changes to the exchange. Perhaps, we
> > should be still sending something indicating that no change happened,
> > otherwise, other nodes will hang forever waiting for the exchange to
> > complete.
> >
> > Am I wrong in my understanding?
> >
> >
> >
> > >
> > > 2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]>:
> > >
> > > > Hi Igniters,
> > > > I 'm starting to implement this patch:
> > > > Can you tell me if there is already a convenient method to see if the
> > > local
> > > > cache was updated last time interval ?
> > > > Regards, Gianfranco
> > > >
> > > >
> > > > --
> > > > Gianfranco Murador
> > > > Igniter and Software Engineer.
> > > >
> > >
> >
>

[Gianfranco Murador]

Hello Alexey,
I mean 'local partition map'. I am trying to investigate the issue, and in
fact,
I need some clarification about the ticket. From my understanding, It is
necessary to refresh the partitions (refreshPartitions())
only if the local partition is changed, or if a transaction has been made
on it. So if I understand I need to add logic and control
in the OnTimeOut method of the inner class . Is it correct ?

/** {@inheritDoc} */
@Override public void onTimeout() {
cctx.kernalContext().closure().runLocalSafe(new Runnable() {
@Override public void run() {
if (!busyLock.readLock().tryLock())
return;

try {
           // onTimeOut we refresh always the partitions
      if (started.compareAndSet(false, true))
refreshPartitions();
}
finally {
busyLock.readLock().unlock();

cctx.time().removeTimeoutObject(ResendTimeoutObject.this);

pendingResend.compareAndSet(ResendTimeoutObject.this, null);
}
}
});
}

Thank you, Regards, Gianfranco

2015-09-01 3:30 GMT+02:00 Dmitriy Setrakyan <[hidden email]>:

> On Mon, Aug 31, 2015 at 5:57 PM, Alexey Goncharuk <
> [hidden email]> wrote:
>
> > As far as I understood Yakov's point, even this message indicating that
> no
> > change happened is redundant because we have message delivery guarantees
> on
> > communication level and no messages can be lost. If a node is waiting
> for a
> > message and receives a message indicating that no change had happened, I
> am
> > not even sure how this node should react: it means that the message with
> an
> > important update somehow was not received (a bug in the code?) and the
> next
> > message indicates that no updates after the lost message were made.
> >
>
> I still would wait for a No-Change empty partition exchange message, rather
> than have no message at all (and wait for a timeout?).
>
> Yakov, can you please chime in and let us all know what you meant by that
> ticket?
>
>
> >
> > 2015-08-31 17:33 GMT-07:00 Dmitriy Setrakyan <[hidden email]>:
> >
> > > On Mon, Aug 31, 2015 at 4:58 PM, Alexey Goncharuk <
> > > [hidden email]> wrote:
> > >
> > > > Gianfranco,
> > > >
> > > > What do you mean by 'local cache' here?
> > > >
> > > > If you are talking about the local partition map, I do not think we
> > have
> > > > such a method. The background exchange that is described in the
> ticket
> > is
> > > > handled in controlled by the ResendTimeoutObject inner class in
> > > > GridCachePartitionExchangeManager. I cannot recall any cases when
> this
> > > > exchange would be needed from the top of my head, but it looks like
> you
> > > > need to do some investigation and code digging to check whether the
> > > > background exchange can be indeed safely removed :)
> > > >
> > >
> > > Alexey, I actually think that this ticket is named wrongly. After
> looking
> > > at the description, it seems that Yakov is suggesting that we do not
> send
> > > the exchange message if there are no changes to the exchange. Perhaps,
> we
> > > should be still sending something indicating that no change happened,
> > > otherwise, other nodes will hang forever waiting for the exchange to
> > > complete.
> > >
> > > Am I wrong in my understanding?
> > >
> > >
> > >
> > > >
> > > > 2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]>:
> > > >
> > > > > Hi Igniters,
> > > > > I 'm starting to implement this patch:
> > > > > Can you tell me if there is already a convenient method to see if
> the
> > > > local
> > > > > cache was updated last time interval ?
> > > > > Regards, Gianfranco
> > > > >
> > > > >
> > > > > --
> > > > > Gianfranco Murador
> > > > > Igniter and Software Engineer.
> > > > >
> > > >
> > >
> >
>

[yzhdanov]

In my view, Alex has 100% understanding on what is hapenning. Let's remove
background exchange if partition map does not change. Gianfranco, I don't
think you should  account for transactions. Only updates to partition
topology matters. Younger nodes should send local updates to the oldest.
The oldest one should spread partitions after some delay buffering possible
updates or similar messages from other nodes.

Hope this helps!
On Sep 1, 2015 11:14, "Gianfranco Murador" <[hidden email]>
wrote:

> Hello Alexey,
> I mean 'local partition map'. I am trying to investigate the issue, and in
> fact,
> I need some clarification about the ticket. From my understanding, It is
> necessary to refresh the partitions (refreshPartitions())
> only if the local partition is changed, or if a transaction has been made
> on it. So if I understand I need to add logic and control
> in the OnTimeOut method of the inner class . Is it correct ?
>
> /** {@inheritDoc} */
> @Override public void onTimeout() {
> cctx.kernalContext().closure().runLocalSafe(new Runnable() {
> @Override public void run() {
> if (!busyLock.readLock().tryLock())
> return;
>
> try {
>            // onTimeOut we refresh always the partitions
>       if (started.compareAndSet(false, true))
> refreshPartitions();
> }
> finally {
> busyLock.readLock().unlock();
>
> cctx.time().removeTimeoutObject(ResendTimeoutObject.this);
>
> pendingResend.compareAndSet(ResendTimeoutObject.this, null);
> }
> }
> });
> }
>
> Thank you, Regards, Gianfranco
>
> 2015-09-01 3:30 GMT+02:00 Dmitriy Setrakyan <[hidden email]>:
>
> > On Mon, Aug 31, 2015 at 5:57 PM, Alexey Goncharuk <
> > [hidden email]> wrote:
> >
> > > As far as I understood Yakov's point, even this message indicating that
> > no
> > > change happened is redundant because we have message delivery
> guarantees
> > on
> > > communication level and no messages can be lost. If a node is waiting
> > for a
> > > message and receives a message indicating that no change had happened,
> I
> > am
> > > not even sure how this node should react: it means that the message
> with
> > an
> > > important update somehow was not received (a bug in the code?) and the
> > next
> > > message indicates that no updates after the lost message were made.
> > >
> >
> > I still would wait for a No-Change empty partition exchange message,
> rather
> > than have no message at all (and wait for a timeout?).
> >
> > Yakov, can you please chime in and let us all know what you meant by that
> > ticket?
> >
> >
> > >
> > > 2015-08-31 17:33 GMT-07:00 Dmitriy Setrakyan <[hidden email]>:
> > >
> > > > On Mon, Aug 31, 2015 at 4:58 PM, Alexey Goncharuk <
> > > > [hidden email]> wrote:
> > > >
> > > > > Gianfranco,
> > > > >
> > > > > What do you mean by 'local cache' here?
> > > > >
> > > > > If you are talking about the local partition map, I do not think we
> > > have
> > > > > such a method. The background exchange that is described in the
> > ticket
> > > is
> > > > > handled in controlled by the ResendTimeoutObject inner class in
> > > > > GridCachePartitionExchangeManager. I cannot recall any cases when
> > this
> > > > > exchange would be needed from the top of my head, but it looks like
> > you
> > > > > need to do some investigation and code digging to check whether the
> > > > > background exchange can be indeed safely removed :)
> > > > >
> > > >
> > > > Alexey, I actually think that this ticket is named wrongly. After
> > looking
> > > > at the description, it seems that Yakov is suggesting that we do not
> > send
> > > > the exchange message if there are no changes to the exchange.
> Perhaps,
> > we
> > > > should be still sending something indicating that no change happened,
> > > > otherwise, other nodes will hang forever waiting for the exchange to
> > > > complete.
> > > >
> > > > Am I wrong in my understanding?
> > > >
> > > >
> > > >
> > > > >
> > > > > 2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]>:
> > > > >
> > > > > > Hi Igniters,
> > > > > > I 'm starting to implement this patch:
> > > > > > Can you tell me if there is already a convenient method to see if
> > the
> > > > > local
> > > > > > cache was updated last time interval ?
> > > > > > Regards, Gianfranco
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Gianfranco Murador
> > > > > > Igniter and Software Engineer.
> > > > > >
> > > > >
> > > >
> > >
> >
>

[dsetrakyan]

On Tue, Sep 1, 2015 at 3:21 AM, Yakov Zhdanov <[hidden email]> wrote:

> In my view, Alex has 100% understanding on what is hapenning. Let's remove
> background exchange if partition map does not change. Gianfranco, I don't
> think you should  account for transactions. Only updates to partition
> topology matters. Younger nodes should send local updates to the oldest.
> The oldest one should spread partitions after some delay buffering possible
> updates or similar messages from other nodes.
>

I think I have finally understood the issue. You are talking about
partition map not changing across the whole cluster, not just on one node,
correct? In that case it absolutely does makes sense, Partition Map
Exchange should not happen if the partition map on any of the nodes did not
change.

>
> Hope this helps!
> On Sep 1, 2015 11:14, "Gianfranco Murador" <[hidden email]>
> wrote:
>
> > Hello Alexey,
> > I mean 'local partition map'. I am trying to investigate the issue, and
> in
> > fact,
> > I need some clarification about the ticket. From my understanding, It is
> > necessary to refresh the partitions (refreshPartitions())
> > only if the local partition is changed, or if a transaction has been made
> > on it. So if I understand I need to add logic and control
> > in the OnTimeOut method of the inner class . Is it correct ?
> >
> > /** {@inheritDoc} */
> > @Override public void onTimeout() {
> > cctx.kernalContext().closure().runLocalSafe(new Runnable() {
> > @Override public void run() {
> > if (!busyLock.readLock().tryLock())
> > return;
> >
> > try {
> >            // onTimeOut we refresh always the partitions
> >       if (started.compareAndSet(false, true))
> > refreshPartitions();
> > }
> > finally {
> > busyLock.readLock().unlock();
> >
> > cctx.time().removeTimeoutObject(ResendTimeoutObject.this);
> >
> > pendingResend.compareAndSet(ResendTimeoutObject.this, null);
> > }
> > }
> > });
> > }
> >
> > Thank you, Regards, Gianfranco
> >
> > 2015-09-01 3:30 GMT+02:00 Dmitriy Setrakyan <[hidden email]>:
> >
> > > On Mon, Aug 31, 2015 at 5:57 PM, Alexey Goncharuk <
> > > [hidden email]> wrote:
> > >
> > > > As far as I understood Yakov's point, even this message indicating
> that
> > > no
> > > > change happened is redundant because we have message delivery
> > guarantees
> > > on
> > > > communication level and no messages can be lost. If a node is waiting
> > > for a
> > > > message and receives a message indicating that no change had
> happened,
> > I
> > > am
> > > > not even sure how this node should react: it means that the message
> > with
> > > an
> > > > important update somehow was not received (a bug in the code?) and
> the
> > > next
> > > > message indicates that no updates after the lost message were made.
> > > >
> > >
> > > I still would wait for a No-Change empty partition exchange message,
> > rather
> > > than have no message at all (and wait for a timeout?).
> > >
> > > Yakov, can you please chime in and let us all know what you meant by
> that
> > > ticket?
> > >
> > >
> > > >
> > > > 2015-08-31 17:33 GMT-07:00 Dmitriy Setrakyan <[hidden email]
> >:
> > > >
> > > > > On Mon, Aug 31, 2015 at 4:58 PM, Alexey Goncharuk <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > > Gianfranco,
> > > > > >
> > > > > > What do you mean by 'local cache' here?
> > > > > >
> > > > > > If you are talking about the local partition map, I do not think
> we
> > > > have
> > > > > > such a method. The background exchange that is described in the
> > > ticket
> > > > is
> > > > > > handled in controlled by the ResendTimeoutObject inner class in
> > > > > > GridCachePartitionExchangeManager. I cannot recall any cases when
> > > this
> > > > > > exchange would be needed from the top of my head, but it looks
> like
> > > you
> > > > > > need to do some investigation and code digging to check whether
> the
> > > > > > background exchange can be indeed safely removed :)
> > > > > >
> > > > >
> > > > > Alexey, I actually think that this ticket is named wrongly. After
> > > looking
> > > > > at the description, it seems that Yakov is suggesting that we do
> not
> > > send
> > > > > the exchange message if there are no changes to the exchange.
> > Perhaps,
> > > we
> > > > > should be still sending something indicating that no change
> happened,
> > > > > otherwise, other nodes will hang forever waiting for the exchange
> to
> > > > > complete.
> > > > >
> > > > > Am I wrong in my understanding?
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > 2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]
> >:
> > > > > >
> > > > > > > Hi Igniters,
> > > > > > > I 'm starting to implement this patch:
> > > > > > > Can you tell me if there is already a convenient method to see
> if
> > > the
> > > > > > local
> > > > > > > cache was updated last time interval ?
> > > > > > > Regards, Gianfranco
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Gianfranco Murador
> > > > > > > Igniter and Software Engineer.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

[Gianfranco Murador]

Thank you , all.


2015-09-01 12:21 GMT+02:00 Yakov Zhdanov <[hidden email]>:

> In my view, Alex has 100% understanding on what is hapenning. Let's remove
> background exchange if partition map does not change. Gianfranco, I don't
> think you should  account for transactions. Only updates to partition
> topology matters. Younger nodes should send local updates to the oldest.
> The oldest one should spread partitions after some delay buffering possible
> updates or similar messages from other nodes.
>
> Hope this helps!
> On Sep 1, 2015 11:14, "Gianfranco Murador" <[hidden email]>
> wrote:
>
> > Hello Alexey,
> > I mean 'local partition map'. I am trying to investigate the issue, and
> in
> > fact,
> > I need some clarification about the ticket. From my understanding, It is
> > necessary to refresh the partitions (refreshPartitions())
> > only if the local partition is changed, or if a transaction has been made
> > on it. So if I understand I need to add logic and control
> > in the OnTimeOut method of the inner class . Is it correct ?
> >
> > /** {@inheritDoc} */
> > @Override public void onTimeout() {
> > cctx.kernalContext().closure().runLocalSafe(new Runnable() {
> > @Override public void run() {
> > if (!busyLock.readLock().tryLock())
> > return;
> >
> > try {
> >            // onTimeOut we refresh always the partitions
> >       if (started.compareAndSet(false, true))
> > refreshPartitions();
> > }
> > finally {
> > busyLock.readLock().unlock();
> >
> > cctx.time().removeTimeoutObject(ResendTimeoutObject.this);
> >
> > pendingResend.compareAndSet(ResendTimeoutObject.this, null);
> > }
> > }
> > });
> > }
> >
> > Thank you, Regards, Gianfranco
> >
> > 2015-09-01 3:30 GMT+02:00 Dmitriy Setrakyan <[hidden email]>:
> >
> > > On Mon, Aug 31, 2015 at 5:57 PM, Alexey Goncharuk <
> > > [hidden email]> wrote:
> > >
> > > > As far as I understood Yakov's point, even this message indicating
> that
> > > no
> > > > change happened is redundant because we have message delivery
> > guarantees
> > > on
> > > > communication level and no messages can be lost. If a node is waiting
> > > for a
> > > > message and receives a message indicating that no change had
> happened,
> > I
> > > am
> > > > not even sure how this node should react: it means that the message
> > with
> > > an
> > > > important update somehow was not received (a bug in the code?) and
> the
> > > next
> > > > message indicates that no updates after the lost message were made.
> > > >
> > >
> > > I still would wait for a No-Change empty partition exchange message,
> > rather
> > > than have no message at all (and wait for a timeout?).
> > >
> > > Yakov, can you please chime in and let us all know what you meant by
> that
> > > ticket?
> > >
> > >
> > > >
> > > > 2015-08-31 17:33 GMT-07:00 Dmitriy Setrakyan <[hidden email]
> >:
> > > >
> > > > > On Mon, Aug 31, 2015 at 4:58 PM, Alexey Goncharuk <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > > Gianfranco,
> > > > > >
> > > > > > What do you mean by 'local cache' here?
> > > > > >
> > > > > > If you are talking about the local partition map, I do not think
> we
> > > > have
> > > > > > such a method. The background exchange that is described in the
> > > ticket
> > > > is
> > > > > > handled in controlled by the ResendTimeoutObject inner class in
> > > > > > GridCachePartitionExchangeManager. I cannot recall any cases when
> > > this
> > > > > > exchange would be needed from the top of my head, but it looks
> like
> > > you
> > > > > > need to do some investigation and code digging to check whether
> the
> > > > > > background exchange can be indeed safely removed :)
> > > > > >
> > > > >
> > > > > Alexey, I actually think that this ticket is named wrongly. After
> > > looking
> > > > > at the description, it seems that Yakov is suggesting that we do
> not
> > > send
> > > > > the exchange message if there are no changes to the exchange.
> > Perhaps,
> > > we
> > > > > should be still sending something indicating that no change
> happened,
> > > > > otherwise, other nodes will hang forever waiting for the exchange
> to
> > > > > complete.
> > > > >
> > > > > Am I wrong in my understanding?
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > 2015-08-28 5:58 GMT-07:00 Gianfranco Murador <[hidden email]
> >:
> > > > > >
> > > > > > > Hi Igniters,
> > > > > > > I 'm starting to implement this patch:
> > > > > > > Can you tell me if there is already a convenient method to see
> if
> > > the
> > > > > > local
> > > > > > > cache was updated last time interval ?
> > > > > > > Regards, Gianfranco
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Gianfranco Murador
> > > > > > > Igniter and Software Engineer.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>