Fwd: Ignite work directory usage?

[dsetrakyan]

Cross sending to dev@.

Would be nice to hear from the Ignite community. How critical is this issue
and does it still exist?

D.

---------- Forwarded message ----------
From: Denis Magda <[hidden email]>
Date: Tue, Jan 9, 2018 at 2:28 PM
Subject: Re: Ignite work directory usage?
To: [hidden email]


It needs to be fixed if the users constantly bump into this issue. However,
I guess it’s not a matter of urgency now.

—
Denis

> On Jan 8, 2018, at 6:30 AM, ilya.kasnacheev <[hidden email]>
wrote:
>
> Hello Denis!
>
> There seems to be cases where matching workDir for different nodes
> (including client nodes) will cause problems:
>
> http://apache-ignite-users.70518.x6.nabble.com/MarshallerCon
textImpl-Failed-to-write-class-name-to-file-td18439.html
>
> There, on Windows, two nodes try to update binary marshaller cache on disk
> at the same time, but, this being on Windows, one of them fails to open
file
> for writing.
>
> On other OSes there also exists case where one instance would open file
for

> writing, and other one, seeing that there is already a file present, tries
> to read it, reads zero bytes and fails at that.
>
> WDYT about this case?
>
> Regards,
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

[Dmitriy Pavlov]

Hi Igniters,

As far as I remember, matching workDir for different nodes in marshaller
context are protected by file locks to avoid concurrent file modifications.

Sergey C. what do you think? Or am I confuse Marhsaller Context with Binary
Meta?

Sincerely,
Dmitry Pavlov

чт, 1 февр. 2018 г. в 5:18, Dmitriy Setrakyan <[hidden email]>:

> Cross sending to dev@.
>
> Would be nice to hear from the Ignite community. How critical is this issue
> and does it still exist?
>
> D.
>
> ---------- Forwarded message ----------
> From: Denis Magda <[hidden email]>
> Date: Tue, Jan 9, 2018 at 2:28 PM
> Subject: Re: Ignite work directory usage?
> To: [hidden email]
>
>
> It needs to be fixed if the users constantly bump into this issue. However,
> I guess it’s not a matter of urgency now.
>
> —
> Denis
>
> > On Jan 8, 2018, at 6:30 AM, ilya.kasnacheev <[hidden email]>
> wrote:
> >
> > Hello Denis!
> >
> > There seems to be cases where matching workDir for different nodes
> > (including client nodes) will cause problems:
> >
> > http://apache-ignite-users.70518.x6.nabble.com/MarshallerCon
> textImpl-Failed-to-write-class-name-to-file-td18439.html
> >
> > There, on Windows, two nodes try to update binary marshaller cache on
> disk
> > at the same time, but, this being on Windows, one of them fails to open
> file
> > for writing.
> >
> > On other OSes there also exists case where one instance would open file
> for
> > writing, and other one, seeing that there is already a file present,
> tries
> > to read it, reads zero bytes and fails at that.
> >
> > WDYT about this case?
> >
> > Regards,
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

[Sergey Chugunov]

Folks,

There are several things here.

Firstly user asked the initial question is sitting on Apache 1.x; it is
clear from exception stack trace he provided.
So although the issue exists for a while it looks like it doesn't hurt
users a lot.

Secondly I examined the code managing marshaller mappings and can say that
it the issue is still here even in the latest version; thus I filed a
ticket [1] to address it.

​[1] https://issues.apache.org/jira/browse/IGNITE-7635

Thanks,
Sergey.

[Valentin Kulichenko]

Sergey,

These mappings are supposed to be the same on all nodes, so if the file
already exists, we can safely ignore this, or use a lock to avoid
concurrent access. Actually, I think we already fixed this in the past,
it's weird that issue came up again.

But in any case, switching marshaller cache to persistence instead of using
these files makes perfect sense to me, and we definitely should do that.

-Val

On Tue, Feb 6, 2018 at 1:04 AM, Sergey Chugunov <[hidden email]>
wrote:

> Folks,
>
> There are several things here.
>
> Firstly user asked the initial question is sitting on Apache 1.x; it is
> clear from exception stack trace he provided.
> So although the issue exists for a while it looks like it doesn't hurt
> users a lot.
>
> Secondly I examined the code managing marshaller mappings and can say that
> it the issue is still here even in the latest version; thus I filed a
> ticket [1] to address it.
>
> ​[1] https://issues.apache.org/jira/browse/IGNITE-7635
>
> Thanks,
> Sergey.
>

[Dmitriy Pavlov]

Hi Val,

Do you mean by
> switching marshaller cache to persistence instead of using these files
makes perfect sense to me,
using 'metastore' for marshaller cache?

Sincerely,
Dmitriy Pavlov

пт, 9 февр. 2018 г. в 1:05, Valentin Kulichenko <
[hidden email]>:

> Sergey,
>
> These mappings are supposed to be the same on all nodes, so if the file
> already exists, we can safely ignore this, or use a lock to avoid
> concurrent access. Actually, I think we already fixed this in the past,
> it's weird that issue came up again.
>
> But in any case, switching marshaller cache to persistence instead of using
> these files makes perfect sense to me, and we definitely should do that.
>
> -Val
>
> On Tue, Feb 6, 2018 at 1:04 AM, Sergey Chugunov <[hidden email]
> >
> wrote:
>
> > Folks,
> >
> > There are several things here.
> >
> > Firstly user asked the initial question is sitting on Apache 1.x; it is
> > clear from exception stack trace he provided.
> > So although the issue exists for a while it looks like it doesn't hurt
> > users a lot.
> >
> > Secondly I examined the code managing marshaller mappings and can say
> that
> > it the issue is still here even in the latest version; thus I filed a
> > ticket [1] to address it.
> >
> > ​[1] https://issues.apache.org/jira/browse/IGNITE-7635
> >
> > Thanks,
> > Sergey.
> >
>

[Valentin Kulichenko]

Dmitry,

I meant the persistence store itself, but just realized that we don't have
a marshaller cache anymore, we use discovery messages instead. However, we
still have the MarshallerMappingFileStore which is basically a persistence
space created specifically for marshaller mappings. I think it would be a
good idea to use something more generic for this (although this is not
critical of course).

In any case, my initial point was that using the same folder by different
nodes for these mappings should not be an issue, because mappings generated
by different nodes are supposed to be always the same. We just need to
avoid weird exceptions.

-Val

On Fri, Feb 9, 2018 at 1:21 AM, Dmitry Pavlov <[hidden email]> wrote:

> Hi Val,
>
> Do you mean by
> > switching marshaller cache to persistence instead of using these files
> makes perfect sense to me,
> using 'metastore' for marshaller cache?
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 9 февр. 2018 г. в 1:05, Valentin Kulichenko <
> [hidden email]>:
>
> > Sergey,
> >
> > These mappings are supposed to be the same on all nodes, so if the file
> > already exists, we can safely ignore this, or use a lock to avoid
> > concurrent access. Actually, I think we already fixed this in the past,
> > it's weird that issue came up again.
> >
> > But in any case, switching marshaller cache to persistence instead of
> using
> > these files makes perfect sense to me, and we definitely should do that.
> >
> > -Val
> >
> > On Tue, Feb 6, 2018 at 1:04 AM, Sergey Chugunov <
> [hidden email]
> > >
> > wrote:
> >
> > > Folks,
> > >
> > > There are several things here.
> > >
> > > Firstly user asked the initial question is sitting on Apache 1.x; it is
> > > clear from exception stack trace he provided.
> > > So although the issue exists for a while it looks like it doesn't hurt
> > > users a lot.
> > >
> > > Secondly I examined the code managing marshaller mappings and can say
> > that
> > > it the issue is still here even in the latest version; thus I filed a
> > > ticket [1] to address it.
> > >
> > > ​[1] https://issues.apache.org/jira/browse/IGNITE-7635
> > >
> > > Thanks,
> > > Sergey.
> > >
> >
>

[Ilya Kasnacheev]

> because mappings generated by different nodes are supposed to be always
the same

Most file systems have atomic rename(). This means we should have two-phase
write of files into the folder in question. Step one - create
"NNNN.class.tmp" file, write data into it; Step two - rename file to
"NNNN.class". Catch "already used" exception and ignore it.

This way weird exceptions are avoided. Should I create a ticket about it?

--
Ilya Kasnacheev

2018-02-09 22:39 GMT+03:00 Valentin Kulichenko <
[hidden email]>:

> Dmitry,
>
> I meant the persistence store itself, but just realized that we don't have
> a marshaller cache anymore, we use discovery messages instead. However, we
> still have the MarshallerMappingFileStore which is basically a persistence
> space created specifically for marshaller mappings. I think it would be a
> good idea to use something more generic for this (although this is not
> critical of course).
>
> In any case, my initial point was that using the same folder by different
> nodes for these mappings should not be an issue, because mappings generated
> by different nodes are supposed to be always the same. We just need to
> avoid weird exceptions.
>
> -Val
>
> On Fri, Feb 9, 2018 at 1:21 AM, Dmitry Pavlov <[hidden email]>
> wrote:
>
> > Hi Val,
> >
> > Do you mean by
> > > switching marshaller cache to persistence instead of using these files
> > makes perfect sense to me,
> > using 'metastore' for marshaller cache?
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 9 февр. 2018 г. в 1:05, Valentin Kulichenko <
> > [hidden email]>:
> >
> > > Sergey,
> > >
> > > These mappings are supposed to be the same on all nodes, so if the file
> > > already exists, we can safely ignore this, or use a lock to avoid
> > > concurrent access. Actually, I think we already fixed this in the past,
> > > it's weird that issue came up again.
> > >
> > > But in any case, switching marshaller cache to persistence instead of
> > using
> > > these files makes perfect sense to me, and we definitely should do
> that.
> > >
> > > -Val
> > >
> > > On Tue, Feb 6, 2018 at 1:04 AM, Sergey Chugunov <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > Folks,
> > > >
> > > > There are several things here.
> > > >
> > > > Firstly user asked the initial question is sitting on Apache 1.x; it
> is
> > > > clear from exception stack trace he provided.
> > > > So although the issue exists for a while it looks like it doesn't
> hurt
> > > > users a lot.
> > > >
> > > > Secondly I examined the code managing marshaller mappings and can say
> > > that
> > > > it the issue is still here even in the latest version; thus I filed a
> > > > ticket [1] to address it.
> > > >
> > > > ​[1] https://issues.apache.org/jira/browse/IGNITE-7635
> > > >
> > > > Thanks,
> > > > Sergey.
> > > >
> > >
> >
>