Apache Ignite Developers - Legacy Mail Archive

IGNITE_HOME for persistence

Classic

List

Threaded

11 messages Options

Dmitriy Setrakyan-2

IGNITE_HOME for persistence

Igniters,

I have just stumbled upon this post on SO:
https://stackoverflow.com/questions/48434929/apache-ignite-persistent-storage

Do we require IGNITE_HOME to be set if the persistence is enabled? If yes,
do we check for it on startup?

D.

dmagda

Re: IGNITE_HOME for persistence

No we don’t. I’ve never touched IGNITE_HOME variable for any other purpose.

As it was suggested, the reported should share the project to reproduce his scenario.

—
Denis

> On Jan 26, 2018, at 9:05 PM, Dmitriy Setrakyan <[hidden email]> wrote:
>
> Igniters,
>
> I have just stumbled upon this post on SO:
> https://stackoverflow.com/questions/48434929/apache-ignite-persistent-storage
>
> Do we require IGNITE_HOME to be set if the persistence is enabled? If yes,
> do we check for it on startup?
>
> D.

Stanislav Lukyanov

RE: IGNITE_HOME for persistence

I checked the code handling the IGNITE_HOME and persistent storage paths,
and here is what the algorithm looks like.

For IGNITE_HOME the following is checked in order; if on any step a value is found then we use it.
- IgniteHome in IgniteConfiguration
- IGNITE_HOME system property
- IGNITE_HOME environment variable
- Current working directory (user.dir) and all its ancestors (all directories are checked to have “bin/” and “config/”)
- Class path entry containing ignite-core classes and all its ancestors

After that, the working directory will be created at one of the following paths
- WorkingDirectory in IgniteConfiguration, if set
- ${IGNITE_HOME}/work, if IGNITE_HOME could be calculated previously
- ${java.io.tmpdir}/work

Persistent storage will be stored in the working directory, unless StoragePath are specified in the config
(same for WAL and WalPath).

The issue here is that if we’ve ended up having persistent DB in the working directory in the /tmp,
then persistence files will be cleared upon restart.
Also, IgniteConfiguration::getIgniteHome claims that Igntie fails if IGNITE_HOME is not set, but that’s not the case.

So, how about actually disallowing to run Ignite when IGNITE_HOME can’t be calculated? Using /tmp for working
directory seems to be an obscure and potentially harmful scenario.
IgniteConfiguration’s documentation can also be adjusted to specify actual steps used to find IgniteHome and WorkingDirectory
if they aren’t set explicitly.
Additionally, I’d suggest not to promote setting IGNITE_HOME system property and environment variable
(e.g. let’s remove it from readmeio). IgniteConfiguration seems to be the most straightforward way to configure Ignite,
and system properties should be used as a backup plan when convenient.

WDYT?

Thanks,
Stan

From: Denis Magda
Sent: 30 января 2018 г. 3:38
To: [hidden email]
Subject: Re: IGNITE_HOME for persistence

No we don’t. I’ve never touched IGNITE_HOME variable for any other purpose.

As it was suggested, the reported should share the project to reproduce his scenario.

—
Denis

dmagda

Re: IGNITE_HOME for persistence

The temp dir is no way to go for the persistence.

I would enforce setting paths for the storage and WAL files if IGNITE_HOME can't be calculated and we end up placing the storage files under the temp.dir.

Alex G., can you join this thread? Seems that there is an oversight on our side that needs to be fixed in 2.4.

—
Denis

> On Jan 30, 2018, at 2:52 AM, Stanislav Lukyanov <[hidden email]> wrote:
>
> I checked the code handling the IGNITE_HOME and persistent storage paths,
> and here is what the algorithm looks like.
>
> For IGNITE_HOME the following is checked in order; if on any step a value is found then we use it.
> - IgniteHome in IgniteConfiguration
> - IGNITE_HOME system property
> - IGNITE_HOME environment variable
> - Current working directory (user.dir) and all its ancestors (all directories are checked to have “bin/” and “config/”)
> - Class path entry containing ignite-core classes and all its ancestors
>
> After that, the working directory will be created at one of the following paths
> - WorkingDirectory in IgniteConfiguration, if set
> - ${IGNITE_HOME}/work, if IGNITE_HOME could be calculated previously
> - ${java.io.tmpdir}/work
>
> Persistent storage will be stored in the working directory, unless StoragePath are specified in the config
> (same for WAL and WalPath).
>
> The issue here is that if we’ve ended up having persistent DB in the working directory in the /tmp,
> then persistence files will be cleared upon restart.
> Also, IgniteConfiguration::getIgniteHome claims that Igntie fails if IGNITE_HOME is not set, but that’s not the case.
>
> So, how about actually disallowing to run Ignite when IGNITE_HOME can’t be calculated? Using /tmp for working
> directory seems to be an obscure and potentially harmful scenario.
> IgniteConfiguration’s documentation can also be adjusted to specify actual steps used to find IgniteHome and WorkingDirectory
> if they aren’t set explicitly.
> Additionally, I’d suggest not to promote setting IGNITE_HOME system property and environment variable
> (e.g. let’s remove it from readmeio). IgniteConfiguration seems to be the most straightforward way to configure Ignite,
> and system properties should be used as a backup plan when convenient.
>
> WDYT?
>
> Thanks,
> Stan
>
> From: Denis Magda
> Sent: 30 января 2018 г. 3:38
> To: [hidden email]
> Subject: Re: IGNITE_HOME for persistence
>
> No we don’t. I’ve never touched IGNITE_HOME variable for any other purpose.
>
> As it was suggested, the reported should share the project to reproduce his scenario.
>
> —
> Denis
>
>> On Jan 26, 2018, at 9:05 PM, Dmitriy Setrakyan <[hidden email]> wrote:
>>
>> Igniters,
>>
>> I have just stumbled upon this post on SO:
>> https://stackoverflow.com/questions/48434929/apache-ignite-persistent-storage
>>
>> Do we require IGNITE_HOME to be set if the persistence is enabled? If yes,
>> do we check for it on startup?
>>
>> D.
>
>

dsetrakyan

Re: IGNITE_HOME for persistence

In reply to this post by Stanislav Lukyanov

I think we should still keep System property and Environment variable
check. Our product has a user base that depends on these properties and
changing them at will is not an option.

I do agree that the product should not start if IGNITE_HOME cannot be
calculated, but I still do not understand how this could be possible.
Setting it to TMP directory seems vary flaky and should be disabled.

This seems to be a blocker for 2.4 release and needs to be fixed. Do we
have an appropriate ticket for this?

D.

On Tue, Jan 30, 2018 at 2:52 AM, Stanislav Lukyanov <[hidden email]>
wrote:

> I checked the code handling the IGNITE_HOME and persistent storage paths,
> and here is what the algorithm looks like.
>
> For IGNITE_HOME the following is checked in order; if on any step a value
> is found then we use it.
> - IgniteHome in IgniteConfiguration
> - IGNITE_HOME system property
> - IGNITE_HOME environment variable
> - Current working directory (user.dir) and all its ancestors (all
> directories are checked to have “bin/” and “config/”)
> - Class path entry containing ignite-core classes and all its ancestors
>
> After that, the working directory will be created at one of the following
> paths
> - WorkingDirectory in IgniteConfiguration, if set
> - ${IGNITE_HOME}/work, if IGNITE_HOME could be calculated previously
> - ${java.io.tmpdir}/work
>
> Persistent storage will be stored in the working directory, unless
> StoragePath are specified in the config
> (same for WAL and WalPath).
>
> The issue here is that if we’ve ended up having persistent DB in the
> working directory in the /tmp,
> then persistence files will be cleared upon restart.
> Also, IgniteConfiguration::getIgniteHome claims that Igntie fails if
> IGNITE_HOME is not set, but that’s not the case.
>
> So, how about actually disallowing to run Ignite when IGNITE_HOME can’t be
> calculated? Using /tmp for working
> directory seems to be an obscure and potentially harmful scenario.
> IgniteConfiguration’s documentation can also be adjusted to specify actual
> steps used to find IgniteHome and WorkingDirectory
> if they aren’t set explicitly.
> Additionally, I’d suggest not to promote setting IGNITE_HOME system
> property and environment variable
> (e.g. let’s remove it from readmeio). IgniteConfiguration seems to be the
> most straightforward way to configure Ignite,
> and system properties should be used as a backup plan when convenient.
>
> WDYT?
>
> Thanks,
> Stan
>
> From: Denis Magda
> Sent: 30 января 2018 г. 3:38
> To: [hidden email]
> Subject: Re: IGNITE_HOME for persistence
>
> No we don’t. I’ve never touched IGNITE_HOME variable for any other purpose.
>
> As it was suggested, the reported should share the project to reproduce
> his scenario.
>
> —
> Denis
>
> > On Jan 26, 2018, at 9:05 PM, Dmitriy Setrakyan <[hidden email]> wrote:
> >
> > Igniters,
> >
> > I have just stumbled upon this post on SO:
> > https://stackoverflow.com/questions/48434929/apache-
> ignite-persistent-storage
> >
> > Do we require IGNITE_HOME to be set if the persistence is enabled? If
> yes,
> > do we check for it on startup?
> >
> > D.
>
>
>

Alexey Goncharuk

Re: IGNITE_HOME for persistence

In reply to this post by dmagda

Denis, Dmitriy,

I do not think the behavior should be in any way specific to the case when
persistence is enabled. We have quite a bit of other information (such as
binary marshaller metadata and marshaller class mappings) that are also
stored in the work folder and may be required after the cluster restart,
moreover, this has been so for years since ignite 1.0.

If you think this is a blocker, please suggest what behavior we want to see
in ignite 2.4. We cannot calculate IGNITE_HOME when Ignite is run in an
embedded mode (in other words, the Ignite distribution package structure is
not preserved). Printing a warning in this case looks good enough for me.

2018-01-30 23:35 GMT+03:00 Denis Magda <[hidden email]>:

> The temp dir is no way to go for the persistence.
>
> I would enforce setting paths for the storage and WAL files if IGNITE_HOME
> can't be calculated and we end up placing the storage files under the
> temp.dir.
>
> Alex G., can you join this thread? Seems that there is an oversight on our
> side that needs to be fixed in 2.4.
>
> —
> Denis
>
> > On Jan 30, 2018, at 2:52 AM, Stanislav Lukyanov <[hidden email]>
> wrote:
> >
> > I checked the code handling the IGNITE_HOME and persistent storage paths,
> > and here is what the algorithm looks like.
> >
> > For IGNITE_HOME the following is checked in order; if on any step a
> value is found then we use it.
> > - IgniteHome in IgniteConfiguration
> > - IGNITE_HOME system property
> > - IGNITE_HOME environment variable
> > - Current working directory (user.dir) and all its ancestors (all
> directories are checked to have “bin/” and “config/”)
> > - Class path entry containing ignite-core classes and all its ancestors
> >
> > After that, the working directory will be created at one of the
> following paths
> > - WorkingDirectory in IgniteConfiguration, if set
> > - ${IGNITE_HOME}/work, if IGNITE_HOME could be calculated previously
> > - ${java.io.tmpdir}/work
> >
> > Persistent storage will be stored in the working directory, unless
> StoragePath are specified in the config
> > (same for WAL and WalPath).
> >
> > The issue here is that if we’ve ended up having persistent DB in the
> working directory in the /tmp,
> > then persistence files will be cleared upon restart.
> > Also, IgniteConfiguration::getIgniteHome claims that Igntie fails if
> IGNITE_HOME is not set, but that’s not the case.
> >
> > So, how about actually disallowing to run Ignite when IGNITE_HOME can’t
> be calculated? Using /tmp for working
> > directory seems to be an obscure and potentially harmful scenario.
> > IgniteConfiguration’s documentation can also be adjusted to specify
> actual steps used to find IgniteHome and WorkingDirectory
> > if they aren’t set explicitly.
> > Additionally, I’d suggest not to promote setting IGNITE_HOME system
> property and environment variable
> > (e.g. let’s remove it from readmeio). IgniteConfiguration seems to be
> the most straightforward way to configure Ignite,
> > and system properties should be used as a backup plan when convenient.
> >
> > WDYT?
> >
> > Thanks,
> > Stan
> >
> > From: Denis Magda
> > Sent: 30 января 2018 г. 3:38
> > To: [hidden email]
> > Subject: Re: IGNITE_HOME for persistence
> >
> > No we don’t. I’ve never touched IGNITE_HOME variable for any other
> purpose.
> >
> > As it was suggested, the reported should share the project to reproduce
> his scenario.
> >
> > —
> > Denis
> >
> >> On Jan 26, 2018, at 9:05 PM, Dmitriy Setrakyan <[hidden email]> wrote:
> >>
> >> Igniters,
> >>
> >> I have just stumbled upon this post on SO:
> >> https://stackoverflow.com/questions/48434929/apache-
> ignite-persistent-storage
> >>
> >> Do we require IGNITE_HOME to be set if the persistence is enabled? If
> yes,
> >> do we check for it on startup?
> >>
> >> D.
> >
> >
>
>

dsetrakyan

Re: IGNITE_HOME for persistence

I think at this point all we can do is print a warning. However, starting
with 3.0, we should require IGNITE_HOME to be set if we cannot determine it
automatically. Do we print a warning already?

On Tue, Jan 30, 2018 at 12:55 PM, Alexey Goncharuk <
[hidden email]> wrote:

> Denis, Dmitriy,
>
> I do not think the behavior should be in any way specific to the case when
> persistence is enabled. We have quite a bit of other information (such as
> binary marshaller metadata and marshaller class mappings) that are also
> stored in the work folder and may be required after the cluster restart,
> moreover, this has been so for years since ignite 1.0.
>
> If you think this is a blocker, please suggest what behavior we want to see
> in ignite 2.4. We cannot calculate IGNITE_HOME when Ignite is run in an
> embedded mode (in other words, the Ignite distribution package structure is
> not preserved). Printing a warning in this case looks good enough for me.
>
> 2018-01-30 23:35 GMT+03:00 Denis Magda <[hidden email]>:
>
> > The temp dir is no way to go for the persistence.
> >
> > I would enforce setting paths for the storage and WAL files if
> IGNITE_HOME
> > can't be calculated and we end up placing the storage files under the
> > temp.dir.
> >
> > Alex G., can you join this thread? Seems that there is an oversight on
> our
> > side that needs to be fixed in 2.4.
> >
> > —
> > Denis
> >
> > > On Jan 30, 2018, at 2:52 AM, Stanislav Lukyanov <
> [hidden email]>
> > wrote:
> > >
> > > I checked the code handling the IGNITE_HOME and persistent storage
> paths,
> > > and here is what the algorithm looks like.
> > >
> > > For IGNITE_HOME the following is checked in order; if on any step a
> > value is found then we use it.
> > > - IgniteHome in IgniteConfiguration
> > > - IGNITE_HOME system property
> > > - IGNITE_HOME environment variable
> > > - Current working directory (user.dir) and all its ancestors (all
> > directories are checked to have “bin/” and “config/”)
> > > - Class path entry containing ignite-core classes and all its ancestors
> > >
> > > After that, the working directory will be created at one of the
> > following paths
> > > - WorkingDirectory in IgniteConfiguration, if set
> > > - ${IGNITE_HOME}/work, if IGNITE_HOME could be calculated previously
> > > - ${java.io.tmpdir}/work
> > >
> > > Persistent storage will be stored in the working directory, unless
> > StoragePath are specified in the config
> > > (same for WAL and WalPath).
> > >
> > > The issue here is that if we’ve ended up having persistent DB in the
> > working directory in the /tmp,
> > > then persistence files will be cleared upon restart.
> > > Also, IgniteConfiguration::getIgniteHome claims that Igntie fails if
> > IGNITE_HOME is not set, but that’s not the case.
> > >
> > > So, how about actually disallowing to run Ignite when IGNITE_HOME can’t
> > be calculated? Using /tmp for working
> > > directory seems to be an obscure and potentially harmful scenario.
> > > IgniteConfiguration’s documentation can also be adjusted to specify
> > actual steps used to find IgniteHome and WorkingDirectory
> > > if they aren’t set explicitly.
> > > Additionally, I’d suggest not to promote setting IGNITE_HOME system
> > property and environment variable
> > > (e.g. let’s remove it from readmeio). IgniteConfiguration seems to be
> > the most straightforward way to configure Ignite,
> > > and system properties should be used as a backup plan when convenient.
> > >
> > > WDYT?
> > >
> > > Thanks,
> > > Stan
> > >
> > > From: Denis Magda
> > > Sent: 30 января 2018 г. 3:38
> > > To: [hidden email]
> > > Subject: Re: IGNITE_HOME for persistence
> > >
> > > No we don’t. I’ve never touched IGNITE_HOME variable for any other
> > purpose.
> > >
> > > As it was suggested, the reported should share the project to reproduce
> > his scenario.
> > >
> > > —
> > > Denis
> > >
> > >> On Jan 26, 2018, at 9:05 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
> > >>
> > >> Igniters,
> > >>
> > >> I have just stumbled upon this post on SO:
> > >> https://stackoverflow.com/questions/48434929/apache-
> > ignite-persistent-storage
> > >>
> > >> Do we require IGNITE_HOME to be set if the persistence is enabled? If
> > yes,
> > >> do we check for it on startup?
> > >>
> > >> D.
> > >
> > >
> >
> >
>

Dmitriy Pavlov

Re: IGNITE_HOME for persistence

In reply to this post by Alexey Goncharuk

Hi Igniters,

Why it can be serious issue for end users? As far as I know IGNITE_HOME is
always set using start scripts, e.g. in ignite.sh.

If Ignite is started using java code from IDE, it may be required to set
storage directory or work or home directory. Until that, TEMP is convinient
for development.

IMO it is quite obvious that persistent store configuration has 2 methods
setPersistentEnabled and setStorageDirectory, isn't it? Moreover all paths
are logged at startup.

+1 for warn user if PDS-enabled ignite instance is started without any
explicit folder specification. But I can't find a reason to consider it
blocker for 2.4.

Sincerely,
Dmitriy Pavlov

вт, 30 янв. 2018 г. в 23:55, Alexey Goncharuk <[hidden email]>:

Dmitriy Pavlov

Re: IGNITE_HOME for persistence

In reply to this post by dsetrakyan

Hi,

do we have a task to print this warning message?

It seems we don't print warning currently.

Sincerely,
Dmitry Pavlov

ср, 31 янв. 2018 г., 0:55 Dmitriy Setrakyan <[hidden email]>:

> I think at this point all we can do is print a warning. However, starting
> with 3.0, we should require IGNITE_HOME to be set if we cannot determine it
> automatically. Do we print a warning already?
>
> On Tue, Jan 30, 2018 at 12:55 PM, Alexey Goncharuk <
> [hidden email]> wrote:
>
> > Denis, Dmitriy,
> >
> > I do not think the behavior should be in any way specific to the case
> when
> > persistence is enabled. We have quite a bit of other information (such as
> > binary marshaller metadata and marshaller class mappings) that are also
> > stored in the work folder and may be required after the cluster restart,
> > moreover, this has been so for years since ignite 1.0.
> >
> > If you think this is a blocker, please suggest what behavior we want to
> see
> > in ignite 2.4. We cannot calculate IGNITE_HOME when Ignite is run in an
> > embedded mode (in other words, the Ignite distribution package structure
> is
> > not preserved). Printing a warning in this case looks good enough for me.
> >
> > 2018-01-30 23:35 GMT+03:00 Denis Magda <[hidden email]>:
> >
> > > The temp dir is no way to go for the persistence.
> > >
> > > I would enforce setting paths for the storage and WAL files if
> > IGNITE_HOME
> > > can't be calculated and we end up placing the storage files under the
> > > temp.dir.
> > >
> > > Alex G., can you join this thread? Seems that there is an oversight on
> > our
> > > side that needs to be fixed in 2.4.
> > >
> > > —
> > > Denis
> > >
> > > > On Jan 30, 2018, at 2:52 AM, Stanislav Lukyanov <
> > [hidden email]>
> > > wrote:
> > > >
> > > > I checked the code handling the IGNITE_HOME and persistent storage
> > paths,
> > > > and here is what the algorithm looks like.
> > > >
> > > > For IGNITE_HOME the following is checked in order; if on any step a
> > > value is found then we use it.
> > > > - IgniteHome in IgniteConfiguration
> > > > - IGNITE_HOME system property
> > > > - IGNITE_HOME environment variable
> > > > - Current working directory (user.dir) and all its ancestors (all
> > > directories are checked to have “bin/” and “config/”)
> > > > - Class path entry containing ignite-core classes and all its
> ancestors
> > > >
> > > > After that, the working directory will be created at one of the
> > > following paths
> > > > - WorkingDirectory in IgniteConfiguration, if set
> > > > - ${IGNITE_HOME}/work, if IGNITE_HOME could be calculated previously
> > > > - ${java.io.tmpdir}/work
> > > >
> > > > Persistent storage will be stored in the working directory, unless
> > > StoragePath are specified in the config
> > > > (same for WAL and WalPath).
> > > >
> > > > The issue here is that if we’ve ended up having persistent DB in the
> > > working directory in the /tmp,
> > > > then persistence files will be cleared upon restart.
> > > > Also, IgniteConfiguration::getIgniteHome claims that Igntie fails if
> > > IGNITE_HOME is not set, but that’s not the case.
> > > >
> > > > So, how about actually disallowing to run Ignite when IGNITE_HOME
> can’t
> > > be calculated? Using /tmp for working
> > > > directory seems to be an obscure and potentially harmful scenario.
> > > > IgniteConfiguration’s documentation can also be adjusted to specify
> > > actual steps used to find IgniteHome and WorkingDirectory
> > > > if they aren’t set explicitly.
> > > > Additionally, I’d suggest not to promote setting IGNITE_HOME system
> > > property and environment variable
> > > > (e.g. let’s remove it from readmeio). IgniteConfiguration seems to be
> > > the most straightforward way to configure Ignite,
> > > > and system properties should be used as a backup plan when
> convenient.
> > > >
> > > > WDYT?
> > > >
> > > > Thanks,
> > > > Stan
> > > >
> > > > From: Denis Magda
> > > > Sent: 30 января 2018 г. 3:38
> > > > To: [hidden email]
> > > > Subject: Re: IGNITE_HOME for persistence
> > > >
> > > > No we don’t. I’ve never touched IGNITE_HOME variable for any other
> > > purpose.
> > > >
> > > > As it was suggested, the reported should share the project to
> reproduce
> > > his scenario.
> > > >
> > > > —
> > > > Denis
> > > >
> > > >> On Jan 26, 2018, at 9:05 PM, Dmitriy Setrakyan <[hidden email]>
> > wrote:
> > > >>
> > > >> Igniters,
> > > >>
> > > >> I have just stumbled upon this post on SO:
> > > >> https://stackoverflow.com/questions/48434929/apache-
> > > ignite-persistent-storage
> > > >>
> > > >> Do we require IGNITE_HOME to be set if the persistence is enabled?
> If
> > > yes,
> > > >> do we check for it on startup?
> > > >>
> > > >> D.
> > > >
> > > >
> > >
> > >
> >
>

dsetrakyan

Re: IGNITE_HOME for persistence

On Tue, Mar 13, 2018 at 7:01 PM, Dmitry Pavlov <[hidden email]>
wrote:

> Hi,
>
> do we have a task to print this warning message?
>
> It seems we don't print warning currently.
>

I do not think we have a ticket. As AG mentioned, this is an issue for
cases when persistence is disabled as well. Dmitriy, do you mind creating
one and assigning it to 2.5 release?

Dmitriy Pavlov

Re: IGNITE_HOME for persistence

Hi Dmitriy, Igniters,

I've created https://issues.apache.org/jira/browse/IGNITE-8456

I hope that someone in community can pick up this ticket. I've marked
ticket as newbie.

Sincerely,
Dmitriy Pavlov

ср, 14 мар. 2018 г. в 2:12, Dmitriy Setrakyan <[hidden email]>:

> On Tue, Mar 13, 2018 at 7:01 PM, Dmitry Pavlov <[hidden email]>
> wrote:
>
> > Hi,
> >
> > do we have a task to print this warning message?
> >
> > It seems we don't print warning currently.
> >
>
> I do not think we have a ticket. As AG mentioned, this is an issue for
> cases when persistence is disabled as well. Dmitriy, do you mind creating
> one and assigning it to 2.5 release?
>