WAL iterator unexpected behavior

[Dmitriy Govorukhin]

Igniters,

I faced a problem with iterate over WAL.

Let's imagine that we write WAL and write some record which depends from
ignite-index.
And then move files to another machine, which does not have ignite-index in
the classpath.
Now try to iterate over WAL.

If iterator can not deserialize some record he try to go next and
execute advanceSegment() for switching to the next segment.
(ignite-index not in the classpath)

1. Iterator skips part of the WAL because advanceRecord() failed to
deserialize record.
2. Iterator does not signal that there was an error.

In my opinion, it is not that we expected.

Any comments?

[dsetrakyan]

Dmitriy,

Thanks for initiating the discussion!

I am not sure I understand the issue fully, but in my view we should not
allow iteration through WAL if we cannot deserialize a portion of it. We
must require that ignite-index is in the classpath, so I would just fail
right away with exception.

D.


On Tue, May 29, 2018 at 6:53 AM, Dmitriy Govorukhin <
[hidden email]> wrote:

> Igniters,
>
> I faced a problem with iterate over WAL.
>
> Let's imagine that we write WAL and write some record which depends from
> ignite-index.
> And then move files to another machine, which does not have ignite-index in
> the classpath.
> Now try to iterate over WAL.
>
> If iterator can not deserialize some record he try to go next and
> execute advanceSegment() for switching to the next segment.
> (ignite-index not in the classpath)
>
> 1. Iterator skips part of the WAL because advanceRecord() failed to
> deserialize record.
> 2. Iterator does not signal that there was an error.
>
> In my opinion, it is not that we expected.
>
> Any comments?
>

[yzhdanov]

This is for offline WAL analysis. So skipping record with proper message is
also a solution. If it is possible, iterator should output a suggestion on
what is missing in classpath. Option to suppress warnings should also
present.

Makes sense?

And final question - did we look at similar utilities from other vendors?

--Yakov

[Dmitriy Govorukhin]

Yakov,

This problem is not for offline only, it applicable for all types of the
iterators.
In general, iterator does not know which class will be needed for
deserialization.

I guess we can expand WAL Iterator factory and provide a method for
creating the iterator with some filter,
which will accept predicate for skipping no interesting records. It will
fix 1 problem.


On Wed, May 30, 2018 at 3:18 PM, Yakov Zhdanov <[hidden email]> wrote:

> This is for offline WAL analysis. So skipping record with proper message is
> also a solution. If it is possible, iterator should output a suggestion on
> what is missing in classpath. Option to suppress warnings should also
> present.
>
> Makes sense?
>
> And final question - did we look at similar utilities from other vendors?
>
> --Yakov
>

[dsetrakyan]

Dmitriy,

I think the behavior for offline and online iterator is fundamentally
different. I do not think it is OK to skip records during normal operation.
In my view, we should report an error and stop. However, when offline, it
is OK to report an error and continue in my view.

D.

On Wed, May 30, 2018 at 5:39 AM, Dmitriy Govorukhin <
[hidden email]> wrote:

> Yakov,
>
> This problem is not for offline only, it applicable for all types of the
> iterators.
> In general, iterator does not know which class will be needed for
> deserialization.
>
> I guess we can expand WAL Iterator factory and provide a method for
> creating the iterator with some filter,
> which will accept predicate for skipping no interesting records. It will
> fix 1 problem.
>
>
> On Wed, May 30, 2018 at 3:18 PM, Yakov Zhdanov <[hidden email]>
> wrote:
>
> > This is for offline WAL analysis. So skipping record with proper message
> is
> > also a solution. If it is possible, iterator should output a suggestion
> on
> > what is missing in classpath. Option to suppress warnings should also
> > present.
> >
> > Makes sense?
> >
> > And final question - did we look at similar utilities from other vendors?
> >
> > --Yakov
> >
>

[Dmitriy Govorukhin]

Dmitriy,

I agree that in normal mode we should stop and report that error according.
I prefer to add ability skip records for offline mode.

On Wed, May 30, 2018 at 5:24 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Dmitriy,
>
> I think the behavior for offline and online iterator is fundamentally
> different. I do not think it is OK to skip records during normal operation.
> In my view, we should report an error and stop. However, when offline, it
> is OK to report an error and continue in my view.
>
> D.
>
> On Wed, May 30, 2018 at 5:39 AM, Dmitriy Govorukhin <
> [hidden email]> wrote:
>
> > Yakov,
> >
> > This problem is not for offline only, it applicable for all types of the
> > iterators.
> > In general, iterator does not know which class will be needed for
> > deserialization.
> >
> > I guess we can expand WAL Iterator factory and provide a method for
> > creating the iterator with some filter,
> > which will accept predicate for skipping no interesting records. It will
> > fix 1 problem.
> >
> >
> > On Wed, May 30, 2018 at 3:18 PM, Yakov Zhdanov <[hidden email]>
> > wrote:
> >
> > > This is for offline WAL analysis. So skipping record with proper
> message
> > is
> > > also a solution. If it is possible, iterator should output a suggestion
> > on
> > > what is missing in classpath. Option to suppress warnings should also
> > > present.
> > >
> > > Makes sense?
> > >
> > > And final question - did we look at similar utilities from other
> vendors?
> > >
> > > --Yakov
> > >
> >
>

[dsetrakyan]

On Wed, May 30, 2018 at 8:04 AM, Dmitriy Govorukhin <
[hidden email]> wrote:

> Dmitriy,
>
> I agree that in normal mode we should stop and report that error according.
> I prefer to add ability skip records for offline mode.
>

Sounds good.

[Dmitriy Govorukhin]

Folks,

I created the issue to solve this issue.

IGNITE-8661 <https://issues.apache.org/jira/browse/IGNITE-8661> WALItreater
is not stopped if can not deserialize record

I suggest to make the following changes:
1. We should only stop iteration on known exceptions
2. Also, need to provide ability skip records by type or some pointer for
the StandaloneWalRecordsIterator

Comments are welcome.

On Thu, May 31, 2018 at 12:53 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Wed, May 30, 2018 at 8:04 AM, Dmitriy Govorukhin <
> [hidden email]> wrote:
>
> > Dmitriy,
> >
> > I agree that in normal mode we should stop and report that error
> according.
> > I prefer to add ability skip records for offline mode.
> >
>
> Sounds good.
>

[dsetrakyan]

Dmitriy, what happens in case of unknown exceptions?


On Thu, May 31, 2018 at 6:35 AM, Dmitriy Govorukhin <
[hidden email]> wrote:

> Folks,
>
> I created the issue to solve this issue.
>
> IGNITE-8661 <https://issues.apache.org/jira/browse/IGNITE-8661>
> WALItreater
> is not stopped if can not deserialize record
>
> I suggest to make the following changes:
> 1. We should only stop iteration on known exceptions
> 2. Also, need to provide ability skip records by type or some pointer for
> the StandaloneWalRecordsIterator
>
> Comments are welcome.
>
> On Thu, May 31, 2018 at 12:53 AM, Dmitriy Setrakyan <[hidden email]
> >
> wrote:
>
> > On Wed, May 30, 2018 at 8:04 AM, Dmitriy Govorukhin <
> > [hidden email]> wrote:
> >
> > > Dmitriy,
> > >
> > > I agree that in normal mode we should stop and report that error
> > according.
> > > I prefer to add ability skip records for offline mode.
> > >
> >
> > Sounds good.
> >
>

[Dmitriy Govorukhin]

Dmitriy,

The iterator will be stopped. and method "it.next()" will throw the
exception.

On Sat, Jun 2, 2018 at 4:27 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Dmitriy, what happens in case of unknown exceptions?
>
>
> On Thu, May 31, 2018 at 6:35 AM, Dmitriy Govorukhin <
> [hidden email]> wrote:
>
> > Folks,
> >
> > I created the issue to solve this issue.
> >
> > IGNITE-8661 <https://issues.apache.org/jira/browse/IGNITE-8661>
> > WALItreater
> > is not stopped if can not deserialize record
> >
> > I suggest to make the following changes:
> > 1. We should only stop iteration on known exceptions
> > 2. Also, need to provide ability skip records by type or some pointer for
> > the StandaloneWalRecordsIterator
> >
> > Comments are welcome.
> >
> > On Thu, May 31, 2018 at 12:53 AM, Dmitriy Setrakyan <
> [hidden email]
> > >
> > wrote:
> >
> > > On Wed, May 30, 2018 at 8:04 AM, Dmitriy Govorukhin <
> > > [hidden email]> wrote:
> > >
> > > > Dmitriy,
> > > >
> > > > I agree that in normal mode we should stop and report that error
> > > according.
> > > > I prefer to add ability skip records for offline mode.
> > > >
> > >
> > > Sounds good.
> > >
> >
>

[dsetrakyan]

I got a bit confused by your initial statement. So, the iterator is stopped
in case of any exception, known or unknown. In that case, sounds good.

D.

On Sun, Jun 3, 2018, 12:11 Dmitriy Govorukhin <[hidden email]>
wrote:

> Dmitriy,
>
> The iterator will be stopped. and method "it.next()" will throw the
> exception.
>
> On Sat, Jun 2, 2018 at 4:27 PM, Dmitriy Setrakyan <[hidden email]>
> wrote:
>
> > Dmitriy, what happens in case of unknown exceptions?
> >
> >
> > On Thu, May 31, 2018 at 6:35 AM, Dmitriy Govorukhin <
> > [hidden email]> wrote:
> >
> > > Folks,
> > >
> > > I created the issue to solve this issue.
> > >
> > > IGNITE-8661 <https://issues.apache.org/jira/browse/IGNITE-8661>
> > > WALItreater
> > > is not stopped if can not deserialize record
> > >
> > > I suggest to make the following changes:
> > > 1. We should only stop iteration on known exceptions
> > > 2. Also, need to provide ability skip records by type or some pointer
> for
> > > the StandaloneWalRecordsIterator
> > >
> > > Comments are welcome.
> > >
> > > On Thu, May 31, 2018 at 12:53 AM, Dmitriy Setrakyan <
> > [hidden email]
> > > >
> > > wrote:
> > >
> > > > On Wed, May 30, 2018 at 8:04 AM, Dmitriy Govorukhin <
> > > > [hidden email]> wrote:
> > > >
> > > > > Dmitriy,
> > > > >
> > > > > I agree that in normal mode we should stop and report that error
> > > > according.
> > > > > I prefer to add ability skip records for offline mode.
> > > > >
> > > >
> > > > Sounds good.
> > > >
> > >
> >
>

[Dmitriy Govorukhin]

Dmitriy,

Sorry, my mistake, I meant unknown exception of course.

On Sun, Jun 3, 2018 at 11:53 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> I got a bit confused by your initial statement. So, the iterator is stopped
> in case of any exception, known or unknown. In that case, sounds good.
>
> D.
>
> On Sun, Jun 3, 2018, 12:11 Dmitriy Govorukhin <
> [hidden email]>
> wrote:
>
> > Dmitriy,
> >
> > The iterator will be stopped. and method "it.next()" will throw the
> > exception.
> >
> > On Sat, Jun 2, 2018 at 4:27 PM, Dmitriy Setrakyan <[hidden email]
> >
> > wrote:
> >
> > > Dmitriy, what happens in case of unknown exceptions?
> > >
> > >
> > > On Thu, May 31, 2018 at 6:35 AM, Dmitriy Govorukhin <
> > > [hidden email]> wrote:
> > >
> > > > Folks,
> > > >
> > > > I created the issue to solve this issue.
> > > >
> > > > IGNITE-8661 <https://issues.apache.org/jira/browse/IGNITE-8661>
> > > > WALItreater
> > > > is not stopped if can not deserialize record
> > > >
> > > > I suggest to make the following changes:
> > > > 1. We should only stop iteration on known exceptions
> > > > 2. Also, need to provide ability skip records by type or some pointer
> > for
> > > > the StandaloneWalRecordsIterator
> > > >
> > > > Comments are welcome.
> > > >
> > > > On Thu, May 31, 2018 at 12:53 AM, Dmitriy Setrakyan <
> > > [hidden email]
> > > > >
> > > > wrote:
> > > >
> > > > > On Wed, May 30, 2018 at 8:04 AM, Dmitriy Govorukhin <
> > > > > [hidden email]> wrote:
> > > > >
> > > > > > Dmitriy,
> > > > > >
> > > > > > I agree that in normal mode we should stop and report that error
> > > > > according.
> > > > > > I prefer to add ability skip records for offline mode.
> > > > > >
> > > > >
> > > > > Sounds good.
> > > > >
> > > >
> > >
> >
>