Igniters,
Right now we are developing WAL shipping process for our internal purposes and we use StandaloneWalRecordsIterator to read WAL files from custom destination. We have bumped into a problem - iterator can be constructed from set of files and dirs, but there's no option to pass WAL pointer to the iterator factory class to start iteration with. It can be worked around (by filtering all records prior to needed pointer), but I think it would be handy to add such option to IgniteWalIteratorFactory API. What do you think? -- Best Regards, Ivan Rakov |
Hi Ivan,
I agree that providing WAL pointer is the better option. Initially, Standalone WAL iterator was developed for debugging utility, so a set of files was perfectly OK. Sincerely, Dmitriy Palov ср, 15 авг. 2018 г. в 20:06, Ivan Rakov <[hidden email]>: > Igniters, > > Right now we are developing WAL shipping process for our internal > purposes and we use StandaloneWalRecordsIterator to read WAL files from > custom destination. We have bumped into a problem - iterator can be > constructed from set of files and dirs, but there's no option to pass > WAL pointer to the iterator factory class to start iteration with. It > can be worked around (by filtering all records prior to needed pointer), > but I think it would be handy to add such option to > IgniteWalIteratorFactory API. > > What do you think? > > -- > Best Regards, > Ivan Rakov > > |
Agree, this should be a great performance boost.
On Wed, Aug 15, 2018 at 10:17 AM, Dmitriy Pavlov <[hidden email]> wrote: > Hi Ivan, > > I agree that providing WAL pointer is the better option. Initially, > Standalone WAL iterator was developed for debugging utility, so a set of > files was perfectly OK. > > Sincerely, > Dmitriy Palov > > ср, 15 авг. 2018 г. в 20:06, Ivan Rakov <[hidden email]>: > > > Igniters, > > > > Right now we are developing WAL shipping process for our internal > > purposes and we use StandaloneWalRecordsIterator to read WAL files from > > custom destination. We have bumped into a problem - iterator can be > > constructed from set of files and dirs, but there's no option to pass > > WAL pointer to the iterator factory class to start iteration with. It > > can be worked around (by filtering all records prior to needed pointer), > > but I think it would be handy to add such option to > > IgniteWalIteratorFactory API. > > > > What do you think? > > > > -- > > Best Regards, > > Ivan Rakov > > > > > |
Thanks for your comments!
I've created a ticket: https://issues.apache.org/jira/browse/IGNITE-9294 Best Regards, Ivan Rakov On 15.08.2018 21:31, Dmitriy Setrakyan wrote: > Agree, this should be a great performance boost. > > On Wed, Aug 15, 2018 at 10:17 AM, Dmitriy Pavlov <[hidden email]> > wrote: > >> Hi Ivan, >> >> I agree that providing WAL pointer is the better option. Initially, >> Standalone WAL iterator was developed for debugging utility, so a set of >> files was perfectly OK. >> >> Sincerely, >> Dmitriy Palov >> >> ср, 15 авг. 2018 г. в 20:06, Ivan Rakov <[hidden email]>: >> >>> Igniters, >>> >>> Right now we are developing WAL shipping process for our internal >>> purposes and we use StandaloneWalRecordsIterator to read WAL files from >>> custom destination. We have bumped into a problem - iterator can be >>> constructed from set of files and dirs, but there's no option to pass >>> WAL pointer to the iterator factory class to start iteration with. It >>> can be worked around (by filtering all records prior to needed pointer), >>> but I think it would be handy to add such option to >>> IgniteWalIteratorFactory API. >>> >>> What do you think? >>> >>> -- >>> Best Regards, >>> Ivan Rakov >>> >>> |
Ivan,
I implemented this issue, please review my changes. https://reviews.ignite.apache.org/ignite/review/IGNT-CR-729 On Thu, Aug 16, 2018 at 3:09 PM Ivan Rakov <[hidden email]> wrote: > Thanks for your comments! > I've created a ticket: https://issues.apache.org/jira/browse/IGNITE-9294 > > Best Regards, > Ivan Rakov > > On 15.08.2018 21:31, Dmitriy Setrakyan wrote: > > Agree, this should be a great performance boost. > > > > On Wed, Aug 15, 2018 at 10:17 AM, Dmitriy Pavlov <[hidden email]> > > wrote: > > > >> Hi Ivan, > >> > >> I agree that providing WAL pointer is the better option. Initially, > >> Standalone WAL iterator was developed for debugging utility, so a set of > >> files was perfectly OK. > >> > >> Sincerely, > >> Dmitriy Palov > >> > >> ср, 15 авг. 2018 г. в 20:06, Ivan Rakov <[hidden email]>: > >> > >>> Igniters, > >>> > >>> Right now we are developing WAL shipping process for our internal > >>> purposes and we use StandaloneWalRecordsIterator to read WAL files from > >>> custom destination. We have bumped into a problem - iterator can be > >>> constructed from set of files and dirs, but there's no option to pass > >>> WAL pointer to the iterator factory class to start iteration with. It > >>> can be worked around (by filtering all records prior to needed > pointer), > >>> but I think it would be handy to add such option to > >>> IgniteWalIteratorFactory API. > >>> > >>> What do you think? > >>> > >>> -- > >>> Best Regards, > >>> Ivan Rakov > >>> > >>> > > |
Dmitriy,
Thanks a lot for stepping in! Your help is appreciated. I've looked through your changes and left a comment in JIRA issue. Best Regards, Ivan Rakov On 16.08.2018 23:56, Dmitriy Govorukhin wrote: > Ivan, > > I implemented this issue, please review my changes. > https://reviews.ignite.apache.org/ignite/review/IGNT-CR-729 > > On Thu, Aug 16, 2018 at 3:09 PM Ivan Rakov <[hidden email]> wrote: > >> Thanks for your comments! >> I've created a ticket: https://issues.apache.org/jira/browse/IGNITE-9294 >> >> Best Regards, >> Ivan Rakov >> >> On 15.08.2018 21:31, Dmitriy Setrakyan wrote: >>> Agree, this should be a great performance boost. >>> >>> On Wed, Aug 15, 2018 at 10:17 AM, Dmitriy Pavlov <[hidden email]> >>> wrote: >>> >>>> Hi Ivan, >>>> >>>> I agree that providing WAL pointer is the better option. Initially, >>>> Standalone WAL iterator was developed for debugging utility, so a set of >>>> files was perfectly OK. >>>> >>>> Sincerely, >>>> Dmitriy Palov >>>> >>>> ср, 15 авг. 2018 г. в 20:06, Ivan Rakov <[hidden email]>: >>>> >>>>> Igniters, >>>>> >>>>> Right now we are developing WAL shipping process for our internal >>>>> purposes and we use StandaloneWalRecordsIterator to read WAL files from >>>>> custom destination. We have bumped into a problem - iterator can be >>>>> constructed from set of files and dirs, but there's no option to pass >>>>> WAL pointer to the iterator factory class to start iteration with. It >>>>> can be worked around (by filtering all records prior to needed >> pointer), >>>>> but I think it would be handy to add such option to >>>>> IgniteWalIteratorFactory API. >>>>> >>>>> What do you think? >>>>> >>>>> -- >>>>> Best Regards, >>>>> Ivan Rakov >>>>> >>>>> >> |
Free forum by Nabble | Edit this page |