Folks,
Quite often we face some concurrency or performance issues. And what is worse, sometimes they spread across several nodes making them hard to debug. We do various things to debug such problems. We add System.outs, measure latencies between algorithm steps, etc.. And once the problem is resolved, we throw away all debugging code. What if add a kind of tracing component to Ignite, so that convenient debugging infrastructure wiil always be ready for use? E.g., assume that we have performance issue in cache operation. Tracing framework could be used as follows: /** Register tracer. */ Cache.onStart() { ... ctx.trace().createProfile("cache_problem"); } /** Put trace inside problematic code. */ Cache.get(...) { TraceEvent traceEvt = ctx.trace().start("cache_problem"); ... traceEvt.checkpoint("LOCKED", [optional args and identifiers]); ... traceEvt.checkpoint("COMMITTED", ...); } /** In the end we dump problematic events. */ Cache.onStop() { ... ctx.profile("cache_problem").dump(System.out, [FILTER PREDICATE]); } Result: 23:20:01.234 [LOCKED=20ms, COMMITTED=152ms, FINISHED=153ms] 23:20:03.211 [LOCKED= 1ms, COMMITTED= 1ms, FINISHED= 2ms] I believe lots of you already did something like this during debug. The idea is to have some reusable component, so that we do not need to reinvent the wheel over and over again. Thoughts? Vladimir. |
How do you disable it? I mean, how do you make sure that it does not affect
performance when it is turned off? D. On Fri, Jan 29, 2016 at 12:31 PM, Vladimir Ozerov <[hidden email]> wrote: > Folks, > > Quite often we face some concurrency or performance issues. And what is > worse, sometimes they spread across several nodes making them hard to > debug. > > We do various things to debug such problems. We add System.outs, measure > latencies between algorithm steps, etc.. And once the problem is resolved, > we throw away all debugging code. > > What if add a kind of tracing component to Ignite, so that convenient > debugging infrastructure wiil always be ready for use? > > E.g., assume that we have performance issue in cache operation. Tracing > framework could be used as follows: > > /** Register tracer. */ > Cache.onStart() { > ... > ctx.trace().createProfile("cache_problem"); > } > > /** Put trace inside problematic code. */ > Cache.get(...) { > TraceEvent traceEvt = ctx.trace().start("cache_problem"); > ... > traceEvt.checkpoint("LOCKED", [optional args and identifiers]); > ... > traceEvt.checkpoint("COMMITTED", ...); > } > > /** In the end we dump problematic events. */ > Cache.onStop() { > ... > ctx.profile("cache_problem").dump(System.out, [FILTER PREDICATE]); > } > > Result: > 23:20:01.234 [LOCKED=20ms, COMMITTED=152ms, FINISHED=153ms] > 23:20:03.211 [LOCKED= 1ms, COMMITTED= 1ms, FINISHED= 2ms] > > I believe lots of you already did something like this during debug. The > idea is to have some reusable component, so that we do not need to reinvent > the wheel over and over again. > > Thoughts? > > Vladimir. > |
+1
I think it is a great idea, still it must be carefully designed to avoid additional performance penalty. BTW, for performance measurements there is already a good component: https://github.com/HdrHistogram/HdrHistogram Sergi 2016-01-30 3:01 GMT+03:00 Dmitriy Setrakyan <[hidden email]>: > How do you disable it? I mean, how do you make sure that it does not affect > performance when it is turned off? > > D. > > On Fri, Jan 29, 2016 at 12:31 PM, Vladimir Ozerov <[hidden email]> > wrote: > > > Folks, > > > > Quite often we face some concurrency or performance issues. And what is > > worse, sometimes they spread across several nodes making them hard to > > debug. > > > > We do various things to debug such problems. We add System.outs, measure > > latencies between algorithm steps, etc.. And once the problem is > resolved, > > we throw away all debugging code. > > > > What if add a kind of tracing component to Ignite, so that convenient > > debugging infrastructure wiil always be ready for use? > > > > E.g., assume that we have performance issue in cache operation. Tracing > > framework could be used as follows: > > > > /** Register tracer. */ > > Cache.onStart() { > > ... > > ctx.trace().createProfile("cache_problem"); > > } > > > > /** Put trace inside problematic code. */ > > Cache.get(...) { > > TraceEvent traceEvt = ctx.trace().start("cache_problem"); > > ... > > traceEvt.checkpoint("LOCKED", [optional args and identifiers]); > > ... > > traceEvt.checkpoint("COMMITTED", ...); > > } > > > > /** In the end we dump problematic events. */ > > Cache.onStop() { > > ... > > ctx.profile("cache_problem").dump(System.out, [FILTER PREDICATE]); > > } > > > > Result: > > 23:20:01.234 [LOCKED=20ms, COMMITTED=152ms, FINISHED=153ms] > > 23:20:03.211 [LOCKED= 1ms, COMMITTED= 1ms, FINISHED= 2ms] > > > > I believe lots of you already did something like this during debug. The > > idea is to have some reusable component, so that we do not need to > reinvent > > the wheel over and over again. > > > > Thoughts? > > > > Vladimir. > > > |
Free forum by Nabble | Edit this page |