Benchmarking

[Anton Vinogradov-2]

Igniters,

All critical code changes should be covered by benchmarks.
Currently, community have no dedicated benchmarking service.

Everyone can run yandstick locally, but this seems to be useless.
You have to run cluster on 4 clients and 4 servers, at least.

Currently it is not possible for me to perform benchmarking using 8+ real
servers.
Also, some automation requred to have reliable check of two branchs with
one feature difference, since yardstick will provide you branch check, not
a branches comparision.

So, my question is:
Is there anybody at community able to perform benchmarks on real hardware
and already have automation to compare 2 branches?

[Pavel Kovalenko]

Igniters,

I would like to add that it would be very nice to have prepared scenarios
in packed docker images with docker-compose, to easily deploy and run it on
AWS environment. This will give the possibility to benchmark any changes
independently and fastly.

2018-08-09 18:51 GMT+03:00 Anton Vinogradov <[hidden email]>:

> Igniters,
>
> All critical code changes should be covered by benchmarks.
> Currently, community have no dedicated benchmarking service.
>
> Everyone can run yandstick locally, but this seems to be useless.
> You have to run cluster on 4 clients and 4 servers, at least.
>
> Currently it is not possible for me to perform benchmarking using 8+ real
> servers.
> Also, some automation requred to have reliable check of two branchs with
> one feature difference, since yardstick will provide you branch check, not
> a branches comparision.
>
> So, my question is:
> Is there anybody at community able to perform benchmarks on real hardware
> and already have automation to compare 2 branches?
>

[daradurvs]

Huge +1, for a public benchmarking tool and the Docker based approach.
On Thu, Aug 9, 2018 at 7:16 PM Pavel Kovalenko <[hidden email]> wrote:

>
> Igniters,
>
> I would like to add that it would be very nice to have prepared scenarios
> in packed docker images with docker-compose, to easily deploy and run it on
> AWS environment. This will give the possibility to benchmark any changes
> independently and fastly.
>
> 2018-08-09 18:51 GMT+03:00 Anton Vinogradov <[hidden email]>:
>
> > Igniters,
> >
> > All critical code changes should be covered by benchmarks.
> > Currently, community have no dedicated benchmarking service.
> >
> > Everyone can run yandstick locally, but this seems to be useless.
> > You have to run cluster on 4 clients and 4 servers, at least.
> >
> > Currently it is not possible for me to perform benchmarking using 8+ real
> > servers.
> > Also, some automation requred to have reliable check of two branchs with
> > one feature difference, since yardstick will provide you branch check, not
> > a branches comparision.
> >
> > So, my question is:
> > Is there anybody at community able to perform benchmarks on real hardware
> > and already have automation to compare 2 branches?
> >

--
Best Regards, Vyacheslav D.

[Ilya Suntsov]

Anton,

I have experience with the yardstick and can help you sometimes.
My possibilities in this matter are limited and it is possible that your
request will be delayed.

Please, try to use the last version of the yardstick and then we can
discuss specific yardstick
issues that you couldn't solve.

Also, you can use dedicated AWS instances for benchmarking.
Pay attention to the fact that sometimes our benchmarks can show unstable
results in case of
non-dedicated instances.




2018-08-09 21:09 GMT+03:00 Vyacheslav Daradur <[hidden email]>:

> Huge +1, for a public benchmarking tool and the Docker based approach.
> On Thu, Aug 9, 2018 at 7:16 PM Pavel Kovalenko <[hidden email]> wrote:
> >
> > Igniters,
> >
> > I would like to add that it would be very nice to have prepared scenarios
> > in packed docker images with docker-compose, to easily deploy and run it
> on
> > AWS environment. This will give the possibility to benchmark any changes
> > independently and fastly.
> >
> > 2018-08-09 18:51 GMT+03:00 Anton Vinogradov <[hidden email]>:
> >
> > > Igniters,
> > >
> > > All critical code changes should be covered by benchmarks.
> > > Currently, community have no dedicated benchmarking service.
> > >
> > > Everyone can run yandstick locally, but this seems to be useless.
> > > You have to run cluster on 4 clients and 4 servers, at least.
> > >
> > > Currently it is not possible for me to perform benchmarking using 8+
> real
> > > servers.
> > > Also, some automation requred to have reliable check of two branchs
> with
> > > one feature difference, since yardstick will provide you branch check,
> not
> > > a branches comparision.
> > >
> > > So, my question is:
> > > Is there anybody at community able to perform benchmarks on real
> hardware
> > > and already have automation to compare 2 branches?
> > >
>
>
>
> --
> Best Regards, Vyacheslav D.
>

--
Best Regards,
Ilya Suntsov
email: [hidden email]
*GridGain Systems*
www.gridgain.com