Python thin client
Locked
123
123
 
|
Igor,
All examples are in 'ignite/modules/platforms/python/examples'. I put examples in separate Python files mostly to be able to automatically confirm their operability. All the lengthy explanations and cross-references are in the main documentation: https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/examples.html To make it easier for end users to navigate in pyignite directories, I added a small README file to the 'examples' directory with a short description and a link to the docs: https://github.com/nobitlost/ignite/blob/ignite-7782/modules/platforms/python/examples/readme.md If you think this README is lacking something regardless of the main pyignite docs, please let me know. On 9/17/18 8:59 PM, Igor Sapego wrote: > Dmitry, > > Sorry, I was not clear enough. What I mean is that Ignite distributed > by both source and binary releases. Binary releases contain platforms > code, which is needed to write your own application in the language, > but does not contain developer stuff, such as tests, documentation > generating scripts, etc. > > Of course, it is more common to use package managers when > developing your application, and of course, we are going to support > this approach. But still, we provide a way for a user to get any client > without any package manager. > > I like the idea of supplying whl package in a binary release, though it > looks like it's going to take more effort to implement it. But I believe > except for the whl package, we will need to supply examples and > instructions, how user can run them. > > What do you think? > > Best Regards, > Igor > > > On Sat, Sep 15, 2018 at 7:03 AM Dmitry Melnichuk < > [hidden email]> wrote: > >> Igor, >> >> I am in doubt here because I am not fully comprehend the meaning of >> "binary release". But if it is somehow related to the "distribution" >> thing, I would dare to suggest the following options: >> >> 1. Copy nothing. Just do >> >> ``` >> $ python setup.py bdist_wheel >> $ twine upload dist/* >> ``` >> >> during the build process (or separately) and let PyPI handle the >> distribution. >> >> This is the most natural and user-friendly way of distributing Python >> packages. End user might only do >> >> ``` >> $ pip install pyignite >> ``` >> >> as it is suggested by my readme file. >> >> 2. Supply the wheel package. It is the file 'pyignite-*.whl' from 'dist' >> directory, that should appear there as the "python setup.py bdist_wheel" >> command finishes. Actually, it can be combined with the first option. >> >> The wheel can then be installed by the end user: >> >> ``` >> $ pip install pyignite-*.whl >> ``` >> >> 3. Supply the following files: >> >> ignite/modules/platforms/python/pyignite/** >> ignite/modules/platforms/python/requirements/** >> ignite/modules/platforms/python/LICENSE >> ignite/modules/platforms/python/README.md >> ignite/modules/platforms/python/setup.py >> >> (** stands for "all the files and sub-folders recursively, starting from >> this folder".) >> >> It is not uncommon or wrong to distribute Python programs as text >> without prior packaging, but, in my personal experience, this is a way >> more suitable for one-time scripts or proprietary programs. For example, >> most of Python web apps relies on git for deployment, without any >> packaging subsystem. >> >> Here's a few words about wheels: >> >> https://wheel.readthedocs.io/ >> >> Here's about uploading to PyPI with twine: >> >> >> https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives >> >> On 9/14/18 9:05 PM, Igor Sapego wrote: >>> Ok, good. >>> >>> Now, what is about installation? Which directories/files >>> need to be copied to ignite's binary release? >>> >>> Best Regards, >>> Igor >>> >> > |
|
|
Great job.
Best Regards, Igor On Tue, Sep 18, 2018 at 11:35 AM Dmitry Melnichuk < [hidden email]> wrote: > Igor, > > All examples are in 'ignite/modules/platforms/python/examples'. > > I put examples in separate Python files mostly to be able to > automatically confirm their operability. All the lengthy explanations > and cross-references are in the main documentation: > > > https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/examples.html > > To make it easier for end users to navigate in pyignite directories, I > added a small README file to the 'examples' directory with a short > description and a link to the docs: > > > https://github.com/nobitlost/ignite/blob/ignite-7782/modules/platforms/python/examples/readme.md > > If you think this README is lacking something regardless of the main > pyignite docs, please let me know. > > On 9/17/18 8:59 PM, Igor Sapego wrote: > > Dmitry, > > > > Sorry, I was not clear enough. What I mean is that Ignite distributed > > by both source and binary releases. Binary releases contain platforms > > code, which is needed to write your own application in the language, > > but does not contain developer stuff, such as tests, documentation > > generating scripts, etc. > > > > Of course, it is more common to use package managers when > > developing your application, and of course, we are going to support > > this approach. But still, we provide a way for a user to get any client > > without any package manager. > > > > I like the idea of supplying whl package in a binary release, though it > > looks like it's going to take more effort to implement it. But I believe > > except for the whl package, we will need to supply examples and > > instructions, how user can run them. > > > > What do you think? > > > > Best Regards, > > Igor > > > > > > On Sat, Sep 15, 2018 at 7:03 AM Dmitry Melnichuk < > > [hidden email]> wrote: > > > >> Igor, > >> > >> I am in doubt here because I am not fully comprehend the meaning of > >> "binary release". But if it is somehow related to the "distribution" > >> thing, I would dare to suggest the following options: > >> > >> 1. Copy nothing. Just do > >> > >> ``` > >> $ python setup.py bdist_wheel > >> $ twine upload dist/* > >> ``` > >> > >> during the build process (or separately) and let PyPI handle the > >> distribution. > >> > >> This is the most natural and user-friendly way of distributing Python > >> packages. End user might only do > >> > >> ``` > >> $ pip install pyignite > >> ``` > >> > >> as it is suggested by my readme file. > >> > >> 2. Supply the wheel package. It is the file 'pyignite-*.whl' from 'dist' > >> directory, that should appear there as the "python setup.py bdist_wheel" > >> command finishes. Actually, it can be combined with the first option. > >> > >> The wheel can then be installed by the end user: > >> > >> ``` > >> $ pip install pyignite-*.whl > >> ``` > >> > >> 3. Supply the following files: > >> > >> ignite/modules/platforms/python/pyignite/** > >> ignite/modules/platforms/python/requirements/** > >> ignite/modules/platforms/python/LICENSE > >> ignite/modules/platforms/python/README.md > >> ignite/modules/platforms/python/setup.py > >> > >> (** stands for "all the files and sub-folders recursively, starting from > >> this folder".) > >> > >> It is not uncommon or wrong to distribute Python programs as text > >> without prior packaging, but, in my personal experience, this is a way > >> more suitable for one-time scripts or proprietary programs. For example, > >> most of Python web apps relies on git for deployment, without any > >> packaging subsystem. > >> > >> Here's a few words about wheels: > >> > >> https://wheel.readthedocs.io/ > >> > >> Here's about uploading to PyPI with twine: > >> > >> > >> > https://packaging.python.org/tutorials/packaging-projects/#uploading-the-distribution-archives > >> > >> On 9/14/18 9:05 PM, Igor Sapego wrote: > >>> Ok, good. > >>> > >>> Now, what is about installation? Which directories/files > >>> need to be copied to ignite's binary release? > >>> > >>> Best Regards, > >>> Igor > >>> > >> > > > > |
|
|
Hi Dmitry,
Not sure if you are aware but all documentation for the thin clients will be moved to readme.io. Here is the ticket for Python thin clients - https://issues.apache.org/jira/browse/IGNITE-9522 I am already working on it. So if you make any changes to the docs in your repo, please let me know so that I don't miss bringing them over to readme.io -Prachi On Tue, Sep 18, 2018 at 4:08 AM, Igor Sapego <[hidden email]> wrote: > Great job. > > Best Regards, > Igor > > > On Tue, Sep 18, 2018 at 11:35 AM Dmitry Melnichuk < > [hidden email]> wrote: > > > Igor, > > > > All examples are in 'ignite/modules/platforms/python/examples'. > > > > I put examples in separate Python files mostly to be able to > > automatically confirm their operability. All the lengthy explanations > > and cross-references are in the main documentation: > > > > > > https://apache-ignite-binary-protocol-client.readthedocs. > io/en/latest/examples.html > > > > To make it easier for end users to navigate in pyignite directories, I > > added a small README file to the 'examples' directory with a short > > description and a link to the docs: > > > > > > https://github.com/nobitlost/ignite/blob/ignite-7782/ > modules/platforms/python/examples/readme.md > > > > If you think this README is lacking something regardless of the main > > pyignite docs, please let me know. > > > > On 9/17/18 8:59 PM, Igor Sapego wrote: > > > Dmitry, > > > > > > Sorry, I was not clear enough. What I mean is that Ignite distributed > > > by both source and binary releases. Binary releases contain platforms > > > code, which is needed to write your own application in the language, > > > but does not contain developer stuff, such as tests, documentation > > > generating scripts, etc. > > > > > > Of course, it is more common to use package managers when > > > developing your application, and of course, we are going to support > > > this approach. But still, we provide a way for a user to get any client > > > without any package manager. > > > > > > I like the idea of supplying whl package in a binary release, though it > > > looks like it's going to take more effort to implement it. But I > believe > > > except for the whl package, we will need to supply examples and > > > instructions, how user can run them. > > > > > > What do you think? > > > > > > Best Regards, > > > Igor > > > > > > > > > On Sat, Sep 15, 2018 at 7:03 AM Dmitry Melnichuk < > > > [hidden email]> wrote: > > > > > >> Igor, > > >> > > >> I am in doubt here because I am not fully comprehend the meaning of > > >> "binary release". But if it is somehow related to the "distribution" > > >> thing, I would dare to suggest the following options: > > >> > > >> 1. Copy nothing. Just do > > >> > > >> ``` > > >> $ python setup.py bdist_wheel > > >> $ twine upload dist/* > > >> ``` > > >> > > >> during the build process (or separately) and let PyPI handle the > > >> distribution. > > >> > > >> This is the most natural and user-friendly way of distributing Python > > >> packages. End user might only do > > >> > > >> ``` > > >> $ pip install pyignite > > >> ``` > > >> > > >> as it is suggested by my readme file. > > >> > > >> 2. Supply the wheel package. It is the file 'pyignite-*.whl' from > 'dist' > > >> directory, that should appear there as the "python setup.py > bdist_wheel" > > >> command finishes. Actually, it can be combined with the first option. > > >> > > >> The wheel can then be installed by the end user: > > >> > > >> ``` > > >> $ pip install pyignite-*.whl > > >> ``` > > >> > > >> 3. Supply the following files: > > >> > > >> ignite/modules/platforms/python/pyignite/** > > >> ignite/modules/platforms/python/requirements/** > > >> ignite/modules/platforms/python/LICENSE > > >> ignite/modules/platforms/python/README.md > > >> ignite/modules/platforms/python/setup.py > > >> > > >> (** stands for "all the files and sub-folders recursively, starting > from > > >> this folder".) > > >> > > >> It is not uncommon or wrong to distribute Python programs as text > > >> without prior packaging, but, in my personal experience, this is a way > > >> more suitable for one-time scripts or proprietary programs. For > example, > > >> most of Python web apps relies on git for deployment, without any > > >> packaging subsystem. > > >> > > >> Here's a few words about wheels: > > >> > > >> https://wheel.readthedocs.io/ > > >> > > >> Here's about uploading to PyPI with twine: > > >> > > >> > > >> > > https://packaging.python.org/tutorials/packaging-projects/# > uploading-the-distribution-archives > > >> > > >> On 9/14/18 9:05 PM, Igor Sapego wrote: > > >>> Ok, good. > > >>> > > >>> Now, what is about installation? Which directories/files > > >>> need to be copied to ignite's binary release? > > >>> > > >>> Best Regards, > > >>> Igor > > >>> > > >> > > > > > > > > |
|
|
In reply to this post by Dmitry Melnichuk
Hi, all
Tried to compare new thin client performance and made similar benchmarks for each one And result not so good for python Ticket with results and bench code: https://issues.apache.org/jira/browse/IGNITE-9824 Py code src: https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb Dmitry, please review results and bench code, maybe somthing wrong or it's expected numbers? -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ |
|
|
Hi, Stepan!
I looked at the benchmark code and the overall methodics, discussed it with fellow programmers, and came up with basically two remarks. First of all, I think, the key for the good (or, at least, unobjectionable) measurement is to isolate the object being measured from the influence of the measurement tool. The usual pattern we use in Python looks like: ``` from time import time bench_start = time() for _ in range(number_of_tests): do_test() bench_end = time() print('Performance is {} tests per second'.format( number_of_tests / (bench_end - bench_start) )) ``` I think, you got the idea: the measurement consists almost solely of the time taken by our subject function `do_test`. As few other code as possible influence the result. Now, let's take a look at your code: https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb Ideally, the `while` loop should include only `cache.put(last_key, some_precomputed_value)`. But instead it also includes: - a series of the `time()` calls, which could be mostly excluded from the measured time, if the measurement done right; each call is probably addresses the HPET device, or network time, or both, - some floating-point calculations, including `round()`, which is hardly necessary, - formatting and output of the intermediate result. I suppose the measurement influence can be quite significant here, but it is at least more or less constant for each test. But if we look at the other benchmarks: https://gist.github.com/pilshchikov/8a4bdb03a8304136c22c9bf7217ee447 [Node.js] https://gist.github.com/pilshchikov/b4351d78ad59e9cd923689c2e387bc80 [PHP] https://gist.github.com/pilshchikov/08096c78b425e00166a2ffa2aa5f49ce [Java] The extra code that influence the measurement is not equivalent across all platforms. For example, PHP's `time()` is most probably lighter than Python `time()`, since it do not give out milliseconds and may address RTC, not HPET. So the platform-to-platform comparison in your benchmark does not look completely fair to me. The second remark concerns not the measurement procedure, but the object being measured. The only client operation being used is OP_CACHE_PUT with a payload of a primitive type. (BTW the type is `Long` in case of the Python client; what about the other clients? Is it `Int`?) I afraid that such an object, even being properly isolated from the measurement tool, would show mostly the throughput of the underlying platform's sockets implementation, not the performance of the client's code. To show the potential of the thin client itself, more varied and fancy tasks needed, i. e. serialization/deserialization of the Complex objects. But it depends on the goal of the benchmarking. If showing off the raw platform agility was intended, than this objection is removed. Dmitry On 10/9/18 10:50 PM, Stepan Pilshchikov wrote: > Hi, all > > Tried to compare new thin client performance and made similar benchmarks for > each one > And result not so good for python > > Ticket with results and bench code: > https://issues.apache.org/jira/browse/IGNITE-9824 > > Py code src: > https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb > > Dmitry, please review results and bench code, maybe somthing wrong or it's > expected numbers? > > > > -- > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > |
|
|
Hi Dmitry,
I agree with your comments on benchmarking code. As more fair alternative we may measure time of loading N elements into the cache, so that it will be necessary to call time() only twice. However, provided that we have real network here, and according to numbers one PUT in Python client requires about 0.3 msec, I hardly can imagine that call to time() or round() may dominate in that response time. But I said, we can easily rule that out by slight benchmark rewrite. As far as serialization, I would prefer to keep primitive objects at the moment, because AFAIK the purpose of the benchmark was to assess underlying infrastructure, looking for some obvious performance issues. Every platform have sockets which use more or less the same API of underlying OS. To the contrast, performance of serialization mechanics may differ widely between platforms depending on implementation. E.g. in Java we spend a lot of time on fine-grained tuning, applied a lot of speculative optimizations. Add to this JIT nature of Java, and you will hardly ever beat Java serialization engine from any interpreted language. So primitive data types make good sense to me. At this point our goal is not make Python equally fast with other platforms, but rather to understand why is it slower than others. Ignite is a product which brings speed to end users. If they do no have speed, they will not use Ignite. So performance questions are always of great importance for us. Vladimir. On Wed, Oct 10, 2018 at 9:57 AM Dmitry Melnichuk < [hidden email]> wrote: > Hi, Stepan! > > I looked at the benchmark code and the overall methodics, discussed it > with fellow programmers, and came up with basically two remarks. > > First of all, I think, the key for the good (or, at least, > unobjectionable) measurement is to isolate the object being measured > from the influence of the measurement tool. The usual pattern we use in > Python looks like: > > ``` > from time import time > > > bench_start = time() > > for _ in range(number_of_tests): > do_test() > > bench_end = time() > > print('Performance is {} tests per second'.format( > number_of_tests / (bench_end - bench_start) > )) > ``` > > I think, you got the idea: the measurement consists almost solely of the > time taken by our subject function `do_test`. As few other code as > possible influence the result. > > Now, let's take a look at your code: > > https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb > > Ideally, the `while` loop should include only `cache.put(last_key, > some_precomputed_value)`. But instead it also includes: > > - a series of the `time()` calls, which could be mostly excluded from > the measured time, if the measurement done right; each call is probably > addresses the HPET device, or network time, or both, > > - some floating-point calculations, including `round()`, which is hardly > necessary, > > - formatting and output of the intermediate result. > > I suppose the measurement influence can be quite significant here, but > it is at least more or less constant for each test. > > But if we look at the other benchmarks: > > https://gist.github.com/pilshchikov/8a4bdb03a8304136c22c9bf7217ee447 > [Node.js] > https://gist.github.com/pilshchikov/b4351d78ad59e9cd923689c2e387bc80 [PHP] > https://gist.github.com/pilshchikov/08096c78b425e00166a2ffa2aa5f49ce > [Java] > > The extra code that influence the measurement is not equivalent across > all platforms. For example, PHP's `time()` is most probably lighter than > Python `time()`, since it do not give out milliseconds and may address > RTC, not HPET. So the platform-to-platform comparison in your benchmark > does not look completely fair to me. > > The second remark concerns not the measurement procedure, but the object > being measured. > > The only client operation being used is OP_CACHE_PUT with a payload of a > primitive type. (BTW the type is `Long` in case of the Python client; > what about the other clients? Is it `Int`?) I afraid that such an > object, even being properly isolated from the measurement tool, would > show mostly the throughput of the underlying platform's sockets > implementation, not the performance of the client's code. To show the > potential of the thin client itself, more varied and fancy tasks needed, > i. e. serialization/deserialization of the Complex objects. > > But it depends on the goal of the benchmarking. If showing off the raw > platform agility was intended, than this objection is removed. > > Dmitry > > On 10/9/18 10:50 PM, Stepan Pilshchikov wrote: > > Hi, all > > > > Tried to compare new thin client performance and made similar benchmarks > for > > each one > > And result not so good for python > > > > Ticket with results and bench code: > > https://issues.apache.org/jira/browse/IGNITE-9824 > > > > Py code src: > > https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb > > > > Dmitry, please review results and bench code, maybe somthing wrong or > it's > > expected numbers? > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > |
|
|
Stepan!
Can you please update the benchmarks based on my suggestions earlier in this thread: use a cleaner time profiling for the loop, remove unnecessary operations (string formatting, rounding operations), stick to the primitive int value for put operations (agree with Vladimir, let's keep it simple). And let me know the results. On 10/10/18 5:46 PM, Vladimir Ozerov wrote: > Hi Dmitry, > > I agree with your comments on benchmarking code. As more fair > alternative we may measure time of loading N elements into the cache, so > that it will be necessary to call time() only twice. However, provided > that we have real network here, and according to numbers one PUT in > Python client requires about 0.3 msec, I hardly can imagine that call to > time() or round() may dominate in that response time. But I said, we can > easily rule that out by slight benchmark rewrite. > > As far as serialization, I would prefer to keep primitive objects at the > moment, because AFAIK the purpose of the benchmark was to assess > underlying infrastructure, looking for some obvious performance issues. > Every platform have sockets which use more or less the same API of > underlying OS. To the contrast, performance of serialization mechanics > may differ widely between platforms depending on implementation. E.g. in > Java we spend a lot of time on fine-grained tuning, applied a lot of > speculative optimizations. Add to this JIT nature of Java, and you will > hardly ever beat Java serialization engine from any interpreted > language. So primitive data types make good sense to me. > > At this point our goal is not make Python equally fast with other > platforms, but rather to understand why is it slower than others. Ignite > is a product which brings speed to end users. If they do no have speed, > they will not use Ignite. So performance questions are always of great > importance for us. > > Vladimir. > > On Wed, Oct 10, 2018 at 9:57 AM Dmitry Melnichuk > <[hidden email] <mailto:[hidden email]>> > wrote: > > Hi, Stepan! > > I looked at the benchmark code and the overall methodics, discussed it > with fellow programmers, and came up with basically two remarks. > > First of all, I think, the key for the good (or, at least, > unobjectionable) measurement is to isolate the object being measured > from the influence of the measurement tool. The usual pattern we use in > Python looks like: > > ``` > from time import time > > > bench_start = time() > > for _ in range(number_of_tests): > do_test() > > bench_end = time() > > print('Performance is {} tests per second'.format( > number_of_tests / (bench_end - bench_start) > )) > ``` > > I think, you got the idea: the measurement consists almost solely of > the > time taken by our subject function `do_test`. As few other code as > possible influence the result. > > Now, let's take a look at your code: > > https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb > > Ideally, the `while` loop should include only `cache.put(last_key, > some_precomputed_value)`. But instead it also includes: > > - a series of the `time()` calls, which could be mostly excluded from > the measured time, if the measurement done right; each call is probably > addresses the HPET device, or network time, or both, > > - some floating-point calculations, including `round()`, which is > hardly > necessary, > > - formatting and output of the intermediate result. > > I suppose the measurement influence can be quite significant here, but > it is at least more or less constant for each test. > > But if we look at the other benchmarks: > > https://gist.github.com/pilshchikov/8a4bdb03a8304136c22c9bf7217ee447 > [Node.js] > https://gist.github.com/pilshchikov/b4351d78ad59e9cd923689c2e387bc80 > [PHP] > https://gist.github.com/pilshchikov/08096c78b425e00166a2ffa2aa5f49ce > [Java] > > The extra code that influence the measurement is not equivalent across > all platforms. For example, PHP's `time()` is most probably lighter > than > Python `time()`, since it do not give out milliseconds and may address > RTC, not HPET. So the platform-to-platform comparison in your benchmark > does not look completely fair to me. > > The second remark concerns not the measurement procedure, but the > object > being measured. > > The only client operation being used is OP_CACHE_PUT with a payload > of a > primitive type. (BTW the type is `Long` in case of the Python client; > what about the other clients? Is it `Int`?) I afraid that such an > object, even being properly isolated from the measurement tool, would > show mostly the throughput of the underlying platform's sockets > implementation, not the performance of the client's code. To show the > potential of the thin client itself, more varied and fancy tasks > needed, > i. e. serialization/deserialization of the Complex objects. > > But it depends on the goal of the benchmarking. If showing off the raw > platform agility was intended, than this objection is removed. > > Dmitry > > On 10/9/18 10:50 PM, Stepan Pilshchikov wrote: > > Hi, all > > > > Tried to compare new thin client performance and made similar > benchmarks for > > each one > > And result not so good for python > > > > Ticket with results and bench code: > > https://issues.apache.org/jira/browse/IGNITE-9824 > > > > Py code src: > > https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb > > > > Dmitry, please review results and bench code, maybe somthing > wrong or it's > > expected numbers? > > > > > > > > -- > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > |
|
|
Dmitry,
Thanks for review Agree with all suggestions. Made and run new benches without any redundant operations (prints, time calculation), only with "put" in "while true" cycle (almost) Case description, environment, results and code in ticket https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) Please review new metrics Because picture is not changed so much, python still have performance issues ср, 10 окт. 2018 г. в 12:30, Dmitry Melnichuk < [hidden email]>: > Stepan! > > Can you please update the benchmarks based on my suggestions earlier in > this thread: use a cleaner time profiling for the loop, remove > unnecessary operations (string formatting, rounding operations), stick > to the primitive int value for put operations (agree with Vladimir, > let's keep it simple). And let me know the results. > > On 10/10/18 5:46 PM, Vladimir Ozerov wrote: > > Hi Dmitry, > > > > I agree with your comments on benchmarking code. As more fair > > alternative we may measure time of loading N elements into the cache, so > > that it will be necessary to call time() only twice. However, provided > > that we have real network here, and according to numbers one PUT in > > Python client requires about 0.3 msec, I hardly can imagine that call to > > time() or round() may dominate in that response time. But I said, we can > > easily rule that out by slight benchmark rewrite. > > > > As far as serialization, I would prefer to keep primitive objects at the > > moment, because AFAIK the purpose of the benchmark was to assess > > underlying infrastructure, looking for some obvious performance issues. > > Every platform have sockets which use more or less the same API of > > underlying OS. To the contrast, performance of serialization mechanics > > may differ widely between platforms depending on implementation. E.g. in > > Java we spend a lot of time on fine-grained tuning, applied a lot of > > speculative optimizations. Add to this JIT nature of Java, and you will > > hardly ever beat Java serialization engine from any interpreted > > language. So primitive data types make good sense to me. > > > > At this point our goal is not make Python equally fast with other > > platforms, but rather to understand why is it slower than others. Ignite > > is a product which brings speed to end users. If they do no have speed, > > they will not use Ignite. So performance questions are always of great > > importance for us. > > > > Vladimir. > > > > On Wed, Oct 10, 2018 at 9:57 AM Dmitry Melnichuk > > <[hidden email] <mailto:[hidden email]>> > > > wrote: > > > > Hi, Stepan! > > > > I looked at the benchmark code and the overall methodics, discussed > it > > with fellow programmers, and came up with basically two remarks. > > > > First of all, I think, the key for the good (or, at least, > > unobjectionable) measurement is to isolate the object being measured > > from the influence of the measurement tool. The usual pattern we use > in > > Python looks like: > > > > ``` > > from time import time > > > > > > bench_start = time() > > > > for _ in range(number_of_tests): > > do_test() > > > > bench_end = time() > > > > print('Performance is {} tests per second'.format( > > number_of_tests / (bench_end - bench_start) > > )) > > ``` > > > > I think, you got the idea: the measurement consists almost solely of > > the > > time taken by our subject function `do_test`. As few other code as > > possible influence the result. > > > > Now, let's take a look at your code: > > > > https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb > > > > Ideally, the `while` loop should include only `cache.put(last_key, > > some_precomputed_value)`. But instead it also includes: > > > > - a series of the `time()` calls, which could be mostly excluded from > > the measured time, if the measurement done right; each call is > probably > > addresses the HPET device, or network time, or both, > > > > - some floating-point calculations, including `round()`, which is > > hardly > > necessary, > > > > - formatting and output of the intermediate result. > > > > I suppose the measurement influence can be quite significant here, > but > > it is at least more or less constant for each test. > > > > But if we look at the other benchmarks: > > > > https://gist.github.com/pilshchikov/8a4bdb03a8304136c22c9bf7217ee447 > > [Node.js] > > https://gist.github.com/pilshchikov/b4351d78ad59e9cd923689c2e387bc80 > > [PHP] > > https://gist.github.com/pilshchikov/08096c78b425e00166a2ffa2aa5f49ce > > [Java] > > > > The extra code that influence the measurement is not equivalent > across > > all platforms. For example, PHP's `time()` is most probably lighter > > than > > Python `time()`, since it do not give out milliseconds and may > address > > RTC, not HPET. So the platform-to-platform comparison in your > benchmark > > does not look completely fair to me. > > > > The second remark concerns not the measurement procedure, but the > > object > > being measured. > > > > The only client operation being used is OP_CACHE_PUT with a payload > > of a > > primitive type. (BTW the type is `Long` in case of the Python client; > > what about the other clients? Is it `Int`?) I afraid that such an > > object, even being properly isolated from the measurement tool, would > > show mostly the throughput of the underlying platform's sockets > > implementation, not the performance of the client's code. To show the > > potential of the thin client itself, more varied and fancy tasks > > needed, > > i. e. serialization/deserialization of the Complex objects. > > > > But it depends on the goal of the benchmarking. If showing off the > raw > > platform agility was intended, than this objection is removed. > > > > Dmitry > > > > On 10/9/18 10:50 PM, Stepan Pilshchikov wrote: > > > Hi, all > > > > > > Tried to compare new thin client performance and made similar > > benchmarks for > > > each one > > > And result not so good for python > > > > > > Ticket with results and bench code: > > > https://issues.apache.org/jira/browse/IGNITE-9824 > > > > > > Py code src: > > > > https://gist.github.com/pilshchikov/8aff4e30d83f8bac20c5a4a9c3917abb > > > > > > Dmitry, please review results and bench code, maybe somthing > > wrong or it's > > > expected numbers? > > > > > > > > > > > > -- > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > |
|
|
Stepan!
Please tell me one thing about how you created the environment for your benchmarks, namely: how did you install the Python client. Did you just do `pip install pyignite`, or did you pull my working branch from the downstream repository and did `pip install -e ignite/modules/platforms/python`? I highly recommend the latter, because PyPI version do not yet include the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. These improvements can have a noticeable positive effect on speed. Dmitry On 10/10/18 11:06 PM, Степан Пильщиков wrote: > Dmitry, > > Thanks for review > > Agree with all suggestions. > > Made and run new benches without any redundant operations (prints, time > calculation), only with "put" in "while true" cycle (almost) > Case description, environment, results and code in ticket > https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) > > Please review new metrics > Because picture is not changed so much, python still have performance issues > |
|
|
Dmitry,
pip install -e from latest sources On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, <[hidden email]> wrote: > Stepan! > > Please tell me one thing about how you created the environment for your > benchmarks, namely: how did you install the Python client. Did you just > do `pip install pyignite`, or did you pull my working branch from the > downstream repository and did `pip install -e > ignite/modules/platforms/python`? > > I highly recommend the latter, because PyPI version do not yet include > the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. > These improvements can have a noticeable positive effect on speed. > > Dmitry > > On 10/10/18 11:06 PM, Степан Пильщиков wrote: > > Dmitry, > > > > Thanks for review > > > > Agree with all suggestions. > > > > Made and run new benches without any redundant operations (prints, time > > calculation), only with "put" in "while true" cycle (almost) > > Case description, environment, results and code in ticket > > https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) > > > > Please review new metrics > > Because picture is not changed so much, python still have performance > issues > > > |
|
|
Stepan,
Thank you for answering my question and for the updated results. I have ran the profiler (cProfile) against the benchmark code you provided and have not found any particular bottleneck. One cycle took me 434 μs, which is the same order of magnitude as your result. I will need more time to analyze the performance of my client and find a way to improve it. But is it a real stopper/blocker for the client itself? If there are no other issues, may we merge and release the client? Plus submit a jira-improvement to further investigate the performance of the Python platform. Dmitry On 10/11/18 4:19 PM, Степан Пильщиков wrote: > Dmitry, > > pip install -e from latest sources > > On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, <[hidden email]> > wrote: > >> Stepan! >> >> Please tell me one thing about how you created the environment for your >> benchmarks, namely: how did you install the Python client. Did you just >> do `pip install pyignite`, or did you pull my working branch from the >> downstream repository and did `pip install -e >> ignite/modules/platforms/python`? >> >> I highly recommend the latter, because PyPI version do not yet include >> the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. >> These improvements can have a noticeable positive effect on speed. >> >> Dmitry >> >> On 10/10/18 11:06 PM, Степан Пильщиков wrote: >>> Dmitry, >>> >>> Thanks for review >>> >>> Agree with all suggestions. >>> >>> Made and run new benches without any redundant operations (prints, time >>> calculation), only with "put" in "while true" cycle (almost) >>> Case description, environment, results and code in ticket >>> https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) >>> >>> Please review new metrics >>> Because picture is not changed so much, python still have performance >> issues >>> >> > |
|
|
Ok, I've filed a ticket [1] for the performance investigation and targeted
it to 2.8. Feel free to assign it to yourself. I'm going to merge the ticket to master and ignite-2.7 branches by the end of the week, if no one have any objections. [1] - https://issues.apache.org/jira/browse/IGNITE-9850 Best Regards, Igor On Thu, Oct 11, 2018 at 12:18 PM Dmitry Melnichuk < [hidden email]> wrote: > Stepan, > > Thank you for answering my question and for the updated results. > > I have ran the profiler (cProfile) against the benchmark code you > provided and have not found any particular bottleneck. One cycle took me > 434 μs, which is the same order of magnitude as your result. > > I will need more time to analyze the performance of my client and find a > way to improve it. But is it a real stopper/blocker for the client > itself? If there are no other issues, may we merge and release the > client? Plus submit a jira-improvement to further investigate the > performance of the Python platform. > > Dmitry > > On 10/11/18 4:19 PM, Степан Пильщиков wrote: > > Dmitry, > > > > pip install -e from latest sources > > > > On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, < > [hidden email]> > > wrote: > > > >> Stepan! > >> > >> Please tell me one thing about how you created the environment for your > >> benchmarks, namely: how did you install the Python client. Did you just > >> do `pip install pyignite`, or did you pull my working branch from the > >> downstream repository and did `pip install -e > >> ignite/modules/platforms/python`? > >> > >> I highly recommend the latter, because PyPI version do not yet include > >> the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. > >> These improvements can have a noticeable positive effect on speed. > >> > >> Dmitry > >> > >> On 10/10/18 11:06 PM, Степан Пильщиков wrote: > >>> Dmitry, > >>> > >>> Thanks for review > >>> > >>> Agree with all suggestions. > >>> > >>> Made and run new benches without any redundant operations (prints, time > >>> calculation), only with "put" in "while true" cycle (almost) > >>> Case description, environment, results and code in ticket > >>> https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) > >>> > >>> Please review new metrics > >>> Because picture is not changed so much, python still have performance > >> issues > >>> > >> > > > > |
|
|
Dmitry,
I've run the tests and it appears that there are a lot of files without Apache licenses in Python client: [1] (See Unapproved licenses list). Can you either add headers or exclude file from the check, if you can not add one? I.e. to this list: [2] [1] - https://ci.ignite.apache.org/repository/download/IgniteTests24Java8_LicensesHeaders/2069024:id/rat.zip%21/target/rat.txt [2] - https://github.com/apache/ignite/blob/master/parent/pom.xml#L817 Best Regards, Igor On Thu, Oct 11, 2018 at 1:13 PM Igor Sapego <[hidden email]> wrote: > Ok, I've filed a ticket [1] for the performance investigation and targeted > it to 2.8. Feel free to assign it to yourself. > > I'm going to merge the ticket to master and ignite-2.7 branches by the > end of the week, if no one have any objections. > > [1] - https://issues.apache.org/jira/browse/IGNITE-9850 > > Best Regards, > Igor > > > On Thu, Oct 11, 2018 at 12:18 PM Dmitry Melnichuk < > [hidden email]> wrote: > >> Stepan, >> >> Thank you for answering my question and for the updated results. >> >> I have ran the profiler (cProfile) against the benchmark code you >> provided and have not found any particular bottleneck. One cycle took me >> 434 μs, which is the same order of magnitude as your result. >> >> I will need more time to analyze the performance of my client and find a >> way to improve it. But is it a real stopper/blocker for the client >> itself? If there are no other issues, may we merge and release the >> client? Plus submit a jira-improvement to further investigate the >> performance of the Python platform. >> >> Dmitry >> >> On 10/11/18 4:19 PM, Степан Пильщиков wrote: >> > Dmitry, >> > >> > pip install -e from latest sources >> > >> > On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, < >> [hidden email]> >> > wrote: >> > >> >> Stepan! >> >> >> >> Please tell me one thing about how you created the environment for your >> >> benchmarks, namely: how did you install the Python client. Did you just >> >> do `pip install pyignite`, or did you pull my working branch from the >> >> downstream repository and did `pip install -e >> >> ignite/modules/platforms/python`? >> >> >> >> I highly recommend the latter, because PyPI version do not yet include >> >> the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. >> >> These improvements can have a noticeable positive effect on speed. >> >> >> >> Dmitry >> >> >> >> On 10/10/18 11:06 PM, Степан Пильщиков wrote: >> >>> Dmitry, >> >>> >> >>> Thanks for review >> >>> >> >>> Agree with all suggestions. >> >>> >> >>> Made and run new benches without any redundant operations (prints, >> time >> >>> calculation), only with "put" in "while true" cycle (almost) >> >>> Case description, environment, results and code in ticket >> >>> https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) >> >>> >> >>> Please review new metrics >> >>> Because picture is not changed so much, python still have performance >> >> issues >> >>> >> >> >> > >> >> |
|
|
Igor,
I fixed my files. Dmitry On 10/13/18 12:14 AM, Igor Sapego wrote: > Dmitry, > > I've run the tests and it appears that there are a lot of files without > Apache > licenses in Python client: [1] (See Unapproved licenses list). > > Can you either add headers or exclude file from the check, if you can not > add one? I.e. to this list: [2] > > [1] - > https://ci.ignite.apache.org/repository/download/IgniteTests24Java8_LicensesHeaders/2069024:id/rat.zip%21/target/rat.txt > [2] - https://github.com/apache/ignite/blob/master/parent/pom.xml#L817 > > Best Regards, > Igor > > > On Thu, Oct 11, 2018 at 1:13 PM Igor Sapego <[hidden email]> wrote: > >> Ok, I've filed a ticket [1] for the performance investigation and targeted >> it to 2.8. Feel free to assign it to yourself. >> >> I'm going to merge the ticket to master and ignite-2.7 branches by the >> end of the week, if no one have any objections. >> >> [1] - https://issues.apache.org/jira/browse/IGNITE-9850 >> >> Best Regards, >> Igor >> >> >> On Thu, Oct 11, 2018 at 12:18 PM Dmitry Melnichuk < >> [hidden email]> wrote: >> >>> Stepan, >>> >>> Thank you for answering my question and for the updated results. >>> >>> I have ran the profiler (cProfile) against the benchmark code you >>> provided and have not found any particular bottleneck. One cycle took me >>> 434 μs, which is the same order of magnitude as your result. >>> >>> I will need more time to analyze the performance of my client and find a >>> way to improve it. But is it a real stopper/blocker for the client >>> itself? If there are no other issues, may we merge and release the >>> client? Plus submit a jira-improvement to further investigate the >>> performance of the Python platform. >>> >>> Dmitry >>> >>> On 10/11/18 4:19 PM, Степан Пильщиков wrote: >>>> Dmitry, >>>> >>>> pip install -e from latest sources >>>> >>>> On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, < >>> [hidden email]> >>>> wrote: >>>> >>>>> Stepan! >>>>> >>>>> Please tell me one thing about how you created the environment for your >>>>> benchmarks, namely: how did you install the Python client. Did you just >>>>> do `pip install pyignite`, or did you pull my working branch from the >>>>> downstream repository and did `pip install -e >>>>> ignite/modules/platforms/python`? >>>>> >>>>> I highly recommend the latter, because PyPI version do not yet include >>>>> the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. >>>>> These improvements can have a noticeable positive effect on speed. >>>>> >>>>> Dmitry >>>>> >>>>> On 10/10/18 11:06 PM, Степан Пильщиков wrote: >>>>>> Dmitry, >>>>>> >>>>>> Thanks for review >>>>>> >>>>>> Agree with all suggestions. >>>>>> >>>>>> Made and run new benches without any redundant operations (prints, >>> time >>>>>> calculation), only with "put" in "while true" cycle (almost) >>>>>> Case description, environment, results and code in ticket >>>>>> https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) >>>>>> >>>>>> Please review new metrics >>>>>> Because picture is not changed so much, python still have performance >>>>> issues >>>>>> >>>>> >>>> >>> >>> > |
|
|
Hi Stepan,
Re your performance questions. Firstly, please pull out the latest sources from the ignite-7782 branch as it has some performance optimizations recently implemented. Secondly, Hotspot is a pretty advanced dynamic adaptive compiler and it's not fair to compare it to a purely interpreted Python language implementation. To get comparable results, please run your Java benchmark with -Djava.compiler=NONE. It turns the dynamic compiler off at runtime. I just did this exercise myself and got comparable scores. Please let me know if you still see any significant difference with the flag set. Thanks, P. On Fri, Oct 12, 2018 at 10:02 AM Dmitry Melnichuk < [hidden email]> wrote: > Igor, > > I fixed my files. > > Dmitry > > On 10/13/18 12:14 AM, Igor Sapego wrote: > > Dmitry, > > > > I've run the tests and it appears that there are a lot of files without > > Apache > > licenses in Python client: [1] (See Unapproved licenses list). > > > > Can you either add headers or exclude file from the check, if you can not > > add one? I.e. to this list: [2] > > > > [1] - > > > https://ci.ignite.apache.org/repository/download/IgniteTests24Java8_LicensesHeaders/2069024:id/rat.zip%21/target/rat.txt > > [2] - https://github.com/apache/ignite/blob/master/parent/pom.xml#L817 > > > > Best Regards, > > Igor > > > > > > On Thu, Oct 11, 2018 at 1:13 PM Igor Sapego <[hidden email]> wrote: > > > >> Ok, I've filed a ticket [1] for the performance investigation and > targeted > >> it to 2.8. Feel free to assign it to yourself. > >> > >> I'm going to merge the ticket to master and ignite-2.7 branches by the > >> end of the week, if no one have any objections. > >> > >> [1] - https://issues.apache.org/jira/browse/IGNITE-9850 > >> > >> Best Regards, > >> Igor > >> > >> > >> On Thu, Oct 11, 2018 at 12:18 PM Dmitry Melnichuk < > >> [hidden email]> wrote: > >> > >>> Stepan, > >>> > >>> Thank you for answering my question and for the updated results. > >>> > >>> I have ran the profiler (cProfile) against the benchmark code you > >>> provided and have not found any particular bottleneck. One cycle took > me > >>> 434 μs, which is the same order of magnitude as your result. > >>> > >>> I will need more time to analyze the performance of my client and find > a > >>> way to improve it. But is it a real stopper/blocker for the client > >>> itself? If there are no other issues, may we merge and release the > >>> client? Plus submit a jira-improvement to further investigate the > >>> performance of the Python platform. > >>> > >>> Dmitry > >>> > >>> On 10/11/18 4:19 PM, Степан Пильщиков wrote: > >>>> Dmitry, > >>>> > >>>> pip install -e from latest sources > >>>> > >>>> On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, < > >>> [hidden email]> > >>>> wrote: > >>>> > >>>>> Stepan! > >>>>> > >>>>> Please tell me one thing about how you created the environment for > your > >>>>> benchmarks, namely: how did you install the Python client. Did you > just > >>>>> do `pip install pyignite`, or did you pull my working branch from the > >>>>> downstream repository and did `pip install -e > >>>>> ignite/modules/platforms/python`? > >>>>> > >>>>> I highly recommend the latter, because PyPI version do not yet > include > >>>>> the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. > >>>>> These improvements can have a noticeable positive effect on speed. > >>>>> > >>>>> Dmitry > >>>>> > >>>>> On 10/10/18 11:06 PM, Степан Пильщиков wrote: > >>>>>> Dmitry, > >>>>>> > >>>>>> Thanks for review > >>>>>> > >>>>>> Agree with all suggestions. > >>>>>> > >>>>>> Made and run new benches without any redundant operations (prints, > >>> time > >>>>>> calculation), only with "put" in "while true" cycle (almost) > >>>>>> Case description, environment, results and code in ticket > >>>>>> https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) > >>>>>> > >>>>>> Please review new metrics > >>>>>> Because picture is not changed so much, python still have > performance > >>>>> issues > >>>>>> > >>>>> > >>>> > >>> > >>> > > > > |
|
|
Hi
Made new run and client definitely become faster I think its right direction for improvements Also rerun java bench with compiler=none option All results you can get in https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) сб, 13 окт. 2018 г. в 4:56, Pavel Petroshenko <[hidden email]>: > Hi Stepan, > > Re your performance questions. > > Firstly, please pull out the latest sources from the ignite-7782 branch as > it has some performance optimizations recently implemented. > > Secondly, Hotspot is a pretty advanced dynamic adaptive compiler and it's > not fair to compare it to a purely interpreted Python language > implementation. To get comparable results, please run your Java benchmark > with -Djava.compiler=NONE. It turns the dynamic compiler off at runtime. I > just did this exercise myself and got comparable scores. > > Please let me know if you still see any significant difference with the > flag set. > > Thanks, > P. > > On Fri, Oct 12, 2018 at 10:02 AM Dmitry Melnichuk < > [hidden email]> wrote: > >> Igor, >> >> I fixed my files. >> >> Dmitry >> >> On 10/13/18 12:14 AM, Igor Sapego wrote: >> > Dmitry, >> > >> > I've run the tests and it appears that there are a lot of files without >> > Apache >> > licenses in Python client: [1] (See Unapproved licenses list). >> > >> > Can you either add headers or exclude file from the check, if you can >> not >> > add one? I.e. to this list: [2] >> > >> > [1] - >> > >> https://ci.ignite.apache.org/repository/download/IgniteTests24Java8_LicensesHeaders/2069024:id/rat.zip%21/target/rat.txt >> > [2] - https://github.com/apache/ignite/blob/master/parent/pom.xml#L817 >> > >> > Best Regards, >> > Igor >> > >> > >> > On Thu, Oct 11, 2018 at 1:13 PM Igor Sapego <[hidden email]> wrote: >> > >> >> Ok, I've filed a ticket [1] for the performance investigation and >> targeted >> >> it to 2.8. Feel free to assign it to yourself. >> >> >> >> I'm going to merge the ticket to master and ignite-2.7 branches by the >> >> end of the week, if no one have any objections. >> >> >> >> [1] - https://issues.apache.org/jira/browse/IGNITE-9850 >> >> >> >> Best Regards, >> >> Igor >> >> >> >> >> >> On Thu, Oct 11, 2018 at 12:18 PM Dmitry Melnichuk < >> >> [hidden email]> wrote: >> >> >> >>> Stepan, >> >>> >> >>> Thank you for answering my question and for the updated results. >> >>> >> >>> I have ran the profiler (cProfile) against the benchmark code you >> >>> provided and have not found any particular bottleneck. One cycle took >> me >> >>> 434 μs, which is the same order of magnitude as your result. >> >>> >> >>> I will need more time to analyze the performance of my client and >> find a >> >>> way to improve it. But is it a real stopper/blocker for the client >> >>> itself? If there are no other issues, may we merge and release the >> >>> client? Plus submit a jira-improvement to further investigate the >> >>> performance of the Python platform. >> >>> >> >>> Dmitry >> >>> >> >>> On 10/11/18 4:19 PM, Степан Пильщиков wrote: >> >>>> Dmitry, >> >>>> >> >>>> pip install -e from latest sources >> >>>> >> >>>> On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, < >> >>> [hidden email]> >> >>>> wrote: >> >>>> >> >>>>> Stepan! >> >>>>> >> >>>>> Please tell me one thing about how you created the environment for >> your >> >>>>> benchmarks, namely: how did you install the Python client. Did you >> just >> >>>>> do `pip install pyignite`, or did you pull my working branch from >> the >> >>>>> downstream repository and did `pip install -e >> >>>>> ignite/modules/platforms/python`? >> >>>>> >> >>>>> I highly recommend the latter, because PyPI version do not yet >> include >> >>>>> the latest improvements I did thanks to Dmitriy “qvad” Sherstobitov. >> >>>>> These improvements can have a noticeable positive effect on speed. >> >>>>> >> >>>>> Dmitry >> >>>>> >> >>>>> On 10/10/18 11:06 PM, Степан Пильщиков wrote: >> >>>>>> Dmitry, >> >>>>>> >> >>>>>> Thanks for review >> >>>>>> >> >>>>>> Agree with all suggestions. >> >>>>>> >> >>>>>> Made and run new benches without any redundant operations (prints, >> >>> time >> >>>>>> calculation), only with "put" in "while true" cycle (almost) >> >>>>>> Case description, environment, results and code in ticket >> >>>>>> https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) >> >>>>>> >> >>>>>> Please review new metrics >> >>>>>> Because picture is not changed so much, python still have >> performance >> >>>>> issues >> >>>>>> >> >>>>> >> >>>> >> >>> >> >>> >> > >> >> |
|
|
Hi,
I've just merged Python thin client to the master and ignite-2.7 Also, I like how the python's results improved and now are comparable to node.js. Great job, guys! Best Regards, Igor On Sat, Oct 13, 2018 at 3:53 PM Степан Пильщиков <[hidden email]> wrote: > Hi > > Made new run and client definitely become faster > I think its right direction for improvements > > Also rerun java bench with compiler=none option > > All results you can get in > https://issues.apache.org/jira/browse/IGNITE-9824 > (last comment) > > сб, 13 окт. 2018 г. в 4:56, Pavel Petroshenko <[hidden email]>: > > > Hi Stepan, > > > > Re your performance questions. > > > > Firstly, please pull out the latest sources from the ignite-7782 branch > as > > it has some performance optimizations recently implemented. > > > > Secondly, Hotspot is a pretty advanced dynamic adaptive compiler and it's > > not fair to compare it to a purely interpreted Python language > > implementation. To get comparable results, please run your Java benchmark > > with -Djava.compiler=NONE. It turns the dynamic compiler off at runtime. > I > > just did this exercise myself and got comparable scores. > > > > Please let me know if you still see any significant difference with the > > flag set. > > > > Thanks, > > P. > > > > On Fri, Oct 12, 2018 at 10:02 AM Dmitry Melnichuk < > > [hidden email]> wrote: > > > >> Igor, > >> > >> I fixed my files. > >> > >> Dmitry > >> > >> On 10/13/18 12:14 AM, Igor Sapego wrote: > >> > Dmitry, > >> > > >> > I've run the tests and it appears that there are a lot of files > without > >> > Apache > >> > licenses in Python client: [1] (See Unapproved licenses list). > >> > > >> > Can you either add headers or exclude file from the check, if you can > >> not > >> > add one? I.e. to this list: [2] > >> > > >> > [1] - > >> > > >> > https://ci.ignite.apache.org/repository/download/IgniteTests24Java8_LicensesHeaders/2069024:id/rat.zip%21/target/rat.txt > >> > [2] - > https://github.com/apache/ignite/blob/master/parent/pom.xml#L817 > >> > > >> > Best Regards, > >> > Igor > >> > > >> > > >> > On Thu, Oct 11, 2018 at 1:13 PM Igor Sapego <[hidden email]> > wrote: > >> > > >> >> Ok, I've filed a ticket [1] for the performance investigation and > >> targeted > >> >> it to 2.8. Feel free to assign it to yourself. > >> >> > >> >> I'm going to merge the ticket to master and ignite-2.7 branches by > the > >> >> end of the week, if no one have any objections. > >> >> > >> >> [1] - https://issues.apache.org/jira/browse/IGNITE-9850 > >> >> > >> >> Best Regards, > >> >> Igor > >> >> > >> >> > >> >> On Thu, Oct 11, 2018 at 12:18 PM Dmitry Melnichuk < > >> >> [hidden email]> wrote: > >> >> > >> >>> Stepan, > >> >>> > >> >>> Thank you for answering my question and for the updated results. > >> >>> > >> >>> I have ran the profiler (cProfile) against the benchmark code you > >> >>> provided and have not found any particular bottleneck. One cycle > took > >> me > >> >>> 434 μs, which is the same order of magnitude as your result. > >> >>> > >> >>> I will need more time to analyze the performance of my client and > >> find a > >> >>> way to improve it. But is it a real stopper/blocker for the client > >> >>> itself? If there are no other issues, may we merge and release the > >> >>> client? Plus submit a jira-improvement to further investigate the > >> >>> performance of the Python platform. > >> >>> > >> >>> Dmitry > >> >>> > >> >>> On 10/11/18 4:19 PM, Степан Пильщиков wrote: > >> >>>> Dmitry, > >> >>>> > >> >>>> pip install -e from latest sources > >> >>>> > >> >>>> On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, < > >> >>> [hidden email]> > >> >>>> wrote: > >> >>>> > >> >>>>> Stepan! > >> >>>>> > >> >>>>> Please tell me one thing about how you created the environment for > >> your > >> >>>>> benchmarks, namely: how did you install the Python client. Did you > >> just > >> >>>>> do `pip install pyignite`, or did you pull my working branch from > >> the > >> >>>>> downstream repository and did `pip install -e > >> >>>>> ignite/modules/platforms/python`? > >> >>>>> > >> >>>>> I highly recommend the latter, because PyPI version do not yet > >> include > >> >>>>> the latest improvements I did thanks to Dmitriy “qvad” > Sherstobitov. > >> >>>>> These improvements can have a noticeable positive effect on speed. > >> >>>>> > >> >>>>> Dmitry > >> >>>>> > >> >>>>> On 10/10/18 11:06 PM, Степан Пильщиков wrote: > >> >>>>>> Dmitry, > >> >>>>>> > >> >>>>>> Thanks for review > >> >>>>>> > >> >>>>>> Agree with all suggestions. > >> >>>>>> > >> >>>>>> Made and run new benches without any redundant operations > (prints, > >> >>> time > >> >>>>>> calculation), only with "put" in "while true" cycle (almost) > >> >>>>>> Case description, environment, results and code in ticket > >> >>>>>> https://issues.apache.org/jira/browse/IGNITE-9824 (last comment) > >> >>>>>> > >> >>>>>> Please review new metrics > >> >>>>>> Because picture is not changed so much, python still have > >> performance > >> >>>>> issues > >> >>>>>> > >> >>>>> > >> >>>> > >> >>> > >> >>> > >> > > >> > >> > |
|
|
Dmitry,
Since Python thin client is now in master, can you please add it to our "Thin clients features" wiki page, so we can track the feature parity of our clients? Or just tell me which features are supported, so I can fill it myself. [1] - https://cwiki.apache.org/confluence/display/IGNITE/Thin+clients+features Best Regards, Igor On Mon, Oct 15, 2018 at 1:38 PM Igor Sapego <[hidden email]> wrote: > Hi, > > I've just merged Python thin client to the master and ignite-2.7 > > Also, I like how the python's results improved and now are comparable to > node.js. > > Great job, guys! > > Best Regards, > Igor > > > On Sat, Oct 13, 2018 at 3:53 PM Степан Пильщиков < > [hidden email]> wrote: > >> Hi >> >> Made new run and client definitely become faster >> I think its right direction for improvements >> >> Also rerun java bench with compiler=none option >> >> All results you can get in >> https://issues.apache.org/jira/browse/IGNITE-9824 >> (last comment) >> >> сб, 13 окт. 2018 г. в 4:56, Pavel Petroshenko <[hidden email]>: >> >> > Hi Stepan, >> > >> > Re your performance questions. >> > >> > Firstly, please pull out the latest sources from the ignite-7782 branch >> as >> > it has some performance optimizations recently implemented. >> > >> > Secondly, Hotspot is a pretty advanced dynamic adaptive compiler and >> it's >> > not fair to compare it to a purely interpreted Python language >> > implementation. To get comparable results, please run your Java >> benchmark >> > with -Djava.compiler=NONE. It turns the dynamic compiler off at >> runtime. I >> > just did this exercise myself and got comparable scores. >> > >> > Please let me know if you still see any significant difference with the >> > flag set. >> > >> > Thanks, >> > P. >> > >> > On Fri, Oct 12, 2018 at 10:02 AM Dmitry Melnichuk < >> > [hidden email]> wrote: >> > >> >> Igor, >> >> >> >> I fixed my files. >> >> >> >> Dmitry >> >> >> >> On 10/13/18 12:14 AM, Igor Sapego wrote: >> >> > Dmitry, >> >> > >> >> > I've run the tests and it appears that there are a lot of files >> without >> >> > Apache >> >> > licenses in Python client: [1] (See Unapproved licenses list). >> >> > >> >> > Can you either add headers or exclude file from the check, if you can >> >> not >> >> > add one? I.e. to this list: [2] >> >> > >> >> > [1] - >> >> > >> >> >> https://ci.ignite.apache.org/repository/download/IgniteTests24Java8_LicensesHeaders/2069024:id/rat.zip%21/target/rat.txt >> >> > [2] - >> https://github.com/apache/ignite/blob/master/parent/pom.xml#L817 >> >> > >> >> > Best Regards, >> >> > Igor >> >> > >> >> > >> >> > On Thu, Oct 11, 2018 at 1:13 PM Igor Sapego <[hidden email]> >> wrote: >> >> > >> >> >> Ok, I've filed a ticket [1] for the performance investigation and >> >> targeted >> >> >> it to 2.8. Feel free to assign it to yourself. >> >> >> >> >> >> I'm going to merge the ticket to master and ignite-2.7 branches by >> the >> >> >> end of the week, if no one have any objections. >> >> >> >> >> >> [1] - https://issues.apache.org/jira/browse/IGNITE-9850 >> >> >> >> >> >> Best Regards, >> >> >> Igor >> >> >> >> >> >> >> >> >> On Thu, Oct 11, 2018 at 12:18 PM Dmitry Melnichuk < >> >> >> [hidden email]> wrote: >> >> >> >> >> >>> Stepan, >> >> >>> >> >> >>> Thank you for answering my question and for the updated results. >> >> >>> >> >> >>> I have ran the profiler (cProfile) against the benchmark code you >> >> >>> provided and have not found any particular bottleneck. One cycle >> took >> >> me >> >> >>> 434 μs, which is the same order of magnitude as your result. >> >> >>> >> >> >>> I will need more time to analyze the performance of my client and >> >> find a >> >> >>> way to improve it. But is it a real stopper/blocker for the client >> >> >>> itself? If there are no other issues, may we merge and release the >> >> >>> client? Plus submit a jira-improvement to further investigate the >> >> >>> performance of the Python platform. >> >> >>> >> >> >>> Dmitry >> >> >>> >> >> >>> On 10/11/18 4:19 PM, Степан Пильщиков wrote: >> >> >>>> Dmitry, >> >> >>>> >> >> >>>> pip install -e from latest sources >> >> >>>> >> >> >>>> On Thu, 11 Oct 2018, 06:37 Dmitry Melnichuk, < >> >> >>> [hidden email]> >> >> >>>> wrote: >> >> >>>> >> >> >>>>> Stepan! >> >> >>>>> >> >> >>>>> Please tell me one thing about how you created the environment >> for >> >> your >> >> >>>>> benchmarks, namely: how did you install the Python client. Did >> you >> >> just >> >> >>>>> do `pip install pyignite`, or did you pull my working branch from >> >> the >> >> >>>>> downstream repository and did `pip install -e >> >> >>>>> ignite/modules/platforms/python`? >> >> >>>>> >> >> >>>>> I highly recommend the latter, because PyPI version do not yet >> >> include >> >> >>>>> the latest improvements I did thanks to Dmitriy “qvad” >> Sherstobitov. >> >> >>>>> These improvements can have a noticeable positive effect on >> speed. >> >> >>>>> >> >> >>>>> Dmitry >> >> >>>>> >> >> >>>>> On 10/10/18 11:06 PM, Степан Пильщиков wrote: >> >> >>>>>> Dmitry, >> >> >>>>>> >> >> >>>>>> Thanks for review >> >> >>>>>> >> >> >>>>>> Agree with all suggestions. >> >> >>>>>> >> >> >>>>>> Made and run new benches without any redundant operations >> (prints, >> >> >>> time >> >> >>>>>> calculation), only with "put" in "while true" cycle (almost) >> >> >>>>>> Case description, environment, results and code in ticket >> >> >>>>>> https://issues.apache.org/jira/browse/IGNITE-9824 (last >> comment) >> >> >>>>>> >> >> >>>>>> Please review new metrics >> >> >>>>>> Because picture is not changed so much, python still have >> >> performance >> >> >>>>> issues >> >> >>>>>> >> >> >>>>> >> >> >>>> >> >> >>> >> >> >>> >> >> > >> >> >> >> >> > |
|
|
Igor,
I do not have access to edit wiki pages, so please fill it. All the features are supported, except for: - Query API: ScanQuery with filter, - Binary API: Get object in BinaryObject form, BinaryObject building, - Additional Features: Best effort affinity, - Type registration: Enum registration, Type name registration. On 10/15/18 9:17 PM, Igor Sapego wrote: > Dmitry, > > Since Python thin client is now in master, can you please add it to our > "Thin clients features" wiki page, so we can track the feature parity > of our clients? Or just tell me which features are supported, so I can fill > it myself. > > [1] - > https://cwiki.apache.org/confluence/display/IGNITE/Thin+clients+features > > Best Regards, > Igor > |
|
|
Dmitry,
I've trying to use python thin client with Ignite 2.7 it dosn't work Please help to investigate Problem reproduced on several others machines Issue: https://issues.apache.org/jira/browse/IGNITE-9908 вт, 16 окт. 2018 г. в 4:36, Dmitry Melnichuk <[hidden email] >: > Igor, > > I do not have access to edit wiki pages, so please fill it. > > All the features are supported, except for: > > - Query API: ScanQuery with filter, > - Binary API: Get object in BinaryObject form, BinaryObject building, > - Additional Features: Best effort affinity, > - Type registration: Enum registration, Type name registration. > > On 10/15/18 9:17 PM, Igor Sapego wrote: > > Dmitry, > > > > Since Python thin client is now in master, can you please add it to our > > "Thin clients features" wiki page, so we can track the feature parity > > of our clients? Or just tell me which features are supported, so I can > fill > > it myself. > > > > [1] - > > https://cwiki.apache.org/confluence/display/IGNITE/Thin+clients+features > > > > Best Regards, > > Igor > > > |
«
Return to Apache Ignite Developers - Legacy Mail Archive
|
1 view|%1 views
| Free forum by Nabble | Edit this page |