Ilya Kasnacheev created IGNITE-12751:
----------------------------------------
Summary: callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows
Key: IGNITE-12751
URL:
https://issues.apache.org/jira/browse/IGNITE-12751 Project: Ignite
Issue Type: Bug
Components: compute
Affects Versions: 2.7.6
Reporter: Ilya Kasnacheev
Attachments: word-count-reproducer.zip
Please consider attached reproducer and linked report.
{{compute.callAsync(jobs, reducer);
Result [res=33, tookMs=81, jobs=5] //warm up
Result [res=99, tookMs=21, jobs=15]
Result [res=330, tookMs=22, jobs=50]
Result [res=990, tookMs=57, jobs=150]
Result [res=3300, tookMs=146, jobs=500]
Result [res=9900, tookMs=231, jobs=1500]
Result [res=33000, tookMs=840, jobs=5000]
Result [res=99000, tookMs=6965, jobs=15000]
Result [res=330000, tookMs=118394, jobs=50000]}}
As soon jobs.size() grows past 5000, performance begins to degrade quadratically.
I don't expect that it will be completely linear, but I would assume that it should stay linear-ish until size() hits at least 100000, given that we see clusters of 100 nodes and it's not unthinkable to expect 1000 jobs to be run on each node. 5000 jobs (which still give OK performance) / 100 nodes is just 50 jobs per node, which becomes limiting factor.
Linked question also mentions OOM event, which may be caused of intermediate storage of (N^2) data.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
