[jira] [Created] (IGNITE-3962) Cassandra parallel data loader

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Created] (IGNITE-3962) Cassandra parallel data loader

Anton Vinogradov (Jira)
Igor Rudyak created IGNITE-3962:
-----------------------------------

             Summary: Cassandra parallel data loader
                 Key: IGNITE-3962
                 URL: https://issues.apache.org/jira/browse/IGNITE-3962
             Project: Ignite
          Issue Type: Improvement
          Components: ignite-cassandra
            Reporter: Igor Rudyak
            Assignee: Igor Rudyak


There should be an option to warm up Ignite caches backed by Cassandra using *loadCache* method and distributing download process among all the nodes of Ignite cluster - downloading portion of data from Cassandra in parallel.

Each Ignite node responsible for a set of Ignite cache partitions. By providing some custom mapping structure to *loadCache* method it's rather easy to map Ignite partitions to "subpartitions" of Cassandra data slice which we are interested in. Such a way it will be possible to download all the data in parallel by utilizing all Ignite cluster nodes and running multiple download thread on each node.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)