Discussion: Upgrade Ignite Spark Module's Spark version to 2.3.0

[Ray]

Spark released its newest version 2.3.0 on Feb 28th, so I'd like to open a
discussion about whether we should upgrade Ignite Spark module to to the
latest version.
In the release notes
https://spark.apache.org/releases/spark-release-2-3-0.html, spark 2.3
introduced many useful new features and as well as many performance and
stability changes.

So in the next release, I think we should support the Spark 2.3 in Ignite
Spark module.
I already created a ticket in jira
https://issues.apache.org/jira/browse/IGNITE-8534, and the patch is ready
for review.

Spark 2.3 dropped support for scala 2.10
https://issues.apache.org/jira/browse/SPARK-19810, so we also need to remove
spark-2.10 module if we decide to support Spark 2.3.

Please share your input, Igniters.



--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[dmagda]

Thanks, Ray,

Personally, I'm fine with the proposed changes. According to Maven stats
spark-2.10 we have 12x times fewer downloads for ignite-spark-2.10 vs.
ignite-spark. So it sounds reasonable to drop the legacy Spark version in
favor of the Spark 2.3 optimizations.

--
Denis


On Mon, May 21, 2018 at 7:31 PM, Ray <[hidden email]> wrote:

> Spark released its newest version 2.3.0 on Feb 28th, so I'd like to open a
> discussion about whether we should upgrade Ignite Spark module to to the
> latest version.
> In the release notes
> https://spark.apache.org/releases/spark-release-2-3-0.html, spark 2.3
> introduced many useful new features and as well as many performance and
> stability changes.
>
> So in the next release, I think we should support the Spark 2.3 in Ignite
> Spark module.
> I already created a ticket in jira
> https://issues.apache.org/jira/browse/IGNITE-8534, and the patch is ready
> for review.
>
> Spark 2.3 dropped support for scala 2.10
> https://issues.apache.org/jira/browse/SPARK-19810, so we also need to
> remove
> spark-2.10 module if we decide to support Spark 2.3.
>
> Please share your input, Igniters.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>