Ignite and Scala 2.10/2.11

[Alexey Kuznetsov-2]

Igniters,

We have an open issue IGNITE-389: Integration with Spark: IgniteRDD.
And Spark uses Scala 2.10 (could be build with Scala 2.11).
Ignite uses Scala 2.11.

So we need to some how solve this situation.
After some investigations (take a look in Spark pom) I found how to build
Ignite with Scala 2.10 and Scala 2.11.

But there two open questions:
 #1. What should be default version of Scala? I think Scala 2.10 (for
compatibility with Spark out of the box).

 #2. We need to upload on maven two versions of modules that depend on
Scala? How we should name them? I think names should be like:
ignite-scalar-${scala.version}.

Any comments?



--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

[Sergi]

How hard will it be to support both versions of Scala in the project?

Also what do you this about GridGain's Visor, will it work with both
versions? I don't think we should break known third party software based on
Apache Ignite.

Sergi

2015-06-01 13:50 GMT+03:00 Alexey Kuznetsov <[hidden email]>:

> Igniters,
>
> We have an open issue IGNITE-389: Integration with Spark: IgniteRDD.
> And Spark uses Scala 2.10 (could be build with Scala 2.11).
> Ignite uses Scala 2.11.
>
> So we need to some how solve this situation.
> After some investigations (take a look in Spark pom) I found how to build
> Ignite with Scala 2.10 and Scala 2.11.
>
> But there two open questions:
>  #1. What should be default version of Scala? I think Scala 2.10 (for
> compatibility with Spark out of the box).
>
>  #2. We need to upload on maven two versions of modules that depend on
> Scala? How we should name them? I think names should be like:
> ignite-scalar-${scala.version}.
>
> Any comments?
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>

[Alexey Kuznetsov-2]

1. I think not so hard. We don't use any specific features of scala 2.11 in
scalar and ignite-visor-console.
2. GG Visor does not depend on ignite-scalar. It uses scala 2.11 from its
own classpath. Not a problem.
3. About third parties - I don't hanged a single line of code. Just
introduced two artifacts: scalar-2.10 and scalar-2.11.

On Mon, Jun 1, 2015 at 9:52 PM, Sergi Vladykin <[hidden email]>
wrote:

> How hard will it be to support both versions of Scala in the project?
>
> Also what do you this about GridGain's Visor, will it work with both
> versions? I don't think we should break known third party software based on
> Apache Ignite.
>
> Sergi
>
> 2015-06-01 13:50 GMT+03:00 Alexey Kuznetsov <[hidden email]>:
>
> > Igniters,
> >
> > We have an open issue IGNITE-389: Integration with Spark: IgniteRDD.
> > And Spark uses Scala 2.10 (could be build with Scala 2.11).
> > Ignite uses Scala 2.11.
> >
> > So we need to some how solve this situation.
> > After some investigations (take a look in Spark pom) I found how to build
> > Ignite with Scala 2.10 and Scala 2.11.
> >
> > But there two open questions:
> >  #1. What should be default version of Scala? I think Scala 2.10 (for
> > compatibility with Spark out of the box).
> >
> >  #2. We need to upload on maven two versions of modules that depend on
> > Scala? How we should name them? I think names should be like:
> > ignite-scalar-${scala.version}.
> >
> > Any comments?
> >
> >
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
> >
>

--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

[nivanov]

Generally, I'd vote for 2.11 not because of any language features but
because of stability (and tooling around). 2.11 is very stable (comparing
to 2.10).

--
Nikita Ivanov


On Mon, Jun 1, 2015 at 8:18 AM, Alexey Kuznetsov <[hidden email]>
wrote:

> 1. I think not so hard. We don't use any specific features of scala 2.11 in
> scalar and ignite-visor-console.
> 2. GG Visor does not depend on ignite-scalar. It uses scala 2.11 from its
> own classpath. Not a problem.
> 3. About third parties - I don't hanged a single line of code. Just
> introduced two artifacts: scalar-2.10 and scalar-2.11.
>
> On Mon, Jun 1, 2015 at 9:52 PM, Sergi Vladykin <[hidden email]>
> wrote:
>
> > How hard will it be to support both versions of Scala in the project?
> >
> > Also what do you this about GridGain's Visor, will it work with both
> > versions? I don't think we should break known third party software based
> on
> > Apache Ignite.
> >
> > Sergi
> >
> > 2015-06-01 13:50 GMT+03:00 Alexey Kuznetsov <[hidden email]>:
> >
> > > Igniters,
> > >
> > > We have an open issue IGNITE-389: Integration with Spark: IgniteRDD.
> > > And Spark uses Scala 2.10 (could be build with Scala 2.11).
> > > Ignite uses Scala 2.11.
> > >
> > > So we need to some how solve this situation.
> > > After some investigations (take a look in Spark pom) I found how to
> build
> > > Ignite with Scala 2.10 and Scala 2.11.
> > >
> > > But there two open questions:
> > >  #1. What should be default version of Scala? I think Scala 2.10 (for
> > > compatibility with Spark out of the box).
> > >
> > >  #2. We need to upload on maven two versions of modules that depend on
> > > Scala? How we should name them? I think names should be like:
> > > ignite-scalar-${scala.version}.
> > >
> > > Any comments?
> > >
> > >
> > >
> > > --
> > > Alexey Kuznetsov
> > > GridGain Systems
> > > www.gridgain.com
> > >
> >
>
>
>
> --
> Alexey Kuznetsov
> GridGain Systems
> www.gridgain.com
>

[dsetrakyan]

On Mon, Jun 1, 2015 at 9:01 AM, Nikita Ivanov <[hidden email]> wrote:

> Generally, I'd vote for 2.11 not because of any language features but
> because of stability (and tooling around). 2.11 is very stable (comparing
> to 2.10).
>

If we go with 2.11, how do we make it work with Spark which runs on Scala
2.10?

[Alexey Kuznetsov-2]

How about to use by default current stable Scala and generate artifacts for
older version (if needed).
So by default Ignite will use Scala 2.11 for now. And on maven will be also
deployed Scala 2.10 compatible artifacts (scalar and visor-console).
For those who want to build manually, maven command will be added to
DEVNOTES: mvn clean package -DskipTests -Dscala-2.10

I implemented (partially)  such approach in ignite-389-2 (SPARK integration
branch).
Alexey G. could you take a look?

On Mon, Jun 1, 2015 at 11:06 PM, Dmitriy Setrakyan <[hidden email]>
wrote:

> On Mon, Jun 1, 2015 at 9:01 AM, Nikita Ivanov <[hidden email]> wrote:
>
> > Generally, I'd vote for 2.11 not because of any language features but
> > because of stability (and tooling around). 2.11 is very stable (comparing
> > to 2.10).
> >
>
> If we go with 2.11, how do we make it work with Spark which runs on Scala
> 2.10?
>

--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

[Branko Čibej]

On 01.06.2015 18:06, Dmitriy Setrakyan wrote:
> On Mon, Jun 1, 2015 at 9:01 AM, Nikita Ivanov <[hidden email]> wrote:
>
>> Generally, I'd vote for 2.11 not because of any language features but
>> because of stability (and tooling around). 2.11 is very stable (comparing
>> to 2.10).
>>
> If we go with 2.11, how do we make it work with Spark which runs on Scala
> 2.10?

Teach Spark to use Scala 2.11, of course.

I find it inconceivable that minor Scala versions are incompatible to
this extent. Looks like a major fail to me ... but not that uncommon,
notice that Ruby has the same lack of backward compatibility.

-- Brane

[Alexey Kuznetsov-2]

Brane,

Spark already has limited Scala 2.11 support.
http://spark.apache.org/docs/latest/building-spark.html#building-for-scala-211
Spark has problems with third party libs: "Scala 2.11 support in Spark does
not support a few features due to dependencies which are themselves not
Scala 2.11 ready. Specifically, Spark’s external Kafka library and JDBC
component are not yet supported in Scala 2.11 builds."

On Tue, Jun 2, 2015 at 4:52 PM, Branko Čibej <[hidden email]> wrote:

> On 01.06.2015 18:06, Dmitriy Setrakyan wrote:
> > On Mon, Jun 1, 2015 at 9:01 AM, Nikita Ivanov <[hidden email]>
> wrote:
> >
> >> Generally, I'd vote for 2.11 not because of any language features but
> >> because of stability (and tooling around). 2.11 is very stable
> (comparing
> >> to 2.10).
> >>
> > If we go with 2.11, how do we make it work with Spark which runs on Scala
> > 2.10?
>
> Teach Spark to use Scala 2.11, of course.
>
> I find it inconceivable that minor Scala versions are incompatible to
> this extent. Looks like a major fail to me ... but not that uncommon,
> notice that Ruby has the same lack of backward compatibility.
>
> -- Brane
>

--
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

[Konstantin Boudnik-2]

On Tue, Jun 02, 2015 at 11:52AM, Branko Čibej wrote:

> On 01.06.2015 18:06, Dmitriy Setrakyan wrote:
> > On Mon, Jun 1, 2015 at 9:01 AM, Nikita Ivanov <[hidden email]> wrote:
> >
> >> Generally, I'd vote for 2.11 not because of any language features but
> >> because of stability (and tooling around). 2.11 is very stable (comparing
> >> to 2.10).
> >>
> > If we go with 2.11, how do we make it work with Spark which runs on Scala
> > 2.10?
>
> Teach Spark to use Scala 2.11, of course.
>
> I find it inconceivable that minor Scala versions are incompatible to
> this extent. Looks like a major fail to me ... but not that uncommon,
> notice that Ruby has the same lack of backward compatibility.

A need to support of 2 *minor* versions of Scala is ludicrous, of course.
[...snipping my own muttering about Scala infamous backward compatibility...]

But here's another catch: if both dev. env. for 2.10 and 2.11 would be
required in order to build Ignite it will complicate the life of the
downstream integrators, most obviously Bigtop, where we'll have to support 2
versions of Scala in the toolchain. Which it unlikely to happen as Bigtop
isn't in business of paying for silly Scala decisions.

Cos