[DISCUSS] Public API deprecation rules

[Alexey Goncharuk-3]

Igniters,

We would like to add some formal requirements to the API deprecation
process. So far the API deprecation was more like intuitive-driven which
recently led to some disagreement and delaying the AI 2.8 release [1][2].

During the discussion [1] we agreed to add an
annotation @IgniteExperimental [3] which marks APIs which are yet not
stable, dangerous, partially implemented or are allowed to be changed in a
future. The reason to release such APIs is to expose the APIs to users and
collect feedback on the usability and possible bugs.

The argument we did not manage to resolve is whether we are allowed to
deprecate the old APIs _before_ the new ones get stable and get released
without @IgniteExperimental annotation. We decided to resolve the
uncertainty with a vote.

The vote we are going to have is reduced to two choices so far:
 * Never deprecate the old APIs unless the new APIs are stable and released
without @IgniteExperimental. The old APIs javadoc may be updated with a
reference to new APIs to encourage users to evaluate new APIs
 * Allow to deprecate the old APIs even when new APIs are marked
with @IgniteExperimental to explicitly notify users that the new APIs are
available

Nikolay, Andrey, please let us know if we should correct the choices
articulation or add another option for the vote.

--AG

[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
[2]
http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
[3] https://issues.apache.org/jira/browse/IGNITE-12559

[dmagda]

The justification and choices look crystal clear to me.

-
Denis


On Wed, Feb 5, 2020 at 1:36 PM Alexey Goncharuk <[hidden email]>
wrote:

> Igniters,
>
> We would like to add some formal requirements to the API deprecation
> process. So far the API deprecation was more like intuitive-driven which
> recently led to some disagreement and delaying the AI 2.8 release [1][2].
>
> During the discussion [1] we agreed to add an
> annotation @IgniteExperimental [3] which marks APIs which are yet not
> stable, dangerous, partially implemented or are allowed to be changed in a
> future. The reason to release such APIs is to expose the APIs to users and
> collect feedback on the usability and possible bugs.
>
> The argument we did not manage to resolve is whether we are allowed to
> deprecate the old APIs _before_ the new ones get stable and get released
> without @IgniteExperimental annotation. We decided to resolve the
> uncertainty with a vote.
>
> The vote we are going to have is reduced to two choices so far:
>  * Never deprecate the old APIs unless the new APIs are stable and released
> without @IgniteExperimental. The old APIs javadoc may be updated with a
> reference to new APIs to encourage users to evaluate new APIs
>  * Allow to deprecate the old APIs even when new APIs are marked
> with @IgniteExperimental to explicitly notify users that the new APIs are
> available
>
> Nikolay, Andrey, please let us know if we should correct the choices
> articulation or add another option for the vote.
>
> --AG
>
> [1]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
> [2]
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
> [3] https://issues.apache.org/jira/browse/IGNITE-12559
>

[agura]

One comment about first choice.

The formulation has one issue from my point of view: Never deprecate
the old APIs unless the new APIs are stable *and released* without
@IgniteExperimental.

Actually we can introduce new stable API and deprecate old API at the
same time without releasing of new API. For example
IgniteCache.withAsync() method was deprecated and new methods
xxxAsync() were added at the same time as stable alternative. In this
case we didn't collect users' feedback because negative users'
feedback about withAsync() was a driver for this change.

On Thu, Feb 6, 2020 at 6:26 AM Denis Magda <[hidden email]> wrote:

>
> The justification and choices look crystal clear to me.
>
> -
> Denis
>
>
> On Wed, Feb 5, 2020 at 1:36 PM Alexey Goncharuk <[hidden email]> wrote:
>>
>> Igniters,
>>
>> We would like to add some formal requirements to the API deprecation
>> process. So far the API deprecation was more like intuitive-driven which
>> recently led to some disagreement and delaying the AI 2.8 release [1][2].
>>
>> During the discussion [1] we agreed to add an
>> annotation @IgniteExperimental [3] which marks APIs which are yet not
>> stable, dangerous, partially implemented or are allowed to be changed in a
>> future. The reason to release such APIs is to expose the APIs to users and
>> collect feedback on the usability and possible bugs.
>>
>> The argument we did not manage to resolve is whether we are allowed to
>> deprecate the old APIs _before_ the new ones get stable and get released
>> without @IgniteExperimental annotation. We decided to resolve the
>> uncertainty with a vote.
>>
>> The vote we are going to have is reduced to two choices so far:
>>  * Never deprecate the old APIs unless the new APIs are stable and released
>> without @IgniteExperimental. The old APIs javadoc may be updated with a
>> reference to new APIs to encourage users to evaluate new APIs
>>  * Allow to deprecate the old APIs even when new APIs are marked
>> with @IgniteExperimental to explicitly notify users that the new APIs are
>> available
>>
>> Nikolay, Andrey, please let us know if we should correct the choices
>> articulation or add another option for the vote.
>>
>> --AG
>>
>> [1]
>> http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
>> [2]
>> http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
>> [3] https://issues.apache.org/jira/browse/IGNITE-12559

[Alexey Goncharuk]

Agree, I was implying this option. Corrected:

The vote we are going to have is reduced to two choices so far:
 * Never deprecate the old APIs unless the new APIs are stable and released
without @IgniteExperimental. The old APIs javadoc may be updated with a
reference to new APIs to encourage users to evaluate new APIs. The
deprecation and new API release may happen simultaneously if the new API is
not marked with @IgniteExperimental or the annotation is removed in the
same release.
 * Allow to deprecate the old APIs even when new APIs are marked
with @IgniteExperimental to explicitly notify users that the new APIs are
available

Nikolay, any comments from your side?

[Nikolay Izhikov-2]

Hello,

I have two notes

> * Allow to deprecate the old APIs even when new APIs are marked with @IgniteExperimental to explicitly notify users that the new APIs are available

I think the main reason is not «notify users that new APIs are available», but
«Notify users that old APIs will be removed in the next major release AND new APIs are available»

> @IgniteExperimental [3] which marks APIs which are yet not stable, dangerous, partially implemented or are allowed to be changed in a future.

It seems too wide definition of «experimental API»
I think we can’t release «dangerous» API.
For me, `@IgniteExperimental` should mark public APIs that can be slightly improved or changed in the next minor release.

> 6 февр. 2020 г., в 13:23, Andrey Gura <[hidden email]> написал(а):
>
> One comment about first choice.
>
> The formulation has one issue from my point of view: Never deprecate
> the old APIs unless the new APIs are stable *and released* without
> @IgniteExperimental.
>
> Actually we can introduce new stable API and deprecate old API at the
> same time without releasing of new API. For example
> IgniteCache.withAsync() method was deprecated and new methods
> xxxAsync() were added at the same time as stable alternative. In this
> case we didn't collect users' feedback because negative users'
> feedback about withAsync() was a driver for this change.
>
> On Thu, Feb 6, 2020 at 6:26 AM Denis Magda <[hidden email]> wrote:
>>
>> The justification and choices look crystal clear to me.
>>
>> -
>> Denis
>>
>>
>> On Wed, Feb 5, 2020 at 1:36 PM Alexey Goncharuk <[hidden email]> wrote:
>>>
>>> Igniters,
>>>
>>> We would like to add some formal requirements to the API deprecation
>>> process. So far the API deprecation was more like intuitive-driven which
>>> recently led to some disagreement and delaying the AI 2.8 release [1][2].
>>>
>>> During the discussion [1] we agreed to add an
>>> annotation @IgniteExperimental [3] which marks APIs which are yet not
>>> stable, dangerous, partially implemented or are allowed to be changed in a
>>> future. The reason to release such APIs is to expose the APIs to users and
>>> collect feedback on the usability and possible bugs.
>>>
>>> The argument we did not manage to resolve is whether we are allowed to
>>> deprecate the old APIs _before_ the new ones get stable and get released
>>> without @IgniteExperimental annotation. We decided to resolve the
>>> uncertainty with a vote.
>>>
>>> The vote we are going to have is reduced to two choices so far:
>>> * Never deprecate the old APIs unless the new APIs are stable and released
>>> without @IgniteExperimental. The old APIs javadoc may be updated with a
>>> reference to new APIs to encourage users to evaluate new APIs
>>> * Allow to deprecate the old APIs even when new APIs are marked
>>> with @IgniteExperimental to explicitly notify users that the new APIs are
>>> available
>>>
>>> Nikolay, Andrey, please let us know if we should correct the choices
>>> articulation or add another option for the vote.
>>>
>>> --AG
>>>
>>> [1]
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
>>> [2]
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Internal-classes-are-exposed-in-public-API-td45146.html
>>> [3] https://issues.apache.org/jira/browse/IGNITE-12559

[Alexey Goncharuk]

Thanks, updated the justification:

The vote we are going to have is reduced to two choices so far:
 * Never deprecate the old APIs unless the new APIs are stable and released
without @IgniteExperimental. The old APIs javadoc may be updated with a
reference to new APIs to encourage users to evaluate new APIs. The
deprecation and new API release may happen simultaneously if the new API is
not marked with @IgniteExperimental or the annotation is removed in the
same release.
 * Allow to deprecate the old APIs even when new APIs are marked with
@IgniteExperimental to explicitly notify users that old APIs will be
removed in the next major release AND new APIs are available

As for the @IgniteExperimental precise meaning - I do not agree that the
experimental essence is related to the API only. As you suggested in the
separate email, we are going to mark MVCC as experimental because it is in
beta meaning that there are currently known scenarios when the new feature
breaks or simply is not implemented. Yet the API for MVCC cache is quite
stable and adds nothing more than a separate cache mode. Not sure if this
changes the vote, though.

[Ivan Pavlukhin]

A little bit more formalism. The thread subject is about API
deprecation in general. But it looks like that discussion boils down
to a replacement of old API with a new one. I suppose there should be
a possibility to deprecate API for removal if the Community decided to
do so.

> @IgniteExperimental [3] which marks APIs which are yet not stable, dangerous, partially implemented or are allowed to be changed in a future.

"Dangerous" part is not clearly stated in javadocs [1]. But I tend to
think about "experimental" features as not-production-ready so
possibly with dangerous bugs. I suppose that there can be another
flavor of API annotations like "UntableAPI" (or non finalized or some
better term) which can be used when an underlying feature is
production-ready but an API can be changed without preserving
backwards compatibility. We might continue discussion in the thread
about experimental API annotations.

[1] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/lang/IgniteExperimental.java

Best regards,
Ivan Pavlukhin

чт, 6 февр. 2020 г. в 14:50, Alexey Goncharuk <[hidden email]>:

>
> Thanks, updated the justification:
>
> The vote we are going to have is reduced to two choices so far:
>  * Never deprecate the old APIs unless the new APIs are stable and released
> without @IgniteExperimental. The old APIs javadoc may be updated with a
> reference to new APIs to encourage users to evaluate new APIs. The
> deprecation and new API release may happen simultaneously if the new API is
> not marked with @IgniteExperimental or the annotation is removed in the
> same release.
>  * Allow to deprecate the old APIs even when new APIs are marked with
> @IgniteExperimental to explicitly notify users that old APIs will be
> removed in the next major release AND new APIs are available
>
> As for the @IgniteExperimental precise meaning - I do not agree that the
> experimental essence is related to the API only. As you suggested in the
> separate email, we are going to mark MVCC as experimental because it is in
> beta meaning that there are currently known scenarios when the new feature
> breaks or simply is not implemented. Yet the API for MVCC cache is quite
> stable and adds nothing more than a separate cache mode. Not sure if this
> changes the vote, though.

[Alexey Goncharuk]

Ivan, thanks, I agree, added this clause:

The vote we are going to have is reduced to two choices so far:
 * Never deprecate the old APIs unless the new APIs are stable and released
without @IgniteExperimental. The old APIs javadoc may be updated with a
reference to new APIs to encourage users to evaluate new APIs. The
deprecation and new API release may happen simultaneously if the new API is
not marked with @IgniteExperimental or the annotation is removed in the
same release.
 * Allow to deprecate the old APIs even when new APIs are marked with
@IgniteExperimental to explicitly notify users that old APIs will be
removed in the next major release AND new APIs are available
Neither of cases prohibits deprecation of an API without a replacement if
community decided so.

Are we good to go?

[Alexey Goncharuk]

I've created a wiki page based on the vote results [1]. Feel free to update
the page if it seems incomplete.

[1]
https://cwiki.apache.org/confluence/display/IGNITE/Development+Guidelines

чт, 6 февр. 2020 г. в 20:17, Alexey Goncharuk <[hidden email]>:

> Ivan, thanks, I agree, added this clause:
>
> The vote we are going to have is reduced to two choices so far:
>  * Never deprecate the old APIs unless the new APIs are stable and
> released without @IgniteExperimental. The old APIs javadoc may be updated
> with a reference to new APIs to encourage users to evaluate new APIs. The
> deprecation and new API release may happen simultaneously if the new API is
> not marked with @IgniteExperimental or the annotation is removed in the
> same release.
>  * Allow to deprecate the old APIs even when new APIs are marked with
> @IgniteExperimental to explicitly notify users that old APIs will be
> removed in the next major release AND new APIs are available
> Neither of cases prohibits deprecation of an API without a replacement if
> community decided so.
>
> Are we good to go?
>