NodeFilter discussion

[Aleksandr Shapkin]

Hi,

I'm working on adding the NodeFilter functionality to the .NET client
https://issues.apache.org/jira/browse/IGNITE-2890

And there are several points I want to highlight regarding the current
implementation:

1) A node filter is being called on every put which seems redundant. This
might be ok unless we don't cross the JNI boundaries. For .NET, it means at
least an additional serialization on every cache operation.
Can we change this behavior and call CU.affinityNode(node, aff.cacheFilter)
only once per assignment? Or at least to wrap it with some cacheable
function?

2) It feels like the default AttributeNodeFilter is suitable for most of
the cases. I found an existing proposal of removing custom filters, but the
thread seems to be abandoned:
http://apache-ignite-developers.2346864.n4.nabble.com/Replacing-NodeFilter-functionality-with-label-approach-td42874.html#a42942

Do we need to support those filters?

3) I noticed that in reality a filter is limited to the DetachedClusterNode
representation. Consider the following stack trace and the #
createBaselineView

 at org.apache.ignite.internal.processors.cache.GridCacheUtils.affinityNode(GridCacheUtils.java:1332)
          at org.apache.ignite.internal.processors.cluster.BaselineTopology.createBaselineView(BaselineTopology.java:329)
          at org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.recalculateBaselineAssignment(GridAffinityAssignmentCache.java:461)


Thus, any NodeFilter depending on a ClusterNode#addresses
or ClusterNode#metrics will fail due to "Operation is not supported on
DetachedClusterNode".

Therefore if we disallow custom node filters completely, the latter issue
would be resolved as well.

Am I missing something?

--
Alex.

[Pavel Tupitsyn]

Alex,

Thank you for diving into this.
Indeed, custom node filters seem to have some issues.

1. For .NET, I'm fairly certain that we don't need custom node filters.
Let's just make AttributeNodeFilter available natively (as a .NET class).

2. For Java, we should at least fix the "call filter on every cache op"
behavior -
should be a simple fix.


On Tue, Jul 14, 2020 at 12:48 PM Aleksandr Shapkin <[hidden email]>
wrote:

> Hi,
>
> I'm working on adding the NodeFilter functionality to the .NET client
> https://issues.apache.org/jira/browse/IGNITE-2890
>
> And there are several points I want to highlight regarding the current
> implementation:
>
> 1) A node filter is being called on every put which seems redundant. This
> might be ok unless we don't cross the JNI boundaries. For .NET, it means at
> least an additional serialization on every cache operation.
> Can we change this behavior and call CU.affinityNode(node, aff.cacheFilter)
> only once per assignment? Or at least to wrap it with some cacheable
> function?
>
> 2) It feels like the default AttributeNodeFilter is suitable for most of
> the cases. I found an existing proposal of removing custom filters, but the
> thread seems to be abandoned:
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Replacing-NodeFilter-functionality-with-label-approach-td42874.html#a42942
>
> Do we need to support those filters?
>
> 3) I noticed that in reality a filter is limited to the DetachedClusterNode
> representation. Consider the following stack trace and the #
> createBaselineView
>
>  at
> org.apache.ignite.internal.processors.cache.GridCacheUtils.affinityNode(GridCacheUtils.java:1332)
>           at
> org.apache.ignite.internal.processors.cluster.BaselineTopology.createBaselineView(BaselineTopology.java:329)
>           at
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.recalculateBaselineAssignment(GridAffinityAssignmentCache.java:461)
>
>
> Thus, any NodeFilter depending on a ClusterNode#addresses
> or ClusterNode#metrics will fail due to "Operation is not supported on
> DetachedClusterNode".
>
> Therefore if we disallow custom node filters completely, the latter issue
> would be resolved as well.
>
> Am I missing something?
>
> --
> Alex.
>

[Valentin Kulichenko]

This might be a topic for Ignite 3.0, but I do agree that the ability to
create custom node filters is unnecessary and error-prone. Attribute-based
filtering should be more than enough.

-Val

On Tue, Jul 14, 2020 at 3:15 AM Pavel Tupitsyn <[hidden email]> wrote:

> Alex,
>
> Thank you for diving into this.
> Indeed, custom node filters seem to have some issues.
>
> 1. For .NET, I'm fairly certain that we don't need custom node filters.
> Let's just make AttributeNodeFilter available natively (as a .NET class).
>
> 2. For Java, we should at least fix the "call filter on every cache op"
> behavior -
> should be a simple fix.
>
>
> On Tue, Jul 14, 2020 at 12:48 PM Aleksandr Shapkin <[hidden email]>
> wrote:
>
> > Hi,
> >
> > I'm working on adding the NodeFilter functionality to the .NET client
> > https://issues.apache.org/jira/browse/IGNITE-2890
> >
> > And there are several points I want to highlight regarding the current
> > implementation:
> >
> > 1) A node filter is being called on every put which seems redundant. This
> > might be ok unless we don't cross the JNI boundaries. For .NET, it means
> at
> > least an additional serialization on every cache operation.
> > Can we change this behavior and call CU.affinityNode(node,
> aff.cacheFilter)
> > only once per assignment? Or at least to wrap it with some cacheable
> > function?
> >
> > 2) It feels like the default AttributeNodeFilter is suitable for most of
> > the cases. I found an existing proposal of removing custom filters, but
> the
> > thread seems to be abandoned:
> >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Replacing-NodeFilter-functionality-with-label-approach-td42874.html#a42942
> >
> > Do we need to support those filters?
> >
> > 3) I noticed that in reality a filter is limited to the
> DetachedClusterNode
> > representation. Consider the following stack trace and the #
> > createBaselineView
> >
> >  at
> >
> org.apache.ignite.internal.processors.cache.GridCacheUtils.affinityNode(GridCacheUtils.java:1332)
> >           at
> >
> org.apache.ignite.internal.processors.cluster.BaselineTopology.createBaselineView(BaselineTopology.java:329)
> >           at
> >
> org.apache.ignite.internal.processors.affinity.GridAffinityAssignmentCache.recalculateBaselineAssignment(GridAffinityAssignmentCache.java:461)
> >
> >
> > Thus, any NodeFilter depending on a ClusterNode#addresses
> > or ClusterNode#metrics will fail due to "Operation is not supported on
> > DetachedClusterNode".
> >
> > Therefore if we disallow custom node filters completely, the latter issue
> > would be resolved as well.
> >
> > Am I missing something?
> >
> > --
> > Alex.
> >
>