[jira] [Created] (IGNITE-11235) Sending Continuous Query with absent class on client knocks out server node

[Anton Vinogradov (Jira)]

Ilya Kasnacheev created IGNITE-11235:
----------------------------------------

             Summary: Sending Continuous Query with absent class on client knocks out server node
                 Key: IGNITE-11235
                 URL: https://issues.apache.org/jira/browse/IGNITE-11235
             Project: Ignite
          Issue Type: Bug
    Affects Versions: 2.7
            Reporter: Ilya Kasnacheev


*In 2.7*, when you create a Continuous Query on client with class which is not present in server classpath, server node will be terminated immediately.
{code}
[2019-02-05 18:20:22,508][ERROR][tcp-disco-msg-worker-#2][root] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Failed to unmarshal discovery custom message: TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0, id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0, verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0, failedNodes=null, isClient=false]]]]
class org.apache.ignite.IgniteException: Failed to unmarshal discovery custom message: TcpDiscoveryCustomEventMessage [msg=null, super=TcpDiscoveryAbstractMessage [sndNodeId=18424a1f-a22a-4401-83f7-2040a30e28f0, id=992de3eb861-18424a1f-a22a-4401-83f7-2040a30e28f0, verifierNodeId=61ff15f0-6337-4e00-ab29-7173a0fb0e1f, topVer=2, pendingIdx=0, failedNodes=null, isClient=false]]
{code}

This means that one rogue client (e.g. with mismatched code base version) can knock whole cluster offline.

I think we should disconnect client from discovery instead when it sends incorrect discovery message.

*In 2.6*, if you tried to create a Continuous Query on client with class which is not present in server classpath, you will get the following erorr on server:
{code}
[15:32:57,844][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to unmarshal discovery custom message.
class org.apache.ignite.IgniteCheckedException: Failed to find class with given class loader for unmarshalling (make sure same versions of all classes are available on all nodes or enable peer-class-loading) [clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
        at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
        at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
        at org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.notifyDiscoveryListener(ServerImpl.java:5476)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5305)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
        at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
        at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
        at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
        at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
        at org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
        at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
        at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
        ... 12 more
[15:32:57,846][SEVERE][tcp-disco-msg-worker-#2][TcpDiscoverySpi] Failed to unmarshal discovery custom message.
class org.apache.ignite.IgniteCheckedException: Failed to find class with given class loader for unmarshalling (make sure same versions of all classes are available on all nodes or enable peer-class-loading) [clsLdr=sun.misc.Launcher$AppClassLoader@764c12b6, cls=com.pany.CQTest$1]
        at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:144)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
        at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:161)
        at org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:82)
        at org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9962)
        at org.apache.ignite.spi.discovery.tcp.messages.TcpDiscoveryCustomEventMessage.message(TcpDiscoveryCustomEventMessage.java:81)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processCustomMessage(ServerImpl.java:5323)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2765)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775)
        at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621)
        at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.lang.ClassNotFoundException: com.pany.CQTest$1
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at org.apache.ignite.internal.util.IgniteUtils.forName(IgniteUtils.java:8608)
        at org.apache.ignite.marshaller.jdk.JdkMarshallerObjectInputStream.resolveClass(JdkMarshallerObjectInputStream.java:59)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
        at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryHandlerV2.readExternal(CacheContinuousQueryHandlerV2.java:169)
        at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
        at org.apache.ignite.internal.processors.continuous.StartRequestData.readExternal(StartRequestData.java:260)
        at java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2076)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2025)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
        at org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:139)
        ... 11 more
{code}

and client thread will wait forever:
{code}
"main" #1 prio=5 os_prio=0 tid=0x00007f403800d000 nid=0x1fa2 waiting on condition [0x00007f403e74a000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
        at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:177)
        at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
        at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery0(CacheContinuousQueryManager.java:694)
        at org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager.executeQuery(CacheContinuousQueryManager.java:517)
        at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.queryContinuous(IgniteCacheProxyImpl.java:569)
        at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:658)
        at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:344)
{code}

But otherwise both client and server are intact and no hanging PME occurs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)