Re: NPE issue with trying to submit Hadoop MapReduce

[dsetrakyan]

Moving this question to the dev list.

Vladimir, any reason why we even allow peerClassLoadingEnabled set to true
in IGFS/HadoopMR? I think the proper approach is to either ignore this
setting or throw exception on startup, no?

D.



On Wed, Dec 9, 2015 at 2:51 AM, Vladimir Ozerov <[hidden email]>
wrote:

> Mike,
>
> Thanks for attaching the project. It helped me got to the bottom of the
> problem. Please remove the following line from example-igfs.xml file:
>
> <property name="peerClassLoadingEnabled" value="true"/>
>
> After that you should be able to use Hadoop Accelerator. This is a kind of
> subtle usability issue. We will address it in the nearest feature ensuring
> that meaningful error/warning is printed in such situations.
> Please let me know if it resolves your problem.
>
> Vladimir.
>
>
> On Wed, Dec 9, 2015 at 3:10 AM, mikep <[hidden email]> wrote:
>
>> Hi Vladimir
>>
>> Heres a recreated example as standalone mini example with the same failing
>> example in case it helps, recreating the same issue.
>>
>> Using intellij,
>>
>> run the IgniteNode class that starts up the ignite node (using spring).
>>
>> then run the TestExample class,
>>
>> this execute a few HDFS commands successfully and sets up a file on
>> IgniteFS
>> using HDFS api.
>> and then executes the same job as mentioned before, on the cluster which
>> suffers the issue.
>>
>>
>> exampleignite.zip
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/n2180/exampleignite.zip
>> >
>>
>> just incase, java version is jdk1.8.0_25.jdk
>>
>> Cheers
>> Mike
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-ignite-users.70518.x6.nabble.com/NPE-issue-with-trying-to-submit-Hadoop-MapReduce-tp2146p2180.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>

[Vladimir Ozerov]

 Currently if peerClassLoadingEnabled=true and Hadoop is in classpath, we
simply do not load Hadoop module silently. This is definitely a problem and
I created a ticket to fix this in 1.6.

On Thu, Dec 10, 2015 at 10:38 AM, Dmitriy Setrakyan <[hidden email]>
wrote:

> Moving this question to the dev list.
>
> Vladimir, any reason why we even allow peerClassLoadingEnabled set to true
> in IGFS/HadoopMR? I think the proper approach is to either ignore this
> setting or throw exception on startup, no?
>
> D.
>
>
>
> On Wed, Dec 9, 2015 at 2:51 AM, Vladimir Ozerov <[hidden email]>
> wrote:
>
>> Mike,
>>
>> Thanks for attaching the project. It helped me got to the bottom of the
>> problem. Please remove the following line from example-igfs.xml file:
>>
>> <property name="peerClassLoadingEnabled" value="true"/>
>>
>> After that you should be able to use Hadoop Accelerator. This is a kind
>> of subtle usability issue. We will address it in the nearest feature
>> ensuring that meaningful error/warning is printed in such situations.
>> Please let me know if it resolves your problem.
>>
>> Vladimir.
>>
>>
>> On Wed, Dec 9, 2015 at 3:10 AM, mikep <[hidden email]>
>> wrote:
>>
>>> Hi Vladimir
>>>
>>> Heres a recreated example as standalone mini example with the same
>>> failing
>>> example in case it helps, recreating the same issue.
>>>
>>> Using intellij,
>>>
>>> run the IgniteNode class that starts up the ignite node (using spring).
>>>
>>> then run the TestExample class,
>>>
>>> this execute a few HDFS commands successfully and sets up a file on
>>> IgniteFS
>>> using HDFS api.
>>> and then executes the same job as mentioned before, on the cluster which
>>> suffers the issue.
>>>
>>>
>>> exampleignite.zip
>>> <
>>> http://apache-ignite-users.70518.x6.nabble.com/file/n2180/exampleignite.zip
>>> >
>>>
>>> just incase, java version is jdk1.8.0_25.jdk
>>>
>>> Cheers
>>> Mike
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-ignite-users.70518.x6.nabble.com/NPE-issue-with-trying-to-submit-Hadoop-MapReduce-tp2146p2180.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>