Simulating network outages

[Anand Kumar Sankaran]

Hi

I am implementing a segmentation resolver to handle split brain scenarios.  I need to simulate network outages to unit test this behavior.  Any suggestions on how to simulate this?  I see that the TcpDiscoverySpi has timeouts, is there a way to mock TcpDiscoverySpi to think that there is a network outage to simulate such conditions?

Thanks.

--
anand

[Alexey Goncharuk]

Hi,

As a starting point you can take a look
at org.apache.ignite.spi.discovery.tcp.ServerImpl#forceNextNodeFailure
which forces DiscoverySpi to drop next node from the topology (not sure if
this is what you need). Another idea is to create a test discovery SPI
which will terminate all it's threads when a test flag is set. Such nodes
also should be segmented from the topology.

Yakov, do you have other ideas?

2016-08-31 0:06 GMT+03:00 Anand Kumar Sankaran <[hidden email]>:

> Hi
>
> I am implementing a segmentation resolver to handle split brain
> scenarios.  I need to simulate network outages to unit test this behavior.
> Any suggestions on how to simulate this?  I see that the TcpDiscoverySpi
> has timeouts, is there a way to mock TcpDiscoverySpi to think that there is
> a network outage to simulate such conditions?
>
> Thanks.
>
> --
> anand
>

[yzhdanov]

You can shutdown and then restart (after some sleep) network stack on the
host. I remember doing that on Fedora several years ago for exactly the
same purposes.



--Yakov

2016-08-31 15:21 GMT+03:00 Alexey Goncharuk <[hidden email]>:

> Hi,
>
> As a starting point you can take a look
> at org.apache.ignite.spi.discovery.tcp.ServerImpl#forceNextNodeFailure
> which forces DiscoverySpi to drop next node from the topology (not sure if
> this is what you need). Another idea is to create a test discovery SPI
> which will terminate all it's threads when a test flag is set. Such nodes
> also should be segmented from the topology.
>
> Yakov, do you have other ideas?
>
> 2016-08-31 0:06 GMT+03:00 Anand Kumar Sankaran <[hidden email]
> >:
>
> > Hi
> >
> > I am implementing a segmentation resolver to handle split brain
> > scenarios.  I need to simulate network outages to unit test this
> behavior.
> > Any suggestions on how to simulate this?  I see that the TcpDiscoverySpi
> > has timeouts, is there a way to mock TcpDiscoverySpi to think that there
> is
> > a network outage to simulate such conditions?
> >
> > Thanks.
> >
> > --
> > anand
> >
>

[Anand Kumar Sankaran]

Thanks Yakov and Alexey.  Appreciate the help.


On 8/31/16, 6:47 AM, "Yakov Zhdanov" <[hidden email]> wrote:

    You can shutdown and then restart (after some sleep) network stack on the
    host. I remember doing that on Fedora several years ago for exactly the
    same purposes.
   
   
   
    --Yakov
   
    2016-08-31 15:21 GMT+03:00 Alexey Goncharuk <[hidden email]>:
   
    > Hi,
    >
    > As a starting point you can take a look
    > at org.apache.ignite.spi.discovery.tcp.ServerImpl#forceNextNodeFailure
    > which forces DiscoverySpi to drop next node from the topology (not sure if
    > this is what you need). Another idea is to create a test discovery SPI
    > which will terminate all it's threads when a test flag is set. Such nodes
    > also should be segmented from the topology.
    >
    > Yakov, do you have other ideas?
    >
    > 2016-08-31 0:06 GMT+03:00 Anand Kumar Sankaran <[hidden email]
    > >:
    >
    > > Hi
    > >
    > > I am implementing a segmentation resolver to handle split brain
    > > scenarios.  I need to simulate network outages to unit test this
    > behavior.
    > > Any suggestions on how to simulate this?  I see that the TcpDiscoverySpi
    > > has timeouts, is there a way to mock TcpDiscoverySpi to think that there
    > is
    > > a network outage to simulate such conditions?
    > >
    > > Thanks.
    > >
    > > --
    > > anand
    > >
    >