+ dev
Pavel Tupitsin, can you please check that org.apache.ignite.configuration.IgniteConfiguration#setConsistentId has it platform counterpart? I could not find it. Raymond, you can explicitly set a bind address for Ignite with public string Localhost { get; set; }. This will make consistent ID to use only 1 address. Also I would suggest you disable ipv6 if you don't use it. Igniters, I think Ignite needs to do these checks and reports: 1. Output the store path and tell its (1) size or state that it is empty and (2) last data file modification date. 2. Output warning if there are other non-empty storage folders under work directory with their sizes and dates. --Yakov 2017-09-05 4:07 GMT+03:00 Raymond Wilson <[hidden email]>: > Dmitriy, > > > > I set up an XML file based on the default one and added the two elements > you noted. > > > > However, this has brought up an issue in that the XML file and an > IgniteConfiguration instance can’t both be provided to the Ignition.Start() > call. So I changed it to use the DiscoverSPI aspect of IgniteConfiguration > and set LocalAddress to “127.0.0.1” and LocalPort to 47500. > > > > This did change the name of the persistence folder to be “127_0_0_1_47500” > as you suggested. > > > > While this resolves my current issue with the folder name changing, it > still seems fragile as network configuration aspects of the server Ignite > is running on have a direct impact on an internal aspect of its > configuration (ie: the location where to store the persisted data). A DHCP > IP lease renewal or an internal DNS domain change or an internal IT > department change to using IPv6 addressing (among other things) could cause > problems when a node restarts and decides the location of its data is > different. > > > > Do you know how GridGain manage this in their enterprise deployments using > persistence? > > > > Thanks, > Raymond. > > > > *From:* Dmitriy Setrakyan [mailto:[hidden email]] > *Sent:* Tuesday, September 5, 2017 11:41 AM > > *To:* user <[hidden email]> > *Cc:* Raymond Wilson <[hidden email]> > *Subject:* Re: Specifying location of persistent storage location > > > > > > On Mon, Sep 4, 2017 at 4:28 PM, Raymond Wilson <[hidden email]> > wrote: > > Hi, > > > > It’s possible this could cause change in the folder name, though I do not > think this is an issue in my case. Below are three different folder names I > have seen. All use the same port number, but differ in terms of the IPV6 > address (I have also seen variations where the IPv6 address is absent in > the folder name). > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_9cc8_92bc_ > 50c9_6794_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > > > , > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_a58c_2f32_ > 8005_b03d_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_38b4_1_858c_ > f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af4b_55b2_582a_47500 > > > > I start the nodes in my local setup in a well defined order so I would > expect the port to be the same. I did once start a second instance by > mistake and did see the port number incremented in the folder name. > > > > Are you suggesting the two changes you note below will result in the same > folder name being chosen every time, unlike above? > > > > > > Yes, exactly. My suggestions will ensure that you explicitly bind to the > same address every time. > > > > > > > > > > > > Thanks, > > Raymond. > > > > *From:* Dmitriy Setrakyan [mailto:[hidden email]] > *Sent:* Tuesday, September 5, 2017 11:17 AM > *To:* user <[hidden email]> > *Cc:* Raymond Wilson <[hidden email]> > *Subject:* Re: Specifying location of persistent storage location > > > > > > > > On Mon, Sep 4, 2017 at 3:37 PM, Raymond Wilson <[hidden email]> > wrote: > > Hi, > > > > I definitely have not had more than one server node running at the same > time (though there have been more than one client node running on the same > machine). > > > > I suspect what is happening is that one or more of the network interfaces > on the machine can have their address change dynamically. What I thought of > as a GUID is actually (I think) an IPv6 address attached to one of the > interfaces. This aspect of the folder name tends to come and go. > > > > You can see from the folder names below that there are quite a number of > addresses involved. This seems to be fragile (and I certainly see the name > of this folder changing frequently), so I think being able to set it to > something concrete would be a good idea. > > > > > > I think I understand what is happening. Ignite starts off with a default > port, and then starts incrementing it with every new node started on the > same host. Perhaps you start server and client nodes in different order > sometimes which causes server to bind to a different port. > > > > To make sure that your server node binds to the same port all the time, > you should try specifying it explicitly in the server node configuration, > like so (forgive me if this snippet does not compile): > > > > > > > > *<property name="discoverySpi"> <bean > class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > <property name="localPort" value="40123"/> </bean></property>* > > > > Please make sure that the client nodes either don't have any port > configured, or have a different port configured. > > > > You should also make sure that Ignite always binds to the desired local > interface on client and server nodes, by specifying IgniteConfiguration.setLocalHost(...) > property, or like so in XML: > > > > *<property name="localHost" value="my.local.ip.address"/>* > > > > If my theory is correct, Ignite should make sure that the clients and > servers cannot theoretically bind to the same port. I will double check it > with the community and file a ticket if needed. > > > > > |
Ignite.NET does not have IgniteConfiguration.ConsistentId, here is the
ticket: https://issues.apache.org/jira/browse/IGNITE-6249 Workaround is to use Spring XML for that particular property (keep everything else in .NET, configs will be merged): https://apacheignite-net.readme.io/docs/configuration#section-spring-xml On Tue, Sep 5, 2017 at 11:51 AM, Yakov Zhdanov <[hidden email]> wrote: > + dev > Pavel Tupitsin, can you please check that > org.apache.ignite.configuration.IgniteConfiguration#setConsistentId has it > platform counterpart? I could not find it. > > Raymond, you can explicitly set a bind address for Ignite with public > string Localhost { get; set; }. This will make consistent ID to use only 1 > address. Also I would suggest you disable ipv6 if you don't use it. > > Igniters, I think Ignite needs to do these checks and reports: > > 1. Output the store path and tell its (1) size or state that it is empty > and (2) last data file modification date. > 2. Output warning if there are other non-empty storage folders under work > directory with their sizes and dates. > > --Yakov > > 2017-09-05 4:07 GMT+03:00 Raymond Wilson <[hidden email]>: > > > Dmitriy, > > > > > > > > I set up an XML file based on the default one and added the two elements > > you noted. > > > > > > > > However, this has brought up an issue in that the XML file and an > > IgniteConfiguration instance can’t both be provided to the > Ignition.Start() > > call. So I changed it to use the DiscoverSPI aspect of > IgniteConfiguration > > and set LocalAddress to “127.0.0.1” and LocalPort to 47500. > > > > > > > > This did change the name of the persistence folder to be > “127_0_0_1_47500” > > as you suggested. > > > > > > > > While this resolves my current issue with the folder name changing, it > > still seems fragile as network configuration aspects of the server Ignite > > is running on have a direct impact on an internal aspect of its > > configuration (ie: the location where to store the persisted data). A > DHCP > > IP lease renewal or an internal DNS domain change or an internal IT > > department change to using IPv6 addressing (among other things) could > cause > > problems when a node restarts and decides the location of its data is > > different. > > > > > > > > Do you know how GridGain manage this in their enterprise deployments > using > > persistence? > > > > > > > > Thanks, > > Raymond. > > > > > > > > *From:* Dmitriy Setrakyan [mailto:[hidden email]] > > *Sent:* Tuesday, September 5, 2017 11:41 AM > > > > *To:* user <[hidden email]> > > *Cc:* Raymond Wilson <[hidden email]> > > *Subject:* Re: Specifying location of persistent storage location > > > > > > > > > > > > On Mon, Sep 4, 2017 at 4:28 PM, Raymond Wilson < > [hidden email]> > > wrote: > > > > Hi, > > > > > > > > It’s possible this could cause change in the folder name, though I do not > > think this is an issue in my case. Below are three different folder > names I > > have seen. All use the same port number, but differ in terms of the IPV6 > > address (I have also seen variations where the IPv6 address is absent in > > the folder name). > > > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_9cc8_92bc_ > > 50c9_6794_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > > > > > > > , > > > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_a58c_2f32_ > > 8005_b03d_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > > 121_1_192_168_178_27_192_168_3_1_2406_e007_38b4_1_858c_ > > f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af4b_55b2_582a_47500 > > > > > > > > I start the nodes in my local setup in a well defined order so I would > > expect the port to be the same. I did once start a second instance by > > mistake and did see the port number incremented in the folder name. > > > > > > > > Are you suggesting the two changes you note below will result in the same > > folder name being chosen every time, unlike above? > > > > > > > > > > > > Yes, exactly. My suggestions will ensure that you explicitly bind to the > > same address every time. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > Raymond. > > > > > > > > *From:* Dmitriy Setrakyan [mailto:[hidden email]] > > *Sent:* Tuesday, September 5, 2017 11:17 AM > > *To:* user <[hidden email]> > > *Cc:* Raymond Wilson <[hidden email]> > > *Subject:* Re: Specifying location of persistent storage location > > > > > > > > > > > > > > > > On Mon, Sep 4, 2017 at 3:37 PM, Raymond Wilson < > [hidden email]> > > wrote: > > > > Hi, > > > > > > > > I definitely have not had more than one server node running at the same > > time (though there have been more than one client node running on the > same > > machine). > > > > > > > > I suspect what is happening is that one or more of the network interfaces > > on the machine can have their address change dynamically. What I thought > of > > as a GUID is actually (I think) an IPv6 address attached to one of the > > interfaces. This aspect of the folder name tends to come and go. > > > > > > > > You can see from the folder names below that there are quite a number of > > addresses involved. This seems to be fragile (and I certainly see the > name > > of this folder changing frequently), so I think being able to set it to > > something concrete would be a good idea. > > > > > > > > > > > > I think I understand what is happening. Ignite starts off with a default > > port, and then starts incrementing it with every new node started on the > > same host. Perhaps you start server and client nodes in different order > > sometimes which causes server to bind to a different port. > > > > > > > > To make sure that your server node binds to the same port all the time, > > you should try specifying it explicitly in the server node configuration, > > like so (forgive me if this snippet does not compile): > > > > > > > > > > > > > > > > *<property name="discoverySpi"> <bean > > class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > > <property name="localPort" value="40123"/> </bean></property>* > > > > > > > > Please make sure that the client nodes either don't have any port > > configured, or have a different port configured. > > > > > > > > You should also make sure that Ignite always binds to the desired local > > interface on client and server nodes, by specifying IgniteConfiguration. > setLocalHost(...) > > property, or like so in XML: > > > > > > > > *<property name="localHost" value="my.local.ip.address"/>* > > > > > > > > If my theory is correct, Ignite should make sure that the clients and > > servers cannot theoretically bind to the same port. I will double check > it > > with the community and file a ticket if needed. > > > > > > > > > > > |
In reply to this post by yzhdanov
Hi Yakov,
Yes, Dmitriy walked me through how setting the LocalHost results in the folder name for the persistent data to be fixed. I also fixed the port number as this is also an aspect of the folder name. Is there a known issue with IPv6 interfaces on a server hosting Ignite? Thanks, Raymond *From:* Yakov Zhdanov [mailto:[hidden email]] *Sent:* Tuesday, September 5, 2017 8:51 PM *To:* [hidden email]; [hidden email] *Cc:* Dmitriy Setrakyan <[hidden email]> *Subject:* Re: Specifying location of persistent storage location + dev Pavel Tupitsin, can you please check that org.apache.ignite.configuration.IgniteConfiguration#setConsistentId has it platform counterpart? I could not find it. Raymond, you can explicitly set a bind address for Ignite with public string Localhost { get; set; }. This will make consistent ID to use only 1 address. Also I would suggest you disable ipv6 if you don't use it. Igniters, I think Ignite needs to do these checks and reports: 1. Output the store path and tell its (1) size or state that it is empty and (2) last data file modification date. 2. Output warning if there are other non-empty storage folders under work directory with their sizes and dates. --Yakov 2017-09-05 4:07 GMT+03:00 Raymond Wilson <[hidden email]>: Dmitriy, I set up an XML file based on the default one and added the two elements you noted. However, this has brought up an issue in that the XML file and an IgniteConfiguration instance can’t both be provided to the Ignition.Start() call. So I changed it to use the DiscoverSPI aspect of IgniteConfiguration and set LocalAddress to “127.0.0.1” and LocalPort to 47500. This did change the name of the persistence folder to be “127_0_0_1_47500” as you suggested. While this resolves my current issue with the folder name changing, it still seems fragile as network configuration aspects of the server Ignite is running on have a direct impact on an internal aspect of its configuration (ie: the location where to store the persisted data). A DHCP IP lease renewal or an internal DNS domain change or an internal IT department change to using IPv6 addressing (among other things) could cause problems when a node restarts and decides the location of its data is different. Do you know how GridGain manage this in their enterprise deployments using persistence? Thanks, Raymond. *From:* Dmitriy Setrakyan [mailto:[hidden email]] *Sent:* Tuesday, September 5, 2017 11:41 AM *To:* user <[hidden email]> *Cc:* Raymond Wilson <[hidden email]> *Subject:* Re: Specifying location of persistent storage location On Mon, Sep 4, 2017 at 4:28 PM, Raymond Wilson <[hidden email]> wrote: Hi, It’s possible this could cause change in the folder name, though I do not think this is an issue in my case. Below are three different folder names I have seen. All use the same port number, but differ in terms of the IPV6 address (I have also seen variations where the IPv6 address is absent in the folder name). 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_9cc8_92bc_50c9_6794_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 , 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_a58c_2f32_8005_b03d_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_121_1_192_168_178_27_192_168_3_1_2406_e007_38b4_1_858c_f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af4b_55b2_582a_47500 I start the nodes in my local setup in a well defined order so I would expect the port to be the same. I did once start a second instance by mistake and did see the port number incremented in the folder name. Are you suggesting the two changes you note below will result in the same folder name being chosen every time, unlike above? Yes, exactly. My suggestions will ensure that you explicitly bind to the same address every time. Thanks, Raymond. *From:* Dmitriy Setrakyan [mailto:[hidden email]] *Sent:* Tuesday, September 5, 2017 11:17 AM *To:* user <[hidden email]> *Cc:* Raymond Wilson <[hidden email]> *Subject:* Re: Specifying location of persistent storage location On Mon, Sep 4, 2017 at 3:37 PM, Raymond Wilson <[hidden email]> wrote: Hi, I definitely have not had more than one server node running at the same time (though there have been more than one client node running on the same machine). I suspect what is happening is that one or more of the network interfaces on the machine can have their address change dynamically. What I thought of as a GUID is actually (I think) an IPv6 address attached to one of the interfaces. This aspect of the folder name tends to come and go. You can see from the folder names below that there are quite a number of addresses involved. This seems to be fragile (and I certainly see the name of this folder changing frequently), so I think being able to set it to something concrete would be a good idea. I think I understand what is happening. Ignite starts off with a default port, and then starts incrementing it with every new node started on the same host. Perhaps you start server and client nodes in different order sometimes which causes server to bind to a different port. To make sure that your server node binds to the same port all the time, you should try specifying it explicitly in the server node configuration, like so (forgive me if this snippet does not compile): *<property name="discoverySpi"> <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> <property name="localPort" value="40123"/> </bean></property>* Please make sure that the client nodes either don't have any port configured, or have a different port configured. You should also make sure that Ignite always binds to the desired local interface on client and server nodes, by specifying IgniteConfiguration.setLocalHost(...) property, or like so in XML: *<property name="localHost" value="my.local.ip.address"/>* If my theory is correct, Ignite should make sure that the clients and servers cannot theoretically bind to the same port. I will double check it with the community and file a ticket if needed. |
In reply to this post by Pavel Tupitsyn
Hi Pavel,
Thanks for the pointer on how to have both the XML and .Net client configuration supplied to Ignite. Raymond. *From:* Pavel Tupitsyn [mailto:[hidden email]] *Sent:* Tuesday, September 5, 2017 8:56 PM *To:* [hidden email] *Cc:* [hidden email]; Dmitriy Setrakyan <[hidden email]> *Subject:* Re: Specifying location of persistent storage location Ignite.NET does not have IgniteConfiguration.ConsistentId, here is the ticket: https://issues.apache.org/jira/browse/IGNITE-6249 Workaround is to use Spring XML for that particular property (keep everything else in .NET, configs will be merged): https://apacheignite-net.readme.io/docs/configuration#section-spring-xml On Tue, Sep 5, 2017 at 11:51 AM, Yakov Zhdanov <[hidden email]> wrote: + dev Pavel Tupitsin, can you please check that org.apache.ignite.configuration.IgniteConfiguration#setConsistentId has it platform counterpart? I could not find it. Raymond, you can explicitly set a bind address for Ignite with public string Localhost { get; set; }. This will make consistent ID to use only 1 address. Also I would suggest you disable ipv6 if you don't use it. Igniters, I think Ignite needs to do these checks and reports: 1. Output the store path and tell its (1) size or state that it is empty and (2) last data file modification date. 2. Output warning if there are other non-empty storage folders under work directory with their sizes and dates. --Yakov 2017-09-05 4:07 GMT+03:00 Raymond Wilson <[hidden email]>: > Dmitriy, > > > > I set up an XML file based on the default one and added the two elements > you noted. > > > > However, this has brought up an issue in that the XML file and an > IgniteConfiguration instance can’t both be provided to the > call. So I changed it to use the DiscoverSPI aspect of IgniteConfiguration > and set LocalAddress to “127.0.0.1” and LocalPort to 47500. > > > > This did change the name of the persistence folder to be “127_0_0_1_47500” > as you suggested. > > > > While this resolves my current issue with the folder name changing, it > still seems fragile as network configuration aspects of the server Ignite > is running on have a direct impact on an internal aspect of its > configuration (ie: the location where to store the persisted data). A DHCP > IP lease renewal or an internal DNS domain change or an internal IT > department change to using IPv6 addressing (among other things) could > problems when a node restarts and decides the location of its data is > different. > > > > Do you know how GridGain manage this in their enterprise deployments using > persistence? > > > > Thanks, > Raymond. > > > > *From:* Dmitriy Setrakyan [mailto:[hidden email]] > *Sent:* Tuesday, September 5, 2017 11:41 AM > > *To:* user <[hidden email]> > *Cc:* Raymond Wilson <[hidden email]> > *Subject:* Re: Specifying location of persistent storage location > > > > > > On Mon, Sep 4, 2017 at 4:28 PM, Raymond Wilson <[hidden email] > > wrote: > > Hi, > > > > It’s possible this could cause change in the folder name, though I do not > think this is an issue in my case. Below are three different folder names > have seen. All use the same port number, but differ in terms of the IPV6 > address (I have also seen variations where the IPv6 address is absent in > the folder name). > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_9cc8_92bc_ > 50c9_6794_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > > > , > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_9e5_1_a58c_2f32_ > 8005_b03d_2406_e007_9e5_1_c5d8_af4b_55b2_582a_47500 > > 0_0_0_0_0_0_0_1_10_0_75_1_10_3_72_117_127_0_0_1_192_168_ > 121_1_192_168_178_27_192_168_3_1_2406_e007_38b4_1_858c_ > f0ab_bc60_54ab_2406_e007_38b4_1_c5d8_af4b_55b2_582a_47500 > > > > I start the nodes in my local setup in a well defined order so I would > expect the port to be the same. I did once start a second instance by > mistake and did see the port number incremented in the folder name. > > > > Are you suggesting the two changes you note below will result in the same > folder name being chosen every time, unlike above? > > > > > > Yes, exactly. My suggestions will ensure that you explicitly bind to the > same address every time. > > > > > > > > > > > > Thanks, > > Raymond. > > > > *From:* Dmitriy Setrakyan [mailto:[hidden email]] > *Sent:* Tuesday, September 5, 2017 11:17 AM > *To:* user <[hidden email]> > *Cc:* Raymond Wilson <[hidden email]> > *Subject:* Re: Specifying location of persistent storage location > > > > > > > > On Mon, Sep 4, 2017 at 3:37 PM, Raymond Wilson <[hidden email] > > wrote: > > Hi, > > > > I definitely have not had more than one server node running at the same > time (though there have been more than one client node running on the same > machine). > > > > I suspect what is happening is that one or more of the network interfaces > on the machine can have their address change dynamically. What I thought > as a GUID is actually (I think) an IPv6 address attached to one of the > interfaces. This aspect of the folder name tends to come and go. > > > > You can see from the folder names below that there are quite a number of > addresses involved. This seems to be fragile (and I certainly see the name > of this folder changing frequently), so I think being able to set it to > something concrete would be a good idea. > > > > > > I think I understand what is happening. Ignite starts off with a default > port, and then starts incrementing it with every new node started on the > same host. Perhaps you start server and client nodes in different order > sometimes which causes server to bind to a different port. > > > > To make sure that your server node binds to the same port all the time, > you should try specifying it explicitly in the server node configuration, > like so (forgive me if this snippet does not compile): > > > > > > > > *<property name="discoverySpi"> <bean > class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi"> > <property name="localPort" value="40123"/> </bean></property>* > > > > Please make sure that the client nodes either don't have any port > configured, or have a different port configured. > > > > You should also make sure that Ignite always binds to the desired local > interface on client and server nodes, by specifying > property, or like so in XML: > > > > *<property name="localHost" value="my.local.ip.address"/>* > > > > If my theory is correct, Ignite should make sure that the clients and > servers cannot theoretically bind to the same port. I will double check it > with the community and file a ticket if needed. > > > > > |
Free forum by Nabble | Edit this page |