cassandra will connect a old no-exist node when I use spring-cassandra-data to connect the cluster

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

cassandra will connect a old no-exist node when I use spring-cassandra-data to connect the cluster

chaoaero
This post has NOT been accepted by the mailing list yet.
I remove two nodes A, B from my cluster several months ago directly by simply shutdown the two virtual machines, I didn't run nodetool decommission If i am not mistaken .

I run "nodetool status" without finding the two dead nodes. But If I use spring-cassandra-data to connect the cluster , some errors occured:
DEBUG com.datastax.driver.core.Connection - [/10.11.215.163:9042-3] connection error
java.net.ConnectException: Connection refused: no further information: /10.11.215.163:9042
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_55]
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) ~[na:1.7.0_55]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150) ~[netty-3.9.0.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105) ~[netty-3.9.0.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79) ~[netty-3.9.0.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318) ~[netty-3.9.0.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ~[netty-3.9.0.Final.jar:na]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_55]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_55]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]
10:05:45.900 [Cassandra Java Driver worker-2] DEBUG com.datastax.driver.core.Cluster - Host /10.136.30.163:9042 is DOWN
10:05:45.901 [Reconnection-0] DEBUG com.datastax.driver.core.Cluster - Failed reconnection to /10.136.30.163:9042 ([/10.136.30.163:9042] Cannot connect), scheduling retry in 2000 milliseconds
10:05:45.901 [Reconnection-1] DEBUG com.datastax.driver.core.Connection - Defuncting connection to /10.11.215.163:9042
com.datastax.driver.core.TransportException: [/10.11.215.163:9042] Cannot connect
        at com.datastax.driver.core.Connection.<init>(Connection.java:104) [cassandra-driver-core-2.0.4.jar:na]
        at com.datastax.driver.core.Connection$Factory.open(Connection.java:445) [cassandra-driver-core-2.0.4.jar:na]
        at com.datastax.driver.core.Cluster$Manager$5.tryReconnect(Cluster.java:1475) [cassandra-driver-core-2.0.4.jar:na]
        at com.datastax.driver.core.AbstractReconnectionHandler.run(AbstractReconnectionHandler.java:91) [cassandra-driver-core-2.0.4.jar:na]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_55]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_55]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_55]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [na:1.7.0_55]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_55]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_55]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_55]

10.11.215.163 is the dead node I mentioned just now.

Now I cannot remove these nodes due to It didn't exsits in "nodetool status" with UUID.

What should I do?
Reply | Threaded
Open this post in threaded view
|

Re: cassandra will connect a old no-exist node when I use spring-cassandra-data to connect the cluster

chaoaero
This post has NOT been accepted by the mailing list yet.
Ask for help