Cassandra 2.0.12 - lots of "Got an IOException in internalRead!"

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Cassandra 2.0.12 - lots of "Got an IOException in internalRead!"

Caraballo, Rafael

In Cassandra 2.0.12, we are seeing a hundreds of these warning in every node…

 

WARN [Thrift-Selector_16] 2015-03-31 00:48:49,700 Message.java (line 364) Got an IOException in internalRead!

java.io.IOException: Connection reset by peer

        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)

        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)

        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)

        at sun.nio.ch.IOUtil.read(IOUtil.java:197)

        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)

        at org.apache.thrift.transport.TNonblockingSocket.read(TNonblockingSocket.java:141)

        at com.thinkaurelius.thrift.util.mem.Buffer.readFrom(Buffer.java:96)

        at com.thinkaurelius.thrift.Message.internalRead(Message.java:360)

        at com.thinkaurelius.thrift.Message.read(Message.java:146)

        at com.thinkaurelius.thrift.TDisruptorServer$SelectorThread.handleRead(TDisruptorServer.java:588)

        at com.thinkaurelius.thrift.TDisruptorServer$SelectorThread.processKey(TDisruptorServer.java:567)

        at com.thinkaurelius.thrift.TDisruptorServer$AbstractSelectorThread.select(TDisruptorServer.java:423)

        at com.thinkaurelius.thrift.TDisruptorServer$AbstractSelectorThread.run(TDisruptorServer.java:383)

 

Rafael Caraballo



This E-mail and any of its attachments may contain Time Warner Cable proprietary information, which is privileged, confidential, or subject to copyright belonging to Time Warner Cable. This E-mail is intended solely for the use of the individual or entity to which it is addressed. If you are not the intended recipient of this E-mail, you are hereby notified that any dissemination, distribution, copying, or action taken in relation to the contents of and attachments to this E-mail is strictly prohibited and may be unlawful. If you have received this E-mail in error, please notify the sender immediately and permanently delete the original and any copy of this E-mail and any printout.
Reply | Threaded
Open this post in threaded view
|

Re: Cassandra 2.0.12 - lots of "Got an IOException in internalRead!"

Michael Shuler
On 03/31/2015 10:15 AM, Caraballo, Rafael wrote:

> In Cassandra 2.0.12, we are seeing a hundreds of these warning in every
> nodeā€¦
>
> WARN [Thrift-Selector_16] 2015-03-31 00:48:49,700 Message.java (line
> 364) Got an IOException in internalRead!
>
> java.io.IOException: Connection reset by peer
>
>          at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>
>          at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)

This is indicative of network issues. Check ifconfig for errors/dropped
values reported on the nodes and talk to your NOC to see if there are
errors on switches, etc.

--
Michael