Cassandra 2.1.3: OOM during bootstrap

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Cassandra 2.1.3: OOM during bootstrap

Nathan Bijnens
We are getting a OOM when adding a new node to an existing cluster. In the heapdump we found that this thread caused the OutOfMemory exception:

"SharedPool-Worker-10" daemon prio=5 tid=440 RUNNABLE

at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:48)
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
at org.apache.cassandra.utils.memory.SlabAllocator.getRegion(SlabAllocator.java:137)
at org.apache.cassandra.utils.memory.SlabAllocator.allocate(SlabAllocator.java:97)
at org.apache.cassandra.utils.memory.ContextAllocator.allocate(ContextAllocator.java:57)
at org.apache.cassandra.utils.memory.ContextAllocator.clone(ContextAllocator.java:47)
Local Variable: java.nio.HeapByteBuffer#3483982
at org.apache.cassandra.utils.memory.MemtableBufferAllocator.clone(MemtableBufferAllocator.java:61)
at org.apache.cassandra.db.Memtable.put(Memtable.java:191)
Local Variable: org.apache.cassandra.db.ArrayBackedSortedColumns#26
Local Variable: org.apache.cassandra.db.AtomicBTreeColumns#138773
at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1180)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:385)
Local Variable: org.apache.cassandra.db.BufferDecoratedKey#152828
Local Variable: org.apache.cassandra.db.commitlog.ReplayPosition#8
Local Variable: java.util.Collections$1#6
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:348)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:214)
at org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:54)
Local Variable: java.net.Inet4Address#208
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
Local Variable: org.apache.cassandra.net.MessageDeliveryTask#55
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
Local Variable: java.util.concurrent.Executors$RunnableAdapter#70
at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
Local Variable: org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask#9
Local Variable: org.apache.cassandra.concurrent.SEPWorker#34
at java.lang.Thread.run(Thread.java:745)

We are using Oracle (and tried with OpenJDK before):
java version "1.7.0_76"
Java(TM) SE Runtime Environment (build 1.7.0_76-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.76-b04, mixed mode)

Best regards,
  Nathan

Reply | Threaded
Open this post in threaded view
|

Re: Cassandra 2.1.3: OOM during bootstrap

Robert Coli-3
On Mon, Mar 30, 2015 at 2:08 AM, Nathan Bijnens <[hidden email]> wrote:
We are getting a OOM when adding a new node to an existing cluster. In the heapdump we found that this thread caused the OutOfMemory exception:

"SharedPool-Worker-10" daemon prio=5 tid=440 RUNNABLE

This type of post is probably better handled as a JIRA ticket at http://issues.apache.org

Please let the list know the URL if you do file a JIRA.

=Rob