COPY command to export a table to CSV file

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

COPY command to export a table to CSV file

Neha Trivedi
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Kiran mk
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.
Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Kiran mk
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.
Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Neha Trivedi
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.

Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Kiran mk
Hi,

Thanks for the info,

Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?

What is the consistency level ?

Best Regardds,
Kiran.M.K.


On Mon, Apr 20, 2015 at 11:55 AM, Neha Trivedi <[hidden email]> wrote:
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.




--
Best Regards,
Kiran.M.K.
Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Neha Trivedi
Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?
it's all default.

What is the consistency level ?
CL = Qurom

Is there any other way to export a table to CSV?

regards
Neha

On Mon, Apr 20, 2015 at 12:21 PM, Kiran mk <[hidden email]> wrote:
Hi,

Thanks for the info,

Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?

What is the consistency level ?

Best Regardds,
Kiran.M.K.


On Mon, Apr 20, 2015 at 11:55 AM, Neha Trivedi <[hidden email]> wrote:
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.




--
Best Regards,
Kiran.M.K.

Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Sebastian Estevez
Try Brian's cassandra-unloader

All the best,


datastax_logo.png

Sebastián Estévez

Solutions Architect | 954 905 8615 | [hidden email]

linkedin.png facebook.png twitter.png g+.png



DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Mon, Apr 20, 2015 at 12:31 PM, Neha Trivedi <[hidden email]> wrote:
Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?
it's all default.

What is the consistency level ?
CL = Qurom

Is there any other way to export a table to CSV?

regards
Neha

On Mon, Apr 20, 2015 at 12:21 PM, Kiran mk <[hidden email]> wrote:
Hi,

Thanks for the info,

Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?

What is the consistency level ?

Best Regardds,
Kiran.M.K.


On Mon, Apr 20, 2015 at 11:55 AM, Neha Trivedi <[hidden email]> wrote:
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.




--
Best Regards,
Kiran.M.K.


Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Serega Sheypak
hi, what happens if unloader meets blob field?

2015-04-20 23:43 GMT+02:00 Sebastian Estevez <[hidden email]>:
Try Brian's cassandra-unloader

All the best,


datastax_logo.png

Sebastián Estévez

Solutions Architect | <a href="tel:954%20905%208615" value="+19549058615" target="_blank">954 905 8615 | [hidden email]

linkedin.png facebook.png twitter.png g+.png



DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Mon, Apr 20, 2015 at 12:31 PM, Neha Trivedi <[hidden email]> wrote:
Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?
it's all default.

What is the consistency level ?
CL = Qurom

Is there any other way to export a table to CSV?

regards
Neha

On Mon, Apr 20, 2015 at 12:21 PM, Kiran mk <[hidden email]> wrote:
Hi,

Thanks for the info,

Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?

What is the consistency level ?

Best Regardds,
Kiran.M.K.


On Mon, Apr 20, 2015 at 11:55 AM, Neha Trivedi <[hidden email]> wrote:
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.




--
Best Regards,
Kiran.M.K.



Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Sebastian Estevez

All the best,


datastax_logo.png

Sebastián Estévez

Solutions Architect | 954 905 8615 | [hidden email]

linkedin.png facebook.png twitter.png g+.png



DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Mon, Apr 20, 2015 at 5:47 PM, Serega Sheypak <[hidden email]> wrote:
hi, what happens if unloader meets blob field?

2015-04-20 23:43 GMT+02:00 Sebastian Estevez <[hidden email]>:
Try Brian's cassandra-unloader

All the best,


datastax_logo.png

Sebastián Estévez

Solutions Architect | <a href="tel:954%20905%208615" value="+19549058615" target="_blank">954 905 8615 | [hidden email]

linkedin.png facebook.png twitter.png g+.png



DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Mon, Apr 20, 2015 at 12:31 PM, Neha Trivedi <[hidden email]> wrote:
Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?
it's all default.

What is the consistency level ?
CL = Qurom

Is there any other way to export a table to CSV?

regards
Neha

On Mon, Apr 20, 2015 at 12:21 PM, Kiran mk <[hidden email]> wrote:
Hi,

Thanks for the info,

Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?

What is the consistency level ?

Best Regardds,
Kiran.M.K.


On Mon, Apr 20, 2015 at 11:55 AM, Neha Trivedi <[hidden email]> wrote:
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.




--
Best Regards,
Kiran.M.K.




Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Neha Trivedi
Thanks Sebastian, I will try it out.
But I am also curious why is the COPY command failing with Out of Memory Error.

regards
Neha

On Tue, Apr 21, 2015 at 4:35 AM, Sebastian Estevez <[hidden email]> wrote:

All the best,


datastax_logo.png

Sebastián Estévez

Solutions Architect | 954 905 8615 | [hidden email]

linkedin.png facebook.png twitter.png g+.png



DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Mon, Apr 20, 2015 at 5:47 PM, Serega Sheypak <[hidden email]> wrote:
hi, what happens if unloader meets blob field?

2015-04-20 23:43 GMT+02:00 Sebastian Estevez <[hidden email]>:
Try Brian's cassandra-unloader

All the best,


datastax_logo.png

Sebastián Estévez

Solutions Architect | <a href="tel:954%20905%208615" value="+19549058615" target="_blank">954 905 8615 | [hidden email]

linkedin.png facebook.png twitter.png g+.png



DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.

On Mon, Apr 20, 2015 at 12:31 PM, Neha Trivedi <[hidden email]> wrote:
Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?
it's all default.

What is the consistency level ?
CL = Qurom

Is there any other way to export a table to CSV?

regards
Neha

On Mon, Apr 20, 2015 at 12:21 PM, Kiran mk <[hidden email]> wrote:
Hi,

Thanks for the info,

Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?

What is the consistency level ?

Best Regardds,
Kiran.M.K.


On Mon, Apr 20, 2015 at 11:55 AM, Neha Trivedi <[hidden email]> wrote:
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.




--
Best Regards,
Kiran.M.K.





Reply | Threaded
Open this post in threaded view
|

Re: COPY command to export a table to CSV file

Neha Trivedi
In reply to this post by Kiran mk
Values in /etc/security/limits.d/cassandra.conf

# Provided by the cassandra package
cassandra  -  memlock  unlimited
cassandra  -  nofile   100000


On Mon, Apr 20, 2015 at 12:21 PM, Kiran mk <[hidden email]> wrote:
Hi,

Thanks for the info,

Does the nproc,nofile,memlock settings in /etc/security/limits.d/cassandra.conf are set to optimum value ?

What is the consistency level ?

Best Regardds,
Kiran.M.K.


On Mon, Apr 20, 2015 at 11:55 AM, Neha Trivedi <[hidden email]> wrote:
hi,

What is the count of records in the column-family ?
      We have about 38,000 Rows in the column-family for which we are trying to export
What  is the Cassandra Version ?
     We are using Cassandra 2.0.11

MAX_HEAP_SIZE and HEAP_NEWSIZE is the default .
The Server is 8 GB.

regards
Neha

On Mon, Apr 20, 2015 at 11:39 AM, Kiran mk <[hidden email]> wrote:
Hi,

check  the MAX_HEAP_SIZE configuration in cassandra-env.sh environment file 

Also HEAP_NEWSIZE ?

What is the Consistency Level you are using ?

Best REgards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:13 AM, Kiran mk <[hidden email]> wrote:
Seems like the is related to JAVA HEAP Memory.

What is the count of records in the column-family ?

What  is the Cassandra Version ?

Best Regards,
Kiran.M.K.

On Mon, Apr 20, 2015 at 11:08 AM, Neha Trivedi <[hidden email]> wrote:
Hello all,

We are getting the OutOfMemoryError on one of the Node and the Node is down, when we run the export command to get all the data from a table.


Regards
Neha




ERROR [ReadStage:532074] 2015-04-09 01:04:00,603 CassandraDaemon.java (line 199) Exception in thread Thread[ReadStage:532074,5,main]
java.lang.OutOfMemoryError: Java heap space
        at org.apache.cassandra.io.util.RandomAccessReader.readBytes(RandomAccessReader.java:347)
        at org.apache.cassandra.utils.ByteBufferUtil.read(ByteBufferUtil.java:392)
        at org.apache.cassandra.utils.ByteBufferUtil.readWithLength(ByteBufferUtil.java:355)
        at org.apache.cassandra.db.ColumnSerializer.deserializeColumnBody(ColumnSerializer.java:124)
        at org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:85)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:75)
        at org.apache.cassandra.db.Column$1.computeNext(Column.java:64)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:88)
        at org.apache.cassandra.db.columniterator.SimpleSliceReader.computeNext(SimpleSliceReader.java:37)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:82)
        at org.apache.cassandra.db.columniterator.LazyColumnIterator.computeNext(LazyColumnIterator.java:59)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.QueryFilter$2.getNext(QueryFilter.java:157)
        at org.apache.cassandra.db.filter.QueryFilter$2.hasNext(QueryFilter.java:140)
        at org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:200)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
        at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:185)
        at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
        at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:101)
        at org.apache.cassandra.db.RowIteratorFactory$2.getReduced(RowIteratorFactory.java:75)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:115)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:98)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)



--
Best Regards,
Kiran.M.K.



--
Best Regards,
Kiran.M.K.




--
Best Regards,
Kiran.M.K.