Error message

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Error message

Teodor Sigaev
Hi!
During bulk load into Cassandra it produces following error message every 2-3
minutes:
ERROR [MEMTABLE-FLUSHER-POOL:7] 2009-09-01 19:17:34,628
DebuggableThreadPoolExecutor.java (line 110) Error in ThreadPoolExecutor
java.util.concurrent.ExecutionException: java.lang.NullPointerException
         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
         at java.util.concurrent.FutureTask.get(FutureTask.java:111)
         at
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:86)
         at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
         at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.NullPointerException
         at org.apache.cassandra.db.ColumnIndexer.doIndexing(ColumnIndexer.java:142)
         at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:62)
         at
org.apache.cassandra.db.ColumnFamilySerializer.serializeWithIndexes(ColumnFamilySerializer.java:77)
         at org.apache.cassandra.db.Memtable.flush(Memtable.java:210)
         at
org.apache.cassandra.db.ColumnFamilyStore$2.run(ColumnFamilyStore.java:1234)
         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
         at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
         ... 2 more


Is it something to worry about? Snapshot
apache-cassandra-incubating-2009-09-01_12-31-33
--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/
Reply | Threaded
Open this post in threaded view
|

Re: Error message

Jonathan Ellis-3
That's a bug...  Somehow you got it to try to write out a row
containing zero columns, which isn't supposed to be allowed.

-Jonathan

2009/9/1 Teodor Sigaev <[hidden email]>:

> Hi!
> During bulk load into Cassandra it produces following error message every
> 2-3 minutes:
> ERROR [MEMTABLE-FLUSHER-POOL:7] 2009-09-01 19:17:34,628
> DebuggableThreadPoolExecutor.java (line 110) Error in ThreadPoolExecutor
> java.util.concurrent.ExecutionException: java.lang.NullPointerException
>        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
>        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
>        at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExecute(DebuggableThreadPoolExecutor.java:86)
>        at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1118)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>        at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.NullPointerException
>        at
> org.apache.cassandra.db.ColumnIndexer.doIndexing(ColumnIndexer.java:142)
>        at
> org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:62)
>        at
> org.apache.cassandra.db.ColumnFamilySerializer.serializeWithIndexes(ColumnFamilySerializer.java:77)
>        at org.apache.cassandra.db.Memtable.flush(Memtable.java:210)
>        at
> org.apache.cassandra.db.ColumnFamilyStore$2.run(ColumnFamilyStore.java:1234)
>        at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>        at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>        ... 2 more
>
>
> Is it something to worry about? Snapshot
> apache-cassandra-incubating-2009-09-01_12-31-33
> --
> Teodor Sigaev                                   E-mail: [hidden email]
>                                                   WWW: http://www.sigaev.ru/
>
Reply | Threaded
Open this post in threaded view
|

Re: Error message

Teodor Sigaev

> That's a bug...  Somehow you got it to try to write out a row
> containing zero columns, which isn't supposed to be allowed.

After rechecking of my application, I tried to play with storage-conf.xml and
cassandra.in.sh. Interesting, that error goes away when MemtableSizeInMB returns
to default (was 256) and max heap size (-Xmx1G) is increased to 3G. Cassandra
cluster is running on 7 boxes with 4G RAM on each.

--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/
Reply | Threaded
Open this post in threaded view
|

Re: Error message

Jonathan Ellis-3
If you can come up with a minimal script to reproduce that would be awesome.

2009/9/2 Teodor Sigaev <[hidden email]>:

>
>> That's a bug...  Somehow you got it to try to write out a row
>> containing zero columns, which isn't supposed to be allowed.
>
> After rechecking of my application, I tried to play with storage-conf.xml
> and cassandra.in.sh. Interesting, that error goes away when MemtableSizeInMB
> returns to default (was 256) and max heap size (-Xmx1G) is increased to 3G.
> Cassandra cluster is running on 7 boxes with 4G RAM on each.
>
> --
> Teodor Sigaev                                   E-mail: [hidden email]
>                                                   WWW: http://www.sigaev.ru/
>
Reply | Threaded
Open this post in threaded view
|

Re: Error message

Teodor Sigaev
> If you can come up with a minimal script to reproduce that would be awesome.
Script is in attachment (sorry, perl version). In short - it inserts a row and
immediately removes it. I've run it three in parallel. Casssandra version is
yesterday's cassandra-0.4.0-final.
INFO - Saved Token not found. Using 17570558338530880605478324248305304996
INFO - Cassandra starting up...
INFO - Standard1 has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(Standard1)@15830327
INFO - Sorting Memtable(Standard1)@15830327
INFO - Writing Memtable(Standard1)@15830327
INFO - Completed flushing /spool/cassandra/data/Keyspace1/Standard1-1-Data.db
INFO - Standard1 has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(Standard1)@22655307
INFO - Sorting Memtable(Standard1)@22655307
INFO - Writing Memtable(Standard1)@22655307
ERROR - Error in executor futuretask
java.util.concurrent.ExecutionException: java.lang.AssertionError
         at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
         at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.logFutur
eExceptions(DebuggableThreadPoolExecutor.java:95)
         at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.afterExe
cute(DebuggableThreadPoolExecutor.java:82)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:887)
         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:907)
         at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.AssertionError
         at org.apache.cassandra.db.ColumnIndexer.doIndexing(ColumnIndexer.java:1
07)
         at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:62
)
         at org.apache.cassandra.db.ColumnFamilySerializer.serializeWithIndexes(C
olumnFamilySerializer.java:78)
         at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:22
2)
         at org.apache.cassandra.db.ColumnFamilyStore$2$1.run(ColumnFamilyStore.j
ava:934)
         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:44
1)
         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec
utor.java:885)
         ... 2 more
INFO - Standard1 has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(Standard1)@14600171
INFO - Sorting Memtable(Standard1)@14600171
INFO - Writing Memtable(Standard1)@14600171
INFO - Completed flushing /spool/cassandra/data/Keyspace1/Standard1-3-Data.db




--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/

test.pl (1K) Download Attachment