RepairException on C* 2.1.3

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

RepairException on C* 2.1.3

Mark Greene
I'm receiving an exception when I run a repair process via: 'nodetool repair -par <keyspace>'

I'm not sure if this is a bug or not but was curious to know if there was something that can be done to remedy this situation?

Full stack trace from the logs:

ERROR [ValidationExecutor:3] 2015-04-17 18:16:56,174 Validator.java:232 - Failed creating a merkle tree for [repair #ee449ac0-e52d-11e4-bce7-8dc78829adc8 on mykeyspace/mycolumnfamily (-6047392565169616230,-6042578405807739912]], /10.0.111.229 (see log for details)
INFO  [AntiEntropySessions:1] 2015-04-17 18:16:56,175 RepairSession.java:260 - [repair #ee450ff0-e52d-11e4-bce7-8dc78829adc8] new session: will sync /10.0.111.229, /10.0.112.183 on range (-301812044562523205,-262462695890469432] for mykeyspace.[mycolumnfamily]
INFO  [AntiEntropySessions:1] 2015-04-17 18:16:56,175 RepairJob.java:163 - [repair #ee450ff0-e52d-11e4-bce7-8dc78829adc8] requesting merkle trees for mycolumnfamily (to [/10.0.112.183, /10.0.111.229])
ERROR [AntiEntropySessions:2] 2015-04-17 18:16:56,181 RepairSession.java:303 - [repair #ee449ac0-e52d-11e4-bce7-8dc78829adc8] session completed with the following error
org.apache.cassandra.exceptions.RepairException: [repair #ee449ac0-e52d-11e4-bce7-8dc78829adc8 on
mykeyspace/mycolumnfamily, (-6047392565169616230,-6042578405807739912]] Validation failed in /10.0.111.229
    at org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:403) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:132) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
ERROR [ValidationExecutor:3] 2015-04-17 18:16:56,181 CassandraDaemon.java:167 - Exception in thread Thread[ValidationExecutor:3,1,main]
java.lang.NullPointerException: null
    at org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:1277) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.io.sstable.SSTableScanner.getScanner(SSTableScanner.java:62) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1640) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1629) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.db.compaction.LeveledCompactionStrategy$LeveledScanner.<init>(LeveledCompactionStrategy.java:262) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getScanners(LeveledCompactionStrategy.java:189) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.db.compaction.WrappingCompactionStrategy.getScanners(WrappingCompactionStrategy.java:357) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:979) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:95) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:617) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_75]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_75]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
ERROR [AntiEntropySessions:2] 2015-04-17 18:16:56,182 CassandraDaemon.java:167 - Exception in thread Thread[AntiEntropySessions:2,5,RMI Runtime]
java.lang.RuntimeException: org.apache.cassandra.exceptions.RepairException: [repair #ee449ac0-e52d-11e4-bce7-8dc78829adc8 on
mykeyspace/mycolumnfamily, (-6047392565169616230,-6042578405807739912]] Validation failed in /10.0.111.229
    at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[guava-16.0.jar:na]
    at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_75]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_75]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_75]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: org.apache.cassandra.exceptions.RepairException: [repair #ee449ac0-e52d-11e4-bce7-8dc78829adc8 on
mykeyspace/mycolumnfamily, (-6047392565169616230,-6042578405807739912]] Validation failed in /10.0.111.229
    at org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:403) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:132) ~[apache-cassandra-2.1.3.jar:2.1.3]
    at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) ~[apache-cassandra-2.1.3.jar:2.1.3]
    ... 3 common frames omitted

Reply | Threaded
Open this post in threaded view
|

Re: RepairException on C* 2.1.3

Robert Coli-3
On Fri, Apr 17, 2015 at 11:40 AM, Mark Greene <[hidden email]> wrote:
I'm receiving an exception when I run a repair process via: 'nodetool repair -par <keyspace>'

This JIRA claims fixed in 2.1.3, but I believe I have heard at least one other report that it isn't :


If I were you, I would :

a) file a JIRA at http://issues.apache.org
b) reply to the list telling us the URL of your issue

=Rob
 
Reply | Threaded
Open this post in threaded view
|

Re: RepairException on C* 2.1.3

Marcus Eriksson
Issue here is that getPosition returns null


On Fri, Apr 17, 2015 at 10:55 PM, Robert Coli <[hidden email]> wrote:
On Fri, Apr 17, 2015 at 11:40 AM, Mark Greene <[hidden email]> wrote:
I'm receiving an exception when I run a repair process via: 'nodetool repair -par <keyspace>'

This JIRA claims fixed in 2.1.3, but I believe I have heard at least one other report that it isn't :


If I were you, I would :

a) file a JIRA at http://issues.apache.org
b) reply to the list telling us the URL of your issue

=Rob