Never dropped droppable tombstoned data with LCS

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Never dropped droppable tombstoned data with LCS

Safa Topal
Hello there,

We have a 3 node cluster with Cassandra 2.0.8 version. I am seeing data that should be dropped already. In JMX, I can see that DroppableTombstoneRatio is 0.68 for the column family and the tombstone_threshold was left as default for the CF. We are using LCS on related CF and replication factor of keyspace is set to 3.

We have experienced some downtimes because of "repair" and for that reason, we are reluctant to run repair again. 

How can we get rid of this tombstoned data without running repair? Or are there any other way to run repair without exhausting the cluster? I have seen a lot about "repair -pr", however I am not sure if it will be suitable for our case.

Regards,
Safa T.
Reply | Threaded
Open this post in threaded view
|

Re: Never dropped droppable tombstoned data with LCS

Robert Coli-3
On Sun, Apr 26, 2015 at 1:50 PM, Safa Topal <[hidden email]> wrote:
We have a 3 node cluster with Cassandra 2.0.8 version. I am seeing data that should be dropped already. In JMX, I can see that DroppableTombstoneRatio is 0.68 for the column family and the tombstone_threshold was left as default for the CF. We are using LCS on related CF and replication factor of keyspace is set to 3.

2.0.8 contains significant bugs, I would upgrade to the HEAD of 2.0.x ASAP.

Regarding non-dropped data :


?
 
We have experienced some downtimes because of "repair" and for that reason, we are reluctant to run repair again. 

Consider increasing your gc_grace_seconds to 34 days and running repair once a month, on the first of the month, until you resolve the issue. Not-running repair on a regular schedule will be fatal to consistency of some data.
 
How can we get rid of this tombstoned data without running repair? Or are there any other way to run repair without exhausting the cluster? I have seen a lot about "repair -pr", however I am not sure if it will be suitable for our case.

Repair -pr should always be used when you are repairing your entire cluster; that's what it's for.

How is repair related to the non-purged data? Repair that kills you with tombstones will probably also kill you without tombstones?

=Rob