DateTieredCompactionStrategy and static columns

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

DateTieredCompactionStrategy and static columns

graham sanderson
I have a potential use case I haven’t had a chance to prototype yet, which would normally be a good candidate for DTCS (i.e. data delivered in order and a fixed TTL), however with every write we’d also be updating some static cells (namely a few key/values in a static map<text.text> CQL column). There could also be explicit deletes of keys in the static map, though that’s not 100% necessary.

Since those columns don’t have TTL, without reading thru the code code and/or trying it, I have no idea what effect this has on DTCS (perhaps it needs to use separate sstables for static columns). Has anyone tried this. If not I eventually will and will report back.

smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DateTieredCompactionStrategy and static columns

Jonathan Haddad
I suspect this will kill the benefit of DTCS, but haven't tested it to be 100% here.

The benefit of DTCS is that sstables are selected for compaction based on the age of the data, not their size.  When you mix TTL'ed data and non TTL'ed data, you end up screwing with the "drop the entire SSTable" optimization.  I don't believe this is any different just because you're mixing in static columns.  What I think will happen is you'll end up with an sstable that's almost entirely TTL'ed with a few static columns that will never get compacted or dropped.  Pretty much the worst scenario I can think of.



On Thu, Apr 30, 2015 at 11:21 AM graham sanderson <[hidden email]> wrote:
I have a potential use case I haven’t had a chance to prototype yet, which would normally be a good candidate for DTCS (i.e. data delivered in order and a fixed TTL), however with every write we’d also be updating some static cells (namely a few key/values in a static map<text.text> CQL column). There could also be explicit deletes of keys in the static map, though that’s not 100% necessary.

Since those columns don’t have TTL, without reading thru the code code and/or trying it, I have no idea what effect this has on DTCS (perhaps it needs to use separate sstables for static columns). Has anyone tried this. If not I eventually will and will report back.