Durability With Cassandra

After an RTFM fail pointed out by Project Chair Jonathan Ellis about how Cassandra handles durability I thought I’d share what I learned.

In a single node configuration by default, Cassandra will not immediately fsync() your write to the commit log from buffer cache to disk. I discovered this (thanks to Jon via the users mailing list) when I was trying to do an insert and immediately tried to shutdown Cassandra and start it again. What I found was if I did that within a second or two, Cassandra’s commit log did not have my insert. If I waited around 5-10 seconds before shutting it down it did. So if you see this behavior this is what is happening. Now, a single node configuration for production is not desirable but I think it’s worth blogging about because it’s a common configuration for someone who is just hacking around and evaluating Cassandra for the first time.

Cassandra will let you achieve guaranteed durability for the commit log if you want. You can change this by changing “CommitLogSync” from periodic to batch and setting “CommitLogSyncBatchWindowInMS” to an appropriate number. This will obviously come with performance trade offs but at least you get to pick what is right for you. Also, if you are running a multi node configuration this may not be needed, assuming you are doing replication for your writes as Cassandra won’t acknowledge the write until that replication has finished. By doing this you virtually eliminate the effects of a hard system crash and can relax the durability of commit log writes for individual nodes.

Cassandra Durability Wiki: http://wiki.apache.org/cassandra/Durability

Advertisement