Wednesday, February 23, 2022

Columnar Versus Row based databases

This year one of my goals was to create more big data content and learn to read more research papers an art that I had forgotten about after I finished grad school. I will try to read atleast 2 papers each month in the Data science space and blog about them. I thought I'd start with something fundamental in big data Row and columnar databases .

Title and Author of Paper

C-Store: A column-oriented DBMS. Stonebraker et al.

Quick synopsis and explanation of the important bits



As a quick recap the paper talks about traditional data bases which implement record-oriented storage attributes of a record are placed contiguosly in storage.When writing to disk, a single write pushes all fields of the record to disk.However, for querying data a read optmised system may be more suited. In comes C-store In C-Store, fast reads are accomplished by storing data organized by column instead of row.Each column value or attribute is stored as a contiguous block. With a column store architecture DBMS can only read the column values required for the query rather than reading the whole row and bringing irrelevant attributes into memory.Since all data within a column is of uniform type the data can be compressed to a more compact form

A visual understanding of Columnar versus Row Suppose I have a an employee table

Employee Location Department
Jac NSW IT
Sally WA Sales


In a row oriented db it will be stored like this

Jac NSW IT Sally WA Sales


In a column oriented db it will be stored like this

Jac Sally NSW WA IT Sales


To summarize the key differences between Columnar versus Row based DBs are

Columnar Row
Columns Stored contiguosly Rows stored contiguously
OLAP Usecase OLTP Usecase
Reads easy Writes easy
Compresses better since all columns are of same data type Does not compress as well

No comments:

Post a Comment