Sunday, February 20, 2011

The golden key to distributed computing is ... Spades & C-CAPS Part 1.

The art of engineering the seemingly outrageously complex logistics behind our modern distributed computing techniques have been manipulated over the years now. It appears no matter how the shape forms, it still forms into the same shape. The current paths in which we've taken to adhere to the abstract image of "true distributed computing" has been one of repetitious erroneousness. As my mom always said,  "the apple doesn't seem to fall far from the tree". Over the past few years I've invested my leisure time to investigating many cloud computing paradigms, the story I've been hearing over and over, time and time again had began to uncover a few analogies constructing in my head that I've been able to apply to a new lightweight concept.

The concept surrounds the "dealer" idea behind the 1930's card game ... Spades, which I'll touch more on a bit later but let's get more familiar with what concepts we have available now. The problem with current distributed computing has forced the developer to sacrifice one principle for the other two.


  • Consistency means that each client always has the same view of the data.
  • Availability means that all clients can always read and write.
  • Partition tolerance means that the system works well across physical network partitions.
In compliance with the CAP Theorem, you are forced to select only two. Seems a bit unfortunate as all permutations of those principles lead to what I call the "Hope Diamond". You see, the Theorem starts off as a simple triangle


After this triangle shape is formed as stated above, you're forced to select two. What if you wanted all three? What if you wanted your cake and wanted to eat is also? turns out you can, but now that you have the cake ... are you going to eat it with a fork or spoon?

The "Hope Diamond" requires two more points within this dimension, let's move to the forth point. Now that you have your consistency, availability and partition tolerance you'll need to choose between the fork and spoon. Will you choose the "CPU"(the processing layer/point) to distribute the processing and leave the data centralized to avoid moving massive amounts of data. This concept will leave your solution within an Tier'd approach only scalable via the extension of the data processing layer "CPU". 

Now let's discuss the fifth point or ... the STORAGE(the data layer/point). Distributing the Storage layer gives you the ability to abstractly guarantee yourself both Availability and Partition tolerance .. Right?, well let's take a mental look at our "Hope Diamond". Our diamond is actually a pyramid without the fifth point, CPU is always the top point of this pyramid  before the pyramid becomes a diamond, our bid to distribute processing power has led us to forget and sometimes ignore the fifth point(STORAGE) and vise versa.


The only way to have true CAP is to apply the theorem to both 4th(processing) 5th(storage) layers equally with ABSOLUTELY NO compensations on either aspect, regardless to your business process' requirements.
So... , where does the card game 'Spades' come in?

.... to be continued.

1 comments:

Twitter Updates