miceda

I’ve created a simple HTTP interface with MochiWeb that allows you to read and write key/Value pairs to Scalaris. The REST “like” interface is very simple:

To write, send a request to: “http://localhost:8002/scalaris/write” with the parameters key=”your_key”, value=”your_value”

To read, “http://localhost:8002/scalaris/read” with the parameters key=”your_key”

The code uses a gen_server process that makes an rpc:call to the Scalaris API.

You can check out the code here: mochiweb-scalaris

After installing Scalaris, running a few nodes, and playing with the API, I wanted to figure out how it actually stores the data. Using this diagram: supervisor.pdf I tracked down the source in the file cs_db_otp.erl ( the db was the obvious clue).

The database is a gen_server that wraps calls to the underlying storage - an Erlang gb_tree. What this means is the data is not actually saved to disk, but rather lives in memory. It appears the each chordnode has its own database.

When you store a Key - Value pair, the database actually records a structure like this in the gb_tree:

 Key,{Value,WriteLock,ReadLock,Version}

On a write, “WriteLock” is set to true and the Version is incremented. On a read, “ReadLock” is set to true. Most of this logic appears to be controlled through the transaction layer.

Other stuff

• Search? There appears to be the ability to search for keys within a given range via a “get_range”. But I haven’t yet found how to call that from the transaction_api
• No delete. As I mentioned in my earlier write up on using the API, there is no way to actually “delete” a key once it’s set. But hey, the code is open-source and since gb_tree has a delete method, it should be possible to add.

One other thing of note, if you’ve implemented any distributed Erlang you know it’s not recommended to run a cluster of Erlang nodes around the Internet using the built in code. Scalaris implements its own TCP layer (the authors mention this in their Videos) for the nodes to communicate. Check out the comm_layer in the source for some ideas if you need to write your own.

After installing Scalaris earlier today, I couldn’t help but jump into the code to see how things work. The users guide shows how to use the Java API and there are some nice examples. But as far as Erlang goes, I couldn’t find any information. Fortunately there are a few unit tests that provided the clues I needed.

Using the Transaction API

First you need to start Scalaris on a few nodes. See “bin/boot.sh” and the “cd_local” scripts. The README describes this. From within one of the erl shells you created with a “cs_local” script you can explore the API. Really pretty easy (and fast):

 %% Write a key
 > transstore.transaction_api:single_write("name", "dave").

 %% Read it back
 > {Value, Version} = transstore.transaction_api:quorum_read("name").
 > Value.
"dave"
> Version.
0

If you write the same key again, the version field will automatically increment. And you can store Erlang structures as the value if you’re looking to store more complicated data. But I couldn’t find a way to remove a key, so I’m not sure if a “delete” exists right now.

A PubSub API !

Digging around I also found a PubSub API that looks pretty interesting. However, it’s appears fixed on a specific goal: JSONRPC to a URL using the jsonrpc in yaws.

Basically you can subscribe to a Topic (Key) with a callback to a URL (Value). The URL must know how to handle the jsonrpc request because the underlying api will send the request to the page on a publish. See this Yaws JSONRPC for more info. Here’s how it appears to work:

%% Subscribe to the topic "ErlangNews". Publish events to the URL.
> pubsub.pubsub_api:subscribe("ErlangNews", "http://localhost:8000/news.yaws").

%% Get the subscibers to the topic
> pubsub.pubsub_api:get_subscribers("ErlangNews").
["http://localhost:8000/news.yaws"]

%% Publish some news...
> pubsub.pubsub_api:publish("ErlangNews", "Welcome to Erlang!").

Underneath the covers, the publish method attempts to make a jsonrpc call on the method “notify” to the URL. Even with this specific goal of JSONRPC it doesn’t look like it would be to difficult to create your own custom implementation - maybe a web based chat system might be interesting…

Rethinking the database

After all these years of using relational databases, my brain is locked in to their structured approach. Having only key value pairs to work with seems straight forward. But once I started brainstorming through a few prototypes I realized mapping all the data into Scalaris will take a little thinking.

I’ve been itching to try out Scalaris since it hit Google code. Well, I finally got the chance today. Why am I so excited about it? Two reasons: 1) I’ve been playing with my own Erlang version of Amazon SQS I built and would like to see if I can take it to the next level with Scalaris. 2) I’m sure there’s something I can learn from the code; IMO studying quality open-source code is one of the best learning resources.

So first I need to get things running. The only difficult part of the install is the rrdtool requirement. I haven’t used rrdtools before (but I may now…looks useful for some work I’m doing) so I need to install it on my Mac. I tried building from source but quickly found I was missing a bunch of dependencies. So I decided to try macports. It did the trick and made life a bit easier. So assuming you have macports installed, here’s what it took for me to install rrdtool:

Building on Mac OS X 10.5

1. sudo port install gettext
2. sudo port install pango
3. sudo port install rrdtool

This took awhile but worked! Now the easy part: just follow the README in the scalaris source. The build went without a hitch!

Once things were installed, I followed the instructions and created a scalaris.local.cfg and fired up the boot server and 2 nodes.

Here are few screen shots:

I ran into problem on the home screen when I tried to enter a key in the form of a URL. YAWS crashed but of course restarted itself. Not sure what all the other screens are telling me yet… so it’s time to dig into the code.

Update…
It’s not a URL that crashes YAWs, it simply data validation. If a field is missing when the HTML form is submitted it causes the error above. So the problem really has nothing to do with Scalaris. I’ve submitted a small patch to the Scalaris team that fixes the problem.