Riak on Erjang

I've spent some time trying to get Riak up and running on Erjang, and here's a note on current status. We're very early, but it is basically "up and humping".

• runs on a "hacked" R13B4 with crypto-2.0 copied from R14A (the new crypto uses NIFs and it's much easier to write NIFs that drivers).
• runs on {storage_backend, riak_kv_fs_backend}; i.e. performance is horroble. Trying to use the dets backend triggered some bug that I could not figure out right away, and bitcask has some NIFs that I didn't want to dive into ... so Justin suggested that I try with this one.

The changes needed in Erjang to get to this are roughly

• a new linked-in driver for Basho's erlang_js module which uses Mozilla Rhino (JVM-based JS implementation). erlang_js still thinks it's using SpiderMonkey so startup messages look a bit strange.
• a fix for erlang:hash(<<binary>>, ...); it was entirely missing. [Reminds me: if someone want to implement phash/phash2 (the portable hash functions) please let me know. What we have now is not at all portable... which is likely to be a problem for riak running on multiple nodes]
• various NIFs for crypto. Crypto is very incomplete so https is unlikely to work. [You're most welcome to add more crypto NIFs; it's quite easy!]
• a new linked-in driver for zlib to support inflate/deflate. Also just enough functionality to run Riak.
• some extra functionality for constructing bit strings. (Which, btw. means that you can now also do <<1,2,3>>. in the shell).
• a few extra options for erlang:system_info/2 and erlang:process_info/2. Some of these are fairly bogus in a context where we don't have per-process heaps, since we cannot answer anything meaningful when asked for the memory usage for a process).
• a bug in the driver interaction: I realized that we need to ensure order-of-delivery for port_command (async) mixed with port_control (sync). The async messages were enqueued and delivered out of order.
• a few of NIFs for erl_ddll (only erl_ddll:loaded_drivers/0 works for now, and NIFs need to be "linked in" from launch time). We need to get the interpreter functional before we can support on_load code to be executed before a module is compiled to JVM code).

Running it...

To start up riak, I use this command:

ej -boot ./releases/0.11.0/riak \
    -embedded \
    -config etc/app.config \
    -name riak@127.0.0.1 \
    -setcookie riak

Which is a drilled down version of bin/riak console.

With the above mentioned fixes which are now in the repo, riak boots and you can do stuff like:

krab$ curl -H "Accept: text/plain" http://127.0.0.1:8098/stats
{
    "vnode_gets": 0,
    "vnode_puts": 0,
    "read_repairs": 0,
    "vnode_gets_total": 42,
    "vnode_puts_total": 33,
    "node_gets": 0,
    "node_gets_total": 14,
    "node_get_fsm_time_mean": "undefined",
    "node_get_fsm_time_median": "undefined",
    "node_get_fsm_time_95": "undefined",
    "node_get_fsm_time_99": "undefined",
    "node_get_fsm_time_100": "undefined",
    "node_puts": 0,
    "node_puts_total": 11,
    "node_put_fsm_time_mean": "undefined",
    "node_put_fsm_time_median": "undefined",
    "node_put_fsm_time_95": "undefined",
    "node_put_fsm_time_99": "undefined",
    "node_put_fsm_time_100": "undefined",
    "read_repairs_total": 0,
    "cpu_nprocs": 143,
    "cpu_avg1": 128,
    "cpu_avg5": 120,
    "cpu_avg15": 110,
    "mem_total": 0062554177.875,
    "mem_allocated": 0060396010.25,
    "nodename": "riak@127.0.0.1",
    "connected_nodes": [

    ],
    "sys_driver_version": "1.5",
    "sys_global_heaps_size": 65470464,
    "sys_heap_type": "shared",
    "sys_logical_processors": 2,
    "sys_otp_release": "R14A",
    "sys_process_count": 282,
    "sys_smp_support": true,
    "sys_system_version": "Erjang [erts-5.7.5]",
    "sys_system_architecture": "java",
    "sys_threads_enabled": true,
    "sys_thread_pool_size": 10,
    "sys_wordsize": 4,
    "ring_members": [
        "riak@127.0.0.1"
    ],
    "ring_num_partitions": 64,
    "ring_ownership": "[{'riak@127.0.0.1',64}]",
    "ring_creation_size": 64,
    "storage_backend": "riak_kv_fs_backend",
    "pbc_connects_total": 0,
    "pbc_connects": 0,
    "pbc_active": 0,
    "riak_kv_version": "0.11.0",
    "os_mon_version": "2.2.5",
    "riak_core_version": "0.11.0",
    "bitcask_version": "1.0.1",
    "luke_version": "0.1",
    "webmachine_version": "1.7.0",
    "mochiweb_version": "1.7.0",
    "erlang_js_version": "0.3",
    "runtime_tools_version": "1.8.3",
    "crypto_version": "1.6.4",
    "sasl_version": "2.1.9",
    "stdlib_version": "1.16.5",
    "kernel_version": "2.13.5"
}

Insert some random stuff like this:

krab$ curl -v -d 'this is a test' -H "Content-Type: text/plain" \
    http://127.0.0.1:8098/riak/test
* About to connect() to 127.0.0.1 port 8098 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8098 (#0)
> POST /riak/test HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: 127.0.0.1:8098
> Accept: */*
> Content-Type: text/plain
> Content-Length: 14
> 
< HTTP/1.1 201 Created
< Vary: Accept-Encoding
< Server: MochiWeb/1.1 WebMachine/1.7 (participate in the frantic)
< Location: /riak/test/V8Va6wNpvtoe1O39WAnfvMaQejJ
< Date: Fri, 25 Jun 2010 09:55:03 GMT
< Content-Type: application/json
< Content-Length: 0
< 
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0

And retrieve it again:

curl -v http://127.0.0.1:8098/riak/test/V8Va6wNpvtoe1O39WAnfvMaQejJ
* About to connect() to 127.0.0.1 port 8098 (#0)
*   Trying 127.0.0.1... connected
* Connected to 127.0.0.1 (127.0.0.1) port 8098 (#0)
> GET /riak/test/V8Va6wNpvtoe1O39WAnfvMaQejJ HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: 127.0.0.1:8098
> Accept: */*
> 
< HTTP/1.1 200 OK
< X-Riak-Vclock: a85hYGBgzmDKBVIszFFp6hlMiYx5rAzc32cc5YMJn+ZkgAp//wEXZmtOYlCs+Y0skQUA
< Vary: Accept-Encoding
< Server: MochiWeb/1.1 WebMachine/1.7 (participate in the frantic)
< Link: ; rel="up"
< Last-Modified: Fri, 25 Jun 2010 09:55:03 GMT
< ETag: 6SFsQFoWZKg7DZ3I6vLDPK
< Date: Fri, 25 Jun 2010 09:55:19 GMT
< Content-Type: text/plain
< Content-Length: 14
< 
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
this is a test

That's basically it; I have not really done any more extensive testing so that will have to come later. Shall be interesting though to see how this performs. I am unlikely to have time to do this this side of summer break though, so you'll have to wait in suspense or go figure it out yourself.