It's been quiet on Erjang for a while, and I've been busy with other things. Going to JavaZone two weeks ago however, got me all psyched up again on my little Erlang on the JVM project.
I met some guys from VMWare/SpringSource and we talked about RabbitMQ, so I spent some late nights these past weeks getting things working. The prospects of this? How about a SpringSource developer wanting to use RabbitMQ? Eventually, this would be a great option as an integrated part of a pure Java development environment for say, Eclipse.
It's at the point now where I can run a good deal of the unit tests, though many still fail. You can read more about the progress below.
The major improvements are
- Mnesia is much more stable now
- TCP/IP error handling is much improved
It's great to work specifically on something like RabbitMQ because it already has a large base of existing test cases. Further, RabbitMQ is plain erlang i.e., no NIFs or linked-in drivers, so it is a perfect candidate for Erjang.
Looking forward to demo/talk about this next week at JAOO. Feel free to follow Erjang at GitHub if you want to stay in the loop on this.
Additions/Fixes...
To get to this point, I added a number of new BIFs, and fixed various bugs. You can see the glory details in the commit log here, but here is an outline of the most important fixes/additions:
- Implement
erlang:hibernatewhich is used ingen_server2and a lot of the other infrastructure modules that come with RabbitMQ. Hibernating is implemented as an exception which is caught at the top-level of the process so as to remove the call stack before going to sleep. - Fix a couple of issues with the tcp_inet driver. Connections shutdown procedures didn't work completely, and rabbit depends on the owner dying to close connections. Further, rabbit uses the tcp option
{exit_on_close, false}to allow outgoing communication to flush before the actual tcp socket is closed, and Erjang had some issues with that too. - Improve the driver select mechanism.
- Implement a few more options for the efile driver. Rabbit uses the
FILE_SETOPTdriver option to configure read-ahead strategies for file usage, and also depends on the integer (file descriptor) returned byFILE_OPEN. - In ETS, I had to implement a few more bif used specifically by Rabbit/Mnesia, such as
ets:repair_continuation,ets:update_counter/3(the 4-tuple variant). - Various fixes to the BEAM -> JVM compiler that I stumbled upon when running Rabbit. There were some cases where the type inference was slightly off, and I optimized the cases when
erlcuses{x,1023}as a scratch register.
I still have an issue with some of the functions in io_lib being non-tail recursive, causing stack overruns. The problem is that this often happens when Erjang tries to print an error message, and so it doesn't print the reason why it dies. And in some cases, the error_logger process itself dies from a stack overrun, bringing down the entire VM.
Going forward, it looks like the issues yet-to-be-fixed are either simple bugs like variants of arguments that are not supported by my BIF implementations, and (more tricky) some cleanup issues with processes.
Stay tuned! I'll talk more about these things at JAOO next week, QCon/SF and ErlangFactory Lite LA in November, YOW! Australia in December. If you want me to come and speak please let me know.
Booting RabbitMQ on Erjang
Booting rabbit right now happens with the following command line
ej -boot /var/lib/rabbitmq/mnesia/rabbit@renaissance/plugins-scratch/rabbit \
-pa ./ebin -name rabbit2@renaissance.local \
-sasl sasl_error_logger '{file,"sasl.log"}'
The bootup sequence looks like this (running the latest rabbitmq 0.9.1 build from hg) on Erjang/OTP R14.
=INFO REPORT==== 29-Sep-2010::12:03:51 ===
alarm_handler: {set,{{disk_almost_full,"/"},[]}}
=INFO REPORT==== 29-Sep-2010::12:03:51 ===
alarm_handler: {set,{{disk_almost_full,"/Volumes/StockMeter"},[]}}
=INFO REPORT==== 29-Sep-2010::12:03:51 ===
alarm_handler: {set,{{disk_almost_full,"/Volumes/Flash"},[]}}
|
=INFO REPORT==== 29-Sep-2010::12:03:51 ===
alarm_handler: {set,{system_memory_high_watermark,[]}}
+---+ +---+
| | | |
| | | |
| | | |
| +---+ +-------+
| |
| RabbitMQ +---+ |
| | | |
| v%%VSN%% +---+ |
| |
+-------------------+
AMQP 0-9-1 / 0-9 / 0-8
Copyright (C) 2007-2010 LShift Ltd., Cohesive Financial Technologies LLC., and Rabbit Technologies Ltd.
Licensed under the MPL. See http://www.rabbitmq.com/
node : rabbit2@renaissance.local
app descriptor : /Users/krab/Projects/rabbitmq-server/ebin/rabbit.app
home dir : /Users/krab
cookie hash : pIGTBl0W0HD2xiNguwnLoQ==
log : tty
sasl log : sasl.log
database dir : /Users/krab/Projects/rabbitmq-server/Mnesia.rabbit2@renaissance.local
erlang version : 5.8
starting file handle cache server ...
=INFO REPORT==== 29-Sep-2010::12:04:05 ===
Limiting to approx 10140 file handles (9124 sockets)
done
starting worker pool ...done
starting codec correctness check ...done
starting database ...found mnesia running id=<2.0.592>
Sep 29, 2010 12:04:05 PM erjang.EModuleManager$FunctionInfo$1 invoke
INFO: MISSING mnesia_sup:prep_stop/1
=INFO REPORT==== 29-Sep-2010::12:04:05 ===
application: mnesia
exited: stopped
type: permanent
=INFO REPORT==== 29-Sep-2010::12:04:06 ===
Process <2.0.2031> exited abnormally without links/monitors
exit reason was: shutdown
erjang.ErlangExit: shutdown
at erjang.m.erlang.ErlProc.exit(ErlProc.java:468)
at erjang.m.mnesia_bup.mnesia_bup.install_fallback_master__2(mnesia_bup.S:207)
at erjang.m.mnesia_bup.mnesia_bup$FN_install_fallback_master__2.go(Unknown Source)
at erjang.EProc.execute(EProc.java:498)
at kilim.Task._runExecute(Task.java:400)
at kilim.WorkerThread.run(WorkerThread.java:47)
done
-- external infrastructure ready
starting logging server ...done
starting exchange type registry ...done
starting exchange type direct ...done
starting exchange type headers ...done
starting exchange type topic ...done
starting exchange type fanout ...done
starting statistics event manager ...done
-- kernel ready
starting guid generator ...done
starting cluster delegate ...done
starting alarm handler ...done
starting memory monitor ...done
starting node monitor ...done
-- core initialized
starting empty DB check ...
=INFO REPORT==== 29-Sep-2010::12:04:07 ===
Memory limit set to 1535MB.
done
starting queue supervisor and queue recovery ...
=INFO REPORT==== 29-Sep-2010::12:04:07 ===
Added vhost <<"/">>
=INFO REPORT==== 29-Sep-2010::12:04:07 ===
Created user <<"guest">>
=INFO REPORT==== 29-Sep-2010::12:04:07 ===
Set user admin flag for user <<"guest">> to true
=INFO REPORT==== 29-Sep-2010::12:04:07 ===
msg_store_transient: using rabbit_msg_store_ets_index to provide index
=INFO REPORT==== 29-Sep-2010::12:04:07 ===
msg_store_persistent: using rabbit_msg_store_ets_index to provide index
=ERROR REPORT==== 29-Sep-2010::12:04:07 ===
msg_store_persistent: rebuilding indices from scratch
done
starting exchange recovery ...done
-- message delivery logic ready
starting error log relay ...done
starting networking ...
=INFO REPORT==== 29-Sep-2010::12:04:07 ===
started TCP Listener on 0.0.0.0:5672
done
-- network listeners available
broker running
Eshell V5.8 (abort with ^G)
(rabbit2@renaissance.local)1>
Running the tests from rabbitmq-java-client
And we can now run some unit tests. I've been poking around in the rabbit-java-client unit tests. Here are some screen shots...
The main tests for rabbitmq-java-client gets most of the way there, but currently dies with a bug in Erjang's ETS implementation when running the 500 producers/consumer test. Well well, can be fixed.
/com/rabbitmq/examples/TestMain.class com.rabbitmq.examples.TestMain : javac v50.0 on 1.6.0_20 Channel 0 fully open. Got message (4 left in q): (0) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Got message (3 left in q): (1) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Got message (2 left in q): (2) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Got message (1 left in q): (3) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Got message (0 left in q): (4) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Drained, remaining in batch = 0. com.rabbitmq.examples.TestMain$BatchedTracingConsumer@6db3f829.handleConsumeOk(amq.ctag-jM3S9gxr4aSTUqoQC71+lA==) com.rabbitmq.examples.TestMain$BatchedTracingConsumer@42698403.handleConsumeOk(amq.ctag-lL8JbFN2bq2oWxUgeDTuFQ==) Async message (0,noack): (5) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (0,ack): (6) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (1,noack): (7) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (1,ack): (8) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (2,noack): (9) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (2,ack): (10) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (3,noack): (11) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (3,ack): (12) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (4,noack): (13) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Async message (4,ack): (14) On the third tone, the time will be Wed Sep 29 12:13:33 CEST 2010 Acking batch. com.rabbitmq.examples.TestMain$BatchedTracingConsumer@6db3f829.handleCancelOk(amq.ctag-jM3S9gxr4aSTUqoQC71+lA==) com.rabbitmq.examples.TestMain$BatchedTracingConsumer@42698403.handleCancelOk(amq.ctag-lL8JbFN2bq2oWxUgeDTuFQ==) About to publish to topic queues About to drain q1 Got message (2 left in q): B Got message (1 left in q): C Got message (0 left in q): D Drained, remaining in batch = 7. About to drain q2 Got message (0 left in q): C Drained, remaining in batch = 9. About to drain q3 Got message (1 left in q): C Got message (0 left in q): D Drained, remaining in batch = 8. About to try mandatory/immediate publications Handling return with body one Returned: #method(reply-code=312,reply-text=NO_ROUTE,exchange=mandatoryTestExchange,routing-key=) - props: #contentHeader (content-type=null, content-encoding=null, headers=null, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null) - body: one Handling return with body two Returned: #method (reply-code=312,reply-text=NO_ROUTE,exchange=mandatoryTestExchange,routing-key=) - props: #contentHeader (content-type=null, content-encoding=null, headers=null, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null) - body: two Handling return with body three Returned: #method (reply-code=313,reply-text=NO_CONSUMERS,exchange=mandatoryTestExchange,routing-key=) - props: #contentHeader (content-type=null, content-encoding=null, headers=null, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null) - body: three Handling return with body four Returned: #method (reply-code=313,reply-text=NO_CONSUMERS,exchange=mandatoryTestExchange,routing-key=) - props: #contentHeader (content-type=null, content-encoding=null, headers=null, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null) - body: four Got message (0 left in q): five Drained, remaining in batch = 0. Completed basic.return testing. Got message (4 left in q): (15) On the third tone, the time will be Wed Sep 29 12:13:34 CEST 2010 Got message (3 left in q): (16) On the third tone, the time will be Wed Sep 29 12:13:34 CEST 2010 Got message (2 left in q): (17) On the third tone, the time will be Wed Sep 29 12:13:34 CEST 2010 Got message (1 left in q): (18) On the third tone, the time will be Wed Sep 29 12:13:34 CEST 2010 Got message (0 left in q): (19) On the third tone, the time will be Wed Sep 29 12:13:34 CEST 2010 Drained, remaining in batch = 0. Handling return with body mandatory Returned: #method (reply-code=312,reply-text=NO_ROUTE,exchange=,routing-key=bogus) - props: #contentHeader (content-type=null, content-encoding=null, headers=null, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null) - body: mandatory Handling return with body immediate Returned: #method (reply-code=313,reply-text=NO_CONSUMERS,exchange=,routing-key=bogus) - props: #contentHeader (content-type=null, content-encoding=null, headers=null, delivery-mode=null, priority=null, correlation-id=null, reply-to=null, expiration=null, message-id=null, timestamp=null, type=null, user-id=null, app-id=null, cluster-id=null) - body: immediate Got message (1 left in q): normal Got message (0 left in q): mandatory Drained, remaining in batch = 8. Closing. Leaving TestMain.run(). WON'T write statistics. WILL use server-side auto-acking.
