OTP Boot Progress...

For a few days I have been spending some time on OTP boot, and there is still some way to go here :-) I am booting an off-the-shelf OTP R13B2. So all you need is erjang.jar, and the path to the pre-loaded beam files.

java -cp erjang-0.1.jar \
    -Derjpath=/sw/lib/erlang/lib/erts-5.7.3/ebin \
    erjang.OTPMain \
        -boot start_clean \
        -root /sw/lib/erlang \
        -init_debug -loader_debug

There is no special versions of the erlang beam files, which is pretty cool - me thinks.

Below I have included the output from the above command. You can compare this progress to running the command "erl -init_debug -loader_debug" on your system. As you will see, there a lots of modules Erjang has not yet loaded, but it seems to me that I have hit that sweet spot where progress is extremely fast, because there is never any doubt about what to do next.

The specific problem below has to do with that Kilim weaver was designed to run as a command-line tool, and I am adapting it to run as part of the virtual machine which is loading the woven code. Not to worry.

So: This is an excellent base for progress. The fundamental mechanics is in place in the VM, and we have the first gen_event server running - the error_logger. From this point forward, it is very simple to pick up the next problem. This is test-driven development indeed. Just someone else wrote all the tests.

If you want to help you're welcome. Just pick up the code here, and search for TODO or throw new NotImplemented().

Next Up: Erlang Term Store (ets). Right now I have hit a compiler problem in compiling ets.beam, and as soon as that is resolved I will have to provide an implementation for the ets BIFs. That will be interesting - because all of the core ets functionality is implemented in BIFs; which is on one hand an excellent opportunity to do some Java code.

Update: Have solved the immediate compiler problem, now pondering options for how to implement ets. Any good suggestions for an open-source embedded Java key-value store? experience with any of these would be appreciated Also still seriously considering H2 Database.

Enjoy!

Kresten

%%%%%%%%%%%%%%%%%%%%%%%%%%
%% output of running 
%%
%%  java -cp erjang-0.1.jar -Derjpath=/sw/lib/erlang/lib/erts-5.7.3/ebin \
%%     erjang.OTPMain -boot start_clean -root /sw/lib/erlang -init_debug -loader_debug
%%
%% Slightly edited and shortened for presentation here.

prompt% ./boot_otp.sh
{get_file,"./start_clean.boot"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/bin/start_clean.boot"}
{return,{ok,<<131,104,3,100,0,6,115,99,114,105,112,116,104,2,107,0,15,79,84,80,...,106>>}}
{progress,preloaded}
{read_file_info,"/sw/lib/erlang/lib/kernel-2.13.3/ebin"}
{return,{ok,{file_info,2346,directory,read,{{2009,11,8},{12,1,32}},{{2009,11,8},{12,1,32}}, {{2009,11,8},{12,1,32}},320,1,0,0,-42,0,0}}}
{read_file_info,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin"}
{return,{ok,{file_info,-28,directory,read,{{2009,11,8},{12,1,31}},{{2009,11,8},{12,1,31}},  {{2009,11,8},{12,1,31}},320,1,0,0,-12425,0,0}}}
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/error_handler.beam"}
{return,{ok,<<70,79,82,49,0,0,17,164,66,69,65,77,65,116,111,109,0,0,1,133,...,0>>}}
{progress,kernel_load_completed}
{read_file_info,"/sw/lib/erlang/lib/kernel-2.13.3/ebin"}
{return,{ok,{file_info,2346,directory,read,{{2009,11,8},{12,1,32}},{{2009,11,8},{12,1,32}}, {{2009,11,8},{12,1,32}},320,1,0,0,-42,0,0}}}
{read_file_info,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin"}
{return,{ok,{file_info,-28,directory,read,{{2009,11,8},{12,1,31}},{{2009,11,8},{12,1,31}},{{2009,11,8},{12,1,31}},320,1,0,0,-12425,0,0}}}
{progress,modules_loaded}   
{read_file_info,"/sw/lib/erlang/lib/kernel-2.13.3/ebin"}
{return,{ok,{file_info,2346,directory,read,{{2009,11,8},{12,1,32}},{{2009,11,8},{12,1,32}},{{2009,11,8},{12,1,32}},320,1,0,0,-42,0,0}}}
{read_file_info,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin"}
{return,{ok,{file_info,-28,directory,read,{{2009,11,8},{12,1,31}},{{2009,11,8},{12,1,31}},{{2009,11,8},{12,1,31}},320,1,0,0,-12425,0,0}}}
{start,heart}
    resolving heart:start/0
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/heart.beam"}
{return,{ok,<<70,79,82,49,0,0,31,156,66,69,65,77,65,116,111,109,0,0,2,80,...,0>>}}
{start,error_logger}
    resolving error_logger:start_link/0
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/error_logger.beam"}
{return,{ok,<<70,79,82,49,0,0,38,204,66,69,65,77,65,116,111,109,0,0,2,154,...,0>>}}
    resolving gen_event:start_link/1
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/gen_event.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/gen_event.beam"}
{return,{ok,<<70,79,82,49,0,0,98,220,66,69,65,77,65,116,111,109,0,0,5,193,...,0>>}}
    resolving gen:start/6
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/gen.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/gen.beam"}
{return,{ok,<<70,79,82,49,0,0,32,140,66,69,65,77,65,116,111,109,0,0,1,213,...,0>>}}
    resolving proc_lib:start_link/5
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/proc_lib.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/proc_lib.beam"}
{return,{ok,<<70,79,82,49,0,0,77,76,66,69,65,77,65,116,111,109,0,0,5,40,...,0>>}}
ignored options to send: [noconnect]
{start,application_controller}
    resolving application_controller:start/1
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/application_controller.beam"}
{return,{ok,<<70,79,82,49,0,0,237,168,66,69,65,77,65,116,111,109,0,0,13,119,...,0>>}}
resolving ets:new/2
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/ets.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/ets.beam"}
{return,{ok,<<70,79,82,49,0,0,162,104,66,69,65,77,65,116,111,109,0,0,10,57,...,0>>}}
[compiling ets...]
Dec 14, 2009 8:39:01 AM erjang.m.erlang.ErlBif load_module
SEVERE: cannot load module
java.lang.InternalError: erjang.m.ets.ets$FN_$2Dtab2file$2F3$2Dfun$2D0$2D__3
    at kilim.analysis.TypeDesc.commonSuperType(TypeDesc.java:242)
    at kilim.analysis.TypeDesc.mergeType(TypeDesc.java:178)
    at kilim.analysis.Value.merge(Value.java:82)
    at kilim.analysis.Frame.merge(Frame.java:88)
    ...
    at erjang.beam.Compiler.compile(Compiler.java:330)
    at erjang.ERT.load_module(ERT.java:618)
    at erjang.m.erlang.ErlBif.load_module(ErlBif.java:1009)
    at erjang.m.erlang.ErlBif.load_module(ErlBif.java:999)
    at erjang.m.erlang.ErlBif$FN_load_module__2.invoke(Unknown Source)
    at erjang.m.init.init.load_mod_code__3(Unknown Source)
    at erjang.m.init.init$FN_load_mod_code__3.go(Unknown Source)
    at erjang.m.init.init.load_mod__2$call(Unknown Source)
    at erjang.m.init.init.ensure_loaded__2(Unknown Source)
    at erjang.m.init.init.ensure_loaded__2$call(Unknown Source)
    at erjang.m.init.init.boot_loop__2(Unknown Source)
    at erjang.m.init.init$FN_boot_loop__2.go(Unknown Source)
    at erjang.EProc.execute(EProc.java:256)
    at kilim.Task._runExecute(Task.java:380)
    at kilim.WorkerThread.run(WorkerThread.java:36)

raise [{ets,new,[ac_tab,[set,public,named_table]]},{application_controller,init,2},{erlang,apply,3}]

{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/io_lib.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/io_lib.beam"}
{return,{ok,<<70,79,82,49,0,0,77,136,66,69,65,77,65,116,111,109,0,0,3,221,...,0>>}}
resolving erl_scan:reserved_word/1
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/erl_scan.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/erl_scan.beam"}
{return,{ok,<<70,79,82,49,0,0,249,28,66,69,65,77,65,116,111,109,0,0,8,172,...,0>>}}
resolving lists:flatten/1
{get_file,"/sw/lib/erlang/lib/kernel-2.13.3/ebin/lists.beam"}
{return,{error,enoent}}
{get_file,"/sw/lib/erlang/lib/stdlib-1.16.3/ebin/lists.beam"}
{return,{ok,<<70,79,82,49,0,1,42,188,66,69,65,77,65,116,111,109,0,0,7,29,...,47>>}}
{"could not start kernel pid",application_controller,"{undef,[{ets,new,[ac_tab,[set,public,named_table]]},{application_controller,init,2},{erlang,apply,2}]}"}
...

exiting PID<init:1> with: erjang.NotImplemented [{erlang,processes,0},{init,alive_processes,0},{init,get_pids,1},{init,kill_all_pids,1},{init,shutdown_pids,3},{init,clear_system,2},{init,new_kernelpid,3},{init,boot_loop,2}]
    at erjang.m.erlang.ErlProc.processes(ErlProc.java:450)
    at erjang.m.erlang.ErlProc$FN_processes__0.invoke(Unknown Source)
    at erjang.m.init.init.alive_processes__0(Unknown Source)
    at erjang.m.init.init.alive_processes__0$call(Unknown Source)
    at erjang.m.init.init.get_pids__1(Unknown Source)
    at erjang.m.init.init.get_pids__1$call(Unknown Source)
    at erjang.m.init.init.kill_all_pids__1(Unknown Source)
    at erjang.m.init.init.kill_all_pids__1$call(Unknown Source)
    at erjang.m.init.init.shutdown_pids__3(Unknown Source)
    at erjang.m.init.init.shutdown_pids__3$call(Unknown Source)
    at erjang.m.init.init.clear_system__2(Unknown Source)
    at erjang.m.init.init.clear_system__2$call(Unknown Source)
    at erjang.m.init.init.new_kernelpid__3(Unknown Source)
    at erjang.m.init.init.new_kernelpid__3$call(Unknown Source)
    at erjang.m.init.init.boot_loop__2(Unknown Source)
    at erjang.m.init.init$FN_boot_loop__2.go(Unknown Source)
    at erjang.EProc.execute(EProc.java:256)
    at kilim.Task._runExecute(Task.java:380)
    at kilim.WorkerThread.run(WorkerThread.java:36)