Virtual Clouds » methodology

perl one liners = magic

ofer affias — Sun, 04 Oct 2009 22:41:24 +0000

a word about the power of unix shell commands: perl one liners, i.e. one line of perl that can save you hours of error-prone hand changes. when i want to do some massive changes in my code this is what i use.

say you need replcae a record name from abc_record to xyz_record, first if you use perl recursivlly it will touch all files and revision control will makr them for commit. you don’t want that. so use grep to filter the relevant files:

grep "#abc_record\b" . -rl

next, do the replace:

perl -i -pe s/"#abc_record\b"/"xyz_record"/g `grep "#abc_record\b" src -rl`

note the use of \b to mark a word boundary.

a more complex replace is if you need to change the code from using a record to use a wrapper function. so for exmaple we have the record:

-record(abc_record, {fld}).

and we want to start use a wrapper module:

-module(rtools).
-export([fld/1]).
 
fld(R) -> R#abc_record.fld.

so we can do it instantly like that:

perl -i -pe s/"\b(\w+)#abc_record\.fld"/"rtools:fld(\1)"/g `grep "#abc_record\.fld" src -rl`

here we have switched from using A#abc_record.fld to record_tools:fld(A)

another trick is to remove complete lines (!), so for exmaple you can get rid of all the -compile(export_all) you have fullishly inserted into your code:

perl -i -nle 'print if !/-compile\(export_all\)/' `grep export_all src -rl`

one safty tip, omit the -i first to try before you actuall change files.

this was just a learn-by-example, more info can be found at:
http://sial.org/howto/perl/one-liner
http://www.unixguide.net/unix/perl_oneliners.shtml

code coverage in erlang

ofer affias — Thu, 23 Oct 2008 22:15:17 +0000

yesterday i found about code coverage under erlang on this nice blog. i wanted to integrate it with my unit testing, so i would be able to see what code of mine is not tested. this can be used to add new tests to cover it, or even remove of unnecessary code.

it took me some time to play with it and come up with a good method of doing that. eventually i have replaced the unit testing makefile line with:

check:
	@for f in $(MODULES); do \
	echo $$f; \
	erl $(FLAGS) -noshell -eval "
	cover:compile($$f, [{i,\"$(INCLUDE)\"}]),\
	$$f:test(),
	cover:analyse_to_file($$f, \"$(DOC)/$${f}_coverage.html\", [html])."\
	-s init stop; \

This actually do 3 things in a raw:

compile each module using cover:compile.
run the test, using the eunit test() function.
dump analysis into html files on doc directory

enjoy!

edit: i found out, that you compile the whole directory for covearge, so the result you get is thecombined coverage for all the specified modules and tests.

OBJECTS := $(patsubst %.erl,$(BIN)/%.$(EMULATOR),$(wildcard *.erl))
MODULES := $(patsubst %.erl,%,$(wildcard *.erl))
SKIP_FILES := $(patsubst %.erl,%,$(wildcard *_tests.erl))
MODULES := $(filter-out $(SKIP_FILES), $(MODULES))
MODULES := [$(subst $(space),$(comma),$(MODULES))]
 
check:
	@echo Testing units...
	@$(TIME) erl $(ERL_LIB_FLAGS) \
	-mnesia dir '"$(DATA_DIR)"' -mnesia debug $(MNESIA) -noshell \
	-eval "\
	cover:compile_directory(\".\", [{i,\"$(INCLUDE)\"},{d,'NODEBUG'}]), \
	T = fun(X) -> io:format(user, \"~-20.s\", [X]), X:test() end, \
	[T(X) || X <- "$(MODULES)"], \
	F = fun(X) -> cover:analyse_to_file(X, \"$(LOG)/\" ++ \
	    atom_to_list(X) ++ \"_coverage.html\", [html]) end, \
	[F(X) || X <- "$(MODULES)"]. \
	" -s init stop;

more on eunit

ofer affias — Wed, 22 Oct 2008 21:32:10 +0000

the use of eunit is very easy and intuitive. in principle every function which ends with _test of arity 0 (i.e. no input arguments) will be automaticly exported by eunit. eunit will also create a function called test() which will call all tests functions which have been defined. this mechanism leaves the programmer with only the actual test writing, and saves him/her the tedious wrappers and procedures. writing a test on a new module become something which take few seconds.

for example, the next function has its own testing function:

flip_sides(Side) ->
	case Side of
		x -> o;
		o -> x
	end.
 
flip_sides_test() ->
	?assert(flip_sides(x) == o),
	?assert(flip_sides(o) == x).

the testing function is invoked automatically by calling the test function of this module, e.g. mymodule:test(). the test function is supplied by the eunit framework, which takes every function that ends with _test() and add it to the testing list of that module.

a more advanced testing method is needed for testing the always loops that keep the state on the erlang module. few problems need to be solved in order to properly test those kind of loops:

initial state loading.
external function calls.
check of the new state, after each event.

the first problem can be solved by adding a new start function for debug use, which takes all the state variables of the always loop as parameters. this allows you to start the always loop in any state you want.

% original start of arity/0
start() -> start([], start).
 
% new function for debug of arity/2
start(State, Status) ->  register(?MODULE, spawn(fun() -> always(State, Status) end)).
 
% the state loop
always(State, Status) ->
	receive
		{From, stop} -> From ! stopped;
		{From, reset} -> always([], start);
		{From, {add, N}} -> always(module2:add(N, State), continue)
	end.

the next problem is the call for external function, shown here as call for module2:add/2, i solved this by adding another parameter to the loop’s state in order to keep a reference to the called function. i have also added it to the start function.

% original start of arity/0
start() -> start([], start, fun module2:add/2).
 
% new function for debug of arity/2
start(State, Status, Func) ->
	register(?MODULE, spawn(fun() -> always(State, Status, Func) end)).
 
% the state loop
always(State, Status, Func) ->
	receive
		{From, stop} -> From ! stopped;
		{From, reset} -> always([], start, Func);
		{From, {add, N}} -> always(Func(N, State), continue, Func)
	end.

adding a reference to the external function allows me to switch to a mocking function as desired. for the new state checking i have added a dump command to the always loop which return me the current state, so i can test it on my testing functions.

% the state loop
always(State, Status, Func) ->
	receive
		{From, stop} -> From ! stopped;
		{From, reset} -> From ! reset, always([], start, Func);
		{From, {add, N}} -> always(Func(N, State), continue, Func);
		{From, dump} -> {State, dump, Func}
	end.

last i’ll show the test function:

always_reset_test() ->
	Fun = fun callback_check_reset_mockup/2,
	start([x, o], continue, Fun),
	rpc(reset),
	?assert(rpc(dump) == {[], start, Fun}),
	stop().

callback_check_reset_mockup(N, State) -> void. % you can check match if applicable

that's it!

lots of ‘e’s…

ofer affias — Tue, 21 Oct 2008 21:22:00 +0000

i have started off a new project, mostly based on erlang, and i have decided to post few of my observations and experiences which i pick along the way. i will start with edoc and eunit, both i have added to my project with zero effort, and got remarkable results.

edoc produces automatic documentation for the project. i have just added the right code to my project makefile, and voula – instance documentation for my functions api. the code itself is:

docs:
erl -noshell -eval "edoc:application($(APPNAME), \".\", [$(DOC_OPTS)])" -s init stop

where:

APPNAME = myappVSN = 0.1
DOC_OPTS = {def, {version, \"$(VSN)\"}}, no_packages

eunit gives an easy way to add unit testing to the project, again, i have added few lines to my makefile:

MODULES = $(patsubst %.erl,%,$(wildcard *.erl))
check:
@for f in $(MODULES); do \
echo $$f; \
erl $(ERL_LIB_FLAGS) -noshell -s $$f test -s init stop; \
done

those few lines takes all the erl files on the current directory and run the test function for each module.

that’s it for the first time.