Spawn many processes erlang

Question

I wanna measure the performance to my database by measuring the time taken to do something as the number of processes increase. The intention is to plot a graph of performance vs number of processes after, anyone has an idea how? i am a beginner in elrlang please helo

Answer 1

Assuming your database is mnesia, this should not be hard. one way would be to have a write function and a read function. However, note that there are several Activity access contexts with mnesia. To test write times, you should NOT use the context of transaction because it returns immediately to the calling process, even before a disc write has occured. However, for disc writes, its important that you look at the context called: sync_transaction. Here is an example:

write(Record)->
    Fun = fun(R)-> mnesia:write(R) end,
    mnesia:activity(sync_transaction,Fun,[Record],mnesia_frag).

The function above will return only when all active replicas of the mnesia table have committed the record onto the data disc file. Hence to test the speed as processes increase, you need to have a record generator,a a process spawner , the write function and finally a timing mechanism. For timing, we have a built in function called: timer:tc/1, timer:tc/2 and timer:tc/3 which returns the exact time it took to execute (completely) a given function. To cut the story short, this is how i would do this:

-module(stress_test).
-compile(export_all).

-define(LIMIT,10000).

-record(book,{
    isbn,
    title,
    price,
    version}).

%% ensure this table is {type,bag}

-record(write_time,{
            isbn,
            num_of_processes,
            write_time
        }).

%% Assuming table (book) already exists
%% Assuming mnesia running already

start()-> 
    ensure_gproc(),
    tv:start(),
    spawn_many(?LIMIT).
 
spawn_many(0)-> ok;
spawn_many(N)->
    spawn(?MODULE,process,[]),
    spawn_many(N - 1).

process()->
    gproc:reg({n, l,guid()},ignored),
    timer:apply_interval(timer:seconds(2),?MODULE,write,[]),
    receive
        <<"stop">> -> exit(normal)
    end.

total_processes()->
    proplists:get_value(size,ets:info(gproc)) div 3.

ensure_gproc()->
    case lists:keymember(gproc,1,application:which_applications()) of
        true -> ok;
        false -> application:start(gproc)
    end.

guid()-> 
    random:seed(now()),
    MD5 = erlang:md5(term_to_binary([random:uniform(152629977),{node(), now(), make_ref()}])),
    MD5List = lists:nthtail(3, binary_to_list(MD5)),
    F = fun(N) -> f("~2.16.0B", [N]) end,
    L = [F(N) || N <- MD5List],
    lists:flatten(L).

generate_record()-> 
    #book{isbn = guid(),title = guid(),price = guid()}.

write()->
    Record = generate_record(),
    Fun = fun(R)-> ok = mnesia:write(R),ok end,
    %% Here is now the actual write we measure
    {Time,ok} = timer:tc(mnesia,activity,[sync_transaction,Fun,[Record],mnesia_frag]),
    %% The we save that time, the number of processes
    %% at that instant
    NoteTime = #write_time{
            isbn = Record#book.isbn,
            num_of_processes = total_processes(),
            write_time = Time
        },
    mnesia:activity(transaction,Fun,[NoteTime],mnesia_frag).

Now there are dependencies here, especially: gproc download and build it into your erlang lib path from here Download Gproc.

To run this, just call: stress_test:start(). The table write_time will help you draw a graph of number of processes against time taken to write. As the number of processes increase from 0 to the upper limit (?LIMIT), we note the time taken to write a given record at the given instant and we also note the number of processes at that time.

---

UPDATE

f(S)-> f(S,[]).
f(S,Args) -> lists:flatten(io_lib:format(S, Args)).

That is the missing function. Apologies....
Remember to study the table write_time, using the application tv, a window is opened in which you can examine the mnesia tables. Use this table to see increasing write times/ or decreasing performance as number of processes increase from time to time.

An element i have left out is to note the actual time of the write action using time() which may be important parameter. You may add it in the table definition of the write_time table.

Answer 2

Also look at http://wiki.basho.com/Benchmarking.html

Answer 3

you might look at tsung http://tsung.erlang-projects.org/