Your problem is that your arr variable's contents is being overwritten with each iteration: the arr[k]
overwrites whatever is in there already. The arr variable needs to be local to the block instead:
mat = []
2.times do # avoid for... use iterators instead
row = []
2.times do
row << rand.to_s
end
mat << row
end