This is generally done using logical indexing:
A = [12346 67 89
1245 0 765
56 0 99
19862 8 675];
mask = (A(:,2) ~= 0);
ans = A(mask, :)
The same but by deleting elements from the original array:
mask = (A(:,2) == 0);
A(mask, :) = []
A very similar question: How can you remove matrix rows in Matlab based on some criteria?