I would say that you are almost there. You only need to convert the record term to a list.
:- use_module(library(apply)).
:- use_module(library(csv)).
get_rows_data(File, Lists):-
csv_read_file(File, Rows, []),
rows_to_lists(Rows, Lists).
rows_to_lists(Rows, Lists):-
maplist(row_to_list, Rows, Lists).
row_to_list(Row, List):-
Row =.. [row|List].