Best DBI method to select 30,000 rows into an array

Question

I am new to Perl and getting to grips with it.

This is a two phased question.

Question 1: The Perl DBI has several methods to fetch data from the database tables.

I am returning one column of data from an Oracle DB into an array in a manner that each resultset is put into its own element, we are talking about 30,000 rows of data.

Here is my code

@arr_oracle_rs = @{$dbh->selectcol_arrayref($oracle_select)}; 

I was wondering if this is the fastest way to do this or should I use another DBI method?

It's hard to tell because of networks latency, current DB load, etc on the quickest and most efficient method to use to get the data and place into an array.

I am just wondering if people with the knowledge can tell me if I am using the correct method for such a task.

https://metacpan.org/pod/DBI#Database-Handle-Methods

selectrow_array()
selectrow_arrayref()
selectrow_hashref()
selectall_arrayref()
selectall_hashref()
selectcol_arrayref()

Question 2:

What is the best method to determine if the select query above ran successfully?

Answer 1

I'm assuming you are using DBD::Oracle.

Don't try and second guess what is happening under the hood. For a start, by default DBD::Oracle fetches multiple rows in one go (see https://metacpan.org/pod/DBD::Oracle#RowCacheSize).

Secondly in your example you already have an array and you copy it to another array - that is a waste of time and memory, i.e., selectcol_arrayref returns a reference to an array and you dereference it then copy it to @arr_oracle_rs. Just use the array ref returned.

Thirdly, we cannot say what is quickest since you've not told us how you are going to work with the returned array. Depending on what you are doing with the array it may actually be quicker to bind the column, repeatedly call fetch and do whatever you need per row (requires less memory and no repeated creation of a scalar) or it may be quicker to get all the rows in one go (requires more memory and lots of scalar creation).

I haven't actually looked at how selectcol_arrayref works but as it has to pick the first column from each row it /might/ be just as well to use selectall_arrayref IF you end up using selectall method.

As in all these things you'll have to benchmark your solutions yourself.

As mpapec said, RaiseError is your friend.