As Access Traverses A Recordset Does It Slow Down As The Index Increases, And Why?

Question

I have the following scenario and was wondering if it was effecting the speed of execution

• Select statement pulls 100,000 rows and puts it into a DAO.recordset
• The recordset manipulates, we'll say, 5000 of those records at a time doing the following
  • Run some expensive string manipulation and functions (already optimized)
  • Insert results into a table
  • Repeat until recordset is empty
• Update original table with results

As the recordset gets further into the index it seems as if the program slows down.

I know that Access doesn't pull all of the records into memory at one time, so why is this going so slowly?

Answer 1

I ended up selecting a small set of data from my recordset using the following logic

Base Query
    If not end or beginning of recordset  then
        Do while Not end of recordset
        DO CODE
        If end of recordset rerun Base Query
    end loop

What this does is pull small sets of data into memory, reducing overhead when traversing your data set. This does require quite a few more hits to the DB, but reduces time significantly, especially if the query has/had been optimized.