As Scott mentioned, you can limit the number of rows read from a text file with the nrows
to read.table
(and its variants like read.csv
).
You can use this in conjunction with the skip
argument to read later chunks in the dataset.
my_file <- "my file.csv"
chunk <- 2000
first <- read.csv(my_file, nrows = chunk)
second <- read.csv(my_file, nrows = chunk, skip = chunk)
third <- read.csv(my_file, nrows = chunk, skip = 2 * chunk)
You may also want to read the "Large memory and out-of-memory data" section of the high-performance computing task view.