error with zoo package, data too long?

StackOverflow https://stackoverflow.com/questions/10112149

  •  30-05-2021
  •  | 
  •  

Question

I want to generate a new column with a running mean of a specified bin width and for that I used the zoo package (rollmean function). My datasheet consists of 1 million rows, and the function applies to the first 500,000, after that it is all NAs produced, any ideas what I may be doing wrong?

library("zoo")

HB<-bin/2
n<-length(dataraw$S)
data<-dataraw[HB:(n-HB),]
sax<- rollmean (dataraw$S, bin)
data2<-cbind(data,sax)

I would aappreciate any help you could give me, I've been stuck with this for far too long. Thanks in advance.

Was it helpful?

Solution

You almost surely have a NA in there which then propagates through rollmean(). A million values by itself does not create trouble as this little experiment shows:

R> library(zoo)
R> X <- zoo(rnorm(1e6), order.by=Sys.time()+seq(1,1e6)*1e-3)
R> Xrm <- rollmean(X, 63)
R> summary(cbind(X, Xrm))
     Index                             X                Xrm         
 Min.   :2012-04-11 13:57:28.9   Min.   :-5.1857   Min.   :-0.5843  
 1st Qu.:2012-04-11 14:01:38.9   1st Qu.:-0.6741   1st Qu.:-0.0866  
 Median :2012-04-11 14:05:48.9   Median :-0.0011   Median :-0.0013  
 Mean   :2012-04-11 14:05:48.9   Mean   :-0.0011   Mean   :-0.0011  
 3rd Qu.:2012-04-11 14:09:58.9   3rd Qu.: 0.6727   3rd Qu.: 0.0838  
 Max.   :2012-04-11 14:14:08.9   Max.   : 4.8914   Max.   : 0.5874  
                                                   NA's   :62       
R>

The only NAs here are due to the introduction of the initial lag at the beginning.

But when I introduce a single NA in X, all hell breaks loose:

R> X[567890] <- NA
R> summary(cbind(X, rollmean(X, 63)))
     Index                             X            rollmean(X, 63) 
 Min.   :2012-04-11 13:57:28.9   Min.   :-5.18574   Min.   :-1      
 1st Qu.:2012-04-11 14:01:38.9   1st Qu.:-0.67413   1st Qu.: 0      
 Median :2012-04-11 14:05:48.9   Median :-0.00110   Median : 0      
 Mean   :2012-04-11 14:05:48.9   Mean   :-0.00109   Mean   : 0      
 3rd Qu.:2012-04-11 14:09:58.9   3rd Qu.: 0.67268   3rd Qu.: 0      
 Max.   :2012-04-11 14:14:08.9   Max.   : 4.89137   Max.   : 1      
                                 NA's   :1          NA's   :432173  
R>

and I end up with 432k trailing NAs.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top