Fit a line with LOESS in R

Question

I have a data set with some points in it and want to fit a line on it. I tried it with the loess function. Unfortunately I get very strange results. See the plot bellow. I expect a line that goes more through the points and over the whole plot. How can I achieve that?

How to reproduce it:

Download the dataset from https://www.dropbox.com/s/ud32tbptyvjsnp4/data.R?dl=1 (only two kb) and use this code:

load(url('https://www.dropbox.com/s/ud32tbptyvjsnp4/data.R?dl=1'))
lw1 = loess(y ~ x,data=data)
plot(y ~ x, data=data,pch=19,cex=0.1)
lines(data$y,lw1$fitted,col="blue",lwd=3)

Any help is greatly appreciated. Thanks!

Answer 1

You've plotted fitted values against y instead of against x. Also, you will need to order the x values before plotting a line. Try this:

lw1 <- loess(y ~ x,data=data)
plot(y ~ x, data=data,pch=19,cex=0.1)
j <- order(data$x)
lines(data$x[j],lw1$fitted[j],col="red",lwd=3)

Answer 2

Unfortunately the data are not available anymore, but an easier way how to fit a non-parametric line (Locally Weighted Scatterplot Smoothing or just a LOESS if you want) is to use following code:

scatter.smooth(y ~ x, span = 2/3, degree = 2)

Note that you can play with parameters span and degree to get arbitrary smoothness.

Answer 3

May be is to late, but you have options with ggplot (and dplyr). First if you want only plot a loess line over points, you can try:

library(ggplot2)
load(url("https://www.dropbox.com/s/ud32tbptyvjsnp4/data.R?dl=1"))
ggplot(data, aes(x, y)) + 
geom_point() +
geom_smooth(method = "loess", se = FALSE)

Other way, is by predict() function using a loess fit. For instance I used dplyr functions to add predictions to new column called "loess":

  library(dplyr)
  data %>%
  mutate(loess = predict(loess(y ~ x, data = data))) %>%
  ggplot(aes(x, y)) +
  geom_point(color = "grey50") +
  geom_line(aes(y = loess))

Update: Added line of code to load the example data provided Update2: Correction on geom_smoot() function name acoording @phi comment