I found out that since the more recent versions of R only produce PDFs and I'm trying to use DVI, I need to explicitly set \SweaveOpts{eps=true}. The bottom line is that PDF images just don't get along well with DVI viewers like Yap. My successful code is as follows:
%
\documentclass{article}
\usepackage{Sweave}
\usepackage{graphicx}
\SweaveOpts{prefix.string=C:/SweavePlots/bar}
\SweaveOpts{eps=true}
\DeclareGraphicsExtensions{.eps}
\title{SweavePlotWithEPS}
\author{user992432}
\begin{document}
\maketitle
\begin{figure}
\begin{center}
<<figname, fig=TRUE, include=TRUE, echo=FALSE>>=
plot(1:10)
@
\caption{I hope this Works}
\label{fig:one}
\end{center}
\end{figure}
\end{document}