of course, after hours of trying to find the answer to this, I post a StackOverflow question and find the answer immediately after. For future Googlers...
First, make sure you are using
<authentication mode="Forms">
</authentication>
Then, inside of this, make sure you are using a "forms" node with a "loginurl" attribute. Set that attribute to your login page URL ("~/Account/Login.aspx" in my case).
<form loginUrl="~/Account/Login.aspx" />
Leaving you this (at the bare-minimum)
<authentication mode="Forms">
<form loginUrl="~/Account/Login.aspx" />
</authentication>