WIF is a set of .NET classes used for authentication with an STS - like ADFS. It enables the development of claims-based applications i.e. the user authenticates and receives a set of attributes from the identity repository in the form of claims inside a SAML token.
WIIF allows an application to easily integrate with the cloud via e.g. Azure Active Directory.
DotNetOpenAuth is a set of .NET classes used mainly for authorization (or consent). The user authenticates somewhere (DotNetOpenAuth supports OpenID) and is asked if they give permission to send a set of attributes from the identity repository to the application. If they agree the application is passed the attributes in a JSON token.
WIF works well in an Enterprise environment where you want SSO across the Microsoft stack. CRM Dynamics, Office 365, Azure, SharePoint all utilize the claims-enabled paradigm.
DotNetOpenAuth works well with stand-alone applications, obviously ones that require some kind of user consent.