Turns out the dll is not copied over to the executing folder of the test runner. Luckily you can specify the specific dll to load out of the GAC.
We added this to the app.config in the unit test project. Note the specific assembly version in the type attribute of the <add />
element
<system.data>
<DbProviderFactories>
<clear />
<add name="Oracle Data Provider" invariant="Oracle.DataAccess.Client"
description=".Net Framework Data Provider for Oracle"
type="Oracle.DataAccess.Client.OracleClientFactory, Oracle.DataAccess, Version=2.112.1.2, Culture=neutral, PublicKeyToken=89b483f429c47342" />
</DbProviderFactories>
</system.data>