EcomDev_PHPUnit displaying errors after setup

Question

I have set up EcomDevs' Magento testing solution using this manual. I have found it a bit of a struggle to get everything that I need to work, so when I ran phpunit in the root of my project and it returned

FAILURES! Tests: 32, Assertions: 43, Failures: 14.

along with all the details of those failures I have been left wondering have I set it up incorrectly or is it areas that the framework is not able to test? I cant find anything online about this problem, so i'm guessing i've done something wrong.

Looking through the list of problems they all seem very similar (install and upgrade arrays are the only area that are failing, example below):

14)     EcomDev_PHPUnitTest_Test_Lib_Constraint_Config_Resource_Script::testGetVersionScriptsDiff with data set "to_1.0.5_one_invalid" ('directory_one_invalid', 'scheme', NULL, '1.0.5')
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
 'expected' => Array (
-        0 => 'install-1.0.0.php'
-        1 => 'upgrade-1.0.0-1.0.1.php'
-        2 => 'upgrade-1.0.1-1.0.5.php'
+        0 => 'install-1.0.5.php'
 )
 'actual' => Array (
-        0 => 'upgrade-1.0.0-1.0.1.php'
 )
)

/Users/myName/Sites/magentoDevTest/app/code/community/EcomDev/PHPUnitTest/Test/Lib/Constraint/Config/Resource/Script.php:82
/usr/local/Cellar/phpunit/3.7.28/libexec/phpunit-3.7.28.phar:614

Obviously I dont want to ignore failures, so how do I resolve this issue?

===EDIT===

I found the cause of my problem by reviewing this link (don't know how i missed it)

installation (step 4)

If it shows that there was no tests found, it means that extension was successfully installed. If it shows some errors, then it means, that your customizations has install scripts that relay on your current database data and you should fix them. Or use your dev database as a base for the tests, but prior first time running the suite.

After reading this I

• deactivated all custom modules (unsuccessful)
• re-copied the dev db and applies it for the test db (incase of any errors) (unsuccessful)
• tried to use the dev db for my tests (returns: "Test DB cannot be the same as the live one. You can change this option by running ecomdev-phpunit.php with magento-config action")(unsuccessful and can only find a reference to magento-config, but can't find the action in any file)
• set my site to the testdb and applied the devdb in local.xml.phpunit and the same failures returned

I am now out of ideas

Answer 1

EcomDev_PHPUnit is refactored at the moment. Sadly broke the master branch, but the dev branch is great, working and useable!

So use this: https://github.com/EcomDev/EcomDev_PHPUnit/tree/dev

Answer 2

Just going to leave this here, in case someone needs help.

If you want to use dev branch, instead of the default modman script
modman clone git://github.com/EcomDev/EcomDev_PHPUnit.git
use
modman clone git://github.com/EcomDev/EcomDev_PHPUnit.git -b dev

If you still encounter the same error Test DB cannot be the same as the live one.

You need to configure PHPUnit to ignore same DB errors by setting <allow_same_db>1</allow_same_db> in app/etc/local.xml.phpunit