Why do functional tests require a running website?

Question

I'm trying to set up functional tests in Drupal 9 using the BrowserTestBase class. The Type of Tests page on drupal.com and Browser test tutorial are quite clear that browser tests will build their own website and browser.

Browser tests create a complete Drupal installation and a virtual web browser and then use the virtual web browser to walk the Drupal install through a series of tests, just like you would do if you were doing it by hand.

However, as soon as I try to run a browser test using run-tests.sh I get an error stating that the --url parameter needs to be provided (this led me down a rabbit hole: Curl error in basic functional test). There is a tutorial on running browser tests, which clarifies that the --url parameter is in fact needed.

In phpunit.xml make the following changes:

• Set the SIMPLETEST_BASE_URL variable to the URL of your site.
• Set the SIMPLETEST_DB variable to point to the URL of your Drupal database.
• If you are placing phpunit.xml somewhere other than core, change the value of the phpunit tag's 'bootstrap' attribute to reflect the new location.
• For kernel and functional tests, set the BROWSERTEST_OUTPUT_DIRECTORY.

The section titled Run All PHPUnit Tests The Way The Testbot Does It sheds a bit of light on this by clarifying that when testbot runs a Browser test, it sets up a drupal site with the testing module enabled.

The first step is to make sure you have a fully working Drupal installation, with the Testing module enabled. The drupal.org testbot assumes that contributed modules will be installed inside the modules/contrib directory and Drupal's application root for unit tests is set to a value assuming this directory structure.

This seems to suggest that the browser test does not "create a complete Drupal installation" but rather requires one, but that leaves more questions about these tests.

• Can the URL be any drupal site with the testing module enabled?
• What changes does the test make to the website?
• Do I need a fresh site everytime I run the tests?
• Do I need to provide login details? Is that handled by the testing module?
• What about the database, what changes are made to it? How do I revert them?

Any kind of clarity on how this works would be appreciated, as I think it will probably answer my other question as well.

Answer 1

You can theoretically run a functional test on a production website. (I am not saying you should though.)

In my experience the testing framework will install a new Drupal site for every individual test and in the end remove these sites. This is done by creating testing tables in your existing database, for which you need to provide the credentials in SIMPLETEST_DB.

As long as the tests do not error, all of this data will be cleaned up at the end of the test and you do not need to worry about anything. If the test unexpectedly stops you might need to clean op some items manually (previously could be done with simepletest module, but I believe there are plans to remove that).

I have no idea if you can use a different site. You need to have database access to that site to install the testing tables. Modules should be available, etc. I do not think it would be reliable.