3

I'm delving into the world of SimpleTest in Drupal. I started off by doing a fresh installation of Drupal 7 and running all the tests. I noticed there are a few tests that are failing, along with 1 test throwing an exception. Should a completely fresh install of Drupal 7 (or Drupal in general) have failing tests? I understand that in HEAD not all tests are going to pass. But, does Drupal get released along with a test suite that includes still failing tests?

I've tried searching the Drupal Core issue queue for some of the failing tests but haven't really found anything.

Edit

Based on comments so far, it seems that it is not intentional that a release of Drupal would include failing tests, even if for known issues that haven't been resolved as part of that release. Given that assumption, what is the best thing to do from there?

Improve this question
3
  • A broken head is a release blocker: what version of Drupal are you testing?
    – user7
    Commented Jan 31, 2012 at 3:30
  • Tests should never fail; they are created to test if the code is working, and if they fail, the code is not working.
    – avpaderno ♦
    Commented Jan 31, 2012 at 5:07
  • This was Drupal 7.10. I agree with the idea that tests should be passing in a release, however I also understand that often times there are known bugs that just don't get fixed in time for the release. I would think the tests included in a Drupal release should all pass, and failing tests for known issues shouldn't be included in the release. However, since I'm just getting into SimpleTest with Drupal, I wanted to get feedback as to whether my assumptions are correct. Failing tests could be indicating a problem in my environment if all tests are meant to be passing.
    – Chaulky
    Commented Jan 31, 2012 at 17:18

3 Answers 3

Reset to default
2

All tests are run on every patch that is posted on drupal.org, fully automated. Nothing is commited unless all tests are passing (It happens that something is commited that breaks the tests (for example if the tests only fail sometimes or the fail was not triggered on the patch for some other reason. However, development comes to a complete halt in such cases until the tests are working again.

But, these automated tests are run on equally configured, highly tuned linux servers (they currently run all of the core tests in ~40 minutes, I'm not going to ask how many hours it took you :)).

So, what is possible is that your system is somehow configured differently, uses different version of PHP, MySQL, the OS or something else. The tests should of course pass on every supported environment, so what I would recommend is creating a core issue, list the not-working tests (including the exact failures, e.g. with screenshots) and a detailed description of your set-up including your versions of PHP, MySQL, Apache, OS, extensions and so on.

I know that DamZ does regularly run all tests on different setups, including PostgreSQL, Windows and so on and I'm pretty sure that most except the Linux/MySQL env still have known failures. Edit: The site for this is on http://drupaltesting.org/, but it looks like it's been broken (the test environment, not the tests) for a few months now..

Improve this answer
0
0

As already stated, Core tests should not fail. In a stable version, no patch against Core is allowed if it breaks any test what so ever. Generally, this requirement is even upheld against the current -dev version, current 8. If you supply a patch that intentionally breaks a test, you are are supposed to supply a new test as well.

There are instances where this requirement can simply not be held up when it comes to large Core re-writes, and exceptions are made, but I strongly doubt this would ever be accepted in a stable version.

Most likely, this is caused by something in your environment instead. I have that issue myself. Tests (in a contrib project) that are known to work by several others, and myself of a different system, randomly fail. I have never taken the time to figure out why.

Since the number of reasons why a test could fail is near inifinite, all the way from a hacked Core to faulty hardware, the only general advice I can give is to whip up a debugger, or, if convenient, replace your server.

Improve this answer
0

I know this is three years old, but the fix for me ended up being elusive, but embarrassingly simple.

My particular dev environment is an Ubuntu 14 Server distro running on a VM on my MacBook. In order to access my sites, I modify my Mac's hosts file to point my servers's IP to my site's host name.

After several hours futile troubleshooting, I realized that I needed to make a similar edit to the hosts file on my server, pointing localhost to my site's host name. As soon as I made that update - BOOM! All green.

Hope this helps someone else!

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.