What is the correct core update workflow?

Question

I want to use composer to manage Drupal 8 dependencies, but I'm not sure what is the right core update workflow. At the moment I'm using drush to update core to latest beta release, but I also have some dependencies in my composer.json file, so after the update I'm using composer install to install all the contrib vendor dependencies. It appears that running composer install overrides some files in the core directory, although I've just updated core to latest version.

I've also tried to manually edit the composer.json file and replace the "drupal/core" line with the specific beta release, e.g. "drupal/core": "~8.0-beta14",, but it still overrides files in the core directory.

What is the right workflow?

Answer 1

I presume that you are using drupal-composer/drupal-project as the basis for your project. If not, take a look at that project, and compare it with yours.

Also, you said that you want to use composer to manage Drupal 8 dependencies, so I presume that you have selected your contrib modules via composer require drupal/devel rather than drush dl devel.

If you are doing all of these things, then you should use composer update to update Drupal core and all of your contrib modules. As long as you retain your composer.lock file, composer install should not change the version of any of your dependencies. You should not use drush pm-update at all. It should not matter to you whether or not files in the core directory are updated or not, as this directory is managed by Composer. You are better off not committing composer-managed directories to your repository, although you can if you wish.

Of course, you should run drush updatedb whenever composer update replaces Drupal core or any module.

To avoid getting development versions, set your minimum stability to 'beta' in your composer.json file using Composer stability flags.

If you are using drupal-composer/drupal-project to manage your site, then all of the root-level files such as README.txt, .htaccess and index.html become owned by your project. That means that you should check them in to your git repository; Composer will not update them, you must update them yourself when they change. These files should change only rarely, but drupal-composer/drupal-project has a script to update these files.

Answer 2

Yes, you can manage Drupal core with composer. There are a couple of things to be aware of though.

You will probably get timeouts due to a number of items composer has to run through, especially if you run in a local VM. If you run composer install you will likely get the composer error:

 [RuntimeException]                                    
  Could not delete core/.nfs0000000000000000000001:

Make sure you use require

{
  "require": {
   "drupal/core": "8.3.*"

Also add an extension to the timeout in the config

    "installer-paths": {
        "core": ["type:drupal-core"],
        "modules/contrib/{$name}": ["type:drupal-module"],
        "profiles/contrib/{$name}": ["type:drupal-profile"],
        "themes/contrib/{$name}": ["type:drupal-theme"],
        "drush/contrib/{$name}": ["type:drupal-drush"],
        "modules/custom/{$name}": ["type:drupal-custom-module"],
        "themes/custom/{$name}": ["type:drupal-custom-theme"]
    }
},

"config":{
            "process-timeout": 1600
       },

Also if that does not work you can run composer install from outside SSH in your VM.

This will bypass any NFS share timeouts and unpack Drupal in the right place.

Answer 3

The following is OK for patch releases 8.4.x > 8.4.y, but not OK for minor releases 8.4.x > 8.5.x. Jump to UPDATE 3 below for what I believe is "the answer" for minor release updates.

1- Backup any files that come with Drupal which you have modified, such as .htaccess, robots.txt, etc. (those 2 are the most commonly changed).

2- [I am told delete lock file is wrong, see UPDATE below] Delete the composer.lock file (in the top level folder of your site). This gets recreated in step 5.

3- Check your composer.json (in the top level folder of your site) and make sure that the "drupal:core" is in the require section and not in a replace section, for example

"require": {
"drupal/core": "^8.4"
},

not

"replace": {
"drupal/core": "^8.4"
},

If "drupal/core" is in the replace section, move it to the require section and delete the replace section. If there are other entries in the replace section, just remove the "drupal/core" not the entire replace section - but I think "drupal/core" is normally the only thing there.

Put what version you want to update to in "drupal/core", examples:

"drupal/core": "^8.5" - will update to the latest version of 8.5. "drupal/core": "8.4.6" - will update to version 8.4.6.

5- Run this (in the top level folder of your site):

composer update drupal/core --with-dependencies

6- If no errors, then do the usual, run the updates and clear cache:

drush updatedb
drush cr

Or if not using drush, go to /update.php to run updates, then to admin/config/development/performance and hit the "Clear all caches" button.

7- If you had backed up files in the first step (.htaccess, robots.txt) put them back. But check to see if Drupal made updates to those files and add those changes to yours.

DONE

If there were errors with the composer update in step 5, it is usually due to issues with versions of the stuff in the vendor folder.

This is a great post in dealing with such issues: https://www.jeffgeerling.com/blog/2018/updating-drupalcore-composer-drupal-core-doesnt-update and read Jeff's other 2 posts on Drupal and Composer to get more knowledge about that.

I was told by 2 people on Twitter that composer.lock should not be deleted (Step 2 above). The composer update drupal/core --with-dependencies command recreates the lock file anyways.

In testing this method I find it works fine for 8.4.3 > 8.4.6 (for example) but I get errors for 8.4.6 > 8.5.x. Will report back when I figure it out.

Example of errors:

Your requirements could not be resolved to an installable set of packages.
  Problem 1
    - symfony/yaml 3.4.x-dev conflicts with symfony/console[v3.2.8].
    - symfony/yaml 3.4.x-dev conflicts with symfony/console[v3.2.8].
    - symfony/yaml 3.4.x-dev conflicts with symfony/console[v3.2.8].
    - drupal/core 8.5.0 requires symfony/yaml ~3.4.5 -> satisfiable by symfony/yaml[3.4.x-dev].
    - Installation request for drupal/core 8.5.0 -> satisfiable by drupal/core[8.5.0].
    - Installation request for symfony/console (locked at v3.2.8, required as ~3.2.8) -> satisfiable by symfony/console[v3.2.8].

This post by Jeff Geerling addresses similar issues, but so far no luck for me: https://www.jeffgeerling.com/blog/2018/updating-drupalcore-composer-drupal-core-doesnt-update

So... the only thing that seems to work for me for 8.4.x > 8.5.x is the "nuclear option" that so many others seem to use, which is run composer update.

I guess that is OK so long as you are sure about the module versions in composer.json. Maybe one should lock them down to the current version. For example:

"drupal/address": "1.3"

rather than:

"drupal/address": "^1.3"

But is the the right answer?

OK the answer that seems to be everywhere is to do the "nuclear option":

A. Delete the /vendor folder.

B. Run composer update and simply update your modules along with core. Or, lock down the module versions in composer.json if you don't want to update them.

One person on Drupal Slack said "the whole philosophy of Composer is that you should always be updating packages, as frequently as possible". Packaged includes modules I think. So it makes some sense I guess.

Once I got from 8.4.6 to 8.5.0, this worked fine to get from 8.5.0 to 8.5.1 composer update drupal/core --with-dependencies just as it did for 8.4.3 to 8.4.6.

I am starting to conclude that "the answer" is that deleting the vendor folder and the composer.lock file, then using composer update is fine, and that one should simply make sure the version numbers for dependencies in the composer.json file are what you want. It's not that big a deal to manage what module versions you want to keep or allow to update in composer.json.

For example:

"drupal/admin_toolbar": "1.18", means stick with 1.18

"drupal/admin_toolbar": "^1.18", means go ahead and update but within 1.x (not 2.x)

This is backed up by a comment (General Redneck) on this post: https://www.jeffgeerling.com/blog/2018/updating-drupalcore-composer-drupal-core-doesnt-update "One of the things I've found as I work in support is that locking down the versions of modules and core is a good idea so that you CAN thermonuke the thing when you want because there are times when some of the various plugins even don't want to behave correctly."

By the way, the composer.lock file is no help with composer update because it gets blown away (as opposed to composer install where he lock file is read):

Running composer install will:

• Check if a composer.lock exists
• If not, perform a composer update to create one
• If composer.lock exists, install the specified versions from the lock file

Running composer update will:

• Check composer.json
• Determine the latest versions to install based on your version specs
• Install the latest versions
• Update composer.lock to reflect the latest versions installed

Ref: https://www.engineyard.com/blog/composer-its-all-about-the-lock-file

I see this is mentioned above: https://github.com/drupal-composer/drupal-project. I have used that and it's fine but it is not a requirement for using Composer with Drupal. It is confusing as it sort of "sounds" like it is from the name. When I first started with Drupal 8 I thought it was required, so built my first D8 site with that, thinking it was best practice.

That "version" of Drupal has he docroot in a /web folder, not in the top folder of the project. Also there is a bunch of stuff added to .gitignore compared to normal Drupal:

/drush/contrib/
/vendor/
/web/core/
/web/modules/contrib/
/web/themes/contrib/
/web/profiles/contrib/
/web/libraries/

So, this version of Drupal is really more meant for sites which are using continuous integration to do a new build of Drupal on every deploy, using composer install. If you deploy with a more normal method, you obviously have to commit all the above stuff to your git repo or it will not get deployed to your server [1], and that stuff is all needed for Drupal to run.

[1] if git is involved with your deploy - if you deploy with SFTP, ignore this.

Answer 4

Using the drupal/core package on packagist.org we can actually manage the core, the contrib modules (, themes and profiles) and the other vendors via composer.

I've setup the following files in my root directory and executed composer install

composer.json

{
  "require": {
    "composer/installers": "^1.0.20",
    "drupal/core": "8.0.*"
  },
  "extra": {
    "installer-paths": {
      "core": ["type:drupal-core"],
      "modules/contrib": ["type:drupal-module"],
      "profiles/contrib": ["type:drupal-profile"],
      "themes/contrib": ["type:drupal-theme"]
    }
  },
  "scripts": {
    "post-install-cmd": [
      "./post_install.sh"
    ]
  }
}

post_install.sh

#!/usr/bin/env bash
export RAW_DRUPAL="https://raw.githubusercontent.com/drupal/drupal/8.0.x"
curl $RAW_DRUPAL/example.gitignore > example.gitignore
curl $RAW_DRUPAL/.gitattributes > .gitattributes
curl $RAW_DRUPAL/.htaccess > .htaccess
curl $RAW_DRUPAL/.csslintrc > .csslintrc
curl $RAW_DRUPAL/.editorconfig > .editorconfig
curl $RAW_DRUPAL/.eslintrc > .eslintrc
curl $RAW_DRUPAL/.eslintignore > .eslintignore
curl $RAW_DRUPAL/index.php > index.php
curl $RAW_DRUPAL/update.php > update.php
curl $RAW_DRUPAL/web.config > web.config
curl $RAW_DRUPAL/autoload.php > autoload.php
curl $RAW_DRUPAL/robots.txt > robots.txt
mkdir -p sites/default
curl $RAW_DRUPAL/sites/example.sites.php > sites/example.sites.php
curl $RAW_DRUPAL/sites/development.services.yml > sites/development.services.yml
curl $RAW_DRUPAL/sites/example.settings.local.php > sites/example.settings.local.php
curl $RAW_DRUPAL/sites/default/default.services.yml > sites/default/default.services.yml
curl $RAW_DRUPAL/sites/default/default.settings.php > sites/default/default.settings.php

Enjoy :)

Answer 5

"drupal/core": "~8.0-beta14" means any release greater than 8.0-beta14 and less than 9! You'll want to remove the tilde to lock it to a specific release. Then make sure to update your lock file by running composer up, and on the target system use composer install.

An easy way to get started is to build the codebase using https://github.com/drupal-composer/drupal-project.

When we need to update something such as upgrading core, you run "composer up" locally. This will update the composer.lock file.

When other developers pull down, or in a deployment script, you would run "composer install", which uses the lock file.

The line in our composer.json for Drupal core is:

"drupal/core": "~8.0",

The tilde () means any release within the 8 number (but not 9).

If you want to lock it to a specific version, you shouldn't use the tilde.

"drupal/core": "8.0-beta14",

then run "composer up" locally, commit the composer.json and composer.lock file, and then run "composer install" on other installations after pulling down the codebase.