Drupal planet https://www.morpht.com/ en Announcing personalisation for Drupal 8 with Recombee https://www.morpht.com/blog/announcing-personalisation-drupal-8-recombee <p>At Morpht, we have been busy experimenting and building proof-of-concept personalisation features on Drupal 8. We started off with content recommendations as one of the cogs in a personalisation machine. <a href="https://www.recombee.com/">Recombee</a> stood out as a great AI-powered content recommendations engine and we decided to develop some contributed modules to integrate it with Drupal.</p> <p>We have implemented three modules which work together:</p> <ul> <li> <a href="https://www.drupal.org/project/search_api_recombee">Search API Recombee</a> indexes content and pushes it across to Recombee.</li> <li> <a href="https://www.drupal.org/project/recombee">Recombee</a> module tracks users and displays recommendations.</li> <li> <a href="https://www.drupal.org/project/json_template">JSON Template</a> module defines a plugin system for transformers and templates and controls how JSON can be transformed client side.</li> </ul> <p>The video below is a demonstration of these three modules and how they combine to deliver a power recommendations system capable of providing content to anonymous and logged in users alike. </p> <article class="align-center media-entity media-entity--type-remote-video media-entity--view-mode-full"><div class="field field--name-field-media-video-embed-field field--type-string field--label-hidden field__item"> <iframe src="/media/oembed?url=https%3A//www.youtube.com/watch%3Fv%3D1EQ_guSlrjI&amp;max_width=854&amp;max_height=480&amp;hash=zTGSLS-4zp-fwzisuqytwFPf1vUGrFccCu1iHWgmC4o" frameborder="0" allowtransparency width="854" height="480" class="media-oembed-content" title='"Recombee and Search API Recombee" by Murray Woodman [Drupal Meetup, 2020-08-04]'></iframe> </div> </article><p><a class="cta-link" href="https://www.morpht.com/sites/morpht/files/2020-08/Personalisation%20with%20Recombee%2C%20Part%202.pdf">Download the slides from the presentation</a><br> (PDF 785KB)</p> <h3>Let's talk</h3> <p>Find out how personalisation can help you increase audience engagement and lift user experience.</p> <p><a class="btn btn-primary" href="/node/106">Contact us</a></p> Mon, 10 Aug 2020 13:03:23 +1000 Murray Woodman https://www.morpht.com/blog/announcing-personalisation-drupal-8-recombee Create content listings on landing pages in Drupal https://www.morpht.com/blog/create-content-listings-landing-pages-drupal <article class="media-entity media-entity--type-remote-video media-entity--view-mode-full-wide"><div class="field field--name-field-media-video-embed-field field--type-string field--label-hidden field__item"> <iframe src="/media/oembed?url=https%3A//www.youtube.com/watch%3Fv%3D-Acpp-xiwlU&amp;max_width=854&amp;max_height=480&amp;hash=B89HEX-g3cV8SoINhtA0jnDTOpQN_W7OW5zerCjWTtE" frameborder="0" allowtransparency width="854" height="480" class="media-oembed-content" title="Listing content on Drupal landing pages"></iframe> </div> </article><h2>Links</h2> <ul> <li><a href="https://www.drupal.org/project/entity_class_formatter">Entity Class Formatter</a></li> <li><a href="https://www.drupal.org/project/entity_reference_display">Entity Reference Display</a></li> <li><a href="https://govcms.convivial.io/">Convivial for GovCMS</a></li> </ul> Tue, 30 Jun 2020 15:58:01 +1000 https://www.morpht.com/blog/create-content-listings-landing-pages-drupal Drupal 8 Architectural Paradigms: The Typed Data API https://www.morpht.com/blog/drupal-8-architectural-paradigms-typed-data-api <article class="media-entity media-entity--type-remote-video media-entity--view-mode-extra-small"><div class="field field--name-field-media-video-embed-field field--type-string field--label-hidden field__item"> <iframe src="/media/oembed?url=https%3A//www.youtube.com/watch%3Fv%3DTUwiQXhknq0&amp;max_width=854&amp;max_height=480&amp;hash=LVTrfCDlpQ_kYd8GwbOHSXVei-7TKKK5vNxYaQC7U1k" frameborder="0" allowtransparency width="854" height="480" class="media-oembed-content" title="Jay Friendly - Drupal 8 Architectural Paradigms"></iframe> </div> </article> Wed, 31 Jul 2019 20:22:57 +1000 Jay Friendly https://www.morpht.com/blog/drupal-8-architectural-paradigms-typed-data-api Debugging Guzzle HTTP request errors in Drupal https://www.morpht.com/blog/debugging-guzzle-http-request-errors-drupal <p>Guzzle makes HTTP requests easy. When they work, it's like magic. However, as with all coding, getting something to work requires debugging, and this is where the Drupal implementation of Guzzle has a major usability problem - any returned messages are truncated, meaning that with the default settings, error messages that can help debug an issue are not accessible to the developer. This article will show developers how they can re-structure their Guzzle queries to log the full error to the Drupal log, instead of a truncated error that does not help fix the issue.</p> <h2>Standard Methodology</h2> <p>Generally, when making a Guzzle request, it is made using a <code>try/catch</code> paradigm, so that the site does not crash in the case of an error. When not using <code>try/catch</code>, a Guzzle error will result in a WSOD, which is as bad as it gets for usability. So let's take a look at an example of how Guzzle would request a page using a standard <code>try/catch</code>:</p> <p><code>try {<br>   $client = \Drupal::httpClient();<br>   $result = $client-&gt;request('GET', 'https://www.google.com');<br> }<br> catch (\Exception $error) {<br>   $logger = \Drupal::logger('HTTP Client error');<br>   $logger-&gt;error($error-&gt;getMessage());<br> }</code></p> <p>This code will request the results of <a href="http://www.google.com">www.google.com</a>, and place them in the <code>$result</code> variable. In the case that the request failed for some reason, the system logs the result of <code>$error-&gt;getMessage()</code> to the Drupal log.</p> <p>The problem, as mentioned in the intro, is that the value returned from <code>$error-&gt;getMessage()</code> contains a truncated version of the response returned from the remote website. If the developer is lucky, the text shown will contain enough information to debug the problem, but rarely is that the case. Often the error message will look something along the lines of:</p> <p><code>Client error: `POST https://exaxmple.com/3.0/users` resulted in a `400 Bad Request` response: {"type":"http://developer.example.com/documentation/guides/error-glossary/","title":"Invalid Resource","stat (truncated...)</code></p> <p>As can be seen, the full response is not shown. The actual details of the problem, and any suggestions as to a solution are not able to be seen. What we want to happen is that the full response details are logged, so we can get some accurate information as to what happened with the request.</p> <h2>Debugging Guzzle Errors</h2> <p>In the code shown above, we used the catch statement to catch <code>\Exception</code>. Generally developers will create a class that extends <code>\Exception</code>, allowing users to catch specific errors, finally catching <code>\Exception</code> as a generic default fallback.</p> <p>When Guzzle hits an error, it throws the exception <code>GuzzleHttp\Exception\GuzzleException</code>. This allows us to catch this exception first to create our own log that contains the full response from the remote server.</p> <p>We can do this, because <code>GuzzleException</code> provides the response object from the original request, which we can use to get the actual response body the remote server sent with the error. We then log that response body to the Drupal log.</p> <p><code>use Drupal\Component\Render\FormattableMarkup;<br> use GuzzleHttp\Exception\GuzzleException;<br> try {<br>   $response = $client-&gt;request($method, $endpoint, $options);<br> }<br> // First try to catch the GuzzleException. This indicates a failed response from the remote API.<br> catch (GuzzleException $error) {<br>   // Get the original response<br>   $response = $error-&gt;getResponse();<br>   // Get the info returned from the remote server.<br>   $response_info = $response-&gt;getBody()-&gt;getContents();<br>   // Using FormattableMarkup allows for the use of &lt;pre/&gt; tags, giving a more readable log item.<br>   $message = new FormattableMarkup('API connection error. Error details are as follows:&lt;pre&gt;@response&lt;/pre&gt;', ['@response' =&gt; print_r(json_decode($response_info), TRUE)]);<br>   // Log the error<br>   watchdog_exception('Remote API Connection', $error, $message);<br> }<br> // A non-Guzzle error occurred. The type of exception is unknown, so a generic log item is created. catch (\Exception $error) {<br>   // Log the error.<br>   watchdog_exception('Remote API Connection', $error, t('An unknown error occurred while trying to connect to the remote API. This is not a Guzzle error, nor an error in the remote API, rather a generic local error ocurred. The reported error was @error', ['@error' =&gt; $error-&gt;getMessage()));<br> }</code></p> <p>With this code, we have caught the Guzzle exception, and logged the actual content of the response from the remote server to the Drupal log. If the exception thrown was any other kind of exception than <code>GuzzleException</code>, we are catching the generic <code>\Exception</code> class, and logging the given error message.</p> <p>By logging the response details, our log entry will now look something like this:</p> <p><code>Remote API connection error. Error details are as follows:</code></p> <p><code>stdClass Object (<br>   [title] =&gt; Invalid Resource<br>   [status] =&gt; 400<br>   [detail] =&gt; The resource submitted could not be validated. For field-specific details, see the 'errors' array.<br>   [errors] =&gt; Array (<br>     [0] =&gt; stdClass Object (<br>       [field] =&gt; some_field<br>       [message] =&gt; Data presented is not one of the accepted values: 'Something', 'something else', or another thing'<br>     )<br>   )<br> )</code></p> <p>* Note that this is just an example, and that each API will give its own response structure.</p> <p>This is a much more valuable debug message than the original truncated message, which left us understanding that there had been an error, but without the information required to fix it.</p> <h2>Summary</h2> <p>Drupal 8 ships with Guzzle, an excellent HTTP client for making requests to other servers. However, the standard debugging method doesn't provide a helpful log message from Guzzle. This article shows how to catch Guzzle errors, so that the full response can be logged, making debugging of connection to remote servers and APIs much easier.</p> <p>Happy Drupaling!</p> <div class="display-xs">Image credit: Marco Verch <a href="https://www.flickr.com/photos/30478819@N08/35337269544">https://www.flickr.com/photos/30478819@N08/35337269544</a> </div> Thu, 06 Jun 2019 03:09:51 +1000 Jay Friendly https://www.morpht.com/blog/debugging-guzzle-http-request-errors-drupal Drupal 8 Configuration - Module developers and the API https://www.morpht.com/blog/drupal-8-configuration-part-5-module-developers-and-api <h2>Background</h2> <p>We live in an age of Drupal complexity. In the early days of Drupal, many developers would have a single Drupal instance/environment (aka copy) that was their production site, where they would test out new modules and develop new functionality. Developing on the live website however sometimes met with disastrous consequences when things went wrong! Over time, technology on the web grew, and nowadays it's fairly standard to have a Drupal project running on multiple environments to allow site development to be run in parallel to a live website without causing disruptions. New functionality is developed first in isolated private copies of the website, put into a testing environment where it is approved by clients, and eventually merged into the live production site.</p> <p>While multiple environments allow for site development without causing disruptions on the live production website, it introduces a new problem; how to ensure consistency between site copies so that they are all working with the correct code.</p> <p>This series of articles will explore the Configuration API, how it enables functionality to be migrated between multiple environments (sites), and ways of using the Configuration API with contributed modules to effectively manage the configuration of a project. This series will consist of the following posts:</p> <ul> <li>Part 1: <a href="/blog/drupal-8-configuration-part-1-configuration-api">The Configuration API</a> </li> <li>Part 2: <a href="/blog/drupal-8-configuration-part-2-how-api-works" hreflang="en">How the API works</a> </li> <li>Part 3: <a href="/blog/drupal-8-configuration-part-3-using-api">Using the API</a> </li> <li>Part 4: <a href="/node/496">Extending the API with contributed modules</a> </li> <li>Part 5: <strong>Module developers and the API</strong> </li> </ul> <p>This article will focus specifically on how developers can manage, declare, and debug configuration in their custom modules.</p> <h2>Configuration Schema</h2> <p>Configuration schema describes the type of configuration a module introduces into the system. Schema definitions are used for things like translating configuration and its values, for typecasting configuration values into their correct data types, and for migrating configuration between systems. Having configuration in the system is not as helpful without metadata that describes what the configuration is. Configuration schemas define the configuration items.</p> <p>Any module that introduces any configuration into the system MUST define the schema for the configuration the module introduces.</p> <p>Configuration schema definitions are declared in <code>[MODULE ROOT]/config/schema/[MODULE NAME].schema.yml</code>, where <code>[MODULE NAME]</code> is the machine name of the module. Schema definitions may define one or multiple configuration objects. Let's look at the configuration schema for the <a href="https://www.drupal.org/project/restrict_ip">Restrict IP module</a> for an example. This module defines a single configuration object, <code>restrict_ip.settings</code>:</p> <p><code>restrict_ip.settings:<br>   type: config_object<br>   label: 'Restrict IP settings'<br>   mapping:<br>     enable:<br>       type: boolean<br>       label: 'Enable module'<br>     mail_address:<br>       type: string<br>       label: 'Contact mail address to show to blocked users'<br>     dblog:<br>       type: boolean<br>       label: 'Log blocked access attempts'<br>     allow_role_bypass:<br>       type: boolean<br>       label: 'Allow IP blocking to be bypassed by roles'<br>     bypass_action:<br>       type: string<br>       label: 'Action to perform for blocked users when bypassing by role is enabled'<br>     white_black_list:<br>       type: integer<br>       label: 'Whether to use a path whitelist, blacklist, or check all pages'<br>     country_white_black_list:<br>       type: integer<br>       label: 'Whether to use a whitelist, blacklist, or neither for countries'<br>     country_list:<br>       type: string<br>       label: 'A colon separated list of countries that should be white/black listed'</code></p> <p>The above schema defines the config object <em>restrict_ip.settings</em> which is of type <em>config_object</em> (defined in <a href="https://cgit.drupalcode.org/drupal/tree/core/config/schema/core.data_types.schema.yml#n108">core.data_types.schema.yml</a>).</p> <p>When this module is enabled, and the configuration is exported, the filename of the configuration will be <code>restrict_ip.settings.yml</code>. This object has the keys <code>enable</code>, <code>mail_address</code>, <code>dblog</code> etc. The schema tells what type of value is to be stored for each of these keys, as well as the label of each key. Note that this label is automatically provided to Drupal for translation.</p> <p>The values can be retrieved from the <code>restrict_ip.settings</code> object as follows:</p> <p><code>$enable_module = \Drupal::config('restrict_ip.settings')-&gt;get('enable');<br> $mail_address = \Drupal::config('restrict_ip.settings')-&gt;get('mail_address');<br> $log = \Drupal::config('restrict_ip.settings')-&gt;get('dblog');</code></p> <p>Note that modules defining custom fields, widgets, and/or formatters must define the schema for those plugins. See <a href="https://www.drupal.org/node/2381105">this page</a> to understand how the schema definitions for these various plugins should be defined.</p> <h3>Default configuration values</h3> <p>If configuration needs to have default values, the default values can be defined in <code>[MODULE ROOT]/config/install/[CONFIG KEY].yml</code> where <code>[CONFIG KEY]</code> is the configuration object name. Each item of configuration defined in the module schema requires its own YML file to set defaults. In the case of the Restrict IP module, there is only one config key, <code>restrict_ip.settings</code>, so there can only be one file to define the default configuration, <code>restrict_ip/config/install/<strong>restrict_ip.settings</strong>.yml</code>. This file will then list the keys of the configuration object, and the default values. In the case of the Restrict IP module, the default values look like this:</p> <p><code>enable: false<br> mail_address: ''<br> dblog: false<br> allow_role_bypass: false<br> bypass_action: 'provide_link_login_page'<br> white_black_list: 0<br> country_white_black_list: 0<br> country_list: ''</code><br>  </p> <p>As can be seen, each of the mapped keys of the <code>restrict_ip.settings</code> config_object in the schema definition are added to this file, with the default values provided for each key. If a key does not have a default value, it can be left out of this file. When the module is enabled, these are the values that will be imported into active configuration as defaults.</p> <h2>Debugging Configuration</h2> <p>When developing a module, it is important to ensure that the configuration schema accurately describes the configuration used in the module. Configuration can be inspected using the <a href="https://www.drupal.org/project/config_inspector">Configuration Inspector module</a>. After enabling your custom module, visit the reports page for the Configuration Inspector at /admin/reports/config-inspector, and it will list any errors in configuration.</p> <figure role="group" class="caption caption-drupal-media"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><source srcset="/sites/morpht/files/styles/full_lg/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=8bi9tjrF 1x, /sites/morpht/files/styles/full_lg_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=jKZIj9JL 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=2Z_WEDnI 1x, /sites/morpht/files/styles/full_md_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=irEnvrAH 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=ux89Klyf 1x, /sites/morpht/files/styles/full_sm_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=ZnwsZAL1 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=xv_X0VAZ 1x, /sites/morpht/files/styles/full_xs_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=YvWn56sr 2x" type="image/png"></source><img src="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20configuration%20inspector%20error%20item.png?itok=xv_X0VAZ" alt="Screenshot of an item with an error in the configuration inspector" typeof="foaf:Image"></picture> </div> </article><figcaption>The Configuration Inspector module errors in configuration schema definitions</figcaption></figure><p>Clicking on 'List' for items with errors will give more details as to the error.</p> <figure role="group" class="caption caption-drupal-media"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><source srcset="/sites/morpht/files/styles/full_lg/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=bSlJWubv 1x, /sites/morpht/files/styles/full_lg_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=E64wzKJj 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=Eyk-3-vt 1x, /sites/morpht/files/styles/full_md_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=jFZ6M5WV 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=7t6JTXsy 1x, /sites/morpht/files/styles/full_sm_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=qY30A21t 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=slKuEsA- 1x, /sites/morpht/files/styles/full_xs_hi/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=-MqdZoJf 2x" type="image/png"></source><img src="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20configuration%20inspector%20expanded%20error%20item.png?itok=slKuEsA-" alt="Screenshot of the Configuration Inspector module showing a configuration item with an error" typeof="foaf:Image"></picture> </div> </article><figcaption>The 'enable' key has an error in schema. The stored value is a boolean, but the configuration definition defines a string</figcaption></figure><p>Using the Configuration Inspector module, you can find where you have errors in your configuration schema definitions. Cleaning up these errors will correctly integrate your module with the Configuration API. In the above screenshot, then type of data in the active schema is a boolean, yet the configuration schema defines it as a string. The solution is to change the schema definition to be a boolean.</p> <h2>Summary</h2> <p>In this final article of this series on the Drupal 8 Configuration API, we looked at configuration schema, how developers can define this schema in their modules and provide defaults, as well as how to debug configuration schema errors. Hopefully this series will give you a fuller understanding of what the Configuration API is, how it can be managed, and how you can use it effectively in your Drupal projects. Happy Drupaling!</p> Thu, 16 May 2019 15:45:44 +1000 Jay Friendly https://www.morpht.com/blog/drupal-8-configuration-part-5-module-developers-and-api Drupal 8 Configuration - Extending the API https://www.morpht.com/blog/drupal-8-configuration-part-4-extending-api-contributed-modules <h2>Background</h2> <p>We live in an age of Drupal complexity. In the early days of Drupal, many developers would have a single Drupal instance/environment (aka copy) that was their production site, where they would test out new modules and develop new functionality. Developing on the live website however sometimes met with disastrous consequences when things went wrong! Over time, technology on the web grew, and nowadays it's fairly standard to have a Drupal project running on multiple environments to allow site development to be run in parallel to a live website without causing disruptions. New functionality is developed first in isolated private copies of the website, put into a testing environment where it is approved by clients, and eventually merged into the live production site.</p> <p>While multiple environments allow for site development without causing disruptions on the live production website, it introduces a new problem; how to ensure consistency between site copies so that they are all working with the correct code.</p> <p>This series of articles will explore the Configuration API, how it enables functionality to be migrated between multiple environments (sites), and ways of using the Configuration API with contributed modules to effectively manage the configuration of a project. This series will consist of the following posts:</p> <ul> <li>Part 1: <a href="/blog/drupal-8-configuration-part-1-configuration-api">The Configuration API</a> </li> <li>Part 2: <a href="/blog/drupal-8-configuration-part-2-how-api-works" hreflang="en">How the API works</a> </li> <li>Part 3: <a href="/blog/drupal-8-configuration-part-3-using-api">Using the API</a> </li> <li><strong>Part 4: Extending the API with contributed modules</strong></li> <li>Part 5: <a href="/node/491">Module developers and the API</a> </li> </ul> <p><a href="/blog/drupal-8-configuration-part-1-configuration-api">Part 1</a> gives the background of the Configuration API, as well as discusses some terminology used within this article, and <a href="/blog/drupal-8-configuration-part-2-how-api-works">Part 2</a> describes how the API works, and <a href="/blog/drupal-8-configuration-part-3-using-api">Part 3</a> explains how to use functionality provided by core, so they are worth a read before beginning this article. </p> <h2>Read-only configuration</h2> <p>In some situations, site builders may want to prevent any configuration changes from being made on the production environment, preventing changes that may cause unexpected issues. For example, clients with admin access could log into the production server, and make what they think is an innocent configuration change, that results in unexpected and drastic consequences. Some site builders consider it to be a best practice to prevent configuration changes on the production server altogether, under the idea that only content should be editable on the production server, and configuration changes should only be made in development and/or staging environments before being tested and pushed to production.</p> <p>The <a href="https://www.drupal.org/project/config_readonly">Config Readonly module</a>, allows for configuration changes through the UI to be disabled on a given environment. It does this by disabling the submit buttons on configuration pages. The module also disables configuration changes using Drush and Drupal console.</p> <figure role="group" class="caption caption-drupal-media"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><source srcset="/sites/morpht/files/styles/full_lg/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=5BNmwXO2 1x, /sites/morpht/files/styles/full_lg_hi/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=7CCuz8kt 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=FER8l3x2 1x, /sites/morpht/files/styles/full_md_hi/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=3yKDm8_0 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=LZbpcx87 1x, /sites/morpht/files/styles/full_sm_hi/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=vGtbAl9i 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=3uC9wDN_ 1x, /sites/morpht/files/styles/full_xs_hi/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=PC8ofNUU 2x" type="image/png"></source><img src="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20config%20readonly%20module.png?itok=3uC9wDN_" alt="Screenshot of the Configuration Readonly module output on configuration pages" typeof="foaf:Image"></picture> </div> </article><figcaption>A configuration form that has been disabled with the Configuration Readonly module</figcaption></figure><p>Note: some configuration forms may still be enabled when using this  module. Module developers must build their forms by extending ConfigFormBase for the Configuration Readonly module to do its magic. If the developer has built the form using other means, the form will not be disabled, and the configuration for that form can be changed through the admin UI.</p> <p>To set up an environment as read-only, add the following line to <code>settings.php</code>, then enable the module:</p> <p><code>$settings['config_readonly'] = TRUE; </code></p> <p>After an environment is set as read-only, changes to configuration can only be made on other environments, then migrated and imported into the active configuration on the read-only environment.</p> <h3 id="blacklist_configuration">Complete split (blacklist) configuration</h3> <p>Sometimes configuration needs to exist on some environments, but not exist in other environments. For example, development modules, like the <a href="https://www.drupal.org/project/devel">Devel module</a>, or UI modules like Views UI (Drupal core) and Menu UI (Drupal core) should not be enabled on production environments, as they add overhead to the server while being unnecessary since the production server should not be used for development.</p> <p>A problem arises when configuration is exported from one environment, and imported into the production environment. All the configuration from the source environment is now the active configuration on the production environment. So any development modules that were enabled on the source environment are now enabled on the production environment. In the case of development modules like Devel, this may only add some overhead to the server, but imagine a module like the <a href="https://www.drupal.org/project/shield">Shield module</a>, which sets up environments to require a username and password before even accessing the site. If this module is accidentally enabled upon import on production, it will block the site from public access - a disaster!</p> <p>The solution to this situation is to <em>blacklist</em> configuration. Blacklisted configuration is blacklisted (removed) from configuration upon export. This functionality is provided by the <a href="https://www.drupal.org/project/config_split">Configuration Split module</a>. This module allows for black-listing configuration either by module, by individual configuration key(s), and/or by wildcard.</p> <p>Note that more detailed directions for creating blacklists can be found on the <a href="https://www.drupal.org/docs/8/modules/configuration-split/creating-a-simple-split-configuration-dev-modules-only-in-dev">documentation page</a>. The following is meant to give an overview of how black lists work.</p> <p>Blacklists are created as part of a configuration profile. Configuration profiles allow for 'splitting' (a divergence in) configuration between environments. Profiles may be created for environment types such <em>development</em>, <em>staging</em> and <em>production</em> allowing for configuration specific to those types of environments. Or profiles could be set up for<em> public non-production</em> environments, that would have the Shield module enabled and configured. While a <em>development</em> profile may apply to all development environments, not all development environments are on publicly accessible URLs, and therefore may not need the Shield module enabled.</p> <p>When setting up a configuration profile, note that the folder name must be the same as the <em>machine_name</em> of the profile.</p> <figure role="group" class="caption caption-drupal-media"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><source srcset="/sites/morpht/files/styles/full_lg/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=2iG9AiW0 1x, /sites/morpht/files/styles/full_lg_hi/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=GEc0V_zU 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=d80UVDpY 1x, /sites/morpht/files/styles/full_md_hi/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=O0opSBR9 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=JgKyaUV6 1x, /sites/morpht/files/styles/full_sm_hi/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=k-_nXvTN 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=fe4xPph0 1x, /sites/morpht/files/styles/full_xs_hi/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=fX8M7ogX 2x" type="image/png"></source><img src="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20Config%20Split%20Profile.png?itok=fe4xPph0" alt="Screenshot of configuration split profile settings" typeof="foaf:Image"></picture> </div> </article><figcaption>Configuration split profile settings</figcaption></figure><p>Note that you must manually create the folder specified above, and that folder can and should be tracked using Git, so it can be use on any environment that enables the profile.</p> <p>Configuration can then be set up to be blacklisted either by module, by configuration key, or by wildcard:</p> <figure role="group" class="caption caption-drupal-media"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><source srcset="/sites/morpht/files/styles/full_lg/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=v6x9_XQS 1x, /sites/morpht/files/styles/full_lg_hi/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=EIOW0XyW 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=8mYdOe1O 1x, /sites/morpht/files/styles/full_md_hi/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=xeqg0zF5 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=5n3f2NVv 1x, /sites/morpht/files/styles/full_sm_hi/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=y2Pi6-Lp 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=5_DwCQTW 1x, /sites/morpht/files/styles/full_xs_hi/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=_PSw4q_H 2x" type="image/png"></source><img src="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20conflig%20split%20complete%20split%20settings.png?itok=5_DwCQTW" alt="Screenshot of configuration split complete-split/blacklist settings" typeof="foaf:Image"></picture> </div> </article><figcaption>Complete split (blacklist) can be set by module, configuration key, or by wildcard</figcaption></figure><p>Finally, environments need to be set up to use a given profile. This is handled by adding the following line to settings.php on the environment:</p> <p><code>$config['config_split.config_split.PROFILEMACHINENAME']['status'] = TRUE;</code></p> <p>Where <code>PROFILEMACHINENAME</code> is the machine_name from the profile you created.</p> <p>Although blacklisted configuration does not become part of the exported archive, it is not ignored altogether. When an environment has the profile enabled, upon export, blacklisted configuration is extracted, then written to the folder specified in the profile. The remaining configuration is written to the default configuration directory. When importing configuration, environments with the profile enabled will first retrieve the configuration from the default configuration directory, then apply any configuration from the folder specified in the profile. Environments not set up to use the profile ignore the configuration in the blacklisted directory altogether on both import and export.</p> <p>This means that a developer can enable the Devel module on their local environment, blacklist it, then export their configuration. The blacklisted configuration never becomes part of the default configuration, and therefore the module will not accidentally be installed on environments with the configuration profile enabled.</p> <h3>Conditional split (grey list) configuration</h3> <p>Grey lists, also provided by the <a href="https://www.drupal.org/project/config_split">Configuration Split module</a>, allow for configuration to differ by environment. With a blacklist (previous section), the configuration only exists in the active database configuration for environments that are set up to use the configuration profile containing the blacklisted configuration. With a grey list, the configuration exists in the active configuration in all environments, but the configuration profiles can be set up to allow environments to use differing values for the configuration.</p> <p><span>Imagine an integration with a remote API requiring a username, password, and endpoint URL. The production server needs integrate with the remote API's production instance, while other environments will integrate with the remote API's sandbox instance. As such, the values to be used will differ by environment:</span></p> <p><u>Production Environment:</u></p> <p>remoteapi.username: <em>ProductionUsername</em><br> remoteapi.password: <em>ProductionPassword</em><br> remoteapi.endpoint:<em> <a href="https://example.com/api">https://example.com/api</a></em></p> <p><u>Other Environments:</u></p> <p>remoteapi.username: <em>SandboxUsername</em><br> remoteapi.password: Sandbox<em>Password</em><br> remoteapi.endpoint:<em> <a href="https://sandbox.example.com/api">https://sandbox.example.com/api</a></em></p> <p><span>A grey list allows for the setup of these values by configuration profile.</span></p> <p><span>You may be remembering that </span><a href="/blog/drupal-8-configuration-part-3-using-api">Part 3</a><span> of this series of articles discussed overriding configuration in </span><code>settings.php</code><span>, and thinking that a grey list sounds like the same thing. After all, the default values for the sandbox instance of the API could be set up as the configuration values, and the production values could be overridden in </span><code>settings.php</code><span> on the production environment, with the same end-result.</span></p> <p><span>The difference is that with a grey list, the remote API values are saved to the configuration profile folder, which is tracked by Git, and therefore can be tracked and migrated between environments. </span>When grey listed configuration is exported, the grey listed configuration is written to the configuration profile folder, in the same manner as blacklisted configuration. When configuration is imported, the default values are retrieved, and the grey list values are used to override the default values, after which the configuration is imported into active configuration.</p> <p>With the configuration override method using <code>settings.php</code>, site builders need to store the various configuration values somewhere outside the project, communicating environment-specific configuration values to each other through some means, to be manually entered on the relevant environment(s). With a grey list, the configuration values are managed with Git, meaning site builders do not need to record them outside the project, nor communicate them to each other through some other means. Site builders simply need to enable the relevant configuration profile in <code>settings.php</code>, and the environment-specific values can then be imported into active configuration from the configuration profile directory. This means that the sandbox API values can be set up as the values used by default on all environments, and a production configuration profile can be enabled on the production environment using the values to connect to the production instance of the remote API.</p> <p>Conditional split items can be selected either from a list, or by manually entering them into the configuration profile:</p> <figure role="group" class="caption caption-drupal-media"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><source srcset="/sites/morpht/files/styles/full_lg/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=7X2a-8ns 1x, /sites/morpht/files/styles/full_lg_hi/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=e2HpyJkU 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=9VYkhdZc 1x, /sites/morpht/files/styles/full_md_hi/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=sHP5uLrH 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=WGSDN3l_ 1x, /sites/morpht/files/styles/full_sm_hi/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=IoLy_X9O 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=Z4o-VbDY 1x, /sites/morpht/files/styles/full_xs_hi/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=CAAQzmNV 2x" type="image/png"></source><img src="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20config%20split%20grey%20list%20settings.png?itok=Z4o-VbDY" alt="Screenshot of configuration split conditional-split/grey list settings" typeof="foaf:Image"></picture> </div> </article><figcaption>Conditional split (grey list) settings can be selected or manually entered</figcaption></figure><p>Finally, n<span>ote that grey lists can actually be used in conjunction with configuration overrides in </span><code>settings.php</code><span>. Grey lists are applied during import and export of configuration from the database. Values in </span><code>settings.php</code><span> are used at runtime, overriding any active configuration. So a developer could choose to set up their local instance of the system to connect to an entirely different instance of the remote API altogether by overriding the values in </span><code>settings.php</code><span>.</span></p> <h3>Ignoring configuration (overwrite protection)</h3> <p>Sometimes developers will want to protect certain configuration items in the database from ever being overwritten. For example imagine a site named <em>Awesome Site</em>, with a module that supplies the core of the site, named <code>awesome_core</code>. Since this module provides the core functionality of the site, it should never be disabled under any circumstances, as that would disable the core functionality of the site. In this case, the configuration for this module can be set to be 'ignored'. Any attempts to import ignored configuration from the file system to the active configuration in database will be skipped, and not imported.</p> <p>Configuration can be ignored using the <a href="https://www.drupal.org/project/config_ignore">Config Ignore module</a>. T<span>he functionality this module provides is similar to the functionality provided by the Config Readonly module discussed earlier, however the Config Readonly module covers the entire configuration of an environment, while the Config Ignore module allows for choosing configuration that should be protected. This configuration is protected by ignoring it altogether on import.</span></p> <p>Configuration can be ignored as follows:</p> <ol> <li>Enable <a href="https://www.drupal.org/project/config_ignore">Config Ignore module</a> on all environments.</li> <li>Navigate to the config ignore UI page, and set the configuration item to be ignored. In the case of preventing the awesome_core module from being disabled, the following would be added:<br><code>core.extension:module.awesome_core</code><br><figure role="group" class="caption caption-drupal-media"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><source srcset="/sites/morpht/files/styles/full_lg/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=oJTE12b7 1x, /sites/morpht/files/styles/full_lg_hi/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=3JbR3mR8 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=TJNce8BC 1x, /sites/morpht/files/styles/full_md_hi/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=ONeqwar- 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=qQR4kv45 1x, /sites/morpht/files/styles/full_sm_hi/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=s-0yoAgH 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=posxd_Xc 1x, /sites/morpht/files/styles/full_xs_hi/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=wo3H9dKT 2x" type="image/png"></source><img src="/sites/morpht/files/styles/full_xs/public/2019-03/screenshot%20-%20configuration%20ignore%20module%20admin%20page.png?itok=posxd_Xc" alt="Screenshot of the Configuration Ignore module admin page" typeof="foaf:Image"></picture> </div> </article><figcaption>Configuration to be ignore is entered one item per line. Wildcards can be used.</figcaption></figure> </li> </ol> <p>This setting will ensure that any attempts to change or remove core.extension:module.awesome_core upon configuration import will be ignored. So if the module is enabled on production, and a developer pushes configuration changes that would uninstall this module, those changes will be ignored, and the module will still be set as enabled after import.</p> <h2>Summary</h2> <p>In this article, we looked at various modules that extend the Configuration API, use cases behind these modules, and how the modules worked. We looked at the <a href="https://www.drupal.org/project/config_readonly">Config Readonly module</a>, the <a href="https://www.drupal.org/project/config_split">Configuration Split module</a>, and the <a href="https://www.drupal.org/project/config_ignore">Config Ignore module</a>, and how to use these modules to manage configuration differences between environments. In the next, final <a href="/blog/drupal-8-configuration-part-5-module-developers-and-api">fifth part</a> of this series, we will look at configuration management for module developers, and how developers can define the schema for the configuration in modules they develop.</p> Wed, 15 May 2019 13:41:44 +1000 Jay Friendly https://www.morpht.com/blog/drupal-8-configuration-part-4-extending-api-contributed-modules Drupal 8 Configuration - Part 2: How the API works https://www.morpht.com/blog/drupal-8-configuration-part-2-how-api-works <h2>Background</h2> <p>We live in an age of Drupal complexity. In the early days of Drupal, many developers would have a single Drupal instance/environment (aka copy) that was their production site, where they would test out new modules and develop new functionality. Developing on the live website however sometimes met with disastrous consequences when things went wrong! Over time, technology on the web grew, and nowadays it's fairly standard to have a Drupal project running on multiple environments to allow site development to be run in parallel to a live website without causing disruptions. New functionality is developed first in isolated private copies of the website, put into a testing environment where it is approved by clients, and eventually merged into the live production site.</p> <p>While multiple environments allow for site development without causing disruptions on the live production website, it introduces a new problem; how to ensure consistency between site copies so that they are all working with the correct code.</p> <p>This series of articles will explore the Configuration API, how it enables functionality to be migrated between multiple environments (sites), and ways of using the Configuration API with contributed modules to effectively manage the configuration of a project. This series will consist of the following posts:</p> <ul> <li>Part 1: <a href="/blog/drupal-8-configuration-part-1-configuration-api">The Configuration API</a> </li> <li><strong><a hreflang="en">Part 2: How the API works</a></strong></li> <li>Part 3: <a href="/node/486">Using the API</a> </li> <li>Part 4: <a href="/node/496">Extending the API with contributed modules</a> </li> <li>Part 5: <a href="/node/491">Module developers and the API</a> </li> </ul> <p><a href="/blog/drupal-8-configuration-part-1-configuration-api">Part 1</a> gives the background of the Configuration API, as well as discusses some terminology used within this article, so it's worth a read before beginning this article.</p> <h2>Active configuration is in the database</h2> <p>In Drupal 8, configuration used at runtime is stored in the database. The values in the database are known as <em>active configuration</em>. In Drupal 7, configuration was known as settings, and stored in the <code>{variable}</code> table. In Drupal 8, configuration is stored in the <code>{config}</code> table. The active configuration is used at runtime by Drupal when preparing responses.</p> <h2>Configuration is backed up to files</h2> <p>The Configuration API enables the ability to export the active database configuration into a series of YML files. These files can also be imported into the database. This means that a developer can create a new Field API field on their local development environment, export the configuration for the new field to files, push those files to to the production environment, then import the configuration into the production environment's active configuration in the database.</p> <p>The configuration values in the database are the live/active values, used by Drupal when responding to requests. The YMLfiles that represent configuration are not required, and are not used at run-time. In fact, in a new system the configuration files don't even exist until/unless someone exports the active configuration from the database. The configuration files are a means to be able to back up and/or migrate configuration between environments. Configuration files are never used in runtime on a site.</p> <h2>Configuration architecture</h2> <p>Let's look at the Configuration API on a more technical level, using a real-world example. The <a href="https://www.drupal.org/project/restrict_ip">Restrict IP</a> module allows users to set a list of rules that whitelist or blacklist users based on their IP address. Upon visiting the module settings page, users are presented with a checkbox that allows them to enable/disable the module functionality.</p> <p>From a data standpoint, checkboxes are booleans; they represent either a <code>true</code> or <code>false</code> value. When exporting the configuration of a site with the Restrict IP module enabled, the relevant configuration key will be saved with a value of either <code>true</code> or <code>false</code> to a .yml file. Modules are required to define the schema for any configuration the module creates. Developers can look at the configuration schema declarations to understand what file(s) will be created, and what values are accepted.</p> <p>Modules declare the schema for their configuration in the <code>[MODULE ROOT]/config/schema</code> directory. In the case of the Restrict IP module, the schema file is <code>restrict_ip/config/schema/restrict_ip.schema.yml</code>. This file contains the following declaration:</p> <p><code>restrict_ip.settings:<br>   type: config_object<br>   label: 'Restrict IP settings'<br>   mapping:<br>     enable:<br>       type: boolean<br>       label: 'Enable module'</code></p> <p>Schema declarations tell the system what the configuration looks like. In this case, the base configuration object is <code>restrict_ip.settings</code>, from the first line. When this configuration is exported to file, the file name will be <code>restrict_ip.settings.yml</code>. In that file will be a declaration of either:</p> <p><code>enable: true</code></p> <p>Or:</p> <p><code>enable: false</code></p> <p>When the file <code>restrict_ip.settings.yml</code> is imported into the active configuration in another environment's database, the value for the <code>enable</code> key will be imported as defined in the file.</p> <p>On top of this, enabled modules are listed in <code>core.extension.yml</code>, which is the configuration that tracks which modules are enabled in a given environment. When the Restrict IP module is enabled in one environment, and configuration files exported from that environment are imported into a different Drupal environment, the Restrict IP module will be enabled due to its existence in <code>core.extension.yml</code>, and the setting <code>enable</code> will have a value of either <code>true</code> or <code>false</code>, depending on what the value was exported from the original environment.</p> <p>Note that if you were to try to import the configuration without having the Restrict IP module in the codebase, an error will be thrown and the configuration import will fail with an error about the Restrict IP module not existing.</p> <h2>Summary</h2> <p>In this article, we looked at how the Drupal 8 Configuration API works on a technical level. We looked at how active configuration lives in the database, and can be exported to files which can then be imported back into the database, or migrated and imported to other Drupal environments. In <a href="/blog/drupal-8-configuration-part-3-using-api">part 3</a> of the series, Using the API, we will look at how to actually use the Configuration API, as well as some contributed modules that extend the functionality of the Configuration API, allowing for more effective management of Drupal 8 projects.</p> Mon, 13 May 2019 12:19:46 +1000 Jay Friendly https://www.morpht.com/blog/drupal-8-configuration-part-2-how-api-works Drupal 8 Configuration - Part 1: The Configuration API https://www.morpht.com/blog/drupal-8-configuration-part-1-configuration-api <h2>Background</h2> <p>We live in an age of Drupal complexity. In the early days of Drupal, many developers would have a single Drupal instance/environment (aka copy) that was their production site, where they would test out new modules and develop new functionality. Developing on the live website however sometimes met with disastrous consequences when things went wrong! Over time, technology on the web grew, and nowadays it's fairly standard to have a Drupal project running on multiple environments to allow site development to be run in parallel to a live website without causing disruptions. New functionality is developed first in isolated private copies of the website, put into a testing environment where it is approved by clients, and eventually merged into the live production site.</p> <p>While multiple environments allow for site development without causing disruptions on the live production website, it introduces a new problem; how to ensure consistency between site copies so that they are all working with the correct code.</p> <p>This series of articles will explore the Configuration API, how it enables functionality to be migrated between multiple environments (sites), and ways of using the Configuration API with contributed modules to effectively manage the configuration of a project. This series will consist of the following posts:</p> <ul> <li><strong>Part 1: The Configuration API</strong></li> <li>Part 2: <a href="https://www.morpht.com/blog/drupal-8-configuration-part-2-how-api-works">How the API works</a> </li> <li>Part 3: <a href="/node/486">Using the API</a> </li> <li>Part 4: <a href="/node/496">Extending the API with contributed modules</a> </li> <li>Part 5: <a href="/node/491">Module developers and the API</a> </li> </ul> <h2>Terminology</h2> <p>Before we get started, let's review some terminology. A Drupal <em>project</em> is the overall combination of the core codebase and all of the <em>environments</em> that are used for development of the project. An <em>environment</em> is a copy/clone of the website that is accessible at a unique domain name (URL). When most users on the web think of a website, they think of the <em>environment</em> accessed at a single URL, like google.com or facebook.com. These URLs are the entry to the live production <em>environments</em> for these websites. However, in addition to the production <em>environment</em>, large projects will have additional <em>environments</em>, where code is developed and tested before being deployed to the production <em>environment</em>. The combination of these <em>environments</em> make up the <em>project</em>. Drupal 8 is built to enable developers to migrate functionality for a project between <em>environments</em>, using the Configuration API.</p> <h2>Components of a Drupal environment</h2> <p>What makes up an environment of a Drupal project? In other words, when making a 'copy' of a Drupal site, what are the elements that need to be migrated/copied to create that copy? A Drupal environment consists of the following elements:</p> <ol> <li> <strong>Codebase</strong>: At the core of any Drupal system is codebase that makes up the 'engine' that runs Drupal. The codebase of any Drupal site consists of Drupal core, modules (both contributed and custom), and themes (again, both contributed and custom). The codebase is a series of files, and these files provide the functionality of a Drupal site.</li> <li> <strong>Configuration</strong>: Configuration is the collection of settings that define how the project will implement the functionality provided by the codebase. The codebase provides the abstract functionality to create things like content types, fields and taxonomies. Developers configure a Drupal site through the admin interface, to create the actual content types, fields, and taxonomies used in the project, that will allow end-users to do things like create pages or make comments. As developers build out the functionality of the project through the admin interface, the settings they choose are saved as the configuration for that environment. Configuration is environment-specific, and the Configuration API provides a means of migrating configuration between environments.</li> <li> <strong>Content</strong>: Content is the data on a site that is specific to the given environment. Content may be created by end-users, or by content admins. While content, like configuration, sometimes needs to be migrated between environments, content should be considered environment-specific. The Migrate API provides a means of migrating content between environments.</li> <li> <strong>Uploaded files</strong>: Most Drupal projects will have files uploaded as content. These are not part of the codebase that provides the functionality of the system, rather they are files that are to be provided to end users. Examples are images, audio/video files and PDFs.</li> <li> <strong>Third party technologies</strong>: Drupal 8 is a hub framework, built to integrate with 3rd party libraries, APIs, softwares, and other technologies. For example, a project may use a 3rd party CSS/JS framework, or swap out the default database caching backend to a Redis caching backend, which provides significant performance improvements. While these third party technologies are not part of Drupal, often the technologies will be required in each environment for the project to run.</li> </ol> <p>The above five elements together make up a Drupal environment. The environment is a fully-functioning Drupal copy (instance), with its own configuration and content. Each environment will have its own domain (URL) at which the environment can be accessed.</p> <h3>Types of environments</h3> <p>Typical projects these days will have a minimum of three environments:</p> <ol> <li> <em>Production</em> environment: The main live environment that end users will use.</li> <li> <em>Staging</em> environments: One or more environments on which site owners can test new functionality or bug fixes before they are deployed to the production environment.</li> <li> <em>Development</em> environments, where new functionality can be developed in isolation. Development environments may be publicly accessible, or may only exist on a developers computer, not accessible to the outside internet.</li> </ol> <h2>Content and configuration - both stored in the database</h2> <p>A major issue that existed with all versions of Drupal core up to and including Drupal 7, was that content and configuration were both stored in the database, and there was no way to separate them. This made it difficult to manage configuration changes, such as adding a field to a content type, between multiple environments. There was no way to export the configuration for that single field, so to ensure consistency full databases were migrated. The problem is that database migrations are an all-or-nothing venture; when migrating from one environment to another, the entire database of the source environment overwrote the entire database on the target environment, meaning all content and configuration was overridden on the target environment.</p> <p>This opens up a problem though. Imagine this scenario:</p> <ol> <li>A developer copies the production database to their local computer.</li> <li>Someone creates new content on the production environment, that becomes part of the database.</li> <li>The developer adds a new field on their local environment.</li> <li>The developer migrates their local database to the production environment, overwriting all configuration and content on the production environment.</li> </ol> <p>With the above scenario, the new field created on the developer's local environment is now on the production environment, however the content created in step 2 is overwritten and lost. As such, the above process is untenable. Databases cannot be migrated 'up' (to production) after a site has gone live. They can only be migrated 'down' (from production).</p> <p>To overcome this problem, a common way to make changes to configuration before Drupal 8 was to make configuration changes on the production environment, then copy the production environment's database down to the other environments. For example, if a developer was writing code that depended on a new field, rather than create the field on their local development environment, they would create the field on the production environment, then copy the production environment database down to their local development environment. Now the field exists on both the production environment as well as their development environment. This solution works, but is akin to killing a mosquito with a sledgehammer - it's overkill. The developer did not need the whole database, only the configuration for that single field. And any changes they had already made in their own environment, such as content for testing, is now wiped out with data from the production environment.</p> <p>While this process worked (and still does continue to work for many projects), it was not ideal. It led to issues where database migrations had to be coordinated with clients and other developers to ensure that it was safe to wipe out content or configuration someone else may still be working with.</p> <h2>The Configuration API</h2> <p>The Drupal 8 Configuration API was created to overcome the above issue, so that developers could migrate configuration - such as Field API fields, content types, and taxonomies - between environments, without having to overwrite the entire database. It decoupled configuration from content. To understand how it makes things better, l<span>et's again review the components of a Drupal site, and the storage mechanisms behind them:</span></p> <ul> <li>Codebase (files)</li> <li>Configuration (database)</li> <li>Content (database)</li> <li>Uploaded files (files)</li> </ul> <p>With the Configuration API, configuration can be exported into files, and these files can be migrated between environments and imported into the new environment. This means that we can now migrate individual items of configuration without having to overwrite the entire database. The Configuration API decouples configuration from content. In the next parts of this series, we will explore how the API work</p> <p>Note 1: the Drupal 7 <a href="https://www.drupal.org/project/features">Features module</a> essentially does the same thing, and though the methodology is different, many of the concepts in the following articles will be relevant to that module as well.</p> <p>Note 2: the Drupal 8 <a href="https://www.drupal.org/docs/8/api/migrate-api/migrate-api-overview">Migrate API</a> was developed in parallel to the Configuration API, allowing for content also to be migrated, again without overwriting the entire database.</p> <h2>Summary</h2> <p>In this article, we looked at an overview of the Configuration API to understand what it is, and why it was created. After reading this article, you should have an understanding of what a Drupal environment is, and what it consists of. You should understand that in Drupal 8, configuration is stored in the database, but can be exported to files, which can then be used to migrate content between environments. In <a href="https://www.morpht.com/blog/drupal-8-configuration-part-2-how-api-works">Part 2 - How the API works</a>, we will take a closer, more technical look at how the API works, and in <a href="/blog/drupal-8-configuration-part-3-using-api">Part 3 - Using the API</a>, we will discuss how to use the API, and some contributed modules that extend its functionality.</p> Fri, 10 May 2019 12:18:26 +1000 Jay Friendly https://www.morpht.com/blog/drupal-8-configuration-part-1-configuration-api Drupal and Composer: Composer for Drupal Developers https://www.morpht.com/blog/drupal-and-composer-part-4-composer-drupal-developers <p class="introduction">As any developer working with Drupal 8 knows, working with Composer has become an integral part of working with Drupal. This can be daunting for those without previous experience working with command line, and can still be a confusing experience for those who do.</p> <p>This is the fourth post in an explorative series of blog posts on Drupal and Composer, hopefully clearing up some of the confusion. The four blog posts on this topic will be as follows:</p> <ul> <li>Part 1: <a href="/blog/drupal-and-composer-part-1-understanding-composer">Understanding Composer</a> </li> <li>Part 2: <a href="/node/391">Managing a Drupal 8 site with Composer</a> </li> <li>Part 3: <a href="/node/396">Converting Management of an Existing Drupal 8 Site to Composer</a> </li> <li>Part 4: Composer for Drupal Developers</li> </ul> <p>If you have not yet read <a href="/node/386">part 1</a> and <a href="/node/391">part 2</a>, then before reading through this post, it is probably best to ensure you understand the concepts outlined in the summaries of those articles before moving forward.</p> <h2>Composer for Drupal Developers</h2> <p>The final section in this series addresses how Drupal developers can integrate profiles, modules and themes with Composer, to manage 3rd party libraries and/or other Drupal modules used by their code. To understand when and why a developer would wish to do this, let's have a look at a use case.</p> <h3>Use Case 1: Instagram Integration</h3> <p>Imagine a developer building a module to integrate with Instagram, pulling the most recent images from an Instagram account and displaying them on the Drupal site. Instagram provides an API for retrieving this information, but before Instagram provides this data, the Drupal site is required to authenticate itself to Instagram, using the OAuth2 protocol. After authenticating, various API calls can be made to retrieve data from the API using the the OAuth2 authentication token.</p> <p>Up to Drupal 7, the Drupal way to do this would be to write a custom OAuth 2 integration (or use the <a href="https://www.drupal.org/project/oauth2_client">OAuth2 Client module</a>), then write custom functions that made the API calls to Instagram, retrieving the data.</p> <p>Drupal 8 however, using Composer, acts like a hub, allowing for the inclusion of 3rd party libraries. This saves developers from having to re-create code within Drupal (aka <em>rebuilding the wheel</em>). For example, Instagram integration can be handled with the <a href="https://github.com/cosenary/Instagram-PHP-API">Instagram-PHP-API library</a>. This library provides OAuth2 integration specific to Instagram, as well as various PHP wrappers that interact with Instagram's API to handle the retrieval of data from the API. Using Composer, our developer can integrate this 3rd party library into Drupal, and use it in their Drupal code. This saves our developer the time and effort of creating and testing the code that integrates with Instagram, as these tasks are handled by the library maintainers. If the library is well maintained, it will be updated in line with updates to Instagram's API, saving developer time (and therefore client money) in maintenance as well. And finally, as the library is open source, it will have the benefit of having eyes on it by PHP developers from various walks, rather than just Drupal developers, making for a stronger, more secure library.</p> <h3>Use Case 2: Drupal Module Dependency</h3> <p>Imagine that it is a requirement that the images retrieved from Instagram be shown in a popup (aka modal or lightbox). This can be handled using the Drupal <a href="https://www.drupal.org/project/colorbox">Colorbox module</a>. As such, the Colorbox module will be set as a dependency in the Instagram module's <code>.info.yml</code> file. When a site builder managing their Drupal project with Composer downloads/manages our developer's Instagram module with Composer, then tries to enable the Instagram module in Drupal, they will get an error from Drupal that the Colorbox module is missing. The problem here is that Drupal is being told the Colorbox module is a dependency, but we are not managing that dependency with Composer, and therefore the code has not been downloaded and does not exist.</p> <p>At this point, it is easy to think that site builders could just add the Colorbox module to Composer with a <code>require</code> command. This would work, and after doing so, they would be able to enable the Instagram module, since the Colorbox module now exists. This opens up a problem however; imagine the site builder later decides to remove our developer's Instagram module. They disable/uninstall it in Drupal, then use <code>composer remove</code> to remove it. Everything looks good - the module has been uninstalled from Drupal, and the code has been removed from the codebase. However, there is one more step to return the system to its original state; disabling/uninstalling the now unnecessary Colorbox module from Drupal and removing the code using Composer. The necessity of this additional step opens up situations where the module will be left on the system due to forgetfulness, a lack of documentation, or changing site builders, creating site bloat due to unnecessary code existing and being executed.</p> <p>The solution to this is to also manage Drupal profile/module/theme dependencies with Composer. In our use case, our developer will list the Colorbox module as a dependency of the Instagram module not just in the Instagram module's <code>.info.yml</code> file, but also as a Composer dependency in <code>composer.json</code>. This way, when the Instagram module is added to a project using Composer, Composer will also download and manage the Colorbox module. And when the site builder removes the Instagram module, Composer will remove the Colorbox module as well (if no other libraries list it as a dependency).</p> <h2>Integrating Drupal Code with Composer</h2> <h3>Step 1: Create a composer.json file</h3> <p>Integrating with Composer requires that a <code>composer.json</code> file be created in the root of the profile/module/theme (from here on out referred to as the <em>Drupal library</em>) folder. This is explained in detail on the Drupal.org documentation page <a href="https://www.drupal.org/docs/8/creating-custom-modules/add-a-composerjson-file">Add a composer.json file</a>. Once this file has been added to the Drupal Library, it should be validated. This can be done by running <code>composer validate</code> on the command line in the same directory that the <code>composer.json</code> file lives in.</p> <p>Note that Drupal.org automatically adds a <code>composer.json</code> file to projects that do not have one. This is so that Composer is able to manage all projects on Drupal.org, not just projects to which maintainers have explicitly added a <code>composer.json</code> file.</p> <h3>Step 2: Declare your dependencies to Composer</h3> <p>All library dependencies, whether 3rd party libraries, or contributed Drupal libraries (modules etc), need to be declared as dependencies to Composer. This is done by calling <code>composer require [LIBRARY NAME]</code>, from within the folder of the Drupal library. This means that if you are adding dependencies for a <em>module</em>, you will navigate to the module folder, and add your dependencies. If you are adding dependencies for a <em>theme</em>, navigate to that folder, and add the dependencies there. The key point here is to not add your dependencies from the wrong folder, as they will be be added to the <code>composer.json</code> file of the wrong package.</p> <h4>Adding Remote Library Dependencies</h4> <p>In the use case example for this article, the <a href="https://github.com/cosenary/Instagram-PHP-API">Instagram-PHP-API library</a> was suggested for integration with Instagram. This library has the Composer key <code>cosenary/instagram</code>. To set this library as a dependency of a module, navigate to the root of the module, and run the following:</p> <p><code>composer require cosenary/instagram</code></p> <p>Running this code results in the following:</p> <ol> <li>The <em>cosenary/instagram</em> library is added as a dependency to the <code>composer.json</code> file, so that Composer knows the package is managed by Composer. The <code>composer.json</code> file will contain something like the following:<br>   <p><code>"require": {<br>   ...<br>   "cosenary/instagram": "^2.3",<br>   ...<br> } </code><br> Note that composer.json is committed to Git.</p> </li> <li> <p>The [MODULE ROOT]/vendor folder is created, and the Instagram-PHP-API library is downloaded into this folder. Composer by default creates the <code>vendor</code> folder in the same directory as the <code>composer.json</code> file. </p> <p> Note that once the Drupal library has been pushed to a repository where it can be managed by Composer, installing the Drupal library with Composer will install the Instagram-PHP-API library to the project's vendor folder. As such, it's a good idea to delete the vendor folder in the module after the new <code>composer.json</code> and <code>composer.lock</code> files have been committed to the remote repository, before requiring the module with Composer.</p> <p> As a best practice, it is a good idea to create a <code>.gitignore</code> file in the module root, and include the vendor directory to ensure that it is never accidentally committed. </p> </li> <li> <p>The <code>composer.lock</code> file is created, locking the installed version of the Instagram-PHP-API library to the Instagram module. This ensures that users of the Instagram library are working with a compatible version of the Instagram-PHP-API library. Note that this file is also committed to Git.</p> </li> <li> <p>Any dependencies of the Instagram-PHP-API library, declared in its own <code>composer.json</code> file, are downloaded.</p> </li> <li> <p>All libraries and dependencies are checked for version conflicts.</p> </li> </ol> <h4>Adding Drupal Library Dependencies</h4> <p>In the use case example for the article, the Drupal Colorbox module (aka library) was mentioned to be a dependency of the Instagram module, and therefore is to be managed with Composer. Adding Drupal modules as dependencies requires an additional step, due to their being hosted on a non-default repository. To add a Drupal library as a dependency:</p> <ol> <li>Edit the <code>composer.json</code> file and add the Drupal repository to it, so that Composer knows where to look for the Drupal libraries, including the Colorbox module: <p> <code>"repositories": [<br>   {<br>     "type": "composer",<br>     "url": "https://packages.drupal.org/8"<br>   }<br> ]</code></p> <p> Now Composer knows where to look for the Colorbox module when the Instagram module is installed.  If developers try to declare Colorbox module as a dependency before this step, they will get an error that Composer cannot find the <code>drupal/colorbox</code> library.<br>  </p> </li> <li>Add the module with composer require: <p> <code>composer require drupal/colorbox</code></p> <p> Note that this will download the Colorbox module to <code>[MODULE ROOT]/vendor/drupal/colorbox</code>. This is NOT a good thing, for while Drupal will be able to find the module in this location, it is not the standard location, and if another copy of the module ends up in other directories that Drupal scans when looking for modules this will create hard to debug issues. So either delete the Colorbox module in the vendor folder right away, or immediately move it to the location where the rest of your contributed modules are located, so it's very clear where it is at.<br>  </p> </li> </ol> <p>After this, both the <code>composer.json</code> file and the the <code>composer.lock</code> file should be committed to Git. The module now has its dependencies declared on both the Instagram-PHP-API library and the Drupal Colorbox module. When site builders install the module to their Drupal project using Composer, the Instagram library will be downloaded to the project (site) <code>vendor</code> folder, and the Drupal Colorbox module will be installed to the <code>web/modules/contrib</code> folder.</p> <h2>Developing With Composer-Managed Drupal Libraries</h2> <p>Now that the module has been integrated with Composer, Composer can be used to manage the module during development. Let's imagine that the Instagram module being developed has the key <code>drupal/insta</code>, and is on Version 8.x-1.x. Downloading the -dev version of the module can be done by appending <code>:1.x-dev</code> when requiring the module:</p> <p><code>composer require drupal/insta:1.x-dev</code></p> <p>This will download the 8.x-1.x-dev version of module to the <code>web/modules/contrib</code> folder, where it can be worked with. A <span>.git</span> folder will be included in the module root, so that changes can be committed to Git. Sounds great, but when our developer tries to push their commits, they will get an error, as the remote URL of the repository is not the repository for maintainers of the module and does not allow commits. As such, we need to change the Git repository's remote URL to the URL of the repository for maintainers, allowing for code to be pushed, and releases to be made.</p> <p>To find the correct Git repository URL, first go to the module's download page on Drupal.org. Make sure you are logged in as a maintainer of the module. If you are a maintainer and have the correct permissions, you will see a tab called <em>version control</em>. Click this tab. Next, copy the Git URL on the page. It will look something like this:</p> <p><code>git@git.drupal.org:project/</code>[MODULE NAME]<code>.git</code></p> <p>Next, navigate to your module folder, and run the following commands:</p> <p><code>git remote rm origin<br> git remote add git@git.drupal.org:project/</code>[MODULE NAME]<code>.git</code></p> <p>The first line of this code removes the incorrect remote Git URL for non-maintainers, and the second line adds the correct Git remote URL for maintainers. After this, you will be able to push changes, including tags (releases).</p> <p>You can also then easily switch between your development and release versions of the module by alternating the following:</p> <p><code>composer require drupal/insta</code></p> <p><code>composer require drupal/insta:1.x-dev</code></p> <p>Note however that when doing any commits, you will need to switch up the remote URL each time you switch to the dev version of the module.</p> <h2 class="display-xxl">Summary</h2> <p>In this final part of the series on using Composer with Drupal, we have looked at how Drupal developers can integrate their custom code with Composer, to ensure that dependencies of the module are correctly managed. We have also looked at how to develop modules that are being managed with Composer. Using these techniques will allow site builders to keep a clean codebase that manages library conflicts.</p> <p>And this concludes my four-part series on Drupal and Composer. I hope you have enjoyed it, and that it shows just how powerful Composer is, and how it can be effectively used to manage Drupal 8 sites. <strong>Happy Composing and Happy Drupaling!</strong></p> Tue, 22 Jan 2019 04:00:00 +1100 Jay Friendly https://www.morpht.com/blog/drupal-and-composer-part-4-composer-drupal-developers Drupal and Composer: Converting an existing Drupal 8 site to Composer https://www.morpht.com/blog/drupal-and-composer-part-3-converting-management-existing-drupal-8-site-composer <p class="introduction">As any developer working with Drupal 8 knows, working with Composer has become an integral part of working with Drupal. This can be daunting for those without previous experience working with command line, and can still be a confusing experience for those who do.</p> <p>This is the third post in an explorative series of blog posts on Drupal and Composer, hopefully clearing up some of the confusion. The four blog posts on this topic will be as follows:</p> <ul> <li>Part 1: <a href="/blog/drupal-and-composer-part-1-understanding-composer">Understanding Composer</a> </li> <li>Part 2: <a href="/node/391">Managing a Drupal 8 site with Composer</a> </li> <li>Part 3: Converting Management of an Existing Drupal 8 Site to Composer</li> <li>Part 4: <a href="/node/416">Composer for Drupal Developers</a> </li> </ul> <p>If you have not yet read <a href="/node/386">part 1</a> and <a href="/node/391">part 2</a>, then before reading through this post, it is probably best to ensure you understand the concepts outlined in the summaries of those articles.</p> <h2>Switching Management of your Drupal site to Composer</h2> <p>So you’ve worked your way through parts one and two of this series, and you now understand what Composer is, how it can be used to work with Drupal 8, and how to start a new Drupal 8 project using Composer. But, you started your current project without using Composer, and want to switch to managing your project using Composer. Where do you start? This article will discusses a few strategies behind converting your existing system to Drupal. Fortunately some automated tools exist for converting existing sites to Composer, and in the situation that neither of these tools work, an overview is provided on how to manually convert an existing site </p> <p>But, before moving on to any of these methods...</p> <p><strong>Take a backup!</strong> As this process will be destructive to your system, make sure you take a backup of your file system, and take a backup of your database. Then go and check to ensure that you have a full backup of the file system, and a full back up of the database.</p> <p>If you skip this step, or do not do it properly, you may end up with an entirely broken system, so don’t skip this step.</p> <h2>Method 1: Composerize (Composer plugin)</h2> <p><a href="https://github.com/grasmash/composerize-drupal">Composerize Drupal</a> is a Composer plugin that has been built to convert existing Drupal installations to use Composer. Instructions are on the download page. If this method doesn't work, you can try the Drupal Composerize module:</p> <h2>Method 2: Composerize (Drupal module)</h2> <p>The <a href="https://www.drupal.org/project/composerize">Composerize module</a> is a Drupal module that is built to convert existing Drupal installations to use Composer. At the time of writing, this module has the following disclaimer on the page:</p> <blockquote> <p>This module is still in development. It supports very basic Drupal 8 setups, but there are many necessary features it still lacks (e.g., support for patches, JavaScript libraries, distributions, etc). We're working on all that stuff, but <strong>this module is definitely not ready for prime time</strong>.</p> </blockquote> <p>If this method doesn't work, you'll likely have to manually convert your site.</p> <h2>Method 3: Manual method</h2> <p>If the above steps fail, your last option is to convert your installation to using the <a href="https://github.com/drupal-composer/drupal-project">Drupal Composer Template</a> manually. N<span>ote that pretty much every system will be different, so these instructions are an overview of the end goal, rather than a complete set of steps that will convert your system. There is a good chance you’ll run into issues along the way that are not covered here, so make sure you took that backup!</span></p> <p>Converting a system to the template requires achieving the following goals:</p> <ul> <li>Setting up the file structure of the system to match the Drupal Composer Template</li> <li>Delete old Composer files</li> <li>Add the Template file system to your Drupal system</li> <li>Set up the configuration directory and the private files directory</li> <li>Set up your profiles, modules and themes to be managed by Composer</li> </ul> <h3>Step 1: Convert your file structure to match the Drupal Composer Template file structure</h3> <p>The Drupal Composer Template is set up to manage the directory above the webroot, as well as the webroot. The webroot is located in the [PROJECT ROOT]/web folder. Therefore you will need this structure:</p> <ul> <li> <code>/</code> - Project root. Contains various scaffolding files such as composer.json and composer.lock, as well as the configuration export folder, and the webroot folder <ul> <li> <code>/web</code> - The webroot. Contains all Drupal core, profile, module and theme files.</li> </ul> </li> </ul> <p>You'll need to set up your Drupal installation to match this structure, and make sure that the server is set up so that the web root is at [PROJECT ROOT]/web.</p> <p><strong>Note</strong>: Some servers require the webroot to be in a directory with a specific name, such as <code>public_html</code>, or <code>www</code>. In this case, you can try setting up symlinks from the required directory name to the <code>/web</code> directory, so that, for example, <code>[PROJECT ROOT]/public_html</code> is a symlink pointing at <code>[PROJECT ROOT]/web</code>. Your Drupal files will then reside in the <code>/web</code> directory (satisfying the Drupal template), and also be accessible at <code>/public_html</code> (satisfying the server requirements). If this does not work, another option would be to edit the composer.json file (which is added in step 3) and change any paths that point at the <code>/web</code> directory, to point at the directory name you are actually using.</p> <h3>Step 2: Delete old Composer files</h3> <p>I'll say it again, because it needs to be said, make sure you took that backup in step 0!</p> <p>If any of the following files or folders exist in your installation, delete them. Note however that you may want to save the composer.json file for reference if you've manually added any libraries into your existing installation using Composer.</p> <ul> <li> <code>[PROJECT ROOT]/vendor</code> directory</li> <li> <code>[PROJECT ROOT]/composer.json</code> file</li> <li> <code>[PROJECT ROOT]/composer.lock</code> file</li> <li> <code>[PROJECT ROOT]/web/vendor</code> directory</li> <li> <code>[PROJECT ROOT]/web/composer.json</code> file</li> <li> <code>[PROJECT ROOT]/web/composer.lock</code> file.</li> </ul> <h3>Step 3: Add the Template file system to your Drupal system</h3> <p>Go <a href="https://github.com/drupal-composer/drupal-project/archive/8.x.zip">here</a>, click 'clone or download' and download the .zip file (note - or clone the Git repository if you prefer)</p> <ol> <li>Save/move the zip file into the project root folder (the directory above the 'web' folder you created above). You can then unpack it using the following command. Before this step, the file <code>/composer.json</code> should not exist. After this step, if you've done it correctly, this file will exist.<br><code>tar -zxvf [FILE] --strip-components=1</code> </li> <li>Run the following command from the project root folder. This command will Install the Composer dependencies as well as create the <code>/vendor</code> directory.<br><code>composer install</code> </li> <li>Run the following to ensure your database is up to date, and caches are cleared.<br><code>drush updb; drush cr;</code> </li> </ol> <h3>Step 4: Set up the configuration directory and the private files directory (Optional)</h3> <p>This next step is optional, however it will make for a more secure system. First, the following directories need to be created if they don't already exist:</p> <ul> <li><code>[PROJECT ROOT]/config/sync</code></li> <li><code>[PROJECT_ROOT]/private</code></li> </ul> <p>The first folder is where exports of the Drupal 8 configuration system will be exported to. The second folder is the private files folder. Creating both of these directories as siblings to the webroot adds security, as the files are not in a web-accessible location. The next thing to do is tell the system of the location of these files. This is done by declaring the folder paths in settings.php. You can do this by adding the following two lines to the bottom of settings.php:</p> <p><code>$config_directories['sync'] = '../config/sync';<br> $settings['file_private_path'] = '../private';</code></p> <p>After this, clear the registry (<code>drush cr;</code>). You can confirm that the configuration directory was properly set by running <code>drush cex sync</code>, and then checking that there are .yml files in the <code>[PROJECT ROOT]/config/sync</code> directory. You can confirm that the private files folder was properly set by going to Admin -&gt; Configuration -&gt; Media -&gt; File System, and confirming that the private files directory is listed as <code>../private</code>.</p> <h3>Step 5: Set up your profiles, modules and themes to be managed by Composer</h3> <p>The final step is to set up Composer to manage Drupal profiles, modules and themes to be managed by Composer. The Drupal Composer Template tracks Core by default, but needs to be informed of the rest of your code. Note that if you do not want to use the most recent version of these profiles/modules/themes, you will need to alter the commands below to set the version you want to install.</p> <p>Drupal profiles, modules and themes can be installed with the following command:</p> <p><code>composer require drupal/[PACKAGE NAME]</code></p> <p>For example, if you were using the Administration Toolbar module (admin_toolbar), you would run:</p> <p><code>composer require drupal/admin_toolbar</code></p> <p>After you have done this, ensure you are up to date with a DB update and cache clear:</p> <p><code>drush updb; drush cr;</code></p> <p>At this point, your system should be converted to the Drupal Composer Template, with contributed code being managed by Composer.</p> <h2>Summary</h2> <p>This article looks at converting exiting Drupal 8 sites to being managed by the <a href="https://github.com/drupal-composer/drupal-project">Drupal Composer Template</a>. Doing this can potentially be automated using the Composerize Composer plugin or the Composerize Drupal module. In situations where this does not work, the manual directions in this article can be used as an alternative.</p> <p>In the next and final part of this series, we'll look at how Drupal developers can integrate 3rd party libraries into their custom Drupal profiles, modules and themes.</p> Thu, 03 Jan 2019 17:11:52 +1100 Jay Friendly https://www.morpht.com/blog/drupal-and-composer-part-3-converting-management-existing-drupal-8-site-composer