Drupal planet https://www.morpht.com/drupal-planet/rss en Drupal and Composer: Part 4 — Composer for Drupal Developers  https://www.morpht.com/blog/drupal-and-composer-part-4-composer-drupal-developers <p class="introduction">As any developer working with Drupal 8 knows, working with Composer has become an integral part of working with Drupal. This can be daunting for those without previous experience working with command line, and can still be a confusing experience for those who do.</p> <p>This is the fourth post in an explorative series of blog posts on Drupal and Composer, hopefully clearing up some of the confusion. The four blog posts on this topic will be as follows:</p> <ul><li>Part 1: <a href="/blog/drupal-and-composer-part-1-understanding-composer">Understanding Composer</a></li> <li>Part 2: <a href="/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer">Managing a Drupal 8 site with Composer</a></li> <li>Part 3: <a href="/blog/drupal-and-composer-part-3-converting-management-existing-drupal-8-site-composer">Converting Management of an Existing Drupal 8 Site to Composer</a></li> <li>Part 4: Composer for Drupal Developers</li> </ul><p>If you have not yet read <a href="https://www.morpht.com/blog/drupal-and-composer-part-1-understanding-composer">part 1</a> and <a href="https://www.morpht.com/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer">part 2</a>, then before reading through this post, it is probably best to ensure you understand the concepts outlined in the summaries of those articles before moving forward.</p> <h2>Composer for Drupal Developers</h2> <p>The final section in this series addresses how Drupal developers can integrate profiles, modules and themes with Composer, to manage 3rd party libraries and/or other Drupal modules used by their code. To understand when and why a developer would wish to do this, let's have a look at a use case.</p> <h3>Use Case 1: Instagram Integration</h3> <p>Imagine a developer building a module to integrate with Instagram, pulling the most recent images from an Instagram account and displaying them on the Drupal site. Instagram provides an API for retrieving this information, but before Instagram provides this data, the Drupal site is required to authenticate itself to Instagram, using the OAuth2 protocol. After authenticating, various API calls can be made to retrieve data from the API using the the OAuth2 authentication token.</p> <p>Up to Drupal 7, the Drupal way to do this would be to write a custom OAuth 2 integration (or use the <a href="https://www.drupal.org/project/oauth2_client">OAuth2 Client module</a>), then write custom functions that made the API calls to Instagram, retrieving the data.</p> <p>Drupal 8 however, using Composer, acts like a hub, allowing for the inclusion of 3rd party libraries. This saves developers from having to re-create code within Drupal (aka <em>rebuilding the wheel</em>). For example, Instagram integration can be handled with the <a href="https://github.com/cosenary/Instagram-PHP-API">Instagram-PHP-API library</a>. This library provides OAuth2 integration specific to Instagram, as well as various PHP wrappers that interact with Instagram's API to handle the retrieval of data from the API. Using Composer, our developer can integrate this 3rd party library into Drupal, and use it in their Drupal code. This saves our developer the time and effort of creating and testing the code that integrates with Instagram, as these tasks are handled by the library maintainers. If the library is well maintained, it will be updated in line with updates to Instagram's API, saving developer time (and therefore client money) in maintenance as well. And finally, as the library is open source, it will have the benefit of having eyes on it by PHP developers from various walks, rather than just Drupal developers, making for a stronger, more secure library.</p> <h3>Use Case 2: Drupal Module Dependency</h3> <p>Imagine that it is a requirement that the images retrieved from Instagram be shown in a popup (aka modal or lightbox). This can be handled using the Drupal <a href="https://www.drupal.org/project/colorbox">Colorbox module</a>. As such, the Colorbox module will be set as a dependency in the Instagram module's <code>.info.yml</code> file. When a site builder managing their Drupal project with Composer downloads/manages our developer's Instagram module with Composer, then tries to enable the Instagram module in Drupal, they will get an error from Drupal that the Colorbox module is missing. The problem here is that Drupal is being told the Colorbox module is a dependency, but we are not managing that dependency with Composer, and therefore the code has not been downloaded and does not exist.</p> <p>At this point, it is easy to think that site builders could just add the Colorbox module to Composer with a <code>require</code> command. This would work, and after doing so, they would be able to enable the Instagram module, since the Colorbox module now exists. This opens up a problem however; imagine the site builder later decides to remove our developer's Instagram module. They disable/uninstall it in Drupal, then use <code>composer remove</code> to remove it. Everything looks good - the module has been uninstalled from Drupal, and the code has been removed from the codebase. However, there is one more step to return the system to its original state; disabling/uninstalling the now unnecessary Colorbox module from Drupal and removing the code using Composer. The necessity of this additional step opens up situations where the module will be left on the system due to forgetfulness, a lack of documentation, or changing site builders, creating site bloat due to unnecessary code existing and being executed.</p> <p>The solution to this is to also manage Drupal profile/module/theme dependencies with Composer. In our use case, our developer will list the Colorbox module as a dependency of the Instagram module not just in the Instagram module's <code>.info.yml</code> file, but also as a Composer dependency in <code>composer.json</code>. This way, when the Instagram module is added to a project using Composer, Composer will also download and manage the Colorbox module. And when the site builder removes the Instagram module, Composer will remove the Colorbox module as well (if no other libraries list it as a dependency).</p> <h2>Integrating Drupal Code with Composer</h2> <h3>Step 1: Create a composer.json file</h3> <p>Integrating with Composer requires that a <code>composer.json</code> file be created in the root of the profile/module/theme (from here on out referred to as the <em>Drupal library</em>) folder. This is explained in detail on the Drupal.org documentation page <a href="https://www.drupal.org/docs/8/creating-custom-modules/add-a-composerjson-file">Add a composer.json file</a>. Once this file has been added to the Drupal Library, it should be validated. This can be done by running <code>composer validate</code> on the command line in the same directory that the <code>composer.json</code> file lives in.</p> <p>Note that Drupal.org automatically adds a <code>composer.json</code> file to projects that do not have one. This is so that Composer is able to manage all projects on Drupal.org, not just projects to which maintainers have explicitly added a <code>composer.json</code> file.</p> <h3>Step 2: Declare your dependencies to Composer</h3> <p>All library dependencies, whether 3rd party libraries, or contributed Drupal libraries (modules etc), need to be declared as dependencies to Composer. This is done by calling <code>composer require [LIBRARY NAME]</code>, from within the folder of the Drupal library. This means that if you are adding dependencies for a <em>module</em>, you will navigate to the module folder, and add your dependencies. If you are adding dependencies for a <em>theme</em>, navigate to that folder, and add the dependencies there. The key point here is to not add your dependencies from the wrong folder, as they will be be added to the <code>composer.json</code> file of the wrong package.</p> <h4>Adding Remote Library Dependencies</h4> <p>In the use case example for this article, the <a href="https://github.com/cosenary/Instagram-PHP-API">Instagram-PHP-API library</a> was suggested for integration with Instagram. This library has the Composer key <code>cosenary/instagram</code>. To set this library as a dependency of a module, navigate to the root of the module, and run the following:</p> <p><code>composer require cosenary/instagram</code></p> <p>Running this code results in the following:</p> <ol><li>The <em>cosenary/instagram</em> library is added as a dependency to the <code>composer.json</code> file, so that Composer knows the package is managed by Composer. The <code>composer.json</code> file will contain something like the following:<br />   <p><code>"require": {<br />   ...<br />   "cosenary/instagram": "^2.3",<br />   ...<br /> } </code><br /> Note that composer.json is committed to Git.</p> </li> <li> <p>The [MODULE ROOT]/vendor folder is created, and the Instagram-PHP-API library is downloaded into this folder. Composer by default creates the <code>vendor</code> folder in the same directory as the <code>composer.json</code> file. </p> <p> Note that once the Drupal library has been pushed to a repository where it can be managed by Composer, installing the Drupal library with Composer will install the Instagram-PHP-API library to the project's vendor folder. As such, it's a good idea to delete the vendor folder in the module after the new <code>composer.json</code> and <code>composer.lock</code> files have been committed to the remote repository, before requiring the module with Composer.</p> <p> As a best practice, it is a good idea to create a <code>.gitignore</code> file in the module root, and include the vendor directory to ensure that it is never accidentally committed. </p> </li> <li> <p>The <code>composer.lock</code> file is created, locking the installed version of the Instagram-PHP-API library to the Instagram module. This ensures that users of the Instagram library are working with a compatible version of the Instagram-PHP-API library. Note that this file is also committed to Git.</p> </li> <li> <p>Any dependencies of the Instagram-PHP-API library, declared in its own <code>composer.json</code> file, are downloaded.</p> </li> <li> <p>All libraries and dependencies are checked for version conflicts.</p> </li> </ol><h4>Adding Drupal Library Dependencies</h4> <p>In the use case example for the article, the Drupal Colorbox module (aka library) was mentioned to be a dependency of the Instagram module, and therefore is to be managed with Composer. Adding Drupal modules as dependencies requires an additional step, due to their being hosted on a non-default repository. To add a Drupal library as a dependency:</p> <ol><li>Edit the <code>composer.json</code> file and add the Drupal repository to it, so that Composer knows where to look for the Drupal libraries, including the Colorbox module: <p> <code>"repositories": [<br />   {<br />     "type": "composer",<br />     "url": "https://packages.drupal.org/8"<br />   }<br /> ]</code></p> <p> Now Composer knows where to look for the Colorbox module when the Instagram module is installed.  If developers try to declare Colorbox module as a dependency before this step, they will get an error that Composer cannot find the <code>drupal/colorbox</code> library.<br />  </p></li> <li>Add the module with composer require: <p> <code>composer require drupal/colorbox</code></p> <p> Note that this will download the Colorbox module to <code>[MODULE ROOT]/vendor/drupal/colorbox</code>. This is NOT a good thing, for while Drupal will be able to find the module in this location, it is not the standard location, and if another copy of the module ends up in other directories that Drupal scans when looking for modules this will create hard to debug issues. So either delete the Colorbox module in the vendor folder right away, or immediately move it to the location where the rest of your contributed modules are located, so it's very clear where it is at.<br />  </p></li> </ol><p>After this, both the <code>composer.json</code> file and the the <code>composer.lock</code> file should be committed to Git. The module now has its dependencies declared on both the Instagram-PHP-API library and the Drupal Colorbox module. When site builders install the module to their Drupal project using Composer, the Instagram library will be downloaded to the project (site) <code>vendor</code> folder, and the Drupal Colorbox module will be installed to the <code>web/modules/contrib</code> folder.</p> <h2>Developing With Composer-Managed Drupal Libraries</h2> <p>Now that the module has been integrated with Composer, Composer can be used to manage the module during development. Let's imagine that the Instagram module being developed has the key <code>drupal/insta</code>, and is on Version 8.x-1.x. Downloading the -dev version of the module can be done by appending <code>:1.x-dev</code> when requiring the module:</p> <p><code>composer require drupal/insta:1.x-dev</code></p> <p>This will download the 8.x-1.x-dev version of module to the <code>web/modules/contrib</code> folder, where it can be worked with. A <span>.git</span> folder will be included in the module root, so that changes can be committed to Git. Sounds great, but when our developer tries to push their commits, they will get an error, as the remote URL of the repository is not the repository for maintainers of the module and does not allow commits. As such, we need to change the Git repository's remote URL to the URL of the repository for maintainers, allowing for code to be pushed, and releases to be made.</p> <p>To find the correct Git repository URL, first go to the module's download page on Drupal.org. Make sure you are logged in as a maintainer of the module. If you are a maintainer and have the correct permissions, you will see a tab called <em>version control</em>. Click this tab. Next, copy the Git URL on the page. It will look something like this:</p> <p><code>git@git.drupal.org:project/</code>[MODULE NAME]<code>.git</code></p> <p>Next, navigate to your module folder, and run the following commands:</p> <p><code>git remote rm origin<br /> git remote add git@git.drupal.org:project/</code>[MODULE NAME]<code>.git</code></p> <p>The first line of this code removes the incorrect remote Git URL for non-maintainers, and the second line adds the correct Git remote URL for maintainers. After this, you will be able to push changes, including tags (releases).</p> <p>You can also then easily switch between your development and release versions of the module by alternating the following:</p> <p><code>composer require drupal/insta</code></p> <p><code>composer require drupal/insta:1.x-dev</code></p> <p>Note however that when doing any commits, you will need to switch up the remote URL each time you switch to the dev version of the module.</p> <h2 class="display-xxl">Summary</h2> <p>In this final part of the series on using Composer with Drupal, we have looked at how Drupal developers can integrate their custom code with Composer, to ensure that dependencies of the module are correctly managed. We have also looked at how to develop modules that are being managed with Composer. Using these techniques will allow site builders to keep a clean codebase that manages library conflicts.</p> <p>And this concludes my four-part series on Drupal and Composer. I hope you have enjoyed it, and that it shows just how powerful Composer is, and how it can be effectively used to manage Drupal 8 sites. <strong>Happy Composing and Happy Drupaling!</strong></p> Tue, 22 Jan 2019 04:00:00 +1100 Jay Friendly https://www.morpht.com/blog/drupal-and-composer-part-4-composer-drupal-developers Drupal and Composer: Part 3 — Converting Management of an Existing Drupal 8 site to Composer https://www.morpht.com/blog/drupal-and-composer-part-3-converting-management-existing-drupal-8-site-composer <p class="introduction">As any developer working with Drupal 8 knows, working with Composer has become an integral part of working with Drupal. This can be daunting for those without previous experience working with command line, and can still be a confusing experience for those who do.</p> <p>This is the third post in an explorative series of blog posts on Drupal and Composer, hopefully clearing up some of the confusion. The four blog posts on this topic will be as follows:</p> <ul><li>Part 1: <a href="/blog/drupal-and-composer-part-1-understanding-composer">Understanding Composer</a></li> <li>Part 2: <a href="/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer">Managing a Drupal 8 site with Composer</a></li> <li>Part 3: Converting Management of an Existing Drupal 8 Site to Composer</li> <li>Part 4: <a href="https://www.morpht.com/blog/drupal-and-composer-part-4-composer-drupal-developers">Composer for Drupal Developers</a></li> </ul><p>If you have not yet read <a href="https://www.morpht.com/blog/drupal-and-composer-part-1-understanding-composer">part 1</a> and <a href="https://www.morpht.com/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer">part 2</a>, then before reading through this post, it is probably best to ensure you understand the concepts outlined in the summaries of those articles.</p> <h2>Switching Management of your Drupal site to Composer</h2> <p>So you’ve worked your way through parts one and two of this series, and you now understand what Composer is, how it can be used to work with Drupal 8, and how to start a new Drupal 8 project using Composer. But, you started your current project without using Composer, and want to switch to managing your project using Composer. Where do you start? This article will discusses a few strategies behind converting your existing system to Drupal. Fortunately some automated tools exist for converting existing sites to Composer, and in the situation that neither of these tools work, an overview is provided on how to manually convert an existing site </p> <p>But, before moving on to any of these methods...</p> <p><strong>Take a backup!</strong> As this process will be destructive to your system, make sure you take a backup of your file system, and take a backup of your database. Then go and check to ensure that you have a full backup of the file system, and a full back up of the database.</p> <p>If you skip this step, or do not do it properly, you may end up with an entirely broken system, so don’t skip this step.</p> <h2>Method 1: Composerize (Composer plugin)</h2> <p><a href="https://github.com/grasmash/composerize-drupal">Composerize Drupal</a> is a Composer plugin that has been built to convert existing Drupal installations to use Composer. Instructions are on the download page. If this method doesn't work, you can try the Drupal Composerize module:</p> <h2>Method 2: Composerize (Drupal module)</h2> <p>The <a href="https://www.drupal.org/project/composerize">Composerize module</a> is a Drupal module that is built to convert existing Drupal installations to use Composer. At the time of writing, this module has the following disclaimer on the page:</p> <blockquote><p>This module is still in development. It supports very basic Drupal 8 setups, but there are many necessary features it still lacks (e.g., support for patches, JavaScript libraries, distributions, etc). We're working on all that stuff, but <strong>this module is definitely not ready for prime time</strong>.</p> </blockquote> <p>If this method doesn't work, you'll likely have to manually convert your site.</p> <h2>Method 3: Manual method</h2> <p>If the above steps fail, your last option is to convert your installation to using the <a href="https://github.com/drupal-composer/drupal-project">Drupal Composer Template</a> manually. N<span>ote that pretty much every system will be different, so these instructions are an overview of the end goal, rather than a complete set of steps that will convert your system. There is a good chance you’ll run into issues along the way that are not covered here, so make sure you took that backup!</span></p> <p>Converting a system to the template requires achieving the following goals:</p> <ul><li>Setting up the file structure of the system to match the Drupal Composer Template</li> <li>Delete old Composer files</li> <li>Add the Template file system to your Drupal system</li> <li>Set up the configuration directory and the private files directory</li> <li>Set up your profiles, modules and themes to be managed by Composer</li> </ul><h3>Step 1: Convert your file structure to match the Drupal Composer Template file structure</h3> <p>The Drupal Composer Template is set up to manage the directory above the webroot, as well as the webroot. The webroot is located in the [PROJECT ROOT]/web folder. Therefore you will need this structure:</p> <ul><li><code>/</code> - Project root. Contains various scaffolding files such as composer.json and composer.lock, as well as the configuration export folder, and the webroot folder <ul><li><code>/web</code> - The webroot. Contains all Drupal core, profile, module and theme files.</li> </ul></li> </ul><p>You'll need to set up your Drupal installation to match this structure, and make sure that the server is set up so that the web root is at [PROJECT ROOT]/web.</p> <p><strong>Note</strong>: Some servers require the webroot to be in a directory with a specific name, such as <code>public_html</code>, or <code>www</code>. In this case, you can try setting up symlinks from the required directory name to the <code>/web</code> directory, so that, for example, <code>[PROJECT ROOT]/public_html</code> is a symlink pointing at <code>[PROJECT ROOT]/web</code>. Your Drupal files will then reside in the <code>/web</code> directory (satisfying the Drupal template), and also be accessible at <code>/public_html</code> (satisfying the server requirements). If this does not work, another option would be to edit the composer.json file (which is added in step 3) and change any paths that point at the <code>/web</code> directory, to point at the directory name you are actually using.</p> <h3>Step 2: Delete old Composer files</h3> <p>I'll say it again, because it needs to be said, make sure you took that backup in step 0!</p> <p>If any of the following files or folders exist in your installation, delete them. Note however that you may want to save the composer.json file for reference if you've manually added any libraries into your existing installation using Composer.</p> <ul><li><code>[PROJECT ROOT]/vendor</code> directory</li> <li><code>[PROJECT ROOT]/composer.json</code> file</li> <li><code>[PROJECT ROOT]/composer.lock</code> file</li> <li><code>[PROJECT ROOT]/web/vendor</code> directory</li> <li><code>[PROJECT ROOT]/web/composer.json</code> file</li> <li><code>[PROJECT ROOT]/web/composer.lock</code> file.</li> </ul><h3>Step 3: Add the Template file system to your Drupal system</h3> <p>Go <a href="https://github.com/drupal-composer/drupal-project/archive/8.x.zip">here</a>, click 'clone or download' and download the .zip file (note - or clone the Git repository if you prefer)</p> <ol><li>Save/move the zip file into the project root folder (the directory above the 'web' folder you created above). You can then unpack it using the following command. Before this step, the file <code>/composer.json</code> should not exist. After this step, if you've done it correctly, this file will exist.<br /><code>tar -zxvf [FILE] --strip-components=1</code></li> <li>Run the following command from the project root folder. This command will Install the Composer dependencies as well as create the <code>/vendor</code> directory.<br /><code>composer install</code></li> <li>Run the following to ensure your database is up to date, and caches are cleared.<br /><code>drush updb; drush cr;</code></li> </ol><h3>Step 4: Set up the configuration directory and the private files directory (Optional)</h3> <p>This next step is optional, however it will make for a more secure system. First, the following directories need to be created if they don't already exist:</p> <ul><li><code>[PROJECT ROOT]/config/sync</code></li> <li><code>[PROJECT_ROOT]/private</code></li> </ul><p>The first folder is where exports of the Drupal 8 configuration system will be exported to. The second folder is the private files folder. Creating both of these directories as siblings to the webroot adds security, as the files are not in a web-accessible location. The next thing to do is tell the system of the location of these files. This is done by declaring the folder paths in settings.php. You can do this by adding the following two lines to the bottom of settings.php:</p> <p><code>$config_directories['sync'] = '../config/sync';<br /> $settings['file_private_path'] = '../private';</code></p> <p>After this, clear the registry (<code>drush cr;</code>). You can confirm that the configuration directory was properly set by running <code>drush cex sync</code>, and then checking that there are .yml files in the <code>[PROJECT ROOT]/config/sync</code> directory. You can confirm that the private files folder was properly set by going to Admin -&gt; Configuration -&gt; Media -&gt; File System, and confirming that the private files directory is listed as <code>../private</code>.</p> <h3>Step 5: Set up your profiles, modules and themes to be managed by Composer</h3> <p>The final step is to set up Composer to manage Drupal profiles, modules and themes to be managed by Composer. The Drupal Composer Template tracks Core by default, but needs to be informed of the rest of your code. Note that if you do not want to use the most recent version of these profiles/modules/themes, you will need to alter the commands below to set the version you want to install.</p> <p>Drupal profiles, modules and themes can be installed with the following command:</p> <p><code>composer require drupal/[PACKAGE NAME]</code></p> <p>For example, if you were using the Administration Toolbar module (admin_toolbar), you would run:</p> <p><code>composer require drupal/admin_toolbar</code></p> <p>After you have done this, ensure you are up to date with a DB update and cache clear:</p> <p><code>drush updb; drush cr;</code></p> <p>At this point, your system should be converted to the Drupal Composer Template, with contributed code being managed by Composer.</p> <h2>Summary</h2> <p>This article looks at converting exiting Drupal 8 sites to being managed by the <a href="https://github.com/drupal-composer/drupal-project">Drupal Composer Template</a>. Doing this can potentially be automated using the Composerize Composer plugin or the Composerize Drupal module. In situations where this does not work, the manual directions in this article can be used as an alternative.</p> <p>In the next and final part of this series, we'll look at how Drupal developers can integrate 3rd party libraries into their custom Drupal profiles, modules and themes.</p> Thu, 03 Jan 2019 17:11:52 +1100 Jay Friendly https://www.morpht.com/blog/drupal-and-composer-part-3-converting-management-existing-drupal-8-site-composer Drupal and Composer: Part 2 — Managing a Drupal 8 site with Composer https://www.morpht.com/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer <p class="lead">As any developer working with Drupal 8 knows, working with Composer has become an integral part of working with Drupal. This can be daunting for those who don't have previous experience working with the command line, and can still be a confusing experience for those who do. This is the second post in an explorative series of blog posts on Drupal and Composer, hopefully clearing up some of the confusion. The four blog posts on this topic will be as follows:</p> <ul><li>Part 1: <a href="/blog/drupal-and-composer-part-1-understanding-composer">Understanding Composer</a></li> <li>Part 2: Managing a Drupal 8 site with Composer</li> <li>Part 3: <a href="/blog/drupal-and-composer-part-3-converting-management-existing-drupal-8-site-composer">Converting Management of an Existing Drupal 8 Site to Composer</a></li> <li>Part 4: <a href="https://www.morpht.com/blog/drupal-and-composer-part-4-composer-drupal-developers">Composer for Drupal Developers</a></li> </ul><p>This article will be difficult to understand without first understanding the concepts explained in <a href="/blog/drupal-and-composer-part-1-understanding-composer">part 1</a>, so If you have not read it, it would probably be worth your while to ensure you understand the concepts outlined in the summary of that article, before proceeding with this one.</p> <h2>Beginning a New Project</h2> <p>Fortunately a lot of work has been put into creating a Composer base (called a template) for Drupal projects. This Drupal Composer template can be found on Github at: <a href="https://github.com/drupal-composer/drupal-project">https://github.com/drupal-composer/drupal-project</a>. Instructions on how to use the template can be found there, and the same instructions are found in the README.md file that comes with the project when installed.</p> <p>Starting a new project with the Drupal Composer template can be done with the following line of code:</p> <p><code>composer create-project drupal-composer/drupal-project:8.x-dev some-dir --stability dev --no-interaction</code></p> <p>This command does a number of things, many of which will be addressed below.</p> <h2>1) A new Composer project is created</h2> <p>The first thing the installation does is to create the directory specified as <em>some-dir</em> in the command, and initializes a Composer project in that directory. Change this to the appropriate name for your project. This is now your new project. The project contains the <em>composer.json</em> and <em>composer.lock</em> files that will be used to manage the code base of the project.</p> <div class="well">Note: The command provided lists the --<em>stability</em> as <em>dev</em>. This can be confusing as developers may think this means they are getting the -dev version of Drupal core. Don't worry, the command given above will always install the current full release of Drupal core, whatever it is at the time when the command is run.</div> <h2>2) Library dependencies (as well as their dependencies) are installed</h2> <p>The Composer template has a number of dependencies included by default, some of which we will take a look at here. These libraries are set by default as requirements in composer.json, and therefore are included when running the install command given earlier.</p> <ul><li><a href="https://www.drush.org/">Drush</a>: Drush is cool. If you don’t know what it is, it’s worth some Google-fu. Anything to be written about Drush has already been written somewhere else, so check it out - it will be worth your while!</li> <li><a href="https://drupalconsole.com/">Drupal Console</a>: Drupal Console is also really cool. See comments on Drush.</li> <li><a href="https://github.com/cweagans/composer-patches">Composer Patches</a>: This one is very cool. This library in and of itself is worth using Composer to manage Drupal projects in my eyes. Even if Composer had no other benefits, this one would be great. First, an explanation of a patch is necessary. A patch is kind of like a band-aid that can be applied to code. Patches allow developers to submit changes, be they bug fixes or new functionality, to the library maintainer. The library maintainer may or may not add the patch to the source code, but in the meantime, other developers can apply the patch to their own systems, both to test if it works, as well as use it if it does. However, when the library the patch has been applied to is updated to a newer version, the patches have to be re-applied. What Composer Patches does is allow developers to track patches applied to the project, and have them applied automatically during the update process. This ensures that bugs don't arise from forgetting to re-apply patches after the update. Patches are tracked by adding them to composer.json. Here is an example: <p> <code>"extra": {<br />   "patches": {<br />     "drupal/core”: {<br />       “Patch description”: "https://www.drupal.org/files/issues/someissue-1543858-30.patch"<br />     }<br />   }<br /> }</code><br />  </p></li> <li> <p>With the above code, the next time <code>composer update drupal/core</code> is run, Composer will attempt to apply the patch found at <a href="https://www.drupal.org/files/issues/someissue-1543858-30.patch">https://www.drupal.org/files/issues/someissue-1543858-30.patch</a> to Drupal core. Note that the description of the patch, given above as "Patch description", is arbitrary, and should be something descriptive. If there is an issue for the patch, a link to the issue is good to add to the description, so developers can quickly look into the status of the patch, see if any updated patches have been released, and check if the patch has been incorporated into the library, rendering the patch unnecessary.</p> </li> <li>And Others: There are many other libraries that are included as part of the Drupal Composer template, but the truth is that I haven’t looked into them. Also note that Drupal core alone has multiple dependencies which have their own dependencies. At the time of writing, 123 libraries are installed into the project with the install command.</li> </ul><p><strong><em>But wait - I don’t use [fill in library here]</em></strong></p> <p>The Composer template is just that - a template. Some people don’t use Drush, some don’t use Drupal console. If these are not needed, they can be removed from a project in the same manner as any Composer library. Example:</p> <p><code>composer remove drush/drush</code></p> <p>The above command will remove Drush from the code managed by the Composer template.</p> <h2>3) The system folder architecture is created</h2> <p>The file system created by the Composer template deserves a close look.</p> <ul><li><strong>/web:</strong> The first thing to notice is that Drupal is installed into the /web directory in the root of the project. This means that the root of the project created with the Composer template is one level above the webroot. When configuring the server, the domain for the server will need to point at the /web directory.</li> <li><strong>/config/sync:</strong> The Composer template sets up the project to store Drupal 8 configuration .yml files in this folder. When running <code>drush cex sync</code> to export configuration, the entire site configuration will be exported to this folder. This is folder is best kept out of the webroot for security purposes.</li> <li><strong>/drush: </strong>This folder holds a few Drush specific items in it. If multiple environments exist for your project, Drush aliases can be set in /drush/sites/self.site.yml, allowing for interaction with your various environments from anywhere within the project.</li> <li><strong>/scripts:</strong> At the time of writing, this folder contains only a single file, /scripts/composer/ScriptHandler.php. This is a really cool file that contains code run by Composer during various processes. <p> The composer.json file in the Drupal Composer template contains the following:<br />  </p> <p><code>"scripts": {<br />   "drupal-scaffold": "DrupalComposer\\DrupalScaffold\\Plugin::scaffold",<br />   "pre-install-cmd": [<br />     "DrupalProject\\composer\\ScriptHandler::checkComposerVersion"<br />   ],<br />   "pre-update-cmd": [<br />     "DrupalProject\\composer\\ScriptHandler::checkComposerVersion"<br />   ],<br />   "post-install-cmd": [<br />     "DrupalProject\\composer\\ScriptHandler::createRequiredFiles"<br />   ],<br />   "post-update-cmd": [<br />     "DrupalProject\\composer\\ScriptHandler::createRequiredFiles"<br />   ]<br /> },</code></p> <p>The code above executes the stated code on pre-install, pre-update, post-install and post-update. Any time either <code>composer install</code> or <code>composer update</code> are executed on the system, the pre and post hook for that call are executed, calling the relevant functions above. Developers can create their own pre/post install and update hooks following the examples shown above.</p> </li> <li> <p><strong>/vendor: </strong>The first thing to note is that the location of this file differs from a vanilla Drupal 8 installation. When installing Drupal manually, the vendor folder is part of the webroot by default. This however could lead to security issues, which is why the Composer template installs it above the webroot. The vendor folder contains most of the libraries that Composer manages for your project. Drupal core, modules, themes and profiles however are saved to other locations (to be discussed in the next section). Everything else is saved to the /vendor folder.</p> </li> </ul><h2>4) Drupal File/Folder Installation Locations are set</h2> <p>As mentioned above, Drupal core is installed into the /web folder. The Composer template also sets up installation locations (directories) for Drupal libraries, modules, themes and profiles so that when Composer installs these, they are put in the appropriate Drupal folder locations. This is the code in composer.json that handles the installation locations:</p> <p><code>"extra": {<br />   "installer-paths": {<br />     "web/core": [<br />       "type:drupal-core"<br />     ],<br />     "web/libraries/{$name}": [<br />       "type:drupal-library"<br />     ],<br />     "web/modules/contrib/{$name}": [<br />       "type:drupal-module"<br />     ],<br />     "web/profiles/contrib/{$name}": [<br />       "type:drupal-profile"<br />     ],<br />     "web/themes/contrib/{$name}": [<br />       "type:drupal-theme"<br />     ],<br />     "drush/contrib/{$name}": [<br />       "type:drupal-drush"<br />     ]<br />   }<br /> }</code></p> <p>The most common library type that a Drupal developer will install will be Drupal modules. So let’s look at the line of code specific to the module installation location:</p> <p><code>"web/modules/contrib/{$name}": [<br />   "type:drupal-module"<br /> ],</code></p> <p>This line of code says that if the type of Composer library is drupal-module, then install it to the <code>/web/modules/contrib/[MODULE MACHINE NAME]</code> folder.</p> <p>But how does Composer know that the library being downloaded is type drupal-module? Well, the key to this is in how Composer libraries are managed in the first place. Throughout this article and the one that precedes it, we have repeatedly looked at the composer.json file that defines this project. Well, every Composer library contains a composer.json file, and every composer.json file that comes with packaged with Drupal modules contains a type declaration as follows:</p> <p><code>"type": "drupal-module",</code></p> <p>When Composer is installing libraries, it looks for a type declaration, and when it finds one, if there is a custom install location set in composer.json for that type, the library is installed to the declared folder. Drupal themes are of type drupal-theme, Drupal profiles are of type drupal-profile, and so on, and they are installed to the folders declared in composer.json.</p> <h2>Managing Custom Drupal Code with Composer</h2> <p>Custom code for a project can be managed with Composer as mentioned in <a href="https://www.morpht.com/blog/drupal-and-composer-part-1-understanding-composer">Part 1</a> of this series. Drupal convention generally separates contributed (3rd party) modules and custom modules into separate folders. To install custom code in the locations according to this convention, the following lines should be added to the installer-paths declaration in composer.json:</p> <p><code>"web/modules/custom/{$name}": [<br />   "type:drupal-custom-module"]<br /> ,<br /> "web/profiles/custom/{$name}": [<br />   "type:drupal-custom-profile"<br /> ],<br /> "web/themes/custom/{$name}": [<br />   "type:drupal-custom-theme"<br /> ],</code></p> <p>This code adds three additional library types for custom modules, custom profiles, and custom themes.</p> <p>Next, you’ll need to add a composer.json file to the custom module/theme/profile. Directions for this can be seen here: <a href="https://www.drupal.org/docs/8/creating-custom-modules/add-a-composerjson-file">https://www.drupal.org/docs/8/creating-custom-modules/add-a-composerjson-file</a>. Note that for the step named Define your module as a PHP package, you should set the type as <code>drupal-custom-[TYPE]</code>, where [TYPE] is one of: module, theme, or profile.</p> <p>Continuing on, make sure the composer.json file containing the type declaration has been pushed to the remote private repository.</p> <p>The last step step is to add your private repository to you project’s composer.json, so that when running <code>composer require my/privatelibrary</code>, Composer knows in which repository to look for the library. Declaring private repositories in composer.json is explained here: <a href="https://getcomposer.org/doc/05-repositories.md#using-private-repositories">https://getcomposer.org/doc/05-repositories.md#using-private-repositories</a>.</p> <p>With the above steps, when running <code>composer install my/library</code>, Composer will find the private repository declared in composer.json, search that repository for my/library, and download it. The composer.json file in my/library tells Composer that it’s of type <code>drupal-custom-[TYPE]</code>, so Drupal will install it into the directory specified for Drupal custom [TYPE].</p> <p>If using Git for version control on your system, you'll probably want to alter the .gitignore file in the Composer project root to ignore the custom folder locations. If you have created a custom module, and will be managing all custom modules with Composer and private repositories, you should probably add the /web/modules/custom folder to .gitignore. If you will be managing some custom modules with Git and not Composer, then you should probably add the custom module you have created to .gitignore as /web/modules/custom/[MODULE NAME].</p> <h2>Managing site settings: settings.php and settings.local.php</h2> <p>This section isn’t actually directly related to Composer and Drupal, but it’s a good step for setting up a project, and we can use the methodology to work with Composer template and Git.</p> <p>Each Drupal installation depends on the file settings.php. This file is loaded as part of Drupal’s bootstrap process on every page load. Site-specific settings are added into this file, such as database connection information.</p> <p>Towards the bottom of settings.php, the following lines can be found:</p> <p><code># if (file_exists($app_root . '/' . $site_path . '/settings.local.php')) {<br /> # include $app_root . '/' . $site_path . '/settings.local.php';<br /> # }</code></p> <p>These lines are commented out by the # symbol at the start of each line. This means that the code is not executed. If these lines are uncommented, by removing the hash symbol at the start of each line, this code is executed. The code looks for a file named settings.local.php, and if it exists, it includes that file. This means any settings in settings.local.php become part of the bootstrap process and are available to Drupal. After uncommenting the code, it will look like this:</p> <p><code>if (file_exists($app_root . '/' . $site_path . '/settings.local.php')) {<br />   include $app_root . '/' . $site_path . '/settings.local.php';<br /> }</code></p> <p>Why do this? This allows settings to be split into two types: settings that will be the same across all environments (eg. production, staging, local installations etc), and local settings that are only relevant to the current Drupal environment in which this file exists. This is done by committing settings.php to Git so it is shared amongst every environment (note - settings.local.php is NOT committed to Git, as it should not be shared). For example, the majority of Drupal installations have a separate database for each environment, meaning that database connection details will be specific to each environment. Therefore, database settings would be put into settings.local.php, as they are not shared between environments. The connection details to a remote API however may be the same regardless of environment, and therefore would be put into settings.php. This ensures any developer working on the system has access to the API details.</p> <p>After splitting up the settings this way, settings.php is committed to Git so it is tracked and shared between environments. </p> <p>The full process is as follows:</p> <ol><li>Install the Drupal Composer template as described earlier in this article</li> <li>Uncomment the code in settings.php as explained above</li> <li>Install Drupal as normal</li> <li>Run <code>git diff settings.php</code> to see what has been added to settings.php as part of the installation process. Anything that shows up should be added to settings.local.php, and removed from settings.php. This will definitely be the database connection, but could be other files as well.</li> <li>Edit .gitignore in the Composer project root, and remove this line: <p>/web/sites/*/settings.php</p> <p> You can now add settings for all environments to settings.php and settings specific to the local environment to settings.local.php.</p></li> </ol><h2>Setting Up the Private Files Directory</h2> <p>After setting up settings.php and settings.local.php as described above, you can now add the following to settings.php:</p> <p><code>$settings['file_private_path'] = ‘../private’;</code></p> <p>Next, create the folder <code>/private</code> in the root of your Composer project. Finally clear the Drupal cache, which will create the file /private/.htaccess. At this point you can now add settings.php and the /private folder to Git. Finally, edit .gitignore and add the following:</p> <p><code># Ignore private files</code></p> <p><code>/private/</code></p> <p class="display-md">This sets up the private file directories across all installations, saving developers having to set it up for each installation. Note that the .gitignore setting will ensure the contents of this folder are ignored by Git, as they should be.</p> <h2>What should I add to Git?</h2> <p>The Drupal Composer template project page states:</p> <blockquote><p>You should create a new git repository, and commit all files not excluded by the .gitignore file.</p> </blockquote> <p>In particular, you will need to ensure you commit composer.json and composer.lock any time composer changes are made.</p> <h2>Is It Safe to Use Composer on a Production Server?</h2> <p>The actual answer to this question is not one I have. I am not a server guy overall, and for that matter, I’m not even that much of a Composer expert. It may be that Composer is entirely safe on a production server, but personally my thoughts are that having a program that can write files to the server from remote servers would seem to open up a potential security risk, and therefore it’s likely better to NOT have Composer on a production server. This comment may lead you to question why I would have wasted the time to write so much on Composer if it’s better to not use it in the first place. But not so fast partner! It really depends on the system architecture and the server setup. Some servers have been set up so that as part of the deployment process, the codebase is built using Composer, but then set up as a read-only file system or a Docker container or some other process ensuring security. This however is a particularly complex server set up. Fortunately there is an alternative for developers who are not working with servers configured in this fashion, which we'll look at next.</p> <h2>Using Composer, without having Composer installed on the production server</h2> <p>There is an in-between solution that allows us to use Composer to manage our projects, even with multiple developers, while not having Composer installed on the production server. In this case, we can use a hybrid of a Composer managed project and the old style of using pure Git for deployment.</p> <p>First, we need to edit the .gitignore folder in the root of our Composer installation. In particular, we need to remove the following code:</p> <p><code># Ignore directories generated by Composer<br /> /drush/contrib/<br /> /vendor/<br /> /web/core/<br /> /web/modules/contrib/<br /> /web/themes/contrib/<br /> /web/profiles/contrib/<br /> /web/libraries/</code></p> <p>The above directories are all created by Composer and managed by Composer, which is why they were originally ignored by Git. However, we will not be managing our production server using Composer, and therefore we want to include these folders into the Git repository rather than ignoring them.</p> <p>After setting up your project, commit the folders listed above to Git. This ensures all the files that are managed by Composer will be part of Git. That way<code>composer install</code> never needs to be run on the production server, since any code that command would download will already be part of Git.</p> <p>What this means now is that any time a developer on the project add/updates code by running either <code>composer update</code> or <code>composer install</code>, they will need to commit not just the composer.json and composer.lock files, but also the added/updated source files that Composer manages, so that all code be available on the production server when checking out the code from Git.</p> <h2>Updating Drupal using Composer</h2> <p>In <a href="/blog/drupal-and-composer-part-1-understanding-composer">part one</a> of this series, I discussed library versions. I am not going to go deep into how the versioning works internally, but I’ll explain how updating works specific to Drupal core. At the time of writing, the current version of Drupal is 8.6.3. The dependency set in composer.json is for Drupal 8.6.*. The * at the end of this means that your project uses upon any version of Drupal 8.6, so when a minor version update comes out for 8.6, for example 8.6.4, Drupal core will be updated to Drupal 8.6.4 when <code>composer update drupal\core</code> is run.</p> <p>However, when Drupal 8.7 is released, it will not be automatically installed, since it does not fit the pattern 8.6.*. To upgrade to Drupal 8.7, the following command is run:</p> <p><code>composer update drupal/core:~8.7</code></p> <p>The <code>~/8.7</code> part of the above command tells Composer to use any version of Drupal 8.7. This means that in the future, when you run <code>composer update drupal/core</code>, minor releases of the 8.7 branch of Drupal core will be installed.</p> <h2>Summary</h2> <p>In this article, we have gone over how Composer is used with Drupal to make project deployment a little smoother, more stable, and consistent between environments. You should have an understanding of:</p> <ul><li>How to set up a new Drupal project using Composer</li> <li>How the following folders relate to the project: <ul><li>/config</li> <li>/drush</li> <li>/scripts</li> <li>/web</li> <li>/vendor</li> </ul></li> <li>How Composer adds Drupal modules and/or themes</li> <li>Drupal library dependencies</li> <li>Managing custom Drupal code with Composer</li> <li>Which items should be committed to Git</li> <li>How to use Composer to manage Drupal projects where the production server does not have Composer</li> <li>How to update Drupal using Composer</li> </ul><p>In the next post, coming soon, we'll look at how to convert an existing Drupal project to being managed by Composer.</p> Tue, 04 Dec 2018 10:04:28 +1100 Jay Friendly https://www.morpht.com/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer Drupal and Composer: Part 1 — Understanding Composer https://www.morpht.com/blog/drupal-and-composer-part-1-understanding-composer <p class="lead">As any developer working with Drupal 8 knows, working with Composer has become an integral part of working with Drupal. This can be daunting for those without previous experience working with the command line, and can still be a confusing experience for those who do. This is the first post in an explorative series of blog posts on Drupal and Composer, hopefully clearing up some of the confusion. The four blog posts on this topic will be as follows:</p> <ul><li>Part 1: Understanding Composer</li> <li>Part 2: <a href="/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer">Managing a Drupal 8 site with Composer</a></li> <li>Part 3: <a href="/blog/drupal-and-composer-part-3-converting-management-existing-drupal-8-site-composer">Converting Management of an Existing Drupal 8 Site to Composer</a></li> <li>Part 4: <a href="https://www.morpht.com/blog/drupal-and-composer-part-4-composer-drupal-developers">Composer for Drupal Developers</a></li> </ul><p>So without further ado, let’s get started.</p> <h2>Composer: What is it?</h2> <p>The Wikipedia page (<a href="https://en.wikipedia.org/wiki/Composer_(software)">https://en.wikipedia.org/wiki/Composer_(software)</a>) describes Composer as follows:</p> <blockquote><p class="lead"><strong>Composer</strong> is an <a href="https://en.wikipedia.org/wiki/Application-level_package_manager">application-level package manager</a> for the <a href="https://en.wikipedia.org/wiki/PHP">PHP</a> <a href="https://en.wikipedia.org/wiki/Programming_language">programming language</a> that provides a standard format for managing dependencies of PHP software and required <a href="https://en.wikipedia.org/wiki/Library_(computing)">libraries</a>.</p> </blockquote> <p>That’s an accurate description, though a little wordy. So let’s break it down a little further to understand what it means.</p> <p>Programmers like to use the term <strong>DRY</strong> - <strong>D</strong>on’t <strong>R</strong>epeat <strong>Y</strong>ourself. This means that whenever possible, code should be re-used, rather than re-written. Traditionally, this referred to code within the codebase of a single application, but with Composer, code can now be shared between applications as well. DRY is another way of saying <em>don’t re-invent the wheel</em>; if someone else has already written code that does what you want to do, rather than writing code that does the same thing, it’s better to re-use the code that that has already been written. For example, the current standard for authentication (aka logging in) to remote systems is the <em>OAuth 2</em> protocol. This is a secure protocol that allows sites or applications to authenticate with other sites, such as Facebook, Google, Twitter, Instagram, and countless others. Writing OAuth 2 integrations is tricky, as the authentication process is somewhat complex. However, other developers have written code that handles OAuth 2 integration, and they have released this code on the internet in the form of a <em>library</em>. A library is basically a set of code that can be re-used by other sites. Using Composer, developers can include this library in a project, and use it to authenticate to the remote API, saving the developer from having to write that code.</p> <p>Composer allows developers to do the following:</p> <ul><li>Download and include a library into a project, with a single command</li> <li>Download and include any libraries that library is dependent upon</li> <li>Check that system requirements are met before installing the library</li> <li>Ensure there are no version conflicts between libraries</li> <li>Update the library and its dependencies with a single command </li> </ul><h2>So how does Composer work?</h2> <p>Composer itself is a software/program. After a user has installed Composer, they can then say ‘Composer: download Library A to my system’. Composer searches remote <em>repositories</em> for libraries. A repository is a server that provides a collection of libraries for download. When Composer finds Library A in a repository, it downloads the library, as well as any libraries that Library A is dependent upon.</p> <div class="well">A note on terminology <p>In this article, the term <em>Library</em> is used. Libraries are also known as <em>Packages</em>, and referred to as such on <a href="https://getcomposer.org/">https://getcomposer.org/</a></p> <p>A <em>project</em> is the codebase, generally for a website or application, that is being managed by Composer. </p></div> <p>By default, the main repository Composer looks at is <a href="https://packagist.org/">https://packagist.org/</a>. This is a site that has been set up specifically for Composer, and contains thousands of public libraries that developers have provided for use. When a user says ‘Composer download Library A’, the Composer program looks for Library A on <a href="https://packagist.org/">https://packagist.org/</a>, the main public Composer repository, and if it finds the Library, it downloads it to your system. If Library A depends upon (aka requires) Library B, then it will also download Library B to your system, and so on. It also checks to make sure that your system has the minimum requirements to handle both Library A and Library B and any other dependencies, and also checks if either of these packages have any conflicts with any other libraries you've installed. If any conflicts are found, Composer shows an error and will not install the libraries until the conflicts have been resolved.</p> <p>While packagist.org is the default repository Composer searches, projects can also define custom repositories that Composer will search for libraries. For example, many developers use Github or Bitbucket, popular services that provide code storage, to store their code in the cloud. A project owner can set up Composer to look for projects in their private Github, Bitbucket, or other repositories, and download libraries from these repositories. This allows for both the public and private code of a project to be managed using Composer.</p> <h2>What happens when I install a library?</h2> <p>Composer manages projects on a technical level using two files: <em>compser.json</em> and <em>composer.lock</em>.  First we’ll look at the <em>composer.json</em> file. This file describes the project. If a developer is using private repositories, the repositories will be declared in this file. Any libraries that the project depends on are written in this file. This file can also be used to set specific folder locations into which libraries should be installed, or set up scripts that are executed as part of the Composer install process. It’s the outline of the entire project.</p> <p>Each library has a <em>name</em>. The name is combined of two parts, first a <em>namespace</em>, which is an arbitrary string that can be anything but is often a company name, or a Github user name etc. The second part is the library name. The two parts are separated by a forward slash, and contain only lower case letters. Drupal modules are all part of the <em>drupal</em> namespace. Libraries are installed using Composer’s <em>require</em> command. Drupal modules can be installed with commands like:</p> <p><code>// Drupal core:<br /> composer require drupal/core<br /> // Drupal module:<br /> composer require drupal/rules<br /> // Drupal theme:<br /> composer require drupal/bootstrap</code></p> <p>When the above commands are run, Composer downloads the library and its dependencies, and adds the library to the <em>composer.json </em>file to indicate that your project uses the library. This means that <em>composer.json</em> is essentially a metadata file describing the codebase of your project, where to get that code, and how to assemble it.</p> <h2>Composer and Git, Multiple Environments and Multiple Developers</h2> <p>Composer and Git work really well with each other. To understand how, let’s first look at traditional site management using Git. Developer A is creating a new Drupal project, purely managed with Git:</p> <ol><li>Developer A downloads Drupal core</li> <li>Developer A creates a new Git repository for the code they have downloaded, and commits the code to the repository</li> <li>Developer A pushes the code to a central repository (often Github or Bitbucket)</li> <li>Developer A <em>checks out</em> (aka pulls) the code to this server.</li> </ol><p>This all sounds good, and it actually works very well. Now let’s imagine that Developer B comes onto the project. Developer B uses Git to download the code from the central repository. At this point, the codebase in Git exists in four locations:</p> <ul><li>Developer A’s computer</li> <li>Developer B’s computer</li> <li>The central repository</li> <li>The production server </li> </ul><p>At the moment, the codebase only consists of Drupal core. The Drupal core code is being managed through Git, which would allow for changes to be tracked in the code, yet it’s very unlikely that either Developer A or Developer B, or indeed any other developers that come on the project, will actually ever edit any of these Drupal core files, as it is a bad practice to edit Drupal core. Drupal core only needs to be tracked by developers who are developing Drupal core, not by projects that are simply using it. So the above setup results in sharing and tracking a bunch of code that is already shared and tracked somewhere else (on Drupal.org).</p> <p>Let’s look at how to start and use Composer to manage a project. Note that this is NOT the best way to use Composer to manage a Drupal site, and is simply an example to show how to use Composer (see part 2 of this series for specifics on how to use Composer to manage a Drupal site).</p> <ol><li>Developer A creates a new project folder and navigates into it.</li> <li>Developer A initializes the project with <code>composer init</code>, which creates a <em>composer.json </em>file in the project folder</li> <li>Developer A adds the Drupal repository at <a href="https://packages.drupal.org/8">https://packages.drupal.org/8</a> to <em>composer.json</em>, so that Drupal core, modules and themes can be installed using Composer</li> <li>Developer A runs <code>composer require drupal/core</code>, which installs Drupal core to the system, as well as any dependencies. It also creates <em>composer.lock</em> (which we'll look at further down the article)</li> <li>Developer A creates a new Git repository, and adds <em>composer.json</em> and <em>composer.lock</em> to the Git repository</li> <li>Developer A pushes <em>composer.json</em> and <em>composer.lock</em> to the central repository</li> <li>Developer A sets up the production server, and <em>checks out</em> the code to this server. At this point, the code consists only of the <em>composer.json</em> and <em>composer.lock </em>files. Additional servers can be set up by checking out the code to any server.</li> <li>Developer A runs <code>composer install</code> on the production server. This pulls all the requirements and dependencies for the project as they are defined in <em>composer.json</em></li> </ol><p>Now when Developer B comes on the project, Developer B uses Git to download the codebase to their local computer. This codebase contains only <em>composer.json</em> and <em>composer.lock.</em> However, when they run <code>composer install</code> they will end up with the exact same codebase as the production server and on Developer A’s machine.</p> <p>Now the codebase exists in the same four locations, however the only code being tracked in the Git repository is the two files used to define the Composer managed project. When an update is made to the project, it is handled by running <code>composer update drupal/core</code>, which will update both <em>composer.json</em> and <em>composer.lock</em>. These files are then updated in the Git repository, as they are the files specific to our project.</p> <p>The difference between the traditional Git method, and the above method using Composer, is that now Drupal core is considered to be an external library, and is not taking up space unnecessarily in our project's Git repository.</p> <h2>Project Versions</h2> <p>Projects can, and pretty much always do, have versions. Drupal 8 uses semantic versioning, meaning that it goes through versions 8.1, 8.2, 8.3… and so on. At the time of writing the current version is 8.6.3. If a new security fix is released, it will be 8.6.4. In time, 8.7.0 will be released.  Composer allows us to work with different versions of libraries. This is a good thing, however it opens up the risk of developers on a project working with different versions of a library, which in turn opens up possibility of bugs. Composer fortunately is built to deal with versions, as we will look at next.</p> <h2>Tracking Project Versions</h2> <p>So how does Composer handle versions, allowing developers to ensure they are always using the same library versions? Welcome the <em>composer.lock</em> file. The <em>composer.lock</em> file essentially acts as a snapshot of the all the versions of all the libraries managed by <em>composer.json</em>. Again, I’ll refer back to the Composer managed site described above. When we first run <code>composer require drupal/core</code> in our project, a few things happen:</p> <ol><li>The current (most recent) version of Drupal is downloaded to the system</li> <li>All libraries that Drupal depends on are also downloaded to the system</li> <li><em>composer.json</em> is updated to show that Drupal is now a dependency of your project</li> <li><em>composer.lock</em> is created/updated to reflect the current versions of all Composer managed libraries</li> </ol><p>So <em>composer.json</em> tracks which libraries are used, and <em>composer.lock</em> is a snapshot tracking which versions of those libraries are currently being used on the project. </p> <h2>Synchronizing Project Versions</h2> <p>The problem with developers using different versions of libraries is that developers may write code that only works on the version of the library that they have, and other developers either don’t yet have, or maybe they are using an outdated version of the library and other developers have updated. Composer projects manage library versions using the commands <code>composer install</code> and <code>composer update</code>. These commands do different things, so next we'll look at the differences between them.</p> <h2>Composer Install and Composer Update</h2> <p>Imagine that Composer didn’t track versions. The following situation would happen (again, this is NOT how it actually works):</p> <ol><li>Drupal 8.5.6 is released.</li> <li>Developer A creates a new project, and sets Drupal core as dependency in <em>composer.json</em>. Developer A has Drupal 8.5.6</li> <li>Drupal 8.6.0 is released</li> <li>Developer B clones the Git project, and installs the codebase using <code>composer install</code>. Composer downloads Drupal core. Developer B has Drupal 8.6.0</li> </ol><p>The two developers are now working on different versions of Drupal. This is dangerous, as any code they write/add may not be compatible with each other's code. Fortunately Composer can track libraries. When a user runs <code>composer install</code>, the versions defined in <em>composer.lock</em> are installed. So when Developer B runs <code>composer install</code>, Drupal 8.5.6 is installed, even though Drupal 8.6.0 has been released, because 8.5.6 is listed as the version being used by the project in <em>composer.json. </em>As such, developers working on Composer managed projects should run <code>composer install</code> each time they pull updates from remote Git repositories containing Composer managed projects.</p> <h2>Updating versions</h2> <p>As has been discussed, the <em>composer.lock</em> file tracks the versions of libraries currently used on the project. This is where the <code>composer update</code> command comes in. Let’s review how to manage version changes for a given library (this is how it actually works):</p> <ol><li>Drupal 8.5.6 is released.</li> <li>Developer A creates a new project, and sets Drupal core as dependency. The <em>composer.lock</em> file records the version of Drupal core used by the project as 8.5.6.</li> <li>Drupal 8.6.0 is released</li> <li>Developer B clones the Git project, and installs the codebase using <code>composer install</code>. The composer.lock file lists the version of Drupal core being used on the project as 8.5.6, so it downloads that version.</li> <li>Developer A sees that a new version of Drupal has been released. Developer A runs <code>composer update drupal/core</code>. Composer installs Drupal 8.6.0 to their system, and updates <em>composer.lock</em> to show the version of Drupal core in use as 8.6.0.</li> <li>Developer A commits this updated <em>composer.lock</em> to Git, and pushes it to the remote repository. </li> <li>Developer B pulls the Git repository, and gets the updated <em>composer.lock</em> file. Developer B then runs <code>composer install</code>, and since the version of Drupal core in registered as being used is now 8.6.0, Composer updates the code to Drupal 8.6.0.</li> </ol><p>Now Developer A and Developer B both have the exact same versions of Drupal on their system. And still the only files managed by Git at this point are <em>composer.json</em> and <em>composer.lock</em>.</p> <h2>Tying it all together</h2> <p>Developers should always run <code>composer.install</code> any time they see that a commit has made changes in the <em>composer.lock</em> file, to ensure that they are on the same codebase as all other developers. Developers should also always run composer.install anytime they switch Git branches, such as between a production and a staging branch. The dependencies of these branches may be very different, and running <code>composer install</code> will update all dependencies to match the current <em>composer.lock</em> snapshot. The <code>composer update</code> command should only be used to update to new versions of libraries, and the <em>composer.lock</em> file should always be committed after running <code>composer update</code>. Finally, any time a developer adds a new dependency to the project, they need to commit both the <em>composer.json</em> file and the <em>composer.lock</em> file to Git.</p> <h2>Summary</h2> <p>Before moving on to the next blog post in this series, you should understand the following:</p> <ul><li>What the <em>composer.json</em> file does</li> <li>What the <em>composer.lock</em> file does</li> <li>When to use <code>composer install</code></li> <li>When to use <code>composer update</code></li> <li>How Git and Composer interact with each other</li> </ul><p>In <a href="/blog/drupal-and-composer-part-2-managing-drupal-8-site-composer">Part 2</a>, we'll look specifically at building and managing a Drupal project using composer.</p> Mon, 26 Nov 2018 09:58:58 +1100 Jay Friendly https://www.morpht.com/blog/drupal-and-composer-part-1-understanding-composer Simple Social Service links for Drupal 8 https://www.morpht.com/blog/simple-social-service-links-drupal-8 <p class="lead">There are couple of online tools, and integration modules to get sharing widget to your site. They rely on JavaScript and the security of your users is questionable. This article will show you how to create a simple yet flexible and safer service sharing widget without line of JavaScript.</p> <p><a></a></p> <h2>Background</h2> <p>The main reason why not to use some of the tools like AddToAny is the security. This is often a case for government or other public facing project such as <a href="https://www.drupal.org/project/govcms8">GovCMS</a>. Sharing widget of these services is not connecting directly to the social service, but it is processed on their servers first. And they can track the user on through the web because of the fingerprint they made. Another reason is that the JS code is often served from a CDN so you don’t know when the code changes and how? Have they put them some malicious script? I don’t want this on my site. And clients often as well. :)</p> <p>Thankfully each service provide a simple way how to share content and we will use that.</p> <p><a></a></p> <h3>Final example</h3> <p>You can see the final result in action with different styling applied at our example <a href="https://govcms8.site-showcase.com/test/extensions">GovCMS 8 demo page</a> (scroll down to the bottom of page).</p> <p><a></a></p> <h2>Site build</h2> <p>First we need to prepare the data structure. For our purpose we will need to create a custom block type, but it can be easily done as a paragraph too.</p> <p><strong>Custom block name:</strong> Social Share<br /><strong>Machine name:</strong> [social_share]</p> <p>And throw in few Boolean fields. One for each service.</p> <p><strong>Field label:</strong> [Human readable name] e.g. “Twitter”<br /><strong>Machine name:</strong> [machine_name] e.g. “social_share_twitter” <em>– this one is important and we will use it later.</em></p> <p>Go to the manage display screen of the block (/admin/structure/block/block-content/manage/social_share/display) and change the Output format to Custom. Then fill in the Custom output for TRUE with the text you like to see on the link e.g. "<strong>Share to twitter</strong>".</p> <p><article class="embedded-entity align-right"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><!--[if IE 9]><video style="display: none;"><![endif]--><source srcset="/sites/morpht/files/styles/full_lg/public/social-share-labels.png?itok=O_13nab4 1x, /sites/morpht/files/styles/full_lg_hi/public/social-share-labels.png?itok=ftSFmscA 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/social-share-labels.png?itok=VzypQ_po 1x, /sites/morpht/files/styles/full_md_hi/public/social-share-labels.png?itok=3wuEAjmr 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/social-share-labels.png?itok=cOjLXoF5 1x, /sites/morpht/files/styles/full_sm_hi/public/social-share-labels.png?itok=PHg1_dJs 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/social-share-labels.png?itok=8cXkpyrH 1x, /sites/morpht/files/styles/full_xs_hi/public/social-share-labels.png?itok=WDuVwujL 2x" type="image/png"></source><!--[if IE 9]></video><![endif]--><img src="/sites/morpht/files/styles/full_xs/public/social-share-labels.png?itok=8cXkpyrH" alt="Social Share labels" typeof="foaf:Image" /></picture></div> </article></article></p> <p>Now we are able to create a new block of the Social share type and check some of these checkboxes. Users will see only the Labels as result.</p> <p><a></a></p> <h2>Theming</h2> <p>The fun part is changing the output of the field from simple label to actual share link.<br /> First we need to know how the final link looks like.<br /> Links examples:</p> <strong>Facebook:</strong> <a href="http://www.facebook.com/share.php?u=[PAGE_URL]&amp;title=[PAGE_TITLE">http://www.facebook.com/share.php?u=[PAGE_URL]&amp;title=[PAGE_TITLE</a>] <strong>Twitter:</strong> <a href="http://twitter.com/intent/tweet?status=[PAGE_TITLE]+[PAGE_URL">http://twitter.com/intent/tweet?status=[PAGE_TITLE]+[PAGE_URL</a>] <strong>LinkedIn:</strong> <a href="http://www.linkedin.com/shareArticle?mini=true&amp;url=[PAGE_URL]&amp;title=[PAGE_TITLE]&amp;source=[BASE_PATH">http://www.linkedin.com/shareArticle?mini=true&amp;url=[PAGE_URL]&amp;title=[PA…</a>] <strong>E-mail:</strong> mailto:?subject=Interesting page [PAGE_TITLE]&amp;body=Check out this site I came across [PAGE_URL] <p>To get it work we need a current page <em>Page URL</em>, <em>Page title</em>, and <em>Base path</em>. Only the page URL is directly accessible from TWIG template. The other two needs to be prepared in preprocess. Lets add these in the theme_name.theme file.</p> <code><span><span>/** * Implements template_preprocess_field(). */</span> <span>function</span> <span>theme_name_preprocess_field</span>(&amp;<span>$variables</span>, <span>$hook</span>) { <span>switch</span> (<span>$variables</span>[<span><span>'</span><span>field_name</span><span>'</span></span>]) { <span>case</span> <span><span>'</span><span>field_social_share_twitter</span><span>'</span></span>: <span>$request</span> = \<span>Drupal</span>::request(); <span>$route_match</span> = \<span>Drupal</span>::routeMatch(); <span>$title</span> = \<span>Drupal</span>::service(<span><span>'</span><span>title_resolver</span><span>'</span></span>) -&gt;getTitle(<span>$request</span>, <span>$route_match</span>-&gt;getRouteObject()); <span>if</span> (<span>is_array</span>(<span>$title</span>)) { <span>$variables</span>[<span><span>'</span><span>node_title</span><span>'</span></span>] = <span>$title</span>[<span><span>'</span><span>#markup</span><span>'</span></span>]; } <span>else</span> { <span>$variables</span>[<span><span>'</span><span>node_title</span><span>'</span></span>] = (<span>string</span>) <span>$title</span>; } <span>$variables</span>[<span><span>'</span><span>base_path</span><span>'</span></span>] = base_path(); <span>break</span>; } } </span></code><p>As we probably will have more then one service we should use the DRY approach here. So we create extra function for the variable generation.</p> <code><span><span>/** * Preprocess field_social_share. */</span> <span>function</span> <span>_theme_name_preprocess_field__social_shares</span>(&amp;<span>$variables</span>) { <span>$request</span> = \<span>Drupal</span>::request(); <span>$route_match</span> = \<span>Drupal</span>::routeMatch(); <span>$title</span> = \<span>Drupal</span>::service(<span><span>'</span><span>title_resolver</span><span>'</span></span>) -&gt;getTitle(<span>$request</span>, <span>$route_match</span>-&gt;getRouteObject()); <span>if</span> (<span>is_array</span>(<span>$title</span>)) { <span>$variables</span>[<span><span>'</span><span>node_title</span><span>'</span></span>] = <span>$title</span>[<span><span>'</span><span>#markup</span><span>'</span></span>]; } <span>else</span> { <span>$variables</span>[<span><span>'</span><span>node_title</span><span>'</span></span>] = (<span>string</span>) <span>$title</span>; } <span>$variables</span>[<span><span>'</span><span>base_path</span><span>'</span></span>] = base_path(); } </span></code><p>And we than call it for various cases. If some service will need more variables it will be easy to add it in different function. So we don’t process whats not required.</p> <code><span><span>/** * Implements template_preprocess_field(). */</span> <span>function</span> <span>theme_name_preprocess_field</span>(&amp;<span>$variables</span>, <span>$hook</span>) { <span>switch</span> (<span>$variables</span>[<span><span>'</span><span>field_name</span><span>'</span></span>]) { <span>case</span> <span><span>'</span><span>field_social_share_facebook</span><span>'</span></span>: _theme_name_preprocess_field__social_shares(<span>$variables</span>); <span>break</span>; <span>case</span> <span><span>'</span><span>field_social_share_twitter</span><span>'</span></span>: _theme_name_preprocess_field__social_shares(<span>$variables</span>); <span>break</span>; <span>case</span> <span><span>'</span><span>field_social_share_linkedin</span><span>'</span></span>: _theme_name_preprocess_field__social_shares(<span>$variables</span>); <span>break</span>; <span>case</span> <span><span>'</span><span>field_social_share_email</span><span>'</span></span>: _theme_name_preprocess_field__social_shares(<span>$variables</span>); <span>break</span>; } } </span></code><p>Now we have the Node title and Base path prepared to be used in field templates.</p> <p>Enable twig debug and look in the markup for the checkbox. You will see couple of suggestions, the one we are looking for is <em>field--field-social-share-twitter.html.twig</em>.</p> <p>As the output should be single link item it is safe to assume we can remove all the labels condition and the single/multiple check as well. On the other hand we need to ensure that if the checkbox is unchecked it will not output any value. That is particularly hard in TWIG as it doesn’t have any universal information about the state of checkbox. It has only access to the actual value. But since we don’t know the value of custom label we cannot use it. However there is a small workaround we can use. Remember we hav not set the FALSE value.<br /> We can check if the field is outputting any #markup. The empty FALSE value will not produce anything, hence the condition will fail.</p> <code><span>{% <span>if</span> item.content[<span><span>'</span><span>#markup</span><span>'</span></span>] %} </span></code><p>Here is the full code for field template:</p> <code><span>{% set classes = [ <span><span>'</span><span>social-share__service</span><span>'</span></span>, <span><span>'</span><span>social-share__service--twitter</span><span>'</span></span>, ] %} {% <span>for</span> item in items %} {% <span>if</span> item.content[<span><span>'</span><span>#markup</span><span>'</span></span>] %} &lt;a {{ attributes.addClass(classes) }} href=<span><span>"</span><span>http://twitter.com/intent/tweet?status=</span><span>{</span><span>{</span><span> node_title }}+</span><span>{</span><span>{</span><span> url('&lt;current&gt;') }}</span><span>"</span></span> title=<span><span>"</span><span>Share to </span><span>{</span><span>{</span><span> item.content }}</span><span>"</span></span>&gt;{{ item.content }}&lt;/a&gt; {% <span>endif</span> %} {% <span>endfor</span> %} </span></code><p> For other services you need to adapt it. But it will still follow the same pattern.</p> <p>And we are done. Now your block should return links to sharing current page to the service.</p> <p><article class="embedded-entity"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><!--[if IE 9]><video style="display: none;"><![endif]--><source srcset="/sites/morpht/files/styles/full_lg/public/social-share-links.png?itok=uNpbduia 1x, /sites/morpht/files/styles/full_lg_hi/public/social-share-links.png?itok=4fWjWNgK 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/social-share-links.png?itok=F9y-7UTS 1x, /sites/morpht/files/styles/full_md_hi/public/social-share-links.png?itok=OYhVKl5E 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/social-share-links.png?itok=DLydSixX 1x, /sites/morpht/files/styles/full_sm_hi/public/social-share-links.png?itok=mL7bhjgw 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/social-share-links.png?itok=atEiMZEh 1x, /sites/morpht/files/styles/full_xs_hi/public/social-share-links.png?itok=MSDoLA_E 2x" type="image/png"></source><!--[if IE 9]></video><![endif]--><img src="/sites/morpht/files/styles/full_xs/public/social-share-links.png?itok=atEiMZEh" alt="Social Share links" typeof="foaf:Image" /></picture></div> </article></article></p> <p><a></a></p> <h2>Pro tip:</h2> <p>So far we have not use any contrib module. But obviously your client would like to have some fancy staying applied. You can add everything in the theme, but that will be only one hardcoded option. For easier live of editors you can use <a href="https://www.drupal.org/project/entity_class_formatter">Entity Class formatter</a> module to easily add classes to the block from a select list. You can provide multiple select list for Size, Color, Rounded corners, Style etc.</p> <p><a></a></p> <h2>Result</h2> <p>At this point we have the simple social share widget ready. We can select which predefined services will show in each instance and how will they look. E.g. On blog post you can have sharing for Twitter, Facebook and Email styled as small rounded icons. But with another instance of the block you can have only large squared LinkedIn icon + label shown on Job offering content type.</p> <p><article class="embedded-entity"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><!--[if IE 9]><video style="display: none;"><![endif]--><source srcset="/sites/morpht/files/styles/full_lg/public/social-share-icons.png?itok=772HrWYT 1x, /sites/morpht/files/styles/full_lg_hi/public/social-share-icons.png?itok=cx8nJTHz 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/social-share-icons.png?itok=hglZuTWR 1x, /sites/morpht/files/styles/full_md_hi/public/social-share-icons.png?itok=VFnK5Jww 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/social-share-icons.png?itok=bWqlP7pA 1x, /sites/morpht/files/styles/full_sm_hi/public/social-share-icons.png?itok=6ohI7Uig 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/social-share-icons.png?itok=Cej12Izq 1x, /sites/morpht/files/styles/full_xs_hi/public/social-share-icons.png?itok=x0GD6Ly3 2x" type="image/png"></source><!--[if IE 9]></video><![endif]--><img src="/sites/morpht/files/styles/full_xs/public/social-share-icons.png?itok=Cej12Izq" alt="Social Share icons" typeof="foaf:Image" /></picture></div> </article></article></p> <p><a></a></p> <h2>Further notes</h2> <p>After I wrote first draft of this article new module appeared which work in very similar way. Give it a try at <a href="https://www.drupal.org/project/better_social_sharing_buttons">Better Social Sharing Buttons</a>. It will be quicker to get up ad running as it has predefined styles and services, but that can be a drawback at the same time. If I need different style, or extra service it can be harder to add it.</p> Thu, 27 Sep 2018 21:46:46 +1000 Petr Illek https://www.morpht.com/blog/simple-social-service-links-drupal-8 Announcing Webform Mass Email port for Drupal 8 https://www.morpht.com/blog/announcing-webform-mass-email-port-drupal-8 <p class="lead">The Webform Mass Email module has now been ported to Drupal 8. Time to get emailing :)</p> <p>The <a href="https://www.drupal.org/project/webform">Webform</a> module has long been a powerhouse for Drupal site builders, providing a flexible solution to one of the central problems of web development: collecting user data, storing it and notifying users when content is added. the Webform module has been extended in many directions, supporting ancillary workflows and use cases.</p> <p>One such use case is sending a mass email to all subscribers to a <a href="https://www.drupal.org/project/webform">Webform</a>. This can be handy for when users are signing up to an event or have registered their interest in a topic. The <a href="https://www.drupal.org/project/webform_mass_email">Webform Mass Email</a> module fills this gap.</p> <h3>The module works as follows</h3> <p>When installed, Webform Mass Email module adds a new sub-tab called "Mass Email" under all Webform's "Results" tab (next to "Submissions", "Download" and "Clear") and one settings sub-tab called "Mass Emails" under global Webform "Configuration" tab (next to "Exporters", "Libraries" and "Advanced").</p> <ol><li>Site builder navigates to the "Mass Emails" tab and set base module settings once. <ul><li>"Cron time" - How much time is being spent per cron run (in seconds).</li> <li>"Allow sending as HTML" - If checked, all emails are processed as HTML formatted.</li> <li>"Log emails" - Should the emails be logged to the system log at admin/reports/dblog?</li> </ul></li> <li>The site builder can then assign "Send Webform Mass Email" permission for roles which are able to send mass emails.</li> <li>Anyone with this permission can then navigate to the "Mass Email" sub-tab of any Webform and send messages. <ul><li>"Email field" - The field that is going to be used as the recipient's email address.</li> <li>"Subject" - Message subject for your email.</li> <li>"Body" - Message text for your email.</li> </ul></li> <li>After you hit the "Send emails" button, messages are inserted into queue and sent when cron is running.</li> </ol><p>The module has now been ported to Drupal 8 and is being supported by the team at Morpht. <a href="https://www.drupal.org/project/webform_mass_email">Check it out</a> and let us know what you think.<br />  </p> Mon, 13 Nov 2017 13:57:19 +1100 Vit Hovezak https://www.morpht.com/blog/announcing-webform-mass-email-port-drupal-8 Announcing Entity Class Formatter for Drupal 8 https://www.morpht.com/blog/announcing-entity-class-formatter-drupal-8 <p class="lead">The Entity Class Formatter is a nifty little module which allows editors to place classes on to entities, allowing their look and feel to be altered in the theme layer or with other modules such as Look and Modifiers. It is now available for Drupal 8.</p> <p><a href="https://www.drupal.org/project/entity_class_formatter">Entity Class Modifier</a> is a humble little module, however, it does open up a lot of possibilities. The most obvious is to use the theme layer to provide styles for the class which has been applied. This makes it possible for the “designer” at design time to can some different styles to pick from. It is however, possible to use the module in a more flexible way and combine it with Modifiers and Looks.</p> <p>Once a class has been defined and added to a field, a “Look Mapping” can be defined, associating the class with a set of Modifiers. The site builder or skilled editor can then go in and define any number of Modifiers which will be fired with the class.</p> <p>For example - a “my-awesome-class” class could be created which is wired into a “field_my_awesome” set of Modifiers. The Modifiers could include a blue linear gradient with white text overlay with generous padding. All of this configuration happens in the UI after deploy. It is a very flexible and powerful system. The system can be configured after deployment and adapted to the styles you wish to apply.</p> <h2>Basic use of Entity Class Formatter</h2> <p>The use of the module is very easy. We can for example define our own class on the article.</p> <p>The first thing we need to do is to enable the module. Once installation is complete we can go and add our custom field. In this little tutorial we will basically add the class onto the article content type. So go to Structure &gt; Content types &gt; Article &gt; Manage fields and add new text field. We can name the field simply "Class" and save it. As we keep everything in default we can save it on the next page too.</p> <p><article class="embedded-entity"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><!--[if IE 9]><video style="display: none;"><![endif]--><source srcset="/sites/morpht/files/styles/full_lg/public/add_text_field.png?itok=GXuck4im 1x, /sites/morpht/files/styles/full_lg_hi/public/add_text_field.png?itok=EF3p5Oqm 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/add_text_field.png?itok=duZnH3to 1x, /sites/morpht/files/styles/full_md_hi/public/add_text_field.png?itok=5pjjrYs9 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/add_text_field.png?itok=MmGoYEVP 1x, /sites/morpht/files/styles/full_sm_hi/public/add_text_field.png?itok=UMHKHivi 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/add_text_field.png?itok=fuOzc1TN 1x, /sites/morpht/files/styles/full_xs_hi/public/add_text_field.png?itok=kWrbE2_7 2x" type="image/png"></source><!--[if IE 9]></video><![endif]--><img src="/sites/morpht/files/styles/full_xs/public/add_text_field.png?itok=fuOzc1TN" alt="Adding of text field" typeof="foaf:Image" /></picture></div> </article></article></p> <p> </p> <p>Now the last thing we need to do to make it work is set the Entity Class on the field in Manage display page. Go to Structure &gt; Content types &gt; Article &gt; Manage display and change the format to "Entity Class". There's no need to any other manipulation with field because it won't render any values which would be visible to the visitor of the page.</p> <p><article class="embedded-entity"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><!--[if IE 9]><video style="display: none;"><![endif]--><source srcset="/sites/morpht/files/styles/full_lg/public/set_formatter.png?itok=82L7P369 1x, /sites/morpht/files/styles/full_lg_hi/public/set_formatter.png?itok=5xADcy5P 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/set_formatter.png?itok=cCFIORbl 1x, /sites/morpht/files/styles/full_md_hi/public/set_formatter.png?itok=hNGWfTzq 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/set_formatter.png?itok=DB07u3O4 1x, /sites/morpht/files/styles/full_sm_hi/public/set_formatter.png?itok=SuPqZjTD 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/set_formatter.png?itok=1Uy0nziE 1x, /sites/morpht/files/styles/full_xs_hi/public/set_formatter.png?itok=W-dFa68k 2x" type="image/png"></source><!--[if IE 9]></video><![endif]--><img src="/sites/morpht/files/styles/full_xs/public/set_formatter.png?itok=1Uy0nziE" alt="Setting of the formatter" typeof="foaf:Image" /></picture></div> </article></article></p> <p> </p> <p>That's it! No we can go to create an article (Content &gt; Add content &gt; Article). Input class to our field...</p> <p><article class="embedded-entity"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><!--[if IE 9]><video style="display: none;"><![endif]--><source srcset="/sites/morpht/files/styles/full_lg/public/add_in_class.png?itok=u58PLoUL 1x, /sites/morpht/files/styles/full_lg_hi/public/add_in_class.png?itok=VwTQ616M 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/add_in_class.png?itok=l94Hi-Dj 1x, /sites/morpht/files/styles/full_md_hi/public/add_in_class.png?itok=qo3lkELp 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/add_in_class.png?itok=EKpDW2p8 1x, /sites/morpht/files/styles/full_sm_hi/public/add_in_class.png?itok=EPlPLJYq 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/add_in_class.png?itok=maklSHiL 1x, /sites/morpht/files/styles/full_xs_hi/public/add_in_class.png?itok=4oxWB_KG 2x" type="image/png"></source><!--[if IE 9]></video><![endif]--><img src="/sites/morpht/files/styles/full_xs/public/add_in_class.png?itok=maklSHiL" alt="Adding of class" typeof="foaf:Image" /></picture></div> </article></article></p> <p>... voila, class is there!</p> <p><article class="embedded-entity"><article class="media-entity media-entity--type-image media-entity--view-mode-full"><div class="field field--name-field-media-image field--type-image field--label-hidden field__item"> <picture><!--[if IE 9]><video style="display: none;"><![endif]--><source srcset="/sites/morpht/files/styles/full_lg/public/inspect_article.png?itok=8jVbfuK6 1x, /sites/morpht/files/styles/full_lg_hi/public/inspect_article.png?itok=tjEWDuVb 2x" media="(min-width: 992px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_md/public/inspect_article.png?itok=ykNhc-2P 1x, /sites/morpht/files/styles/full_md_hi/public/inspect_article.png?itok=0pDbKXA- 2x" media="(min-width: 768px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_sm/public/inspect_article.png?itok=w9ddRZ6X 1x, /sites/morpht/files/styles/full_sm_hi/public/inspect_article.png?itok=MrkbnQl1 2x" media="(min-width: 576px)" type="image/png"></source><source srcset="/sites/morpht/files/styles/full_xs/public/inspect_article.png?itok=NR9wgAj8 1x, /sites/morpht/files/styles/full_xs_hi/public/inspect_article.png?itok=uX0RSA84 2x" type="image/png"></source><!--[if IE 9]></video><![endif]--><img src="/sites/morpht/files/styles/full_xs/public/inspect_article.png?itok=NR9wgAj8" alt="Checking if class is on place" typeof="foaf:Image" /></picture></div> </article></article></p> <h2>Similar but different</h2> <p>There are a couple of modules out there which are similar but are different enough for them not to be totally suited to our requirements.</p> <p><a href="https://www.drupal.org/project/classy_paragraphs">Classy Paragraphs</a>, available in Drupal 8, has been influential in the Paragraphs ecosystem and has opened the way for creative designs. It was intended to apply to Paragraphs only and is quite structured in the way classes are applied through a custom field type. The Entity Class Formatter module is more flexible in that it has been designed to apply to all entity types. It can also handle a variety of field types (text, select lists, entity references) and is able to adapt to the data at hand. So, Entity Class Formatter has a similar aim - it is just somewhat more generic.</p> <p><a href="https://www.drupal.org/project/field_formatter_css_class">Field Formatter CSS Class</a>, available in Drupal 7, also uses a field formatter approach to applying the class to the entity. It does have more complexity than this module because it deals with several layers of nesting. The Entity Class Formatter is very simple and only applies to the parent entity of the field.</p> <div class="well"><a href="https://www.drupal.org/project/entity_class_formatter">Entity Class Formatter</a> was inspired by Classy Paragraphs (thanks) and is supported by the team at Morpht.</div> Fri, 10 Nov 2017 14:06:37 +1100 Jan Pára https://www.morpht.com/blog/announcing-entity-class-formatter-drupal-8 Announcing Enforce Profile Field for Drupal 8 https://www.morpht.com/blog/announcing-enforce-profile-field-drupal-8 <p class="lead"> The Enforce Profile Field is a new module which allows editors to enforce the completion of one or more fields in order to access content on a site. It is now available for Drupal 8.</p> <p>Sometimes you need to collect a variety of profile data for different users. The data may be needed for regulatory compliance or marketing reasons. In some cases you need a single field and in others it may be several. You may also wish to collect the information when a user access certain parts of the site.</p> <p>The <a href="https://www.drupal.org/project/enforce_profile_field">Enforce Profile Field</a> module comes to the rescue in cases such as these, forcing users to complete their profile before being able to move onto the page they want to see. This may sound harsh, however, collecting data as you need it is a more subtle way of collecting data than enforcing it all at registration time.</p> <p>The implementation consists mainly from a new Field Type called "Enforce profile" and hook_entity_view_alter().</p> <h3>The module works as follows</h3> <ol><li>Site builder defines a “form display” for the user type bundle and specify fields associated with it to collect data. <ol><li>The fields should not be required, as this allows the user to skip them on registration and profile editing.</li> <li>In addition the <a href="https://www.drupal.org/project/form_mode_manager">Form Mode Manager</a> module can be used to display the “form display” as a "tab" on a user profile page.</li> </ol></li> <li>The site builder places an Enforce profile field onto an entity type bundle, such as a node article or page.</li> <li>The Enforce profile field requires some settings: <ol><li>A "User's form mode" to be utilized for additional field information extraction (created in the first step).</li> <li>An "Enforced view modes" that require some profile data to be filled in before being able to access them. You should usually select the "Full content" view mode and rather not include view modes like "Teaser" or "Search".</li> </ol></li> <li>The editor creates content, an article or page, and can then select which fields need to be enforced. <ol><li>The editor is provided with multi-select of "User's form mode" fields.</li> <li>Selecting nothing is equal to no access change, no profile data enforcement.</li> </ol></li> <li>A new user navigates to the content and is redirected to the profile tab and is informed that they need to complete the fields.</li> <li>Fields are completed, form submitted and the user redirected back to the content. <ol><li>In case the user doesn't provide all enforced fields, the profile tab is displayed again with the message what fields need to be filled in.</li> </ol></li> </ol><h3>Why to use the Enforce Profile Field to collect an additional profile data?</h3> <ul><li>You may need customer's information to generate a coupon or access token.</li> <li>You may just want to know better with whom you share information.</li> <li>Your users know exactly what content requires their additional profile data input rather than satisfying a wide range of requirements during registration. It just makes it easier for them.</li> <li>The new profile data can be synced to a CRM or other system if required to.</li> </ul><p>Let us know what you think.<br />  </p> Wed, 08 Nov 2017 10:42:36 +1100 Dalibor Matura https://www.morpht.com/blog/announcing-enforce-profile-field-drupal-8 Announcing Webform Invitation port for Drupal 8 https://www.morpht.com/blog/announcing-webform-invitation-port-drupal-8 <p class="lead"> The Webform Invitation module has now been ported to Drupal 8.</p> <p>Most of us know how powerful the <a href="https://www.drupal.org/project/webform">Webform</a> module is in Drupal, allowing us to collect and retain data from users. It has been extended in many ways and is a vital part of most Drupal websites.</p> <p>The <a href="https://www.drupal.org/project/webform_invitation">Webform Invitation</a> module extends Webform and allows submissions to be limited by invitation through the entry of a code which validates the submitted against a list of known codes. This pattern can be used to apply “coupons” and handle any situation where you need to limit the users who are able to submit.</p> <p>All generated codes can be downloaded as XLS spreadsheet with direct links leading to the Webform URL with "code" parameter inside, so it's then automatically filled into "Invitation Code" text field - you can use this spreadsheet for any of your favorite mass emailing service.</p> <h3>The module works as follows</h3> <p>When installed, Webform Invitation module adds a new tab to all Webforms called "Invitation" (next to "View", "Test", "Results" and "Build"). There are also four sub-tabs called "Settings" (default action), "List codes", "Generate", and "Download".</p> <ol><li>Site builder defines new Webform or update existing one, where invitations are needed.</li> <li>The Site builder navigates to the "Invitation" tab and enables invitations for the current Webform. <ul><li>This option is adding new "webform_invitation_code" text field element on the first position inside "Build" section and you can adapt the position or other element details as required.</li> <li>When option is disabled, the "webform_invitation_code" element is removed from any position.</li> </ul></li> <li>Then the Webform is locked for any submission, because there are no invitation codes available and site builder needs to generate some. It's possible by navigating to "Generate" sub-tab and setting number of generated codes and their type (MD5 or custom). <ul><li>"Number of codes to generate" is 25 by default, but you can increase this number as needed.</li> <li>"Type of tokens" is set to MD5 hash of 32 characters length, but you can switch to custom token and set their length and character sets to be used.</li> <li>After clicking on "Generate", you are redirected to "List codes" sub-tab and you are ready for sending these invitation codes out and users are able to fill submissions. You can monitor if specific codes were used and for which submission.</li> <li>The site builder can generate new codes of different type any time.</li> </ul></li> <li>If site builder wants to export all unused invitation codes, he can do it on "Download" sub-tab. <ul><li>It provides XLS spreadsheet with direct links to the Webform URL with "code" parameter inside.</li> <li>This code is automatically filled into "Invitation Code" text field.</li> </ul></li> </ol><p>The <a href="https://www.drupal.org/project/webform_invitation">Webform Invitation</a> module is supported by the team at Morpht. Give it a try and let us know what you think.<br />  </p> Tue, 07 Nov 2017 21:05:37 +1100 Vit Hovezak https://www.morpht.com/blog/announcing-webform-invitation-port-drupal-8 Creating comments via email in Drupal 7 https://www.morpht.com/blog/creating-comments-email-drupal-7 <p>For a recent project we needed to allow users to comment on content via email. The idea was that when new comments were created, Drupal would send out notification emails that could be replied to. The system had to be able to handle long email chains and a few edge cases. Email is tricky and we learned a lot about the wild world of email. We are sharing our experiences here.</p> <p><a href="https://www.drupal.org/project/mailcomment">Mail Comment</a> lets you reply via email to notifications from your website. In this article we’ll show you how to reply to content via email and outline the mechanisms that make it work. There are quite a few modules involved in the process and they all have an important role to play.</p> <h3>Modules</h3> <p>You will need the following modules installed:</p> <ul><li><a href="https://www.drupal.org/project/rules">Rules</a></li> <li><a href="https://www.drupal.org/project/message">Message</a></li> <li><a href="https://www.drupal.org/project/message_notify">Message Notify</a></li> <li><a href="https://www.drupal.org/project/mailcomment">Mail Comment</a></li> <li>Mail Comment Message Notify</li> <li><a href="https://www.drupal.org/project/mailhandler">Mailhandler</a></li> <li><a href="https://www.drupal.org/project/feeds">Feeds</a></li> <li><a href="https://www.drupal.org/project/feeds_comment_processor">Feeds Comment Processor</a></li> </ul><h2>The Workflow</h2> <p>Here’s a typical workflow and how each module fits into the process:</p> <ol><li>User creates a comment</li> <li>Drupal sends a notification email to recipients (Rules + Mail Comment Message Notify)</li> <li>A recipient replies to this notification email via their email client</li> <li>The reply email is sent to the “From” header which goes to our mailbox (Mailhandler)</li> <li>Drupal imports the email from this mailbox (Feeds importer)</li> <li>Drupal validates the email’s authenticity (Mail Comment)</li> <li>Drupal maps the email’s metadata to comment fields (Feeds + Mail Comment)</li> <li>Drupal creates a comment (Feeds Comment Processor)</li> </ol><h2>The Steps</h2> <p>With the basic concepts in hand – let’s build out this functionality.</p> <h3>Create a Message type</h3> <p>The Message framework provides an entity type designed specifically for dealing with messages. Messages are great for creating and sending notifications. Create a message type via Structure -&gt; Message types -&gt; Add message type. This message type will represent a notification that occurs when someone posts a comment.</p> <p>In the “Description” field label your message “New Comment”. The mandatory “Message text” field will become the notification email subject. The form requires a save before correctly displaying “Message text” fields and their associated view modes (a UI bug). Put any value in this field as we’ll be changing it after saving this form.</p> <p>If you’ve previously enabled Mail Comment then you will see a very important checkbox “Allow users to reply to this message by email”. This has to be ticked.</p> <p>Save the form and edit the newly created message type. Now you will see two special view modes which will be displayed as the notification email’s Subject and Body.</p> <p>In “Message text -&gt; View modes: Notify - Email subject” enter a subject:</p> New comment created on [nid]<p>In “Message text -&gt; View modes: Notify - Email” enter a body:</p> [comment:user:mail] says: [comment:value] View: [comment:node:url]<p>Save the form and then click “Manage fields”. Create two fields:</p> <ul><li>field_message_rendered_subject (Text field)</li> <li>field_message_rendered_body (Text area)</li> </ul><p>These fields are used by Message Notify to save the output of the Subject and Body view modes for use with email.</p> <h3>Create a notification rule</h3> <p>Now that the message type is set up we can create a notification rule that is triggered whenever someone creates a comment.</p> <ol><li>Create a rule that reacts on the event “After saving a new comment”.</li> <li>Add the action “Send Message with Message Notify”.</li> <li>In the “Data selector” select the comment that triggered this rule.</li> <li>Set “Rendered subject field” to “Rendered subject” and “Rendered body field” to “Rendered body”.</li> <li>Set the “The recipient of the email” to any email address you’d like.</li> </ol><h3>Configure Mail Comment</h3> <p>Mail Comment works its magic on both sides. It modifies outgoing emails so that they can be replied to, and authenticates incoming emails and extracts metadata. Mail Comment does this by embedding a unique string in the outgoing email that contains email metadata and a hash, which we will go into more detail later.</p> <p>Go to Configuration -&gt; System -&gt; Mail Comment (/admin/config/system/mailcomment).</p> <p>Set “Reply-To address“ to the email account that the Feeds importer will be importing from.<br /> “Insert reply to text” is a handy option to have enabled. It will help Mail Comment seperate the latest content in an email from the email’s thread. Other settings are fine to leave at their defaults.</p> <h3>Create a Mailhandler Mailbox</h3> <p>Email replies will be sent to a mailbox that you control. Mailhandler allows you to create a connection to this mailbox. A Feeds importer will then be importing emails from this mailbox via Mailhandler.</p> <p>Create a mailbox via Structure -&gt; Mailhandler Mailboxes -&gt; Add (admin/structure/mailhandler/add).<br /> The “Username” is likely the same value as Mail Comment’s “Reply-To” email address.</p> <h3>Create a Feeds importer</h3> <p>The Feeds importer extracts the email via Mailhandler's mailbox. It runs via cron.</p> <p>Create a Feeds importer via Structure -&gt; Feeds importers -&gt; Add importer. (/admin/structure/feeds/create)<br /> Label it “New Comments”.</p> <h4>Fetcher</h4> <p>Configure the fetcher’s “Message filter” to “Comments only”. This filter works by checking if an email’s In-Reply-To header is set. At this time of writing the filter doesn’t handle all email situations correctly. Please see <a href="https://www.drupal.org/node/1370286#comment-11960587">this comment</a> for a solution.</p> <h4>Parser</h4> <p>Select “Mailhandler IMAP stream parser” with the “Authentication plugin” as “Mail Comment Authentication”. Mail Comment authenticates and validates the incoming email to ensure that it was originally sent from the website and replied to by a valid user.</p> <h4>Processor</h4> <p>Feeds’ importer processor creates the comment. Select “Comment processor” as your processor. For our particular project at Morpht we created messages instead of comments with “Message entity processor” and a custom module – but that’s another story.</p> <p>The Mapping configuration here is where all the magic happens. Metadata is pulled from the email and exposed as a source value. The following source values need to be mapped from source to target:</p> <ul><li>Message ID (message_id) -&gt; GUID (guid)</li> <li>Parent NID (parent_nid) -&gt; Node id (nid)</li> <li>User ID (authenticated_uid) -&gt; User ID (uid)</li> <li>Subject (subject) -&gt; Title (subject)</li> <li>Body (Text) (body_text) -&gt; Comment (comment_body)</li> </ul><p>Message ID and Parent NID are source values created by Mail Comment from the email’s metadata.</p> <h2>Summary</h2> <p>The steps above detail how to reply to notifications sent when someone create a comment. Similar steps can be used to send notifications when someone creates a node. Create a new message type called “new_node” and follow the steps above, replacing the comment specific configuration with node config.</p> <h2>Testing</h2> <p>Test the system by creating a comment on a node through your website’s UI. A notification email should be sent out to recipients, including your testing email address. Reply to this email, run cron and the comment should be imported. To assist your debugging checkout Content -&gt; Messages and the <a href="https://www.drupal.org/project/maillog">Maillog</a> module.</p> <h2>Dealing with email clients</h2> <p>If you’d like to know what’s going on under the hood feel free to read below. There’s a reward for reading on. This section will also show you how to reply to long email chains and still have the email imported by Drupal. It also fixes a few bugs with the current version of Mail Comment at the time of this writing.</p> <h3>The Message-ID</h3> &lt;<a href="mailto:1234.4666.5056.1486965753.cb17604ac1434f792f04f724d2af7ee6@example.com">1234.4666.5056.1486965753.cb17604ac1434f792f04f724d2af7ee6@example.com</a>&gt;<p>Mail Comment stores important information in an email by creating a custom string full of metadata about the original comment. This string is sent in the email inside the Message-ID header. Message-ID is a standard email header that email clients use to insert identifying information about the email. Usually it’s a big hash but in our case it’s a dot separated string of values containing:</p> <ul><li>Node ID</li> <li>Comment ID</li> <li>User ID</li> <li>Timestamp</li> <li>Hash</li> </ul><p>The format is four digits and one hash divided by dots, an @ symbol and finally the domain that the email was sent from.</p> <h3>Long email chains</h3> <p>Mail Comment’s metadata in the Message-ID header will be lost as soon as someone replies to an email. The email client replying to the email will overwrite Message-ID with its own value. Thankfully there is another email header specifically designed to store a history of all Message-IDs used in the email chain’s lifetime: References. The References header stores each Message-ID in chronological order like so:</p> &lt;<a href="mailto:4431.5196.7356.1487571103.20e2a3d69bef550a990d7c751beb84de@example.com">4431.5196.7356.1487571103.20e2a3d69bef550a990d7c751beb84de@example.com</a>&gt; &lt;<a href="mailto:CAPbhD2K4MvcCPqSNRhp33P+CoNkY5xyb+qNqeJBuhMrxeMCBsg@mail.gmail.com">CAPbhD2K4MvcCPqSNRhp33P+CoNkY5xyb+qNqeJBuhMrxeMCBsg@mail.gmail.com</a>&gt; &lt;CAPbhD2JZdgQj==1AxDWqaR2N=<a href="mailto:jrQtvrz3hQ-03b9jkgJg5eoFA@mail.gmail.com">jrQtvrz3hQ-03b9jkgJg5eoFA@mail.gmail.com</a>&gt;<p>As you can see the original Message-ID is on the first line.</p> <p>Email clients send emails in unexpected formats and can sometimes break the Message-ID metadata. This is especially true with Microsoft Outlook/Exchange, which inserts a seemingly random value at the beginning of the References header (which is actually the third value of the dot separated string).</p> 7356 &lt;<a href="mailto:4431.5196.7356.1487571103.20e2a3d69bef550a990d7c751beb84de@domain.com">4431.5196.7356.1487571103.20e2a3d69bef550a990d7c751beb84de@domain.com</a>&gt; &lt;<a href="mailto:CAPbhD2K4MvcCPqSNRhp33P+CoNkY5xyb+qNqeJBuhMrxeMCBsg@mail.gmail.com">CAPbhD2K4MvcCPqSNRhp33P+CoNkY5xyb+qNqeJBuhMrxeMCBsg@mail.gmail.com</a>&gt; &lt;CAPbhD2JZdgQj==1AxDWqaR2N=<a href="mailto:jrQtvrz3hQ-03b9jkgJg5eoFA@mail.gmail.com">jrQtvrz3hQ-03b9jkgJg5eoFA@mail.gmail.com</a>&gt; <h4>Our Patch</h4> <p>We have <a href="https://www.drupal.org/node/2852329">created a patch</a> to better detail with oddly-formatted References headers and long email chains. Feel free to apply.</p> <h3>HTML being sent as plain text</h3> <p>Microsoft Outlook/Exchange occasionally sends out badly formatted HTML in the plain text version of the email. There’s no silver bullet solution. We came up with a simple fix that just strips out all the bad HTML leaving you the plain text.</p> <p>In your custom module create the following function:</p> <p><code>&lt;?php<br /> /**<br />  * Implements hook_feeds_after_parse()<br />  */<br /> function mymodule_feeds_after_parse(FeedsSource $source, FeedsParserResult $result) {<br />   foreach ($result-&gt;items as &amp;$email) {<br />     // Filter html<br />     $email['body_text'] = _mymodule_filter_html($email['body_text']);<br />     $email['body_html'] = _mymodule_filter_html($email['body_html']);<br />   }<br /> }</code></p> <p>This hook will run just after feeds has parsed in values, and call the filter html function below:</p> <p><code>&lt;?php<br /> /**<br />  * Strip html<br />  */<br /> function _mymodule_filter_html($text) {<br />   // remove all tags<br />   $text_remove = strip_tags($text);<br />   // decode html special characters<br />   $text_remove = html_entity_decode($text_remove, ENT_QUOTES);<br />   // replace multiple empty lines with on<br />   $text_remove = preg_replace('/\n(\s*\n)+/', "\n\n", $text_remove);<br />   return $text_remove;</code><br /><code>}</code></p> <p>Once these two functions are pasted in, clear Drupal’s cache (drush cc all).</p> <h2>Conclusion</h2> <p>Now you know how to create content in Drupal from email! The basic ingredients are Mail Comment, Feeds importer and a few modules gluing it all together.</p> <p>You can use similar techniques to create nodes from email too. Just be mindful that there are security implications doing this as you can’t always trust the sender and contents of an email. Mail Comment authenticates incoming mail by creating a hash and this is harder to do when importing an email without a Mail Comment generated Message-ID.</p> <p>As Drupal heads more and more towards being a REST based backend it’s easy to forget the more traditional means of inputting data. Email is ancient on the technology timeline, but its convenience coupled with Drupal’s well defined data structure can be a great combo. Happy mailing!</p> Wed, 10 May 2017 00:40:25 +1000 Maedi Prichard https://www.morpht.com/blog/creating-comments-email-drupal-7