You demonstrated AI in Drupal before the recent ChatGPT and OpenAI disruption. What got you interested in AI enough to create a module?
We have followed AI for a few years and have seen steady progress in natural language processing and similar tech. We have viewed the integration of AI into CMSs such as Drupal as a natural progression where we can improve the lives of editors by helping them with their day-to-day tasks. We have been talking to customers about this functionality, so it was a fun challenge to make it work in Drupal.
One of the key drivers at Morpht is to position Drupal as a content experience platform (CXP) rather than just a CMS. This means we view content as a valuable resource that can be used and adapted to serve a number of purposes that extend beyond just the presentation of the content on a website. The burgeoning AI ecosystem represents a great opportunity for Drupal as content is the source material for many new services being developed. Don't take the content to the Ai; bring the AI to the content.
These ideas lead us to develop the Augmentor module, an extensible system for integrating AI into Drupal. We are actively working on different use cases to improve personalisation with Augmentor, a contributed module available on Drupal.org. Our session at last year's Drupal South was just the beginning of where we are going with AI and LLMs.
We have to admit that even we were taken aback by the recent “growth spurt" in LLMs and the general interest in using AI to generate and augment content. It's impressive to see how quickly people have taken to this new technology and how it is now being used. This certainly is an amazing time to be alive.
What is the thinking behind Augmentor?
Augmentor was created out of the recognition that many solutions in Drupal are bespoke and tend to solve the same issues for different use cases. We surveyed the work done in the broader Drupal community, and it became clear that the same old "plumbing" issues needed to be solved each time - integrating AI into the UI and the editor experience. Augmentor aimed to provide an implementation for standard UI features so that future AI modules did not need to reinvent the wheel on this. We have been working on integrating AI into the node edit form and the WYSIWYG editor. More integrations are in the pipeline.
Augmentor creates a pluggable system for AI integrations. We looked at common patterns for abstracting AI integrations and built them for maximum composability. The basic pattern for Augmentor is that augmentations take an input (file, HTML, text) and return a keyed array of output. That is the heart of it. Once we had this common pattern down, we were able to integrate a variety of external services.
Then we built a set of essential integrations: Open AI (GPT3 completion), Google Text to Speech, Google Speech to Text and Google Cloud Vision API. This allowed us to grab the low-hanging fruit for integrating LLMs for text generation and analysing images for keyword extraction. We recently integrated NLP Cloud, which provides a wide array of tools for processing textual content. This has been helpful for extracting concepts from controlled vocabularies. So now we are not just dealing with "tags" but also terms contained in Drupal's taxonomy system. This is a big win for editors who want to automatically add categories to their content.
These are the kinds of things you can do with Augmentor and its plugins:
- Generate article summaries
- Generate articles from a title or an idea.
- Extract keywords for tagging.
- Extract taxonomy terms from controlled vocabularies
- Extract keywords from images
- Extra NSFW metadata from images along a number of dimensions (violence, nudity, etc.)
- Generate text from speech and vice versa.
What are your goals for the Augmentor project?
We want to kick off an ecosystem around AI in Drupal and get people to think about how to make their customers’ lives easier. Adding a new integration is pretty straightforward, as all the developer needs to handle is the connection and the configuration. We recently added OpenAI, ChatGPT, and NLP Cloud, which took a few hours. The developer just needs to focus on using the library of their choice and ensuring the UI handles the various parameters that need to be passed into it.
It would be great to see the addition of new sub-modules and plugins supporting new AI systems. We are also working on extending the integration points into Drupal. If you are considering working on AI in Drupal, look at the Augmentor AI module and its ecosystem. You can also catch up with the devs in the Augmentor Slack channel to discuss your ideas.
What of the future?
The AI landscape is evolving very rapidly. That is an understatement. One thing is for sure: things will continue to change and develop. ChatGPT and GPT4 have taken the world by storm. This doesn't mean that they are the only game in town. New models are being released all of the time, with capabilities in different domains. That is why we have taken a neutral approach to handling AI in Drupal. Augmentor can be extended to new systems that emerge and adapt as new services come along. We are interested in what the community comes up with in terms of integrations.
Any upcoming features you can discuss?
So far, we have integrated Augmentor with the node edit screen. We have been focussed on improving the editor experience on a node-by-node basis. It is important to consider the "editor" as part of the workflow, as they will review the generated content before being saved into the CMS. This is a sensible approach; however, it lacks the benefits of AI's scalability. Things will really start to be fun when we can process content in bulk, unlocking the power of AI across the whole corpus, not just on individual pieces of content.
We have been looking at the Entity Condition Action (ECA) module, which offers a lot of promise in this area. ECA can be seen as a replacement for the Rules modules, which were very popular in Drupal 7. ECA hooks into Drupal APIs (events and actions) and can be used to wire together business logic. It makes a lot of sense to expose an Action for Augmentors so they can be integrated into the ECA ecosystem. This allows an editor to augment content automatically whenever a node is created or saved. It is also possible to process several nodes at once using this approach.
This is very exciting, as it offers a pathway to uplifting large amounts of content in a straightforward manner. This provides a lot of potential for clients sitting on a large amount of content that could do with a bit of love - I've seen a lot of sites with poor summary metadata or inconsistently tagged with categories, for example.
What else? It's all about unlocking the value which is in your content. If a system can be put together well, it should be possible to automatically build custom knowledge bases and then use these to drive applications such as chat and search. It is all about automating processes and making the data available to smart services, which can present that data in new ways. This is a really exciting area.
Finally, there is the area of semantic search. LLMs can extract "embeddings" that capture content's semantic meaning rather than relying on keywords. We have seen services such as Algolia (Search.io) and Recombee start incorporating these services into their search and recommendation engines. This is another area where I expect to see a lot of activity in the near future.
Ultimately we want to unlock the value stored in content. The solutions we are building are about something other than content spinning or replacing content creators. We wish to provide a range of tools to help creators and editors carry out their tasks and for site owners who want to make the most of their precious content. After all, data matures like wine, and applications like fish.
