Media embedding in Drupal 8.8
2007 is the year of my first DrupalCon, and the year the #1 most wanted end-user feature was Better media handling
. 2019 is the year that Drupal will finally have it. Doing things right takes time!
Back then I never would’ve believed I would someday play a small role in making it happen :)
Without further ado, and without using a mouse:
The text editor assisted in producing this HTML:
<p>Let's talk about llamas!</p>
<drupal-media alt="A beautiful llama!" data-align="center" data-entity-type="media" data-entity-uuid="84911dc4-c086-4781-afc3-eb49b7380ff5"></drupal-media>
<p>(I like llamas, okay?)</p>
If you’re wondering why something seemingly so simple could have taken such a long time, read on for a little bit of Drupal history! (By no means a complete history.)
2007 and Drupal five
Twelve years ago, in Dries’ State of Drupal talk 1, Better media handling
was deemed super important. I attended a session about it — this is the (verbatim) session description:
- Drupal’s core features for file management and media handling
- common problems and requirements (restrictions, performance issues, multi-lingual content, dependencies between nodes and files)
- first approaches: own node types for managing, improved filemananger.module (example: Bloomstreet,European Resistance Archive, Director’s Cut Commercials)
- next step: generic media management module with pluggable media types, mutli server infrastructure, different protocols, file systems, file encoding/transcoding
It’s surprisingly relevant today.
By the way, you can still look at the session’s slides or even watch it!
2007–2013 (?)
The era of the venerable Media module, from which many lessons were learned, but which never quite reached the required level of usability for inclusion in Drupal core.
2013 (?) – 2019
The Media initiative started around 2013 (I think?), with the Media entity module as the first area of focus. After years of monumental work by many hundreds of Drupal contributors (yes, really!), only one missing puzzle piece was left: WYSIWYG embedding of media. The first thing I worked on after joining Acquia was shipping a WYSIWYG editor with Drupal 8.0, so I was asked to take this on.
To help you understand the massive scale of the Media Initiative: this last puzzle piece represents only the last few percent of work!
Drupal has always focused on content modeling and structured content. WYSIWYG embedding of media should not result in blobs of HTML being embedded. So we’re using domain-specific markup (<drupal-media>) to continue to respect structured content principles. The result is document transclusion combined with an assistive “WYSIWYG” editing UX — which we wished for in 2013.
A little less than two months ago, we added the MediaEmbed text filter to Drupal 8.8 (domain-specific markup), then we made those have previews using CKEditor Widgets for assistive “WYSIWYG” editing, followed by media library integration and per-embed metadata overriding (for example overriding alt, as shown in the screencast).
I was responsible for coming up with an architecture that addressed all needs. To maximally learn from past lessons, I first worked on stabilizing the contributed Entity Embed module. So this builds on many years worth of work on the Entity Embed module from Dave Reid, cs_shadow and slashrsm! Thanks to the historical backlog of reported issues there, we are able to avoid those same problems in Drupal core. The architecture of the CKEditor integration was designed in close collaboration with Krzysztof Krzton of the CKEditor team (thanks CKSource!), and was also informed by the problems reported against that module. So this stabilization work benefited both Drupal core as well as tens of thousands of existing sites using Entity Embed.
I then was able to “just” lift Drupal core patches out of the Entity Embed module (omitting some features that do not make sense in Drupal core). But it’s phenaproxima (thanks Acquia!), oknate and rainbreaw who got this actually committed to Drupal core!
Complete media management shipped in increments
Fortunately, for many (most?) Drupal 8 sites, this will not require significant rework, only gradual change. Drupal 8.8 will ship with complete media management, but it’ll be the fifth Drupal core release in a little over two years that adds layers of functionality in order to arrive at that complete solution:
- Drupal 8.4 added foundational Media API support, but still required contributed modules to be able to use it
- Drupal 8.5 made Media work out-of-the-box
- Drupal 8.6 added oEmbed support (enabling YouTube videos for example) and added an experimental Media Library
- Drupal 8.7 made the Media Library UI-complete: bulk uploads, massively improved UX
- Drupal 8.8 will contain the key thing that was blocking Media Library from being marked stable (non-experimental): WYSIWYG integration
Today is the perfect moment to start looking into adopting it shortly after Drupal 8.8 ships in December!
Thanks to phenaproxima for feedback on this post!
-
See slide 40! ↩︎
Comments
Exciting news! How about migration from Drupal 7 file to
<drupal-media>tags?For migration, see https://www.drupal.org/project/drupal/issues/2930514 and especially https://www.drupal.org/project/drupal/issues/2885002 which is part of that.
Migration support is still being worked on, and will certainly be opt-in. Automatically migrating would certainly disrupt some existing workflows.
To add some details to this history, from 2007 to 2013 two solutions were available for media handling, the famous Media module and the Scald suite which already enabled reusability of media content and drag and drop of Scald atoms into WYSIWYG-enabled fields. The fusion of the ideas led to the media entity into Drupal 8.
Thanks for making the history more complete :)
I’d love to read a detailed history about this! I kept it very high-level because I was not involved for the full ride.
I’d love a detailed longread like https://wimleers.com/blog/api-first-drupal-file-uploads-572-comments-summarized about how Media in Drupal core came to be!
Such a great leap in Drupal’s history, I’ve been waiting for this moment for years with the Scald team! Great work Wim, thank you for it and this feels like the landing of Apollo 11, what a great timing!
I wouldn’t quite compare it to that, but I’m glad your so excited! :D
With Drupal 8, I use the contrib Juicebox module to create photo galleries. These consist of thousands of jpg files (and thumbnail versions of jpg files) spread across dozens of Galleries. I upload the files to a directory structure that I manage using FTP within
sites/default/files. If I want to re-use these jpg files within an article content type I use the contrib IMCE module. It lets me browse through my directory structure and insert the image into the article using CKEditor. It works fine.So, now what happens? Will media library “discover” my existing files and directory structure? Do I leave the existing files where they are and start adding new files to Media Library? Is there a conversion or migration step. My current Galleries depend on a pre-specified set of files in a pre-specified directory structure. If this changes my galleries go ker-plunck.
It’s going to be up to https://www.drupal.org/project/juicebox to provide an update path. There actually already is an issue for adding Media support to the Juicebox module, and it’s been open for more than 2.5 years: https://www.drupal.org/project/juicebox/issues/2840494. It also seems that the Juicebox module is not actively maintained: the latest release is a beta, which is more than 2.5 years old too…
Then again, this module seems highly optimized for one particular use case that you could also choose to continue to use this.
There is no simple answer here: because it depends.
The high-level meta issue for migrating into Media is https://www.drupal.org/project/drupal/issues/2930514.
I use Juicebox and views to make image galleries from core media entities and it works fine. It sounds like that issue was maybe for the contrib version of Media? Works fine with core.
Why only media? Same goes for the media gallery module which provides the browser to populate media reference fields. That’s like making moderation only available for nodes.
There’s so many reasons to only do embedding of media and not arbitrary entities! This was discussed in detail at https://www.drupal.org/project/drupal/issues/2801307.
Here are some of the high-level reasons:
Nice work Wim, media embedding is finally about to make it into “core”. I haven’t yet tried out 8.8.x however I did recently try the varbase distribution and was very impressed with it’s media handling, very very nice.
https://www.drupal.org/project/varbase
I’m glad to see you’re excited about this too! I see that https://www.drupal.org/project/varbase has pioneered integrating the many prior contributed Media modules into a coherent whole. Do you plan to transition it from Entity Browser to Media Library?
Congrats !!! , Media is finally here , a few years after Wordpress did it , but importantly we did a million times better. Thanks to everybody in the media team for all the work. The demonstration video with the keyboard is super awesome and a nice touch.
Glad you like it so much! :D
I’m not following Drupal so closely these days but this was one of main things I looked for few years, and finally it’s here! Reading this story brought some good memories :) Congrats and thank you for all the work you and others put into it!
Thanks for the kind words and sharing the good memories — and hopefully see you back in the Drupal community soon? :)
I have a rather confusing hotch-potch of modules that don’t quite do a good job. Which ones of the following will I no longer need in Drupal 8.8?
Great question!
You won’t need any of those modules anymore :)
mediaentities. The 3 modules you listed could handle arbitrary entity types, but because of that could not optimize the UX as much.The Media and Media Library modules ship with Drupal core. So it’s only a matter of enabling those, adding that filter to your text format and adding that button to your text editor. That’s it :)
I wouldn’t say that Media Library supersedes Entity Browser or any of its related modules; I think this has been confusing for people (and disappointing for some). But, the fact of the matter is that Media Library was built to be simple and fairly limited, on purpose. Entity Browser is a LOT more flexible and you can select anything with it. It’s got superpowers. Media Library should work for most standard use cases with a little flexibility, but if you need to really take control and build a very customized UI, or support things that aren’t media, Entity Browser is the way to go. Same thing goes for Entity Embed – Media Library provides the most important media-specific functionality, but Entity Embed is where it’s at for more complicated uses.
You’re of course right! Note I did say
However, that is indeed still a too simplistic answer from me. If you’re doing “typical” media stuff, then my answer is correct. As soon as you have complex needs, Entity Browser + Entity Embed are indeed still the way to go.
Thanks Wim. At last I am doing the deed: I have ditched the modules I listed above and mostly I have what I need except for an issue I raised in Drupal Answers about aligning embedded Media (Images)
Actually 8.8.1 core Media meets my needs without any additional modules. Thanks! I updated the Drupal docs Embedding media with CKEditor with a few details after I resolved my issue re my filtering out HTML incorrectly
Excellent — that’s exactly what the intent and hope was! It’s nice to see that being confirmed!
I guess I’ll just have to install 8.8 dev locally and see what happens. But, quickly … if I have an article content type with an image “embedded” (maybe I should say “inserted”?) with codes like:
<p><img alt="billsuegayfrank.jpg" height="703" src="/drupal8/sites/default/files/styles/large/public/inline- images/billsuegayfrank.jpg"width="938" /></p>For a module such as Juicebox which uses an XML file to set up the layout and options for a photo gallery, contains an images and thumbnails directory for the images and uses a Javascript program to display the Gallery, forcing a transition to media library would seem unnecessary and disruptive.
There is no forced transition to Media Library. Juicebox and any other contributed Drupal module will continue to work as-is. Every site can choose for itself to migrate to Media + Media Library. By default, nothing happens. Precisely for the reason you cite: to avoid disruption!
Congratulations to the whole Media Initiative team, and especially thank you, Wim, for taking the time to communicate these milestone achievements publicly.
Wauw! I was in that session in 2007 as well.
We’ve come a long way, and it sure is great to see this in Core!
A hat tip to all involved, but also a nod of recognition to all to forged their own path.
For instance, we used the venerable Scald Module (https://www.drupal.org/project/scald) integrated with EnterMedia for asset management (https://entermediadb.org/) to power https://news.un.org/ !
I look forward to seeing the same kind of power and usability be available in core.
Wow! What an amazing coincidence! :D
Hi, thank you so much for your amazing work and this blog post! Keyboard accessibility is really so important but often overlooked. I also would like to ask what keys you are using. According to the CKEditor docs, it’s
ALT+F10to enter the toolbar, tab + arrows to navigate between options, space or enter to select/insert … and it all works, but I cannot reproduce the finalEdit imagepart. Would be very greatful if you could give a hint :)Problems with oEmbed content regarding adding remote videos with Media Module. Complicated solutions to a big potential security risk make me question its use, especially since it is a known issue—it ALWAYS makes it a risk without clear instructions as to how to resolve. Why should we have to resolve it in the first place on a core module? It’s unusable with that kind of risk.
Which concrete risks are you referring to?