XB week 5: chaos theory

We started the week off with a first MVP config entity: the component config entity. We have to start somewhere and then iterate — and iterate we can: thanks to Felix “f.mazeikis”, #3444417 landed! Most crucially this means we now have both a low-level unit test and a kernel test that ensures the validation for this config entity is thorough — of course I’m bringing config validation to Experience Builder (XB) from the very start!

It doesn’t do much: it allows “opting in” single directory components (SDCs) to XB, and now only those are available in the PoC admin UI. Next up is #3452397: expanding the admin UI and the config entity to allow defining default values for its SDC props. This will allow the UI to immediately generate a preview of the component, which otherwise may (depending on the component) not render anything visible (imagine: a <h2> without text, an <img> without an image, etc.).

But literally everything about this config entity is subject to change:

• its name (issue: #3455036)
• the whole “opting in” principle (issue comment: #3444424-7
• what it does exactly (issue: #3446083)
• whether it should handle only exposing SDCs 1:1 in XB, or whether it should also handle other component types (issue: #3454519)
• we may or may not need this to tie existing SDCs to design tokens, but none of that is clearly defined yet either.

How did we get here?

Which brings me to the main subject for this week: I have a theory that explains the chaos that I think now many of you see.

We rushed to open the 0.x branch to invite contribution, because Dries announced XB at DrupalCon. This wrongly gave the impression that there was a very detailed plan, with a precise architecture in mind. That is not the case, and mea culpa for giving that impression. That branch was simply the combination of different PoCs merged together.

Some additional context: back in March, from one day to the next, I along with Ben “bnjmnm”, Ted “tedbow”, Alex “effulgentsia”, Tim and Lauri, plus Felix and Jesse (both from the Site Studio team, both very experienced) I were asked to stop everything we were doing (for me: dropping config validation) and start estimating the product requirements.

~4 weeks of non-stop grueling meetings ¹ to assess and estimate the 64 requirements, with Lauri providing visual examples and additional context for the product requirements he crafted (after careful research: #3452440). We looked at multiple radical options and evaluated which of these was viable:

1. building on top of one of the page building solutions that already exist in the Drupal ecosystem
2. partnering with one of the existing page building systems, if they were willing to relicense under GPL-2+ and we’d be able to fit them onto Drupal’s data modeling strengths
3. building something new (but reusing the parts in the Drupal ecosystem that we can build on top of, such as SDC, field types for data modeling, etc.)

The result: 60 pages worth of notes ², with for each of the 42 “Required for MVP” product requirements (out of a total of 64), there now was an outline of how it could be implemented, a best+realistic+worst case estimate (week-level accuracy), which skillsets needed and how many people. It’s during this process that it became clear that only one choice made sense: the last one. That was the conclusion we reached because the existing solutions in the ecosystem fall short (see #3452440), Site Studio does not meet all of XB’s requirements and some of its architectural choices conflict with XB’s requirements and we did not end up finding a magical unicorn that meshed perfectly with Drupal.

It’s during this time that I made my biggest mistake yet: because the request for estimating this was coming from within Acquia, I assumed this was intended to be built by a team of Acquians. Because … why else create estimates?! If it’s built by a mix of paid full-time, paid part-time and volunteer community members, there’s little point in estimating, right?!
Well, I was wrong: turns out the intent was to get a sense of how feasible it was to achieve in roughly which timeline. But I and my fellow team members were so deep into this enormous estimation exercise based on very high-level requirements that thinking about capturing this information in a more shareable form was a thought that simply did not occur…

So, choice 3 it was, with people that have deep Layout Builder knowledge (Ted & Tim), while bringing in expertise from the Site Studio team (Jesse & Felix) and strong front-end expertise (Ben & Harumi “hooroomoo”) … because next we were asked to narrow the estimates for the highest-variance estimates, to bring it down to a higher degree of confidence. That’s where the “FTEs”, “Best Case Size (weeks)”, “Realistic Case Size (weeks)”, “Worst Case Size (weeks)”, “Most Probable Case Size (calculated, FTE weeks)”, “Variance” columns in the requirements spreadsheet come in.

For the next ~4 weeks, we built PoCs for the requirements where our estimates had either the highest variance or the highest number, to both narrow and reduce the estimates. That’s where all the branches on the experience_builder project come from! For example:

• Harumi’s poc-codemirror for editing component styles (not in 0.x)
• Felix’ research__component_export_poc to verify we could actually transport default file content as part of the regular config import/export system (not in 0.x)
• Ben’s cypress_starter
• Ben’s easyblocks_backend (not in 0.x)
• Jesse’s poc_reactrouter (not in 0.x)
• Ted’s research_sb_config for config-defined SDCs (not in 0.x)
• my research__design_versions (not in 0.x)
• …

Week 1 = witch pot

After the ~8 chaotic weeks above, it was DrupalCon (Dries also wrote about this, just before DrupalCon, based on the above work!), and then … it was week 1.

In that first week, we took a handful of branches that seemed to contain the pieces most sensible to start iterating on an actual implementation, threw that into a witch pot, and called that 0.x!

Now that all of you know the full origin story, and many of you have experienced the subsequent ~4 equally chaotic weeks, you’re all caught up! :D

Going forward: order from chaos

So, how are we going to improve things?

• Tuesday: issue queue 100% triaged — each issue that is for a later phase is now Postponed and each issue that needs input from someone has that person assigned.
• Friday: adopt Architecture Decision Records (ADRs): #3454669) — many thanks to Lullabot for leading the way
• Friday: the first high-level diagrams landed, and thanks to Kyle “ctrlADel”, they follow the C4 model
• More to come in week 6!

Actual progress?

So besides the backstory and attempts to bring more order & overview, what happened?

The UI gained panning support (#3452623), which really makes it feel like it’s coming alive:

… you can also try an imperfect/limited version of the current UI on the statically hosted demo UI thanks to #3450311 by Lee “larowlan”.

We also got two eslint CI jobs running now (#3450307): one for the typical Drupal stuff, one for the XB UI (which is written in TypeScript) and reduced the overhead of PHPStan Level 8 (again thanks to Lee, in #3454501) … and a bunch more low-level things that were improved.

Various things are in progress … expect an update for those next week :)

Thanks to Lauri for reviewing this!

Week 5 was June 10–16, 2024.