Designing a Jupyter Extensions with IPyDrawio#

Diagrams can effectively support the many phases of the software development cycle. What IPyDrawio, and the underlying general-purpose drawio UI, lack in features versus more specialized tools, when used in the right context to help a software team communicate between itself, and sometimes more importantly, its users.

Note

This tutorial looks at how different kinds of diagram artifacts can be created that support the process of building Jupyter Extensions, using IPyDrawio itself as a subject. Each of the images below renders directly in the browser for this documentation, and ais editable within IPyDrawio.

Extensions for JupyterLab and Jupyter Server are often developed, and can be distributed, together as a single Python package. While a single author can build, test, document, and ship new features without any kinds of pictures, individual, and especially team- and community-driven, processes can be enhanced by different visual artifacts that live next to the code.

Hint

Using an editable, but widely-viewable format like .dio.png or .dio.svg means these assets can be used “at-rest,” though without advanced features like multiple pages and interactivity, and works well with the visual diff tools on e.g. GitHub.

Gathering Ideas#

As you start thinking about your extension, who will use it, and how you might build it, IPyDrawio diagrams can enable structured problem-solving approaches effective at capturing, and then leveraging, visual concepts to capture goals and trade-offs.

Mind Maps#

Mind Maps are an effective way to capture “stream of consciousness”.

Hint

Search for mind in the Custom Diagram… panel. This task also lends itself to the sketch theme, which trades UI simplicity for hiding some of the more advanced features.

a mind map of ipydrawio

Note

Newer versions of MermaidJS also text-based mind maps. These are not yet available on many places where other MermaidJS diagrams work, such as GitHub.

Seven Management and Planning Tools#

The Seven Management and Planning Tools are suitable for large and small projects.

Note

While there are no “prebuilt” templates for these diagrams, many of the “basic” and and “tree” shape palettes have ready-made shapes with pre-configured connectors that preserve intent, and work well with the auto-layout tools.

an affinity diagram of ipydrawio

Hint

Use the Ctrl (or Cmd on MacOS) with existing arrows to quickly make copies.

MermaidJS Diagrams#

The mermaid.js drawio plugin (enabled by default) provides a way to write pithy text descriptions, including:

Flowchart, Sequence diagram, Class Diagram, State Diagram, Entity Relationship Diagram, User Journey, Gantt, Pie Chart, Requirement Diagram, Mindmap, Timeline

Note

From the drawio menu, select InsertAdvancedMermaid. Paste some mermaid there. The resulting object will not be editable with shape-editing tools, but you will be able to adjust the mermaid syntax later by clicking on it, which can be worth much more over time.

Hint

Here are some more interactive editing experience than the textarea provided by drawio.

mermaid diagrams

If these kinds of diagrams are helpful for your extension, consider more advanced tools like Papyrus that support more formal models and diagram standards.

Visual Design#

While IPyDrawio isn’t as full-featured as properietary tools, or even open source tools such as the excellent Inkscape, it can be effective, and in some ways, more approachable, than full design suites.

Wireframing#

Low-fidelity wireframes, which often include filler text like lorem ipsum, placeholder images, etc. provide a quick way to capture the essential views of a novel UI without getting a team hung up on details.

a wireframe of the ipydrawio custom diagram panel

Hint

Use primarily basic shapes: a box with a word in it says “button” just as well as something with gradients and carefully-kerned fonts.

Note

For single-page/stage views, the sketch theme can work well, while min provides a nice balance between unobtrusiveness and power.

Comps#

High-fidelity mockups, also known as “composites” or “comps,” eventually provide the language used between the designer role and the developer role… even if these roles are filled by the same person.

Hint

Search for jupyter in the Custom Diagram… panel. The JupyterLab Mockups contains recreations of a number of common views, such as the Launcher and Notebook, which can be recombined, remixed, or expanded to suggest new features.

Note

One of the “full” themes (e.g. kennedy, dark or atlas) is recommended, as using multiple pages, cross-page links, and auto-layouts are more readily discoverable than with min or sketch.

a comp of a future feature for IPyDrawio

Testing#

While not a particularly accessible semantic model, native drawio documents and SVG are well-behaved XML, and far less opaque than “professional” UML/SysML tools. Especially when using some of the richer shapes, such as those in Trees, some data will be preserved, and can be used, for example, to validate repository integrity.

Note

For example, if you are building an extension composed of many classes, you can use either simple string comparison, regular expressions, or XPath queries (provided by both browser JS and python) to verify all classes are included in a key diagram.

a deployment diagram of ipydrawio packages

from pathlib import Path

import lxml.etree as ET

Because the sources and targets of the dotted edges are not preserved on the path SVG elements, we use the fill colors to fill in the missing data.

packages = {
    "#dae8fc": "ipydrawio",
    "#f8cecc": "ipydrawio_export",
    "#d5e8d4": "ipydrawio_mathjax",
}
ships = {
    py_pkg: {
        js_pkg["dest"] for js_pkg in __import__(py_pkg)._jupyter_labextension_paths()
    }
    for fill, py_pkg in packages.items()
}
def test_deployment(
    ns="@deathbeds/",
    diagram="../../_static/tutorials/designing-jupyter-extensions/deployment.dio.svg",
):
    deployment = Path(diagram)
    et = ET.fromstring(deployment.read_text(encoding="utf-8"))
    # get everything with a data-label, which gets preserved
    should_ship_in = {
        f"""{ns}{e.attrib["data-label"]}""": packages.get(e.xpath("./*[1]/@fill")[0])
        for e in et.xpath("//*[@data-label]")
    }
    for py_package, ships_exts in ships.items():
        print(py_package)
        print("... observed", sorted(ships_exts))
        should_ship_exts = {
            js_name
            for js_name, py_name in should_ship_in.items()
            if py_name == py_package
        }
        print("... expected", sorted(should_ship_exts))
        mismatches = ships_exts ^ should_ship_exts
        assert not mismatches, f"mismatches: {mismatches}"
        print(f"... {py_package} is 🚢-shape!")

Note

For documentation purposes, we just run this directly here, but this could be easily run as part of a regular unit test suite, and an “offending” diagram included in a rich test output such as pytest-html.

test_deployment()
ipydrawio
... observed ['@deathbeds/ipydrawio', '@deathbeds/ipydrawio-jupyter-templates', '@deathbeds/ipydrawio-notebook', '@deathbeds/ipydrawio-webpack']
... expected ['@deathbeds/ipydrawio', '@deathbeds/ipydrawio-jupyter-templates', '@deathbeds/ipydrawio-notebook', '@deathbeds/ipydrawio-webpack']
... ipydrawio is 🚢-shape!
ipydrawio_export
... observed ['@deathbeds/ipydrawio-pdf']
... expected ['@deathbeds/ipydrawio-pdf']
... ipydrawio_export is 🚢-shape!
ipydrawio_mathjax
... observed ['@deathbeds/ipydrawio-mathjax']
... expected ['@deathbeds/ipydrawio-mathjax']
... ipydrawio_mathjax is 🚢-shape!

Now, if a new package is added to either the code, or the diagram, and the other side is not updated, this test will fail, helping to keep the documentation up-to-date.

Documentation#

Once these diagrams exist, they can be used directly within documentation sites. While alternatives exist for working directly with .drawio files, such as sphinxcontrib-drawio, again keeping the files in standards-compliant, but editable, formats such as .dio.png and .dio.svg are more likely to stay editable and viewable over time.

Hint

myst-nb is highly recommended (and used on this documentation site) for embedding various kinds of images without a lot of the hassle of some of the finer points of .rst editing.

Another approach is generating diagrams, with tools such as:

These offer an intermediate representation, as code or some other grammar, which can be more effective at keeping documentation up-to-date than hand editing diagrams.