{
"cells": [
{
"cell_type": "markdown",
"id": "05473d59-5b2e-4511-a2ce-428930f2af24",
"metadata": {},
"source": [
"# Designing a Jupyter Extensions with IPyDrawio"
]
},
{
"cell_type": "markdown",
"id": "c0a849e0-cdfe-4c8f-8417-d232999cc1b4",
"metadata": {},
"source": [
"Diagrams can effectively support the many phases of the software development cycle. What IPyDrawio, and the underlying general-purpose drawio UI, lack in features versus more specialized tools, when used in the right context to help a software team communicate between itself, and sometimes more importantly, its users."
]
},
{
"cell_type": "markdown",
"id": "c6517292-f934-4365-a6e3-3c223f14bca9",
"metadata": {},
"source": [
"```{note}\n",
"This tutorial looks at how different kinds of diagram artifacts can be created that support the process of building _Jupyter Extensions_, using IPyDrawio itself as a subject. Each of the images below rendered directly in the browser for this documentation, as well as editable within IPyDrawio.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "7a5ef077-3d16-4e61-ad47-39b03eb60612",
"metadata": {},
"source": [
"Extensions for [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html) and [Jupyter Server](https://jupyter-server.readthedocs.io/en/latest/developers/extensions.html?highlight=distributing#distributing-a-server-extension) often developed, and can be distributed, together as a single Python package. While a single author can build, test, document and ship new features _without_ any kinds of pictures, individuals and especially team- and community-driven processes can be enhanced by different visual artifacts that _live next to the code_."
]
},
{
"cell_type": "markdown",
"id": "150c79c8-4349-4fde-bc3c-d7dfa1c74d1a",
"metadata": {},
"source": [
"```{hint}\n",
"Using an editable, but widely-viewable format like `.dio.png` or `.dio.svg` means these assets can be used \"at-rest,\" though without advanced features like multiple pages and interactivity, and works well with the visual diff tools on e.g. GitHub.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "8c9e0cd8-e623-4c15-9ca9-43f2f35bca8b",
"metadata": {},
"source": [
"## Gathering Ideas\n",
"\n",
"As you start thinking about your extension, who will use it, and how you might build it, IPyDrawio diagrams can capture structured problem-solving approaches effective at capturing, and then leveraging, visual concepts to capture goals and trade-offs."
]
},
{
"cell_type": "markdown",
"id": "c1e0fe74-ddc5-4892-bf73-827bba9eeb94",
"metadata": {},
"source": [
"### Mind Maps\n",
"[Mind Maps](https://en.wikipedia.org/wiki/Mind_map) are an effective way to capture \"stream of consciousness\"."
]
},
{
"cell_type": "markdown",
"id": "605ed2a3-1a8b-416f-82b1-d1493c986cfe",
"metadata": {},
"source": [
"```{hint}\n",
"Search for `mind` in the _Custom Diagram..._ panel. This task also lends itself to the `sketch` theme, which trades UI simplicity for hiding some of the more advanced features.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "93667624-a6e6-4e58-bb97-3d76c312f353",
"metadata": {},
"source": [
"[![a mind map of ipydrawio][mindmap]][mindmap]\n",
"\n",
"[mindmap]: ../../_static/tutorials/designing-jupyter-extensions/mindmap.dio.svg"
]
},
{
"cell_type": "markdown",
"id": "bec77aa2-80cf-4702-b222-5c40e021b4a1",
"metadata": {},
"source": [
"### Seven Management and Planning Tools\n",
"The [Seven Management and Planning Tools](https://en.wikipedia.org/wiki/Seven_management_and_planning_tools) are suitable for large and small projects.\n",
"\n",
"```{note}\n",
"While there are no \"prebuilt\" templates for these diagrams, many of the \"basic\" and and \"tree\" shape palettes have ready-made shapes with pre-configured connectors that preserve intent, and work well with the auto-layout tools.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "de02a784-aa8c-49a5-a1c2-e321c42912bd",
"metadata": {},
"source": [
"[![an affinity diagram of ipydrawio][affinity]][affinity]\n",
"\n",
"[affinity]: ../../_static/tutorials/designing-jupyter-extensions/affinity.dio.svg"
]
},
{
"cell_type": "markdown",
"id": "3d8c47d4-84dd-4907-95ca-5d8c898c4c6b",
"metadata": {},
"source": [
"```{hint}\n",
"Use the Ctrl (or Cmd on MacOS) with existing arrows to quickly make copies. \n",
"```"
]
},
{
"cell_type": "markdown",
"id": "14fe2c2b-de91-4aee-92d1-179d9754468b",
"metadata": {},
"source": [
"### Mermaid Diagrams\n",
"\n",
"The [mermaid.js](https://mermaid-js.github.io/mermaid) drawio plugin (enabled by default) provides a way to write pithy text descriptions, including:\n",
"\n",
"> _Flowchart, Sequence diagram, Class Diagram, State Diagram, Entity Relationship Diagram, User Journey, Gantt, Pie Chart, Requirement Diagram_"
]
},
{
"cell_type": "markdown",
"id": "e7db97c9-fe37-49f2-b08a-c24b9bdff6b7",
"metadata": {},
"source": [
"```{note}\n",
"From the drawio menu, select _Insert_ ▸ _Advanced_ ▸ _Mermaid_. Paste some mermaid there. The resulting object will _not_ be editable with shape-editing tools, but you _will_ be able to adjust the mermaid syntax later by clicking on it, which can be worth _much_ more over time. \n",
"```"
]
},
{
"cell_type": "markdown",
"id": "eab9baf5-f7c5-4ae3-992c-b682943f9fbf",
"metadata": {},
"source": [
"```{hint}\n",
"Here are some more interactive editing experience than the `textarea` provided by drawio.\n",
"- [jupyterlab-markup](https://github.com/agoose77/jupyterlab-markup)\n",
"- [mermaid live editor](https://mermaid-js.github.io/mermaid-live-editor)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "0e4be3cd-a8d6-41a7-913a-645ef4487a5f",
"metadata": {},
"source": [
"[![mermaid diagrams][mermaid]][mermaid]\n",
"\n",
"[mermaid]: ../../_static/tutorials/designing-jupyter-extensions/mermaid.dio.svg"
]
},
{
"cell_type": "markdown",
"id": "e57bd8c7-6576-4938-a102-1eeaf33b2626",
"metadata": {},
"source": [
"If these kinds of diagrams are helpful for your extension, consider more advanced tools like [Papyrus](https://www.eclipse.org/papyrus) that support more formal models and diagram standards."
]
},
{
"cell_type": "markdown",
"id": "39f30c2f-f937-4c67-98a1-1cc4670f2709",
"metadata": {},
"source": [
"## Visual Design\n",
"\n",
"While IPyDrawio isn't as full-featured as properietary tools, or even open source tools such as the excellent [Inkscape](https://inkscape.org), it can be effective, and in some ways, more approachable, than full design suites."
]
},
{
"cell_type": "markdown",
"id": "4d22eebd-b5b7-4a8f-89a5-977b61852562",
"metadata": {},
"source": [
"### Wireframing\n",
"\n",
"Low-fidelity wireframes, which often include filler text like _lorem ipsum_, placeholder images, etc. provide a quick way to capture the essential views of a novel UI without getting a team hung up on details."
]
},
{
"cell_type": "markdown",
"id": "69f3416e-884b-4292-84cf-59e54e40a592",
"metadata": {},
"source": [
"[![a wireframe of the ipydrawio custom diagram panel][wireframe]][wireframe]\n",
"\n",
"\n",
"[wireframe]: ../../_static/tutorials/designing-jupyter-extensions/wireframe.dio.svg"
]
},
{
"cell_type": "markdown",
"id": "202e9e97-1a90-4c9f-829f-c2fd55c96f13",
"metadata": {},
"source": [
"```{hint}\n",
"Use primarily _basic shapes_: a box with a word in it says \"button\" just as well as something with gradients and carefully-kerned fonts.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "da515eab-71a5-488f-b40a-c361ea26ae71",
"metadata": {},
"source": [
"```{note}\n",
"For single-page/stage views, the `sketch` theme can work well, while `min` provides a nice balance between unobtrusiveness and power.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "a9cede37-b50d-4c49-805f-e904db806715",
"metadata": {},
"source": [
"### Comps\n",
"\n",
"High-fidelity mockups, or \"comps,\" eventually provide the language used between the designer role and the developer role... even if these roles are filled by the same person."
]
},
{
"cell_type": "markdown",
"id": "2ab27c36-9864-4348-b649-d43970a553d5",
"metadata": {},
"source": [
"```{hint}\n",
"Search for `jupyter` in the _Custom Diagram..._ panel. The _JupyterLab Mockups_ contains recreations of a number of common views, such as the _Launcher_ and _Notebook_, which can be recombined, remixed, or expanded to suggest new features. \n",
"```"
]
},
{
"cell_type": "markdown",
"id": "0ab07900-ff38-4502-839f-d2e31efc6129",
"metadata": {},
"source": [
"```{note}\n",
"One of the \"full\" themes (e.g. `kennedy`, `dark` or `atlas`) is recommended, as using multiple pages, cross-page links, and auto-layouts are more readily discoverable than with `min` or `sketch`.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "2ed75f4f-d1ae-4d30-a082-1f7f5c3ce326",
"metadata": {},
"source": [
"[![a comp of a future feature for IPyDrawio][comp]][comp]\n",
"\n",
"[comp]: ../../_static/tutorials/designing-jupyter-extensions/comp.dio.svg"
]
},
{
"cell_type": "markdown",
"id": "98e88667-f9ef-4ae3-8746-348a20b86598",
"metadata": {},
"source": [
"## Testing\n",
"\n",
"While not a _particularly_ accessible semantic model, native drawio documents and SVG _are_ well-behaved XML, and far less opaque than \"professional\" UML/SysML tools. Especially when using some of the richer shapes, such as those in _Trees_, some data will be preserved, and can be used, for example, to validate repository integrity.\n",
"\n",
"\n",
"```{note}\n",
"For example, if you are building an extension composed of many classes, you can use either simple string comparison, regular expressions, or XPath queries (provided by both [browser JS](https://developer.mozilla.org/en-US/docs/Web/XPath) and [python](https://docs.python.org/3/library/xml.etree.elementtree.html#elementtree-xpath)) to verify all classes are included in a key diagram.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "af1ef92c-69cd-4197-ba34-a25dd428f160",
"metadata": {},
"source": [
"[![a deployment diagram of ipydrawio packages][deployment]][deployment]\n",
"\n",
"[deployment]: ../../_static/tutorials/designing-jupyter-extensions/deployment.dio.svg"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9976fbb6-e3fb-43b4-a30d-a0ef1d182bbc",
"metadata": {},
"outputs": [],
"source": [
"import lxml.etree as ET\n",
"from pathlib import Path"
]
},
{
"cell_type": "markdown",
"id": "97be3409-eb2b-4766-a873-e956a8e139b3",
"metadata": {},
"source": [
"Because the sources and targets of the dotted edges are not preserved on the `path` SVG elements, we use the fill colors to fill in the missing data."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "168ffd51-051f-40b2-83f5-c1d3c08a3a8d",
"metadata": {},
"outputs": [],
"source": [
"packages = {\n",
" \"#dae8fc\": \"ipydrawio\",\n",
" \"#f8cecc\": \"ipydrawio_export\",\n",
" \"#d5e8d4\": \"ipydrawio_mathjax\"\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "45a41a7b-3f73-4e74-8c85-9cf2d7d8b012",
"metadata": {},
"outputs": [],
"source": [
"ships = {\n",
" py_pkg: {\n",
" js_pkg[\"dest\"]\n",
" for js_pkg\n",
" in __import__(py_pkg)._jupyter_labextension_paths()\n",
" }\n",
" for fill, py_pkg in packages.items()\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ebbd6b25-bcf2-4047-8c72-adde387ec0ee",
"metadata": {},
"outputs": [],
"source": [
"def test_deployment(\n",
" ns=\"@deathbeds/\",\n",
" diagram=\"../../_static/tutorials/designing-jupyter-extensions/deployment.dio.svg\"\n",
"):\n",
" deployment = Path(diagram)\n",
" et = ET.fromstring(deployment.read_text(encoding=\"utf-8\"))\n",
" # get everything with a data-label, which gets preserved\n",
" should_ship_in = {\n",
" f\"\"\"{ns}{e.attrib[\"data-label\"]}\"\"\":\n",
" packages.get(e.xpath(\"./*[1]/@fill\")[0])\n",
" for e in et.xpath(\"//*[@data-label]\")\n",
" }\n",
" for py_package, ships_exts in ships.items():\n",
" print(py_package)\n",
" print(\"... observed\", sorted(ships_exts))\n",
" should_ship_exts = {\n",
" js_name for js_name, py_name in should_ship_in.items()\n",
" if py_name == py_package\n",
" }\n",
" print(\"... expected\", sorted(should_ship_exts))\n",
" mismatches = ships_exts ^ should_ship_exts\n",
" assert not mismatches, f\"mismatches: {mismatches}\"\n",
" print(f\"... {py_package} is 🚢-shape!\")"
]
},
{
"cell_type": "markdown",
"id": "458ea4b9-1365-48d1-9bf9-7d7aa8d04645",
"metadata": {},
"source": [
"```{note}\n",
"For documentation purposes, we just run this directly here, but this could be easily run as part of a regular unit test suite, and an \"offending\" diagram included in a rich test output such as [pytest-html](https://github.com/pytest-dev/pytest-html).\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "42de7083-e97a-4b52-a647-9dd718d20b08",
"metadata": {},
"outputs": [],
"source": [
"test_deployment()"
]
},
{
"cell_type": "markdown",
"id": "9dd02cf8-959a-4698-9749-34e3422ef9fc",
"metadata": {},
"source": [
"Now, if a new package is added to either the code, or the diagram, and the other side is _not_ updated, this test will fail, helping to keep the documentation up-to-date."
]
},
{
"cell_type": "markdown",
"id": "0178416c-7f47-4ae2-81bf-850e14c8d22a",
"metadata": {},
"source": [
"## Documentation\n",
"\n",
"Once these diagrams exist, they can be used directly within documentation sites. While alternatives exist for working directly with `.drawio` files, such as [sphinxcontrib-drawio](https://pypi.org/project/sphinxcontrib-drawio), again keeping the files in standards-compliant, but editable, formats such as `.dio.png` and `.dio.svg` are more likely to stay editable and viewable over time."
]
},
{
"cell_type": "markdown",
"id": "0165bbc2-700b-47c8-b0b6-fbc9789f2cb5",
"metadata": {},
"source": [
"```{hint}\n",
"[myst-nb](https://github.com/executablebooks/MyST-NB) is highly recommended (and used on this documentation site) for embedding various kinds of images without a lot of the hassle of some of the finer points of `.rst` editing.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "63e1049d-cfa4-49d2-9dea-db554172899a",
"metadata": {},
"source": [
"Another approach is generating diagrams, with tools such as:\n",
"\n",
"- [graphviz2drawio](https://github.com/hbmartin/graphviz2drawio) \n",
"- [n2g](https://github.com/dmulyalin/N2G)\n",
"- [drawio-network-plot](https://github.com/amroashram/drawio_network_plot)\n",
"- [python-drawio](https://github.com/chmduquesne/python-drawio)\n",
"\n",
"These offer an intermediate representation, as code or some other grammar, which can be more effective at keeping documentation up-to-date than hand editing diagrams."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}