{ "cells": [ { "cell_type": "markdown", "id": "05473d59-5b2e-4511-a2ce-428930f2af24", "metadata": {}, "source": [ "# Designing a Jupyter Extensions with IPyDrawio" ] }, { "cell_type": "markdown", "id": "c0a849e0-cdfe-4c8f-8417-d232999cc1b4", "metadata": {}, "source": [ "Diagrams can effectively support the many phases of the software development cycle. What IPyDrawio, and the underlying general-purpose drawio UI, lack in features versus more specialized tools, when used in the right context to help a software team communicate between itself, and sometimes more importantly, its users." ] }, { "cell_type": "markdown", "id": "c6517292-f934-4365-a6e3-3c223f14bca9", "metadata": {}, "source": [ "```{note}\n", "This tutorial looks at how different kinds of diagram artifacts can be created that support the process of building _Jupyter Extensions_, using IPyDrawio itself as a subject. Each of the images below rendered directly in the browser for this documentation, as well as editable within IPyDrawio.\n", "```" ] }, { "cell_type": "markdown", "id": "7a5ef077-3d16-4e61-ad47-39b03eb60612", "metadata": {}, "source": [ "Extensions for [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html) and [Jupyter Server](https://jupyter-server.readthedocs.io/en/latest/developers/extensions.html?highlight=distributing#distributing-a-server-extension) often developed, and can be distributed, together as a single Python package. While a single author can build, test, document and ship new features _without_ any kinds of pictures, individuals and especially team- and community-driven processes can be enhanced by different visual artifacts that _live next to the code_." ] }, { "cell_type": "markdown", "id": "150c79c8-4349-4fde-bc3c-d7dfa1c74d1a", "metadata": {}, "source": [ "```{hint}\n", "Using an editable, but widely-viewable format like `.dio.png` or `.dio.svg` means these assets can be used \"at-rest,\" though without advanced features like multiple pages and interactivity, and works well with the visual diff tools on e.g. GitHub.\n", "```" ] }, { "cell_type": "markdown", "id": "8c9e0cd8-e623-4c15-9ca9-43f2f35bca8b", "metadata": {}, "source": [ "## Gathering Ideas\n", "\n", "As you start thinking about your extension, who will use it, and how you might build it, IPyDrawio diagrams can capture structured problem-solving approaches effective at capturing, and then leveraging, visual concepts to capture goals and trade-offs." ] }, { "cell_type": "markdown", "id": "c1e0fe74-ddc5-4892-bf73-827bba9eeb94", "metadata": {}, "source": [ "### Mind Maps\n", "[Mind Maps](https://en.wikipedia.org/wiki/Mind_map) are an effective way to capture \"stream of consciousness\"." ] }, { "cell_type": "markdown", "id": "605ed2a3-1a8b-416f-82b1-d1493c986cfe", "metadata": {}, "source": [ "```{hint}\n", "Search for `mind` in the _Custom Diagram..._ panel. This task also lends itself to the `sketch` theme, which trades UI simplicity for hiding some of the more advanced features.\n", "```" ] }, { "cell_type": "markdown", "id": "93667624-a6e6-4e58-bb97-3d76c312f353", "metadata": {}, "source": [ "[![a mind map of ipydrawio][mindmap]][mindmap]\n", "\n", "[mindmap]: ../../_static/tutorials/designing-jupyter-extensions/mindmap.dio.svg" ] }, { "cell_type": "markdown", "id": "bec77aa2-80cf-4702-b222-5c40e021b4a1", "metadata": {}, "source": [ "### Seven Management and Planning Tools\n", "The [Seven Management and Planning Tools](https://en.wikipedia.org/wiki/Seven_management_and_planning_tools) are suitable for large and small projects.\n", "\n", "```{note}\n", "While there are no \"prebuilt\" templates for these diagrams, many of the \"basic\" and and \"tree\" shape palettes have ready-made shapes with pre-configured connectors that preserve intent, and work well with the auto-layout tools.\n", "```" ] }, { "cell_type": "markdown", "id": "de02a784-aa8c-49a5-a1c2-e321c42912bd", "metadata": {}, "source": [ "[![an affinity diagram of ipydrawio][affinity]][affinity]\n", "\n", "[affinity]: ../../_static/tutorials/designing-jupyter-extensions/affinity.dio.svg" ] }, { "cell_type": "markdown", "id": "3d8c47d4-84dd-4907-95ca-5d8c898c4c6b", "metadata": {}, "source": [ "```{hint}\n", "Use the Ctrl (or Cmd on MacOS) with existing arrows to quickly make copies. \n", "```" ] }, { "cell_type": "markdown", "id": "14fe2c2b-de91-4aee-92d1-179d9754468b", "metadata": {}, "source": [ "### Mermaid Diagrams\n", "\n", "The [mermaid.js](https://mermaid-js.github.io/mermaid) drawio plugin (enabled by default) provides a way to write pithy text descriptions, including:\n", "\n", "> _Flowchart, Sequence diagram, Class Diagram, State Diagram, Entity Relationship Diagram, User Journey, Gantt, Pie Chart, Requirement Diagram_" ] }, { "cell_type": "markdown", "id": "e7db97c9-fe37-49f2-b08a-c24b9bdff6b7", "metadata": {}, "source": [ "```{note}\n", "From the drawio menu, select _Insert_ ▸ _Advanced_ ▸ _Mermaid_. Paste some mermaid there. The resulting object will _not_ be editable with shape-editing tools, but you _will_ be able to adjust the mermaid syntax later by clicking on it, which can be worth _much_ more over time. \n", "```" ] }, { "cell_type": "markdown", "id": "eab9baf5-f7c5-4ae3-992c-b682943f9fbf", "metadata": {}, "source": [ "```{hint}\n", "Here are some more interactive editing experience than the `textarea` provided by drawio.\n", "- [jupyterlab-markup](https://github.com/agoose77/jupyterlab-markup)\n", "- [mermaid live editor](https://mermaid-js.github.io/mermaid-live-editor)\n", "```" ] }, { "cell_type": "markdown", "id": "0e4be3cd-a8d6-41a7-913a-645ef4487a5f", "metadata": {}, "source": [ "[![mermaid diagrams][mermaid]][mermaid]\n", "\n", "[mermaid]: ../../_static/tutorials/designing-jupyter-extensions/mermaid.dio.svg" ] }, { "cell_type": "markdown", "id": "e57bd8c7-6576-4938-a102-1eeaf33b2626", "metadata": {}, "source": [ "If these kinds of diagrams are helpful for your extension, consider more advanced tools like [Papyrus](https://www.eclipse.org/papyrus) that support more formal models and diagram standards." ] }, { "cell_type": "markdown", "id": "39f30c2f-f937-4c67-98a1-1cc4670f2709", "metadata": {}, "source": [ "## Visual Design\n", "\n", "While IPyDrawio isn't as full-featured as properietary tools, or even open source tools such as the excellent [Inkscape](https://inkscape.org), it can be effective, and in some ways, more approachable, than full design suites." ] }, { "cell_type": "markdown", "id": "4d22eebd-b5b7-4a8f-89a5-977b61852562", "metadata": {}, "source": [ "### Wireframing\n", "\n", "Low-fidelity wireframes, which often include filler text like _lorem ipsum_, placeholder images, etc. provide a quick way to capture the essential views of a novel UI without getting a team hung up on details." ] }, { "cell_type": "markdown", "id": "69f3416e-884b-4292-84cf-59e54e40a592", "metadata": {}, "source": [ "[![a wireframe of the ipydrawio custom diagram panel][wireframe]][wireframe]\n", "\n", "\n", "[wireframe]: ../../_static/tutorials/designing-jupyter-extensions/wireframe.dio.svg" ] }, { "cell_type": "markdown", "id": "202e9e97-1a90-4c9f-829f-c2fd55c96f13", "metadata": {}, "source": [ "```{hint}\n", "Use primarily _basic shapes_: a box with a word in it says \"button\" just as well as something with gradients and carefully-kerned fonts.\n", "```" ] }, { "cell_type": "markdown", "id": "da515eab-71a5-488f-b40a-c361ea26ae71", "metadata": {}, "source": [ "```{note}\n", "For single-page/stage views, the `sketch` theme can work well, while `min` provides a nice balance between unobtrusiveness and power.\n", "```" ] }, { "cell_type": "markdown", "id": "a9cede37-b50d-4c49-805f-e904db806715", "metadata": {}, "source": [ "### Comps\n", "\n", "High-fidelity mockups, or \"comps,\" eventually provide the language used between the designer role and the developer role... even if these roles are filled by the same person." ] }, { "cell_type": "markdown", "id": "2ab27c36-9864-4348-b649-d43970a553d5", "metadata": {}, "source": [ "```{hint}\n", "Search for `jupyter` in the _Custom Diagram..._ panel. The _JupyterLab Mockups_ contains recreations of a number of common views, such as the _Launcher_ and _Notebook_, which can be recombined, remixed, or expanded to suggest new features. \n", "```" ] }, { "cell_type": "markdown", "id": "0ab07900-ff38-4502-839f-d2e31efc6129", "metadata": {}, "source": [ "```{note}\n", "One of the \"full\" themes (e.g. `kennedy`, `dark` or `atlas`) is recommended, as using multiple pages, cross-page links, and auto-layouts are more readily discoverable than with `min` or `sketch`.\n", "```" ] }, { "cell_type": "markdown", "id": "2ed75f4f-d1ae-4d30-a082-1f7f5c3ce326", "metadata": {}, "source": [ "[![a comp of a future feature for IPyDrawio][comp]][comp]\n", "\n", "[comp]: ../../_static/tutorials/designing-jupyter-extensions/comp.dio.svg" ] }, { "cell_type": "markdown", "id": "98e88667-f9ef-4ae3-8746-348a20b86598", "metadata": {}, "source": [ "## Testing\n", "\n", "While not a _particularly_ accessible semantic model, native drawio documents and SVG _are_ well-behaved XML, and far less opaque than \"professional\" UML/SysML tools. Especially when using some of the richer shapes, such as those in _Trees_, some data will be preserved, and can be used, for example, to validate repository integrity.\n", "\n", "\n", "```{note}\n", "For example, if you are building an extension composed of many classes, you can use either simple string comparison, regular expressions, or XPath queries (provided by both [browser JS](https://developer.mozilla.org/en-US/docs/Web/XPath) and [python](https://docs.python.org/3/library/xml.etree.elementtree.html#elementtree-xpath)) to verify all classes are included in a key diagram.\n", "```" ] }, { "cell_type": "markdown", "id": "af1ef92c-69cd-4197-ba34-a25dd428f160", "metadata": {}, "source": [ "[![a deployment diagram of ipydrawio packages][deployment]][deployment]\n", "\n", "[deployment]: ../../_static/tutorials/designing-jupyter-extensions/deployment.dio.svg" ] }, { "cell_type": "code", "execution_count": null, "id": "9976fbb6-e3fb-43b4-a30d-a0ef1d182bbc", "metadata": {}, "outputs": [], "source": [ "import lxml.etree as ET\n", "from pathlib import Path" ] }, { "cell_type": "markdown", "id": "97be3409-eb2b-4766-a873-e956a8e139b3", "metadata": {}, "source": [ "Because the sources and targets of the dotted edges are not preserved on the `path` SVG elements, we use the fill colors to fill in the missing data." ] }, { "cell_type": "code", "execution_count": null, "id": "168ffd51-051f-40b2-83f5-c1d3c08a3a8d", "metadata": {}, "outputs": [], "source": [ "packages = {\n", " \"#dae8fc\": \"ipydrawio\",\n", " \"#f8cecc\": \"ipydrawio_export\",\n", " \"#d5e8d4\": \"ipydrawio_mathjax\"\n", "}" ] }, { "cell_type": "code", "execution_count": null, "id": "45a41a7b-3f73-4e74-8c85-9cf2d7d8b012", "metadata": {}, "outputs": [], "source": [ "ships = {\n", " py_pkg: {\n", " js_pkg[\"dest\"]\n", " for js_pkg\n", " in __import__(py_pkg)._jupyter_labextension_paths()\n", " }\n", " for fill, py_pkg in packages.items()\n", "}" ] }, { "cell_type": "code", "execution_count": null, "id": "ebbd6b25-bcf2-4047-8c72-adde387ec0ee", "metadata": {}, "outputs": [], "source": [ "def test_deployment(\n", " ns=\"@deathbeds/\",\n", " diagram=\"../../_static/tutorials/designing-jupyter-extensions/deployment.dio.svg\"\n", "):\n", " deployment = Path(diagram)\n", " et = ET.fromstring(deployment.read_text(encoding=\"utf-8\"))\n", " # get everything with a data-label, which gets preserved\n", " should_ship_in = {\n", " f\"\"\"{ns}{e.attrib[\"data-label\"]}\"\"\":\n", " packages.get(e.xpath(\"./*[1]/@fill\")[0])\n", " for e in et.xpath(\"//*[@data-label]\")\n", " }\n", " for py_package, ships_exts in ships.items():\n", " print(py_package)\n", " print(\"... observed\", sorted(ships_exts))\n", " should_ship_exts = {\n", " js_name for js_name, py_name in should_ship_in.items()\n", " if py_name == py_package\n", " }\n", " print(\"... expected\", sorted(should_ship_exts))\n", " mismatches = ships_exts ^ should_ship_exts\n", " assert not mismatches, f\"mismatches: {mismatches}\"\n", " print(f\"... {py_package} is 🚢-shape!\")" ] }, { "cell_type": "markdown", "id": "458ea4b9-1365-48d1-9bf9-7d7aa8d04645", "metadata": {}, "source": [ "```{note}\n", "For documentation purposes, we just run this directly here, but this could be easily run as part of a regular unit test suite, and an \"offending\" diagram included in a rich test output such as [pytest-html](https://github.com/pytest-dev/pytest-html).\n", "```" ] }, { "cell_type": "code", "execution_count": null, "id": "42de7083-e97a-4b52-a647-9dd718d20b08", "metadata": {}, "outputs": [], "source": [ "test_deployment()" ] }, { "cell_type": "markdown", "id": "9dd02cf8-959a-4698-9749-34e3422ef9fc", "metadata": {}, "source": [ "Now, if a new package is added to either the code, or the diagram, and the other side is _not_ updated, this test will fail, helping to keep the documentation up-to-date." ] }, { "cell_type": "markdown", "id": "0178416c-7f47-4ae2-81bf-850e14c8d22a", "metadata": {}, "source": [ "## Documentation\n", "\n", "Once these diagrams exist, they can be used directly within documentation sites. While alternatives exist for working directly with `.drawio` files, such as [sphinxcontrib-drawio](https://pypi.org/project/sphinxcontrib-drawio), again keeping the files in standards-compliant, but editable, formats such as `.dio.png` and `.dio.svg` are more likely to stay editable and viewable over time." ] }, { "cell_type": "markdown", "id": "0165bbc2-700b-47c8-b0b6-fbc9789f2cb5", "metadata": {}, "source": [ "```{hint}\n", "[myst-nb](https://github.com/executablebooks/MyST-NB) is highly recommended (and used on this documentation site) for embedding various kinds of images without a lot of the hassle of some of the finer points of `.rst` editing.\n", "```" ] }, { "cell_type": "markdown", "id": "63e1049d-cfa4-49d2-9dea-db554172899a", "metadata": {}, "source": [ "Another approach is generating diagrams, with tools such as:\n", "\n", "- [graphviz2drawio](https://github.com/hbmartin/graphviz2drawio) \n", "- [n2g](https://github.com/dmulyalin/N2G)\n", "- [drawio-network-plot](https://github.com/amroashram/drawio_network_plot)\n", "- [python-drawio](https://github.com/chmduquesne/python-drawio)\n", "\n", "These offer an intermediate representation, as code or some other grammar, which can be more effective at keeping documentation up-to-date than hand editing diagrams." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.0" } }, "nbformat": 4, "nbformat_minor": 5 }