{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "05473d59-5b2e-4511-a2ce-428930f2af24",
   "metadata": {},
   "source": [
    "# Designing a Jupyter Extensions with IPyDrawio"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0a849e0-cdfe-4c8f-8417-d232999cc1b4",
   "metadata": {},
   "source": [
    "Diagrams can effectively support the many phases of the software development cycle. What IPyDrawio, and the underlying general-purpose drawio UI, lack in features versus more specialized tools, when used in the right context to help a software team communicate between itself, and sometimes more importantly, its users."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6517292-f934-4365-a6e3-3c223f14bca9",
   "metadata": {},
   "source": [
    "```{note}\n",
    "This tutorial looks at how different kinds of diagram artifacts can be created that support the process of building _Jupyter Extensions_, using IPyDrawio itself as a subject. Each of the images below renders directly in the browser for this documentation, and ais editable within IPyDrawio.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a5ef077-3d16-4e61-ad47-39b03eb60612",
   "metadata": {},
   "source": [
    "Extensions for [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/extension/extension_dev.html) and [Jupyter Server](https://jupyter-server.readthedocs.io/en/latest/developers/extensions.html?highlight=distributing#distributing-a-server-extension) are often developed, and can be distributed, together as a single Python package. While a single author can build, test, document, and ship new features _without_ any kinds of pictures, individual, and especially team- and community-driven, processes can be enhanced by different visual artifacts that _live next to the code_."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "150c79c8-4349-4fde-bc3c-d7dfa1c74d1a",
   "metadata": {},
   "source": [
    "```{hint}\n",
    "Using an editable, but widely-viewable format like `.dio.png` or `.dio.svg` means these assets can be used \"at-rest,\" though without advanced features like multiple pages and interactivity, and works well with the visual diff tools on e.g. GitHub.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c9e0cd8-e623-4c15-9ca9-43f2f35bca8b",
   "metadata": {},
   "source": [
    "## Gathering Ideas\n",
    "\n",
    "As you start thinking about your extension, who will use it, and how you might build it, IPyDrawio diagrams can enable structured problem-solving approaches effective at capturing, and then leveraging, visual concepts to capture goals and trade-offs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1e0fe74-ddc5-4892-bf73-827bba9eeb94",
   "metadata": {},
   "source": [
    "### Mind Maps\n",
    "[Mind Maps](https://en.wikipedia.org/wiki/Mind_map) are an effective way to capture \"stream of consciousness\"."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "605ed2a3-1a8b-416f-82b1-d1493c986cfe",
   "metadata": {},
   "source": [
    "```{hint}\n",
    "Search for `mind` in the _Custom Diagram..._ panel. This task also lends itself to the `sketch` theme, which trades UI simplicity for hiding some of the more advanced features.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93667624-a6e6-4e58-bb97-3d76c312f353",
   "metadata": {},
   "source": [
    "[![a mind map of ipydrawio][mindmap]][mindmap]\n",
    "\n",
    "[mindmap]: ../../_static/tutorials/designing-jupyter-extensions/mindmap.dio.svg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4201146e-6199-4841-92af-b97ac8b70e43",
   "metadata": {},
   "source": [
    "```{note}\n",
    "Newer versions of MermaidJS also [text-based mind maps](https://mermaid.js.org/syntax/mindmap.html). These are not yet available on many places where other MermaidJS diagrams work, such as GitHub.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bec77aa2-80cf-4702-b222-5c40e021b4a1",
   "metadata": {},
   "source": [
    "### Seven Management and Planning Tools\n",
    "The [Seven Management and Planning Tools](https://en.wikipedia.org/wiki/Seven_management_and_planning_tools) are suitable for large and small projects.\n",
    "\n",
    "```{note}\n",
    "While there are no \"prebuilt\" templates for these diagrams, many of the \"basic\" and and \"tree\" shape palettes have ready-made shapes with pre-configured connectors that preserve intent, and work well with the auto-layout tools.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de02a784-aa8c-49a5-a1c2-e321c42912bd",
   "metadata": {},
   "source": [
    "[![an affinity diagram of ipydrawio][affinity]][affinity]\n",
    "\n",
    "[affinity]: ../../_static/tutorials/designing-jupyter-extensions/affinity.dio.svg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d8c47d4-84dd-4907-95ca-5d8c898c4c6b",
   "metadata": {},
   "source": [
    "```{hint}\n",
    "Use the <kbd>Ctrl</kbd> (or <kbd>Cmd</kbd> on MacOS) with existing arrows to quickly make copies.  \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14fe2c2b-de91-4aee-92d1-179d9754468b",
   "metadata": {},
   "source": [
    "### MermaidJS Diagrams\n",
    "\n",
    "The [mermaid.js](https://mermaid.js.org) drawio plugin (enabled by default) provides a way to write pithy text descriptions, including:\n",
    "\n",
    "> _Flowchart, Sequence diagram, Class Diagram, State Diagram, Entity Relationship Diagram, User Journey, Gantt, Pie Chart, Requirement Diagram, Mindmap, Timeline_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7db97c9-fe37-49f2-b08a-c24b9bdff6b7",
   "metadata": {},
   "source": [
    "```{note}\n",
    "From the _drawio_ menu, select _Insert_ ▸ _Advanced_ ▸ _Mermaid_. Paste some mermaid there. The resulting object will _not_ be editable with shape-editing tools, but you _will_ be able to adjust the mermaid syntax later by clicking on it, which can be worth _much_ more over time. \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eab9baf5-f7c5-4ae3-992c-b682943f9fbf",
   "metadata": {},
   "source": [
    "```{hint}\n",
    "Here are some more interactive editing experience than the `textarea` provided by drawio.\n",
    "- [jupyterlab-markup](https://github.com/agoose77/jupyterlab-markup)\n",
    "- [mermaid live editor](https://mermaid-js.github.io/mermaid-live-editor)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e4be3cd-a8d6-41a7-913a-645ef4487a5f",
   "metadata": {},
   "source": [
    "[![mermaid diagrams][mermaid]][mermaid]\n",
    "\n",
    "[mermaid]: ../../_static/tutorials/designing-jupyter-extensions/mermaid.dio.svg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e57bd8c7-6576-4938-a102-1eeaf33b2626",
   "metadata": {},
   "source": [
    "If these kinds of diagrams are helpful for your extension, consider more advanced tools like [Papyrus](https://www.eclipse.org/papyrus) that support more formal models and diagram standards."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39f30c2f-f937-4c67-98a1-1cc4670f2709",
   "metadata": {},
   "source": [
    "## Visual Design\n",
    "\n",
    "While IPyDrawio isn't as full-featured as properietary tools, or even open source tools such as the excellent [Inkscape](https://inkscape.org), it can be effective, and in some ways, more approachable, than full design suites."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d22eebd-b5b7-4a8f-89a5-977b61852562",
   "metadata": {},
   "source": [
    "### Wireframing\n",
    "\n",
    "Low-fidelity wireframes, which often include filler text like _lorem ipsum_, placeholder images, etc. provide a quick way to capture the essential views of a novel UI without getting a team hung up on details."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69f3416e-884b-4292-84cf-59e54e40a592",
   "metadata": {},
   "source": [
    "[![a wireframe of the ipydrawio custom diagram panel][wireframe]][wireframe]\n",
    "\n",
    "\n",
    "[wireframe]: ../../_static/tutorials/designing-jupyter-extensions/wireframe.dio.svg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "202e9e97-1a90-4c9f-829f-c2fd55c96f13",
   "metadata": {},
   "source": [
    "```{hint}\n",
    "Use primarily _basic shapes_: a box with a word in it says \"button\" just as well as something with gradients and carefully-kerned fonts.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da515eab-71a5-488f-b40a-c361ea26ae71",
   "metadata": {},
   "source": [
    "```{note}\n",
    "For single-page/stage views, the `sketch` theme can work well, while `min` provides a nice balance between unobtrusiveness and power.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a9cede37-b50d-4c49-805f-e904db806715",
   "metadata": {},
   "source": [
    "### Comps\n",
    "\n",
    "High-fidelity mockups, also known as \"composites\" or \"comps,\" eventually provide the language used between the designer role and the developer role... even if these roles are filled by the same person."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ab27c36-9864-4348-b649-d43970a553d5",
   "metadata": {},
   "source": [
    "```{hint}\n",
    "Search for `jupyter` in the _Custom Diagram..._ panel. The _JupyterLab Mockups_ contains recreations of a number of common views, such as the _Launcher_ and _Notebook_, which can be recombined, remixed, or expanded to suggest new features. \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ab07900-ff38-4502-839f-d2e31efc6129",
   "metadata": {},
   "source": [
    "```{note}\n",
    "One of the \"full\" themes (e.g. `kennedy`, `dark` or `atlas`) is recommended, as using multiple pages, cross-page links, and auto-layouts are more readily discoverable than with `min` or `sketch`.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ed75f4f-d1ae-4d30-a082-1f7f5c3ce326",
   "metadata": {},
   "source": [
    "[![a comp of a future feature for IPyDrawio][comp]][comp]\n",
    "\n",
    "[comp]: ../../_static/tutorials/designing-jupyter-extensions/comp.dio.svg"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98e88667-f9ef-4ae3-8746-348a20b86598",
   "metadata": {},
   "source": [
    "## Testing\n",
    "\n",
    "While not a _particularly_ accessible semantic model, native drawio documents and SVG _are_ well-behaved XML, and far less opaque than \"professional\" UML/SysML tools. Especially when using some of the richer shapes, such as those in _Trees_, some data will be preserved, and can be used, for example, to validate repository integrity.\n",
    "\n",
    "\n",
    "```{note}\n",
    "For example, if you are building an extension composed of many classes, you can use either simple string comparison, regular expressions, or XPath queries (provided by both [browser JS](https://developer.mozilla.org/en-US/docs/Web/XPath) and [python](https://docs.python.org/3/library/xml.etree.elementtree.html#elementtree-xpath)) to verify all classes are included in a key diagram.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af1ef92c-69cd-4197-ba34-a25dd428f160",
   "metadata": {},
   "source": [
    "[![a deployment diagram of ipydrawio packages][deployment]][deployment]\n",
    "\n",
    "[deployment]: ../../_static/tutorials/designing-jupyter-extensions/deployment.dio.svg"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9976fbb6-e3fb-43b4-a30d-a0ef1d182bbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import lxml.etree as ET"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97be3409-eb2b-4766-a873-e956a8e139b3",
   "metadata": {},
   "source": [
    "Because the sources and targets of the dotted edges are not preserved on the `path` SVG elements, we use the fill colors to fill in the missing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "168ffd51-051f-40b2-83f5-c1d3c08a3a8d",
   "metadata": {},
   "outputs": [],
   "source": [
    "packages = {\n",
    "    \"#dae8fc\": \"ipydrawio\",\n",
    "    \"#f8cecc\": \"ipydrawio_export\",\n",
    "    \"#d5e8d4\": \"ipydrawio_mathjax\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45a41a7b-3f73-4e74-8c85-9cf2d7d8b012",
   "metadata": {},
   "outputs": [],
   "source": [
    "ships = {\n",
    "    py_pkg: {\n",
    "        js_pkg[\"dest\"] for js_pkg in __import__(py_pkg)._jupyter_labextension_paths()\n",
    "    }\n",
    "    for fill, py_pkg in packages.items()\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ebbd6b25-bcf2-4047-8c72-adde387ec0ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "def test_deployment(\n",
    "    ns=\"@deathbeds/\",\n",
    "    diagram=\"../../_static/tutorials/designing-jupyter-extensions/deployment.dio.svg\",\n",
    "):\n",
    "    deployment = Path(diagram)\n",
    "    et = ET.fromstring(deployment.read_text(encoding=\"utf-8\"))\n",
    "    # get everything with a data-label, which gets preserved\n",
    "    should_ship_in = {\n",
    "        f\"\"\"{ns}{e.attrib[\"data-label\"]}\"\"\": packages.get(e.xpath(\"./*[1]/@fill\")[0])\n",
    "        for e in et.xpath(\"//*[@data-label]\")\n",
    "    }\n",
    "    for py_package, ships_exts in ships.items():\n",
    "        print(py_package)\n",
    "        print(\"... observed\", sorted(ships_exts))\n",
    "        should_ship_exts = {\n",
    "            js_name\n",
    "            for js_name, py_name in should_ship_in.items()\n",
    "            if py_name == py_package\n",
    "        }\n",
    "        print(\"... expected\", sorted(should_ship_exts))\n",
    "        mismatches = ships_exts ^ should_ship_exts\n",
    "        assert not mismatches, f\"mismatches: {mismatches}\"\n",
    "        print(f\"... {py_package} is 🚢-shape!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "458ea4b9-1365-48d1-9bf9-7d7aa8d04645",
   "metadata": {},
   "source": [
    "```{note}\n",
    "For documentation purposes, we just run this directly here, but this could be easily run as part of a regular unit test suite, and an \"offending\" diagram included in a rich test output such as [pytest-html](https://github.com/pytest-dev/pytest-html).\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42de7083-e97a-4b52-a647-9dd718d20b08",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_deployment()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9dd02cf8-959a-4698-9749-34e3422ef9fc",
   "metadata": {},
   "source": [
    "Now, if a new package is added to either the code, or the diagram, and the other side is _not_ updated, this test will fail, helping to keep the documentation up-to-date."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0178416c-7f47-4ae2-81bf-850e14c8d22a",
   "metadata": {},
   "source": [
    "## Documentation\n",
    "\n",
    "Once these diagrams exist, they can be used directly within documentation sites. While alternatives exist for working directly with `.drawio` files, such as [sphinxcontrib-drawio](https://pypi.org/project/sphinxcontrib-drawio), again keeping the files in standards-compliant, but editable, formats such as `.dio.png` and `.dio.svg` are more likely to stay editable and viewable over time."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0165bbc2-700b-47c8-b0b6-fbc9789f2cb5",
   "metadata": {},
   "source": [
    "```{hint}\n",
    "[myst-nb](https://github.com/executablebooks/MyST-NB) is highly recommended (and used on this documentation site) for embedding various kinds of images without a lot of the hassle of some of the finer points of `.rst` editing.\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63e1049d-cfa4-49d2-9dea-db554172899a",
   "metadata": {},
   "source": [
    "Another approach is generating diagrams, with tools such as:\n",
    "\n",
    "- [graphviz2drawio](https://github.com/hbmartin/graphviz2drawio) \n",
    "- [n2g](https://github.com/dmulyalin/N2G)\n",
    "- [drawio-network-plot](https://github.com/amroashram/drawio_network_plot)\n",
    "- [python-drawio](https://github.com/chmduquesne/python-drawio)\n",
    "\n",
    "These offer an intermediate representation, as code or some other grammar, which can be more effective at keeping documentation up-to-date than hand editing diagrams."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}