Working with Custom Shape Libraries#

When working with the ipydrawio TypeScript API, custom shape libraries can be added by providing a fully-resolved, absolute URL to an .xml file in clibs.

From the Widget API, this is somewhat more complicated, but might be worth it for certain custom cases such as issue #80, where a fully-kernel-driven solution is desirable, despite the gotchas (see below).

import base64
import json
import urllib.parse
import zlib

import ipydrawio

A Library#

A library is, at its core, a list of shape descriptions. The best way to learn more about these is investigating the drawio documentation, which covers building them interactively. When done, you’d end up with some data like this.

library = [
    {
        "w": 80,
        "h": 80,
        "aspect": "fixed",
        "title": "Source",
        "xml": '<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Source" interArrivalTime="" id="2"><mxCell style="rhombus;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>',
    },
    {
        "w": 80,
        "h": 80,
        "aspect": "fixed",
        "title": "Queue",
        "xml": '<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Queue" capacity="" id="2"><mxCell style="ellipse;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>',
    },
    {
        "w": 80,
        "h": 80,
        "aspect": "fixed",
        "title": "Machine",
        "xml": '<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Machine" processingTime="" id="2"><mxCell style="whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>',
    },
    {
        "w": 80,
        "h": 80,
        "aspect": "fixed",
        "title": "Exit",
        "xml": '<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="" type="Exit" id="2"><mxCell style="rhombus;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry width="80" height="80" as="geometry"/></mxCell></object></root></mxGraphModel>',
    },
]

JSON Encoding#

Each element in the list has an xml attribute which is, as it suggests, XML, and must be carefully escaped. Because this will go through a URL parser, an XML parser, a JSON parser, and then another XML parser, the recommended approach is to use drawio’s semi-convoluted base64/zlib technique.

zlib_opts = {"wbits": -15}


def inflate(deflated):
    infl = zlib.decompressobj(**zlib_opts)
    return urllib.parse.unquote(
        infl.decompress(base64.b64decode(deflated)) + infl.flush(),
    )


def deflate(inflated):
    defl = zlib.compressobj(**zlib_opts)
    return base64.b64encode(
        defl.compress(urllib.parse.quote(inflated).encode("utf-8")) + defl.flush(),
    ).decode("utf-8")

Again, due to the number of parsers involved, it’s best to avoid extra spaces in the data URI.

library_json = json.dumps(
    [dict(shape.items(), xml=deflate(shape["xml"])) for shape in library],
    separators=(",", ":"),
)
library_json

'[{"w":80,"h":80,"aspect":"fixed","title":"Source","xml":"dVFLEoIwDD1N13y6cavguHKlFyiQoXVay4QocHsLAQUdF5nJS97LV8jM9SdUjT77CqyQRyEz9J7Yc30G1oo0NpWQuUjTOFj0J5cEC7BRCHf6ofviBiWFvFXF2Cif2KOAhgYYX/wDS+CouRPgHtE8lb0aB2vF0nGC22FaGuzMRe1d8WiFPHTaEFwaVY6ZLmwbYprcOEUSXC76BCTo/66SrHqdwDsgHAKlMxVpZuxiVmkwtaZtTLWM67dyuUs0j85gvhKDzx+irye9AA=="},{"w":80,"h":80,"aspect":"fixed","title":"Queue","xml":"dVG7DoMwDPyazLyWzoWKqUPVLwhgkVShiYIp8Pc1GFpoxRDJdz77bEckaTPkXjp1tRUYkVxEknprkaNmSMEYEYe6Ekkm4jikFxzkInoEnfTwxD+5LR5QIuWNLCajbFZPBTg6YHzroAMmS+lkqXHcClejGe5naHE0SxMitGsJnHulEe7UZ8r0tCRxCpvJPKKQm77AIwyHG0QbrxxsA+hHkvS6QsWKU8hVCnStcM/JlnH9qVzPESyjM1iOw+B7/uDnb94="},{"w":80,"h":80,"aspect":"fixed","title":"Machine","xml":"dVHLEoMgDPwazj649FztePLU/gBqRuiAOJCp+vcFwVbb8ZCZbLKbDYHQQs2VYSOvdQeS0BuhhdEaQ6bmAqQkeSo6QkuS56mL5KSXuXBwZAYG/KPr5gktur5kjTcqV7YX4DJCwDVruRggzjG6BWvF0D+Egr1gM1zhcReLi4zciQuE+8hajyf3REKvHJW3zlwaRr3AIMyn+2c7hwq0AjSLo0yiQx4YlzSoOIie47HGbMD9R7kdI4kLBxBPE8D3+MnPz7wB"},{"w":80,"h":80,"aspect":"fixed","title":"Exit","xml":"dVFLEoIwDD1N13y6cS0wrFx5ggIZWqe1nRIFbm9LQEWHRWbykpe8fBgvzFR74eTFdqAZrxgvvLVInpkK0JrlqeoYL1mep8GSg1wWLEAnPNzxj26bG7QY8lo0Uahc2LEAZweEq0khxbaeC2UvN+CsV76X1jSPgfHzKBXC1Yk2ZsawT4hJNFEnCy41fYJHmA6Hzb60arAG0M+BMqoOJTFOKVVJUL3EfUwMhPt35bZ5so5OYL0Dgc+lk583vAA="}]'

XML Encoding#

The JSON is wrapped inside an XML document, with a top-level tag of mxlibrary.

library_xml = f"""<mxlibrary>{library_json}</mxlibrary><!-- /my library -->"""
library_xml

'<mxlibrary>[{"w":80,"h":80,"aspect":"fixed","title":"Source","xml":"dVFLEoIwDD1N13y6cavguHKlFyiQoXVay4QocHsLAQUdF5nJS97LV8jM9SdUjT77CqyQRyEz9J7Yc30G1oo0NpWQuUjTOFj0J5cEC7BRCHf6ofviBiWFvFXF2Cif2KOAhgYYX/wDS+CouRPgHtE8lb0aB2vF0nGC22FaGuzMRe1d8WiFPHTaEFwaVY6ZLmwbYprcOEUSXC76BCTo/66SrHqdwDsgHAKlMxVpZuxiVmkwtaZtTLWM67dyuUs0j85gvhKDzx+irye9AA=="},{"w":80,"h":80,"aspect":"fixed","title":"Queue","xml":"dVG7DoMwDPyazLyWzoWKqUPVLwhgkVShiYIp8Pc1GFpoxRDJdz77bEckaTPkXjp1tRUYkVxEknprkaNmSMEYEYe6Ekkm4jikFxzkInoEnfTwxD+5LR5QIuWNLCajbFZPBTg6YHzroAMmS+lkqXHcClejGe5naHE0SxMitGsJnHulEe7UZ8r0tCRxCpvJPKKQm77AIwyHG0QbrxxsA+hHkvS6QsWKU8hVCnStcM/JlnH9qVzPESyjM1iOw+B7/uDnb94="},{"w":80,"h":80,"aspect":"fixed","title":"Machine","xml":"dVHLEoMgDPwazj649FztePLU/gBqRuiAOJCp+vcFwVbb8ZCZbLKbDYHQQs2VYSOvdQeS0BuhhdEaQ6bmAqQkeSo6QkuS56mL5KSXuXBwZAYG/KPr5gktur5kjTcqV7YX4DJCwDVruRggzjG6BWvF0D+Egr1gM1zhcReLi4zciQuE+8hajyf3REKvHJW3zlwaRr3AIMyn+2c7hwq0AjSLo0yiQx4YlzSoOIie47HGbMD9R7kdI4kLBxBPE8D3+MnPz7wB"},{"w":80,"h":80,"aspect":"fixed","title":"Exit","xml":"dVFLEoIwDD1N13y6cS0wrFx5ggIZWqe1nRIFbm9LQEWHRWbykpe8fBgvzFR74eTFdqAZrxgvvLVInpkK0JrlqeoYL1mep8GSg1wWLEAnPNzxj26bG7QY8lo0Uahc2LEAZweEq0khxbaeC2UvN+CsV76X1jSPgfHzKBXC1Yk2ZsawT4hJNFEnCy41fYJHmA6Hzb60arAG0M+BMqoOJTFOKVVJUL3EfUwMhPt35bZ5so5OYL0Dgc+lk583vAA="}]</mxlibrary><!-- /my library -->'

This, in turn, must be transformed into a Data URI. The whole thing can’t be base64 encoded, because drawio expects a semicolon-separated list of ids.

GOTCHA: Use of data URIs relies on a NASTY PATCH applied when packaging @deathbeds/jupyterlab-drawio-webpack: by default, the upstream would rewrite this into a proxied request, which ipydrawio don’t support. Usually, the name of the library will be derived from the filename, which is usually the last path component after the / … in this case, the whole document is the path, so we make do with some hacks.

URL Encoding#

library_data_uri = f"data:application/xml,{library_xml}"

URL Params#

Finally, the most reliable means of communicating with drawio is via its URL parameters, exposed on the widget as url_params

GOTCHA url_params should be set before the widget is displayed to avoid extra dialogs.

The clibs parameters accepts a list of “library keys,” each with different formats. We are interest in U (for URL) library.

Some others that might be worth exploring some time include L for Local, which works with an IndexedDB instance… but is not guaranteed to be configured by the time a document loads.

Note, we also override the stealth default… stealth isn’t strictly going to worsen the privacy posture, as all of the other providers are still disabled.

url_params = dict(ipydrawio.Diagram._default_url_params(None))
url_params.update(
    clibs=f"U{library_data_uri}",
    stealth="0",
)
url_params

{'db': '0',
 'drafts': '0',
 'gapi': '0',
 'gh': '0',
 'gl': '0',
 'od': '0',
 'p': 'ex;tips;svgdata;sql;anim;trees;replay;anon;flow;webcola;tags',
 'picker': '0',
 'stealth': '0',
 'svg-warning': '0',
 'thumb': '0',
 'tr': '0',
 'ui': 'min',
 'clibs': 'Udata:application/xml,<mxlibrary>[{"w":80,"h":80,"aspect":"fixed","title":"Source","xml":"dVFLEoIwDD1N13y6cavguHKlFyiQoXVay4QocHsLAQUdF5nJS97LV8jM9SdUjT77CqyQRyEz9J7Yc30G1oo0NpWQuUjTOFj0J5cEC7BRCHf6ofviBiWFvFXF2Cif2KOAhgYYX/wDS+CouRPgHtE8lb0aB2vF0nGC22FaGuzMRe1d8WiFPHTaEFwaVY6ZLmwbYprcOEUSXC76BCTo/66SrHqdwDsgHAKlMxVpZuxiVmkwtaZtTLWM67dyuUs0j85gvhKDzx+irye9AA=="},{"w":80,"h":80,"aspect":"fixed","title":"Queue","xml":"dVG7DoMwDPyazLyWzoWKqUPVLwhgkVShiYIp8Pc1GFpoxRDJdz77bEckaTPkXjp1tRUYkVxEknprkaNmSMEYEYe6Ekkm4jikFxzkInoEnfTwxD+5LR5QIuWNLCajbFZPBTg6YHzroAMmS+lkqXHcClejGe5naHE0SxMitGsJnHulEe7UZ8r0tCRxCpvJPKKQm77AIwyHG0QbrxxsA+hHkvS6QsWKU8hVCnStcM/JlnH9qVzPESyjM1iOw+B7/uDnb94="},{"w":80,"h":80,"aspect":"fixed","title":"Machine","xml":"dVHLEoMgDPwazj649FztePLU/gBqRuiAOJCp+vcFwVbb8ZCZbLKbDYHQQs2VYSOvdQeS0BuhhdEaQ6bmAqQkeSo6QkuS56mL5KSXuXBwZAYG/KPr5gktur5kjTcqV7YX4DJCwDVruRggzjG6BWvF0D+Egr1gM1zhcReLi4zciQuE+8hajyf3REKvHJW3zlwaRr3AIMyn+2c7hwq0AjSLo0yiQx4YlzSoOIie47HGbMD9R7kdI4kLBxBPE8D3+MnPz7wB"},{"w":80,"h":80,"aspect":"fixed","title":"Exit","xml":"dVFLEoIwDD1N13y6cS0wrFx5ggIZWqe1nRIFbm9LQEWHRWbykpe8fBgvzFR74eTFdqAZrxgvvLVInpkK0JrlqeoYL1mep8GSg1wWLEAnPNzxj26bG7QY8lo0Uahc2LEAZweEq0khxbaeC2UvN+CsV76X1jSPgfHzKBXC1Yk2ZsawT4hJNFEnCy41fYJHmA6Hzb60arAG0M+BMqoOJTFOKVVJUL3EfUwMhPt35bZ5so5OYL0Dgc+lk583vAA="}]</mxlibrary><!-- /my library -->'}

More URL Params#

A number of other parameters can be useful for custom embedding purposes, such as using a minimal ui, hiding the default libs, disabling additional plugins.

url_params.update(ui="min", libs="0", p="")

Use The Source#

We should now be able to use the desired shapes in the diagram. Unlike url_params, the value of the diagram’s source can be updated immediately.

d.source.value = """<mxfile version="15.8.7" type="embed">
    <diagram id="x" name="My Diagram">
        <mxGraphModel dx="1687" dy="681" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="850" pageHeight="1100" math="0" shadow="0"><root><mxCell id="0"/><mxCell id="1" parent="0"/><object label="apple" type="Source" interArrivalTime="" id="2"><mxCell style="rhombus;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry x="100" y="90" width="80" height="80" as="geometry"/></mxCell></object><object label="banana" type="Queue" capacity="" id="3"><mxCell style="ellipse;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry x="250" y="90" width="80" height="80" as="geometry"/></mxCell></object><object label="cherry" type="Machine" processingTime="" id="4"><mxCell style="whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry x="385" y="90" width="80" height="80" as="geometry"/></mxCell></object><object label="date" type="Exit" id="5"><mxCell style="rhombus;whiteSpace=wrap;html=1;" vertex="1" parent="1"><mxGeometry x="510" y="90" width="80" height="80" as="geometry"/></mxCell></object></root>
        </mxGraphModel>
    </diagram>
</mxfile>"""