Rendering FreeMarker and Python page templates with 11ty
Post #12 published on by Tobias Fedder
The combination of the FreeMarker Generator (Java) and the Chameleon library (Python) enabled me to render both, FreeMarker macros and Python page templates in the same repository using the static site generator Eleventy (11ty).
How?
While clicking through 11ty’s docs to look something up for this website, I stumbled upon the list of templating languages that 11ty supports — either directly or through plugins. One of them caught my eye: Custom.
It turns out 11ty makes it easy to add templating languages through the Configuration API’s addExtension(). That function takes a string for the file extension and an object to define further handling of templates with that file extension. The compile property of that object expects a function taking the template’s content or the template file’s path and should return yet another function. That returned function will be called with the data from 11ty’s Data Cascade for that template; it’s meant to return the result of rendering the template with the given data.
That’s where the FreeMarker Generator comes in. It takes a template written in the FreeMarker Template Language (FTL) as input and outputs the resulting text. The template can use data that’s provided, for example a JSON string. Lastly, additional FTL files containing macros can be made available from a given directory. Macros are reusable blocks or snippets of templates. It’s used via its command line interface (CLI).
freemarker-generator -i=Hi_${_.name} --template-dir=./my_macros --param=input={"name":"Tobi"}
At least that is almost how it works. Now back to the compile function from 11ty. In that function I run a command that uses the CLI, providing the template string, the stringified JSON of the data from the Data Cascade and the path to the directory containing the macros. The resulting text is the HTML that 11ty writes to the output directory.
Some tweaks are necessary for this. To use the macros and the data in the templates the following line of FTL needs to be in the beginning of every template.
<#import "./macros.ftl" as m/><#assign json = tools.jsonpath.parse(input)><#assign _ = json.read("$")>
Simply injecting that line as a prefix to each FTL template inside of the compile function does the job. That is, if there is a file called macros.ftl. A FTL file can contain many macros. But each file has to be imported with a unique name. That’s why I use a naming convention. The names of all files defining FTL macros end with _macro.ftl. For example whatever.ftl uses/showcases the macro defined/developed in whatever_macro.ftl. On 11ty’s before event all the macros are written to the macros.ftl file, which is outside of 11ty’s input directory. Then only that one file needs to be imported in the FTL templates. Returning early from the compile function whenever the filename ends with _macros.ftl prevents 11ty from rendering empty useless files.
Another tweak is the removal of all circular references from the data in order to stringify the data to JSON. Removing the collections from the data accomplishes that. So, yeah, you’re not going to write your entire 11ty site in FreeMarker, sorry.
The same is true for using Python Page Templates (PT). A short Python script that uses the Chameleon library achieves basically everything I just described for FTL, except it’s PT this time, including the Macro Expansion Template Attribute Language (METAL), which brings the concept of macros to PT.
Here is a short re‐write of this proof of concept using publicly available resources, without any corporate software architecture overhead, that I put on GitHub as Metalmarkerty, in case you’re interested. It really doesn’t do anything except rendering HTML, from both FTL and PT template files, using one macro that uses another macro within. It’s also a dev container, so it should be possible to try it in a code space on GitHub.
Potential improvements
So it works. But it’s not great. The rendering is slow. On my machine it takes around 350 ms per FTL template and about 100 ms per PT file. That isn’t that surprising, considering that every template starts the programm, loads libraries and macro files, then processes one template and tears everything down; just to redo all of that for the next template. I assume it’d be much more performant to start a programm/service once on 11ty’s before event and call it for every template and data pair, then stop it on 11ty’s after event.
So far it just renders some loose HTML files. Ideally it could show the result within a page that contains documentation from a markdown file — like whatever.md to stick with that example. Maybe using containment or <iframe>s? Ideally that page would show in which template languages that component has already been implemented.
Whenever a component is defined in more than one template language, it could be useful to check whether the resulting HTML is actually the same. Minifying the HTML, including sorting attributes and classes, then diffing the results could be a way of achieving that.
There are many ways to use a component. Finding a way to create multiple results per template from data‐defined variants with (too) short or long text, empty arguments for optional parameters, etc., based on some kind of iteration (11ty’s built‐in pagination maybe?) would help highlight weaknesses in design or implementation.
If you take a look at the repository you'll see that I used Deno as the JS-runtime. I use it for this website and I have fun trying it out — maybe that’s a future blog post. Deno’s compatibility with the Node modules, from Deno 2 onward, is really good. But it’s not perfect. Even with the little bit of file system interaction here I ran into small problems. If someone depended on the use or output of this system, then I’d use NodeJS for it, because that is the runtime 11ty is built for. Most advantages of Deno disappear pretty quickly, when running software that is built for NodeJS anyway. Again, maybe that’s a topic for another blog post. I’m sure there is more stuff that could be improved, when building on this.
Why though?
Still here? Wondering why anyone would do that to begin with? Okay.
I’m part of the only web dev team in our organisation. We are tasked with developing and customising many web services, ranging from classic websites based on a content management system, to customer portals consisting of countless, never‐ending, sometimes complicated forms (hello German bureaucracy). Some of these services are written in Java, others in Python. All of these use server-side rendering and therefore either FTL, or PT including METAL.
While there are different web services for a reason, they serve different purposes and therefore differ somewhat in their UI. It’s hopefully not too hard to imagine, that some parts of the UI could be shared between these and that they ideally look alike across services. With the template files placed in each services’ repository that’s hard to achieve though. It’d be easier, if the components were developed and distributed from one central place.
We were once in reach of a first building block for such a thing. As part of a migration and relaunch of one of our services a UX‐design agency created a new design for us. To our delight it didn’t just consist of beautiful images and concepts. It also included template files, CSS and JS as well. On top of that the files were organized in Fractal, a tool to showcase and develop the collection of components, their markup and notes.
The only downside, that template language was handlebars. Handlebars is a fine JS‐based templating language. It’s a good fit for an UX- and front-end-minded agency, but it’s also a template language foreign to us and our backends. Unfortunately that’s one of the reasons why it quickly deteriorated (why have a third template language just for the style guide, right?). A tool similar to Fractal, that renders mock data, FTL, and PT to HTML could really benefit us.
I had the FreeMarker Generator on my radar for quite a while for that reason. When I found 11ty’s Custom Template Languages page it clicked for me. It became clear that I had to try using the FreeMarker Generator with 11ty. And in case of success, I needed to find or build a similar tool for PT and use it with 11ty as well. A few months after that realisation our team hold what we call innovation days. We basically reserved two days for the team to try things we think might help our work. One colleague joined me in this effort of rendering FTL and PT organized through 11ty. At the end of the second day the result of this experiment was quite convincing. At least it was promising enough for the team to decide that this was the one idea to continue to pursue the year after. Hooray, a chance for a better organised set of UI elements after all.
What’s next?
Nothing. It’s on hold. Software development and IT operations will be outsourced in the coming years. Not just outsourced but also merged with other IT service providers from similar organisations. Many working groups consisting of current and future colleagues have been formed to determine how that should look like. One of these groups is concerned with creating a shared design system. As long as there isn’t even a plan for the future — at least not that detailled — it’s unclear whether or not a web‐based style guide and component library, that renders PT and FTL, will be useful for us at all in the long run.
Anyways, you read this whole blog post. Maybe that’s because some of it could be useful for you. So, you tell me: what’s next?