RTFM: a Really Tiny Flexible Manual... generator

[airstruck]

Hmm, that acronym needs some work.

Anyway, here's an early preview of a tiny ldoc alternative I've been working on over the past few days.

Features

- It's a very small single file, currently around 500 lines including the default template.

- It's very configurable, via an optional configuration file and the command line.

- It's simple. It doesn't do any parsing of the source other than looking for docblocks.

- It's extensible. In fact, you can monkeypatch it right in the config file if you want.

Options

Options work like this. Internally there's a thing called a "generator," and that has fields like "input" (a "Reader") and "output" (a "Writer"). Those things in turn have their own fields that can affect their behavior. The configuration file runs with the generator as its environment, so in the configuration file you can write, for example:

Code: Select all

-- rtfmconf.lua

template.title = 'API Docs' -- the main title to show on our docs
template.path = 'stuff/template.html' -- path to our custom template
output.path = 'docs/index.html' -- writes to stdout if not present
tag.field.sort = 'type,name' -- sort field tags by type, then name

Command line options work the same way. They're written like --some.option=value and should appear before the list of source files to read. Any options passed on the command line override those in the configuration file. For example, you can run the tool like this:

Code: Select all

lua rtfm.lua --template.title='Awesome API Docs' --tag.field.sort='type,name' game/foo.lua game/bar.lua

Preview

Here's a quick look at part of RTFM's own docs, generated with the built-in template.

preview.png (43.51 KiB) Viewed 5969 times

Where's the manual?

Well, it can document itself, naturally. I want to get the built-in docs to the point where no other documentation is necessary, but it's not there yet (that's partly what this thread's for; the stuff I'm writing here should end up there eventually).

You can generate the docs like this:

Code: Select all

lua rtfm.lua rtfm.lua > rtfm.html

Feedback?

I'm posting this early because I'd love to get some feedback on it, even though there are still things I still want to add. Please let me know what you think, good or bad.

Download

I'll put it on github under the MIT license when it's more polished, but it can live here for now.

rtfm.lua

(16.38 KiB) Downloaded 192 times

[bartbes]

I see you're generating HTML directly, is that just the default template's fault, or is it actually only set up to generate html?

[airstruck]

I see you're generating HTML directly, is that just the default template's fault, or is it actually only set up to generate html?

Yeah, for the default template I wanted to have output that could be viewed directly without any extra steps, so HTML seemed like the obvious choice. The template abstraction in this script is just a way to embed Lua in otherwise plain text with some escape sequences, so you could easily create a markdown template, or JSON, or XML, or some other Lua format or whatever.

Right now the escape sequences are hard-coded; <@ foo @> is a regular escape (for control flow or whatever) and <@= foo @> will output the result of an expression. Those escape sequences will be configurable eventually, just haven't gotten to it yet. Two upvalues are available in templates; "self" is the template instance and "tags" is the tag data extracted by the parser.

The only thing HTML-specific about the template abstraction is an alternate escape sequence, <script>@ foo @</script>, that's useful for getting text editor syntax highlighting not to freak out if you want to define some longer functions inside an HTML template. Why would you want to define them in the template instead of making them members of the template abstraction, you ask? Because then someone would complain about the abstraction being too specialized.

[davisdude]

I like it. I know you want it on GitHub when it's more polished, but IMO it would be more beneficial to put it up now.

[airstruck]

I like it. I know you want it on GitHub when it's more polished, but IMO it would be more beneficial to put it up now.

Fair enough, will put it up soon, just need to get this template looking halfway decent again after tearing it all apart

[airstruck]

Here you go: https://github.com/airstruck/rtfm

Not totally happy with the template yet but it'll do for now. Will add readme and example stuff soon.

[airstruck]

The imperative style of the templates is bothering me. It works, but loops and functions within the template can be hard to follow. I thought about this a bit and am leaning towards adding something like XSL's "template" and "apply-templates" elements to allow templates to be written in a more declarative way. This would allow for keeping everything "flat," achieve some level of XSL-style polymorphism, and make templates much more readable.

I've come up with something resembling a simple "xpath for Lua" for selecting nodes (tags), it seems to work pretty well. This would be used for template's "match" and apply-templates' "select" parameters. What I can't decide on is what these new additions should be called, and what the escape sequences should look like. I'd like it to fit with the escape sequences described above, but there's no precedent for opening and closing "tags" yet, which "template" would need (apply-templates would be "self-terminating," so that should be easier, can just be a regular escaped function call).

I'd like to get suggestions on how these new additions should look. I suppose it could just be a regular function call to start a template and another one to end it, but it might be better to have some dedicated syntax to make it a little more compact and less brittle and awkward. If anyone familiar with the basics of XSL has any ideas, I'd love to hear them.

[EntranceJew]

Have you considered consulting luastache? Imo doc templates could look as dumb as WordPress templates as long as they can loop and have hierarchy for nested data I'm fine.

It's hard for me to sound enthused about another template Lang, when most PHP templates (smarty, blade) all feel similar enough. Especially if I am only going to write a handful of templates ever, tweaking as needed.

[airstruck]

Have you considered consulting luastache?

I can't find that anywhere, is it a Lua port of Mustache?

I've never used mustache, so I don't know if it can do what I'm talking about. The idea is that all your templates are defined at the top level, there are no "subtemplates." Each one is responsible for transforming any nodes in a set (context) matching certain criteria (selector). They can call "apply-templates," which applies a template to a subset of their context nodes. Templates are assigned specificity based on query complexity, like CSS. When applying templates, only the one with the highest specificity is applied.

Generally you define a bunch of templates and then call apply-templates on the root node of your data. Templates call apply-templates when they want another template to deal with the child data of their own contexts. They don't decide which template is applied, another template is chosen by specificity (this is where XSL's polymorphism comes from).

Imo doc templates could look as dumb as WordPress templates as long as they can loop and have hierarchy for nested data I'm fine.

That's basically how they are now. The templating system is pretty much identical to CGILua (or PHP, or classic ASP) with different escape characters. I want to put the important stuff from XSL on top of it (selectors, xsl:templates and apply-templates) because I think it can be a lot cleaner to deal with this kind of structured data in a declarative way. If Luastache or something else can do this, I'll definitely take a look.

It's hard for me to sound enthused about another template Lang, when most PHP templates (smarty, blade) all feel similar enough. Especially if I am only going to write a handful of templates ever, tweaking as needed.

Yeah, agreed. Of course you wouldn't have to use this part of it if you didn't want to. It would be nice to do this as regular function calls so there's no weird syntax learning curve. Maybe just an "apply" function for apply-templates, and a "define" function taking a callback for xsl:template (and optional mode and selector params for each).

Thanks, that's something to think about.

[airstruck]

Updated with experimental rule-based template transformations discussed above. The idea is that templates should be more manageable and easier to read, reason about and modify since things are naturally broken up into small chunks with a more singular purpose.

Within templates, `define` is similar to xsl:template, and `defer` and `descend` are similar to xsl:apply-templates. Selectors are just strings with Lua code that is evaluated with a "tag" as the environment. Priority is given to the last matching transformation rule, so rules should be written in order of lowest to highest specificity. Take a look at the built-in default template for an example. Of course this is all optional, and templates can still be written as they were before.

This is actually a feature-stripped version of what I originally had, so if anyone using this needs something closer to xpath selectors (features like slash, double-slash, dot, and so on) let me know what you need it for and I'll look at putting it back in.

Next up will be ldoc-style @sections, a few minor tag additions like @extends, and more thorough self-documentation. Also taking requests.