The Machine

The parts of a build that reach outside it: other programs, the network, encrypted files, other scripts, and what the last run remembered.

sys

local stdout, stderr, code = wattle.sys.exec("git", { "rev-parse", "HEAD" })

wattle.sys.stream("ffmpeg", { "-i", "in.mov", "out.mp4" }, function(line)
  print(line)
end)

exec returns stdout, stderr and the exit code, and waits. stream pipes output into a callback line by line, for jobs long enough that you want to see them working.

Both streams are read at once and arrive interleaved, in the order they were written; the callback cannot tell which one a line came from. Reading them at once rather than one after the other is the whole reason stream works on ffmpeg, which writes progress to stderr faster than a 64 KB pipe can hold it and blocks forever if nobody is draining it.

Arguments never go through a shell. There is no quoting to get wrong because there is no shell to quote for.

http

local body, status, headers = wattle.http.get(url)
local body, status, headers = wattle.http.post(url, { body = "...", headers = {} })

A 404 comes back as 404 rather than an error; only a connection failure raises. That distinction matters when a build is allowed to shrug at a missing resource and not allowed to shrug at a network that is not there.

Pair it with file.cache so a build does not re-fetch on every run:

wattle.file.cache(o("data.json"), function()
  return (wattle.http.get("https://example.com/data.json"))
end)

secrets

wattle.secrets.read(path)

Decrypts and returns a file’s contents, so an encrypted KDL or JSON file can be handed straight to a parser.

Encryption is ChaCha20-Poly1305. The key comes from WATTLE_MASTER_KEY if it is set, and otherwise from wattle/master.key. Run wattle secrets init to generate one, then add it to your .gitignore and back it up somewhere. Nothing can be recovered without it.

wattle secrets edit decrypts into a directory it creates with mode 0700 and removes when the editor exits.

lua

local posts = wattle.lua.run("wattle/posts.lua", { config = config })

Runs another Lua file with its own environment and returns its value, for splitting a large build across several scripts. The context table is what the script can see of the caller.

manifest

wattle.manifest.has_changed(key, content)
wattle.manifest.track(key, content)
wattle.manifest.get_hash(key)
wattle.manifest.clear(key)
wattle.manifest.save()

Content hashes remembered between builds, under $XDG_CACHE_HOME/wattle/<project>-<hash>/, so a script can skip work whose input has not changed:

if wattle.manifest.has_changed(item.file, wattle.file.read(item.file)) then
  expensive(item)
  wattle.manifest.track(item.file, wattle.file.read(item.file))
end

Saved automatically after a build that succeeded, and not after one that failed. A failed build has already written some of its outputs and not others, and a manifest recording that state would make the next run skip exactly the work that did not finish.

Pipeline State

wattle.inputs, wattle.output, wattle.gathered, wattle.processed, wattle.transformed.

inputs is writable and the rest are records of what happened. See how a build works for where each one is filled in.

Where Things Are Written

XDG_CACHE_HOME falling back to $HOME/.cache is the usual rule. A build sandbox usually has neither, which is worth knowing before you spend an afternoon on it: point both somewhere writable and the failures go away.