github.com/suchi-dms/suchi/plugin-api) with a handful of value types
and interfaces. Everything a plugin needs to talk to core lives there.
Core imports plugin-api; plugins import plugin-api; nobody imports the
other direction. That’s the seam.
Plugins are Go packages compiled into your distro binary — there is no
runtime plugin loader. A “plugin” is what you get when a private
distro’s
plugins/index.go blank-imports your package + wires it in
main.go. This keeps the security posture honest (no arbitrary code
paths at runtime) at the cost of one recompile per plugin swap.The surface
plugin-api is deliberately tiny — 150-odd lines total. Every plugin
implements one or more of these interfaces.
Plus these value types:
Event— a job coming off the outbox:Kind,DocID,Time,Payload.Principal— the authenticated actor: user or scoped token.BlobRef— content-addressed blob pointer:SHA256,Size.AuditEvent— flat, JSON-shaped audit record.
plugin-api/kinds.go — KindIngest, KindSniff,
KindOCR, KindStorage, KindSearch, KindAuth, KindClassify,
KindNotify, KindExport, KindConvert, KindBarcode. These are
strings, not types: they’re a shared vocabulary for config files and
manifests, not a runtime dispatch.
Design rules
Idempotent Handle. The outbox retries with backoff on failure. Any side effect your handler performs may be applied more than once for the same (kind, doc_id, payload). Guard by writing state you can dedup against — a hash, aWHERE NOT EXISTS, a unique index — before making
the observable change.
No hidden state. Everything a handler needs to survive a restart
must be either passed in Event.Payload or read from the DB by DocID.
Package-level maps that “remember what we’ve seen” break on process
kill.
One tx per write. If your handler updates multiple tables, wrap them
in a single db.WriteTx. SQLite is a single-writer database — a partial
update is worse than no update.
Small dep footprint. Anything you import becomes part of the final
suchi binary. A plugin that pulls in 40 MB of transitive deps to
classify a PDF loses to a subprocess that shells out.
Fail loud, not quiet. Return errors from Handle — the outbox logs
them, backs off, eventually parks the job as dead. Swallowing errors
“to be resilient” hides real bugs from the operator.
Worked example — a Subscriber
Say we want to auto-tag every ingestedapplication/pdf document with
pdf. The right seam is a Subscriber on post-ingest.
1. Structure the package
go.mod:
2. Implement Subscriber
3. Test it
Usecore/db.LoadMigrations to bring up a fresh SQLite. Existing
plugins (plugins/llm-classifier/handler_test.go,
core/pipeline/postingest/postingest_test.go) show the pattern.
4. Wire it into a private distro
In your fork ofdistro/cmd/suchi/main.go, alongside the other
disp.Register calls:
Worked example — an Authenticator
Say your org uses signed JWTs from an internal identity broker. Wire that as a link in the auth chain:main.go before the local-auth link:
(nil, nil) when the request isn’t
yours; the next link takes over.
Worked example — an AuditSink
Say you want to forward every audit event to a syslog daemon:AuditSink.Emit is called inline on the write path — keep it fast, or
buffer + fire-and-forget internally. Returning an error only logs and
drops; core’s audit_events table is the durable record.
Testing patterns
Every existing plugin uses the same fixture recipe. Steal it:Handle directly — you don’t need the
dispatcher. For Authenticators, construct an *http.Request with
httptest.NewRequest and call Authenticate.
Guard tests behind t.Skip if the plugin shells out to a binary the
CI host doesn’t have (see core/pipeline/heic/heic_test.go for the
pattern — checks magick on PATH before running).
Registration cookbook
Almost every plugin wires the same way in a private distro’smain.go. Copy-paste from an existing block:
- Subscriber:
disp.Register(myplugin.New(d, log)). Seedistro/cmd/suchi/main.go— the block that registerspostingest,view,webhookdispatch,workflowsubscribers is the template. - Authenticator: append to
authChain.Authenticators(order matters — first non-nilPrincipalwins). - AuditSink: no core-side wiring today; the interface is reserved
in
plugin-api/audit.go. A futureaudit.RegisterSink(s)call will slot in the same way; plan for it now by keeping your sink stateless and its constructor pure.
What NOT to do
- Don’t fork core to add a feature that fits behind an interface.
If your plugin needs a new callback point, extend
plugin-apifirst (send a PR upstream). Forks rot. - Don’t touch
documents.original_blob. The CAS layer owns content-addressed blobs; plugins write metadata + archives, never the raw ingest. Rewriting the original invalidates dedup and audit. - Don’t hold DB write locks across network I/O. SQLite has one
writer; a slow LLM call inside a
WriteTxfreezes every other writer. Do the network call outside, buffer the result, then open the tx. - Don’t invent new job kinds without a corresponding Subscriber registered at boot. An orphaned kind just piles up in the outbox as unclaimed rows.
- Don’t emit personally identifiable information into audit events.
Core’s
audit_eventstable already carries actor + action + object id;Before/Aftermaps are for diffable state, not raw document content.
When to prefer a subprocess
If your plugin shells out toqpdf/tesseract/magick/msgconvert
style tools, use core/sandbox.Run rather than os/exec directly.
It caps stdout, kills on timeout, removes network access, and matches
the deployment convention every other pipeline stage uses. See
core/pipeline/heic/heic.go as the reference.
Referencing other plugins
Each is a working example under 500 lines. Read one that matches your
seam before writing your own.