Search
core/searchquery parses bounded flat clauses, resolves names at the API boundary
and compiles parameterized SQL/FTS. Each text clause matches the porter word index
(documents_fts) or the substring trigram index (documents_trigram), so search
does not depend on the documents’ language. core/api/search_query.go keeps ranked Search,
sortable Documents and saved views on the same document-set contract while
preserving response shapes. See Search.
core/api/document_scope.go also owns the no-correspondent predicate used by
Documents and Ask your documents. It applies inside the authorized document SQL,
so counts, pages, retrieval candidates, and cited sources cannot broaden beyond
the visible scope.
Automations and ask questions
Local matching proposes unresolved metadata for review before deterministic user automations.core/automations owns triggers/conditions/actions;
core/approvals owns human state machines. Each automation action definition
pairs validation and execution in one instance registry. Save-time validation and
execution-time validation use that definition in the caller’s transaction.
Document/system admission and refile admin rechecks remain outside the handler;
only built-in discard can revisit Trash. Custom callbacks are trusted compiled
code, not an authorization or inference interface. Enqueue extension work in the
same transaction as its state change; keep network/subprocess calls outside it.
An ask automation remains an automation definition, not a second workflow
engine. core/automations/ask.go validates the bounded question/answer/action
mapping and selects matching document_added questions from one read snapshot.
The LLM transport receives only model-safe {automation_id, question, allowed_answers} objects. plugins/llm-classifier/handler.go keeps local action
mappings and custom-field baselines outside the prompt, maps validated answers
before ordinary classifier fields, and submits the resulting fixed effects
through the existing automatic/document-change approval paths. The configured
question lives only with its automation definition; raw answers and evidence
quotes have no durable run-history table.
Before classification transport, the handler loads at most 100 most-used
correspondent names and 100 most-used tag names from the document’s filing
system. The provider sees those bounded names, not catalog IDs. Under the final
writer transaction, the handler reloads the current complete catalog and maps an
exact or one conservative unambiguous name variant to its existing ID; ambiguous
values remain ordinary new proposals.
Approval runs and recovery
Automations owns the action contract. Integrations must not add another workflow engine. Advancement jobs carry system/run revision; handlers run outside the writer, then their transactional effects/tasks recheck revision/status. Tasks retain originating revision so retries cannot reopen resolved visits. Accepted decisions/timeouts increment revision and clear deadlines before queued advancement; timeout selection, expiration and enqueue are one transaction. The periodic sweep also cancels superseded document-change reviews; it never accepts a filing suggestion. Human metadata writes reconcile affected reviews in the same transaction. Cancellation records asuperseded transition and expires open
tasks, while document-change resolutions and queued effects still recheck current
source, field and access state. See
Approvals.
Approval run/task IDs use AUTOINCREMENT; run allocation also skips retained legacy
job references. Upgrade recovery requires complete entry-job history and a resolved
current-state task for human decisions. Only initial-visit legacy timeouts have a
provable origin. Ambiguous jobs remain dead for operator review; retry cannot bind
them to a new state. Migration expires tasks from terminal or different states.
Filing-tree changes
core/jd/presetfile validates bounded offline TOML/HuML before
expansion/generated System/49. Omitted formats default to TOML; HuML remains an
explicit first-class format. Marshal reverses only validated generated rows
and revalidates output; no YAML fallback, runtime catalog, passthrough fields or
separate UI schema.
core/jd/sets.go owns the bounded built-in set catalog and deterministic composed
trees. core/jd/preset_change.go resolves ready-made/composed targets, semantic
replacement defaults, document counts and the selected system’s current
provenance. core/jd/importer remains the single preview/apply engine for those
changes and custom/CLI import.
The preview hash binds source, destination/first-import choice, memberships,
seeds, remaps, replacements, source preset, selected sets and relevant target
state. Apply rechecks the active API admin and plan under the writer, resolves
local symbols, updates reviewed replacement categories in place, preserves IDs
and local forks, suspends the source preset’s owned rules, activates the target,
and atomically commits tree/provenance/jobs. Unrelated systems do not stale a plan.
Only a safe initially unconfigured archive replaces bootstrap. Custom imports
later merge additively; reviewed built-in/composed changes may replace category
meaning explicitly. Refile stays post-Apply and explicit.
Archive setup has no completion mutation: core/settings.LoadSetupState projects
the first-admin time plus the selected system’s preset provenance, selected set
IDs and filing-tree choice. core/api.SetupState compares the stored ID, version
and canonical content hash with the built-in catalog before identifying the
choice as ready-made; an imported file can reuse a catalog ID without acquiring
that identity. The browser reads this through GET /api/admin/setup/state.
POST /api/admin/setup/preset/preview produces the bound review and
POST /api/admin/setup/preset/apply consumes its expected_state_hash.
core/taxonomy/exporter exports consistent archive revision-1 tree snapshots.
Filing trees carry no automations; every rule is archive-owned. Import provenance
is neither installed-pack state nor whole-archive attribution. JD and
Automations own authoring/operator details.
Dates and document questions
core/intelligence validates facts; API source visibility/fact filters precede
counts/pages. Exact-day precision excludes month/year placeholders from Calendar
days. Document dates owns precision/review. On-demand
document questions in core/api/chat.go use explicit callbacks to the configured
model; they are not background work or a general agent runtime. Preserve text
bounds, review decisions, and human edits; see
Ask your documents.
Automatic application and review eligibility
core/documentstate captures source and field generations before inference.
The schema stores each ask as a one-to-one automation_asks definition plus a
conditional action column. Monotonic document counters and triggers in the
baseline schema include same-value reassertions; explicit no-op junction
removals advance intent in their domain writer. core/similar.TopDocsInTx keeps
selection, metadata and supporting snapshots in one read transaction. Matching
and model naming context use owner-level visibility, never an administrator’s
corpus-wide bypass.
The classifier performs provider work outside the writer. Before committing it
rechecks runtime identity, target source and naming-context authorization. For
ask automations it also reloads and compares the complete ordered matching
question/answer/action signatures. Any mismatch discards the whole ask-answer
batch while leaving independently validated ordinary classifier output eligible;
there is no partial per-answer salvage or extra provider retry.
settings.ResolveAutoApply reads the shared policy under the writer: a missing
value enables automatic application; malformed values or errors fail closed.
core/approvals.ApplyAutomaticDocumentChangeInTx checks that policy, the explicit
per-result threshold and score, and current source, field, supporters and
destination guards. Only model/archive sources can take this path; explicit
rules stay independent. Review-required changes use
core/approvals.ProposeDocumentChangeInTx, which invokes shared review
eligibility with host-validated checks and atomically stores bounded
source/human bindings, optional automation attribution, and review jobs.
Unsupported, stale or contradictory candidates cannot gain mutation authority.
Automatic and reviewed metadata use a shared guarded value writer, not duplicate
field-update SQL. Typed custom-field effects reuse core/customfield validation,
storage and rendering; their proposal binds the prior presence plus a one-way
rendered-value fingerprint rather than adding another revision table. Machine
policy and interactive human authorization remain distinct: automatic
application never fabricates a reviewer or locks language. Within a multi-action
batch, successful automatic writes advance the application baseline without
discarding the original snapshot checks against human races. Successful writes
cancel competing reviews made stale by the new field generation.
Metadata resolution stores the interactive principal’s session digest and expiry,
not a bearer cookie. plugin-api.Principal keeps these fields off the wire;
local-cookie authentication supplies them, not API/OIDC bearer authentication.
core/approvals rechecks the exact resolved visit, session, current actor,
membership/ACLs, source/supporters, field generation and destination at the
eventual write. Expired queued authorization can be retired for a fresh review
without rebinding the old decision. Mutation, required provenance and render work
commit together. Existing audit sinks remain best-effort, not that commit boundary.
Dates stay in document_intelligence, not workflow runs. The classifier validates
the raw date’s exact UTF-8 span and normalized value against target text; the
resolve API repeats grounding and authorization under the writer. Each new date’s
own score is checked against the model/date threshold, independently of overall
classification confidence. Automatic acceptance records threshold-auto-v1 and
confidence_threshold, with null reviewer/time. In-flight human decisions,
source generation and exact evidence remain authoritative. Rescans replace pending
candidates with non-reused IDs while preserving accepted/rejected values and
original attribution. Legacy pending rows have no fabricated source proof.
Settings/startup never bulk-accept pending dates. See Document dates.
Purging a document removes its facts/evidence and document-bound approval state
through existing domain ownership. Full database backup retains history and
generations; native takeout remains partial reingestion, not restored review
authority. Classifier-owned review markers are excluded from automation match
snapshots; explicit internal rules remain independent. Approvals
and Automations own the available actions and recovery flow.
Classifier review-tag cleanup is narrowly tracked by document_tags.classifier_owned,
default false including migrated assignments. The LLM handler marks only assignments
it creates, never adopts existing ones, and removes an earlier marker only in the
successful high-confidence result transaction when no longer requested. Explicit
adds, approved suggestions, rules, source metadata and user renames/merges clear
ownership in their own transactions; human reassertions during model calls survive.
Other model tags remain additive. This is not a general provenance ledger; see
LLM classifier.
Human document metadata PATCH and bulk metadata/tag writers remove only
classifier-owned needs-review after successful ACL checks, in the edit’s write
transaction. Explicitly adding the tag first transfers ownership to the human;
Trash, restore and rescan do not resolve it. Document detail refreshes its
displayed tag pills after a successful edit while the marker is visible; the
server owns the provenance decision.