Apache Tika (document text extraction)

Module id: tika · Kind: sidecar · Configured by: TIKA_BASE_URL

Apache Tika is the workhorse text extractor behind uploads: it turns PDFs, Word, Excel, PowerPoint and dozens of other formats into plain text that Synaplan chunks, embeds and searches. It is part of every Compose install and is the only module that is on by default.

What it adds

  • Text extraction for Office documents and PDFs on upload, in chat attachments and in Saved Tasks.
  • The tika step of the Documents extraction chain (structured_office, office_convert, tika, pdf_vision on a fresh install — see AI infrastructure → Extraction).
  • The quality thresholds every extractor is measured against (TIKA_MIN_LENGTH, TIKA_MIN_ENTROPY): a result with too little useful text is rejected and the chain moves on.

Without it

Plain text, Markdown, CSV and HTML are read natively and do not need Tika. For Office files and PDFs the extraction chain simply skips the tika step and continues with the remaining adapters — Office files still yield text through the structured Office reader, PDFs fall to pdf_vision, which sends page images to a vision model and therefore costs tokens. Uploads never fail because Tika is missing; they just extract slower or less completely.

If you run Docling, Tika is also Docling's safety net: a Docling outage hands the file to Tika.

How to enable

Compose (dev, minimal, production)

Nothing to do. docker-compose.yml, docker-compose-minimal.yml and the production deploy/compose.yaml all start the tika service and set TIKA_BASE_URL=http://tika:9998 for the backend and worker.

An external or shared Tika

Point the backend at any Tika server that speaks the standard REST API:

TIKA_BASE_URL=https://tika.internal.example.com
TIKA_HTTP_USER=synaplan       # only if the server is behind basic auth
TIKA_HTTP_PASS=…

Restart backend and worker after changing backend/.env (docker compose restart backend worker).

Turning it off deliberately

TIKA_BASE_URL=disabled

Use the literal disabled rather than deleting the line: the backend requires the variable to be present at start.

Check it works

  • Operate → Feature Status → Apache Tika shows Available with Tika is running and the server version. Tika is not reachable means the URL is set but the container did not answer; TIKA_BASE_URL is unset or disabled means the module is off.
  • Operate → System configuration → Processing → Tika → Test connection runs the same probe on demand.
  • Operate → AI infrastructure → Extraction → Test with a file: upload a PDF and see whether tika won, how long it took, and why other adapters were skipped.
  • Command line: docker compose exec backend php bin/console app:modules:list.

Tika's JVM warms up for a while after start; the container's health check waits for /version to answer within three seconds, so a Needs setup badge in the first minute after docker compose up is normal.

Configuration reference

Variable Default Meaning
TIKA_BASE_URL http://tika:9998 (Compose) Base URL of the Tika server. disabled switches the module off. Required — the backend refuses to start without a value.
TIKA_HTTP_USER / TIKA_HTTP_PASS empty Basic-auth credentials for a protected Tika.
TIKA_TIMEOUT_MS 30000 Per-request timeout. Raise for very large scans.
TIKA_RETRIES 2 Retries on a failed request.
TIKA_RETRY_BACKOFF_MS 1000 Pause between retries.
TIKA_MIN_LENGTH 10 Minimum characters an extraction must yield to count as usable. Applies to every adapter on the families listed in EXTRACTION.QUALITY.apply_to (default pdf).
TIKA_MIN_ENTROPY 3.0 Minimum Shannon entropy of the text — rejects garbage such as a page of repeated glyphs.

The dev Compose file publishes Tika on localhost:9999 for debugging (curl localhost:9999/version); production does not expose it.

Troubleshooting

  • PDF uploads take a minute and the answer is "no text found". The scan has no text layer. Put pdf_vision after tika in the Documents chain (it is there by default), or run Docling with OCR in front of it.
  • Extraction looks fine in Test but chat answers miss table cells. Tika flattens tables into lines. Docling keeps table structure and headings — that is what it is for.
  • Tika is not reachable right after start. Wait for the JVM; check docker compose logs tika.

Related