Hister: vanhemmat versiot ja versiohistoria

Changelog
- 0b3b8cd [enh] release v0.20.0

Changelog
- 118a73a [enh] release v0.19.0

Changelog
- 8ba16c2 [enh] release v0.17.0

New Features
- Public Search Mode: added unauthenticated read-only access to global search and previews while keeping write operations protected.
- Documentation Datasets: ships metadata and tooling to import reference documentation (Go, PowerShell, Python, Rust, Node.js, MDN).
- StackExchange Extractor: replaced the StackOverflow extractor with a generic StackExchange extractor indexing answers with metadata.
- Local File Previews: local Markdown and Org files now render as HTML previews with automatic title extraction and clean syntax.
- MCP History & Capabilities: added get_history tool to the MCP endpoint, date filters support, and semantic search capability reporting.
- Browser Extension Upgrades: added configurable hotkeys for indexing controls and badge indicators showing if a page is already indexed.
- Import & Export Improvements: supports HTML files, multiple input files, compressed JSON, date-range filtering, and directory imports.
- Search Sorting & JSON Output: added sorting by newest date, domain, or most visited pages, and a format=json parameter to the search endpoint.
- History RSS Feed: the history endpoint and UI now expose an RSS feed for recently indexed documents.

Enhancements
- File indexing startup runs in the background to avoid blocking server start.
- Watched directory indexing now uses a proper, non-duplicating queue.
- Added CLI indexing flag to explicitly allow sensitive content when requested.
- Semantic search concurrency limits and merging of shared and user-specific vector results.
- Redesigned history timeline displaying indexed version counts and day-unloaded item indicators.
- Refined web UI layout, dark theme, search input, and configurable web title/subtitle branding.
- Added type facets and optimized rendering to hide inactive facet categories with single candidates.
- Built-in yt-dlp is now pre-installed in the official Docker image.

Bug Fixes
- Fixed Lobsters extraction to match updated HTML structure.
- Fixed document label preservation and batch user ID handling.
- Fixed text wrapping and layout squeeze on history and search views.
- Improved Base URL consistency for CSRF checks and invalid configurations.
- Recovered PDF parsing panics and addressed memory issues in PDF dependencies.
- Prevented reindexing from stopping entirely when a single document extractor aborts.
- Fixed database migration side-effects on UpdatedAt timestamps.

New Features
- Document Versioning: track changes to documents upon re-indexing with diff changelogs in the preview panel.
- Priority Rules: re-introduced rules to pin matched URLs to the top of search results.
- PDF Indexing: full-text extraction for local PDFs and support for direct upload via the browser extension.
- Faceted Filtering: filter results by language or domain with live updating counts.
- Embedded Video Extractor: detects and renders embedded videos (YouTube, Vimeo) directly in the preview panel.
- Notion Extractor: clean text extraction from public Notion URLs.
- Optimized Storage: HTML content and favicons are now stored gzip-compressed externally to reduce Bleve index size.
- User-Specific Directory Indexing: bind indexed local paths to specific users for shared instances.
- Inline Label Editing: manage document labels directly from search result cards.
- Browser Import: added Ladybird support and automatic history database detection.
- File Deletion Tracking: automatic index cleanup when a watched local file is deleted.
- MCP Preview: added Model Context Protocol endpoint for document previews.

Enhancements
- Added a copy-to-clipboard button next to result URLs.
- Added a preview extractor selector to switch views without re-indexing.
- Rules table improvements (sortable columns, filterable values, and regex validation).
- New CLI flags (--delay, --timeout, --user-agent) directly on hister index.
- Standardized config file discovery lookup order across platforms.
- Improved yt-dlp diagnostic messages and debug logging.
- URLs found during recursive crawls are now inserted in batches.

Bug Fixes
- Fixed Docker BASE_URL override logic via config file.
- Fixed search results behavior to enforce all queried terms instead of any.
- Fixed history page infinite scrolling and result paging sort keys.
- Prevented HTML from being redundant written during re-indexing.
- Fixed timezone display handling to respect browser configuration.

New Features
- Mastodon Extractor: dedicated support for indexing toots as separate documents with specific metadata.
- UI Improvements: full-screen preview mode, infinite scroll for search results, and image lightbox.
- History Page: added interactive preview panel and keyboard navigation (consistent with search page).
- WebDriver BiDi: new crawler backend using W3C protocol to drive existing browser sessions efficiently.
- Management: direct deletion of all search results, keyboard shortcuts for deletion, and editable rules/aliases in the UI.
- Search: support for exact phrase matching and metadata-specific query filtering (metadata.key:value).
- Robots.txt: the crawler now respects robots.txt rules by default.

Enhancements
- Resizable split-screen preview panel with persistent width.
- Improved semantic search relevance via punctuation-based text chunking.
- OIDC/OAuth improvements, including configurable userinfo endpoints and password login toggle.
- CLI: clearer error messages and version strings in server logs.
- Duplicate prevention for skip/priority rules and search aliases.

Bug Fixes
- Fixed search term escaping and phrase query application logic.
- Resolved API deletion issues from the browser extension.
- Fixed OIDC scope forwarding and TUI client access token handling.
- UI layout fixes for the authentication page on small screens.
- Fixed field inclusion (text, html, language) in search results.

New Features
- Semantic search via sentence-embedding models (SQLite/sqlite-vec or PostgreSQL/pgvector backends).
- OAuth 2.0 / OpenID Connect authentication support.
- MCP (Model Context Protocol) server endpoint at /api/mcp.
- Persistent crawler state: recursive crawl jobs survive interruptions and can be resumed by job ID.
- New extractors: Wikipedia, GitHub project, Lobste.rs, yt-dlp, JSON-LD.
- OpenSearch suggestions endpoint (/api/suggest) for browser search-as-you-type completions.

Enhancements
- --backend flag available on hister index, hister index -r, and hister import-browser (headless Chrome support).
- Per-invocation header and cookie injection for index commands.
- CLI search: --limit, --fields, --html flags; paging support; list-urls --offline mode.
- Quoted field queries (e.g. url:"file:///C:/path with spaces").
- Preview panel: clickable title, maximised content width, JSON-LD metadata, dark theme fixes.
- NixOS module improvements: systemd/launchd hardening, environmentFile option, openFirewall opt-in.
- Executable size reduced ~70 MB via trimmed lingua-go fork.
- Timezone data bundled into the binary.

Bug Fixes
- file:// URLs handled correctly in the UI.
- URLs no longer lowercased during query building.
- History view correctly filtered per-user in multi-user mode.
- File indexing fixed on Windows.
- Various crawler, extension, and Nix module fixes.

New Features
- Web crawler: new `hister index -r <url>` command crawls sites recursively (configurable depth, link count, allowed/excluded domains and URL patterns).
- PostgreSQL backend as an alternative to SQLite.
- Extractor pipeline overhaul: configurable extractors with explicit states, new pkg.go.dev and Stack Overflow extractors.
- Desktop readability panel: focused search results load in a split-pane reader on screens wider than 1280 px.

Enhancements
- HTML sanitizer (bluemonday) applied to all extracted content.
- `metadata` field added to documents for arbitrary key/value data.
- Admin users can create global indexes or indexes on behalf of other users.
- `hister index` skips already-indexed URLs by default; pass `--force` to reindex.
- URL and domain wildcard matching automatically anchors to start and end.
- `import` command renamed to `import-browser`.
- Various UI improvements (ToC in API docs, document indexed date in preview, search query in tab title, WebSocket optimisations).

Bug Fixes
- Browser history database opened read-only to avoid lock conflicts.
- History entries deleted when their associated document is deleted.
- Fixed field-specific alternation parts in query parser.
- HTML field no longer leaks into search results.
- URL changes after HTTP redirects now resolved correctly.
- Various crawler, dark mode and mobile layout fixes.

What's Changed
- Multi-user support.
- Better authentication method for the extension.
- Enhanced delete. Queries can be used to delete multiple items. --dry flag.
- Reindex does not require server stop anymore.
- Enhanced history view.
- Extension feedback if the active tab has a matching skip rule.
- Add pages/domains to skip rules directly from the extension popup.
- Plenty of bugfixes.