LWT 3.0.0 — The Great Rewrite

[HugoFara]

This is the largest release in LWT's history. The entire application has been restructured from the ground up — new architecture, new frontend, new database engine, new security model — while preserving full compatibility with your existing data.

---

Highlights

Completely new architecture

• MVC + Front Controller: All requests route through index.php with proper Controllers (17 classes), Services (35 classes), and Views. The root folder no longer contains dozens of PHP files. (Refactor PHP files in a manageable way #212, The root folder has too many files #213, Separate business and code logic #215). Contributing becomes easy!
• Extensible Language Parsers (Special language parsers #222): Clean parser architecture using the Strategy Pattern — Regex (default), Character (CJK), and MeCab (Japanese). New parsers can be added by implementing ParserInterface and registering in ParserRegistry. We can now parse Chinese, Korean, and more!

Modern frontend

• Alpine.js + TypeScript: 90+ TypeScript files with reactive Alpine.js components replace the legacy jQuery/iframe/overlib stack. (Use a front end framework such as Vue #208, Move away from raw JS #209, Replace jQuery by a modern framework #211). The app will feel easier to use and be faster.
• Lucide SVG icons: All 97 legacy Fugue PNG icons replaced with scalable, CSS-customizable SVG icons via the IconHelper class. Modern icons!
• Iframe removal (Use AJAX instead of dynamic PHP calls in iframes #166): Legacy iframe-based layout replaced with Alpine.js components and REST API calls. Only dictionary iframes remain for external dictionary display. It brings better mobile support.

Multi-user support (#221)

• Dedicated users table with proper foreign key relationships.
• User authentication, registration, and role-based access (user/admin) with per-user data isolation.
• All user-owned data (languages, texts, words, tags, feeds) linked to the users table with CASCADE delete.

Database overhaul (#220)

• All tables migrated from MyISAM to InnoDB — ACID transactions, row-level locking, crash recovery.
• 16 foreign key constraints for referential integrity across tables.
• All SQL queries in validation functions now use proper input sanitization — multiple SQL injection vulnerabilities fixed.

---

Added

• Notes field for terms (#128): Personal notes separate from translations, with database migration, updated entity classes, service layer, UI forms, and API endpoints.
• Inline Markdown in translations and notes (#126): Support for **bold**, *italic*, ~~strikethrough~~, and [links](url). Rendered in word modals, annotations, word lists, test screens, and detail views. Custom lightweight parsers in both TypeScript and PHP with XSS protection.
• Official support for PHP 8.3 and 8.4.
• Modular Monolith + MVC Architecture (#212, #215): Complete restructure with Controllers in src/backend/Controllers/, Services in src/backend/Services/, Views in src/backend/Views/, Router in src/backend/Router/. Then, the architecture was refactored as Modular Monolith.
• Front Controller Pattern (#213): All requests route through index.php. Only index.php and router.php remain in the root folder.
• Alpine.js Frontend Framework (#208): Reactive UI components reducing reliance on PHP-rendered pages.
• TypeScript Migration (#209): 90+ .ts files in src/frontend/js/ with proper module structure (api/, core/, reading/, testing/, feeds/, languages/, etc.).
• jQuery Removal (#211): Replaced with modern vanilla JavaScript and Alpine.js.
• Overlib Replacement (#214): The 20-year-old overlib tooltip library has been removed.
• HoverIntent Replacement (#210): jQuery plugin replaced with a native TypeScript implementation.
• Extensible Language Parsers (#222): Strategy Pattern with ParserInterface, ParserRegistry, ParsingCoordinator, and value objects (Token, ParserResult, ParserConfig). Each language specifies its parser via the LgParserType column. Language edit form dynamically adjusts to the selected parser.
• Multi-user support (#221): Dedicated users table with foreign keys, authentication, registration, and role-based access (user/admin).

Changed

• Database Engine Migration (#220): All permanent tables converted from MyISAM to InnoDB (ACID transactions, row-level locking, foreign keys, crash recovery).
• Inter-Table Foreign Keys (#220): 16 foreign key constraints added. Language references, text references, word references, and tag references all enforce referential integrity with CASCADE behavior. Migration: 20251221_120000_add_inter_table_foreign_keys.sql.
• TTS Settings Storage (#186): Text-to-Speech settings now use browser localStorage instead of cookies. Includes automatic migration from old cookie format.
• Icon Migration to Lucide SVG: All 97 legacy Fugue PNG icons (16×16) replaced with scalable Lucide SVG icons. IconHelper class provides a unified rendering interface.
• Removed SYSTEM_VARIABLES_ADMIN Privilege Requirement (#167): No more elevated database privileges for word imports. Replaced with chunked batch inserts (500 rows per batch) that stream files line-by-line.

Fixed

• Tag Duplicate Key Error (#120): Fixed rare error when saving a word with a tag from a stale session cache. Changed to INSERT IGNORE for race condition handling.
• Japanese Annotations (#101): Fixed off-by-one index in annotationToJson() causing ~50% of annotations to fail.
• Text Parsing (#114): Last word of text now recognized when text ends without punctuation.
• PHP 8.3 Compatibility: Fixed deprecation warnings in langFromDict() and targetLangFromDict() (null checks before parse_str()).
• Database Maintenance: Fixed initWordCount() generating invalid SQL for CJK languages. Word count now defaults to 1 minimum.
• SQL Prefix Queries: Fixed prefixQuery() to handle DROP TABLE IF EXISTS and case-insensitive SQL keywords.
• Punctuation Formatting (#125): Punctuation marks no longer break onto separate lines — wrapped in non-breaking groups.

Security

• Session security cookie to protect against session hijacking.
• Fixed SQL injection vulnerabilities in validateLang(), validateText(), validateTag(), and validateArchTextTag() — all validation functions now properly sanitize non-numeric input.
• Fixed SQL injection vulnerabilities in inc/tags.php: getTextTitle(), saveWordTags(), saveTextTags(), and saveArchivedTextTags() now validate numeric input.
• Fixed SQL syntax errors in tag list functions when handling empty lists or failed tag creation.

Deprecated

• PHP 8.0 testing & support removed.
• PHP 8.1 support removed.
• Legacy table prefix system ($tbpref) is deprecated in favor of user-based multi-tenancy. Existing prefixed tables will be migrated automatically (if things go well). It has been a long way from Give each wordpress user their own database instance. #76 to Replace $tbpref #164

Removed

• Orphaned Frame Settings (#116): Removed all 7 legacy frame-related settings from the admin settings page (frame heights, widths for text and test interfaces). These became non-functional after the iframe replacement (Use AJAX instead of dynamic PHP calls in iframes #166). Also removed the dead initFrameResizable() TypeScript function.

Breaking Changes

Back up your database before upgrading.

• textitems2.Ti2WoID now uses NULL instead of 0 for unknown words (required for FK constraints).
• Legacy table prefix system ($tbpref) is deprecated — existing prefixed tables are migrated automatically.
• Frame-related settings removed (non-functional since iframe replacement). ("Height of right top frame" not working #116)
• Elevated database privileges (SYSTEM_VARIABLES_ADMIN) no longer required. (Remove SYSTEM_ADMIN_VARIABLES privilege #167)

Upgrading

# Docker (recommended)
docker compose pull && docker compose up -d

# Manual
composer install
npm install && npm run build:all
# Database migrations run automatically on first request

Contributors

Thanks to @ProgramComputer, @quopquai, @rahimnathwani and everyone who filed issues and participated in discussions. Particular thanks to @jzohrab, creator of Lute. Discussions with him, and his work on Lute, has been foundational to many of the DB changes. Have a look at his guide to managing database changes.

Full Changelog: 2.10.0...3.0.0

---

This discussion was created from the release LWT 3.0.0 — The Great Rewrite.

[TheNomad11]

Image ghcr.io/hugofara/lwt:3.0.0 Error manifest unknown

latest points to 2.9.1

[HugoFara]

True, there where incompatibilities with MeCab and some more exotic architectures, which blocked the build of the Docker image. I released 3.0.1 so that they won't have to build MeCab, and I updated the installer to use the latest published image anyway.