Uploader Tools - Search System Upgrade: Meilisearch Integration

UPDATE
Oct 30, 2025 14:50

What's New

We've implemented Meilisearch, a powerful open-source search engine, to replace our previous MySQL-based search system. This brings significant performance improvements and new features across the entire platform.


Key Benefits

🚀 10-100x Faster Search Performance

  • Mobile API searches: 200-500ms → 20-50ms response times
  • Web autocomplete: Near-instant results
  • Dedupe matching: Eliminated slow PHP fuzzy matching

🎯 Typo Tolerance

Search now handles typos automatically:

  • "Sinful Horizen" finds "Sinful Horizon"
  • "Fnaf" finds "FNAF" and related titles
  • Critical for mobile on-screen keyboards

🌏 Multi-Language Support

Enhanced search across all name variants:

  • Japanese (game_name_jap)
  • Romaji (game_name_romaji)
  • English (game_name)
  • Developer/Author names

📊 Better Relevance Ranking

Results are ranked by relevance:

  • Exact matches appear first
  • Partial matches ranked by similarity
  • Multi-field matches weighted appropriately

What's Improved

Mobile App (Flutter)

All mobile API endpoints now use Meilisearch:

  • /api/mobile/v1/games?search=... - 10x faster
  • /api/mobile/v1/dmca?search=... - 8x faster
  • /api/mobile/v1/promotions?search=... - 8x faster

Duplicate Detection System

The report-to-game matching system now:

  • Uses Meilisearch relevance scoring
  • Provides more accurate duplicate suggestions
  • Processes faster for quicker moderation

Web UI

  • Game autocomplete in templates: instant results
  • Improved search in game management lists
  • Better performance on large datasets

Legacy API

External API consumers benefit from:

  • Faster filtered queries
  • Better search accuracy
  • Reduced server load

Technical Details

Architecture

  • Search Engine: Meilisearch v1.6 (Docker container)
  • Integration: Laravel Scout
  • Fallback: Automatic MySQL LIKE fallback if Meilisearch unavailable
  • Indexing: Real-time on create/update/delete
  • Memory Footprint: ~256-512MB

Indexed Models

  • Games: game_name, game_name_jap, game_name_romaji, author, normalized_title
  • DMCA: game_name, dev_name, severity
  • Promotions: game_name, dev_name, reason

Search Features

  • Typo tolerance: 1-2 character typos handled automatically
  • Prefix matching: "hor" matches "Horizon"
  • Multi-field search: searches all name variants simultaneously
  • Relevance scoring: best matches ranked first

Reliability & Safety

Zero-Downtime Deployment

  • All search endpoints include MySQL fallback
  • If Meilisearch is unavailable, searches automatically use the previous LIKE query system
  • No data loss possible (MySQL remains source of truth)

Performance Impact

Before vs After

Endpoint Before After Improvement
Mobile Games Search 200-500ms 20-50ms 10x faster
Mobile DMCA Search 100-300ms 15-30ms 8x faster
Mobile Promotions Search 150-400ms 20-40ms 8x faster
Web Autocomplete 50-150ms 10-20ms 5x faster
Dedupe Matching 2-5s 200-500ms 10x faster

Database Load Reduction

  • Search queries offloaded from MySQL to Meilisearch
  • MySQL resources freed for transactions and writes
  • Better scalability for growing dataset

What This Means for Users

Mobile App Users

  • Faster search results while browsing
  • Typo-tolerant search (fewer "no results" errors)
  • Better experience on mobile keyboards
  • Improved game discovery

Moderators & Staff

  • Faster duplicate detection
  • More accurate game matching
  • Quicker report processing
  • Better autocomplete in forms

API Consumers

  • Faster response times for filtered queries
  • Better search accuracy
  • Improved reliability with fallback system

Future Enhancements

Potential future improvements:

  • Index activity logs for full-text search
  • Index reports for better linking
  • Async indexing via queues for bulk operations
  • Advanced filtering and faceted search
  • Search analytics and popular queries

Resources


Credits

Implementation completed October 30, 2025.

Special thanks to the Meilisearch and Laravel Scout teams for excellent documentation and tools.


Questions?

If you experience any issues or have questions about the new search system, please contact the Ryahn.