April 16, 2026
Collecting and normalizing overseas game data
A collection and normalization pipeline for official overseas game data.
Introduction
Arcade data
#Arcade documents
eric

Overseas game names and product information

Adding an overseas arcade requires more than an address. The same game may have different names across countries, and manufacturer and version classifications also need to align. As expansion began in the second half of 2025, these differences affected shared search and filters. Copying a name from an official page could make one product look like several.

Instead of entering every initial record manually, we built a Python collector to read official manufacturer and cabinet sites. It extracts fields, arranges them into the service’s format and loads them into the backend. Bringing different sources into that common structure is normalization. Collecting official information also required deciding how the service identifies the same product.

Mermaid
flowchart TD
    accTitle: From official sources to reference data
    accDescr: Extract required fields from official cabinet and manufacturer pages, normalize them and import them into reference tables.
    S[Official cabinet and manufacturer pages] --> E[Extract required fields]
    E --> N[Normalize to a common structure]
    N --> I[Separate import]
    I --> R[Manufacturers, series and versions]

Collected material and shared reference data

Manufacturers, series and versions are not specific to one arcade. They are reference data shared by many venues. Hard-coding them into interface conditions would require program changes for a name correction, so a separate import stage links them through game_manufacturer, game_series and game_series_version.

Country-specific names and global or regional visibility are stored at the same stage. For example, different local names can retain one product relationship while the interface chooses the appropriate display name. Screens do not need to reinterpret the original page or create duplicate series just because names differ.

Mermaid
flowchart TD
    accTitle: Shared identity and regional presentation
    accDescr: Shared products and versions have country-specific names and visibility scopes used by country-specific reads.
    P[Shared products and versions] --> N[Country-specific display names]
    P --> V[Global and regional visibility]
    N --> Q[Country-specific reads and screens]
    V --> Q

Normalization and repeatable imports

The collector selects relevant fields and fits them to product and version structures. Retaining source text preserves its wording, but does not establish the service’s fields and relationships. Normalization decides which records names and versions belong to.

Imports are idempotent to avoid creating duplicate entries when material is collected again. Idempotence means repeating the same input produces the same result. Data preparation can therefore run independently of query logic, although a consistent repeated result does not establish source accuracy.

Mermaid
flowchart TD
    accTitle: Repeated imports versus on-site facts
    accDescr: Repeated imports of the same input produce the same references; installation at an arcade is checked separately.
    S[Same official material] --> I[Idempotent import]
    I --> R[Same reference data]
    R --> A[Candidate installation links]
    A --> C[Check installation on site separately]

Using initial data and checking installations

This made initial overseas cabinet and version data less dependent on manual entry. Country-specific display and reference relationships support linking installed games and provided a foundation for initial material covering roughly 300–400 Southeast Asian arcades. That figure describes the initial collection scope at the time.

An official page documenting a version does not identify which arcade has it installed. Installation and current business status still need on-site verification. The collector supplies starting product references that users can connect to conditions they subsequently check and correct.

Reference: Engineering notes.

Musecat
All About Arcades