Changelog

All notable changes to this project will be documented in this file.

[2025.12.09]

Added

  • Landing Page: Added a new landing page at the root URL (/)
    • Hero section with app screenshot
    • Feature highlights and privacy information
    • OGP meta tags for social media sharing
  • English Documentation: Added English translation of all user documentation (12 pages)
    • Getting Started, Data Table, Statistics, Graph Builder, Custom Graph, SQL Editor, Crosstab, Reports, etc.
    • Static HTML generation for English docs
  • Create Dataset: Create new empty datasets from the File menu
  • Selected Rows to Dataset: Create a derived dataset from selected rows in the Selected Rows tab
  • Sandbox Mode: Added sandbox mode for embedding MIDAS in documentation with interactive demos
  • Bin Statistics - Boundary/Center: Added boundary and center parameters for histogram bin alignment

Changed

  • URL Change: The application URL has changed from / to /app.

Fixed

  • Statistics Tab - Kebab Menu: Added kebab menu to all graphs (histogram, scatter plot, time series, date distribution) with "Add to Report" and "Open in Graph Builder" options
  • NULL Display: Improved NULL value display with distinct styling
  • Tab Creation Modal: Fixed infinite loop error in TabCreationComboBox
  • Primary Dataset Edit: Invalidate dependent element caches when editing primary datasets
  • Stale Model Blocking: Block predictions and diagnostics for stale models instead of deleting them

[2025.12.05]

Added

  • Data Table - Edit Mode: Added editing capabilities for primary datasets
    • Inline cell editing with double-click
    • Add new rows
    • Delete rows
  • Project Diff: Compare current project state with saved MDS files
    • Detect changes in datasets, models, and reports
    • Visual diff UI with detailed change information
    • Clickable dataset ID links to open corresponding tabs
  • Report - Markdown Editor: Migrated report editor to CodeMirror with syntax highlighting, autocomplete for element references, and collapsible element preview section
  • Report - Aspect Ratio: Added aspect ratio setting for report elements
  • Report - Element Menu: Added menu button in inline preview with Duplicate and Open Source Dataset options
  • Report - Multi-Reference Warning: Show warning when modifying elements that are referenced multiple times
  • Data Table - Cell Tooltip: Show tooltip with full cell content on hover (500ms delay)
  • Graph Builder - Facet Category Limit: Added Max Facet Categories setting in Settings > Graph tab to prevent browser freezing when selecting columns with too many categories
  • AI Agent Command Interface: Added x: command system for AI agent integration
    • x:tab - Create and configure analysis tabs (SQL Editor, Statistics, GLM, Graph Builder, etc.)
    • x:dataset - Create derived datasets via SQL queries
    • x:report - Add graphs and text to reports
    • x:model - Execute GLM models
    • x:status, x:list, x:describe - Query project state
  • Console Mode: Added console mode UI for AI agents to execute commands and view results
  • Console Mode: Display only the latest command result in UI (full history available in browser console)
  • AI Agent Command - Dataset: Auto-resolve targetDatasetId from SQL query for sql_query type operations
  • Crosstab - Multi-Value Fields: Crosstab now supports multiple value fields with aggregation
  • Custom Graph - Facet Column Types: Facet now accepts all column types (Int64, Float64, datetime, etc.), not just categorical columns

Fixed

  • Custom Graph - Legend Ordinal Order: Fixed legend not respecting ordinal order when ordinal variable is assigned to Fill, Color, Stroke, Shape, or Linetype aesthetics
  • Report - Element Menu (Firefox): Fixed report element menu not clickable in Firefox
  • Report - View Settings: Fixed View Settings not working for graphs in reports
  • Report - Embed Mode: Fixed report tab disappearing when navigating back from embed mode
  • Report - Embed Mode Save: Save project to OPFS before entering embed mode to prevent data loss
  • Data Table Menu - Checkbox UI: Fixed "Save data with project" checkbox not visually updating when clicked directly on the checkbox
  • Data Table - File Selection: Fixed file selection to use input element so it can be opened from command palette
  • Custom Graph - Int64 Axis Tick: Fixed decimal tick values appearing on Int64 type axis
  • SQL Query Editor - INTERVAL Type: Display DuckDB INTERVAL type in human-readable format
  • Storage Management: Display project names instead of project IDs in the Saved Projects list
  • Dataset Rename: Automatically update SQL queries when renaming a dataset that is referenced by SQL Query operations
  • SQL Query Editor: Fixed false positive detection of multiple statements when semicolon appears inside string literals
  • Project Lineage: Fixed table references in SQL subqueries not being detected for lineage tracking
    • WHERE clause subqueries (IN, NOT IN, EXISTS, comparison)
    • Scalar subqueries in SELECT list
    • HAVING clause subqueries
    • FROM clause subqueries (derived tables)
    • Nested subqueries
  • SQL Query Editor: Fixed UNION/INTERSECT/EXCEPT queries being incorrectly rejected as unsafe operations
  • E2E Test - Quick Access: Fixed test using CSS class selector that didn't work in production builds

[2025.12.01]

Added

  • Open from URL - CSV Support: Load CSV files directly from URLs with progress display, cancellation, and Content-Type validation
  • Application Settings: Added settings dialog with trusted URL whitelist management for Open from URL
  • Report - Add Text Button: Added "Add Text" button to report header kebab menu

Fixed

  • Graph Builder - X-Axis Label Rotation: Improved label rotation logic to compare label width with band width
  • Data Table - Filter Expression: Unified filter expression syntax to DuckDB SQL standard (quotes and operators)
  • Settings - UI Separation: Separated Settings and Storage Management into distinct buttons
  • DuckDB Worker - Initialization: Fixed DuckDB being initialized twice in worker
  • Data Table - Save Filtered Dataset: Automatically opens the new tab after saving filtered dataset
  • Time Series - Resize: Fixed Time Series graph not resizing with container
  • Create New Tab - Linear Regression: Removed deprecated Linear Regression option from dialog

[2025.11.30]

Added

  • PWA Support: Added Progressive Web App support for offline access and installation
  • DuckDB Worker - SQL Cancel: DuckDB now runs in a Web Worker with SQL query cancellation support
  • Data Table - Save Filtered Data: Filter results can now be saved as derived datasets
  • Graph Builder - Tile Ordinal Axis: Tile (Heatmap) now supports ordinal axis scales

Fixed

  • Data Table - Reload Cache: Fixed transitive derived dataset cache not being cleared on primary dataset reload
  • SQL Editor - DATE Type: Fixed DATE type displaying as numbers in SQL query results
  • Graph Builder - Dataset Selection: Fixed ID duplication error in dataset selection
  • SQL Editor - Materialized View: Fixed Materialized View checkbox state not persisting correctly

[2025.11.27]

Added

  • Help Menu - Documentation: Added Documentation submenu with Japanese/English links
  • Graph Builder - Geom Tile: Added geom_tile for creating heatmaps and tile-based visualizations
    • Supports continuous color scales for numeric fill values
  • Graph Builder - Summary Statistics: Added stat_summary transformation for aggregate visualizations
    • Mean, median, min, max, sum, and custom percentile functions
    • Output selection (y, ymin, ymax) for different geometry types
    • Complete option to show all x-axis categories including those with no data
  • Graph Builder - Fixed Fill Color: Added option to set a fixed fill color independent of data
  • Dataset Metadata - Description: Added ability to edit dataset descriptions in Dataset Metadata modal

Fixed

  • Graph Builder - Facets: Fixed aesthetic scale domains to be unified across all facet panels
  • Graph Builder - Facets: Fixed "Complete missing combinations" option to apply consistently across all panels
  • Graph Builder - Alpha Aesthetic: Enabled Alpha aesthetic mapping for Bar, Line, Area, and Tile geometries

[2025.11.25]

Added

  • Project Lineage - Context Menu: Right-click context menu for lineage graph nodes
    • Open/Delete menu for datasets, models, and reports (Focus/Close for tabs)
    • Shows dependencies in warning dialog before deletion
    • Deletes all dependent objects when confirmed
  • Documentation Link: Added "Documentation (ja)" link in About dialog for quick access to Japanese documentation
  • MDS URL Duplicate Detection: Prevent duplicate projects when opening same URL multiple times
    • Shows confirmation dialog when opening an MDS URL that was previously loaded
    • Three options: Open Existing (use cached version), Open as New (create new copy), or Cancel
    • Displays project information (name, last modified, fetched time) in confirmation dialog
    • Tracks source URL and fetch timestamp for each project (sourceUrl, sourceUrlFetchedAt)
    • Helps users avoid accidentally creating duplicate projects from the same source
  • Embed Mode - Local Projects: Enter Embed mode from locally opened projects
    • "Embed Mode" menu item in report menu (⋮ button)
    • Browser back button returns to normal ProjectView
    • Auto-loads project from storage when accessing embed URL directly

Fixed

  • Embed Mode - External MDS Loading: Fixed /?mds=<URL>&embed=true not working at root path
    • Root path with mds parameter now correctly renders EmbedView
    • Shows proper error message when mds parameter is missing in embed mode

Removed

  • Report HTML Export: Removed the "Export Report as HTML" feature
    • The hacky DOM-cloning approach had reliability issues with CSS and SVG rendering
    • Use browser's print function (Cmd+P / Ctrl+P) and "Save as PDF" as an alternative

[2025.11.24]

Added

  • Embed Mode: Embed reports in external websites using iframe
    • URL format: ?mds=<URL>&embed=true&report-id=<ID> or &report-name=<NAME>
    • Clean view-only display without menus or editing UI
    • "Created with MIDAS" attribution in bottom-right corner
    • Falls back to first report if no report specified
  • Graph Builder - Responsive Height: Graphs now automatically adjust to container height changes
  • URL-based MDS Loading: Load MDS project files directly from URLs using ?mds= parameter
    • Share projects by simply sharing a URL
    • Three-tier security validation: blocks cloud metadata endpoints, URL-based whitelist, warning dialogs for untrusted sources
    • 10MB file size limit to prevent DoS attacks
    • Protects users by warning about public file hosting services (GitHub, S3, etc.) where unverified content may exist
    • SEO-friendly canonical URLs (excludes tracking parameters)

Fixed

  • Graph Builder - Pair Plot: Fixed Pair Plot to scale dynamically based on container size instead of using fixed dimensions
  • Crosstab - Bar in Cell: Fixed minimum values showing no bar in Bar in Cell visual mode
  • Statistics: Fixed date/datetime column statistics (Earliest/Latest/Time span) not displaying

Changed

  • ID Generation: Unified all ID generation to use UUID v4 format with type-specific prefixes
    • All elements (graphs, reports, datasets, tabs) now use globally unique identifiers
    • Prefixes (graph-, report-, dataset-, etc.) maintained for better debugging experience
    • Eliminates potential ID collision issues in distributed or concurrent usage scenarios

[2025.11.21]

Added

  • Custom Graph - Facets: Facet Wrap and Facet Grid functionality
    • Split data by categorical variables to create multiple sub-graphs
    • Facet Wrap: Grid layout with single variable
    • Facet Grid: 2D grid with row and column variables
    • Horizontal scrolling for viewing all panels when exceeding screen width
  • Custom Graph - Density (KDE): Kernel Density Estimation statistical transformation
    • Added as independent layer statistics option alongside Identity/Count/Bin
    • Bandwidth parameter configuration
    • Y Scale options (count/density) for both Bin and Density statistics
    • Automatic "Density" Y-axis title generation
    • Grouped density curves by categorical aesthetics (color/fill/stroke)
  • Custom Graph - Secondary Y Axis: Dual Y-axis support for overlaying different scales
    • Configure right-side Y axis independently (scale type, title, domain, ticks)
    • Per-layer Y axis selection (Primary/Secondary)
    • Synchronized zoom behavior across both axes
    • Available in Cartesian coordinate system only
  • Custom Graph - Categorical Color Palette: Discrete color palette for categorical variables
    • Optimized color selection for categorical data visualization
  • Documentation: Enhanced documentation site features
    • Left sidebar navigation for improved browsing
    • 404 error page for invalid documentation URLs
  • Graph Builder - Responsive Preview: Dynamic preview height adjustment based on content

Fixed

  • Custom Graph - Color/Fill Specification: Unified color and fill aesthetics to match ggplot2 conventions
    • Bar: fill for bar color, color for border (removed color fallback to fill)
    • Area: fill for area color, color for border, changed grouping from color to fill
    • Point/Line: No changes (already compliant)
    • Added fillScale and strokeScale support across all layers
  • Custom Graph - Layer Aesthetics: Show only supported aesthetics per geometry type
    • Point: x, y, color, fill, stroke, size, shape, alpha
    • Line: x, y, color, size, linetype, alpha
    • Bar: x, y, fill, color, alpha (stroke not supported)
    • Area: x, y, fill, color, alpha
    • Prevents configuration of unsupported aesthetics that would be ignored
  • Custom Graph - Bar Position Settings: Position (Identity/Stack/Dodge) now applies correctly
  • Custom Graph - Bar Display: Corrected bar rendering issues
  • Custom Graph - Stack Position: Fixed stacked bar/area positioning calculation
  • Custom Graph - Bar Axis Overlap: Bars no longer overlap with axis lines
  • Custom Graph - Bin Statistics Aesthetics: Aesthetics (color/fill/size) now preserved when using Bin statistics
  • Custom Graph - Legend Titles: Column names now display correctly in legend titles instead of column IDs
  • Custom Graph - Bar Layer: Disabled color aesthetic for Bar layers (fill is the correct aesthetic for bars)
  • Custom Graph - Query Result Categorical Axes: Categorical columns from Query Result datasets now recognized properly
  • Custom Graph - Axis Labels: Automatic generation of axis labels (axis titles) from column names
  • Custom Graph - X Axis Tick Overlap: Fixed overlapping tick labels on X axis
  • Custom Graph - Facet Settings Layout: Improved Columns and Rows layout in Facet configuration UI
  • Custom Graph - Facet Responsive Sizing: Facet panels now resize dynamically based on parent container dimensions
  • Custom Graph - Facet Label Clipping: X axis labels in faceted graphs no longer get clipped
  • Custom Graph - Density with Faceting: Density curves and faceting can now be used simultaneously
  • Custom Graph - Flip Coordinate Histograms: Numeric histograms now render correctly when using flip coordinate system
  • Custom Graph - Density Y Axis Requirement: Fixed Density statistics specification to correctly indicate Y axis is not required
  • Graph Builder - Preview Scroll: Fixed scroll behavior in preview container
  • Graph Builder - Dataset Dropdown: Corrected dataset dropdown display issues

[2025.11.17]

Added

  • Custom Graph: Grammar of Graphics-based graph creation feature
    • Multiple geometry types (Point/Line/Bar/Area)
    • Layer-based multi-layer support
    • Statistical transformations (Identity/Count/Bin) and position adjustments (Identity/Stack/Dodge)
    • Categorical scale and color scale configuration UI
    • Additional aesthetic channels (size, fill, stroke, alpha)
    • Coordinate system transformations (cartesian/flip) and legend rendering
    • Support for ordinal/nominal columns
    • Category order customization with Reverse button
    • Two-pane layout with collapsible configuration sections

Fixed

  • Project Lineage: Fixed issue where SQL dependencies with multiple tables were not tracked correctly
    • Added support for CROSS_PRODUCT (comma-separated tables)
    • Removed duplicate edge creation
  • Graph Builder: Fixed issue where previous rendering remained when settings changed
  • DerivedDataSet: Fixed recursive evaluation of parent datasets
  • Data Table: Fixed conflict between cell copy feature and row context menu

Performance

  • DuckDB: Improved DuckDB initialization speed for SQL evaluation

[2025.11.05-2]

Fixed

  • Column Type Conversion: Fixed issue where excluded rows were not preserved after project save and reload when using EXCLUDE mode (#174)
  • Histogram: Fixed rendering issue for datasets with 1000+ rows (#176)
  • Graph Builder - Bar Chart: Fixed D3 scale replacement issue causing incorrect bar positioning

[2025.11.05]

Added

  • Dummy Coding: Transform categorical variables into dummy variables for GLM analysis
    • Option to keep original columns alongside dummy variables
    • Scale transformation functionality for categorical variables
    • Boolean columns included as-is by default
  • Dataset Metadata: Display dataset information modal showing row count, column count, data types, and other metadata
  • GLM: Web Worker implementation for non-blocking computation
    • Real-time progress tracking with iteration details
    • Convergence history saved as dataset for analysis
    • Coefficient display organized by variable in convergence history table

Fixed

  • Convert Column Type: Fixed error when dataset names contain spaces
  • Filtered Data: Fixed issue where filtered datasets created from DerivedDataSets failed to load after saving and reopening project files
  • GLM: Fixed handling of negative predicted values (μ<0) in Gaussian GLM
  • GLM Progress Dialog: Fixed layout shift during progress updates
  • GLM Convergence History: Fixed data type issues with theta_iter column and improved log-likelihood display
  • Project View: Prevented unintended scrolling behavior
  • Dummy Coding: Fixed persistence of dummy-coded datasets in project files

Performance

  • GLM: 4.7x performance improvement using TypedArray optimization

[2025.11.01]

Added

  • Release Notes: View release history from Help menu and About dialog

[2025.10.31]

Added

  • Graph Builder: Filter functionality to subset data in visualizations
  • Graph Builder - Scatter Plot: Display count of overlapping points in tooltips
  • File Format: MDS project files are now compressed with gzip, reducing file size

Fixed

  • Project Management: Project names are now properly saved and displayed in the project list
  • Project Loading: Improved performance by loading metadata only when listing projects

[2025.10.23]

Added

  • Graph Builder - Bar Chart: Top N filtering and custom sorting options
  • SQL Editor: Overwrite existing datasets functionality
  • Project Lineage: Display dependencies in duplicate name dialog

Fixed

  • Performance: Fixed stack overflow when clicking interval scale columns with large datasets
  • Graph Builder: Fixed issue where user-selected DerivedDataSet was ignored
  • Data Table: Fixed abnormally large column headers
  • Data Table: Improved row number header z-index for better scrolling display
  • UI: Hidden scrollbar in tab header area
  • SQL Editor: Fixed initial query not displaying
  • SQL Query Viewer: Improved color scheme and spacing

[2025.10.22]

Added

  • GLM: Prediction functionality for trained models
  • Graph Builder - Histogram: Faceted mode with statistical annotations
  • Graph Builder - Pair Plot: Multi-variable scatter plot matrix visualization
  • Graph Builder - Scatter Plot: 2D density visualization with configurable color scales
  • SQL Editor: Integrated CodeMirror for improved editing experience
  • Data Management: Dataset deletion with dependency checking

Fixed

  • GLM: Store feature metadata by column ID, resolve names from training dataset during prediction
  • SQL: Allow referencing unevaluated DerivedDataSet in FROM clause
  • Pair Plot: Unified axis positions and colors between histograms and scatter plots
  • UI: Export project dialog now properly displays as single modal

[2025.10.19]

Fixed

  • Sample Data: Fixed country name formatting in Gapminder dataset

[2025.10.18]

Added

  • Initial release