jfraeysd/fetch_ml
0
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
fetch_ml/docs/_site/adr/ADR-002-use-sqlite-for-local-development/index.html
Jeremie Fraeys ea15af1833 Fix multi-user authentication and clean up debug code 
- Fix YAML tags in auth config struct (json -> yaml)
- Update CLI configs to use pre-hashed API keys
- Remove double hashing in WebSocket client
- Fix port mapping (9102 -> 9103) in CLI commands
- Update permission keys to use jobs:read, jobs:create, etc.
- Clean up all debug logging from CLI and server
- All user roles now authenticate correctly:
  * Admin: Can queue jobs and see all jobs
  * Researcher: Can queue jobs and see own jobs
  * Analyst: Can see status (read-only access)

Multi-user authentication is now fully functional.
2025-12-06 12:35:32 -05:00

1922 lines
No EOL
36 KiB
HTML
Raw Blame History

<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="Secure Machine Learning Platform">
<link rel="prev" href="../ADR-001-use-go-for-api-server/">
<link rel="next" href="../ADR-003-use-redis-for-job-queue/">
<link rel="icon" href="../../assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.1, mkdocs-material-9.7.0">
<title>ADR-002: Use SQLite for Local Development - Fetch ML Documentation</title>
<link rel="stylesheet" href="../../assets/stylesheets/main.618322db.min.css">
<link rel="stylesheet" href="../../assets/stylesheets/palette.ab4e12ef.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<script>__md_scope=new URL("../..",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="default" data-md-color-primary="blue" data-md-color-accent="blue">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
<a href="#adr-002-use-sqlite-for-local-development" class="md-skip">
Skip to content
</a>
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="../.." title="Fetch ML Documentation" class="md-header__button md-logo" aria-label="Fetch ML Documentation" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
Fetch ML Documentation
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
ADR-002: Use SQLite for Local Development
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="" data-md-color-scheme="default" data-md-color-primary="blue" data-md-color-accent="blue" aria-hidden="true" type="radio" name="__palette" id="__palette_0">
<input class="md-option" data-md-color-media="" data-md-color-scheme="slate" data-md-color-primary="blue" data-md-color-accent="blue" aria-hidden="true" type="radio" name="__palette" id="__palette_1">
</form>
<script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<a href="javascript:void(0)" class="md-search__icon md-icon" title="Share" aria-label="Share" data-clipboard data-clipboard-text="" data-md-component="search-share" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M18 16.08c-.76 0-1.44.3-1.96.77L8.91 12.7c.05-.23.09-.46.09-.7s-.04-.47-.09-.7l7.05-4.11c.54.5 1.25.81 2.04.81a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3c0 .24.04.47.09.7L8.04 9.81C7.5 9.31 6.79 9 6 9a3 3 0 0 0-3 3 3 3 0 0 0 3 3c.79 0 1.5-.31 2.04-.81l7.12 4.15c-.05.21-.08.43-.08.66 0 1.61 1.31 2.91 2.92 2.91s2.92-1.3 2.92-2.91A2.92 2.92 0 0 0 18 16.08"/></svg>
</a>
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://github.com/jfraeys/fetch_ml" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
GitHub
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<nav class="md-tabs" aria-label="Tabs" data-md-component="tabs">
<div class="md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="../.." class="md-tabs__link">
Home
</a>
</li>
<li class="md-tabs__item">
<a href="../../quick-start/" class="md-tabs__link">
Getting Started
</a>
</li>
<li class="md-tabs__item">
<a href="../../development-setup/" class="md-tabs__link">
Development
</a>
</li>
<li class="md-tabs__item">
<a href="../../deployment/" class="md-tabs__link">
Operations & Production
</a>
</li>
<li class="md-tabs__item">
<a href="../../security/" class="md-tabs__link">
Security
</a>
</li>
<li class="md-tabs__item">
<a href="../../configuration-schema/" class="md-tabs__link">
Reference
</a>
</li>
<li class="md-tabs__item md-tabs__item--active">
<a href="../" class="md-tabs__link">
Architecture Decisions
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary md-nav--lifted md-nav--integrated" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="../.." title="Fetch ML Documentation" class="md-nav__button md-logo" aria-label="Fetch ML Documentation" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
Fetch ML Documentation
</label>
<div class="md-nav__source">
<a href="https://github.com/jfraeys/fetch_ml" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
GitHub
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../.." class="md-nav__link">
<span class="md-ellipsis">
Home
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_2" >
<label class="md-nav__link" for="__nav_2" id="__nav_2_label" tabindex="0">
<span class="md-ellipsis">
Getting Started
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_2_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_2">
<span class="md-nav__icon md-icon"></span>
Getting Started
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../quick-start/" class="md-nav__link">
<span class="md-ellipsis">
Quick Start
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../installation/" class="md-nav__link">
<span class="md-ellipsis">
Simple Installation Guide
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../first-experiment/" class="md-nav__link">
<span class="md-ellipsis">
First Experiment
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3" id="__nav_3_label" tabindex="0">
<span class="md-ellipsis">
Development
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
Development
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../development-setup/" class="md-nav__link">
<span class="md-ellipsis">
Development Setup
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../testing/" class="md-nav__link">
<span class="md-ellipsis">
Testing Guide
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../architecture/" class="md-nav__link">
<span class="md-ellipsis">
Homelab Architecture
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../cli-reference/" class="md-nav__link">
<span class="md-ellipsis">
CLI Reference
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../zig-cli/" class="md-nav__link">
<span class="md-ellipsis">
Zig CLI Guide
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../queue/" class="md-nav__link">
<span class="md-ellipsis">
Task Queue Architecture
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../smart-defaults/" class="md-nav__link">
<span class="md-ellipsis">
Smart Defaults
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../cicd/" class="md-nav__link">
<span class="md-ellipsis">
CI/CD Pipeline
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_4" >
<label class="md-nav__link" for="__nav_4" id="__nav_4_label" tabindex="0">
<span class="md-ellipsis">
Operations & Production
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_4_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_4">
<span class="md-nav__icon md-icon"></span>
Operations & Production
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../deployment/" class="md-nav__link">
<span class="md-ellipsis">
ML Experiment Manager - Deployment Guide
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../environment-variables/" class="md-nav__link">
<span class="md-ellipsis">
Environment Variables
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../production-monitoring/" class="md-nav__link">
<span class="md-ellipsis">
Production Monitoring Deployment Guide (Linux)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../operations/" class="md-nav__link">
<span class="md-ellipsis">
Operations Runbook
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../redis-ha/" class="md-nav__link">
<span class="md-ellipsis">
Redis High Availability (Optional)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../release-checklist/" class="md-nav__link">
<span class="md-ellipsis">
Release Checklist
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_5" >
<label class="md-nav__link" for="__nav_5" id="__nav_5_label" tabindex="0">
<span class="md-ellipsis">
Security
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_5_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_5">
<span class="md-nav__icon md-icon"></span>
Security
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../security/" class="md-nav__link">
<span class="md-ellipsis">
Security Guide
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../api-key-process/" class="md-nav__link">
<span class="md-ellipsis">
FetchML API Key Process
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../user-permissions/" class="md-nav__link">
<span class="md-ellipsis">
User Permissions in Fetch ML
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle md-toggle--indeterminate" type="checkbox" id="__nav_6" >
<label class="md-nav__link" for="__nav_6" id="__nav_6_label" tabindex="0">
<span class="md-ellipsis">
Reference
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_6_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_6">
<span class="md-nav__icon md-icon"></span>
Reference
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../../configuration-schema/" class="md-nav__link">
<span class="md-ellipsis">
Configuration Schema
</span>
</a>
</li>
<li class="md-nav__item">
<a href="../../troubleshooting/" class="md-nav__link">
<span class="md-ellipsis">
Troubleshooting
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item md-nav__item--active md-nav__item--section md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_7" checked>
<div class="md-nav__link md-nav__container">
<a href="../" class="md-nav__link ">
<span class="md-ellipsis">
Architecture Decisions
</span>
</a>
<label class="md-nav__link " for="__nav_7" id="__nav_7_label" tabindex="">
<span class="md-nav__icon md-icon"></span>
</label>
</div>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_7_label" aria-expanded="true">
<label class="md-nav__title" for="__nav_7">
<span class="md-nav__icon md-icon"></span>
Architecture Decisions
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="../ADR-001-use-go-for-api-server/" class="md-nav__link">
<span class="md-ellipsis">
ADR-001: Use Go for API Server
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--active">
<input class="md-nav__toggle md-toggle" type="checkbox" id="__toc">
<label class="md-nav__link md-nav__link--active" for="__toc">
<span class="md-ellipsis">
ADR-002: Use SQLite for Local Development
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<a href="./" class="md-nav__link md-nav__link--active">
<span class="md-ellipsis">
ADR-002: Use SQLite for Local Development
</span>
</a>
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
<label class="md-nav__title" for="__toc">
<span class="md-nav__icon md-icon"></span>
Table of contents
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item">
<a href="#status" class="md-nav__link">
<span class="md-ellipsis">
Status
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#context" class="md-nav__link">
<span class="md-ellipsis">
Context
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#decision" class="md-nav__link">
<span class="md-ellipsis">
Decision
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#consequences" class="md-nav__link">
<span class="md-ellipsis">
Consequences
</span>
</a>
<nav class="md-nav" aria-label="Consequences">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#positive" class="md-nav__link">
<span class="md-ellipsis">
Positive
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#negative" class="md-nav__link">
<span class="md-ellipsis">
Negative
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#options-considered" class="md-nav__link">
<span class="md-ellipsis">
Options Considered
</span>
</a>
<nav class="md-nav" aria-label="Options Considered">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#postgresql" class="md-nav__link">
<span class="md-ellipsis">
PostgreSQL
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#mysql" class="md-nav__link">
<span class="md-ellipsis">
MySQL
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#in-memory-databases-redis-etc" class="md-nav__link">
<span class="md-ellipsis">
In-memory databases (Redis, etc.)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#no-database-file-based-storage" class="md-nav__link">
<span class="md-ellipsis">
No database (file-based storage)
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#rationale" class="md-nav__link">
<span class="md-ellipsis">
Rationale
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="../ADR-003-use-redis-for-job-queue/" class="md-nav__link">
<span class="md-ellipsis">
ADR-003: Use Redis for Job Queue
</span>
</a>
</li>
</ul>
</nav>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1 id="adr-002-use-sqlite-for-local-development">ADR-002: Use SQLite for Local Development<a class="headerlink" href="#adr-002-use-sqlite-for-local-development" title="Permanent link">&para;</a></h1>
<h2 id="status">Status<a class="headerlink" href="#status" title="Permanent link">&para;</a></h2>
<p>Accepted</p>
<h2 id="context">Context<a class="headerlink" href="#context" title="Permanent link">&para;</a></h2>
<p>For local development and testing, we needed a database solution that:
- Requires minimal setup and configuration
- Works well with Go's database drivers
- Supports the same SQL features as production databases
- Allows easy reset and recreation of test data
- Doesn't require external services running locally</p>
<h2 id="decision">Decision<a class="headerlink" href="#decision" title="Permanent link">&para;</a></h2>
<p>We chose SQLite as the default database for local development and testing environments.</p>
<h2 id="consequences">Consequences<a class="headerlink" href="#consequences" title="Permanent link">&para;</a></h2>
<h3 id="positive">Positive<a class="headerlink" href="#positive" title="Permanent link">&para;</a></h3>
<ul>
<li>Zero configuration - database is just a file</li>
<li>Fast performance for local development workloads</li>
<li>Easy to reset by deleting the database file</li>
<li>Excellent Go driver support (mattn/go-sqlite3)</li>
<li>Supports most SQL features we need</li>
<li>Portable across different development machines</li>
<li>No external dependencies or services to manage</li>
</ul>
<h3 id="negative">Negative<a class="headerlink" href="#negative" title="Permanent link">&para;</a></h3>
<ul>
<li>Limited to single connection at a time (file locking)</li>
<li>Not suitable for production multi-user scenarios</li>
<li>Some advanced SQL features may not be available</li>
<li>Different behavior compared to PostgreSQL in production</li>
</ul>
<h2 id="options-considered">Options Considered<a class="headerlink" href="#options-considered" title="Permanent link">&para;</a></h2>
<h3 id="postgresql">PostgreSQL<a class="headerlink" href="#postgresql" title="Permanent link">&para;</a></h3>
<p><strong>Pros:</strong>
- Production-grade database
- Excellent feature support
- Good Go driver support
- Consistent with production environment</p>
<p><strong>Cons:</strong>
- Requires external service installation and configuration
- Higher resource usage
- More complex setup for new developers
- Overkill for simple local development</p>
<h3 id="mysql">MySQL<a class="headerlink" href="#mysql" title="Permanent link">&para;</a></h3>
<p><strong>Pros:</strong>
- Popular and well-supported
- Good Go drivers available</p>
<p><strong>Cons:</strong>
- Requires external service
- More complex setup
- Different SQL dialect than PostgreSQL</p>
<h3 id="in-memory-databases-redis-etc">In-memory databases (Redis, etc.)<a class="headerlink" href="#in-memory-databases-redis-etc" title="Permanent link">&para;</a></h3>
<p><strong>Pros:</strong>
- Very fast
- No persistence needed for some tests</p>
<p><strong>Cons:</strong>
- Limited query capabilities
- Not suitable for complex relational data
- Different data model than production</p>
<h3 id="no-database-file-based-storage">No database (file-based storage)<a class="headerlink" href="#no-database-file-based-storage" title="Permanent link">&para;</a></h3>
<p><strong>Pros:</strong>
- Simple implementation
- No dependencies</p>
<p><strong>Cons:</strong>
- Limited query capabilities
- No transaction support
- Hard to scale to complex data needs</p>
<h2 id="rationale">Rationale<a class="headerlink" href="#rationale" title="Permanent link">&para;</a></h2>
<p>SQLite provides the perfect balance of simplicity and functionality for local development. It requires zero setup - developers can just run the application and it works. The file-based nature makes it easy to reset test data by deleting the database file. While it differs from our production PostgreSQL database, it supports the same core SQL features needed for development and testing.</p>
<p>The main limitation is single-writer access, but this is acceptable for local development where typically only one developer is working with the database at a time. For integration tests that need concurrent access, we can use PostgreSQL or Redis.</p>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"annotate": null, "base": "../..", "features": ["navigation.instant", "navigation.tracking", "navigation.tabs", "navigation.sections", "navigation.expand", "navigation.indexes", "toc.integrate", "search.highlight", "search.share"], "search": "../../assets/javascripts/workers/search.7a47a382.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="../../assets/javascripts/bundle.e71a0d61.min.js"></script>
</body>
</html>
Reference in a new issue View git blame Copy permalink