# Data Layer: Storage and Persistence Rules Single abstraction for reading and writing app data so **guest** (localStorage + in-memory) and **auth** (Supabase) stay in sync. All project-scoped writes go through this layer; features do not implement their own "if Supabase then persist, else localStorage." **Cursor rule:** `.cursor/rules/data-layer.mdc` — follow when adding or changing code that reads/writes project data, stakeholders, checklists, etc. --- ## The rule **For any project-scoped write:** update local state (in-memory / localStorage) for guest, and if `projectId` is a UUID and Supabase client exists, persist that entity to Supabase. The data layer owns both. Call the layer; don’t duplicate the logic in features. --- ## Who should read - **Projects list:** `getStoredProjects()` (or `Blueprint.storage.getProjects()` when using storage API) - **Current project id:** `getResolvedCurrentProjectId()` or `Blueprint.currentProject.get()` — see [Unified Current Project API](UNIFIED_CURRENT_PROJECT_API.md) - **Frameworks:** `getStoredFrameworks()` / `Blueprint.storage.getFrameworks()` - **Guides:** `getStoredGuides()` / `Blueprint.storage.getGuides()` - **Checklist sections / GA audit standard / interview bank, etc.:** use `Blueprint.data.getChecklistSections()`, `Blueprint.data.getGaAuditStandardChecklist()`, etc. as documented in the code. --- ## Who should write (use the data layer) Use these instead of manually calling `setStoredProjects` and then remembering to call a persist function. | Entity | Read | Write (data layer) | |--------|------|--------------------| | **Project (full or patch)** | `getStoredProjects()`, find by id | `saveProject(projectId, patch)` — updates local and persists to Supabase when UUID + client | | **Stakeholders** | From project: `project.stakeholders` | `saveProjectStakeholders(projectId, list)` | | **Implementation checklist** | `Blueprint.storage.getChecks()` or project.checklist | `saveProjectChecklist(projectId, checklist)` or `Blueprint.storage.setChecks(checks)` (which uses saveProjectChecklist internally). **One checklist per project** (regardless of number of GA audits or other data). | | **Projects list (replace whole list)** | `getStoredProjects()` | `setStoredProjects(projects)` — already updates local and, when useSupabaseData + client, persists via `persistProjectsToSupabase` | | **Frameworks** | `getStoredFrameworks()` | `setStoredFrameworks(frameworks)` — same pattern | | **Guides** | `getStoredGuides()` | `setStoredGuides(guides)` — same pattern | When you add a new project-scoped field (e.g. GA audit checkbox state), add a `saveProjectX(projectId, value)` that updates local project and calls the corresponding persist when `projectId` is a UUID. --- ## Where things live | Storage | Guest | Auth (signed in) | |---------|--------|-------------------| | **Projects** | localStorage `crossDomainProjects` + in-memory | Supabase `projects` + in-memory `Blueprint.supabaseProjects` | | **Stakeholders** | On project in localStorage / in-memory | On project in Supabase `projects.stakeholders` | | **Implementation checklist** | On project in localStorage / in-memory | Supabase `projects.checklist` | | **Frameworks** | localStorage `crossDomainFrameworks` | Supabase `frameworks` | | **Guides** | In-memory only (no localStorage for list when Supabase in use) | Supabase `guides` + in-memory `Blueprint.supabaseGuides` | | **GA audit (standard sections)** | User override in localStorage / user_settings | Supabase `user_settings` or `app_settings` | | **GA audit (per-project state)** | Project field / localStorage `gaAuditChecklist` | Project row when column exists; otherwise local only | | **Interview question bank / templates** | localStorage + user_settings | Supabase `user_settings` | --- ## Checklist when adding a feature 1. **Read** via the data layer: `getStoredProjects()`, `getResolvedCurrentProjectId()`, or the appropriate getter. 2. **Write** via the data layer: `saveProject()`, `saveProjectStakeholders()`, `saveProjectChecklist()`, or `setStoredProjects` / `setStoredFrameworks` / `setStoredGuides` for full-list updates. 3. Do **not** only call `setStoredProjects(projects)` after mutating without going through a save helper when the mutation is a single-entity update (e.g. stakeholders or checklist). Use the helper so Supabase persist happens when projectId is a UUID. --- ## Checklist when fixing "only in this browser" or "not syncing" 1. **Local state:** Did we update in-memory and (for guest) localStorage? 2. **Supabase:** For UUID projects and when client exists, did we persist the same entity (e.g. stakeholders, checklist)? 3. Prefer adding or using a shared **save-this-entity** function instead of ad-hoc persist calls. --- ## Saving pattern (use everywhere) **Rule:** For any write (add, update, or **delete**), the data layer must: 1. **Update local state first** (in-memory, and localStorage for guest where applicable). 2. **Then** perform the remote operation (Supabase persist or delete). **Why:** If you do the remote operation first and only update local state in the callback, another path (e.g. a save, auto-save, or persist triggered elsewhere) can run with the **old** list and re-insert or overwrite the change. Updating local state first ensures `getStored*()` never returns stale data, so any concurrent persist uses the correct list. --- ## Delete / remove pattern (guides, frameworks, list items) When the user deletes an item (e.g. a guide, a framework, or an item in a list), follow this order so the delete survives refresh and no concurrent persist re-inserts the item. ### Steps (follow in order) 1. **Capture identity** before changing anything: e.g. `var idToDelete = currentGuideId;` and, if you need it for a DB lookup, the full item `var itemToDelete = guides.filter(...)[0];`. 2. **Update in-memory immediately:** Compute the new list without the item (e.g. `var filtered = guides.filter(function (g) { return g.id !== idToDelete; });`), then set the canonical in-memory store (e.g. `Blueprint.supabaseGuides = filtered.slice();`) and call the list setter (e.g. **setStoredGuides(filtered)**). The setter will perform the Supabase delete for the removed id(s) and persist the remaining list. 3. **Update UI immediately:** Clear selection, refresh dropdowns/lists, show toast (e.g. "Deleted"). Do not wait for the Supabase delete promise. 4. **Then** (optionally) fire the remote delete if your setter does not already do it (e.g. `storage.setGuides` already deletes by `id` for each `deletedId`; you may still call `client.from('guides').delete().eq('id', idToDelete)` for clarity or to ensure the row is gone). In the delete’s `.then()`, only handle errors (e.g. log or show toast); do **not** call the list setter again or update local state again. ### Do not - **Do not** update in-memory only inside the Supabase delete `.then()`. That leaves a window where another handler can call `persist*(getStored*())` with the old list and re-insert the item. - **Do not** use localStorage for the guides/frameworks list when Supabase is in use; the source of truth is in-memory (`Blueprint.supabaseGuides` / `Blueprint.supabaseFrameworks`) and Supabase. The app does not read or write the guides list to localStorage. ### Reference - **Guides delete:** `js/app-init.js` — guide delete button: capture id/item → `setStoredGuides(filtered)` + UI update + toast → then Supabase delete. No state update inside the delete `.then()`. --- ## Implementation checklist (no errors / checks not taking) - **One checklist per project:** Each project has a single `project.checklist`; there is only one per project regardless of how many GA audits or other data exist. - **Read:** Use `Blueprint.storage.getChecks()` or `getStoredChecks()` (they use the current project from `getResolvedCurrentProjectId()`). - **Write:** Use `Blueprint.storage.setChecks(checks)` or `saveProjectChecklist(projectId, checklist)`. Do not mutate `project.checklist` and then only call `setStoredProjects`; use the data layer so Supabase persist runs when the project is a UUID. - If checkboxes don’t persist: ensure the checklist section uses `setStoredChecks` / `Blueprint.storage.setChecks` on toggle, and that a project is selected (so `getCurrentProjectIdForStorage()` returns a valid id). See **js/storage.js** (`getChecks`/`setChecks`) and **js/app-data.js** (`saveProjectChecklist`, `persistProjectChecklist`). ## GA Audit checklist (one per project) - **One checklist per project:** There are many audit *runs*, but the GA Audit checklist **checked state** is a single object per project (`project.ga_audit_checklist`). It survives refresh when persisted to Supabase. - **Read:** `getGaAuditChecklistState()` (app-init.js) returns the current project’s `ga_audit_checklist` from `getStoredProjects()`. - **Write:** `setGaAuditChecklistState(state)` → `saveProjectGaAuditChecklist(projectId, state)` → updates local project and calls `persistProjectGaAuditChecklist` when project is a UUID. - **Database:** Requires **migration 080** (`080_projects_ga_audit_checklist.sql`): adds `projects.ga_audit_checklist` JSONB. Run it in Supabase Dashboard → SQL Editor if the column is missing; otherwise persist will fail and checks won’t survive refresh. --- ## Adding new checklist functionality (step-by-step) When adding a **new** project-scoped checklist (or any “checked state” that should persist per project and survive refresh), follow these steps exactly so data persists and the UI always reflects the current project. ### 1. Database - Add a **migration** that adds a JSONB column to `public.projects`, e.g. `my_checklist JSONB DEFAULT '{}'`. - Run the migration in Supabase Dashboard → SQL Editor (or your normal migration path). - If the column is missing, persist will fail and checks will not survive refresh. ### 2. Map DB ↔ app shape (js/app-data.js) - **rowToProject(r):** Map the DB column into the project object, e.g. `myChecklist: (r.my_checklist && typeof r.my_checklist === 'object') ? r.my_checklist : {}`. - **projectToRow(p):** If this checklist is included in full project upserts, add the field; otherwise document “persisted only via persistProjectMyChecklist” (like `ga_audit_checklist`). ### 3. Save helper (js/app-data.js) - Add **saveProjectMyChecklist(projectId, state)** that: - Returns early if `!projectId`. - Gets projects via `getStoredProjects()`, finds the project by id, sets `projects[idx].myChecklist = payload` (and `updatedAt`), calls **setStoredProjects(projects)**. - If `isUuid(projectId)`, calls **persistProjectMyChecklist(projectId, payload)**. - Do **not** mutate the project and then only call `setStoredProjects` without a persist; the save helper must call the persist when the project is a UUID. ### 4. Persist function (js/app-data.js) - Add **persistProjectMyChecklist(projectId, state)** that: - Returns early if `!Blueprint.useSupabaseData || !projectId || !isUuid(projectId)` or no Supabase client. - **Do not** add a guard on `Blueprint.projectsLoadedFromSupabase === false`. If you do, persist will be skipped during or right after load and checks will not survive refresh. - Calls `client.from('projects').update({ my_checklist: payload, updated_at: updatedAt }).eq('id', projectId)` and handles errors (e.g. log or warn). ### 5. Read path (get state for current project) - Add a getter (e.g. **getMyChecklistState()**) that: - Gets current project id via **getResolvedCurrentProjectId()** (or equivalent). - If no projectId, return a safe default (e.g. `{}`). - Gets projects via **getStoredProjects()**, finds the project by id, returns `project.myChecklist` or `{}`. - **Do not** return empty just because “load is in progress” or a flag like `projectsLoadedFromSupabase === false`; that causes the UI to show no checks and can overwrite state. ### 6. Write path (checkbox / UI toggle) - On toggle, call a setter (e.g. **setMyChecklistState(state)**) that: - Gets current project id via **getResolvedCurrentProjectId()**. - If no projectId, optionally write to localStorage only or no-op. - Otherwise calls **saveProjectMyChecklist(projectId, state)** (so local state and Supabase persist both happen). ### 7. Restore state and re-render when project changes (js/app-init.js loadProject) - In **loadProject(projectId)**: - After loading the project `p`, set the checklist state from the project: e.g. **setMyChecklistState(p.myChecklist || {})**. - Then **force a re-render** of the checklist UI (e.g. **requestMyChecklistRender(null, true)** or equivalent with `force: true`). If you only request a render without `force`, the “already rendered, no result change” logic may skip re-rendering and the DOM will keep showing the **previous** project’s checkboxes even though the in-memory state is correct. ### 8. Render the checklist UI - When building the checklist HTML, call the **getter** (e.g. getMyChecklistState()) at **render time** and use that to set each item’s checked state (e.g. `isChecked`, CSS class `checked`). - Ensure the checklist is re-rendered when the section is shown or when the project changes (see step 7); if the render is skipped when content “already exists,” pass **force: true** on project load/switch so the DOM is rebuilt from the current project’s state. ### 9. Summary table (for your new checklist) | Step | What | Why | |------|------|-----| | 1 | Migration: add JSONB column to `projects` | Persist has a target column; without it, checks don’t survive refresh. | | 2 | rowToProject / projectToRow | Load and (if used) full upsert see the field. | | 3 | saveProjectX(projectId, value) | Single place that updates local + persist when UUID. | | 4 | persistProjectX — no `projectsLoadedFromSupabase` guard | Guard causes persist to be skipped; checks then don’t survive refresh. | | 5 | Getter uses current project only; no “loading” empty return | Prevents empty state overwriting real data and wrong UI. | | 6 | Setter calls saveProjectX on toggle | Ensures every toggle persists. | | 7 | loadProject: set state then **force** re-render | Without force, UI keeps showing previous project’s checkboxes. | | 8 | Render uses getter at render time; force re-render on project change | DOM always matches current project’s state. | Reference implementations: **Implementation checklist** (storage.js getChecks/setChecks, app-data saveProjectChecklist/persistProjectChecklist), **GA Audit checklist** (app-init getGaAuditChecklistState/setGaAuditChecklistState, app-data saveProjectGaAuditChecklist/persistProjectGaAuditChecklist, loadProject + requestGaAuditChecklistRender(null, true)). --- ## API reference (js/app-data.js) - `saveProject(projectId, patch)` — merge `patch` onto project, update list, persist full projects when Supabase in use - `saveProjectStakeholders(projectId, list)` — set project.stakeholders, update list, call `persistProjectStakeholders` when UUID - `saveProjectChecklist(projectId, checklist)` — set project.checklist, update list, call `persistProjectChecklist` when UUID - `getStoredProjects()` / `setStoredProjects(projects)` — full list read/write (setStoredProjects already persists when useSupabaseData + client) - `persistProjectStakeholders(projectId, list)` — Supabase-only update (normally use `saveProjectStakeholders`) - `persistProjectChecklist(projectId, checklist)` — Supabase-only update (normally use `saveProjectChecklist`) Guides and frameworks: `setStoredGuides` / `setStoredFrameworks` already update local and persist when client exists; use them for full-list updates. --- ## Migrated call sites The following use the data layer (with fallback to legacy path when the layer is not available): - **app.js:** `syncStakeholderListFromDom`, `saveStakeholderTakeawaysModal` → `saveProjectStakeholders`; project name, notes, clientName, clientCompany, webUrl, notes modal, measurementId, apiSecret, ga4PropertyId, containerId → `saveProject`; `setStoredChecks` → `saveProjectChecklist`. - **js/app-init.js:** Add/remove stakeholder, stakeholder blur (role/name/focus/interviewed) → `saveProjectStakeholders`. - **js/storage.js:** `Blueprint.storage.setChecks` → `saveProjectChecklist` when a project is selected.