Self-Learning Intelligence Architecture

Raw source
-> project/source attribution
-> structured signal extraction
-> packet/card compilation
-> assistant/tool response
-> user action, correction, rejection, or outcome
-> feedback event
-> promotion review
-> active memory, retrieval weight, prompt constraint, or workflow rule

​

Current Tables To Build On

​

Raw Knowledge And Retrieval

​

Packet-First Project Intelligence

​

Compiler And Attribution Ledgers

​

Existing Feedback And Memory

​

Additive Tables To Build First

​

ai_feedback_events

create table public.ai_feedback_events (
  id uuid primary key default gen_random_uuid(),
  created_at timestamptz not null default now(),
  user_id uuid null references auth.users(id) on delete set null,
  project_id integer null references public.projects(id) on delete set null,
  target_id uuid null references public.intelligence_targets(id) on delete set null,
  session_id uuid null,
  source_table text null,
  source_record_id text null,
  event_type text not null,
  event_family text not null,
  surface text not null,
  subject_type text not null,
  subject_id text null,
  signal text not null,
  reason_category text null,
  free_text text null,
  before_snapshot jsonb not null default '{}'::jsonb,
  after_snapshot jsonb not null default '{}'::jsonb,
  source_context jsonb not null default '{}'::jsonb,
  metadata jsonb not null default '{}'::jsonb
);

• retrieval
• attribution
• assistant_response
• tool_action
• task_generation
• packet_quality
• document_review
• user_preference
• workflow_outcome
• eval_failure

• positive
• negative
• corrected
• accepted
• ignored
• completed
• failed
• needs_review
• stale
• conflicting

• (project_id, event_family, created_at desc)
• (target_id, event_family, created_at desc)
• (subject_type, subject_id, created_at desc)
• (source_table, source_record_id)
• (event_type, signal, created_at desc)

• service role writes all events
• users can insert events for their own interactions
• admins can read all
• project members can read project-scoped non-private events
• private preference/memory events require user ownership or admin access

​

ai_learning_promotions

create table public.ai_learning_promotions (
  id uuid primary key default gen_random_uuid(),
  created_at timestamptz not null default now(),
  reviewed_at timestamptz null,
  reviewed_by uuid null references auth.users(id) on delete set null,
  status text not null default 'candidate',
  promotion_type text not null,
  project_id integer null references public.projects(id) on delete set null,
  target_id uuid null references public.intelligence_targets(id) on delete set null,
  source_event_ids uuid[] not null default '{}',
  destination_table text null,
  destination_record_id text null,
  confidence numeric not null default 0.5,
  risk_level text not null default 'low',
  proposed_learning jsonb not null,
  review_notes text null,
  expires_at timestamptz null,
  superseded_by uuid null references public.ai_learning_promotions(id)
);

• agent_learnings for prevention prompts and repeated failure patterns
• ai_memories for user/team facts, preferences, lessons, and commitments
• ai_task_feedback.promoted = true for approved positive task examples
• document_attribution_candidates.status = approved|rejected for project-linking corrections
• insight_cards / insight_card_evidence for packet-quality corrections
• future retrieval scoring table for source/chunk ranking rules

​

ai_retrieval_feedback

create table public.ai_retrieval_feedback (
  id uuid primary key default gen_random_uuid(),
  created_at timestamptz not null default now(),
  user_id uuid null references auth.users(id) on delete set null,
  project_id integer null references public.projects(id) on delete set null,
  target_id uuid null references public.intelligence_targets(id) on delete set null,
  session_id uuid null,
  tool_name text not null,
  query_text text not null,
  source_document_id text null references public.document_metadata(id) on delete set null,
  source_chunk_id text null,
  rank integer null,
  score numeric null,
  cited boolean not null default false,
  user_referenced boolean not null default false,
  used_in_answer boolean not null default false,
  outcome text not null default 'unknown',
  metadata jsonb not null default '{}'::jsonb
);

• helpful
• unhelpful
• wrong_project
• stale
• unsupported
• unknown

​

Ingestion Flows

​

Microsoft Graph Teams, Outlook, And SharePoint

1. Backend Graph service ingests messages, files, and attachments.
2. Source row lands in document_metadata with source IDs, source URL/path, title/subject, participants, source timestamps, source_system, source_metadata, and raw_text or summary.
3. Text is chunked into document_chunks when it has durable retrieval value.
4. Project attribution scores explicit IDs, project aliases, client names, addresses, participants, thread/channel/path/title context.
5. High-confidence attribution updates document_metadata.project_id.
6. Medium/low-confidence attribution creates document_attribution_candidates and an ai_feedback_events review signal.
7. source_intelligence_jobs queues signal extraction.
8. High-confidence signals promote to insight_cards and insight_card_evidence.
9. packet_refresh_jobs queues packet regeneration.

​

Fireflies Meetings

1. Transcript/summary lands in document_metadata.
2. Transcript content chunks into document_chunks.
3. Meeting participants and title drive project attribution.
4. Extract decisions, commitments, open questions, schedule risk, financial exposure, and owner/action signals.
5. Promote supported items to cards only when an evidence excerpt exists.
6. Later user corrections write ai_feedback_events, ai_retrieval_feedback, and candidate promotions when repeated or high-impact.

​

User Chat And AI Assistant

1. User message enters frontend/src/app/api/ai-assistant/chat/route.ts.
2. Intent chooses packet-first advisor path, source lookup path, action/tool path, or memory path.
3. Assistant metadata records tool trace, retrieved sources, response quality, data parts/widgets, and selected learnings.
4. Post-response jobs run conversation memory extraction, learning usage recording, and retrieval trace recording.
5. User feedback, edit, regeneration, copied answer, accepted widget, or ignored action writes ai_feedback_events.
6. Approved promotions write to agent_learnings, ai_memories, or a domain table.

​

Task Generation

1. Assistant proposes or creates a task through action tools.
2. Task snapshot is captured in ai_task_feedback when user rates it.
3. Good feedback can become a promoted positive example after review.
4. Bad feedback creates or updates an agent_learnings candidate with a prevention prompt.
5. Future task creation pulls promoted positive examples, active agent learnings, and current project packet/source evidence.

​

Project Intelligence Pages

1. User views a packet, opens a card, expands evidence, follows source links, or marks a card wrong/stale/useful.
2. Each meaningful interaction writes ai_feedback_events.
3. Card corrections update intelligence_reviews.
4. Approved corrections update cards, evidence, or attribution candidates.
5. Packet refresh regenerates the target packet and records why.

​

Feedback Events To Capture First

​

Promotion Rules

1. Human review approves it.
2. The same failure repeats at least twice with similar signatures.
3. A high-confidence workflow outcome proves it worked.
4. A deterministic eval fails and maps to a clear prevention rule.

​

Promote To agent_learnings

​

Promote To ai_memories

​

Promote Positive Task Examples

​

Promote Retrieval Weight

​

Promote Attribution Rules

​

Promote Packet Rules

• Put financial exposure before schedule commentary for owner updates.
• Do not show Teams-only risk as high confidence without a confirming source.
• Move stale cards older than 14 days without new evidence below active blockers.

​

Fail-Loud Rules

​

Verification Gates To Build First

​

Gate 1: Schema And RLS

npx supabase gen types typescript --project-id "lgveqfnpkxvzbnnwuled" --schema public > frontend/src/types/database.types.ts
npm run db:migrations:verify-applied -- supabase/migrations/<new_feedback_migration>.sql

• ai_feedback_events, ai_learning_promotions, and ai_retrieval_feedback exist in generated types.
• FK types match actual tables: projects.id is integer, document_metadata.id is text, document_chunks.chunk_id is text, packet/card IDs are uuid.
• anon/authenticated cannot read private feedback from other users.
• service role can insert events and promotions.

​

Gate 2: Feedback Event Writer

• assistant thumbs down writes the existing chat feedback path if still used and always writes ai_feedback_events
• task feedback writes ai_task_feedback and a normalized event
• packet/card feedback writes intelligence_reviews and a normalized event
• failures return generic-free error messages with table and cause

​

Gate 3: Retrieval Trace

• every source lookup answer records tool name, query, source document IDs, chunk IDs, rank, cited flag, and answer usage
• downvoted source-grounded answers mark involved chunks as unknown or unhelpful, not bad forever
• source lookup questions do not answer from packet summary alone

​

Gate 4: Promotion Queue

• one-off thumbs-down creates candidate only
• repeated similar thumbs-down promotes to agent_learnings.status = active only when threshold is met
• approved positive task feedback becomes a future few-shot example
• rejected/expired/superseded promotions are not injected

​

Gate 5: Packet Improvement Loop

• correcting a card creates a review item and feedback event
• approving correction updates card/evidence and enqueues packet refresh
• refreshed packet includes changed card ranking/status
• stale/missing source coverage is visible in packet JSON

​

Gate 6: Assistant Prompt Injection Safety

• agent_learnings context is scoped by project/tool/tags
• private ai_memories are only injected for the owning user
• team memories are not created from low-confidence private preferences
• max learning context stays bounded
• injected learning usage is recorded in agent_learning_usages

​

First Implementation Order

​

Implemented Retrieval Learning Loop

​

Implemented Data Model

​

Implemented Admin Surface

• status tabs for candidate, approved, applied, rejected, and superseded promotions
• metrics for candidates, approvals, applied learnings, active/paused weights, superseded promotions, and audit activity
• Generate action for retrieval promotion scans
• approve/reject controls
• impact preview for retrieval-weight promotions
• apply, pause, resume, and supersede controls
• learning activity feed backed by ai_feedback_events

​

Implemented Flow

assistant semantic search
-> retrieval traces are written to ai_retrieval_feedback
-> admin runs promotion scan
-> repeated helpful/problem signals create ai_learning_promotions candidates
-> admin approves/rejects
-> approved retrieval weights can be previewed
-> admin applies promotion
-> ai_retrieval_weights row becomes active
-> semanticSearch applies bounded ranking multiplier
-> admin can pause, resume, or supersede
-> every lifecycle action writes ai_feedback_events

​

Current Safety Rules

• No ranking hint is active until admin approval and apply.
• Candidate generation requires repeated grouped signals.
• Boosts require repeated helpful/cited/used signals without problem signals.
• Problem candidates come from repeated unhelpful, wrong-project, stale, or unsupported outcomes.
• Multipliers are bounded between 0.65 and 1.5.
• Paused and superseded weights are ignored by semantic search.
• Lifecycle actions are auditable in ai_feedback_events.

​

Current Verification

• route conflict check
• focused ESLint
• focused TypeScript filtering for touched self-learning files
• unit guardrail for retrieval-weight scoring behavior

​

Remaining Gaps

• Browser-level smoke test for generate -> approve -> preview -> apply -> pause/resume.
• Live semantic-search replay preview.
• Destination writers for agent_learnings, ai_memories, task examples, and attribution rules.
• Packet/card feedback loop.

1. Add ai_feedback_events, ai_learning_promotions, and ai_retrieval_feedback.
2. Add frontend/src/lib/ai/services/feedback-event-service.ts.
3. Wire assistant thumbs, task feedback, and packet/card feedback into normalized events.
4. Add central retrieval trace capture.
5. Add promotion review and destination writers for agent_learnings, ai_memories, task examples, and attribution candidates.
6. Extend task generation with approved positive examples and active prevention prompts.
7. Add packet card feedback after the event and promotion infrastructure is proven.

​

Product Behavior Rules

• Feedback capture must be available in normal product surfaces, not only chat.
• The assistant must say when it is using memory versus source truth.
• Human-approved project facts should become project intelligence, not private memory.
• Personal preferences should not change team-visible outputs unless explicitly promoted.
• Financial, legal, contractual, and client-facing outputs require evidence and review before learning changes behavior.
• The system should prefer a missing-data warning over confident unsupported advice.

​

Open Questions

1. Which admin surface should own promotion review: existing feedback inbox, Project Intelligence review queue, or a dedicated AI Training page?
2. Should ai_feedback_events.session_id use chat_history.session_id UUID only, or support text session IDs from older paths?
3. Which events should be visible to non-admin project members?
4. What is the first owner-facing output to learn from edits: Project Status Report, owner update email, or packet executive summary?
5. How long should low-confidence attribution candidates stay active before archiving?

​

Recommended First Build

1. Add ai_feedback_events and write to it from task feedback and assistant thumbs.
2. Add ai_retrieval_feedback and record source/chunk usage for source lookup answers.
3. Add ai_learning_promotions and promote only reviewed/repeated failures into agent_learnings.
4. Extend task generation to use approved positive examples and active prevention prompts.
5. Add packet card feedback after the event and promotion infrastructure is proven.