All Projects
LLM Fine-TuningLegal AILegal

Phi-4 Fine-Tuning on the U.S. Constitution

Fine-tuned a 14B LLM on a full legal text to produce a reference-backed legal QA assistant, with measured before/after performance gains.

Build a Similar AI SystemWatch Demo
Phi-4 Fine-Tuning on the U.S. Constitution — LLM Fine-Tuning preview
LLM Fine-Tuning

Workflow & Architecture

How this system works

Step 1
Training data

U.S. Constitution structured into instruction–response pairs for domain tuning.

Step 2
Fine-tuning

Phi-4 trained with LoRA/QLoRA via Unsloth on legal QA and summarization tasks.

Step 3
Benchmarking

Before/after evaluation on 99 test queries for depth and citation accuracy.

Step 4
Chat API

FastAPI serves the fine-tuned model to the React chat interface.

Step 5
Cited answers

Responses include explicit references to source constitutional text.

Overview

Used Unsloth (LoRA/QLoRA) to fine-tune Phi-4 on instruction–response pairs built from the U.S. Constitution, with a FastAPI backend and React frontend powering a chat interface with explicit source citations.

The Problem

Demonstrate that domain fine-tuning measurably improves an LLM's depth and accuracy on a specialized text, versus using it out of the box.

The Solution

Fine-tuned Phi-4 on structured Constitution-based instruction-response pairs with before/after benchmarking, reference-backed answers, and a production chat interface.

Key Features

  • Structured instruction-tuning data built from raw legal text
  • Before/after benchmarking of the fine-tuned model
  • Reference-backed answers with explicit citations to source text
  • QA, summarization, and amendment interpretation tasks
  • FastAPI backend with React chat frontend

Outcomes & Business Value

  • Average response length improved 32→47 words after fine-tuning
  • Explicit source references improved from 45 to 78 out of 99 test queries
  • Measurable, benchmarked gain from domain-specific LLM fine-tuning

Related Projects