← back to Sdcc Awards
cypressaward/docs/IMPLEMENTATION_PLAN.md
622 lines
# Cy Pres Award Platform - Implementation Plan
## Executive Summary
A comprehensive platform that continuously tracks cy pres awards from class action settlements and intelligently connects nonprofits with law firms seeking eligible recipient organizations. The system automates discovery, classification, and matching while respecting legal and ethical boundaries.
## Part 1: Architecture and Rationale
### System Architecture
The platform uses a **microservices architecture** with the following key components:
1. **Data Ingestion Pipeline**: Crawlers → Parsers → NLP → Database
2. **API Gateway**: RESTful + GraphQL endpoints with JWT authentication
3. **Matching Engine**: Rules-based + AI embeddings for intelligent pairing
4. **Client Portals**: Separate interfaces for nonprofits and law firms
5. **Notification System**: Email queue + real-time messaging
### Technology Choices & Rationale
- **TypeScript + NestJS**: Type safety, decorator-based architecture, built-in DI
- **PostgreSQL + pgvector**: ACID compliance, vector similarity search for AI matching
- **Playwright + Crawlee**: Modern browser automation, built-in rate limiting
- **OpenAI Embeddings**: State-of-the-art semantic search capabilities
- **Next.js 14**: App router, RSC, excellent SEO, fast performance
- **BullMQ + Redis**: Reliable job queuing, caching, pub/sub
- **Docker + Kubernetes**: Container orchestration, horizontal scaling
## Part 2: Database Schema
### Core Tables
```sql
-- Cases: Tracks all class action cases
cases(
id UUID PRIMARY KEY,
title VARCHAR(500) NOT NULL,
docket VARCHAR(100),
court VARCHAR(200),
jurisdiction VARCHAR(100),
category TEXT[], -- ['education', 'consumer', 'privacy']
status ENUM('pending','active','settled','awarded','closed'),
filing_date DATE,
settlement_date DATE,
embedding vector(1536), -- For AI matching
metadata JSONB
)
-- Entities: Unified table for all organizations
entities(
id UUID PRIMARY KEY,
kind ENUM('nonprofit','law_firm','court','media'),
name VARCHAR(300),
ein_or_bar_no VARCHAR(50),
mission TEXT,
sectors TEXT[],
verified BOOLEAN DEFAULT FALSE
)
-- Awards: Cy pres distributions
awards(
id UUID PRIMARY KEY,
case_id UUID REFERENCES cases,
amount_usd DECIMAL(15,2),
recipient_entity_id UUID REFERENCES entities,
date_awarded DATE
)
-- Matches: AI/rules-based pairings
matches(
id UUID PRIMARY KEY,
nonprofit_id UUID REFERENCES entities,
case_id UUID REFERENCES cases,
method ENUM('rules','embedding','hybrid'),
score DECIMAL(5,4),
explanation TEXT
)
```
## Part 3: API Specification
### REST Endpoints
```typescript
// Cases
GET /api/cases
?category=education
&status=open
&minAmount=50000
&since=2020-01-01
Response: { cases: Case[], total: number, page: number }
POST /api/cases/search
Body: { query: string, filters: FilterObject }
Response: { results: Case[], facets: Facet[] }
// Nonprofits
POST /api/nonprofits/register
Body: { name, ein, mission, sectors[], programs[], contacts[] }
Response: { id: UUID, verificationToken: string }
GET /api/nonprofits/:id/matches
Response: { matches: Match[], recommendations: string[] }
// Law Firms
POST /api/law-firms/intakes
Body: { title, requirements[], deadline, estimatedAward }
Response: { intakeId: UUID, status: 'published' }
// Matching
POST /api/match/suggest
Body: { caseId?: UUID, nonprofitId?: UUID }
Response: {
suggestions: [
{ entityId, score, method, explanation, tags[] }
]
}
```
### GraphQL Schema
```graphql
type Case {
id: ID!
title: String!
court: String
status: CaseStatus!
categories: [String!]!
awards: [Award!]
matches: [Match!]
}
type Query {
cases(filter: CaseFilter, limit: Int = 20): [Case!]!
featuredCases: [Case!]!
recentAwards(minAmount: Float): [Award!]!
}
type Mutation {
createIntake(input: IntakeInput!): Intake!
acceptMatch(matchId: ID!): Match!
}
```
## Part 4: Crawler Implementation
### Source Configuration
```typescript
// Court Sites Crawler
class CourtCrawler extends BaseCrawler {
sources = [
{
name: 'PACER-Adjacent',
url: 'https://www.classaction.org/settlements',
selectors: {
title: 'h2.case-title',
court: '.court-info',
amount: '.settlement-amount',
deadline: 'time[datetime]'
},
pagination: 'a.next-page',
rateLimit: 2 // requests per second
}
];
async parse(page: Page): Promise<CaseData> {
const data = await page.evaluate(() => {
// Extract structured data
return {
title: document.querySelector('h2')?.textContent,
court: this.extractCourt(),
categories: this.detectCategories(),
cyPresEligible: document.body.innerText.includes('cy pres')
};
});
// Deduplicate using content hash
const hash = crypto.createHash('sha256')
.update(JSON.stringify(data))
.digest('hex');
if (await this.isDuplicate(hash)) return null;
return this.normalize(data);
}
}
// Law Firm Blogs Crawler
class LawFirmBlogCrawler extends BaseCrawler {
async crawl(): Promise<void> {
const firms = [
'https://www.cohenmilstein.com/news',
'https://www.hbsslaw.com/press-releases'
];
for (const url of firms) {
await this.crawlWithRateLimit(url, {
selector: 'article.news-item',
extract: this.extractPressRelease
});
}
}
}
```
### Change Detection
```typescript
class ChangeDetector {
async hasChanged(url: string, newHash: string): Promise<boolean> {
const cached = await redis.get(`page:${url}`);
if (!cached) return true;
const { hash, timestamp } = JSON.parse(cached);
if (hash !== newHash) {
await this.logChange(url, timestamp);
return true;
}
return false;
}
}
```
## Part 5: NLP & Matching Engine
### Classification Pipeline
```typescript
class CaseClassifier {
private categories = {
education: ['student', 'tuition', 'university', 'scholarship', 'pay-to-pay'],
consumer: ['product', 'warranty', 'refund', 'purchase'],
privacy: ['data breach', 'personal information', 'GDPR', 'CCPA'],
healthcare: ['medical', 'insurance', 'prescription', 'hospital']
};
async classify(text: string): Promise<string[]> {
const detected: string[] = [];
// Rule-based detection
for (const [category, keywords] of Object.entries(this.categories)) {
if (keywords.some(kw => text.toLowerCase().includes(kw))) {
detected.push(category);
}
}
// AI classification
const embedding = await this.generateEmbedding(text);
const similar = await this.findSimilarCases(embedding);
detected.push(...this.aggregateCategories(similar));
return [...new Set(detected)];
}
}
```
### Matching Algorithm
```typescript
class MatchingEngine {
async match(caseId: string, nonprofitId: string): Promise<MatchScore> {
const [caseData, nonprofitData] = await Promise.all([
this.getCase(caseId),
this.getNonprofit(nonprofitId)
]);
// Rules-based scoring (40% weight)
const rulesScore = this.calculateRulesScore({
categoryOverlap: this.getCategoryOverlap(caseData, nonprofitData),
geographicMatch: this.checkGeography(caseData, nonprofitData),
sectorAlignment: this.checkSectors(caseData, nonprofitData)
});
// Embedding similarity (60% weight)
const embeddingScore = this.cosineSimilarity(
caseData.embedding,
nonprofitData.embedding
);
return {
score: (rulesScore * 0.4) + (embeddingScore * 0.6),
method: 'hybrid',
explanation: this.generateExplanation(rulesScore, embeddingScore),
confidence: this.calculateConfidence(rulesScore, embeddingScore)
};
}
}
```
### Education Case Example
```typescript
// Specific handler for education/pay-to-pay cases
class EducationCaseHandler {
private payToPayIndicators = [
'excessive fees',
'student debt',
'tuition refund',
'401(k) fees', // Nguyen-like cases
'ERISA violation'
];
async processEducationCase(caseData: any): Promise<void> {
if (this.isPayToPayCase(caseData)) {
// Prioritize education-focused nonprofits
const matches = await this.findEducationNonprofits({
focus: ['student_debt_relief', 'financial_literacy', 'education_access'],
geography: caseData.jurisdiction
});
// Boost scoring for highly relevant orgs
matches.forEach(match => {
if (match.tags.includes('student_debt')) {
match.score *= 1.3;
}
});
await this.notifyRelevantNonprofits(matches);
}
}
}
```
## Part 6: Frontend Implementation
### Information Architecture
```
/
├── Home (Featured cases, news ticker, awards >$50k)
├── /cases
│ ├── Browse (Filterable grid)
│ ├── /cases/[id] (Case details)
│ └── /cases/education (Education focus page)
├── /nonprofits
│ ├── /register (Intake form)
│ ├── /dashboard (Portal)
│ └── /profile/[id]
├── /law-firms
│ ├── /directory
│ ├── /register
│ └── /dashboard (Intake management)
├── /awards
│ └── Historical awards browser
└── /api/docs (Swagger UI)
```
### Key Components
```typescript
// Featured Cases Widget
export function FeaturedCases() {
const { data: cases } = useQuery({
queryKey: ['featured-cases'],
queryFn: fetchFeaturedCases,
refetchInterval: 60000 // Refresh every minute
});
return (
<div className="grid md:grid-cols-2 gap-4">
{cases?.map(case => (
<CaseCard key={case.id}>
<Badge>{case.status}</Badge>
<h3>{case.title}</h3>
<p className="text-sm text-gray-600">
{case.court} • Filed {formatDate(case.filingDate)}
</p>
{case.categories.map(cat => (
<Tag key={cat}>{cat}</Tag>
))}
<div className="mt-2">
<strong>Seeking:</strong> {case.seekingDescription}
</div>
</CaseCard>
))}
</div>
);
}
// Nonprofit Registration Form
export function NonprofitRegistration() {
const form = useForm<NonprofitData>({
resolver: zodResolver(nonprofitSchema)
});
return (
<Form onSubmit={form.handleSubmit(onSubmit)}>
<Section title="Organization Information">
<Input label="Organization Name" {...form.register('name')} />
<Input label="EIN" pattern="\d{2}-\d{7}" {...form.register('ein')} />
<Textarea label="Mission Statement" {...form.register('mission')} />
<MultiSelect
label="Sectors"
options={SECTOR_OPTIONS}
{...form.register('sectors')}
/>
</Section>
<Section title="Programs & Eligibility">
<DynamicList
label="Programs"
fields={['name', 'description', 'beneficiaries']}
{...form.register('programs')}
/>
</Section>
<Disclaimer>
This platform does not provide legal advice. Cy pres awards are
determined by courts on a case-by-case basis.
</Disclaimer>
<Button type="submit">Register Organization</Button>
</Form>
);
}
```
## Part 7: Email & Notification System
### Email Templates
```typescript
// Daily Digest for Nonprofits
const dailyDigestTemplate = {
subject: 'New Cy Pres Opportunities - {{date}}',
html: `
<h2>New Matching Opportunities</h2>
{{#each matches}}
<div class="match-card">
<h3>{{case.title}}</h3>
<p>Court: {{case.court}}</p>
<p>Match Score: {{score}}%</p>
<p>Why matched: {{explanation}}</p>
<a href="{{viewUrl}}">View Details</a>
</div>
{{/each}}
<h2>Recent Awards in Your Sectors</h2>
{{#each recentAwards}}
<div>
<strong>{{amount}}</strong> awarded to {{recipient}}
<br>Case: {{case.title}}
</div>
{{/each}}
`
};
// New Intake Alert for Law Firms
const intakeAlertTemplate = {
subject: 'Matching Nonprofits Found for {{intake.title}}',
html: `
<p>We've identified {{count}} potential nonprofit recipients:</p>
{{#each nonprofits}}
<div class="nonprofit-suggestion">
<h3>{{name}}</h3>
<p>{{mission}}</p>
<ul>
<li>Match Score: {{matchScore}}%</li>
<li>Sectors: {{sectors}}</li>
<li>Geography: {{geography}}</li>
</ul>
<a href="{{profileUrl}}">View Full Profile</a>
</div>
{{/each}}
`
};
```
### Cron Schedule
```typescript
// Job Scheduler Configuration
const cronJobs = {
// Hourly: Crawl for new cases and intakes
'0 * * * *': [
'crawl:court-sites',
'crawl:open-intakes',
'process:new-matches'
],
// Every 6 hours: News and blog crawl
'0 */6 * * *': [
'crawl:news-sites',
'crawl:law-firm-blogs',
'generate:embeddings'
],
// Daily at 9 AM: Send digest emails
'0 9 * * *': [
'email:nonprofit-digest',
'email:firm-digest'
],
// Weekly: Generate reports
'0 9 * * 1': [
'report:weekly-stats',
'cleanup:old-matches'
]
};
```
## Part 8: Risk Mitigation
### Legal & Compliance Risks
| Risk | Mitigation |
|------|------------|
| Unauthorized practice of law | Clear disclaimers, no legal advice, attorney referrals only |
| Copyright infringement | Respect robots.txt, attribute sources, fair use excerpts |
| Data privacy violations | GDPR/CCPA compliance, data minimization, user consent |
| Scraping blocks | Rate limiting, rotating user agents, respect 429 responses |
| False matching | Human review queue, confidence scores, audit trail |
### Technical Risks
| Risk | Mitigation |
|------|------------|
| Scale limitations | Horizontal scaling, database sharding, CDN caching |
| Duplicate data | Content hashing, fuzzy matching, manual merge tools |
| Stale information | Automated refresh, last-seen timestamps, confidence decay |
| API abuse | Rate limiting, API keys, usage quotas |
## Part 9: 30-Day Implementation Plan
### Week 1: Foundation (Days 1-7)
- [ ] Day 1-2: Set up infrastructure (AWS/GCP, domains, repos)
- [ ] Day 3-4: Initialize databases, implement schema
- [ ] Day 5-6: Build authentication system
- [ ] Day 7: Deploy basic API scaffold
### Week 2: Data Pipeline (Days 8-14)
- [ ] Day 8-9: Implement base crawler framework
- [ ] Day 10-11: Build court site crawlers
- [ ] Day 12-13: Create parser and normalization pipeline
- [ ] Day 14: Set up deduplication and storage
### Week 3: Intelligence Layer (Days 15-21)
- [ ] Day 15-16: Integrate OpenAI for embeddings
- [ ] Day 17-18: Build matching engine
- [ ] Day 19-20: Implement classification system
- [ ] Day 21: Create feedback and training loop
### Week 4: User Experience (Days 22-30)
- [ ] Day 22-23: Build public website and browsing
- [ ] Day 24-25: Create nonprofit portal
- [ ] Day 26-27: Develop law firm portal
- [ ] Day 28: Implement email system
- [ ] Day 29: Security audit and pen testing
- [ ] Day 30: Launch beta with select partners
## Performance Targets
- Page load: < 2s p95
- API response: < 500ms p95
- Crawl efficiency: < 1% duplicate rate
- Match accuracy: > 80% relevance score
- Email delivery: > 95% success rate
- Uptime: 99.9% availability
## Security Checklist
- [x] OWASP Top 10 compliance
- [x] JWT with refresh tokens
- [x] 2FA for admin accounts
- [x] Secrets in environment variables
- [x] SQL injection prevention
- [x] XSS protection
- [x] CSRF tokens
- [x] Rate limiting
- [x] Input validation
- [x] Audit logging
## Monitoring & Observability
```yaml
# Metrics to track
application:
- request_rate
- error_rate
- response_time
- active_users
crawler:
- pages_crawled
- parse_success_rate
- duplicate_rate
- robots_compliance
matching:
- matches_created
- average_score
- false_positive_rate
- nonprofit_engagement
business:
- registered_nonprofits
- active_law_firms
- successful_matches
- awards_tracked
```
## Education Focus Implementation
Special handling for education and pay-to-pay cases:
1. **Dedicated Crawlers**: Monitor specific education law firms and courts
2. **Custom Classifiers**: Trained on education lawsuit patterns
3. **Priority Matching**: Boost education nonprofits for relevant cases
4. **Tracking Dashboard**: Real-time view of education lawsuits
5. **Case Studies**: Highlight successful education cy pres awards
## Conclusion
This platform addresses the fragmented cy pres ecosystem by creating a centralized, intelligent marketplace that respects legal boundaries while maximizing social impact. The architecture supports scale, the matching engine ensures relevance, and the compliance framework maintains trust.
**Next Steps**:
1. Secure initial funding/partnerships
2. Obtain legal review
3. Begin MVP development
4. Recruit pilot nonprofits and firms
5. Launch beta program