Val Town Code SearchReturn to Val Town

91.  **Examine `/backend/services/pdf.processing.service.ts` completely**
10
11    - Confirm it contains both `processPdfWithOpenAI()` and `processEmailContentWithOpenAI()` functions
12    - Note the exported types: `PdfExtractionResult` and `EmailExtractionResult`
13    - Verify the file's primary context is email processing (both content and attachments)
423.  **Update import in `/backend/emails/transaction.attachments.email.ts`**
43
44    - Change: `import { processPdfWithOpenAI, processEmailContentWithOpenAI } from "../services/pdf.processing.service.ts";`
45    - To: `import { processPdfWithOpenAI, processEmailContentWithOpenAI } from "../services/email.processing.service.ts";`
46
474.  **Delete `/backend/services/pdf.processing.service.ts`**
52
53- \[ \] New file `/backend/services/email.processing.service.ts` created with updated header
54- \[ \] Functions `processPdfWithOpenAI` and `processEmailContentWithOpenAI` remain identical
55- \[ \] Types `PdfExtractionResult` and `EmailExtractionResult` remain identical
56- \[ \] Import updated in `/backend/services/notion.transactions.service.ts`

13
14*   Read `/backend/emails/transaction.attachments.email.ts` to understand current PDF processing flow
15*   Note that it uses `processPdfWithOpenAI()` which returns `PdfExtractionResult` with structured data
16*   Identify the data being extracted: amount, date, vendor, description, type, confidence
17
1161.  Add import: `import { createNotionAttachment } from "../services/notion.transactions.service.ts";`
117    
1182.  In the PDF processing loop, after each `processPdfWithOpenAI()` call, add:
119    
120

1Instructions for Implementing OpenAI PDF Processing in Val Town
2===============================================================
3
5-------
6
7User wants to process PDF attachments from emails using OpenAI to extract structured financial data (amount, date, vendor, description). The goal is to log results for verification, not store them.
8
9Critical Requirements
13
14*   **Step 1**: Extract text from PDF locally using a PDF parsing library
15*   **Step 2**: Send extracted text to OpenAI chat completions (NOT Files API)
16*   **Do NOT** use OpenAI Files API, file attachments, or file\_search tools
17*   **Do NOT** overcomplicate with file uploads or special OpenAI endpoints
18
19### 2\. PDF Text Extraction Library Selection
24*   **Handle errors gracefully** with multiple fallback approaches
25
26### 3\. OpenAI Integration
27
28*   Use standard `openai` package from esm.sh: `https://esm.sh/openai@4.67.3`
29*   Use regular chat completions endpoint
30*   Include extracted text directly in the prompt
35
36*   Create `/backend/services/pdf.processing.service.ts`
37*   Export `processPdfWithOpenAI(file: File)` function
38*   Update existing email trigger to import and use the service
39*   Update documentation in README files
45
46    // /backend/services/pdf.processing.service.ts
47    import { OpenAI } from "https://esm.sh/openai@4.67.3";
48    
49    // Define interface for results
63        processingTime: number;
64        textLength?: number;
65        openaiTokens?: number;
66      };
67    }
75    }
76    
77    // OpenAI analysis function
78    async function analyzeTextWithOpenAI(openai: OpenAI, text: string): Promise<any> {
79      // Use detailed extraction prompt
80      // Send to chat completions (NOT Files API)
84    
85    // Main export function
86    export async function processPdfWithOpenAI(file: File): Promise<PdfExtractionResult> {
87      // Validate file (PDF, size limits)
88      // Extract text locally
89      // Initialize OpenAI client
90      // Analyze text with OpenAI
91      // Parse and return structured results
92      // Comprehensive error handling and logging
97
98    // /backend/emails/transaction.attachments.email.ts
99    import { processPdfWithOpenAI } from "../services/pdf.processing.service.ts";
100    
101    // In the main function, after detecting PDFs:
102    for (const pdfFile of pdfFiles) {
103      const result = await processPdfWithOpenAI(pdfFile);
104      // Log results and handle errors
105    }
111*   Update services README with new service
112*   Update email trigger README with processing capability
113*   Add OPENAI\_API\_KEY to environment variables section
114
115Critical Implementation Details
131    
132
133### OpenAI Prompt
134
135    const extractionPrompt = `
167### Common Pitfalls to Avoid
168
1691.  **Don't use OpenAI Files API** \- Use regular chat completions
1702.  **Don't use file attachments** \- Send text directly in prompt
1713.  **Import libraries dynamically** \- Avoid top-level imports that cause Node.js issues
175### Environment Variables
176
177*   Ensure `OPENAI_API_KEY` is set
178*   Document in README requirements section
179
182*   Log extracted text preview (first 200 chars)
183*   Log text length for verification
184*   Log OpenAI response and token usage
185*   Log processing time and success/failure rates
186

5---------
6
7Modify the existing PDF-only email trigger to also process emails without PDF attachments by extracting financial data and "view in browser" URLs using OpenAI, then saving to the same Notion database.
8
9Key Implementation Strategy
10---------------------------
11
12*   **Use OpenAI for ALL extraction** (financial data + URLs) - do NOT use regex patterns
13*   **Maintain existing PDF processing** \- only add email content processing as a new branch
14*   **Use same Notion database schema** \- save URLs as external links in "Files & media" property
33        type: 'receipt' | 'invoice' | null;
34        confidence: 'high' | 'medium' | 'low';
35        viewUrl: string | null;  // NEW: URL extracted by OpenAI
36      };
37      error?: string;
39        contentLength: number;
40        processingTime: number;
41        openaiTokens?: number;
42      };
43    }
48**File**: `/backend/services/pdf.processing.service.ts`
49
50Add new function that processes email content with OpenAI:
51
52    async function analyzeEmailContentWithOpenAI(openai: OpenAI, content: string): Promise<any> {
53      const extractionPrompt = `
54    You are an expert financial document analyzer. I will provide you with email content that contains financial information.
81    `;
82    
83      return await openai.chat.completions.create({
84        model: "gpt-4o",
85        messages: [{ role: "user", content: extractionPrompt }],
89    }
90    
91    export async function processEmailContentWithOpenAI(content: string): Promise<EmailExtractionResult> {
92      // Implementation similar to processPdfWithOpenAI but for email content
93      // Include viewUrl in the returned data structure
94    }
141Add import for new functions:
142
143    import { processPdfWithOpenAI, processEmailContentWithOpenAI } from "../services/pdf.processing.service.ts";
144    import { createNotionAttachment, uploadPdfToNotionPage, createNotionEmailAttachment } from "../services/notion.transactions.service.ts";
145    
156      
157      try {
158        // Process email content with OpenAI (extracts data + URL)
159        const emailProcessingResult = await processEmailContentWithOpenAI(emailContent);
160        
161        if (emailProcessingResult.success && emailProcessingResult.data?.viewUrl) {
162          console.log('🔗 OpenAI extracted view URL:', emailProcessingResult.data.viewUrl);
163        }
164        
165        // Save to Notion (URL comes from OpenAI result)
166        const notionResult = await createNotionEmailAttachment(
167          emailProcessingResult,
192Key points to document:
193
194*   **AI-Powered URL Extraction**: OpenAI extracts URLs as part of data extraction
195*   **Dual Processing Modes**: PDF attachments vs email content
196*   **Same Database Schema**: Both modes use identical Notion fields
209### DO:
210
211*   Let OpenAI handle all extraction (data + URLs) in one call
212*   Add email processing as a new branch, not a replacement
213*   Use the same field mapping for both PDF and email modes
218
2191.  Test existing PDF processing still works
2202.  Test email content processing with OpenAI
2213.  Verify URLs are correctly saved to Notion
2224.  Check error handling for various edge cases
226
227*   Emails with PDFs: Use existing PDF processing (unchanged)
228*   Emails without PDFs: Process email content with OpenAI, extract financial data + URLs, save to same Notion database with URL as external link
229*   Same comprehensive logging and error handling for both modes
230*   No breaking changes to existing functionality

31*   **No inference**: Do NOT assume or infer document type - only base on explicit keyword presence
32
33### 3\. OpenAI Prompt Enhancement
34
35Modify the `analyzeTextWithOpenAI` function's extraction prompt:
36
37**Add new instruction (#5):**
78
79*   Do NOT modify the PDF text extraction logic
80*   Do NOT change the OpenAI client initialization
81*   Do NOT alter error handling patterns
82*   Do NOT modify the existing data fields (amount, date, vendor, description, confidence)
137
138*   ✅ Adds type field to TypeScript interface
139*   ✅ Updates OpenAI prompt with type detection instructions
140*   ✅ Includes type in JSON response format specification
141*   ✅ Adds type to console logging output

6
7
8## 02-openai.md
9
10

9  "z-ai/glm-4.5-air:free",
10  // "morph/morph-v3-large",
11  // "openai/gpt-4o-mini",
12  // "anthropic/claude-3-haiku",
13  // "meta-llama/llama-3.1-8b-instruct",

94Note: When changing a SQLite table's schema, change the table's name (e.g., add _2 or _3) to create a fresh table.
95
96### OpenAI
97
98```ts
99import { OpenAI } from "https://esm.town/v/std/openai";
100const openai = new OpenAI();
101const completion = await openai.chat.completions.create({
102  messages: [
103    { role: "user", content: "Say hello in a creative way" },

276
277// ============================================================================
278// OPENAI INTEGRATION (40 lines)
279// ============================================================================
280
281async function callOpenAI(prompt: string): Promise<SimpleAnalysisResult> {
282  const requestBody = {
283    model: 'gpt-4o-mini',
287  };
288
289  // Log the full request being sent to OpenAI
290  console.log('=== OPENAI REQUEST BEING SENT ===');
291  console.log('Model:', requestBody.model);
292  console.log('Max completion tokens:', requestBody.max_completion_tokens);
293  console.log('Response format:', requestBody.response_format);
294  console.log('=== PROMPT BEING SENT TO OPENAI ===');
295  console.log(prompt);
296  console.log('=== END PROMPT ===');
297  console.log('Request body size (chars):', JSON.stringify(requestBody).length);
298  console.log('=== END OPENAI REQUEST ===');
299
300  const response = await fetch('https://api.openai.com/v1/chat/completions', {
301    method: 'POST',
302    headers: {
303      'Authorization': `Bearer ${Deno.env.get('OPENAI_API_KEY')}`,
304      'Content-Type': 'application/json'
305    },
308
309  if (!response.ok) {
310    throw new Error(`OpenAI API error: ${response.status} - ${response.statusText}`);
311  }
312
314  const content = data.choices[0].message.content;
315  
316  // Log the response from OpenAI
317  console.log('=== OPENAI RESPONSE RECEIVED ===');
318  console.log('Usage:', data.usage);
319  console.log('Model used:', data.model);
320  console.log('Response content length:', content?.length || 0);
321  console.log('=== OPENAI RESPONSE CONTENT ===');
322  console.log(content);
323  console.log('=== END OPENAI RESPONSE ===');
324  
325  try {
326    return JSON.parse(content);
327  } catch (error) {
328    console.error('Failed to parse OpenAI response:', content);
329    throw new Error('Invalid JSON response from OpenAI');
330  }
331}
410    const prompt = buildAnalysisPrompt(teamStats, roster);
411    
412    console.log('Calling OpenAI for enhanced analysis...');
413    
414    // Get AI analysis
415    const analysis = await callOpenAI(prompt);
416    
417    console.log(`Analysis complete. Grade: ${analysis.overall_grade}, Recommendations: ${analysis.waiver_recommendations.length}`);

100Note: When changing a SQLite table's schema, change the table's name (e.g., add _2 or _3) to create a fresh table.
101
102### OpenAI
103```ts
104import { OpenAI } from "https://esm.town/v/std/openai";
105const openai = new OpenAI();
106const completion = await openai.chat.completions.create({
107  messages: [
108    { role: "user", content: "Say hello in a creative way" },