1---
2title: "Post-mortem: A Backward Incompatible Database Migration"
3description: Val runs failed due to a database migration that was not backward compatible
4pubDate: 2025-04-02T00:00:00.000Z
5author: Sophie Houser
6---
7
8Today at 10:11am we experienced a 12-minute outage, which caused HTTP vals to return 503 errors and other types of vals to fail. In the end, the root cause was a deployment timing issue where database migrations were deployed successfully, but our application code deployment hung for several minutes. The new database migrations were incompatible with the old application code and crashed the process.
9
10We aim to make all database migrations maintain backward compatibility, but in this case, we only discovered through the delayed deployment feedback that the new migrations were not compatible with previous versions.
11
12## Timeline
27## Next Steps
28
29Reliability is important to us and we’ve taken steps to make sure this doesn’t happen again. We’ve added a test to ensure database migrations are backward compatible, which we’ll run before we deploy any new code that includes database migrations.
15
16[React Hono Example](https://www.val.town/x/stevekrouse/reactHonoExample) is a
17fuller featured example project, with a SQLite database table, queries,
18client-side CSS and a favicon, and some shared code that runs on both client and
19server.
1---
2title: "Post-mortem: A Backward Incompatible Database Migration"
3description: Val runs failed due to a database migration that was not backward compatible
4pubDate: 2025-04-02T00:00:00.000Z
5author: Sophie Houser
6---
7
8Today at 10:11am we experienced a 12-minute outage, which caused HTTP vals to return 503 errors and other types of vals to fail. In the end, the root cause was a deployment timing issue where database migrations were deployed successfully, but our application code deployment hung for several minutes. The new database migrations were incompatible with the old application code and crashed the process.
9
10We aim to make all database migrations maintain backward compatibility, but in this case, we only discovered through the delayed deployment feedback that the new migrations were not compatible with previous versions.
11
12## Timeline
27## Next Steps
28
29Reliability is important to us and we’ve taken steps to make sure this doesn’t happen again. We’ve added a test to ensure database migrations are backward compatible, which we’ll run before we deploy any new code that includes database migrations.
46 "slug": "codegen",
47 "link": "/blog/codegen",
48 "description": "Like Claude Artifacts, but with a backend and database",
49 "pubDate": "Thu, 22 Aug 2024 00:00:00 GMT",
50 "author": "JP Posma",
16- [ ] Rebuild as React Router?
17- [ ] opentownie as a pr bot
18- [ ] give it the ability to see its own client-side and server-side logs by building a middleware that shoves them into a SQL light database date and then give it a tool to access them
19- [ ] do a browser use or screenshot thing to give it access to its own visual output
20- [ ] Have it default to creating a new branch off main and then embedding and iframe to the resulting http val and give you a link to a pr opening url
7export const thinkTool = tool({
8 description:
9 "Use the tool to think about something. It will not obtain new information or change the database, but just append the thought to the log. Use it when complex reasoning or some cache memory is needed.",
10 parameters: z.object({
11 thought: z.string().describe("A thought to think about."),
166```
167├── backend/
168│ ├── database/
169│ │ ├── migrations.ts # Schema definitions
170│ │ ├── queries.ts # DB query functions
228- Handle API calls properly with proper error catching
229
230### Database Patterns
231- Run migrations on startup or comment out for performance
232- Change table names when modifying schemas rather than altering
9- The **client-side entrypoint** is `/frontend/index.html`, which in turn imports `/frontend/index.tsx`, which in turn imports the React app from `/frontend/App.tsx`.
10
11[React Hono Example](https://www.val.town/x/stevekrouse/reactHonoExample) is a fuller featured example project, with a SQLite database table, queries, client-side CSS and a favicon, and some shared code that runs on both client and server.
14- [ ] Rebuild as React Router?
15- [ ] opentownie as a pr bot
16- [ ] give it the ability to see its own client-side and server-side logs by building a middleware that shoves them into a SQL light database date and then give it a tool to access them or use our trpc API in that tool
17- [ ] do a browser use or screenshot thing to give it access to its own visual output
18- [ ] Have it default to creating a new branch off main and then embedding and iframe to the resulting http val and give you a link to a pr opening url
7export const thinkTool = tool({
8 description:
9 "Use the tool to think about something. It will not obtain new information or change the database, but just append the thought to the log. Use it when complex reasoning or some cache memory is needed.",
10 parameters: z.object({
11 thought: z.string().describe("A thought to think about."),