Val Town Code SearchReturn to Val Town

139## The Live Dev Experience
140
141One of the most important use cases of `vt` is "watch" functionality. There's a lot of reasons for this. One of the big ones is that, as you (and maybe your favorite LLM) edit your code locally, it is really handy to remove the friction of syncing with Val Town.
142
143From the beginning, my plan for this was to implement "git" behavior first (pushing and pulling), and then just doing file system watching to add live syncing using [Deno.watchFs](https://docs.deno.com/api/deno/~/Deno.watchFs) to handle file system events.
159## Some Nitty Gritty
160
161Internally, `vt` is broken up into two main parts: the `vt lib`, where the actual logic for `push`, `pull`, and other "git" operations is defined, and the `vt cmd` lib, where the CLI logic is laid out. Eventually, we want to make `vt` a esm library, where we expose the functionality of the `vt lib` component.
162
163For both cases, we're using [Deno's native testing framework](https://docs.deno.com/runtime/fundamentals/testing/). For `vt lib`, it's pretty straightforward, where we run tests in temp directories doing things like pulls or pushes, and then use the `sdk` to verify changes.
165For the command library, the CLI framework we're using for `vt` (cliffy) has a handy [Snapshot test module](https://cliffy.io/docs@v1.0.0-rc.7/testing/snapshot)  that `vt` doesn't use. We decided against using snapshot tests mostly because we lose a lot of fine grain control. Instead, we took inspiration from cliffy in running the CLI tests as subprocesses, and make a lot of use of Deno's `assertStringIncludes` on `stdout`. One ongoing issue we've had with testing is that we've encountered a lot of "resource leaks." It's really cool that `Deno` tests can detect unawaited promises or unawaited/cancelled request bodies, but it doesn't do a good job of helping us locate where the issues are.
166
167`vt` stores configuration in `<your system configuration directory>/vt/config.yaml`, and loads it using `zod`.  There's some interesting mechanics where we have multiple layers of configuration, like local and global, and prioritize configuration options in order. Once again, [Deno's standard library](https://jsr.io/@std/collections/doc/deep-merge) has been really handy in building out these components, like the `deepMerge` function.
168
169To figure out where your system configuration directory is, usually I'd use [npm's xdg-portable](https://www.npmjs.com/package/xdg-portable), which gets us your os-aware configuration directory, but it turns out that using this via `npm:xdg-portable` [doesn't work with Deno](https://github.com/rivy/js.xdg-portable/issues/3), and we can't use the officially recommended http import version since `jsr`, the registry we publish `vt` to, doesn't support http imports. I looked into this, and it seemed like an issue with their build process not including their Deno code. The solution I decided on? Fork [xdg-portable](https://github.com/404Wolf/xdg-portable-deno) to be Deno native! In the process, I removed a ton of bloat.
206But then we realized that, without rename detection, if you move a val with configuration -- like cron config, or custom endpoint HTTP vals, then doing the deletion/creation would cause you to lose all your config! And so, we added rename detection to `vt`.
207
208The rename detection algorithm is a bit complicated -- it works by looking at all files that got deleted and that got created, and then considering a file as renamed if a created file is sufficiently similar to a file that got deleted. When iterating over created files to see if a deleted file is similar enough to one of them, we use some heuristics to filter out files that could not possibly be similar enough, like the file length. Then we compute the Levenshtein distance between a given deleted file and created file, and consider a given created file "renamed" if it is above some theshold similar to a deleted file, and if it is similar enough to multiple, then the one it is most similar to [as it turns out, Deno's standard library has a super efficient edit distance function](https://jsr.io/@std/text). Git does fancy [directory rename detection](https://git-scm.com/docs/directory-rename-detection/2.22.0), which is something that, at least for now, `vt` does not do.
209
210Because rename detection is relatively expensive, it is internally implemented as an optional operation that doesn't always get used for every `vt` operation. For example, there isn't a lot of reason to do rename detection for `vt pull` -- it would really just be for reporting.

1export default async function (req: Request): Promise<Response> {
2  return Response.json({ ok: true })
3}

7});
8
9export default async function(interval: Interval) {
10  const subdomain = "valtownfornotion";
11  const items = await blob.list(subdomain);

7});
8
9export async function createDatabasePagesParallel(databaseId: string, pages: []) {
10  const creations = pages.map((page) =>
11    notion.pages.create({
43
44// delete all database rows
45export async function deleteAllDatabasePagesParallel(databaseId: string) {
46  const pageIds = await notion.databases.query({
47    database_id: databaseId,
59}
60
61export async function getDatabaseId(databaseTitle: string) {
62  // getDatabaseId(databaseTitle)
63  const database = await notion.databases
66}
67
68export async function getDatabaseTitle(databaseId: string) {
69  // getDatabaseTitle(databaseId)
70  const database = await notion.databases.retrieve({ database_id: databaseId });
73}
74
75export async function getDatabaseParentPageId(databaseId: string) {
76  let currentBlockId = databaseId;
77
98}
99
100export async function getCalloutProperties(askingFor: string) {
101  const blockProperties = (askingFor != "val.town")
102    ? {
121}
122
123export async function findChildDatabaseBlocks(rootBlockId: string, blockIdentifier: string) {
124  const matchingBlocks = [];
125
126  async function searchBlock(blockId) {
127    const res = await notion.blocks.children.list({ block_id: blockId });
128
165}
166
167export async function findCalloutBlocks(rootBlockId: string, blockIdentifier: string) {
168  const matchingBlocks = [];
169
170  async function searchBlock(blockId) {
171    const res = await notion.blocks.children.list({ block_id: blockId });
172
200}
201
202export async function getBostonTime() {
203  const now = new Date();
204
213}
214
215export async function listChildrenChildPages(blockId: string) {
216  try {
217    const response = await notion.blocks.children.list({
229}
230
231export async function addFavicon(data?: any) {
232  const pageId = data?.id;
233  // use notion if nothing if the URL property is empty
234  const url = await getWebhookPropertyValue(data, "Website") || "notion.com";
235  // send the URL value to the function that will get the site's favicon location
236  const faviconURL = await helpers.scrapeFaviconUrl(url);
237  console.log({ "Verified favicon URL": faviconURL });
242}
243
244export async function setPageStatus(object: any) {
245  const { data, section, step } = object;
246  const pageId = data?.id;
282}
283
284export async function updatePageIcon(pageId: string, faviconURL?: string) {
285  try {
286    const response = await notion.pages.update({
302}
303
304export async function listChildren(pageId: string) {
305  // get page properties
306  try {
318}
319
320// Helper function to get property value from webhook payload
321export async function getWebhookPropertyValue(payload: any, propertyName: any) {
322  let result = null;
323  // check to see if the whole payload was sent, or just the data object
357}
358
359export async function getNotionPage(pageId: string) {
360  // get page properties
361  try {

1export async function setBlobKey(object: any) {
2  const { slug, clientPageId, containerId } = object;
3  const blobKey = [ // key_legend: "[subdomain]--[id of guest page]--[id of container to be reset]",
11}
12
13export async function slugify(str: string) {
14  //  const containerTitleSlug = (await helpers.setContainerTitle(c.req.headers.get("x-container-title"))).split(" ").join("-");
15  return (str.replace(/[^\w\s]|_/g, "").replace(/\s+/g, " ").trim().toLowerCase()).split(" ").join("-");
16}
17
18// Function to scrape favicon URL from a website
19export async function scrapeFaviconUrl(url: string) {
20  try {
21    // Ensure URL has protocol
75}
76
77export async function extractCamelCaseWords(str: string) {
78  if (!str) return [];
79

58
59  // if the blob exists that maps to this callout for this user, use the id property inside it
60  // otherwise call the function to find the id of the callout for this user
61  // const blockId = blobject // if therre's a blob, then this isn't the first call
62  //   ? blobject.id // use the id in the blob to speed up subsequent calls

4import { DateTime } from "https://esm.sh/luxon@3.4.4";
5
6export async function testDailyBrief() {
7  try {
8    const testChatId = Deno.env.get("TEST_TELEGRAM_CHAT_ID");

2// Run this script manually to create the database table
3
4export default async function setupTelegramChatDb() {
5  try {
6    // Import SQLite module

13} from "../memoryUtils.ts";
14
15async function generateBriefingContent(anthropic, memories, today, isSunday) {
16  try {
17    const weekdaysHelp = generateWeekDays(today);
96}
97
98export async function sendDailyBriefing(chatId?: string, today?: DateTime) {
99  // Get API keys from environment
100  const apiKey = Deno.env.get("ANTHROPIC_API_KEY");
135  const lastSunday = today.startOf("week").minus({ days: 1 });
136
137  // Fetch relevant memories using the utility function
138  const memories = await getRelevantMemories();
139
216}
217
218function generateWeekDays(today) {
219  let output = [];
220
239// console.log(weekDays);
240
241// Export a function that calls sendDailyBriefing with no parameters
242// This maintains backward compatibility with existing cron jobs
243export default async function (overrideToday?: DateTime) {
244  return await sendDailyBriefing(undefined, overrideToday);
245}

16In a normal server environment, you would likely use a middleware [like this one](https://hono.dev/docs/getting-started/nodejs#serve-static-files) to serve static files. Some frameworks or deployment platforms automatically make any content inside a `public/` folder public. 
17
18However in Val Town you need to handle this yourself, and it can be suprisingly difficult to read and serve files in a Val Town Project. This template uses helper functions from [stevekrouse/utils/serve-public](https://www.val.town/x/stevekrouse/utils/branch/main/code/serve-public/README.md), which handle reading project files in a way that will work across branches and forks, automatically transpiles typescript to javascript, and assigns content-types based on the file's extension.
19
20### `index.html`
26## CRUD API Routes
27
28This app has two CRUD API routes: for reading and inserting into the messages table. They both speak JSON, which is standard. They import their functions from `/backend/database/queries.ts`. These routes are called from the React app to refresh and update data.
29
30## Errors