Val Town Code SearchReturn to Val Town

27  useEffect(() => {
28    if ((age.length > 0 && budget.length > 0) || buttonPressed) {
29      fetchSuggestions();
30      setButtonPressed(false);
31    }
32  }, [age, budget, buttonPressed]);
33
34  const fetchSuggestions = async (isMoreSuggestions: boolean = false) => {
35    setLoading(true);
36    setError(null);
40
41    try {
42      const response = await fetch("/api/shopping-assistant", {
43        method: "POST",
44        headers: { "Content-Type": "application/json" },
60      setShownSuggestions(prevShown => [...prevShown, ...data.suggestions.map(s => s.text)]);
61    } catch (err) {
62      console.error("Error in fetchSuggestions:", err);
63      setError(`An error occurred while fetching suggestions: ${err.message}`);
64    } finally {
65      setLoading(false);
73
74  const handleMoreSuggestions = async () => {
75    await fetchSuggestions(true);
76  };
77

9Fun fact generation about the text (e.g., average word length)
10Summary generation
11The tool is implemented in JavaScript using Hono as a lightweight framework and integrates web APIs for fetching external content and analyzing it in real-time.
12
13Code Structure
16Hono: Used for routing and handling API requests.
17encode (from JS Base64): Utility for encoding data.
18parse (from Node HTML Parser): Parses HTML content, used in fetchUrlContent to clean and extract text.
19Core Modules
20
62Usage: Included in /analyse for quick overviews.
63cleanText(text)
64Purpose: Cleans HTML and unwanted characters from the fetched content.
65Parameters:
66
67text (string): Raw HTML/text. Returns: Cleaned text string.
68Usage: Essential for preprocessing content in fetchUrlContent.
69fetchUrlContent(url)
70Purpose: Fetches HTML content from a URL, removes unwanted elements, and extracts main text.
71Parameters:
72
73url (string): URL to fetch and clean content from. Returns: Cleaned text string or an error message.
74Usage: This function powers the URL input functionality.
75Core Functionalities
76Text Analysis Form (HTML)
77The form on the main page allows users to submit either raw text or a URL. The client-side JavaScript processes the form submission and, if needed, triggers a URL fetch to obtain content.
78
79Backend Processing (Routes)
81GET /: Serves the main HTML page.
82POST /analyse: Analyzes submitted text and returns word count, sentiment, summary, TF-IDF, and word frequency results.
83POST /fetch-url: Retrieves and cleans content from a URL, then prepares it for analysis.
84Dark Mode and Interactivity
85
90Testing
91
92Integration Tests: Ensure calculateTFIDF, calculateWordFrequency, and fetchUrlContent produce consistent outputs.
93Client-Side Testing: Regularly test URL fetching functionality to ensure robustness, particularly with dynamic sites.
94Error Handling
95
96Maintain try/catch blocks in asynchronous functions like fetchUrlContent to capture network issues.
97Update error messages to be user-friendly and provide feedback if inputs are invalid or exceed size limits.
98Data Sanitization
99
100Verify that cleanText removes sensitive information from fetched URLs to prevent accidental disclosure.
101Rate Limits and API Usage
102
119The Radical Text Analyser code primarily uses the following APIs:
1201. Hono Framework API
121    * Purpose: Used to set up routing and serve responses for different endpoints (GET /, POST /analyse, and POST /fetch-url).
122    * Usage: The Hono framework handles HTTP requests and responses, forming the backbone of the server-side API.
1232. External Content Fetching API (via fetch)
124    * Purpose: Retrieves HTML content from external URLs when users input a URL for analysis.
125    * Usage: The fetchUrlContent function uses fetch to make a GET request to the user-provided URL, which allows the application to process content from web pages.
126    * Error Handling: Includes logic to check response status and handle various network errors.
1273. Google Fonts API

9  const variable: string = await blob.getJSON(blobKey) ?? "";
10
11  const response = await fetch(scrapeURL);
12  const body = await response.text();
13  const $ = cheerio.load(body);

71# Long-term memory
72
73At some point the user might ask you to do something with "memory". Things like "remember", "save to memory", "forget", "update memory", etc. Please use corresponding actions to achieve those tasks. User might also ask you to perform some task with the context of your "memory" - in that case fetch all memories before proceeding with the task. The memories should be formed in a clear and purely informative language, void of unnecessary adjectives or decorative language forms. An exception to that rule might be a case when the language itself is the integral part of information (snippet of writing to remember for later, examples of some specific language forms, quotes, etc.).
74
75Structure of a memory:

25
26  useEffect(() => {
27    fetchComments();
28    fetchRating();
29  }, []);
30
31  const fetchComments = async () => {
32    const response = await fetch(`/${type}/${item.id}/comments`);
33    if (response.ok) {
34      const data = await response.json();
37  };
38
39  const fetchRating = async () => {
40    const response = await fetch(`/${type}/${item.id}/rating`);
41    if (response.ok) {
42      const data = await response.json();
46
47  const addComment = async () => {
48    const response = await fetch(`/${type}/${item.id}/comments`, {
49      method: 'POST',
50      headers: { 'Content-Type': 'application/json' },
53    if (response.ok) {
54      setNewComment('');
55      fetchComments();
56    }
57  };
58
59  const addRating = async (newRating) => {
60    const response = await fetch(`/${type}/${item.id}/rating`, {
61      method: 'POST',
62      headers: { 'Content-Type': 'application/json' },
108
109  useEffect(() => {
110    fetchInitialData();
111    fetchFavorites();
112  }, [page]);
113
114  const fetchInitialData = async () => {
115    const response = await fetch(`/search?type=${page}&query=`);
116    if (response.ok) {
117      const data = await response.json();
123  };
124
125  const fetchFavorites = async () => {
126    const response = await fetch("/favorites");
127    if (response.ok) {
128      const data = await response.json();
133  const handleSearch = async (e) => {
134    if (e.key === "Enter" || e.type === "click") {
135      const response = await fetch(`/search?type=${page}&query=${encodeURIComponent(search)}`);
136      if (response.ok) {
137        const data = await response.json();
146  const toggleFavorite = async (item, type) => {
147    const method = favorites[type].some(fav => fav.id === item.id) ? 'DELETE' : 'POST';
148    const response = await fetch(`/favorites/${type}`, {
149      method,
150      headers: { 'Content-Type': 'application/json' },
152    });
153    if (response.ok) {
154      fetchFavorites();
155    }
156  };
157
158  const addLostPet = async (petData) => {
159    const response = await fetch("/lostPets", {
160      method: "POST",
161      headers: { 'Content-Type': 'application/json' },
163    });
164    if (response.ok) {
165      fetchInitialData();
166    }
167  };

11    setLoading(true);
12    try {
13      const response = await fetch("/create-checkout-session", {
14        method: "POST",
15        headers: {

222    }
223
224    const response = await fetch("/generate", {
225      method: "POST",
226      headers: { "Content-Type": "application/json" },

1import { fetch } from 'https://esm.town/v/std/fetch';
2
3export type NtfyPayload = {
22
23  const body = JSON.stringify({ ...payload, topic });
24  const res = await fetch(server, {
25    method: 'POST',
26    headers: {

55  }
56}
57// Link Fetching Helper
58export class LinkFetcher {
59  static discoveredUrls = new Set();
60  static pageDepths = new Map();
61  static processQueue = [];
62
63  static async fetchLinksFromWebsite(websiteUrl) {
64    console.log(`\nFetching links from website: ${websiteUrl}`);
65    console.log("Initial HTML structure analysis begin");
66    const query = `
79
80    console.log(`Making LSD API request for ${websiteUrl}`);
81    const response = await fetch(
82      `https://lsd.so/api?query=${encodeURIComponent(query)}`,
83    );
84
85    if (!response.ok) {
86      console.log(`Failed to fetch links from ${websiteUrl}`);
87      return [];
88    }
148      const currentBatch = pagesToProcess.splice(0, 10);
149      const batchPromises = currentBatch.map(async (pageUrl) => {
150        console.log(`Fetching links from: ${pageUrl}`);
151        const pageQuery = `
152          SELECT 
162
163        console.log(`Making LSD API request for ${pageUrl}`);
164        const response = await fetch(
165          `https://lsd.so/api?query=${encodeURIComponent(pageQuery)}`,
166        );
167
168        if (!response.ok) {
169          console.log(`Failed to fetch links from ${pageUrl}`);
170          return [];
171        }
235    }
236
237    console.log(`Successfully fetched ${allLinks.length} links from ${discoveredPages.size} pages`);
238    return allLinks;
239  }
279    try {
280      console.log(`Attempting HEAD request for: ${url}`);
281      const response = await fetch(url, { method: "HEAD" });
282
283      if (!response.ok) {
284        console.log(`HEAD request failed, attempting GET request for: ${url}`);
285        const getResponse = await fetch(url, { method: "GET" });
286        return { ok: getResponse.ok, status: getResponse.status };
287      }
431    };
432
433    // Fetch all links with progress tracking
434    const links = await LinkFetcher.fetchLinksFromWebsite(websiteUrl);
435    const totalPages = new Set(links.map(link =>
436      new URL(link.link, websiteUrl).origin + new URL(link.link, websiteUrl).pathname

1/** @jsxImportSource https://esm.sh/hono@latest/jsx **/
2
3import { modifyFetchHandler } from "https://esm.town/v/andreterron/codeOnValTown?v=50";
4import { iframeHandler } from "https://esm.town/v/nbbaier/iframeHandler";
5import { resetStyle } from "https://esm.town/v/nbbaier/resetStyle";
16import { verifyToken } from "https://esm.town/v/pomdtr/verifyToken";
17import { ResultSet, sqlite } from "https://esm.town/v/std/sqlite";
18import { reloadOnSaveFetchMiddleware } from "https://esm.town/v/stevekrouse/reloadOnSave";
19import { Hono } from "npm:hono";
20import type { FC } from "npm:hono/jsx";
175});
176
177export const handler = app.fetch;
178export default iframeHandler(modifyFetchHandler(passwordAuth(handler, { verifyPassword: verifyToken })));