52
53## 2. Data Collection and Use
54<apiName> allows users to store, retrieve, list, and delete data. The data stored can be of any type as inputted by the user. We do not restrict or control the content of the data stored. The API operates as a public database, where data can be accessed by any user. We do not tie or track data per user.
55
56## 3. Data Security
4
5- Create a new metadata object. Also has support for getting the previous metadata and only adding new vals, but that's currently disabled.
6- Get all val names from the [database of public vals](https://www.val.town/v/sqlite/db), made by [Achille Lacoin](https://www.val.town/u/pomdtr).
7- Put val names in batches. Vals in the same batch will have their embeddings stored in the same blob, at different offsets.
8- Iterate through all each batch, get code for all the vals, get embeddings from OpenAI, and store the result in a blob.
8
9I implemented three backends, which you can switch between in the search UI. Check out these vals for details on their implementation.
10- **Neon:** storing and searching embeddings using the [pg_vector](https://neon.tech/docs/extensions/pgvector) extension in Neon's Postgres database.
11 - Searching: [janpaul123/semanticSearchNeon](https://www.val.town/v/janpaul123/semanticSearchNeon)
12 - Indexing: [janpaul123/indexValsNeon](https://www.val.town/v/janpaul123/indexValsNeon)
18 - Indexing: [janpaul123/indexValsTurso](https://www.val.town/v/janpaul123/indexValsTurso)
19
20All implementations use the [database of public vals](https://www.val.town/v/sqlite/db), made by [Achille Lacoin](https://www.val.town/u/pomdtr), which is refreshed every hour. The Neon implementation updates every 10 minutes, and the other ones are not updated. I also forked Achille's [search UI](https://www.val.town/v/pomdtr/val_town_search) for this val.
21
22Please share any feedback and suggestions, and feel free to fork our vals to improve them. This is a playground for semantic search before we implement it in the product for real!
4
5- Create the `vals_embeddings` and `vss_vals_embeddings` tables in Turso if they don't already exist.
6- Get all val names from the [database of public vals](https://www.val.town/v/sqlite/db), made by [Achille Lacoin](https://www.val.town/u/pomdtr).
7- Get all val names from the `vals_embeddings` table and compute the difference (which ones are missing).
8- Iterate through all missing vals, get their code, get embeddings from OpenAI, and store the result in Turso.
14
15## 2. Data Collection and Use
16${apiName} allows users to store, retrieve, list, and delete data. The data stored can be of any type as inputted by the user. We do not restrict or control the content of the data stored. ${apiName} serves as a public database accessible to anyone with an API key.
17
18## 3. User Restrictions
23
24## 5. Data Management
25Given the nature of ${apiName}, there are no user accounts or user identification measures. The API operates like a public database where data can be added, viewed, and deleted by any user. Users should be aware that any data they input can be accessed, modified, or deleted by other users.
26
27## 6. Data Security
54 return { error: "already exists!" };
55 }
56 await users.push(user); // Assume insertion into a database
57 return { ok: true, users };
58 }
9## Contributing
10
11Forking this repo should mostly work, except for the sqlite database. You'll need to create the table & populate it with some data. [This script](https://www.val.town/v/stevekrouse/dateme_sqlite) *should* do it, but I think it has a couple bugs. If you're interested in contributing to this project contact me or comment on this val and I'll get it working for ya!
12
13## Todos
14
15- [ ] Make the SQLite database forkable and build a widget/workflow for that, ie fix @stevekrouse/dateme_sqlite
16- [ ] Require an email (that isn't shared publicly)
17 - [ ] Verify the email address with a "magic link"
33}
34
35// DATABASE
36const kv = new DenoSyntheticKV("passkeys_example");
37
2import {
3 convertSqliteType,
4 DatabaseHeader,
5 DatabaseResultSet,
6 DatabaseRow,
7 SqliteLikeBaseDriver,
8} from "https://esm.sh/@libsqlstudio/gui/driver";
9
10export function transformRawResult(raw: ResultSet): DatabaseResultSet {
11 const headerSet = new Set();
12
13 const headers: DatabaseHeader[] = raw.columns.map((colName, colIdx) => {
14 const colType = raw.columnTypes[colIdx];
15 let renameColName = colName;
34 a[b.name] = r[idx];
35 return a;
36 }, {} as DatabaseRow)
37 );
38
60 }
61
62 async transaction(stmts: InStatement[]): Promise<DatabaseResultSet[]> {
63 const r = await fetch(`/api/batch`, {
64 method: "POST",
80 }
81
82 async query(stmt: InStatement): Promise<DatabaseResultSet> {
83 const r = await fetch(`/api/execute`, {
84 method: "POST",
33}
34
35// DATABASE
36const kv = new DenoSyntheticKV("passkeys_example");
37