Keep sim_proxy profile data up to date

Interpret channel URLs the same as a channel #mention

docs/api.md

@@ -0,0 +1,52 @@
+# API
+
+There is a web API for getting information about things that are bridged with Out Of Your Element.
+
+The base URL is the URL of the particular OOYE instance, for example, https://bridge.cadence.moe.
+
+No authentication is required.
+
+I'm happy to add more endpoints, just ask for them.
+
+## Endpoint: GET /api/message
+
+|Query parameter|Type|Description|
+|---------------|----|-----------|
+|`message_id`|regexp `/^[0-9]+$/`|Discord message ID to look up information for|
+
+Response:
+
+```typescript
+{
+  source: "matrix" | "discord"                // Which platform the message originated on
+  matrix_author?: {                             // Only for Matrix messages; should be up-to-date rather than historical data
+    displayname: string,                        // Matrix user's current display name
+    avatar_url: string | null,                  // Absolute HTTP(S) URL to download the Matrix user's current avatar
+    mxid: string                                // Matrix user ID, can never change
+  },
+  events: [                                   // Data about each individual event
+    {
+      metadata: {                               // Data from OOYE's database about how bridging was performed
+        sender: string,                           // Same as matrix user ID
+        event_id: string,                         // Unique ID of the event on Matrix, can never change
+        event_type: "m.room.message" | string,    // Event type
+        event_subtype: "m.text" | string | null,  // For m.room.message events, this is the msgtype property
+        part: 0 | 1,                              // For multi-event messages, 0 if this is the first part
+        reaction_part: 0 | 1,                     // For multi-event messages, 0 if this is the last part
+        room_id: string,                          // Room ID that the event was sent in, linked to the Discord channel
+        source: number
+      },
+      raw: {                                    // Raw historical event data from the Matrix API. Contains at least these properties:
+        content: any,                             // The only non-metadata property, entirely client-generated
+        type: string,
+        room_id: string,
+        sender: string,
+        origin_server_ts: number,
+        unsigned?: any,
+        event_id: string,
+        user_id: string
+      }
+    }
+  ]
+}
+```

docs/developer-orientation.md

@@ -89,14 +89,14 @@ Whether you read those or not, I'm more than happy to help you 1-on-1 with codin
 
 # Dependency justification
 
-Total transitive production dependencies: 139
+Total transitive production dependencies: 137
 
 ### <font size="+2">🦕</font>
 
-* (31) better-sqlite3: SQLite3 is the best database, and this is the best library for it.
+* (31) better-sqlite3: SQLite is the best database, and this is the best library for it.
 * (27) @cloudrac3r/pug: Language for dynamic web pages. This is my fork. (I released code that hadn't made it to npm, and removed the heavy pug-filters feature.)
 * (16) stream-mime-type@1: This seems like the best option. Version 1 is used because version 2 is ESM-only.
-* (10) h3: Web server. OOYE needs this for the appservice listener, authmedia proxy, and more. 14 transitive dependencies is on the low end for a web server.
+* (9) h3: Web server. OOYE needs this for the appservice listener, authmedia proxy, self-service, and more.
 * (11) sharp: Image resizing and compositing. OOYE needs this for the emoji sprite sheets.
 
 ### <font size="-1">🪱</font>
@@ -118,11 +118,9 @@ Total transitive production dependencies: 139
 * (1) enquirer: Interactive prompting for the initial setup rather than forcing users to edit YAML non-interactively.
 * (0) entities: Looks fine. No dependencies.
 * (0) get-relative-path: Looks fine. No dependencies.
-* (0) get-stream: Only needed if content_length_workaround is true.
 * (1) heatsync: Module hot-reloader that I trust.
 * (1) js-yaml: Will be removed in the future after registration.yaml is converted to JSON.
 * (0) lru-cache: For holding unused nonce in memory and letting them be overwritten later if never used.
-* (0) minimist: It's already pulled in by better-sqlite3->prebuild-install.
 * (0) prettier-bytes: It does what I want and has no dependencies.
 * (0) snowtransfer: Discord API library with bring-your-own-caching that I trust.
 * (0) try-to-catch: Not strictly necessary, but it's already pulled in by supertape, so I may as well.

docs/get-started.md

@@ -11,7 +11,7 @@ You'll need:
 
 Follow these steps:
 
-1. [Get Node.js version 20 or later](https://nodejs.org/en/download/prebuilt-installer). If you're on Linux, you may prefer to install through system's package manager, though Debian and Ubuntu have hopelessly out of date packages.
+1. [Get Node.js version 22 or later](https://nodejs.org/en/download/prebuilt-installer). If you're on Linux, you may prefer to install through system's package manager, though Debian and Ubuntu have hopelessly out of date packages.
 
 1. Switch to a normal user account. (i.e. do not run any of the following commands as root or sudo.)
 
@@ -32,6 +32,16 @@ Follow these steps:
 
 1. Start the bridge: `npm run start`
 
+## Update
+
+New versions are announced in [#updates](https://matrix.to/#/#ooye-updates:cadence.moe) and listed on [releases](https://gitdab.com/cadence/out-of-your-element/releases). Here's how to update:
+
+1. Fetch the repo and checkout the latest release tag.
+
+1. Install dependencies: `npm install`
+
+1. Restart the bridge: Stop the currently running process, and then start the new one with `npm run start`
+
 # Get Started
 
 Visit the website on the domain name you set up, and click the button to add the bot to your Discord server.
@@ -71,7 +81,7 @@ server {
 	client_max_body_size 5M;
 
 	location / {
-		add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;
+		add_header Strict-Transport-Security "max-age=63072000; includeSubDomains" always;
 		proxy_pass http://127.0.0.1:6693;
 	}
 }

docs/why-does-the-bridge-have-a-website.md

@@ -50,10 +50,6 @@ When you use `npm run setup` as part of OOYE's initial setup, it will guide you
 
 When OOYE is running, the web server runs on port 6693. (To use a different port or a UNIX socket, edit registration.yaml's `socket` setting and restart.)
 
-It doesn't have to have its own dedicated domain name, you can also use a sub-path on an existing domain, like the domain of your Matrix homeserver. See the end of this document for more information.
+It doesn't have to have its own dedicated domain name, you can also use a sub-path on an existing domain, like the domain of your Matrix homeserver. You are likely already using a reverse proxy to run your homeserver, so this should just be a configuration change.
 
-You are likely already using a reverse proxy for running your homeserver, so this should just be a configuration change.
-
-## Example configurations
-
-[See the Get Started document for examples.](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md).
+[See here for sample configurations!](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md#appendix)

package-lock.json

@@ -1,19 +1,19 @@
 {
   "name": "out-of-your-element",
-  "version": "1.1.1",
+  "version": "3.1.0",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "out-of-your-element",
-      "version": "1.1.1",
+      "version": "3.1.0",
       "license": "AGPL-3.0-or-later",
       "dependencies": {
-        "@chriscdn/promise-semaphore": "^2.0.1",
-        "@cloudrac3r/discord-markdown": "^2.6.3",
+        "@chriscdn/promise-semaphore": "^3.0.1",
+        "@cloudrac3r/discord-markdown": "^2.6.6",
         "@cloudrac3r/giframe": "^0.4.3",
         "@cloudrac3r/html-template-tag": "^5.0.1",
-        "@cloudrac3r/in-your-element": "^1.0.0",
+        "@cloudrac3r/in-your-element": "^1.1.1",
         "@cloudrac3r/mixin-deep": "^3.0.1",
         "@cloudrac3r/pngjs": "^7.0.3",
         "@cloudrac3r/pug": "^4.0.4",
@@ -21,63 +21,38 @@
         "@stackoverflow/stacks": "^2.5.4",
         "@stackoverflow/stacks-icons": "^6.0.2",
         "ansi-colors": "^4.1.3",
-        "better-sqlite3": "^11.1.2",
+        "better-sqlite3": "^12.2.0",
         "chunk-text": "^2.0.1",
-        "cloudstorm": "^0.11.2",
-        "discord-api-types": "^0.37.119",
+        "cloudstorm": "^0.14.0",
+        "discord-api-types": "^0.38.19",
         "domino": "^2.1.6",
         "enquirer": "^2.4.1",
         "entities": "^5.0.0",
         "get-relative-path": "^1.0.2",
-        "get-stream": "^6.0.1",
-        "h3": "^1.12.0",
+        "h3": "^1.15.1",
         "heatsync": "^2.7.2",
         "htmx.org": "^2.0.4",
-        "lru-cache": "^10.4.3",
-        "minimist": "^1.2.8",
+        "lru-cache": "^11.0.2",
         "prettier-bytes": "^1.0.4",
         "sharp": "^0.33.4",
-        "snowtransfer": "^0.12.0",
+        "snowtransfer": "^0.15.0",
         "stream-mime-type": "^1.0.2",
         "try-to-catch": "^3.0.1",
         "uqr": "^0.1.2",
         "xxhash-wasm": "^1.0.2",
-        "zod": "^3.23.8"
+        "zod": "^4.0.17"
       },
       "devDependencies": {
         "@cloudrac3r/tap-dot": "^2.0.3",
-        "@types/node": "^18.16.0",
+        "@types/node": "^22.17.1",
         "c8": "^10.1.2",
         "cross-env": "^7.0.3",
-        "supertape": "^10.4.0"
+        "supertape": "^11.3.0"
       },
       "engines": {
         "node": ">=20"
       }
     },
-    "../in-your-element": {
-      "name": "@cloudrac3r/in-your-element",
-      "version": "0.0.0",
-      "extraneous": true,
-      "license": "AGPL-3.0-or-later",
-      "dependencies": {
-        "h3": "^1.12.0",
-        "zod": "^3.23.8"
-      },
-      "devDependencies": {
-        "@cloudrac3r/tap-dot": "^2.0.2",
-        "@types/node": "^18.19.42",
-        "c8": "^10.1.2",
-        "cross-env": "^7.0.3",
-        "mock-req": "^0.2.0",
-        "readable-mock-req": "^0.2.2",
-        "supertape": "^10.7.2",
-        "try-to-catch": "^3.0.1"
-      },
-      "engines": {
-        "node": ">=18"
-      }
-    },
     "../tap-dot": {
       "name": "@cloudrac3r/tap-dot",
       "version": "2.0.0",
@@ -144,9 +119,10 @@
       }
     },
     "node_modules/@chriscdn/promise-semaphore": {
-      "version": "2.0.9",
-      "resolved": "https://registry.npmjs.org/@chriscdn/promise-semaphore/-/promise-semaphore-2.0.9.tgz",
-      "integrity": "sha512-kKXJcm5gM8FN8O8U20H19/85b8R33K0Q2u5cnm9mfblK/7QcNChlOhCTWgnrr8wYiuF1ZbYIZcioxW79QfjnmQ=="
+      "version": "3.1.1",
+      "resolved": "https://registry.npmjs.org/@chriscdn/promise-semaphore/-/promise-semaphore-3.1.1.tgz",
+      "integrity": "sha512-ALLLLYlPfd/QZLptcVi6HQRK1zaCDWZoqYYw+axLmCatFs4gVTSZ5nqlyxwFe4qwR/K84HvOMa9hxda881FqMA==",
+      "license": "MIT"
     },
     "node_modules/@cloudcmd/stub": {
       "version": "4.0.1",
@@ -249,9 +225,10 @@
       }
     },
     "node_modules/@cloudrac3r/discord-markdown": {
-      "version": "2.6.3",
-      "resolved": "https://registry.npmjs.org/@cloudrac3r/discord-markdown/-/discord-markdown-2.6.3.tgz",
-      "integrity": "sha512-9pELy0wk0SiAfdj8QQDUpxZkzFum1c3/ybCG7DKs9EQQMPgP0AF7tNGEIfX76eJVisdGKftzuNd0xIfeGFKJxg==",
+      "version": "2.6.6",
+      "resolved": "https://registry.npmjs.org/@cloudrac3r/discord-markdown/-/discord-markdown-2.6.6.tgz",
+      "integrity": "sha512-4FNO7WmACPvcTrQjeLQLr9WRuP7JDUVUGFrRJvmAjiMs2UlUAsShfSRuU2SCqz3QqmX8vyJ06wy2hkjTTyRtbw==",
+      "license": "MIT",
       "dependencies": {
         "simple-markdown": "^0.7.3"
       }
@@ -270,13 +247,13 @@
       }
     },
     "node_modules/@cloudrac3r/in-your-element": {
-      "version": "1.0.0",
-      "resolved": "https://registry.npmjs.org/@cloudrac3r/in-your-element/-/in-your-element-1.0.0.tgz",
-      "integrity": "sha512-g6vdxNJtc9+Y0djClrc0xNwL6DFiEQ9ikWHBBDJ3iKAWygdjANnFJ/Q1DVMmNqUYRsQIN3yH1aIQICKA2CDXhQ==",
+      "version": "1.1.1",
+      "resolved": "https://registry.npmjs.org/@cloudrac3r/in-your-element/-/in-your-element-1.1.1.tgz",
+      "integrity": "sha512-AKp9vnSDA9wzJl4O3C/LA8jgI5m1r0M3MRBQGHcVVL22SrrZMdcy+kWjlZWK343KVLOkuTAISA2D+Jb/zyZS6A==",
       "license": "AGPL-3.0-or-later",
       "dependencies": {
         "h3": "^1.12.0",
-        "zod": "^3.23.8"
+        "zod": "^4.0.17"
       },
       "engines": {
         "node": ">=18"
@@ -753,6 +730,29 @@
         "url": "https://opencollective.com/libvips"
       }
     },
+    "node_modules/@isaacs/balanced-match": {
+      "version": "4.0.1",
+      "resolved": "https://registry.npmjs.org/@isaacs/balanced-match/-/balanced-match-4.0.1.tgz",
+      "integrity": "sha512-yzMTt9lEb8Gv7zRioUilSglI0c0smZ9k5D65677DLWLtWJaXIS3CqcGyUFByYKlnUj6TkjLVs54fBl6+TiGQDQ==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": "20 || >=22"
+      }
+    },
+    "node_modules/@isaacs/brace-expansion": {
+      "version": "5.0.0",
+      "resolved": "https://registry.npmjs.org/@isaacs/brace-expansion/-/brace-expansion-5.0.0.tgz",
+      "integrity": "sha512-ZT55BDLV0yv0RBm2czMiZ+SqCGO7AvmOM3G/w2xhVPH+te0aKgFjmBvGlL1dH+ql2tgGO3MVrbb3jCKyvpgnxA==",
+      "dev": true,
+      "license": "MIT",
+      "dependencies": {
+        "@isaacs/balanced-match": "^4.0.1"
+      },
+      "engines": {
+        "node": "20 || >=22"
+      }
+    },
     "node_modules/@isaacs/cliui": {
       "version": "8.0.2",
       "resolved": "https://registry.npmjs.org/@isaacs/cliui/-/cliui-8.0.2.tgz",
@@ -836,16 +836,37 @@
         "node": ">=8"
       }
     },
-    "node_modules/@jest/schemas": {
-      "version": "29.6.3",
-      "resolved": "https://registry.npmjs.org/@jest/schemas/-/schemas-29.6.3.tgz",
-      "integrity": "sha512-mo5j5X+jIZmJQveBKeS/clAueipV7KgiX1vMgCxam1RNYiqE1w62n0/tJJnHtjW8ZHcQco5gY85jA3mi0L+nSA==",
+    "node_modules/@jest/diff-sequences": {
+      "version": "30.0.1",
+      "resolved": "https://registry.npmjs.org/@jest/diff-sequences/-/diff-sequences-30.0.1.tgz",
+      "integrity": "sha512-n5H8QLDJ47QqbCNn5SuFjCRDrOLEZ0h8vAHCK5RL9Ls7Xa8AQLa/YxAc9UjFqoEDM48muwtBGjtMY5cr0PLDCw==",
       "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": "^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0"
+      }
+    },
+    "node_modules/@jest/get-type": {
+      "version": "30.0.1",
+      "resolved": "https://registry.npmjs.org/@jest/get-type/-/get-type-30.0.1.tgz",
+      "integrity": "sha512-AyYdemXCptSRFirI5EPazNxyPwAL0jXt3zceFjaj8NFiKP9pOi0bfXonf6qkf82z2t3QWPeLCWWw4stPBzctLw==",
+      "dev": true,
+      "license": "MIT",
+      "engines": {
+        "node": "^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0"
+      }
+    },
+    "node_modules/@jest/schemas": {
+      "version": "30.0.5",
+      "resolved": "https://registry.npmjs.org/@jest/schemas/-/schemas-30.0.5.tgz",
+      "integrity": "sha512-DmdYgtezMkh3cpU8/1uyXakv3tJRcmcXxBOcO0tbaozPwpmh4YMsnWrQm9ZmZMfa5ocbxzbFk6O4bDPEc/iAnA==",
+      "dev": true,
+      "license": "MIT",
       "dependencies": {
-        "@sinclair/typebox": "^0.27.8"
+        "@sinclair/typebox": "^0.34.0"
       },
       "engines": {
-        "node": "^14.15.0 || ^16.10.0 || >=18.0.0"
+        "node": "^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0"
       }
     },
     "node_modules/@jridgewell/resolve-uri": {
@@ -894,16 +915,17 @@
       }
     },
     "node_modules/@putout/cli-keypress": {
-      "version": "2.0.0",
-      "resolved": "https://registry.npmjs.org/@putout/cli-keypress/-/cli-keypress-2.0.0.tgz",
-      "integrity": "sha512-EXJv2HaXM+5scjoxE6Tf+o4+pxwL1tYJZJBDMygrF7cocjirGcU05GgNr9WHOaUPaVOpVjVU98ugYD7XJLmMkw==",
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/@putout/cli-keypress/-/cli-keypress-3.0.0.tgz",
+      "integrity": "sha512-RwODGTbcWNaulEPvVPdxH/vnddf5dE627G3s8gyou3kexa6zQerQHvbKFX0wywNdA3HD2O/9STPv/r5mjXFUgw==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "ci-info": "^4.0.0",
         "fullstore": "^3.0.0"
       },
       "engines": {
-        "node": ">=16"
+        "node": ">=20"
       }
     },
     "node_modules/@putout/cli-validate-args": {
@@ -920,15 +942,16 @@
       }
     },
     "node_modules/@sinclair/typebox": {
-      "version": "0.27.8",
-      "resolved": "https://registry.npmjs.org/@sinclair/typebox/-/typebox-0.27.8.tgz",
-      "integrity": "sha512-+Fj43pSMwJs4KRrH/938Uf+uAELIgVBmQzg/q1YG10djyfA3TnrU8N8XzqCh/okZdszqBQTZf96idMfE5lnwTA==",
-      "dev": true
+      "version": "0.34.38",
+      "resolved": "https://registry.npmjs.org/@sinclair/typebox/-/typebox-0.34.38.tgz",
+      "integrity": "sha512-HpkxMmc2XmZKhvaKIZZThlHmx1L0I/V1hWK1NubtlFnr6ZqdiOpV72TKudZUNQjZNsyDBay72qFEhEvb+bcwcA==",
+      "dev": true,
+      "license": "MIT"
     },
     "node_modules/@stackoverflow/stacks": {
-      "version": "2.5.4",
-      "resolved": "https://registry.npmjs.org/@stackoverflow/stacks/-/stacks-2.5.4.tgz",
-      "integrity": "sha512-k11SesaE+bZXNoa7IDinkpmu1BgVq7xIV1Gl1fZ5SM00hPi2S/vCyjIupLvJbSt1PAJTlnpBoUIM1ubb0Y7qFg==",
+      "version": "2.8.4",
+      "resolved": "https://registry.npmjs.org/@stackoverflow/stacks/-/stacks-2.8.4.tgz",
+      "integrity": "sha512-FfA7Bw7a0AQrMw3/bG6G4BUrZ698F7Cdk6HkR9T7jdaufORkiX5d16wI4j4b5Sqm1FwkaZAF+ZSKLL1w0tAsew==",
       "license": "MIT",
       "dependencies": {
         "@hotwired/stimulus": "^3.2.2",
@@ -936,9 +959,10 @@
       }
     },
     "node_modules/@stackoverflow/stacks-icons": {
-      "version": "6.1.0",
-      "resolved": "https://registry.npmjs.org/@stackoverflow/stacks-icons/-/stacks-icons-6.1.0.tgz",
-      "integrity": "sha512-l5M+gcBeJBAJaxX4ByWhT/eeWhFusdalP/sq0z0uiUM+pwMfDU/pEm2u4DWVYSvBKaskbH3K0oILWaeDvEfGmA=="
+      "version": "6.6.1",
+      "resolved": "https://registry.npmjs.org/@stackoverflow/stacks-icons/-/stacks-icons-6.6.1.tgz",
+      "integrity": "sha512-upa2jajYTKAHfILFbPWMsml0nlh4fbIEb2V9SS0txjOJEoZE2oBnNJXbg29vShp7Nyn1VwrMjaraX63WkKT07w==",
+      "license": "MIT"
     },
     "node_modules/@supertape/engine-loader": {
       "version": "2.0.0",
@@ -953,16 +977,17 @@
       }
     },
     "node_modules/@supertape/formatter-fail": {
-      "version": "3.0.2",
-      "resolved": "https://registry.npmjs.org/@supertape/formatter-fail/-/formatter-fail-3.0.2.tgz",
-      "integrity": "sha512-mSBnNprfLFmGvZkP+ODGroPLFCIN5BWE/06XaD5ghiTVWqek7eH8IDqvKyEduvuQu1O5tvQiaTwQsyxvikF+2w==",
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/@supertape/formatter-fail/-/formatter-fail-4.0.0.tgz",
+      "integrity": "sha512-+isArOXmGkIqH14PQoq2WhJmSwO8rzpQnhurVMuBmC+kYB96R95kRdjo/KO9d9yP1KoSjum0kX94s0SwqlZ8yA==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
-        "@supertape/formatter-tap": "^3.0.3",
+        "@supertape/formatter-tap": "^4.0.0",
         "fullstore": "^3.0.0"
       },
       "engines": {
-        "node": ">=16"
+        "node": ">=20"
       }
     },
     "node_modules/@supertape/formatter-json-lines": {
@@ -978,10 +1003,11 @@
       }
     },
     "node_modules/@supertape/formatter-progress-bar": {
-      "version": "6.1.0",
-      "resolved": "https://registry.npmjs.org/@supertape/formatter-progress-bar/-/formatter-progress-bar-6.1.0.tgz",
-      "integrity": "sha512-BVnLW08BMbF/Xf9DNxTtc5V5Ong4VCj0w46Ts2cc1EboX+RQGuxGO0/wrzTBTt4t30iUzFhG/t2g280MfLHutQ==",
+      "version": "7.0.0",
+      "resolved": "https://registry.npmjs.org/@supertape/formatter-progress-bar/-/formatter-progress-bar-7.0.0.tgz",
+      "integrity": "sha512-JDCT86hFJkoaqE/KS8BQsRaYiy3ipMpf0j+o+vwQMcFYm0mgG35JwbotBMUQM7LFifh68bTqU4xuewy7kUS1EA==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "chalk": "^5.3.0",
         "ci-info": "^4.0.0",
@@ -990,14 +1016,15 @@
         "once": "^1.4.0"
       },
       "engines": {
-        "node": ">=18"
+        "node": ">=20"
       }
     },
     "node_modules/@supertape/formatter-progress-bar/node_modules/chalk": {
-      "version": "5.3.0",
-      "resolved": "https://registry.npmjs.org/chalk/-/chalk-5.3.0.tgz",
-      "integrity": "sha512-dLitG79d+GV1Nb/VYcCDFivJeK1hiukt9QjRNVOsUtTy1rR1YJsmpGGTZ3qJos+uw7WmWF4wUwBd9jxjocFC2w==",
+      "version": "5.5.0",
+      "resolved": "https://registry.npmjs.org/chalk/-/chalk-5.5.0.tgz",
+      "integrity": "sha512-1tm8DTaJhPBG3bIkVeZt1iZM9GfSX2lzOeDVZH9R9ffRHpmHvxZ/QhgQH/aDTkswQVt+YHdXAdS/In/30OjCbg==",
       "dev": true,
+      "license": "MIT",
       "engines": {
         "node": "^12.17.0 || ^14.13 || >=16.0.0"
       },
@@ -1006,28 +1033,31 @@
       }
     },
     "node_modules/@supertape/formatter-short": {
-      "version": "2.0.1",
-      "resolved": "https://registry.npmjs.org/@supertape/formatter-short/-/formatter-short-2.0.1.tgz",
-      "integrity": "sha512-zxFrZfCccFV+bf6A7MCEqT/Xsf0Elc3qa0P3jShfdEfrpblEcpSo0T/Wd9jFwc7uHA3ABgxgcHy7LNIpyrFTCg==",
+      "version": "3.0.0",
+      "resolved": "https://registry.npmjs.org/@supertape/formatter-short/-/formatter-short-3.0.0.tgz",
+      "integrity": "sha512-lKiIMekxQgkF4YBj/IiFoRUQrF/Ow7D8zt9ZEBdHTkRys30vhRFn9557okECKGdpnAcSsoTHWwgikS/NPc3g/g==",
       "dev": true,
+      "license": "MIT",
       "engines": {
-        "node": ">=16"
+        "node": ">=20"
       }
     },
     "node_modules/@supertape/formatter-tap": {
-      "version": "3.0.3",
-      "resolved": "https://registry.npmjs.org/@supertape/formatter-tap/-/formatter-tap-3.0.3.tgz",
-      "integrity": "sha512-U5OuMotfYhGo9cZ8IgdAXRTH5Yy8yfLDZzYo1upTPTwlJJquKwtvuz7ptiB7BN3OFr5YakkDYlFxOYPcLo7urg==",
+      "version": "4.0.0",
+      "resolved": "https://registry.npmjs.org/@supertape/formatter-tap/-/formatter-tap-4.0.0.tgz",
+      "integrity": "sha512-cupeiik+FeTQ24d0fihNdS901Ct720UhUqgtPl2DiLWadEIT/B8+TIB4MG60sTmaE8xclbCieanbS/I94CQTPw==",
       "dev": true,
+      "license": "MIT",
       "engines": {
-        "node": ">=16"
+        "node": ">=20"
       }
     },
     "node_modules/@supertape/formatter-time": {
-      "version": "1.0.2",
-      "resolved": "https://registry.npmjs.org/@supertape/formatter-time/-/formatter-time-1.0.2.tgz",
-      "integrity": "sha512-QihQWA/3LSNuODHrL8MGNHkdRunaEqNQkuMUDGNgEQO8MYBB0d83WGlNxDFGjn4kRlq47hovw3Skq7Btb2i2JA==",
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/@supertape/formatter-time/-/formatter-time-2.0.0.tgz",
+      "integrity": "sha512-5UPvVHwpg5ZJmz0nII2f5rBFqNdMxHQnBybetmhgkSDIZHb+3NTPz/VrDggZERWOGxmIf4NKebaA+BWHTBQMeA==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "chalk": "^5.3.0",
         "ci-info": "^4.0.0",
@@ -1037,14 +1067,15 @@
         "timer-node": "^5.0.7"
       },
       "engines": {
-        "node": ">=18"
+        "node": ">=20"
       }
     },
     "node_modules/@supertape/formatter-time/node_modules/chalk": {
-      "version": "5.3.0",
-      "resolved": "https://registry.npmjs.org/chalk/-/chalk-5.3.0.tgz",
-      "integrity": "sha512-dLitG79d+GV1Nb/VYcCDFivJeK1hiukt9QjRNVOsUtTy1rR1YJsmpGGTZ3qJos+uw7WmWF4wUwBd9jxjocFC2w==",
+      "version": "5.5.0",
+      "resolved": "https://registry.npmjs.org/chalk/-/chalk-5.5.0.tgz",
+      "integrity": "sha512-1tm8DTaJhPBG3bIkVeZt1iZM9GfSX2lzOeDVZH9R9ffRHpmHvxZ/QhgQH/aDTkswQVt+YHdXAdS/In/30OjCbg==",
       "dev": true,
+      "license": "MIT",
       "engines": {
         "node": "^12.17.0 || ^14.13 || >=16.0.0"
       },
@@ -1076,13 +1107,13 @@
       "dev": true
     },
     "node_modules/@types/node": {
-      "version": "18.19.76",
-      "resolved": "https://registry.npmjs.org/@types/node/-/node-18.19.76.tgz",
-      "integrity": "sha512-yvR7Q9LdPz2vGpmpJX5LolrgRdWvB67MJKDPSgIIzpFbaf9a1j/f5DnLp5VDyHGMR0QZHlTr1afsD87QCXFHKw==",
+      "version": "22.18.0",
+      "resolved": "https://registry.npmjs.org/@types/node/-/node-22.18.0.tgz",
+      "integrity": "sha512-m5ObIqwsUp6BZzyiy4RdZpzWGub9bqLJMvZDD0QMXhxjqMHMENlj+SqF5QxoUwaQNFe+8kz8XM8ZQhqkQPTgMQ==",
       "dev": true,
       "license": "MIT",
       "dependencies": {
-        "undici-types": "~5.26.4"
+        "undici-types": "~6.21.0"
       }
     },
     "node_modules/@types/prop-types": {
@@ -1208,13 +1239,17 @@
       ]
     },
     "node_modules/better-sqlite3": {
-      "version": "11.8.1",
-      "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-11.8.1.tgz",
-      "integrity": "sha512-9BxNaBkblMjhJW8sMRZxnxVTRgbRmssZW0Oxc1MPBTfiR+WW21e2Mk4qu8CzrcZb1LwPCnFsfDEzq+SNcBU8eg==",
+      "version": "12.2.0",
+      "resolved": "https://registry.npmjs.org/better-sqlite3/-/better-sqlite3-12.2.0.tgz",
+      "integrity": "sha512-eGbYq2CT+tos1fBwLQ/tkBt9J5M3JEHjku4hbvQUePCckkvVf14xWj+1m7dGoK81M/fOjFT7yM9UMeKT/+vFLQ==",
       "hasInstallScript": true,
+      "license": "MIT",
       "dependencies": {
         "bindings": "^1.5.0",
         "prebuild-install": "^7.1.1"
+      },
+      "engines": {
+        "node": "20.x || 22.x || 23.x || 24.x"
       }
     },
     "node_modules/bindings": {
@@ -1272,10 +1307,11 @@
       }
     },
     "node_modules/brace-expansion": {
-      "version": "2.0.1",
-      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.1.tgz",
-      "integrity": "sha512-XnAIvQ8eM+kC6aULx6wuQiwVsnzsi9d3WxzV3FpWTGA19F621kwdbsAcFKXgKUHZWsy+mY6iL1sHTxWEFCytDA==",
+      "version": "2.0.2",
+      "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.0.2.tgz",
+      "integrity": "sha512-Jt0vHyM+jmUBqojB7E1NIYadt0vI0Qxjxd2TErW94wDz+E2LAm5vKMXXwg6ZZBTHPuUlDgQHKXvjGBdfcF1ZDQ==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "balanced-match": "^1.0.0"
       }
@@ -1352,9 +1388,9 @@
       }
     },
     "node_modules/ci-info": {
-      "version": "4.0.0",
-      "resolved": "https://registry.npmjs.org/ci-info/-/ci-info-4.0.0.tgz",
-      "integrity": "sha512-TdHqgGf9odd8SXNuxtUBVx8Nv+qZOejE6qyqiy5NtbYYQOeFa6zmHkxlPzmaLxWWHsU6nJmB7AETdVPi+2NBUg==",
+      "version": "4.3.0",
+      "resolved": "https://registry.npmjs.org/ci-info/-/ci-info-4.3.0.tgz",
+      "integrity": "sha512-l+2bNRMiQgcfILUi33labAZYIWlH1kWDp+ecNo5iisRKrbm0xcRyCww71/YU0Fkw0mAFpz9bJayXPjey6vkmaQ==",
       "dev": true,
       "funding": [
         {
@@ -1362,6 +1398,7 @@
           "url": "https://github.com/sponsors/sibiraj-s"
         }
       ],
+      "license": "MIT",
       "engines": {
         "node": ">=8"
       }
@@ -1371,6 +1408,7 @@
       "resolved": "https://registry.npmjs.org/cli-progress/-/cli-progress-3.12.0.tgz",
       "integrity": "sha512-tRkV3HJ1ASwm19THiiLIXLO7Im7wlTuKnvkYaTkyoAPefqjNg7W7DHKUlGRxy9vxDvbyCYQkQozvptuMkGCg8A==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "string-width": "^4.2.3"
       },
@@ -1414,16 +1452,16 @@
       }
     },
     "node_modules/cloudstorm": {
-      "version": "0.11.4",
-      "resolved": "https://registry.npmjs.org/cloudstorm/-/cloudstorm-0.11.4.tgz",
-      "integrity": "sha512-fk0tAyZmUBWrxELyXaKh19s1RJucmhmvTMfB/LrvdRHdUvc20VkD7qCrFaQHSQ/+kzwhSHVY43zNAjtz93pH9A==",
+      "version": "0.14.1",
+      "resolved": "https://registry.npmjs.org/cloudstorm/-/cloudstorm-0.14.1.tgz",
+      "integrity": "sha512-x95WCKg818E1rE1Ru45NPD3RoIq0pg3WxwvF0GE7Eq07pAeLcjSRqM1lUmbmfjdOqZrWdSRYA1NETVZ8QhVrIA==",
       "license": "MIT",
       "dependencies": {
-        "discord-api-types": "^0.37.119",
-        "snowtransfer": "^0.12.0"
+        "discord-api-types": "^0.38.21",
+        "snowtransfer": "^0.15.0"
       },
       "engines": {
-        "node": ">=16.15.0"
+        "node": ">=22.0.0"
       }
     },
     "node_modules/color": {
@@ -1517,9 +1555,10 @@
       }
     },
     "node_modules/crossws": {
-      "version": "0.3.4",
-      "resolved": "https://registry.npmjs.org/crossws/-/crossws-0.3.4.tgz",
-      "integrity": "sha512-uj0O1ETYX1Bh6uSgktfPvwDiPYGQ3aI4qVsaC/LWpkIzGj1nUYm5FK3K+t11oOlpN01lGbprFCH4wBlKdJjVgw==",
+      "version": "0.3.5",
+      "resolved": "https://registry.npmjs.org/crossws/-/crossws-0.3.5.tgz",
+      "integrity": "sha512-ojKiDvcmByhwa8YYqbQI/hg7MEU0NC03+pSdEq4ZUnZR9xXpwk7E43SMNGkn+JxJGPFtNvQ48+vV2p+P1ml5PA==",
+      "license": "MIT",
       "dependencies": {
         "uncrypto": "^0.1.3"
       }
@@ -1563,9 +1602,9 @@
       "integrity": "sha512-mEQCMmwJu317oSz8CwdIOdwf3xMif1ttiM8LTufzc3g6kR+9Pe236twL8j3IYT1F7GfRgGcW6MWxzZjLIkuHIg=="
     },
     "node_modules/destr": {
-      "version": "2.0.3",
-      "resolved": "https://registry.npmjs.org/destr/-/destr-2.0.3.tgz",
-      "integrity": "sha512-2N3BOUU4gYMpTP24s5rF5iP7BDr7uNTCs4ozw3kf/eKfvWSIu93GEBi5m427YoyJoeOzQ5smuu4nNAPGb8idSQ==",
+      "version": "2.0.5",
+      "resolved": "https://registry.npmjs.org/destr/-/destr-2.0.5.tgz",
+      "integrity": "sha512-ugFTXCtDZunbzasqBxrK93Ik/DRYsO6S/fedkWEMKqt04xZ4csmnmwGDBAb07QWNaGMAmnTIemsYZCksjATwsA==",
       "license": "MIT"
     },
     "node_modules/detect-libc": {
@@ -1576,19 +1615,14 @@
         "node": ">=8"
       }
     },
-    "node_modules/diff-sequences": {
-      "version": "29.6.3",
-      "resolved": "https://registry.npmjs.org/diff-sequences/-/diff-sequences-29.6.3.tgz",
-      "integrity": "sha512-EjePK1srD3P08o2j4f0ExnylqRs5B9tJjcp9t1krH2qRi8CCdsYfwe9JgSLurFBWwq4uOlipzfk5fHNvwFKr8Q==",
-      "dev": true,
-      "engines": {
-        "node": "^14.15.0 || ^16.10.0 || >=18.0.0"
-      }
-    },
     "node_modules/discord-api-types": {
-      "version": "0.37.119",
-      "resolved": "https://registry.npmjs.org/discord-api-types/-/discord-api-types-0.37.119.tgz",
-      "integrity": "sha512-WasbGFXEB+VQWXlo6IpW3oUv73Yuau1Ig4AZF/m13tXcTKnMpc/mHjpztIlz4+BM9FG9BHQkEXiPto3bKduQUg=="
+      "version": "0.38.22",
+      "resolved": "https://registry.npmjs.org/discord-api-types/-/discord-api-types-0.38.22.tgz",
+      "integrity": "sha512-2gnYrgXN3yTlv2cKBISI/A8btZwsSZLwKpIQXeI1cS8a7W7wP3sFVQOm3mPuuinTD8jJCKGPGNH399zE7Un1kA==",
+      "license": "MIT",
+      "workspaces": [
+        "scripts/actions/documentation"
+      ]
     },
     "node_modules/doctypes": {
       "version": "1.1.0",
@@ -1733,12 +1767,13 @@
       "dev": true
     },
     "node_modules/foreground-child": {
-      "version": "3.2.1",
-      "resolved": "https://registry.npmjs.org/foreground-child/-/foreground-child-3.2.1.tgz",
-      "integrity": "sha512-PXUUyLqrR2XCWICfv6ukppP96sdFwWbNEnfEMt7jNsISjMsvaLNinAHNDYyvkyU+SZG2BTSbT5NjG+vZslfGTA==",
+      "version": "3.3.1",
+      "resolved": "https://registry.npmjs.org/foreground-child/-/foreground-child-3.3.1.tgz",
+      "integrity": "sha512-gIXjKqtFuWEgzFRJA9WCQeSJLZDjgJUOMCMzxtvFq/37KojM1BFGufqsCy0r4qSQmYLsZYMeyRqzIWOMup03sw==",
       "dev": true,
+      "license": "ISC",
       "dependencies": {
-        "cross-spawn": "^7.0.0",
+        "cross-spawn": "^7.0.6",
         "signal-exit": "^4.0.1"
       },
       "engines": {
@@ -1795,17 +1830,6 @@
         "source-map": "^0.6.1"
       }
     },
-    "node_modules/get-stream": {
-      "version": "6.0.1",
-      "resolved": "https://registry.npmjs.org/get-stream/-/get-stream-6.0.1.tgz",
-      "integrity": "sha512-ts6Wi+2j3jQjqi70w5AlN8DFnkSwC+MqmxEzdEALB2qXZYV3X/b1CTfgPLGJNMeAWxdPfU8FO1ms3NUfaHCPYg==",
-      "engines": {
-        "node": ">=10"
-      },
-      "funding": {
-        "url": "https://github.com/sponsors/sindresorhus"
-      }
-    },
     "node_modules/github-from-package": {
       "version": "0.0.0",
       "resolved": "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz",
@@ -1833,19 +1857,19 @@
       }
     },
     "node_modules/h3": {
-      "version": "1.15.0",
-      "resolved": "https://registry.npmjs.org/h3/-/h3-1.15.0.tgz",
-      "integrity": "sha512-OsjX4JW8J4XGgCgEcad20pepFQWnuKH+OwkCJjogF3C+9AZ1iYdtB4hX6vAb5DskBiu5ljEXqApINjR8CqoCMQ==",
+      "version": "1.15.4",
+      "resolved": "https://registry.npmjs.org/h3/-/h3-1.15.4.tgz",
+      "integrity": "sha512-z5cFQWDffyOe4vQ9xIqNfCZdV4p//vy6fBnr8Q1AWnVZ0teurKMG66rLj++TKwKPUP3u7iMUvrvKaEUiQw2QWQ==",
+      "license": "MIT",
       "dependencies": {
         "cookie-es": "^1.2.2",
-        "crossws": "^0.3.3",
+        "crossws": "^0.3.5",
         "defu": "^6.1.4",
-        "destr": "^2.0.3",
+        "destr": "^2.0.5",
         "iron-webcrypto": "^1.2.1",
-        "node-mock-http": "^1.0.0",
-        "ohash": "^1.1.4",
+        "node-mock-http": "^1.0.2",
         "radix3": "^1.1.2",
-        "ufo": "^1.5.4",
+        "ufo": "^1.6.1",
         "uncrypto": "^0.1.3"
       }
     },
@@ -1871,9 +1895,9 @@
       }
     },
     "node_modules/heatsync": {
-      "version": "2.7.2",
-      "resolved": "https://registry.npmjs.org/heatsync/-/heatsync-2.7.2.tgz",
-      "integrity": "sha512-1djRg4eufv5q+CRy5SuZSiV3j53KIDSGkDubJB+vXY1OE+AnTkw5HIJxi+0vEjHjX+wbH5syYQunQ/ElAgoEmg==",
+      "version": "2.8.2",
+      "resolved": "https://registry.npmjs.org/heatsync/-/heatsync-2.8.2.tgz",
+      "integrity": "sha512-zO5ivWP1NYoYmngdqVxzeQGX2Q68rfLkXKbO8Dhcguj5eS2eBDVpcWPh3+KCQagM7xYP5QVzvrUryWDu4mt6Eg==",
       "license": "MIT",
       "dependencies": {
         "backtracker": "^4.0.0"
@@ -1894,9 +1918,10 @@
       "dev": true
     },
     "node_modules/htmx.org": {
-      "version": "2.0.4",
-      "resolved": "https://registry.npmjs.org/htmx.org/-/htmx.org-2.0.4.tgz",
-      "integrity": "sha512-HLxMCdfXDOJirs3vBZl/ZLoY+c7PfM4Ahr2Ad4YXh6d22T5ltbTXFFkpx9Tgb2vvmWFMbIc3LqN2ToNkZJvyYQ=="
+      "version": "2.0.6",
+      "resolved": "https://registry.npmjs.org/htmx.org/-/htmx.org-2.0.6.tgz",
+      "integrity": "sha512-7ythjYneGSk3yCHgtCnQeaoF+D+o7U2LF37WU3O0JYv3gTZSicdEFiI/Ai/NJyC5ZpYJWMpUb11OC5Lr6AfAqA==",
+      "license": "0BSD"
     },
     "node_modules/ieee754": {
       "version": "1.2.1",
@@ -2030,27 +2055,19 @@
       }
     },
     "node_modules/jest-diff": {
-      "version": "29.7.0",
-      "resolved": "https://registry.npmjs.org/jest-diff/-/jest-diff-29.7.0.tgz",
-      "integrity": "sha512-LMIgiIrhigmPrs03JHpxUh2yISK3vLFPkAodPeo0+BuF7wA2FoQbkEg1u8gBYBThncu7e1oEDUfIXVuTqLRUjw==",
+      "version": "30.0.5",
+      "resolved": "https://registry.npmjs.org/jest-diff/-/jest-diff-30.0.5.tgz",
+      "integrity": "sha512-1UIqE9PoEKaHcIKvq2vbibrCog4Y8G0zmOxgQUVEiTqwR5hJVMCoDsN1vFvI5JvwD37hjueZ1C4l2FyGnfpE0A==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
-        "chalk": "^4.0.0",
-        "diff-sequences": "^29.6.3",
-        "jest-get-type": "^29.6.3",
-        "pretty-format": "^29.7.0"
+        "@jest/diff-sequences": "30.0.1",
+        "@jest/get-type": "30.0.1",
+        "chalk": "^4.1.2",
+        "pretty-format": "30.0.5"
       },
       "engines": {
-        "node": "^14.15.0 || ^16.10.0 || >=18.0.0"
-      }
-    },
-    "node_modules/jest-get-type": {
-      "version": "29.6.3",
-      "resolved": "https://registry.npmjs.org/jest-get-type/-/jest-get-type-29.6.3.tgz",
-      "integrity": "sha512-zrteXnqYxfQh7l5FHyL38jL39di8H8rHoecLH3JNxH3BwOrBsNeabdap5e0I23lD4HHI8W5VFBZqG4Eaq5LNcw==",
-      "dev": true,
-      "engines": {
-        "node": "^14.15.0 || ^16.10.0 || >=18.0.0"
+        "node": "^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0"
       }
     },
     "node_modules/js-stringify": {
@@ -2058,6 +2075,13 @@
       "resolved": "https://registry.npmjs.org/js-stringify/-/js-stringify-1.0.2.tgz",
       "integrity": "sha512-rtS5ATOo2Q5k1G+DADISilDA6lv79zIiwFd6CcjuIxGKLFm5C+RLImRscVap9k55i+MOZwgliw+NejvkLuGD5g=="
     },
+    "node_modules/json-with-bigint": {
+      "version": "3.4.4",
+      "resolved": "https://registry.npmjs.org/json-with-bigint/-/json-with-bigint-3.4.4.tgz",
+      "integrity": "sha512-AhpYAAaZsPjU7smaBomDt1SOQshi9rEm6BlTbfVwsG1vNmeHKtEedJi62sHZzJTyKNtwzmNnrsd55kjwJ7054A==",
+      "dev": true,
+      "license": "MIT"
+    },
     "node_modules/just-kebab-case": {
       "version": "4.2.0",
       "resolved": "https://registry.npmjs.org/just-kebab-case/-/just-kebab-case-4.2.0.tgz",
@@ -2080,10 +2104,13 @@
       }
     },
     "node_modules/lru-cache": {
-      "version": "10.4.3",
-      "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-10.4.3.tgz",
-      "integrity": "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ==",
-      "license": "ISC"
+      "version": "11.1.0",
+      "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.1.0.tgz",
+      "integrity": "sha512-QIXZUBJUx+2zHUdQujWejBkcD9+cs94tLn0+YL8UrCh+D5sCXZ4c7LaEH48pNwRY3MLDgqUFyhlCyjJPf1WP0A==",
+      "license": "ISC",
+      "engines": {
+        "node": "20 || >=22"
+      }
     },
     "node_modules/make-dir": {
       "version": "4.0.0",
@@ -2185,9 +2212,10 @@
       }
     },
     "node_modules/node-mock-http": {
-      "version": "1.0.0",
-      "resolved": "https://registry.npmjs.org/node-mock-http/-/node-mock-http-1.0.0.tgz",
-      "integrity": "sha512-0uGYQ1WQL1M5kKvGRXWQ3uZCHtLTO8hln3oBjIusM75WoesZ909uQJs/Hb946i2SS+Gsrhkaa6iAO17jRIv6DQ=="
+      "version": "1.0.2",
+      "resolved": "https://registry.npmjs.org/node-mock-http/-/node-mock-http-1.0.2.tgz",
+      "integrity": "sha512-zWaamgDUdo9SSLw47we78+zYw/bDr5gH8pH7oRRs8V3KmBtu8GLgGIbV2p/gRPd3LWpEOpjQj7X1FOU3VFMJ8g==",
+      "license": "MIT"
     },
     "node_modules/object-assign": {
       "version": "4.1.1",
@@ -2197,11 +2225,6 @@
         "node": ">=0.10.0"
       }
     },
-    "node_modules/ohash": {
-      "version": "1.1.4",
-      "resolved": "https://registry.npmjs.org/ohash/-/ohash-1.1.4.tgz",
-      "integrity": "sha512-FlDryZAahJmEF3VR3w1KogSEdWX3WhA5GPakFx4J81kEAiHyLMpdLLElS8n8dfNadMgAne/MywcvmogzscVt4g=="
-    },
     "node_modules/once": {
       "version": "1.4.0",
       "resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
@@ -2287,6 +2310,12 @@
         "url": "https://github.com/sponsors/isaacs"
       }
     },
+    "node_modules/path-scurry/node_modules/lru-cache": {
+      "version": "10.4.3",
+      "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-10.4.3.tgz",
+      "integrity": "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ==",
+      "dev": true
+    },
     "node_modules/peek-readable": {
       "version": "4.1.0",
       "resolved": "https://registry.npmjs.org/peek-readable/-/peek-readable-4.1.0.tgz",
@@ -2330,17 +2359,18 @@
       "integrity": "sha512-dLbWOa4xBn+qeWeIF60qRoB6Pk2jX5P3DIVgOQyMyvBpu931Q+8dXz8X0snJiFkQdohDDLnZQECjzsAj75hgZQ=="
     },
     "node_modules/pretty-format": {
-      "version": "29.7.0",
-      "resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-29.7.0.tgz",
-      "integrity": "sha512-Pdlw/oPxN+aXdmM9R00JVC9WVFoCLTKJvDVLgmJ+qAffBMxsV85l/Lu7sNx4zSzPyoL2euImuEwHhOXdEgNFZQ==",
+      "version": "30.0.5",
+      "resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-30.0.5.tgz",
+      "integrity": "sha512-D1tKtYvByrBkFLe2wHJl2bwMJIiT8rW+XA+TiataH79/FszLQMrpGEvzUVkzPau7OCO0Qnrhpe87PqtOAIB8Yw==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
-        "@jest/schemas": "^29.6.3",
-        "ansi-styles": "^5.0.0",
-        "react-is": "^18.0.0"
+        "@jest/schemas": "30.0.5",
+        "ansi-styles": "^5.2.0",
+        "react-is": "^18.3.1"
       },
       "engines": {
-        "node": "^14.15.0 || ^16.10.0 || >=18.0.0"
+        "node": "^18.14.0 || ^20.0.0 || ^22.0.0 || >=24.0.0"
       }
     },
     "node_modules/pretty-format/node_modules/ansi-styles": {
@@ -2348,6 +2378,7 @@
       "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-5.2.0.tgz",
       "integrity": "sha512-Cxwpt2SfTzTtXcfOlzGEee8O+c+MmUgGrNiBcXnuWxuFJHe6a5Hz7qwhwe5OgaSYI0IJvkLqWX1ASG+cJOkEiA==",
       "dev": true,
+      "license": "MIT",
       "engines": {
         "node": ">=10"
       },
@@ -2458,10 +2489,11 @@
       "license": "MIT"
     },
     "node_modules/react-is": {
-      "version": "18.2.0",
-      "resolved": "https://registry.npmjs.org/react-is/-/react-is-18.2.0.tgz",
-      "integrity": "sha512-xWGDIW6x921xtzPkhiULtthJHoJvBbF3q26fzloPCK0hsvxtPVelvftw3zjbHWSkR2km9Z+4uxbDDK/6Zw9B8w==",
-      "dev": true
+      "version": "18.3.1",
+      "resolved": "https://registry.npmjs.org/react-is/-/react-is-18.3.1.tgz",
+      "integrity": "sha512-/LLMVyas0ljjAtoYiPqYiL8VWXzUUdThrmU5+n20DZv+a+ClRoevUzw5JxU+Ieh5/c87ytoTBV9G1FiKfNJdmg==",
+      "dev": true,
+      "license": "MIT"
     },
     "node_modules/readable-web-to-node-stream": {
       "version": "3.0.2",
@@ -2687,12 +2719,12 @@
       }
     },
     "node_modules/snowtransfer": {
-      "version": "0.12.0",
-      "resolved": "https://registry.npmjs.org/snowtransfer/-/snowtransfer-0.12.0.tgz",
-      "integrity": "sha512-EmVTAeSXtA7ZlTqwmZxe5JwRTm4FOXEOqMOzGu8fdVSoqXjcWgQ8IfaIRu/54FamOMjOmcxnpTyNPj5MUqWxpA==",
+      "version": "0.15.0",
+      "resolved": "https://registry.npmjs.org/snowtransfer/-/snowtransfer-0.15.0.tgz",
+      "integrity": "sha512-kEDGKtFiH5nSkHsDZonEUuDx99lUasJoZ7AGrgvE8HzVG59vjvqc//C+pjWj4DuJqTj4Q+Z1L/M/MYNim8F2VA==",
       "license": "MIT",
       "dependencies": {
-        "discord-api-types": "^0.37.119"
+        "discord-api-types": "^0.38.21"
       },
       "engines": {
         "node": ">=16.15.0"
@@ -2897,41 +2929,126 @@
       }
     },
     "node_modules/supertape": {
-      "version": "10.10.0",
-      "resolved": "https://registry.npmjs.org/supertape/-/supertape-10.10.0.tgz",
-      "integrity": "sha512-Zxww3DePaNlRJgy4XVukEU98254DWwNbV0Ch1jJcCWZxD0AJM9fIJG1bbFmVXXdYe0G0+YnpfrP12nVM2K+cEg==",
+      "version": "11.3.0",
+      "resolved": "https://registry.npmjs.org/supertape/-/supertape-11.3.0.tgz",
+      "integrity": "sha512-2LP36xLtxsb3bBYrfvWIilhWpA/vs7/vIgElpsqEhZZ0vcOAMlhMIxH6eHAl5u9KcxGD28IrJrw8lREqeMtZeQ==",
       "dev": true,
+      "license": "MIT",
       "dependencies": {
         "@cloudcmd/stub": "^4.0.0",
-        "@putout/cli-keypress": "^2.0.0",
+        "@putout/cli-keypress": "^3.0.0",
         "@putout/cli-validate-args": "^2.0.0",
         "@supertape/engine-loader": "^2.0.0",
-        "@supertape/formatter-fail": "^3.0.0",
+        "@supertape/formatter-fail": "^4.0.0",
         "@supertape/formatter-json-lines": "^2.0.0",
-        "@supertape/formatter-progress-bar": "^6.0.0",
-        "@supertape/formatter-short": "^2.0.0",
-        "@supertape/formatter-tap": "^3.0.0",
-        "@supertape/formatter-time": "^1.0.0",
+        "@supertape/formatter-progress-bar": "^7.0.0",
+        "@supertape/formatter-short": "^3.0.0",
+        "@supertape/formatter-tap": "^4.0.0",
+        "@supertape/formatter-time": "^2.0.0",
         "@supertape/operator-stub": "^3.0.0",
         "cli-progress": "^3.8.2",
         "flatted": "^3.3.1",
         "fullstore": "^3.0.0",
-        "glob": "^10.0.0",
-        "jest-diff": "^29.0.1",
+        "glob": "^11.0.1",
+        "jest-diff": "^30.0.3",
+        "json-with-bigint": "^3.4.4",
         "once": "^1.4.0",
         "resolve": "^1.17.0",
         "stacktracey": "^2.1.7",
         "strip-ansi": "^7.0.0",
         "try-to-catch": "^3.0.0",
         "wraptile": "^3.0.0",
-        "yargs-parser": "^21.0.0"
+        "yargs-parser": "^22.0.0"
       },
       "bin": {
         "supertape": "bin/tracer.mjs",
         "tape": "bin/tracer.mjs"
       },
       "engines": {
-        "node": ">=18"
+        "node": ">=20"
+      }
+    },
+    "node_modules/supertape/node_modules/glob": {
+      "version": "11.0.3",
+      "resolved": "https://registry.npmjs.org/glob/-/glob-11.0.3.tgz",
+      "integrity": "sha512-2Nim7dha1KVkaiF4q6Dj+ngPPMdfvLJEOpZk/jKiUAkqKebpGAWQXAq9z1xu9HKu5lWfqw/FASuccEjyznjPaA==",
+      "dev": true,
+      "license": "ISC",
+      "dependencies": {
+        "foreground-child": "^3.3.1",
+        "jackspeak": "^4.1.1",
+        "minimatch": "^10.0.3",
+        "minipass": "^7.1.2",
+        "package-json-from-dist": "^1.0.0",
+        "path-scurry": "^2.0.0"
+      },
+      "bin": {
+        "glob": "dist/esm/bin.mjs"
+      },
+      "engines": {
+        "node": "20 || >=22"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/isaacs"
+      }
+    },
+    "node_modules/supertape/node_modules/jackspeak": {
+      "version": "4.1.1",
+      "resolved": "https://registry.npmjs.org/jackspeak/-/jackspeak-4.1.1.tgz",
+      "integrity": "sha512-zptv57P3GpL+O0I7VdMJNBZCu+BPHVQUk55Ft8/QCJjTVxrnJHuVuX/0Bl2A6/+2oyR/ZMEuFKwmzqqZ/U5nPQ==",
+      "dev": true,
+      "license": "BlueOak-1.0.0",
+      "dependencies": {
+        "@isaacs/cliui": "^8.0.2"
+      },
+      "engines": {
+        "node": "20 || >=22"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/isaacs"
+      }
+    },
+    "node_modules/supertape/node_modules/minimatch": {
+      "version": "10.0.3",
+      "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.0.3.tgz",
+      "integrity": "sha512-IPZ167aShDZZUMdRk66cyQAW3qr0WzbHkPdMYa8bzZhlHhO3jALbKdxcaak7W9FfT2rZNpQuUu4Od7ILEpXSaw==",
+      "dev": true,
+      "license": "ISC",
+      "dependencies": {
+        "@isaacs/brace-expansion": "^5.0.0"
+      },
+      "engines": {
+        "node": "20 || >=22"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/isaacs"
+      }
+    },
+    "node_modules/supertape/node_modules/path-scurry": {
+      "version": "2.0.0",
+      "resolved": "https://registry.npmjs.org/path-scurry/-/path-scurry-2.0.0.tgz",
+      "integrity": "sha512-ypGJsmGtdXUOeM5u93TyeIEfEhM6s+ljAhrk5vAvSx8uyY/02OvrZnA0YNGUrPXfpJMgI1ODd3nwz8Npx4O4cg==",
+      "dev": true,
+      "license": "BlueOak-1.0.0",
+      "dependencies": {
+        "lru-cache": "^11.0.0",
+        "minipass": "^7.1.2"
+      },
+      "engines": {
+        "node": "20 || >=22"
+      },
+      "funding": {
+        "url": "https://github.com/sponsors/isaacs"
+      }
+    },
+    "node_modules/supertape/node_modules/yargs-parser": {
+      "version": "22.0.0",
+      "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-22.0.0.tgz",
+      "integrity": "sha512-rwu/ClNdSMpkSrUb+d6BRsSkLUq1fmfsY6TOpYzTwvwkg1/NRG85KBy3kq++A8LKQwX6lsu+aWad+2khvuXrqw==",
+      "dev": true,
+      "license": "ISC",
+      "engines": {
+        "node": "^20.19.0 || ^22.12.0 || >=23"
       }
     },
     "node_modules/supports-color": {
@@ -2959,9 +3076,10 @@
       }
     },
     "node_modules/tar-fs": {
-      "version": "2.1.1",
-      "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.1.tgz",
-      "integrity": "sha512-V0r2Y9scmbDRLCNex/+hYzvp/zyYjvFbHPNgVTKfQvVrb6guiE/fxP+XblDNR011utopbkex2nM4dHNV6GDsng==",
+      "version": "2.1.3",
+      "resolved": "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.3.tgz",
+      "integrity": "sha512-090nwYJDmlhwFwEW3QQl+vaNnxsO2yVsd45eTKRBzSzu+hlb1w2K9inVq5b0ngXuLVqQ4ApvsUHHnu/zQNkWAg==",
+      "license": "MIT",
       "dependencies": {
         "chownr": "^1.1.1",
         "mkdirp-classic": "^0.5.2",
@@ -3033,10 +3151,11 @@
       }
     },
     "node_modules/timer-node": {
-      "version": "5.0.7",
-      "resolved": "https://registry.npmjs.org/timer-node/-/timer-node-5.0.7.tgz",
-      "integrity": "sha512-M1aP6ASmuVD0PSxl5fqjCAGY9WyND3DHZ8RwT5I8o7469XE53Lb5zbPai20Dhj7TProyaapfVj3TaT0P+LoSEA==",
-      "dev": true
+      "version": "5.0.9",
+      "resolved": "https://registry.npmjs.org/timer-node/-/timer-node-5.0.9.tgz",
+      "integrity": "sha512-zXxCE/5/YDi0hY9pygqgRqjRbrFRzigYxOudG0I3syaqAAmX9/w9sxex1bNFCN6c1S66RwPtEIJv65dN+1psew==",
+      "dev": true,
+      "license": "MIT"
     },
     "node_modules/to-fast-properties": {
       "version": "2.0.0",
@@ -3103,9 +3222,9 @@
       }
     },
     "node_modules/ufo": {
-      "version": "1.5.4",
-      "resolved": "https://registry.npmjs.org/ufo/-/ufo-1.5.4.tgz",
-      "integrity": "sha512-UsUk3byDzKd04EyoZ7U4DOlxQaD14JUKQl6/P7wiX4FNvUfm3XL246n9W5AmqwW5RSFJ27NAuM0iLscAOYUiGQ==",
+      "version": "1.6.1",
+      "resolved": "https://registry.npmjs.org/ufo/-/ufo-1.6.1.tgz",
+      "integrity": "sha512-9a4/uxlTWJ4+a5i0ooc1rU7C7YOw3wT+UGqdeNNHWnOF9qcMBgLRS+4IYUqbczewFx4mLEig6gawh7X6mFlEkA==",
       "license": "MIT"
     },
     "node_modules/uncrypto": {
@@ -3115,10 +3234,11 @@
       "license": "MIT"
     },
     "node_modules/undici-types": {
-      "version": "5.26.5",
-      "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-5.26.5.tgz",
-      "integrity": "sha512-JlCMO+ehdEIKqlFxk6IfVoAUVmgz7cU7zD/h9XZ0qzeosSHmUJVOzSQvvYSYWXkFXC+IfLKSIffhv0sVZup6pA==",
-      "dev": true
+      "version": "6.21.0",
+      "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
+      "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==",
+      "dev": true,
+      "license": "MIT"
     },
     "node_modules/uqr": {
       "version": "0.1.2",
@@ -3327,9 +3447,9 @@
       }
     },
     "node_modules/zod": {
-      "version": "3.24.2",
-      "resolved": "https://registry.npmjs.org/zod/-/zod-3.24.2.tgz",
-      "integrity": "sha512-lY7CDW43ECgW9u1TcT3IoXHflywfVqDYze4waEz812jR/bZ8FHDsl7pFQoSZTz5N+2NqRXs8GBwnAwo3ZNxqhQ==",
+      "version": "4.1.5",
+      "resolved": "https://registry.npmjs.org/zod/-/zod-4.1.5.tgz",
+      "integrity": "sha512-rcUUZqlLJgBC33IT3PNMgsCq6TzLQEG/Ei/KTCU0PedSWRMAXoOUN+4t/0H+Q8bdnLPdqUYnvboJT0bn/229qg==",
       "license": "MIT",
       "funding": {
         "url": "https://github.com/sponsors/colinhacks"

package.json

@@ -1,6 +1,6 @@
 {
   "name": "out-of-your-element",
-  "version": "3.0.0",
+  "version": "3.1.0",
   "description": "A bridge between Matrix and Discord",
   "main": "index.js",
   "repository": {
@@ -18,11 +18,11 @@
     "node": ">=20"
   },
   "dependencies": {
-    "@chriscdn/promise-semaphore": "^2.0.1",
-    "@cloudrac3r/discord-markdown": "^2.6.3",
+    "@chriscdn/promise-semaphore": "^3.0.1",
+    "@cloudrac3r/discord-markdown": "^2.6.6",
     "@cloudrac3r/giframe": "^0.4.3",
     "@cloudrac3r/html-template-tag": "^5.0.1",
-    "@cloudrac3r/in-your-element": "^1.0.0",
+    "@cloudrac3r/in-your-element": "^1.1.1",
     "@cloudrac3r/mixin-deep": "^3.0.1",
     "@cloudrac3r/pngjs": "^7.0.3",
     "@cloudrac3r/pug": "^4.0.4",
@@ -30,35 +30,33 @@
     "@stackoverflow/stacks": "^2.5.4",
     "@stackoverflow/stacks-icons": "^6.0.2",
     "ansi-colors": "^4.1.3",
-    "better-sqlite3": "^11.1.2",
+    "better-sqlite3": "^12.2.0",
     "chunk-text": "^2.0.1",
-    "cloudstorm": "^0.11.2",
-    "discord-api-types": "^0.37.119",
+    "cloudstorm": "^0.14.0",
+    "discord-api-types": "^0.38.19",
     "domino": "^2.1.6",
     "enquirer": "^2.4.1",
     "entities": "^5.0.0",
     "get-relative-path": "^1.0.2",
-    "get-stream": "^6.0.1",
-    "h3": "^1.12.0",
+    "h3": "^1.15.1",
     "heatsync": "^2.7.2",
     "htmx.org": "^2.0.4",
-    "lru-cache": "^10.4.3",
-    "minimist": "^1.2.8",
+    "lru-cache": "^11.0.2",
     "prettier-bytes": "^1.0.4",
     "sharp": "^0.33.4",
-    "snowtransfer": "^0.12.0",
+    "snowtransfer": "^0.15.0",
     "stream-mime-type": "^1.0.2",
     "try-to-catch": "^3.0.1",
     "uqr": "^0.1.2",
     "xxhash-wasm": "^1.0.2",
-    "zod": "^3.23.8"
+    "zod": "^4.0.17"
   },
   "devDependencies": {
     "@cloudrac3r/tap-dot": "^2.0.3",
-    "@types/node": "^18.16.0",
+    "@types/node": "^22.17.1",
     "c8": "^10.1.2",
     "cross-env": "^7.0.3",
-    "supertape": "^10.4.0"
+    "supertape": "^11.3.0"
   },
   "scripts": {
     "start": "node --enable-source-maps start.js",
@@ -66,6 +64,6 @@
     "addbot": "node addbot.js",
     "test": "cross-env FORCE_COLOR=true supertape --no-check-assertions-count --format tap --no-worker test/test.js | tap-dot",
     "test-slow": "cross-env FORCE_COLOR=true supertape --no-check-assertions-count --format tap --no-worker test/test.js -- --slow | tap-dot",
-    "cover": "c8 -o test/coverage --skip-full -x db/migrations -x src/matrix/file.js -x src/matrix/api.js -x src/matrix/mreq.js -x src/d2m/converters/rlottie-wasm.js -r html -r text supertape --no-check-assertions-count --format fail --no-worker test/test.js -- --slow"
+    "cover": "c8 -o test/coverage --skip-full -x db/migrations -x src/matrix/file.js -x src/matrix/api.js -x src/d2m/converters/rlottie-wasm.js -r html -r text supertape --no-check-assertions-count --format fail --no-worker test/test.js -- --slow"
   }
 }

readme.md

@@ -6,6 +6,8 @@ Modern Matrix-to-Discord appservice bridge, created by [@cadence:cadence.moe](ht
 
 [![Releases](https://img.shields.io/gitea/v/release/cadence/out-of-your-element?gitea_url=https%3A%2F%2Fgitdab.com&style=plastic&color=green)](https://gitdab.com/cadence/out-of-your-element/releases) [![Discuss on Matrix](https://img.shields.io/badge/discuss-%23out--of--your--element-white?style=plastic)](https://matrix.to/#/#out-of-your-element:cadence.moe)
 
+![](https://cadence.moe/i/f42a3f)
+
 ## Why a new bridge?
 
 * Modern: Supports new Discord features like replies, threads and stickers, and new Matrix features like edits, spaces and space membership.
@@ -39,4 +41,4 @@ For more information about features, [see the user guide.](https://gitdab.com/ca
 
 ## Get started!
 
-[Read the installation instructions →](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/install.md)
+[Read the installation instructions →](https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md)

scripts/backfill.js

@@ -0,0 +1,79 @@
+#!/usr/bin/env node
+// @ts-check
+
+console.log("-=- This script is experimental. It WILL mess up the room history on Matrix. -=-")
+console.log()
+
+const {channel: channelID} = require("minimist")(process.argv.slice(2), {string: ["channel"]})
+if (!channelID) {
+	console.error("Usage: ./scripts/backfill.js --channel=<channel id here>")
+	process.exit(1)
+}
+
+const assert = require("assert/strict")
+const sqlite = require("better-sqlite3")
+const backfill = new sqlite("scripts/backfill.db")
+backfill.prepare("CREATE TABLE IF NOT EXISTS backfill (channel_id TEXT NOT NULL, message_id INTEGER NOT NULL, PRIMARY KEY (channel_id, message_id))").run()
+
+const HeatSync = require("heatsync")
+
+const {reg} = require("../src/matrix/read-registration")
+const passthrough = require("../src/passthrough")
+
+const sync = new HeatSync({watchFS: false})
+const db = new sqlite("ooye.db")
+Object.assign(passthrough, {sync, db})
+
+const DiscordClient = require("../src/d2m/discord-client")
+
+const discord = new DiscordClient(reg.ooye.discord_token, "half")
+passthrough.discord = discord
+
+const orm = sync.require("../src/db/orm")
+passthrough.from = orm.from
+passthrough.select = orm.select
+
+/** @type {import("../src/d2m/event-dispatcher")}*/
+const eventDispatcher = sync.require("../src/d2m/event-dispatcher")
+
+const roomID = passthrough.select("channel_room", "room_id", {channel_id: channelID}).pluck().get()
+if (!roomID) {
+	console.error("Please choose a channel that's already bridged.")
+	process.exit(1)
+}
+
+;(async () => {
+	await discord.cloud.connect()
+	console.log("Connected, waiting for data about requested channel...")
+
+	discord.cloud.on("event", event)
+})()
+
+const preparedInsert = backfill.prepare("INSERT INTO backfill (channel_id, message_id) VALUES (?, ?)")
+
+async function event(event) {
+	if (event.t !== "GUILD_CREATE") return
+	const channel = event.d.channels.find(c => c.id === channelID)
+	if (!channel) return
+	const guild_id = event.d.id
+
+	let last = backfill.prepare("SELECT cast(max(message_id) as TEXT) FROM backfill WHERE channel_id = ?").pluck().get(channelID) || "0"
+	console.log(`OK, processing messages for #${channel.name}, continuing from ${last}`)
+
+	while (last) {
+		const messages = await discord.snow.channel.getChannelMessages(channelID, {limit: 50, after: String(last)})
+		messages.reverse() // More recent messages come first -> More recent messages come last
+		for (const message of messages) {
+			const simulatedGatewayDispatchData = {
+				guild_id,
+				backfill: true,
+				...message
+			}
+			await eventDispatcher.onMessageCreate(discord, simulatedGatewayDispatchData)
+			preparedInsert.run(channelID, message.id)
+		}
+		last = messages.at(-1)?.id
+	}
+
+	process.exit()
+}

scripts/migrate-from-old-bridge.js

@@ -2,8 +2,7 @@
 // @ts-check
 
 const assert = require("assert").strict
-/** @type {any} */ // @ts-ignore bad types from semaphore
-const Semaphore = require("@chriscdn/promise-semaphore")
+const {Semaphore} = require("@chriscdn/promise-semaphore")
 const sqlite = require("better-sqlite3")
 const HeatSync = require("heatsync")
 

scripts/setup.js

@@ -17,8 +17,6 @@ const {SnowTransfer} = require("snowtransfer")
 const DiscordTypes = require("discord-api-types/v10")
 const {createApp, defineEventHandler, toNodeListener} = require("h3")
 
-const args = require("minimist")(process.argv.slice(2), {string: ["emoji-guild"]})
-
 // Move database file if it's still in the old location
 if (fs.existsSync("db")) {
 	if (fs.existsSync("db/ooye.db")) {
@@ -50,24 +48,13 @@ passthrough.select = orm.select
 let registration = require("../src/matrix/read-registration")
 let {reg, getTemplateRegistration, writeRegistration, readRegistration, checkRegistration, registrationFilePath} = registration
 
+const {setupEmojis} = require("../src/m2d/actions/setup-emojis")
+
 function die(message) {
 	console.error(message)
 	process.exit(1)
 }
 
-async function uploadAutoEmoji(snow, guild, name, filename) {
-	let emoji = guild.emojis.find(e => e.name === name)
-	if (!emoji) {
-		console.log(`   Uploading ${name}...`)
-		const data = fs.readFileSync(filename, null)
-		emoji = await snow.guildAssets.createEmoji(guild.id, {name, image: "data:image/png;base64," + data.toString("base64")})
-	} else {
-		console.log(`   Reusing ${name}...`)
-	}
-	db.prepare("REPLACE INTO auto_emoji (name, emoji_id, guild_id) VALUES (?, ?, ?)").run(emoji.name, emoji.id, guild.id)
-	return emoji
-}
-
 async function suggestWellKnown(serverUrlPrompt, url, otherwise) {
 	try {
 		var json = await fetch(`${url}/.well-known/matrix/client`).then(res => res.json())
@@ -141,6 +128,7 @@ function defineEchoHandler() {
 		console.log("OOYE has its own web server. It needs to be accessible on the public internet.")
 		console.log("You need to enter a public URL where you will be able to host this web server.")
 		console.log("OOYE listens on localhost:6693, so you will probably have to set up a reverse proxy.")
+		console.log("Examples: https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md#appendix")
 		console.log("Now listening on port 6693. Feel free to send some test requests.")
 		/** @type {{bridge_origin: string}} */
 		const bridgeOriginResponse = await prompt({
@@ -169,7 +157,7 @@ function defineEchoHandler() {
 		console.log("Go to https://discord.com/developers, create or pick an app, go to the Bot section, and reset the token.")
 		/** @type {SnowTransfer} */ // @ts-ignore
 		let snow = null
-		/** @type {{id: string, flags: number, redirect_uris: string[]}} */ // @ts-ignore
+		/** @type {{id: string, flags: number, redirect_uris: string[], description: string}} */ // @ts-ignore
 		let client = null
 		/** @type {{discord_token: string}} */
 		const discordTokenResponse = await prompt({
@@ -207,6 +195,34 @@ function defineEchoHandler() {
 			})
 		}
 
+		console.log("Would you like to require a password to add your bot to servers? This will discourage others from using your bridge.")
+		console.log("Important: To make it truly private, you MUST ALSO disable Public Bot in the Discord bot configuration page.")
+		/** @type {{web_password: string}} */
+		const passwordResponse = await prompt({
+			type: "text",
+			name: "web_password",
+			message: "Choose a simple password (optional)"
+		})
+
+		console.log("To fulfill license obligations, I recommend mentioning Out Of Your Element in your Discord bot's profile.")
+		console.log("On the Discord bot configuration page, go to General and add something like this to the description:")
+		console.log(cyan("Powered by **Out Of Your Element**"))
+		console.log(cyan("https://gitdab.com/cadence/out-of-your-element"))
+		await prompt({
+			type: "invisible",
+			name: "description",
+			message: "Press Enter to acknowledge",
+			validate: async token => {
+				process.stdout.write(magenta("checking, please wait..."))
+				client = await snow.requestHandler.request(`/applications/@me`, {}, "get", "json")
+				if (client.description?.match(/out.of.your.element/i)) {
+					return true
+				} else {
+					return "Description must name or link Out Of Your Element"
+				}
+			}
+		})
+
 		console.log("What is your Discord client secret?")
 		console.log(`You can find it in the application's OAuth2 section: https://discord.com/developers/applications/${client.id}/oauth2`)
 		/** @type {{discord_client_secret: string}} */
@@ -244,7 +260,8 @@ function defineEchoHandler() {
 				...bridgeOriginResponse,
 				server_origin: serverOrigin,
 				...discordTokenResponse,
-				...clientSecretResponse
+				...clientSecretResponse,
+				...passwordResponse
 			}
 		}
 		registration.reg = reg
@@ -331,47 +348,13 @@ function defineEchoHandler() {
 
 	console.log("✅ Matrix appservice login works...")
 
-	// upload the L1 L2 emojis to some guild
-	const emojis = db.prepare("SELECT name FROM auto_emoji WHERE name = 'L1' OR name = 'L2'").pluck().all()
-	if (emojis.length !== 2) {
-		// If an argument was supplied, always use that one
-		let guild = null
-		if (args["emoji-guild"]) {
-			if (typeof args["emoji-guild"] === "string") {
-				guild = await discord.snow.guild.getGuild(args["emoji-guild"])
-			}
-			if (!guild) return die(`Error: You asked emojis to be uploaded to guild ID ${args["emoji-guild"]}, but the bot isn't in that guild.`)
-		}
-		// Otherwise, check if we have already registered an auto emoji guild
-		if (!guild) {
-			const guildID = passthrough.select("auto_emoji", "guild_id", {name: "_"}).pluck().get()
-			if (guildID) {
-				guild = await discord.snow.guild.getGuild(guildID, false)
-			}
-		}
-		// Otherwise, check if we should create a new guild
-		if (!guild) {
-			const guilds = await discord.snow.user.getGuilds({limit: 11, with_counts: false})
-			if (guilds.length < 10) {
-				console.log("   Creating a guild for emojis...")
-				guild = await discord.snow.guild.createGuild({name: "OOYE Emojis"})
-			}
-		}
-		// Otherwise, it's the user's problem
-		if (!guild) {
-			return die(`Error: The bot needs to upload some emojis. Please say where to upload them to. Run setup again with --emoji-guild=GUILD_ID`)
-		}
-		// Upload those emojis to the chosen location
-		db.prepare("REPLACE INTO auto_emoji (name, emoji_id, guild_id) VALUES ('_', '_', ?)").run(guild.id)
-		await uploadAutoEmoji(discord.snow, guild, "L1", join(__dirname, "../docs/img/L1.png"))
-		await uploadAutoEmoji(discord.snow, guild, "L2", join(__dirname, "../docs/img/L2.png"))
-	}
+	// upload the L1 L2 emojis to user emojis
+	await setupEmojis()
 	console.log("✅ Emojis are ready...")
 
 	// set profile data on discord...
 	const avatarImageBuffer = await fetch("https://cadence.moe/friends/out_of_your_element.png").then(res => res.arrayBuffer())
 	await discord.snow.user.updateSelf({avatar: "data:image/png;base64," + Buffer.from(avatarImageBuffer).toString("base64")})
-	await discord.snow.requestHandler.request(`/applications/@me`, {}, "patch", "json", {description: "Powered by **Out Of Your Element**\nhttps://gitdab.com/cadence/out-of-your-element"})
 	console.log("✅ Discord profile updated...")
 
 	// set profile data on homeserver...

scripts/start-server.js

@@ -21,12 +21,7 @@ const DiscordClient = require("../src/d2m/discord-client")
 const discord = new DiscordClient(reg.ooye.discord_token, "half")
 passthrough.discord = discord
 
-const app = createApp()
-const router = createRouter()
-app.use(router)
-const server = createServer(toNodeListener(app))
-server.listen(reg.socket || new URL(reg.url).port)
-const as = Object.assign(new EventEmitter(), {app, router, server}) // @ts-ignore
+const {as} = require("../src/matrix/appservice")
 passthrough.as = as
 
 const orm = sync.require("../src/db/orm")

src/d2m/actions/add-reaction.js

@@ -25,7 +25,7 @@ async function addReaction(data) {
 	if (!parentID) return // Nothing can be done if the parent message was never bridged.
 	assert.equal(typeof parentID, "string")
 
-	const key = await emojiToKey.emojiToKey(data.emoji)
+	const key = await emojiToKey.emojiToKey(data.emoji, data.message_id)
 	const shortcode = key.startsWith("mxc://") ? `:${data.emoji.name}:` : undefined
 
 	const roomID = await createRoom.ensureRoom(data.channel_id)

src/d2m/actions/create-room.js

@@ -40,6 +40,8 @@ const PRIVACY_ENUMS = {
 
 const DEFAULT_PRIVACY_LEVEL = 0
 
+const READ_ONLY_ROOM_EVENTS_DEFAULT_POWER = 50
+
 /** @type {Map<string, Promise<string>>} channel ID -> Promise<room ID> */
 const inflightRoomCreate = new Map()
 
@@ -54,6 +56,7 @@ function convertNameAndTopic(channel, guild, customName) {
 	let channelPrefix =
 		( parentChannel?.type === DiscordTypes.ChannelType.GuildForum ? ""
 		: channel.type === DiscordTypes.ChannelType.PublicThread ? "[⛓️] "
+		: channel.type === DiscordTypes.ChannelType.AnnouncementThread ? "[⛓️] "
 		: channel.type === DiscordTypes.ChannelType.PrivateThread ? "[🔒⛓️] "
 		: channel.type === DiscordTypes.ChannelType.GuildVoice ? "[🔊] "
 		: "")
@@ -145,9 +148,10 @@ async function channelToKState(channel, guild, di) {
 		"m.room.join_rules/": join_rules,
 		/** @type {Ty.Event.M_Power_Levels} */
 		"m.room.power_levels/": {
-			events_default: everyoneCanSend ? 0 : 50,
+			events_default: everyoneCanSend ? 0 : READ_ONLY_ROOM_EVENTS_DEFAULT_POWER,
 			events: {
-				"m.reaction": 0
+				"m.reaction": 0,
+				"m.room.redaction": 0 // only affects redactions of own events, required to be able to un-react
 			},
 			notifications: {
 				room: everyoneCanMentionEveryone ? 0 : 20
@@ -175,8 +179,16 @@ async function channelToKState(channel, guild, di) {
 		}
 	}
 
+	// Don't overwrite room topic if the topic has been customised
 	if (hasCustomTopic) delete channelKState["m.room.topic/"]
 
+	// Don't add a space parent if it's self service
+	// (The person setting up self-service has already put it in their preferred space to be able to get this far.)
+	const autocreate = select("guild_active", "autocreate", {guild_id: guild.id}).pluck().get()
+	if (autocreate === 0 && ![DiscordTypes.ChannelType.PrivateThread, DiscordTypes.ChannelType.PublicThread, DiscordTypes.ChannelType.AnnouncementThread].includes(channel.type)) {
+		delete channelKState[`m.space.parent/${parentSpaceID}`]
+	}
+
 	return {spaceID: parentSpaceID, privacyLevel, channelKState}
 }
 
@@ -221,8 +233,8 @@ async function createRoom(channel, guild, spaceID, kstate, privacyLevel) {
 		return roomID
 	})
 
-	// Put the newly created child into the space, no need to await this
-	_syncSpaceMember(channel, spaceID, roomID)
+	// Put the newly created child into the space
+	await _syncSpaceMember(channel, spaceID, roomID, guild.id)
 
 	return roomID
 }
@@ -391,7 +403,7 @@ async function _syncRoom(channelID, shouldActuallySync) {
 	db.prepare("UPDATE channel_room SET name = ? WHERE room_id = ?").run(channel.name, roomID)
 
 	// sync room as space member
-	const spaceApply = _syncSpaceMember(channel, spaceID, roomID)
+	const spaceApply = _syncSpaceMember(channel, spaceID, roomID, guild.id)
 	await Promise.all([roomApply, spaceApply])
 
 	return roomID
@@ -503,14 +515,25 @@ async function unbridgeDeletedChannel(channel, guildID) {
  * @param {DiscordTypes.APIGuildTextChannel} channel
  * @param {string} spaceID
  * @param {string} roomID
+ * @param {string} guild_id
  * @returns {Promise<string[]>}
  */
-async function _syncSpaceMember(channel, spaceID, roomID) {
+async function _syncSpaceMember(channel, spaceID, roomID, guild_id) {
+	// If space is self-service then only permit changes to space parenting for threads
+	// (The person setting up self-service has already put it in their preferred space to be able to get this far.)
+	const autocreate = select("guild_active", "autocreate", {guild_id}).pluck().get()
+	if (autocreate === 0 && ![DiscordTypes.ChannelType.PrivateThread, DiscordTypes.ChannelType.PublicThread, DiscordTypes.ChannelType.AnnouncementThread].includes(channel.type)) {
+		return []
+	}
+
 	const spaceKState = await ks.roomToKState(spaceID)
 	let spaceEventContent = {}
 	if (
 		channel.type !== DiscordTypes.ChannelType.PrivateThread // private threads do not belong in the space (don't offer people something they can't join)
-		&& !channel["thread_metadata"]?.archived // archived threads do not belong in the space (don't offer people conversations that are no longer relevant)
+		&& (
+			!channel["thread_metadata"]?.archived // archived threads do not belong in the space (don't offer people conversations that are no longer relevant)
+			|| discord.channels.get(channel.parent_id || "")?.type === DiscordTypes.ChannelType.GuildForum
+		)
 	) {
 		spaceEventContent = {
 			via: [reg.ooye.server_name]
@@ -536,6 +559,7 @@ async function createAllForGuild(guildID) {
 }
 
 module.exports.DEFAULT_PRIVACY_LEVEL = DEFAULT_PRIVACY_LEVEL
+module.exports.READ_ONLY_ROOM_EVENTS_DEFAULT_POWER = READ_ONLY_ROOM_EVENTS_DEFAULT_POWER
 module.exports.PRIVACY_ENUMS = PRIVACY_ENUMS
 module.exports.createRoom = createRoom
 module.exports.ensureRoom = ensureRoom

src/d2m/actions/create-room.test.js

@@ -154,7 +154,8 @@ test("channel2room: read-only discord channel", async t => {
 		"m.room.power_levels/": {
 			events_default: 50, // <-- it should be read-only!
 			events: {
-				"m.reaction": 0
+				"m.reaction": 0,
+				"m.room.redaction": 0
 			},
 			notifications: {
 				room: 20,

src/d2m/actions/create-space.js

@@ -129,16 +129,10 @@ async function _syncSpace(guild, shouldActuallySync) {
 		// don't try to update rooms with custom avatars though
 		const roomsWithCustomAvatars = select("channel_room", "room_id", {}, "WHERE custom_avatar IS NOT NULL").pluck().all()
 
-		const state = await ks.kstateToState(spaceKState)
-		const childRooms = state.filter(({type, state_key, content}) => {
-			return type === "m.space.child" && "via" in content && !roomsWithCustomAvatars.includes(state_key)
-		}).map(({state_key}) => state_key)
-
-		for (const roomID of childRooms) {
-			const avatarEventContent = await api.getStateEvent(roomID, "m.room.avatar", "")
-			if (avatarEventContent.url !== newAvatarState.url) {
-				await api.sendState(roomID, "m.room.avatar", "", newAvatarState)
-			}
+		for await (const room of api.generateFullHierarchy(spaceID)) {
+			if (room.avatar_url === newAvatarState.url) continue
+			if (roomsWithCustomAvatars.includes(room.room_id)) continue
+			await api.sendState(room.room_id, "m.room.avatar", "", newAvatarState)
 		}
 	}
 

src/d2m/actions/edit-message.js

@@ -22,9 +22,7 @@ async function editMessage(message, guild, row) {
 	if (row && row.speedbump_webhook_id === message.webhook_id) {
 		// Handle the PluralKit public instance
 		if (row.speedbump_id === "466378653216014359") {
-			const root = await registerPkUser.fetchMessage(message.id)
-			assert(root.member)
-			senderMxid = await registerPkUser.ensureSimJoined(root, roomID)
+			senderMxid = await registerPkUser.syncUser(message.id, message.author, roomID, false)
 		}
 	}
 

src/d2m/actions/register-pk-user.js

@@ -5,7 +5,7 @@ const {reg} = require("../../matrix/read-registration")
 const Ty = require("../../types")
 
 const passthrough = require("../../passthrough")
-const {sync, db, select} = passthrough
+const {sync, db, select, from} = passthrough
 /** @type {import("../../matrix/api")} */
 const api = sync.require("../../matrix/api")
 /** @type {import("../../matrix/file")} */
@@ -20,6 +20,20 @@ const registerUser = sync.require("./register-user")
  * @prop {string} id
  */
 
+/** @returns {Promise<Ty.PkMessage>} */
+async function fetchMessage(messageID) {
+	try {
+		var res = await fetch(`https://api.pluralkit.me/v2/messages/${messageID}`)
+	} catch (networkError) {
+		// Network issue, raise a more readable message
+		throw new Error(`Failed to connect to PK API: ${networkError.toString()}`)
+	}
+	if (!res.ok) throw new Error(`PK API returned an error: ${await res.text()}`)
+	const root = await res.json()
+	if (!root.member) throw new Error(`PK API didn't return member data: ${JSON.stringify(root)}`)
+	return root
+}
+
 /**
  * A sim is an account that is being simulated by the bridge to copy events from the other side.
  * @param {Ty.PkMessage} pkMessage
@@ -95,6 +109,7 @@ async function ensureSimJoined(pkMessage, roomID) {
 }
 
 /**
+ * Generate profile data based on webhook displayname and configured avatar.
  * @param {Ty.PkMessage} pkMessage
  * @param {WebhookAuthor} author
  */
@@ -115,54 +130,47 @@ async function memberToStateContent(pkMessage, author) {
 
 /**
  * Sync profile data for a sim user. This function follows the following process:
- * 1. Join the sim to the room if needed
- * 2. Make an object of what the new room member state content would be, including uploading the profile picture if it hasn't been done before
- * 3. Compare against the previously known state content, which is helpfully stored in the database
- * 4. If the state content has changed, send it to Matrix and update it in the database for next time
- * @param {WebhookAuthor} author
- * @param {Ty.PkMessage} pkMessage
- * @param {string} roomID
+ * 1. Look up data about proxy user from API
+ * 2. If this fails, try to use previously cached data (won't sync)
+ * 3. Create and join the sim to the room if needed
+ * 4. Make an object of what the new room member state content would be, including uploading the profile picture if it hasn't been done before
+ * 5. Compare against the previously known state content, which is helpfully stored in the database
+ * 6. If the state content has changed, send it to Matrix and update it in the database for next time
+ * @param {string} messageID to call API with
+ * @param {WebhookAuthor} author for profile data
+ * @param {string} roomID room to join member to
+ * @param {boolean} shouldActuallySync whether to actually sync updated user data or just ensure it's joined
  * @returns {Promise<string>} mxid of the updated sim
  */
-async function syncUser(author, pkMessage, roomID) {
+async function syncUser(messageID, author, roomID, shouldActuallySync) {
+	try {
+		// API lookup
+		var pkMessage = await fetchMessage(messageID)
+		db.prepare("REPLACE INTO sim_proxy (user_id, proxy_owner_id, displayname) VALUES (?, ?, ?)").run(pkMessage.member.uuid, pkMessage.sender, author.username)
+	} catch (e) {
+		// Fall back to offline cache
+		const senderMxid = from("sim_proxy").join("sim", "user_id").join("sim_member", "mxid").where({displayname: author.username, room_id: roomID}).pluck("mxid").get()
+		if (!senderMxid) throw e
+		return senderMxid
+	}
+
+	// Create and join the sim to the room if needed
 	const mxid = await ensureSimJoined(pkMessage, roomID)
-	// Update the sim_proxy table, so mentions can look up the original sender later
-	db.prepare("INSERT OR IGNORE INTO sim_proxy (user_id, proxy_owner_id, displayname) VALUES (?, ?, ?)").run(pkMessage.member.uuid, pkMessage.sender, author.username)
-	// Sync the member state
+
+	if (shouldActuallySync) {
+		// Build current profile data
 		const content = await memberToStateContent(pkMessage, author)
 		const currentHash = registerUser._hashProfileContent(content, 0)
 		const existingHash = select("sim_member", "hashed_profile_content", {room_id: roomID, mxid}).safeIntegers().pluck().get()
-	// only do the actual sync if the hash has changed since we last looked
+
+		// Only do the actual sync if the hash has changed since we last looked
 		if (existingHash !== currentHash) {
 			await api.sendState(roomID, "m.room.member", mxid, content, mxid)
 			db.prepare("UPDATE sim_member SET hashed_profile_content = ? WHERE room_id = ? AND mxid = ?").run(currentHash, roomID, mxid)
 		}
+	}
+
 	return mxid
 }
 
-/** @returns {Promise<Ty.PkMessage>} */
-async function fetchMessage(messageID) {
-	// Their backend is weird. Sometimes it says "message not found" (code 20006) on the first try, so we make multiple attempts.
-	let attempts = 0
-	do {
-		try {
-			var res = await fetch(`https://api.pluralkit.me/v2/messages/${messageID}`)
-			if (res.ok) return res.json()
-			var errorGetter = res.json
-		} catch (e) {
-			// Catch any network issues too.
-			errorGetter = e.toString
-		}
-
-		// I think the backend needs some time to update.
-		await new Promise(resolve => setTimeout(resolve, 1500))
-	} while (++attempts < 3)
-
-	throw new Error(`PK API returned an error after ${attempts} tries: ${JSON.stringify(await errorGetter())}`)
-}
-
-module.exports._memberToStateContent = memberToStateContent
-module.exports.ensureSim = ensureSim
-module.exports.ensureSimJoined = ensureSimJoined
 module.exports.syncUser = syncUser
-module.exports.fetchMessage = fetchMessage

src/d2m/actions/register-user.js

@@ -3,6 +3,7 @@
 const assert = require("assert").strict
 const {reg} = require("../../matrix/read-registration")
 const DiscordTypes = require("discord-api-types/v10")
+const Ty = require("../../types")
 const mixin = require("@cloudrac3r/mixin-deep")
 
 const passthrough = require("../../passthrough")
@@ -15,6 +16,8 @@ const file = sync.require("../../matrix/file")
 const utils = sync.require("../../discord/utils")
 /** @type {import("../converters/user-to-mxid")} */
 const userToMxid = sync.require("../converters/user-to-mxid")
+/** @type {import("./create-room")} */
+const createRoom = sync.require("./create-room")
 /** @type {import("xxhash-wasm").XXHashAPI} */ // @ts-ignore
 let hasher = null
 // @ts-ignore
@@ -97,12 +100,12 @@ async function ensureSimJoined(user, roomID) {
 
 /**
  * @param {DiscordTypes.APIUser} user
- * @param {Omit<DiscordTypes.APIGuildMember, "user">} member
+ * @param {Omit<DiscordTypes.APIGuildMember, "user"> | undefined} member
  */
 async function memberToStateContent(user, member, guildID) {
 	let displayname = user.username
 	if (user.global_name) displayname = user.global_name
-	if (member.nick) displayname = member.nick
+	if (member?.nick) displayname = member.nick
 
 	const content = {
 		displayname,
@@ -117,7 +120,7 @@ async function memberToStateContent(user, member, guildID) {
 		}
 	}
 
-	if (member.avatar || user.avatar) {
+	if (member?.avatar || user.avatar) {
 		// const avatarPath = file.userAvatar(user) // the user avatar only
 		const avatarPath = file.memberAvatar(guildID, user, member) // the member avatar or the user avatar
 		content["moe.cadence.ooye.member"].avatar = avatarPath
@@ -130,13 +133,16 @@ async function memberToStateContent(user, member, guildID) {
 /**
  * https://gitdab.com/cadence/out-of-your-element/issues/9
  * @param {DiscordTypes.APIUser} user
- * @param {Omit<DiscordTypes.APIGuildMember, "user">} member
+ * @param {Omit<DiscordTypes.APIGuildMember, "user"> | undefined} member
  * @param {DiscordTypes.APIGuild} guild
  * @param {DiscordTypes.APIGuildChannel} channel
  * @returns {number} 0 to 100
  */
 function memberToPowerLevel(user, member, guild, channel) {
+	if (!member) return 0
+
 	const permissions = utils.getPermissions(member.roles, guild.roles, user.id, channel.permission_overwrites)
+	const everyonePermissions = utils.getPermissions([], guild.roles, undefined, channel.permission_overwrites)
 	/*
 	 * PL 100 = Administrator = People who can brick the room. RATIONALE:
 	 * 	- Administrator.
@@ -156,8 +162,14 @@ function memberToPowerLevel(user, member, guild, channel) {
 	 * 	- Moderate Members.
 	 */
 	if (utils.hasSomePermissions(permissions, ["ManageMessages", "ManageNicknames", "ManageThreads", "KickMembers", "BanMembers", "MuteMembers", "DeafenMembers", "ModerateMembers"])) return 50
+	/* PL 50 = if room is read-only but the user has been specially allowed to send messages */
+	const everyoneCanSend = utils.hasPermission(everyonePermissions, DiscordTypes.PermissionFlagsBits.SendMessages)
+	const userCanSend = utils.hasPermission(permissions, DiscordTypes.PermissionFlagsBits.SendMessages)
+	if (!everyoneCanSend && userCanSend) return createRoom.READ_ONLY_ROOM_EVENTS_DEFAULT_POWER
 	/* PL 20 = Mention Everyone for technical reasons. */
-	if (utils.hasSomePermissions(permissions, ["MentionEveryone"])) return 20
+	const everyoneCanMentionEveryone = utils.hasPermission(everyonePermissions, DiscordTypes.PermissionFlagsBits.MentionEveryone)
+	const userCanMentionEveryone = utils.hasPermission(permissions, DiscordTypes.PermissionFlagsBits.MentionEveryone)
+	if (!everyoneCanMentionEveryone && userCanMentionEveryone) return 20
 	return 0
 }
 
@@ -179,7 +191,7 @@ function _hashProfileContent(content, powerLevel) {
  * 4. Compare against the previously known state content, which is helpfully stored in the database
  * 5. If the state content or power level have changed, send them to Matrix and update them in the database for next time
  * @param {DiscordTypes.APIUser} user
- * @param {Omit<DiscordTypes.APIGuildMember, "user">} member
+ * @param {Omit<DiscordTypes.APIGuildMember, "user"> | undefined} member
  * @param {DiscordTypes.APIGuildChannel} channel
  * @param {DiscordTypes.APIGuild} guild
  * @param {string} roomID
@@ -192,16 +204,14 @@ async function syncUser(user, member, channel, guild, roomID) {
 	const currentHash = _hashProfileContent(content, powerLevel)
 	const existingHash = select("sim_member", "hashed_profile_content", {room_id: roomID, mxid}).safeIntegers().pluck().get()
 	// only do the actual sync if the hash has changed since we last looked
-	if (existingHash !== currentHash) {
+	const hashHasChanged = existingHash !== currentHash
+	// however, do not overwrite pre-existing data if we already have data and `member` is not accessible, because this would replace good data with bad data
+	const wouldOverwritePreExisting = existingHash && !member
+	if (hashHasChanged && !wouldOverwritePreExisting) {
 		// Update room member state
 		await api.sendState(roomID, "m.room.member", mxid, content, mxid)
 		// Update power levels
-		const powerLevelsStateContent = await api.getStateEvent(roomID, "m.room.power_levels", "")
-		const oldPowerLevel = powerLevelsStateContent.users?.[mxid] || 0
-		mixin(powerLevelsStateContent, {users: {[mxid]: powerLevel}})
-		if (powerLevel === 0) delete powerLevelsStateContent.users[mxid] // keep the event compact
-		const sendPowerLevelAs = powerLevel < oldPowerLevel ? mxid : undefined // bridge bot won't not have permission to demote equal power users, so do this action as themselves
-		await api.sendState(roomID, "m.room.power_levels", "", powerLevelsStateContent, sendPowerLevelAs)
+		await api.setUserPower(roomID, mxid, powerLevel)
 		// Update cached hash
 		db.prepare("UPDATE sim_member SET hashed_profile_content = ? WHERE room_id = ? AND mxid = ?").run(currentHash, roomID, mxid)
 	}
@@ -245,3 +255,4 @@ module.exports.ensureSim = ensureSim
 module.exports.ensureSimJoined = ensureSimJoined
 module.exports.syncUser = syncUser
 module.exports.syncAllUsersInRoom = syncAllUsersInRoom
+module.exports._memberToPowerLevel = memberToPowerLevel

src/d2m/actions/register-user.test.js

@@ -1,10 +1,12 @@
-const {_memberToStateContent} = require("./register-user")
+const {_memberToStateContent, _memberToPowerLevel} = require("./register-user")
 const {test} = require("supertape")
-const testData = require("../../../test/data")
+const data = require("../../../test/data")
+const mixin = require("@cloudrac3r/mixin-deep")
+const DiscordTypes = require("discord-api-types/v10")
 
 test("member2state: without member nick or avatar", async t => {
 	t.deepEqual(
-		await _memberToStateContent(testData.member.kumaccino.user, testData.member.kumaccino, testData.guild.general.id),
+		await _memberToStateContent(data.member.kumaccino.user, data.member.kumaccino, data.guild.general.id),
 		{
 			avatar_url: "mxc://cadence.moe/UpAeIqeclhKfeiZNdIWNcXXL",
 			displayname: "kumaccino",
@@ -24,7 +26,7 @@ test("member2state: without member nick or avatar", async t => {
 
 test("member2state: with global name, without member nick or avatar", async t => {
 	t.deepEqual(
-		await _memberToStateContent(testData.member.papiophidian.user, testData.member.papiophidian, testData.guild.general.id),
+		await _memberToStateContent(data.member.papiophidian.user, data.member.papiophidian, data.guild.general.id),
 		{
 			avatar_url: "mxc://cadence.moe/JPzSmALLirnIprlSMKohSSoX",
 			displayname: "PapiOphidian",
@@ -44,7 +46,7 @@ test("member2state: with global name, without member nick or avatar", async t =>
 
 test("member2state: with member nick and avatar", async t => {
 	t.deepEqual(
-		await _memberToStateContent(testData.member.sheep.user, testData.member.sheep, testData.guild.general.id),
+		await _memberToStateContent(data.member.sheep.user, data.member.sheep, data.guild.general.id),
 		{
 			avatar_url: "mxc://cadence.moe/rfemHmAtcprjLEiPiEuzPhpl",
 			displayname: "The Expert's Submarine",
@@ -61,3 +63,64 @@ test("member2state: with member nick and avatar", async t => {
 		}
 	)
 })
+
+test("member2power: default to zero if member roles unknown", async t => {
+	const power = _memberToPowerLevel(data.user.clyde_ai, null, data.guild.data_horde, data.channel.saving_the_world)
+	t.equal(power, 0)
+})
+
+test("member2power: unremarkable = 0", async t => {
+	const power = _memberToPowerLevel(data.user.clyde_ai, {
+		roles: []
+	}, data.guild.data_horde, data.channel.general)
+	t.equal(power, 0)
+})
+
+test("member2power: can mention everyone = 20", async t => {
+	const power = _memberToPowerLevel(data.user.clyde_ai, {
+		roles: ["684524730274807911"]
+	}, data.guild.data_horde, data.channel.general)
+	t.equal(power, 20)
+})
+
+test("member2power: can send messages in protected channel due to role = 50", async t => {
+	const power = _memberToPowerLevel(data.user.clyde_ai, {
+		roles: ["684524730274807911"]
+	}, data.guild.data_horde, data.channel.saving_the_world)
+	t.equal(power, 50)
+})
+
+test("member2power: can send messages in protected channel due to user override = 50", async t => {
+	const power = _memberToPowerLevel(data.user.clyde_ai, {
+		roles: []
+	}, data.guild.data_horde, mixin({}, data.channel.saving_the_world, {
+		permission_overwrites: data.channel.saving_the_world.permission_overwrites.concat({
+			type: DiscordTypes.OverwriteType.member,
+			id: data.user.clyde_ai.id,
+			allow: String(DiscordTypes.PermissionFlagsBits.SendMessages),
+			deny: "0"
+		})
+	}))
+	t.equal(power, 50)
+})
+
+test("member2power: can kick users = 50", async t => {
+	const power = _memberToPowerLevel(data.user.clyde_ai, {
+		roles: ["682789592390281245"]
+	}, data.guild.data_horde, data.channel.general)
+	t.equal(power, 50)
+})
+
+test("member2power: can manage channels = 100", async t => {
+	const power = _memberToPowerLevel(data.user.clyde_ai, {
+		roles: ["665290147377578005"]
+	}, data.guild.data_horde, data.channel.saving_the_world)
+	t.equal(power, 100)
+})
+
+test("member2power: pathfinder use case", async t => {
+	const power = _memberToPowerLevel(data.user.jerassicore, {
+		roles: ["1235396773510647810", "1359752622130593802", "1249165855632265267", "1380768596929806356", "1380756348190462015"]
+	}, data.guild.pathfinder, data.channel.character_art)
+	t.equal(power, 50)
+})

src/d2m/actions/remove-reaction.js

@@ -43,7 +43,7 @@ async function removeSomeReactions(data) {
  * @param {Ty.Event.Outer<Ty.Event.M_Reaction>[]} reactions
  */
 async function removeReaction(data, reactions) {
-	const key = await emojiToKey.emojiToKey(data.emoji)
+	const key = await emojiToKey.emojiToKey(data.emoji, data.message_id)
 	return converter.removeReaction(data, reactions, key)
 }
 
@@ -52,7 +52,7 @@ async function removeReaction(data, reactions) {
  * @param {Ty.Event.Outer<Ty.Event.M_Reaction>[]} reactions
  */
 async function removeEmojiReaction(data, reactions) {
-	const key = await emojiToKey.emojiToKey(data.emoji)
+	const key = await emojiToKey.emojiToKey(data.emoji, data.message_id)
 	const discordPreferredEncoding = await emoji.encodeEmoji(key, undefined)
 	db.prepare("DELETE FROM reaction WHERE message_id = ? AND encoded_emoji = ?").run(data.message_id, discordPreferredEncoding)
 

src/d2m/actions/send-message.js

@@ -31,20 +31,17 @@ async function sendMessage(message, channel, guild, row) {
 	if (!dUtils.isWebhookMessage(message)) {
 		if (message.author.id === discord.application.id) {
 			// no need to sync the bot's own user
-		} else if (message.member) { // available on a gateway message create event
+		} else {
 			senderMxid = await registerUser.syncUser(message.author, message.member, channel, guild, roomID)
-		} else { // well, good enough...
-			senderMxid = await registerUser.ensureSimJoined(message.author, roomID)
 		}
 	} else if (row && row.speedbump_webhook_id === message.webhook_id) {
 		// Handle the PluralKit public instance
 		if (row.speedbump_id === "466378653216014359") {
-			const pkMessage = await registerPkUser.fetchMessage(message.id)
-			senderMxid = await registerPkUser.syncUser(message.author, pkMessage, roomID)
+			senderMxid = await registerPkUser.syncUser(message.id, message.author, roomID, true)
 		}
 	}
 
-	const events = await messageToEvent.messageToEvent(message, guild, {}, {api})
+	const events = await messageToEvent.messageToEvent(message, guild, {}, {api, snow: discord.snow})
 	const eventIDs = []
 	if (events.length) {
 		db.prepare("INSERT OR IGNORE INTO message_channel (message_id, channel_id) VALUES (?, ?)").run(message.id, message.channel_id)

src/d2m/actions/update-pins.js

@@ -25,10 +25,18 @@ function convertTimestamp(timestamp) {
  * @param {number?} convertedTimestamp
  */
 async function updatePins(channelID, roomID, convertedTimestamp) {
-	const discordPins = await discord.snow.channel.getChannelPinnedMessages(channelID)
-	const pinned = pinsToList.pinsToList(discordPins)
+	try {
+		var discordPins = await discord.snow.channel.getChannelPinnedMessages(channelID)
+	} catch (e) {
+		if (e.message === `{"message": "Missing Access", "code": 50001}`) {
+			return // Discord sends channel pins update events even for channels that the bot can't view/get pins in, just ignore it
+		}
+		throw e
+	}
 
 	const kstate = await ks.roomToKState(roomID)
+	const pinned = pinsToList.pinsToList(discordPins, kstate)
+
 	const diff = ks.diffKState(kstate, {"m.room.pinned_events/": {pinned}})
 	await ks.applyKStateDiffToRoom(roomID, diff)
 

src/d2m/converters/edit-to-changes.js

@@ -22,6 +22,10 @@ function eventCanBeEdited(ev) {
 	return true
 }
 
+function eventIsText(ev) {
+	return ev.old.event_type === "m.room.message" && (ev.old.event_subtype === "m.text" || ev.old.event_subtype === "m.notice")
+}
+
 /**
  * @param {import("discord-api-types/v10").GatewayMessageCreateDispatchData} message
  * @param {import("discord-api-types/v10").APIGuild} guild
@@ -121,6 +125,20 @@ async function editToChanges(message, guild, api) {
 	unchangedEvents.push(...eventsToReplace.filter(ev => !eventCanBeEdited(ev))) // Move them from eventsToRedact to unchangedEvents.
 	eventsToReplace = eventsToReplace.filter(eventCanBeEdited)
 
+	// Now, everything in eventsToReplace has the potential to have changed, but did it actually?
+	// (Example: if a URL preview was generated or updated, the message text won't have changed.)
+	// Only way to detect this is by text content. So we'll remove text events from eventsToReplace that have the same new text as text currently in the event.
+	for (let i = eventsToReplace.length; i--;) { // move backwards through array
+		const event = eventsToReplace[i]
+		if (!eventIsText(event)) continue // not text, can't analyse
+		const oldEvent = await api.getEvent(roomID, eventsToReplace[i].old.event_id)
+		const oldEventBodyWithoutQuotedReply = oldEvent.content.body?.replace(/^(>.*\n)*\n*/sm, "")
+		if (oldEventBodyWithoutQuotedReply !== event.newInnerContent.body) continue // event changed, must replace it
+		// Move it from eventsToRedact to unchangedEvents.
+		unchangedEvents.push(...eventsToReplace.filter(ev => ev.old.event_id === event.old.event_id))
+		eventsToReplace = eventsToReplace.filter(ev => ev.old.event_id !== event.old.event_id)
+	}
+
 	// We want to maintain exactly one part = 0 and one reaction_part = 0 database row at all times.
 	// This would be disrupted if existing events that are (reaction_)part = 0 will be redacted.
 	// If that is the case, pick a different existing or newly sent event to be (reaction_)part = 0.
@@ -193,4 +211,3 @@ function makeReplacementEventContent(oldID, newFallbackContent, newInnerContent)
 }
 
 module.exports.editToChanges = editToChanges
-module.exports.makeReplacementEventContent = makeReplacementEventContent

src/d2m/converters/edit-to-changes.test.js

@@ -4,7 +4,14 @@ const data = require("../../../test/data")
 const Ty = require("../../types")
 
 test("edit2changes: edit by webhook", async t => {
-	const {senderMxid, eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.edit_by_webhook, data.guild.general, {})
+	let called = 0
+	const {senderMxid, eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.edit_by_webhook, data.guild.general, {
+		getEvent(roomID, eventID) {
+			called++
+			t.equal(eventID, "$zXSlyI78DQqQwwfPUSzZ1b-nXzbUrCDljJgnGDdoI10")
+			return {content: {body: "dummy"}}
+		}
+	})
 	t.deepEqual(eventsToRedact, [])
 	t.deepEqual(eventsToSend, [])
 	t.deepEqual(eventsToReplace, [{
@@ -28,10 +35,15 @@ test("edit2changes: edit by webhook", async t => {
 	}])
 	t.equal(senderMxid, null)
 	t.deepEqual(promotions, [])
+	t.equal(called, 1)
 })
 
 test("edit2changes: bot response", async t => {
 	const {senderMxid, eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.bot_response, data.guild.general, {
+		getEvent(roomID, eventID) {
+			t.equal(eventID, "$fdD9OZ55xg3EAsfvLZza5tMhtjUO91Wg3Otuo96TplY")
+			return {content: {body: "dummy"}}
+		},
 		async getJoinedMembers(roomID) {
 			t.equal(roomID, "!hYnGGlPHlbujVVfktC:cadence.moe")
 			return new Promise(resolve => {
@@ -123,7 +135,14 @@ test("edit2changes: add caption back to that image (due to it having a reaction,
 })
 
 test("edit2changes: stickers and attachments are not changed, only the content can be edited", async t => {
-	const {eventsToRedact, eventsToReplace, eventsToSend} = await editToChanges(data.message_update.edited_content_with_sticker_and_attachments, data.guild.general, {})
+	let called = 0
+	const {eventsToRedact, eventsToReplace, eventsToSend} = await editToChanges(data.message_update.edited_content_with_sticker_and_attachments, data.guild.general, {
+		getEvent(roomID, eventID) {
+			called++
+			t.equal(eventID, "$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qdFv4")
+			return {content: {body: "dummy"}}
+		}
+	})
 	t.deepEqual(eventsToRedact, [])
 	t.deepEqual(eventsToSend, [])
 	t.deepEqual(eventsToReplace, [{
@@ -145,10 +164,16 @@ test("edit2changes: stickers and attachments are not changed, only the content c
 			}
 		}
 	}])
+	t.equal(called, 1)
 })
 
 test("edit2changes: edit of reply to skull webp attachment with content", async t => {
-	const {eventsToRedact, eventsToReplace, eventsToSend} = await editToChanges(data.message_update.edit_of_reply_to_skull_webp_attachment_with_content, data.guild.general, {})
+	const {eventsToRedact, eventsToReplace, eventsToSend} = await editToChanges(data.message_update.edit_of_reply_to_skull_webp_attachment_with_content, data.guild.general, {
+		getEvent(roomID, eventID) {
+			t.equal(eventID, "$vgTKOR5ZTYNMKaS7XvgEIDaOWZtVCEyzLLi5Pc5Gz4M")
+			return {content: {body: "dummy"}}
+		}
+	})
 	t.deepEqual(eventsToRedact, [])
 	t.deepEqual(eventsToSend, [])
 	t.deepEqual(eventsToReplace, [{
@@ -177,7 +202,12 @@ test("edit2changes: edit of reply to skull webp attachment with content", async
 })
 
 test("edit2changes: edits the text event when multiple rows have part = 0 (should never happen in real life, but make sure the safety net works)", async t => {
-	const {eventsToRedact, eventsToReplace, eventsToSend} = await editToChanges(data.message_update.edited_content_with_sticker_and_attachments_but_all_parts_equal_0, data.guild.general, {})
+	const {eventsToRedact, eventsToReplace, eventsToSend} = await editToChanges(data.message_update.edited_content_with_sticker_and_attachments_but_all_parts_equal_0, data.guild.general, {
+		getEvent(roomID, eventID) {
+			t.equal(eventID, "$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qd999")
+			return {content: {body: "dummy"}}
+		}
+	})
 	t.deepEqual(eventsToRedact, [])
 	t.deepEqual(eventsToSend, [])
 	t.deepEqual(eventsToReplace, [{
@@ -202,7 +232,12 @@ test("edit2changes: edits the text event when multiple rows have part = 0 (shoul
 })
 
 test("edit2changes: promotes the text event when multiple rows have part = 1 (should never happen in real life, but make sure the safety net works)", async t => {
-	const {eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.edited_content_with_sticker_and_attachments_but_all_parts_equal_1, data.guild.general, {})
+	const {eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.edited_content_with_sticker_and_attachments_but_all_parts_equal_1, data.guild.general, {
+		getEvent(roomID, eventID) {
+			t.equal(eventID, "$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qd111")
+			return {content: {body: "dummy"}}
+		}
+	})
 	t.deepEqual(eventsToRedact, [])
 	t.deepEqual(eventsToSend, [])
 	t.deepEqual(eventsToReplace, [{
@@ -279,32 +314,31 @@ test("edit2changes: generated embed", async t => {
 })
 
 test("edit2changes: generated embed on a reply", async t => {
-	const {senderMxid, eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.embed_generated_on_reply, data.guild.general, {})
-	t.deepEqual(eventsToRedact, [])
-	t.deepEqual(eventsToReplace, [{
-		oldID: "$UTqiL3Zj3FC4qldxRLggN1fhygpKl8sZ7XGY5f9MNbF",
-		newContent: {
-			$type: "m.room.message",
+	let called = 0
+	const {senderMxid, eventsToRedact, eventsToReplace, eventsToSend, promotions} = await editToChanges(data.message_update.embed_generated_on_reply, data.guild.general, {
+		getEvent(roomID, eventID) {
+			called++
+			t.equal(eventID, "$UTqiL3Zj3FC4qldxRLggN1fhygpKl8sZ7XGY5f9MNbF")
+			return {
+				type: "m.room.message",
+				content: {
 					// Unfortunately the edited message doesn't include the message_reference field. Fine. Whatever. It looks normal if you're using a good client.
 					body: "> a Discord user: [Replied-to message content wasn't provided by Discord]"
-				+ "\n\n* https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM",
+						+ "\n\nhttps://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM",
 					format: "org.matrix.custom.html",
-			formatted_body: "<mx-reply><blockquote><a href=\"https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM\">In reply to</a> a Discord user<br>[Replied-to message content wasn't provided by Discord]</blockquote></mx-reply>* <a href=\"https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM\">https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM</a>",
+					formatted_body: "<mx-reply><blockquote><a href=\"https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM\">In reply to</a> a Discord user<br>[Replied-to message content wasn't provided by Discord]</blockquote></mx-reply><a href=\"https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM\">https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM</a>",
 					"m.mentions": {},
-			"m.new_content": {
-				body: "https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM",
-				format: "org.matrix.custom.html",
-				formatted_body: "<a href=\"https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM\">https://matrix.to/#/!BnKuBPCvyfOkhcUjEu:cadence.moe/$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM</a>",
-				"m.mentions": {},
-				msgtype: "m.text",
-			},
 					"m.relates_to": {
 						event_id: "$UTqiL3Zj3FC4qldxRLggN1fhygpKl8sZ7XGY5f9MNbF",
 						rel_type: "m.replace",
 					},
 					msgtype: "m.text",
-		},
-	}])
+				}
+			}
+		}
+	})
+	t.deepEqual(eventsToRedact, [])
+	t.deepEqual(eventsToReplace, [])
 	t.deepEqual(eventsToSend, [{
 		$type: "m.room.message",
 		msgtype: "m.notice",
@@ -324,4 +358,5 @@ test("edit2changes: generated embed on a reply", async t => {
 		"nextEvent": true,
 	}])
 	t.equal(senderMxid, "@_ooye_cadence:cadence.moe")
+	t.equal(called, 1)
 })

src/d2m/converters/emoji-to-key.js

@@ -8,9 +8,10 @@ const file = sync.require("../../matrix/file")
 
 /**
  * @param {import("discord-api-types/v10").APIEmoji} emoji
+ * @param {string} message_id
  * @returns {Promise<string>}
  */
-async function emojiToKey(emoji) {
+async function emojiToKey(emoji, message_id) {
 	let key
 	if (emoji.id) {
 		// Custom emoji
@@ -30,7 +31,10 @@ async function emojiToKey(emoji) {
 		// Default emoji
 		const name = emoji.name
 		assert(name)
-		key = name
+		// If the reaction was used on Matrix already, it might be using a different arrangement of Variation Selector 16 characters.
+		// We'll use the same arrangement that was originally used, otherwise a duplicate of the emoji will appear as a separate reaction.
+		const originalEncoding = select("reaction", "original_encoding", {message_id, encoded_emoji: encodeURIComponent(name)}).pluck().get()
+		key = originalEncoding || name
 	}
 	return key
 }

src/d2m/converters/message-to-event.embeds.test.js

@@ -321,6 +321,25 @@ test("message2event embeds: youtube video", async t => {
 	}])
 })
 
+test("message2event embeds: tenor gif should show a video link without a provider", async t => {
+	const events = await messageToEvent(data.message_with_embeds.tenor_gif, data.guild.general, {}, {})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		msgtype: "m.text",
+		body: "@Realdditors: get real https://tenor.com/view/get-real-gif-26176788",
+		format: "org.matrix.custom.html",
+		formatted_body: "<font color=\"#ff4500\">@Realdditors</font> get real <a href=\"https://tenor.com/view/get-real-gif-26176788\">https://tenor.com/view/get-real-gif-26176788</a>",
+		"m.mentions": {}
+	}, {
+		$type: "m.room.message",
+		msgtype: "m.notice",
+		body: "| 🎞️ https://media.tenor.com/Bz5pfRIu81oAAAPo/get-real.mp4",
+		format: "org.matrix.custom.html",
+		formatted_body: "<blockquote><p>🎞️ https://media.tenor.com/Bz5pfRIu81oAAAPo/get-real.mp4</p></blockquote>",
+		"m.mentions": {}
+	}])
+})
+
 test("message2event embeds: if discord creates an embed preview for a discord channel link, don't copy that embed", async t => {
 	const events = await messageToEvent(data.message_with_embeds.discord_server_included_punctuation_bad_discord, data.guild.general, {}, {
 		api: {

src/d2m/converters/message-to-event.js

@@ -33,8 +33,10 @@ function getDiscordParseCallbacks(message, guild, useHTML) {
 		user: node => {
 			const mxid = select("sim", "mxid", {user_id: node.id}).pluck().get()
 			const interaction = message.interaction_metadata || message.interaction
-			const username = message.mentions.find(ment => ment.id === node.id)?.username
+			const username = message.mentions?.find(ment => ment.id === node.id)?.username
+				|| message.referenced_message?.mentions?.find(ment => ment.id === node.id)?.username
 				|| (interaction?.user.id === node.id ? interaction.user.username : null)
+				|| (message.author.id === node.id ? message.author.username : null)
 				|| node.id
 			if (mxid && useHTML) {
 				return `<a href="https://matrix.to/#/${mxid}">@${username}</a>`
@@ -204,7 +206,7 @@ async function attachmentToEvent(mentions, attachment) {
  * - includeEditFallbackStar: false
  * - alwaysReturnFormattedBody: false - formatted_body will be skipped if it is the same as body because the message is plaintext. if you want the formatted_body to be returned anyway, for example to merge it with another message, then set this to true.
  * - scanTextForMentions: true - needs to be set to false when converting forwarded messages etc which may be from a different channel that can't be scanned.
- * @param {{api: import("../../matrix/api")}} di simple-as-nails dependency injection for the matrix API
+ * @param {{api: import("../../matrix/api"), snow?: import("snowtransfer").SnowTransfer}} di simple-as-nails dependency injection for the matrix API
  */
 async function messageToEvent(message, guild, options = {}, di) {
 	const events = []
@@ -401,7 +403,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 			const id = match[3]
 			const name = match[2]
 			const animated = !!match[1]
-			return emojiToKey.emojiToKey({id, name, animated}) // Register the custom emoji if needed
+			return emojiToKey.emojiToKey({id, name, animated}, message.id) // Register the custom emoji if needed
 		}))
 
 		async function transformParsedVia(parsed) {
@@ -412,8 +414,10 @@ async function messageToEvent(message, guild, options = {}, di) {
 						node.via = await getViaServersMemo(node.row.room_id)
 					}
 				}
-				if (Array.isArray(node.content)) {
-					await transformParsedVia(node.content)
+				;for (const maybeChildNodesArray of [node, node.content, node.items]) {
+					if (Array.isArray(maybeChildNodesArray)) {
+						await transformParsedVia(maybeChildNodesArray)
+					}
 				}
 			}
 			return parsed
@@ -477,14 +481,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 			}
 			if (repliedToContent == "") repliedToContent = "[Media]"
 			else if (!repliedToContent) repliedToContent = "[Replied-to message content wasn't provided by Discord]"
-			const repliedToHtml = markdown.toHTML(repliedToContent, {
-				discordCallback: getDiscordParseCallbacks(message, guild, true)
-			})
-			const repliedToBody = markdown.toHTML(repliedToContent, {
-				discordCallback: getDiscordParseCallbacks(message, guild, false),
-				discordOnly: true,
-				escapeHTML: false,
-			})
+			const {body: repliedToBody, html: repliedToHtml} = await transformContent(repliedToContent)
 			if (repliedToEventRow) {
 				// Generate a reply pointing to the Matrix event we found
 				html = `<mx-reply><blockquote><a href="https://matrix.to/#/${repliedToEventRow.room_id}/${repliedToEventRow.event_id}">In reply to</a> ${repliedToUserHtml}`
@@ -496,19 +493,11 @@ async function messageToEvent(message, guild, options = {}, di) {
 			} else { // repliedToUnknownEvent
 				// This reply can't point to the Matrix event because it isn't bridged, we need to indicate this.
 				assert(message.referenced_message)
-				const dateDifference = new Date(message.timestamp).getTime() - new Date(message.referenced_message.timestamp).getTime()
-				const oneHour = 60 * 60 * 1000
-				if (dateDifference < oneHour) {
-					var dateDisplay = "n"
-				} else if (dateDifference < 25 * oneHour) {
-					var dateDisplay = ` ${Math.floor(dateDifference / oneHour)}-hour-old`
-				} else {
-					var dateDisplay = ` ${Math.round(dateDifference / (24 * oneHour))}-day-old`
-				}
-				html = `<blockquote>In reply to a${dateDisplay} unbridged message from ${repliedToDisplayName}:`
+				const dateDisplay = dUtils.howOldUnbridgedMessage(message.referenced_message.timestamp, message.timestamp)
+				html = `<blockquote>In reply to ${dateDisplay} from ${repliedToDisplayName}:`
 					+ `<br>${repliedToHtml}</blockquote>`
 					+ html
-				body = (`In reply to a${dateDisplay} unbridged message:\n${repliedToDisplayName}: `
+				body = (`In reply to ${dateDisplay}:\n${repliedToDisplayName}: `
 					+ repliedToBody).split("\n").map(line => "> " + line).join("\n")
 					+ "\n\n" + body
 			}
@@ -615,6 +604,49 @@ async function messageToEvent(message, guild, options = {}, di) {
 		await addTextEvent(body, html, msgtype)
 	}
 
+	// Then scheduled events
+	if (message.content && di?.snow) {
+		for (const match of [...message.content.matchAll(/discord\.gg\/([A-Za-z0-9]+)\?event=([0-9]{18,})/g)]) { // snowflake has minimum 18 because the events feature is at least that old
+			const invite = await di.snow.invite.getInvite(match[1], {guild_scheduled_event_id: match[2]})
+			const event = invite.guild_scheduled_event
+			if (!event) continue // the event ID provided was not valid
+
+			const formatter = new Intl.DateTimeFormat("en-NZ", {month: "long", day: "numeric", hour: "numeric", minute: "2-digit", timeZoneName: "shortGeneric"}) // 9 June at 3:00 pm NZT
+			const rep = new mxUtils.MatrixStringBuilder()
+
+			// Add time
+			if (event.scheduled_end_time) {
+				// @ts-ignore - no definition available for formatRange
+				rep.addParagraph(`Scheduled Event - ${formatter.formatRange(new Date(event.scheduled_start_time), new Date(event.scheduled_end_time))}`)
+			} else {
+				rep.addParagraph(`Scheduled Event - ${formatter.format(new Date(event.scheduled_start_time))}`)
+			}
+
+			// Add details
+			rep.addLine(`## ${event.name}`, tag`<strong>${event.name}</strong>`)
+			if (event.description) rep.addLine(event.description)
+
+			// Add location
+			if (event.entity_metadata?.location) {
+				rep.addParagraph(`📍 ${event.entity_metadata.location}`)
+			} else if (invite.channel?.name) {
+				const roomID = select("channel_room", "room_id", {channel_id: invite.channel.id}).pluck().get()
+				if (roomID) {
+					const via = await getViaServersMemo(roomID)
+					rep.addParagraph(`🔊 ${invite.channel.name} - https://matrix.to/#/${roomID}?${via}`, tag`🔊 ${invite.channel.name} - <a href="https://matrix.to/#/${roomID}?${via}">${invite.channel.name}</a>`)
+				} else {
+					rep.addParagraph(`🔊 ${invite.channel.name}`)
+				}
+			}
+
+			// Send like an embed
+			let {body, formatted_body: html} = rep.get()
+			body = body.split("\n").map(l => "| " + l).join("\n")
+			html = `<blockquote>${html}</blockquote>`
+			await addTextEvent(body, html, "m.notice")
+		}
+	}
+
 	// Then attachments
 	if (message.attachments) {
 		const attachmentEvents = await Promise.all(message.attachments.map(attachmentToEvent.bind(null, mentions)))
@@ -640,7 +672,7 @@ async function messageToEvent(message, guild, options = {}, di) {
 		const rep = new mxUtils.MatrixStringBuilder()
 
 		// Provider
-		if (embed.provider?.name) {
+		if (embed.provider?.name && embed.provider.name !== "Tenor") {
 			if (embed.provider.url) {
 				rep.addParagraph(`via ${embed.provider.name} ${embed.provider.url}`, tag`<sub><a href="${embed.provider.url}">${embed.provider.name}</a></sub>`)
 			} else {

src/d2m/converters/message-to-event.test.js

@@ -532,6 +532,43 @@ test("message2event: simple reply to matrix user, reply fallbacks disabled", asy
 	}])
 })
 
+test("message2event: reply to matrix user with mention", async t => {
+	const events = await messageToEvent(data.message.reply_to_matrix_user_mention, data.guild.general, {}, {
+		api: {
+			getEvent: mockGetEvent(t, "!kLRqKKUQXcibIMtOpl:cadence.moe", "$7P2O_VTQNHvavX5zNJ35DV-dbJB1Ag80tGQP_JzGdhk", {
+				type: "m.room.message",
+				content: {
+					msgtype: "m.text",
+					body: "@_ooye_extremity:cadence.moe you owe me $30",
+					format: "org.matrix.custom.html",
+					formatted_body: "<a href=\"https://matrix.to/#/@_ooye_extremity:cadence.moe\">@_ooye_extremity:cadence.moe</a> you owe me $30"
+				},
+				sender: "@cadence:cadence.moe"
+			})
+		}
+	})
+	t.deepEqual(events, [{
+		$type: "m.room.message",
+		"m.relates_to": {
+			"m.in_reply_to": {
+				event_id: "$7P2O_VTQNHvavX5zNJ35DV-dbJB1Ag80tGQP_JzGdhk"
+			}
+		},
+		"m.mentions": {
+			user_ids: [
+				"@cadence:cadence.moe"
+			]
+		},
+		msgtype: "m.text",
+		body: "> okay 🤍 yay 🤍: @extremity: you owe me $30\n\nkys",
+		format: "org.matrix.custom.html",
+		formatted_body:
+			'<mx-reply><blockquote><a href="https://matrix.to/#/!kLRqKKUQXcibIMtOpl:cadence.moe/$7P2O_VTQNHvavX5zNJ35DV-dbJB1Ag80tGQP_JzGdhk">In reply to</a> <a href="https://matrix.to/#/@cadence:cadence.moe">okay 🤍 yay 🤍</a>'
+			+ '<br><a href="https://matrix.to/#/@_ooye_extremity:cadence.moe">@extremity</a> you owe me $30</blockquote></mx-reply>'
+			+ 'kys'
+	}])
+})
+
 test("message2event: reply with a video", async t => {
 	const events = await messageToEvent(data.message.reply_with_video, data.guild.general, {
 		api: {
@@ -1165,3 +1202,174 @@ test("message2event: don't scan forwarded messages for mentions", async t => {
 		}
 	])
 })
+
+test("message2event: invite no details embed if no event", async t => {
+	const events = await messageToEvent({content: "https://discord.gg/placeholder?event=1381190945646710824"}, {}, {}, {
+		snow: {
+			invite: {
+				getInvite: async () => ({...data.invite.irl, guild_scheduled_event: null})
+			}
+		}
+	})
+	t.deepEqual(events, [
+		{
+			$type: "m.room.message",
+			body: "https://discord.gg/placeholder?event=1381190945646710824",
+			format: "org.matrix.custom.html",
+			formatted_body: "<a href=\"https://discord.gg/placeholder?event=1381190945646710824\">https://discord.gg/placeholder?event=1381190945646710824</a>",
+			"m.mentions": {},
+			msgtype: "m.text",
+		}
+	])
+})
+
+test("message2event: irl invite event renders embed", async t => {
+	const events = await messageToEvent({content: "https://discord.gg/placeholder?event=1381190945646710824"}, {}, {}, {
+		snow: {
+			invite: {
+				getInvite: async () => data.invite.irl
+			}
+		}
+	})
+	t.deepEqual(events, [
+		{
+			$type: "m.room.message",
+			body: "https://discord.gg/placeholder?event=1381190945646710824",
+			format: "org.matrix.custom.html",
+			formatted_body: "<a href=\"https://discord.gg/placeholder?event=1381190945646710824\">https://discord.gg/placeholder?event=1381190945646710824</a>",
+			"m.mentions": {},
+			msgtype: "m.text",
+		},
+		{
+			$type: "m.room.message",
+			msgtype: "m.notice",
+			body: `| Scheduled Event - 8 June at 10:00 pm NZT – 9 June at 12:00 am NZT`
+				+ `\n| ## forest exploration`
+				+ `\n| `
+				+ `\n| 📍 the dark forest`,
+			format: "org.matrix.custom.html",
+			formatted_body: `<blockquote><p>Scheduled Event - 8 June at 10:00 pm NZT – 9 June at 12:00 am NZT</p>`
+				+ `<strong>forest exploration</strong>`
+				+ `<p>📍 the dark forest</p></blockquote>`,
+			"m.mentions": {}
+		}
+	])
+})
+
+test("message2event: vc invite event renders embed", async t => {
+	const events = await messageToEvent({content: "https://discord.gg/placeholder?event=1381174024801095751"}, {}, {}, {
+		snow: {
+			invite: {
+				getInvite: async () => data.invite.vc
+			}
+		}
+	})
+	t.deepEqual(events, [
+		{
+			$type: "m.room.message",
+			body: "https://discord.gg/placeholder?event=1381174024801095751",
+			format: "org.matrix.custom.html",
+			formatted_body: "<a href=\"https://discord.gg/placeholder?event=1381174024801095751\">https://discord.gg/placeholder?event=1381174024801095751</a>",
+			"m.mentions": {},
+			msgtype: "m.text",
+		},
+		{
+			$type: "m.room.message",
+			msgtype: "m.notice",
+			body: `| Scheduled Event - 9 June at 3:00 pm NZT`
+				+ `\n| ## Cooking (Netrunners)`
+				+ `\n| Short circuited brain interfaces actually just means your brain is medium rare, yum.`
+				+ `\n| `
+				+ `\n| 🔊 Cooking`,
+			format: "org.matrix.custom.html",
+			formatted_body: `<blockquote><p>Scheduled Event - 9 June at 3:00 pm NZT</p>`
+				+ `<strong>Cooking (Netrunners)</strong><br>Short circuited brain interfaces actually just means your brain is medium rare, yum.`
+				+ `<p>🔊 Cooking</p></blockquote>`,
+			"m.mentions": {}
+		}
+	])
+})
+
+test("message2event: vc invite event renders embed with room link", async t => {
+	const events = await messageToEvent({content: "https://discord.gg/placeholder?event=1381174024801095751"}, {}, {}, {
+		api: {
+			getJoinedMembers: async () => ({
+				joined: {
+					"@_ooye_bot:cadence.moe": {display_name: null, avatar_url: null},
+				}
+			})
+		},
+		snow: {
+			invite: {
+				getInvite: async () => data.invite.known_vc
+			}
+		}
+	})
+	t.deepEqual(events, [
+		{
+			$type: "m.room.message",
+			body: "https://discord.gg/placeholder?event=1381174024801095751",
+			format: "org.matrix.custom.html",
+			formatted_body: "<a href=\"https://discord.gg/placeholder?event=1381174024801095751\">https://discord.gg/placeholder?event=1381174024801095751</a>",
+			"m.mentions": {},
+			msgtype: "m.text",
+		},
+		{
+			$type: "m.room.message",
+			msgtype: "m.notice",
+			body: `| Scheduled Event - 9 June at 3:00 pm NZT`
+				+ `\n| ## Cooking (Netrunners)`
+				+ `\n| Short circuited brain interfaces actually just means your brain is medium rare, yum.`
+				+ `\n| `
+				+ `\n| 🔊 Hey. - https://matrix.to/#/!FuDZhlOAtqswlyxzeR:cadence.moe?via=cadence.moe`,
+			format: "org.matrix.custom.html",
+			formatted_body: `<blockquote><p>Scheduled Event - 9 June at 3:00 pm NZT</p>`
+				+ `<strong>Cooking (Netrunners)</strong><br>Short circuited brain interfaces actually just means your brain is medium rare, yum.`
+				+ `<p>🔊 Hey. - <a href="https://matrix.to/#/!FuDZhlOAtqswlyxzeR:cadence.moe?via=cadence.moe">Hey.</a></p></blockquote>`,
+			"m.mentions": {}
+		}
+	])
+})
+
+test("message2event: channel links are converted even inside lists (parser post-processer descends into list items)", async t => {
+	let called = 0
+	const events = await messageToEvent({
+		content: "1. Don't be a dick"
+		+ "\n2. Follow rule number 1"
+		+ "\n3. Follow Discord TOS"
+		+ "\n4. Do **not** post NSFW content, shock content, suggestive content"
+		+ "\n5. Please keep <#176333891320283136> professional and helpful, no random off-topic joking"
+		+ "\nThis list will probably change in the future"
+	}, data.guild.general, {}, {
+		api: {
+			getJoinedMembers(roomID) {
+				called++
+				t.equal(roomID, "!qzDBLKlildpzrrOnFZ:cadence.moe")
+				return {
+					joined: {
+						"@quadradical:federated.nexus": {
+							membership: "join",
+							display_name: "quadradical"
+						}
+					}
+				}
+			}
+		}
+	})
+	t.deepEqual(events, [
+		{
+			$type: "m.room.message",
+			body: "1. Don't be a dick"
+			+ "\n2. Follow rule number 1"
+			+ "\n3. Follow Discord TOS"
+			+ "\n4. Do **not** post NSFW content, shock content, suggestive content"
+			+ "\n5. Please keep #wonderland professional and helpful, no random off-topic joking"
+			+ "\nThis list will probably change in the future",
+			format: "org.matrix.custom.html",
+			formatted_body: "<ol start=\"1\"><li>Don't be a dick</li><li>Follow rule number 1</li><li>Follow Discord TOS</li><li>Do <strong>not</strong> post NSFW content, shock content, suggestive content</li><li>Please keep <a href=\"https://matrix.to/#/!qzDBLKlildpzrrOnFZ:cadence.moe?via=cadence.moe&via=federated.nexus\">#wonderland</a> professional and helpful, no random off-topic joking</li></ol>This list will probably change in the future",
+			"m.mentions": {},
+			msgtype: "m.text"
+		}
+	])
+	t.equal(called, 1)
+})

src/d2m/converters/pins-to-list.js

@@ -4,16 +4,28 @@ const {select} = require("../../passthrough")
 
 /**
  * @param {import("discord-api-types/v10").RESTGetAPIChannelPinsResult} pins
+ * @param {{"m.room.pinned_events/"?: {pinned?: string[]}}} kstate
  */
-function pinsToList(pins) {
+function pinsToList(pins, kstate) {
+	let alreadyPinned = kstate["m.room.pinned_events/"]?.pinned || []
+
+	// If any of the already pinned messages are bridged messages then remove them from the already pinned list.
+	//   * If a bridged message is still pinned then it'll be added back in the next step.
+	//   * If a bridged message was unpinned from Discord-side then it'll be unpinned from our side due to this step.
+	//   * Matrix-only unbridged messages that are pinned will remain pinned.
+	alreadyPinned = alreadyPinned.filter(event_id => {
+		const messageID = select("event_message", "message_id", {event_id}).pluck().get()
+		return !messageID || pins.find(m => m.id === messageID) // if it is bridged then remove it from the filter
+	})
+
 	/** @type {string[]} */
 	const result = []
 	for (const message of pins) {
 		const eventID = select("event_message", "event_id", {message_id: message.id, part: 0}).pluck().get()
-		if (eventID) result.push(eventID)
+		if (eventID && !alreadyPinned.includes(eventID)) result.push(eventID)
 	}
 	result.reverse()
-	return result
+	return alreadyPinned.concat(result)
 }
 
 module.exports.pinsToList = pinsToList

src/d2m/converters/pins-to-list.test.js

@@ -3,10 +3,59 @@ const data = require("../../../test/data")
 const {pinsToList} = require("./pins-to-list")
 
 test("pins2list: converts known IDs, ignores unknown IDs", t => {
-	const result = pinsToList(data.pins.faked)
+	const result = pinsToList(data.pins.faked, {})
 	t.deepEqual(result, [
 		"$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qdFv4",
 		"$mtR8cJqM4fKno1bVsm8F4wUVqSntt2sq6jav1lyavuA",
 		"$X16nfVks1wsrhq4E9SSLiqrf2N8KD0erD0scZG7U5xg"
 	])
 })
+
+test("pins2list: already pinned duplicate items are not moved", t => {
+	const result = pinsToList(data.pins.faked, {
+		"m.room.pinned_events/": {
+			pinned: [
+				"$mtR8cJqM4fKno1bVsm8F4wUVqSntt2sq6jav1lyavuA"
+			]
+		}
+	})
+	t.deepEqual(result, [
+		"$mtR8cJqM4fKno1bVsm8F4wUVqSntt2sq6jav1lyavuA",
+		"$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qdFv4",
+		"$X16nfVks1wsrhq4E9SSLiqrf2N8KD0erD0scZG7U5xg"
+	])
+})
+
+test("pins2list: already pinned unknown items are not moved", t => {
+	const result = pinsToList(data.pins.faked, {
+		"m.room.pinned_events/": {
+			pinned: [
+				"$unknown1",
+				"$mtR8cJqM4fKno1bVsm8F4wUVqSntt2sq6jav1lyavuA",
+				"$unknown2"
+			]
+		}
+	})
+	t.deepEqual(result, [
+		"$unknown1",
+		"$mtR8cJqM4fKno1bVsm8F4wUVqSntt2sq6jav1lyavuA",
+		"$unknown2",
+		"$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qdFv4",
+		"$X16nfVks1wsrhq4E9SSLiqrf2N8KD0erD0scZG7U5xg"
+	])
+})
+
+test("pins2list: bridged messages can be unpinned", t => {
+	const result = pinsToList(data.pins.faked.slice(0, -2), {
+		"m.room.pinned_events/": {
+			pinned: [
+				"$mtR8cJqM4fKno1bVsm8F4wUVqSntt2sq6jav1lyavuA",
+				"$lnAF9IosAECTnlv9p2e18FG8rHn-JgYKHEHIh5qdFv4"
+			]
+		}
+	})
+	t.deepEqual(result, [
+		"$mtR8cJqM4fKno1bVsm8F4wUVqSntt2sq6jav1lyavuA",
+		"$X16nfVks1wsrhq4E9SSLiqrf2N8KD0erD0scZG7U5xg",
+	])
+})

src/d2m/discord-client.js

@@ -62,9 +62,6 @@ class DiscordClient {
 		addEventLogger("error", "Error")
 		addEventLogger("disconnected", "Disconnected")
 		addEventLogger("ready", "Ready")
-		this.snow.requestHandler.on("requestError", (requestID, error) => {
-			console.error("request error:", error)
-		})
 	}
 }
 

src/d2m/discord-packets.js

@@ -157,59 +157,17 @@ const utils = {
 		}
 
 		// Event dispatcher for OOYE bridge operations
-		if (listen === "full") {
+		if (listen === "full" && message.t) {
 			try {
-				if (message.t === "GUILD_UPDATE") {
-					await eventDispatcher.onGuildUpdate(client, message.d)
-
-				} else if (message.t === "GUILD_EMOJIS_UPDATE" || message.t === "GUILD_STICKERS_UPDATE") {
-					await eventDispatcher.onExpressionsUpdate(client, message.d)
-
-				} else if (message.t === "CHANNEL_UPDATE") {
-					await eventDispatcher.onChannelOrThreadUpdate(client, message.d, false)
-
-				} else if (message.t === "CHANNEL_PINS_UPDATE") {
-					await eventDispatcher.onChannelPinsUpdate(client, message.d)
-
-				} else if (message.t === "CHANNEL_DELETE") {
-					await eventDispatcher.onChannelDelete(client, message.d)
-
-				} else if (message.t === "THREAD_CREATE") {
-					// @ts-ignore
-					await eventDispatcher.onThreadCreate(client, message.d)
-
-				} else if (message.t === "THREAD_UPDATE") {
-					// @ts-ignore
-					await eventDispatcher.onChannelOrThreadUpdate(client, message.d, true)
-
-				} else if (message.t === "MESSAGE_CREATE") {
-					await eventDispatcher.onMessageCreate(client, message.d)
-
-				} else if (message.t === "MESSAGE_UPDATE") {
-					await eventDispatcher.onMessageUpdate(client, message.d)
-
-				} else if (message.t === "MESSAGE_DELETE") {
-					await eventDispatcher.onMessageDelete(client, message.d)
-
-				} else if (message.t === "MESSAGE_DELETE_BULK") {
-					await eventDispatcher.onMessageDeleteBulk(client, message.d)
-
-				} else if (message.t === "TYPING_START") {
-					await eventDispatcher.onTypingStart(client, message.d)
-
-				} else if (message.t === "MESSAGE_REACTION_ADD") {
-					await eventDispatcher.onReactionAdd(client, message.d)
-
-				} else if (message.t === "MESSAGE_REACTION_REMOVE" || message.t === "MESSAGE_REACTION_REMOVE_EMOJI" || message.t === "MESSAGE_REACTION_REMOVE_ALL") {
+				if (message.t === "MESSAGE_REACTION_REMOVE" || message.t === "MESSAGE_REACTION_REMOVE_EMOJI" || message.t === "MESSAGE_REACTION_REMOVE_ALL") {
 					await eventDispatcher.onSomeReactionsRemoved(client, message.d)
 
 				} else if (message.t === "INTERACTION_CREATE") {
 					await interactions.dispatchInteraction(message.d)
 
-				} else if (message.t === "PRESENCE_UPDATE") {
-					eventDispatcher.onPresenceUpdate(client, message.d)
+				} else if (message.t in eventDispatcher) {
+					await eventDispatcher[message.t](client, message.d)
 				}
-
 			} catch (e) {
 				// Let OOYE try to handle errors too
 				await eventDispatcher.onError(client, e, message)

src/d2m/event-dispatcher.js

@@ -2,7 +2,6 @@
 
 const assert = require("assert").strict
 const DiscordTypes = require("discord-api-types/v10")
-const util = require("util")
 const {sync, db, select, from} = require("../passthrough")
 
 /** @type {import("./actions/send-message")}) */
@@ -27,21 +26,18 @@ const updatePins = sync.require("./actions/update-pins")
 const api = sync.require("../matrix/api")
 /** @type {import("../discord/utils")} */
 const dUtils = sync.require("../discord/utils")
-/** @type {import("../m2d/converters/utils")} */
-const mxUtils = require("../m2d/converters/utils")
 /** @type {import("./actions/speedbump")} */
 const speedbump = sync.require("./actions/speedbump")
 /** @type {import("./actions/retrigger")} */
 const retrigger = sync.require("./actions/retrigger")
 /** @type {import("./actions/set-presence")} */
 const setPresence = sync.require("./actions/set-presence")
+/** @type {import("../m2d/event-dispatcher")} */
+const matrixEventDispatcher = sync.require("../m2d/event-dispatcher")
 
-/** @type {any} */ // @ts-ignore bad types from semaphore
-const Semaphore = require("@chriscdn/promise-semaphore")
+const {Semaphore} = require("@chriscdn/promise-semaphore")
 const checkMissedPinsSema = new Semaphore()
 
-let lastReportedEvent = 0
-
 // Grab Discord events we care about for the bridge, check them, and pass them on
 
 module.exports = {
@@ -51,48 +47,14 @@ module.exports = {
 	 * @param {import("cloudstorm").IGatewayMessage} gatewayMessage
 	 */
 	async onError(client, e, gatewayMessage) {
-		console.error("hit event-dispatcher's error handler with this exception:")
-		console.error(e) // TODO: also log errors into a file or into the database, maybe use a library for this? or just wing it? definitely need to be able to store the formatted event body to load back in later
-		console.error(`while handling this ${gatewayMessage.t} gateway event:`)
-		console.dir(gatewayMessage.d, {depth: null})
-
-		if (gatewayMessage.t === "TYPING_START") return
-
-		if (Date.now() - lastReportedEvent < 5000) return
-		lastReportedEvent = Date.now()
-
 		const channelID = gatewayMessage.d["channel_id"]
 		if (!channelID) return
 		const roomID = select("channel_room", "room_id", {channel_id: channelID}).pluck().get()
 		if (!roomID) return
 
-		let stackLines = null
-		if (e.stack) {
-			stackLines = e.stack.split("\n")
-			let cloudstormLine = stackLines.findIndex(l => l.includes("/node_modules/cloudstorm/"))
-			if (cloudstormLine !== -1) {
-				stackLines = stackLines.slice(0, cloudstormLine - 2)
-			}
-		}
+		if (gatewayMessage.t === "TYPING_START") return
 
-		const builder = new mxUtils.MatrixStringBuilder()
-		builder.addLine("\u26a0 Bridged event from Discord not delivered", "\u26a0 <strong>Bridged event from Discord not delivered</strong>")
-		builder.addLine(`Gateway event: ${gatewayMessage.t}`)
-		builder.addLine(e.toString())
-		if (stackLines) {
-			builder.addLine(`Error trace:\n${stackLines.join("\n")}`, `<details><summary>Error trace</summary><pre>${stackLines.join("\n")}</pre></details>`)
-		}
-		builder.addLine("", `<details><summary>Original payload</summary><pre>${util.inspect(gatewayMessage.d, false, 4, false)}</pre></details>`)
-		await api.sendEvent(roomID, "m.room.message", {
-			...builder.get(),
-			"moe.cadence.ooye.error": {
-				source: "discord",
-				payload: gatewayMessage
-			},
-			"m.mentions": {
-				user_ids: ["@cadence:cadence.moe"]
-			}
-		})
+		await matrixEventDispatcher.sendError(roomID, "Discord", gatewayMessage.t, e, gatewayMessage)
 	},
 
 	/**
@@ -146,13 +108,24 @@ module.exports = {
 			})
 			// console.log(`[check missed messages] got ${messages.length} messages; last message that IS bridged is at position ${latestBridgedMessageIndex} in the channel`)
 			if (latestBridgedMessageIndex === -1) latestBridgedMessageIndex = 1 // rather than crawling the ENTIRE channel history, let's just bridge the most recent 1 message to make it up to date.
+
+			// We get member data so that we can accurately update any changes to nickname or permissions that have occurred in the meantime
+			// The rate limit is lax enough that the backlog will still be pretty quick (at time of writing, 5 per 1 second per guild)
+			/** @type {Map<string, DiscordTypes.APIGuildMember | undefined>} id -> member: cache members for the run because people talk to each other */
+			const members = new Map()
+
+			// Send in order
 			for (let i = Math.min(messages.length, latestBridgedMessageIndex)-1; i >= 0; i--) {
-				const simulatedGatewayDispatchData = {
+				const message = messages[i]
+
+				if (!members.has(message.author.id)) members.set(message.author.id, await client.snow.guild.getGuildMember(guild.id, message.author.id).catch(() => undefined))
+				await module.exports.MESSAGE_CREATE(client, {
 					guild_id: guild.id,
+					member: members.get(message.author.id),
+					// @ts-ignore
 					backfill: true,
-					...messages[i]
-				}
-				await module.exports.onMessageCreate(client, simulatedGatewayDispatchData)
+					...message
+				})
 			}
 		}
 	},
@@ -199,7 +172,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.APIThreadChannel} thread
 	 */
-	async onThreadCreate(client, thread) {
+	async THREAD_CREATE(client, thread) {
 		const channelID = thread.parent_id || undefined
 		const parentRoomID = select("channel_room", "room_id", {channel_id: channelID}).pluck().get()
 		if (!parentRoomID) return // Not interested in a thread if we aren't interested in its wider channel (won't autocreate)
@@ -211,7 +184,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayGuildUpdateDispatchData} guild
 	 */
-	async onGuildUpdate(client, guild) {
+	async GUILD_UPDATE(client, guild) {
 		const spaceID = select("guild_space", "space_id", {guild_id: guild.id}).pluck().get()
 		if (!spaceID) return
 		await createSpace.syncSpace(guild)
@@ -220,19 +193,26 @@ module.exports = {
 	/**
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayChannelUpdateDispatchData} channelOrThread
-	 * @param {boolean} isThread
 	 */
-	async onChannelOrThreadUpdate(client, channelOrThread, isThread) {
+	async CHANNEL_UPDATE(client, channelOrThread) {
 		const roomID = select("channel_room", "room_id", {channel_id: channelOrThread.id}).pluck().get()
 		if (!roomID) return // No target room to update the data on
 		await createRoom.syncRoom(channelOrThread.id)
 	},
 
+	/**
+	 * @param {import("./discord-client")} client
+	 * @param {DiscordTypes.GatewayChannelUpdateDispatchData} thread
+	 */
+	async THREAD_UPDATE(client, thread) {
+		await module.exports.CHANNEL_UPDATE(client, thread)
+	},
+
 	/**
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayChannelPinsUpdateDispatchData} data
 	 */
-	async onChannelPinsUpdate(client, data) {
+	async CHANNEL_PINS_UPDATE(client, data) {
 		const roomID = select("channel_room", "room_id", {channel_id: data.channel_id}).pluck().get()
 		if (!roomID) return // No target room to update pins in
 		const convertedTimestamp = updatePins.convertTimestamp(data.last_pin_timestamp)
@@ -243,7 +223,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayChannelDeleteDispatchData} channel
 	 */
-	async onChannelDelete(client, channel) {
+	async CHANNEL_DELETE(client, channel) {
 		const guildID = channel["guild_id"]
 		if (!guildID) return // channel must have been a DM channel or something
 		const roomID = select("channel_room", "room_id", {channel_id: channel.id}).pluck().get()
@@ -256,7 +236,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayMessageCreateDispatchData} message
 	 */
-	async onMessageCreate(client, message) {
+	async MESSAGE_CREATE(client, message) {
 		if (message.author.username === "Deleted User") return // Nothing we can do for deleted users.
 		const channel = client.channels.get(message.channel_id)
 		if (!channel || !("guild_id" in channel) || !channel.guild_id) return // Nothing we can do in direct messages.
@@ -286,7 +266,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayMessageUpdateDispatchData} data
 	 */
-	async onMessageUpdate(client, data) {
+	async MESSAGE_UPDATE(client, data) {
 		// Based on looking at data they've sent me over the gateway, this is the best way to check for meaningful changes.
 		// If the message content is a string then it includes all interesting fields and is meaningful.
 		// Otherwise, if there are embeds, then the system generated URL preview embeds.
@@ -304,7 +284,7 @@ module.exports = {
 		if (affected) return
 
 		// Check that the sending-to room exists, and deal with Eventual Consistency(TM)
-		if (retrigger.eventNotFoundThenRetrigger(data.id, module.exports.onMessageUpdate, client, data)) return
+		if (retrigger.eventNotFoundThenRetrigger(data.id, module.exports.MESSAGE_UPDATE, client, data)) return
 
 		/** @type {DiscordTypes.GatewayMessageCreateDispatchData} */
 		// @ts-ignore
@@ -322,7 +302,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayMessageReactionAddDispatchData} data
 	 */
-	async onReactionAdd(client, data) {
+	async MESSAGE_REACTION_ADD(client, data) {
 		if (data.user_id === client.user.id) return // m2d reactions are added by the discord bot user - do not reflect them back to matrix.
 		await addReaction.addReaction(data)
 	},
@@ -339,9 +319,9 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayMessageDeleteDispatchData} data
 	 */
-	async onMessageDelete(client, data) {
+	async MESSAGE_DELETE(client, data) {
 		speedbump.onMessageDelete(data.id)
-		if (retrigger.eventNotFoundThenRetrigger(data.id, module.exports.onMessageDelete, client, data)) return
+		if (retrigger.eventNotFoundThenRetrigger(data.id, module.exports.MESSAGE_DELETE, client, data)) return
 		await deleteMessage.deleteMessage(data)
 	},
 
@@ -349,7 +329,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayMessageDeleteBulkDispatchData} data
 	 */
-		async onMessageDeleteBulk(client, data) {
+	async MESSAGE_DELETE_BULK(client, data) {
 		await deleteMessage.deleteMessageBulk(data)
 	},
 
@@ -357,7 +337,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayTypingStartDispatchData} data
 	 */
-	async onTypingStart(client, data) {
+	async TYPING_START(client, data) {
 		const roomID = select("channel_room", "room_id", {channel_id: data.channel_id}).pluck().get()
 		if (!roomID) return
 		const mxid = from("sim").join("sim_member", "mxid").where({user_id: data.user_id, room_id: roomID}).pluck("mxid").get()
@@ -370,9 +350,17 @@ module.exports = {
 
 	/**
 	 * @param {import("./discord-client")} client
-	 * @param {DiscordTypes.GatewayGuildEmojisUpdateDispatchData | DiscordTypes.GatewayGuildStickersUpdateDispatchData} data
+	 * @param {DiscordTypes.GatewayGuildEmojisUpdateDispatchData} data
 	 */
-	async onExpressionsUpdate(client, data) {
+	async GUILD_EMOJIS_UPDATE(client, data) {
+		await createSpace.syncSpaceExpressions(data, false)
+	},
+
+	/**
+	 * @param {import("./discord-client")} client
+	 * @param {DiscordTypes.GatewayGuildStickersUpdateDispatchData} data
+	 */
+	async GUILD_STICKERS_UPDATE(client, data) {
 		await createSpace.syncSpaceExpressions(data, false)
 	},
 
@@ -380,7 +368,7 @@ module.exports = {
 	 * @param {import("./discord-client")} client
 	 * @param {DiscordTypes.GatewayPresenceUpdateDispatchData} data
 	 */
-	onPresenceUpdate(client, data) {
+	PRESENCE_UPDATE(client, data) {
 		const status = data.status
 		if (!status) return
 		setPresence.presenceTracker.incomingPresence(data.user.id, data.guild_id, status)

src/db/migrations/0022-auto-emoji-without-guild.sql

@@ -0,0 +1,11 @@
+BEGIN TRANSACTION;
+
+DROP TABLE auto_emoji;
+
+CREATE TABLE auto_emoji (
+	name	TEXT NOT NULL,
+	emoji_id	TEXT NOT NULL,
+	PRIMARY KEY (name)
+) WITHOUT ROWID;
+
+COMMIT;

src/db/migrations/0023-add-original-encoding-to-reaction.sql

@@ -0,0 +1,5 @@
+BEGIN TRANSACTION;
+
+ALTER TABLE reaction ADD COLUMN original_encoding TEXT;
+
+COMMIT;

src/db/migrations/0024-add-direct.sql

@@ -0,0 +1,9 @@
+BEGIN TRANSACTION;
+
+CREATE TABLE direct (
+	mxid TEXT NOT NULL,
+	room_id TEXT NOT NULL,
+	PRIMARY KEY (mxid)
+) WITHOUT ROWID;
+
+COMMIT;

src/db/orm-defs.d.ts

@@ -1,4 +1,9 @@
 export type Models = {
+	auto_emoji: {
+		name: string
+		emoji_id: string
+	}
+
 	channel_room: {
 		channel_id: string
 		room_id: string
@@ -14,6 +19,18 @@ export type Models = {
 		custom_topic: number
 	}
 
+	direct: {
+		mxid: string
+		room_id: string
+	}
+
+	emoji: {
+		emoji_id: string
+		name: string
+		animated: number
+		mxc_url: string
+	}
+
 	event_message: {
 		event_id: string
 		message_id: string
@@ -55,6 +72,10 @@ export type Models = {
 		mxc_url: string
 	}
 
+	media_proxy: {
+		permitted_hash: number
+	}
+
 	member_cache: {
 		room_id: string
 		mxid: string
@@ -99,27 +120,11 @@ export type Models = {
 		webhook_token: string
 	}
 
-	emoji: {
-		emoji_id: string
-		name: string
-		animated: number
-		mxc_url: string
-	}
-
 	reaction: {
 		hashed_event_id: number
 		message_id: string
 		encoded_emoji: string
-	}
-
-	auto_emoji: {
-		name: string
-		emoji_id: string
-		guild_id: string
-	}
-
-	media_proxy: {
-		permitted_hash: number
+		original_encoding: string | null
 	}
 }
 

src/discord/utils.js

@@ -136,6 +136,24 @@ function getPublicUrlForCdn(url) {
 	return `${reg.ooye.bridge_origin}/download/discord${match[1]}/${match[2]}/${match[3]}/${match[4]}`
 }
 
+/**
+ * @param {string} oldTimestamp
+ * @param {string} newTimestamp
+ * @returns {string} "a x-day-old unbridged message"
+ */
+function howOldUnbridgedMessage(oldTimestamp, newTimestamp) {
+	const dateDifference = new Date(newTimestamp).getTime() - new Date(oldTimestamp).getTime()
+	const oneHour = 60 * 60 * 1000
+	if (dateDifference < oneHour) {
+		return "an unbridged message"
+	} else if (dateDifference < 25 * oneHour) {
+		var dateDisplay = `a ${Math.floor(dateDifference / oneHour)}-hour-old unbridged message`
+	} else {
+		var dateDisplay = `a ${Math.round(dateDifference / (24 * oneHour))}-day-old unbridged message`
+	}
+	return dateDisplay
+}
+
 module.exports.getPermissions = getPermissions
 module.exports.hasPermission = hasPermission
 module.exports.hasSomePermissions = hasSomePermissions
@@ -145,3 +163,4 @@ module.exports.isEphemeralMessage = isEphemeralMessage
 module.exports.snowflakeToTimestampExact = snowflakeToTimestampExact
 module.exports.timestampToSnowflakeInexact = timestampToSnowflakeInexact
 module.exports.getPublicUrlForCdn = getPublicUrlForCdn
+module.exports.howOldUnbridgedMessage = howOldUnbridgedMessage

src/m2d/actions/add-reaction.js

@@ -31,10 +31,14 @@ async function addReaction(event) {
 			// not adding it to the database otherwise a m->d removal would try calling the API
 			return
 		}
+		if (e.message?.includes("Unknown Emoji")) {
+			// happens if a matrix user tries to add on to a super reaction
+			return
+		}
 		throw e
 	}
 
-	db.prepare("REPLACE INTO reaction (hashed_event_id, message_id, encoded_emoji) VALUES (?, ?, ?)").run(utils.getEventIDHash(event.event_id), messageID, discordPreferredEncoding)
+	db.prepare("REPLACE INTO reaction (hashed_event_id, message_id, encoded_emoji, original_encoding) VALUES (?, ?, ?, ?)").run(utils.getEventIDHash(event.event_id), messageID, discordPreferredEncoding, key)
 }
 
 module.exports.addReaction = addReaction

src/m2d/actions/send-event.js

@@ -62,7 +62,7 @@ async function resolvePendingFiles(message) {
 /** @param {Ty.Event.Outer_M_Room_Message | Ty.Event.Outer_M_Room_Message_File | Ty.Event.Outer_M_Sticker} event */
 async function sendEvent(event) {
 	const row = select("channel_room", ["channel_id", "thread_parent"], {room_id: event.room_id}).get()
-	if (!row) return // allow the bot to exist in unbridged rooms, just don't do anything with it
+	if (!row) return [] // allow the bot to exist in unbridged rooms, just don't do anything with it
 	let channelID = row.channel_id
 	let threadID = undefined
 	if (row.thread_parent) {

src/m2d/actions/setup-emojis.js

@@ -0,0 +1,26 @@
+// @ts-check
+
+const fs = require("fs")
+const {join} = require("path")
+
+const passthrough = require("../../passthrough")
+
+const {id} = require("../../../addbot")
+
+async function setupEmojis() {
+	const {discord, db} = passthrough
+	const emojis = await discord.snow.assets.getAppEmojis(id)
+	for (const name of ["L1", "L2"]) {
+		const existing = emojis.items.find(e => e.name === name)
+		if (existing) {
+			db.prepare("REPLACE INTO auto_emoji (name, emoji_id) VALUES (?, ?)").run(existing.name, existing.id)
+		} else {
+			const filename = join(__dirname, "../../../docs/img", `${name}.png`)
+			const data = fs.readFileSync(filename, null)
+			const uploaded = await discord.snow.assets.createAppEmoji(id, {name, image: "data:image/png;base64," + data.toString("base64")})
+			db.prepare("REPLACE INTO auto_emoji (name, emoji_id) VALUES (?, ?)").run(uploaded.name, uploaded.id)
+		}
+	}
+}
+
+module.exports.setupEmojis = setupEmojis

src/m2d/converters/event-to-message.js

@@ -19,6 +19,8 @@ const dUtils = sync.require("../../discord/utils")
 const file = sync.require("../../matrix/file")
 /** @type {import("./emoji-sheet")} */
 const emojiSheet = sync.require("./emoji-sheet")
+/** @type {import("../actions/setup-emojis")} */
+const setupEmojis = sync.require("../actions/setup-emojis")
 
 /** @type {[RegExp, string][]} */
 const markdownEscapes = [
@@ -154,6 +156,27 @@ turndownService.addRule("listItem", {
 	}
 })
 
+turndownService.addRule("table", {
+	filter: "table",
+	replacement: function (content, node, options) {
+		const trs = node.querySelectorAll("tr").cache
+		/** @type {{text: string, tag: string}[][]} */
+		const tableText = trs.map(tr => [...tr.querySelectorAll("th, td")].map(cell => ({text: cell.textContent, tag: cell.tagName})))
+		const tableTextByColumn = tableText[0].map((col, i) => tableText.map(row => row[i]))
+		const columnWidths = tableTextByColumn.map(col => Math.max(...col.map(cell => cell.text.length)))
+		const resultRows = tableText.map((row, rowIndex) =>
+			row.map((cell, colIndex) =>
+				cell.text.padEnd(columnWidths[colIndex])
+			).join("  ")
+		)
+		const tableHasHeader = tableText[0].slice(1).some(cell => cell.tag === "TH")
+		if (tableHasHeader) {
+			resultRows.splice(1, 0, "-".repeat(columnWidths.reduce((a, c) => a + c + 2)))
+		}
+		return "```\n" + resultRows.join("\n") + "```"
+	}
+})
+
 /** @type {string[]} SPRITE SHEET EMOJIS FEATURE: mxc urls for the currently processing message */
 let endOfMessageEmojis = []
 turndownService.addRule("emoji", {
@@ -315,7 +338,7 @@ function getUserOrProxyOwnerID(mxid) {
  * At the time of this executing, we know what the end of message emojis are, and we know that at least one of them is unknown.
  * This function will strip them from the content and generate the correct pending file of the sprite sheet.
  * @param {string} content
- * @param {{id: string, name: string}[]} attachments
+ * @param {{id: string, filename: string}[]} attachments
  * @param {({name: string, mxc: string} | {name: string, mxc: string, key: string, iv: string} | {name: string, buffer: Buffer})[]} pendingFiles
  * @param {(mxc: string) => Promise<Buffer | undefined>} mxcDownloader function that will download the mxc URLs and convert to uncompressed PNG data. use `getAndConvertEmoji` or a mock.
  */
@@ -329,9 +352,9 @@ async function uploadEndOfMessageSpriteSheet(content, attachments, pendingFiles,
 	// Create a sprite sheet of known and unknown emojis from the end of the message
 	const buffer = await emojiSheet.compositeMatrixEmojis(endOfMessageEmojis, mxcDownloader)
 	// Attach it
-	const name = "emojis.png"
-	attachments.push({id: String(attachments.length), name})
-	pendingFiles.push({name, buffer})
+	const filename = "emojis.png"
+	attachments.push({id: String(attachments.length), filename})
+	pendingFiles.push({name: filename, buffer})
 	return content
 }
 
@@ -426,7 +449,7 @@ async function checkWrittenMentions(content, senderMxid, roomID, guild, di) {
 					allowedMentionsParse: ["everyone"]
 				}
 			}
-		} else {
+		} else if (writtenMentionMatch[1].length < 40) { // the API supports up to 100 characters, but really if you're searching more than 40, something messed up
 			const results = await di.snow.guild.searchGuildMembers(guild.id, {query: writtenMentionMatch[1]})
 			if (results[0]) {
 				assert(results[0].user)
@@ -458,6 +481,23 @@ const attachmentEmojis = new Map([
 	["m.file", "📄"]
 ])
 
+async function getL1L2ReplyLine(called = false) {
+	// @ts-ignore
+	const autoEmoji = new Map(select("auto_emoji", ["name", "emoji_id"], {}, "WHERE name = 'L1' OR name = 'L2'").raw().all())
+	if (autoEmoji.size === 2) {
+		return `<:L1:${autoEmoji.get("L1")}><:L2:${autoEmoji.get("L2")}>`
+	}
+	/* c8 ignore start */
+	if (called) {
+		// Don't know how this could happen, but just making sure we don't enter an infinite loop.
+		console.warn("Warning: OOYE is missing data to format replies. To fix this: `npm run setup`")
+		return ""
+	}
+	await setupEmojis.setupEmojis()
+	return getL1L2ReplyLine(true)
+	/* c8 ignore stop */
+}
+
 /**
  * @param {Ty.Event.Outer_M_Room_Message | Ty.Event.Outer_M_Room_Message_File | Ty.Event.Outer_M_Sticker | Ty.Event.Outer_M_Room_Message_Encrypted_File} event
  * @param {import("discord-api-types/v10").APIGuild} guild
@@ -486,6 +526,7 @@ async function eventToMessage(event, guild, di) {
 	}
 
 	let content = event.content.body // ultimate fallback
+	/** @type {{id: string, filename: string}[]} */
 	const attachments = []
 	/** @type {({name: string, mxc: string} | {name: string, mxc: string, key: string, iv: string} | {name: string, buffer: Buffer})[]} */
 	const pendingFiles = []
@@ -493,7 +534,45 @@ async function eventToMessage(event, guild, di) {
 	const ensureJoined = []
 
 	// Convert content depending on what the message is
-	if (event.type === "m.room.message" && (event.content.msgtype === "m.text" || event.content.msgtype === "m.emote")) {
+	// Handle images first - might need to handle their `body`/`formatted_body` as well, which will fall through to the text processor
+	let shouldProcessTextEvent = event.type === "m.room.message" && (event.content.msgtype === "m.text" || event.content.msgtype === "m.emote")
+	if (event.type === "m.room.message" && (event.content.msgtype === "m.file" || event.content.msgtype === "m.video" || event.content.msgtype === "m.audio" || event.content.msgtype === "m.image")) {
+		content = ""
+		const filename = event.content.filename || event.content.body
+		if ("file" in event.content) {
+			// Encrypted
+			assert.equal(event.content.file.key.alg, "A256CTR")
+			attachments.push({id: "0", filename})
+			pendingFiles.push({name: filename, mxc: event.content.file.url, key: event.content.file.key.k, iv: event.content.file.iv})
+		} else {
+			// Unencrypted
+			attachments.push({id: "0", filename})
+			pendingFiles.push({name: filename, mxc: event.content.url})
+		}
+		// Check if we also need to process a text event for this image - if it has a caption that's different from its filename
+		if ((event.content.body && event.content.filename && event.content.body !== event.content.filename) || event.content.formatted_body) {
+			shouldProcessTextEvent = true
+		}
+	}
+	if (event.type === "m.sticker") {
+		content = ""
+		let filename = event.content.body
+		if (event.type === "m.sticker") {
+			let mimetype
+			if (event.content.info?.mimetype?.includes("/")) {
+				mimetype = event.content.info.mimetype
+			} else {
+				const res = await di.api.getMedia(event.content.url, {method: "HEAD"})
+				if (res.status === 200) {
+					mimetype = res.headers.get("content-type")
+				}
+				if (!mimetype) throw new Error(`Server error ${res.status} or missing content-type while detecting sticker mimetype`)
+			}
+			filename += "." + mimetype.split("/")[1]
+		}
+		attachments.push({id: "0", filename})
+		pendingFiles.push({name: filename, mxc: event.content.url})
+	} else if (shouldProcessTextEvent) {
 		// Handling edits. If the edit was an edit of a reply, edits do not include the reply reference, so we need to fetch up to 2 more events.
 		// this event ---is an edit of--> original event ---is a reply to--> past event
 		await (async () => {
@@ -507,7 +586,7 @@ async function eventToMessage(event, guild, di) {
 			if (!messageIDsToEdit.length) return
 
 			// Ok, it's an edit.
-			event.content = event.content["m.new_content"]
+			event = {...event, content: event.content["m.new_content"]}
 
 			// Is it editing a reply? We need special handling if it is.
 			// Get the original event, then check if it was a reply
@@ -568,9 +647,7 @@ async function eventToMessage(event, guild, di) {
 				return
 			}
 
-			// @ts-ignore
-			const autoEmoji = new Map(select("auto_emoji", ["name", "emoji_id"], {}, "WHERE name = 'L1' OR name = 'L2'").raw().all())
-			replyLine = `<:L1:${autoEmoji.get("L1")}><:L2:${autoEmoji.get("L2")}>`
+			replyLine = await getL1L2ReplyLine()
 			const row = from("event_message").join("message_channel", "message_id").select("channel_id", "message_id").where({event_id: repliedToEventId}).and("ORDER BY part").get()
 			if (row) {
 				replyLine += `https://discord.com/channels/${guild.id}/${row.channel_id}/${row.message_id} `
@@ -780,40 +857,6 @@ async function eventToMessage(event, guild, di) {
 			// @ts-ignore bad type from turndown
 			content = turndownService.escape(content)
 		}
-	} else if (event.type === "m.room.message" && (event.content.msgtype === "m.file" || event.content.msgtype === "m.video" || event.content.msgtype === "m.audio" || event.content.msgtype === "m.image")) {
-		content = ""
-		const filename = event.content.filename || event.content.body
-		// A written `event.content.body` will be bridged to Discord's image `description` which is like alt text.
-		// Bridging as description rather than message content in order to match Matrix clients (Element, Neochat) which treat this as alt text or title text.
-		const description = (event.content.body !== event.content.filename && event.content.filename && event.content.body) || undefined
-		if ("url" in event.content) {
-			// Unencrypted
-			attachments.push({id: "0", description, filename})
-			pendingFiles.push({name: filename, mxc: event.content.url})
-		} else {
-			// Encrypted
-			assert.equal(event.content.file.key.alg, "A256CTR")
-			attachments.push({id: "0", description, filename})
-			pendingFiles.push({name: filename, mxc: event.content.file.url, key: event.content.file.key.k, iv: event.content.file.iv})
-		}
-	} else if (event.type === "m.sticker") {
-		content = ""
-		let filename = event.content.body
-		if (event.type === "m.sticker") {
-			let mimetype
-			if (event.content.info?.mimetype?.includes("/")) {
-				mimetype = event.content.info.mimetype
-			} else {
-				const res = await di.api.getMedia(event.content.url, {method: "HEAD"})
-				if (res.status === 200) {
-					mimetype = res.headers.get("content-type")
-				}
-				if (!mimetype) throw new Error(`Server error ${res.status} or missing content-type while detecting sticker mimetype`)
-			}
-			filename += "." + mimetype.split("/")[1]
-		}
-		attachments.push({id: "0", filename})
-		pendingFiles.push({name: filename, mxc: event.content.url})
 	}
 
 	content = displayNameRunoff + replyLine + content

src/m2d/converters/event-to-message.test.js

@@ -3770,7 +3770,7 @@ test("event2message: text attachments work", async t => {
 				username: "cadence [they]",
 				content: "",
 				avatar_url: "https://bridge.example.org/download/matrix/cadence.moe/azCAhThKTojXSZJRoWwZmhvU",
-				attachments: [{id: "0", description: undefined, filename: "chiki-powerups.txt"}],
+				attachments: [{id: "0", filename: "chiki-powerups.txt"}],
 				pendingFiles: [{name: "chiki-powerups.txt", mxc: "mxc://cadence.moe/zyThGlYQxvlvBVbVgKDDbiHH"}]
 			}]
 		}
@@ -3806,14 +3806,14 @@ test("event2message: image attachments work", async t => {
 				username: "cadence [they]",
 				content: "",
 				avatar_url: "https://bridge.example.org/download/matrix/cadence.moe/azCAhThKTojXSZJRoWwZmhvU",
-				attachments: [{id: "0", description: undefined, filename: "cool cat.png"}],
+				attachments: [{id: "0", filename: "cool cat.png"}],
 				pendingFiles: [{name: "cool cat.png", mxc: "mxc://cadence.moe/IvxVJFLEuksCNnbojdSIeEvn"}]
 			}]
 		}
 	)
 })
 
-test("event2message: image attachments can have a custom description", async t => {
+test("event2message: image attachments can have a plaintext caption", async t => {
 	t.deepEqual(
 		await eventToMessage({
 			type: "m.room.message",
@@ -3840,10 +3840,62 @@ test("event2message: image attachments can have a custom description", async t =
 			messagesToEdit: [],
 			messagesToSend: [{
 				username: "cadence [they]",
-				content: "",
+				content: "Cat emoji surrounded by pink hearts",
 				avatar_url: "https://bridge.example.org/download/matrix/cadence.moe/azCAhThKTojXSZJRoWwZmhvU",
-				attachments: [{id: "0", description: "Cat emoji surrounded by pink hearts", filename: "cool cat.png"}],
-				pendingFiles: [{name: "cool cat.png", mxc: "mxc://cadence.moe/IvxVJFLEuksCNnbojdSIeEvn"}]
+				attachments: [{id: "0", filename: "cool cat.png"}],
+				pendingFiles: [{name: "cool cat.png", mxc: "mxc://cadence.moe/IvxVJFLEuksCNnbojdSIeEvn"}],
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}]
+		}
+	)
+})
+
+test("event2message: image attachments can have a formatted caption", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			content: {
+				body: "this event has `formatting`",
+				filename: "5740.jpg",
+				format: "org.matrix.custom.html",
+				formatted_body: "this event has <code>formatting</code>",
+				info: {
+					h: 1340,
+					mimetype: "image/jpeg",
+					size: 226689,
+					thumbnail_info: {
+						h: 670,
+						mimetype: "image/jpeg",
+						size: 80157,
+						w: 540
+					},
+					thumbnail_url: "mxc://thomcat.rocks/XhLsOCDBYyearsLQgUUrbAvw",
+					w: 1080,
+					"xyz.amorgan.blurhash": "KHJQG*55ic-.}?0M58J.9v"
+				},
+				msgtype: "m.image",
+				url: "mxc://thomcat.rocks/RTHsXmcMPXmuHqVNsnbKtRbh"
+			},
+			origin_server_ts: 1740607766895,
+			sender: "@cadence:cadence.moe",
+			type: "m.room.message",
+			event_id: "$NqNqVgukiQm1nynm9vIr9FIq31hZpQ3udOd7cBIW46U",
+			room_id: "!BnKuBPCvyfOkhcUjEu:cadence.moe"
+		}),
+		{
+			ensureJoined: [],
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "this event has `formatting`",
+				avatar_url: "https://bridge.example.org/download/matrix/cadence.moe/azCAhThKTojXSZJRoWwZmhvU",
+				attachments: [{id: "0", filename: "5740.jpg"}],
+				pendingFiles: [{name: "5740.jpg", mxc: "mxc://thomcat.rocks/RTHsXmcMPXmuHqVNsnbKtRbh"}],
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
 			}]
 		}
 	)
@@ -3892,7 +3944,7 @@ test("event2message: encrypted image attachments work", async t => {
 				username: "cadence [they]",
 				content: "",
 				avatar_url: "https://bridge.example.org/download/matrix/cadence.moe/azCAhThKTojXSZJRoWwZmhvU",
-				attachments: [{id: "0", description: undefined, filename: "image.png"}],
+				attachments: [{id: "0", filename: "image.png"}],
 				pendingFiles: [{
 					name: "image.png",
 					mxc: "mxc://heyquark.com/LOGkUTlVFrqfiExlGZNgCJJX",
@@ -3904,6 +3956,91 @@ test("event2message: encrypted image attachments work", async t => {
 	)
 })
 
+test("event2message: evil encrypted image attachment works", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			sender: "@austin:tchncs.de",
+			type: "m.room.message",
+			content: {
+				body: "Screenshot 2025-06-29 at 13.36.46.png",
+				file: {
+					hashes: {
+						sha256: "Vh1apd8wSFu/BpUdQbIrKUzFB0Uu+l1octgZL+aVGTQ"
+					},
+					iv: "sd33K7pSZNMAAAAAAAAAAA",
+					key: {
+						alg: "A256CTR",
+						ext: true,
+						k: "-nyqk1eqI-g-ND59P9qHp310-Qyc2A5gSAYm1BxopSg",
+						key_ops: [
+							"encrypt",
+							"decrypt"
+						],
+						kty: "oct"
+					},
+					url: "mxc://tchncs.de/eac5f83fa97cd74062daf75dfa04d6e5356897281939377544214085632",
+					v: "v2"
+				},
+				info: {
+					h: 682,
+					mimetype: "image/png",
+					"org.matrix.msc4230.is_animated": false,
+					size: 1813154,
+					thumbnail_file: {
+						hashes: {
+							sha256: "o3xykQwfsTUf5Y8qP5fjT7qBv5lAT3rtkmPpise5eQw"
+						},
+						iv: "SNxIZsJkju4AAAAAAAAAAA",
+						key: {
+							alg: "A256CTR",
+							ext: true,
+							k: "CcibYjzzSDexOWBbcBh_kCDiLibg8vUZthz5CnxV0es",
+							key_ops: [
+								"encrypt",
+								"decrypt"
+							],
+							kty: "oct"
+						},
+						url: "mxc://tchncs.de/ecd811d913ed1b240ebfc81517a5de2c3a1e9d401939377537079574528",
+						v: "v2"
+					},
+					thumbnail_info: {
+						h: 600,
+						mimetype: "image/png",
+						size: 451773,
+						w: 507
+					},
+					thumbnail_url: null,
+					w: 577,
+					"xyz.amorgan.blurhash": "TqN1Ais=t1~qRjWFxURiWCM{ofof"
+				},
+				"m.mentions": {},
+				msgtype: "m.image",
+				url: null
+			},
+			event_id: "$UKMbzTlqlyLYN78utVEtiivABFvOe39nx5trHwqNmeQ",
+			room_id: "!iSyXgNxQcEuXoXpsSn:pussthecat.org"
+		}),
+		{
+			ensureJoined: [],
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "Austin Huang",
+				content: "",
+				avatar_url: "https://bridge.example.org/download/matrix/tchncs.de/090a2b5e07eed2f71e84edad5207221e6c8f8b8e",
+				attachments: [{id: "0", filename: "Screenshot 2025-06-29 at 13.36.46.png"}],
+				pendingFiles: [{
+					name: "Screenshot 2025-06-29 at 13.36.46.png",
+					mxc: "mxc://tchncs.de/eac5f83fa97cd74062daf75dfa04d6e5356897281939377544214085632",
+					key: "-nyqk1eqI-g-ND59P9qHp310-Qyc2A5gSAYm1BxopSg",
+					iv: "sd33K7pSZNMAAAAAAAAAAA"
+				}]
+			}]
+		}
+	)
+})
+
 test("event2message: stickers work", async t => {
 	t.deepEqual(
 		await eventToMessage({
@@ -4485,6 +4622,42 @@ test("event2message: @room in the middle of a link is not converted", async t =>
 	)
 })
 
+test("event2message: table", async t => {
+	t.deepEqual(
+		await eventToMessage({
+			type: "m.room.message",
+			sender: "@cadence:cadence.moe",
+			content: {
+				msgtype: "m.text",
+				body: "wrong body",
+				format: "org.matrix.custom.html",
+				formatted_body: "content<table><thead><tr><th>Col 1</th><th>Col 2</th><th>Col 3</th></tr></thead><tbody><tr><th>Apple</th><td>Banana</td><td>Cherry</td></tr><tr><th>Aardvark</th><td>Bee</td><td>Crocodile</td></tr><tr><td>Argon</td><td>Boron</td><td>Carbon</td></tr></tbody></table>more content"
+			},
+			room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe",
+			event_id: "$SiXetU9h9Dg-M9Frcw_C6ahnoXZ3QPZe3MVJR5tcB9A"
+		}),
+		{
+			messagesToDelete: [],
+			messagesToEdit: [],
+			messagesToSend: [{
+				username: "cadence [they]",
+				content: "content```"
+				+ "\nCol 1     Col 2   Col 3  "
+				+ "\n---------------------------"
+				+ "\nApple     Banana  Cherry "
+				+ "\nAardvark  Bee     Crocodile"
+				+ "\nArgon     Boron   Carbon   ```"
+				+ "more content",
+				avatar_url: undefined,
+				allowed_mentions: {
+					parse: ["users", "roles"]
+				}
+			}],
+			ensureJoined: []
+		}
+	)
+})
+
 slow()("event2message: unknown emoji at the end is reuploaded as a sprite sheet", async t => {
 	const messages = await eventToMessage({
 		type: "m.room.message",

src/m2d/event-dispatcher.js

@@ -28,51 +28,126 @@ const {reg} = require("../matrix/read-registration")
 
 let lastReportedEvent = 0
 
-function guard(type, fn) {
-	return async function(event, ...args) {
-		try {
-			return await fn(event, ...args)
-		} catch (e) {
-			console.error(`Exception while processing a ${type} Matrix event:`)
-			console.dir(event, {depth: null})
+/**
+ * This function is adapted from Evan Kaufman's fantastic work.
+ * The original function and my adapted function are both MIT licensed.
+ * @url https://github.com/EvanK/npm-loggable-error/
+ * @param {number} [depth]
+ * @returns {string}
+*/
+function stringifyErrorStack(err, depth = 0) {
+	let collapsed = " ".repeat(depth);
+	if (!(err instanceof Error)) {
+		return collapsed + err
+	}
 
-			if (Date.now() - lastReportedEvent < 5000) return
+	// add full stack trace if one exists, otherwise convert to string
+	let stackLines = String(err?.stack ?? err).replace(/^/gm, " ".repeat(depth)).trim().split("\n")
+	let cloudstormLine = stackLines.findIndex(l => l.includes("/node_modules/cloudstorm/"))
+	if (cloudstormLine !== -1) {
+		stackLines = stackLines.slice(0, cloudstormLine - 2)
+	}
+	collapsed += stackLines.join("\n")
+
+	const props = Object.getOwnPropertyNames(err).filter(p => !["message", "stack"].includes(p))
+
+	// only break into object notation if we have additional props to dump
+	if (props.length) {
+		const dedent = " ".repeat(depth);
+		const indent = " ".repeat(depth + 2);
+
+		collapsed += " {\n";
+
+		// loop and print each (indented) prop name
+		for (let property of props) {
+			collapsed += `${indent}[${property}]: `;
+
+			// if another error object, stringify it too
+			if (err[property] instanceof Error) {
+				collapsed += stringifyErrorStack(err[property], depth + 2).trimStart();
+			}
+			// otherwise stringify as JSON
+			else {
+				collapsed += JSON.stringify(err[property]);
+			}
+
+			collapsed += "\n";
+		}
+
+		collapsed += `${dedent}}\n`;
+	}
+
+	return collapsed;
+}
+
+/**
+ * @param {string} roomID
+ * @param {"Discord" | "Matrix"} source
+ * @param {any} type
+ * @param {any} e
+ * @param {any} payload
+ */
+async function sendError(roomID, source, type, e, payload) {
+	console.error(`Error while processing a ${type} ${source} event:`)
+	console.error(e)
+	console.dir(payload, {depth: null})
+
+	if (Date.now() - lastReportedEvent < 5000) return null
 	lastReportedEvent = Date.now()
 
-			const cloudflareErrorTitle = e.toString().match(/<!DOCTYPE html>.*?<title>discord\.com \| ([^<]*)<\/title>/s)?.[1]
-			if (cloudflareErrorTitle) {
-				return api.sendEvent(event.room_id, "m.room.message", {
-					msgtype: "m.text",
-					body: `\u26a0 Matrix event not delivered to Discord. Cloudflare error: ${cloudflareErrorTitle}.`,
-					format: "org.matrix.custom.html",
-					formatted_body: `\u26a0 <strong>Matrix event not delivered to Discord</strong><br>Cloudflare error: ${cloudflareErrorTitle}`,
-					"moe.cadence.ooye.error": {
-						source: "matrix",
-						payload: event
-					}
-				})
+	let errorIntroLine = e.toString()
+	if (e.cause) {
+		errorIntroLine += ` (cause: ${e.cause})`
 	}
 
-			let stackLines = e.stack.split("\n")
-			api.sendEvent(event.room_id, "m.room.message", {
-				msgtype: "m.text",
-				body: "\u26a0 Matrix event not delivered to Discord. See formatted content for full details.",
-				format: "org.matrix.custom.html",
-				formatted_body: "\u26a0 <strong>Matrix event not delivered to Discord</strong>"
-					+ `<br>Event type: ${type}`
-					+ `<br>${e.toString()}`
-					+ `<br><details><summary>Error trace</summary>`
-					+ `<pre>${stackLines.join("\n")}</pre></details>`
-					+ `<details><summary>Original payload</summary>`
-					+ `<pre>${util.inspect(event, false, 4, false)}</pre></details>`,
+	const builder = new utils.MatrixStringBuilder()
+
+	const cloudflareErrorTitle = errorIntroLine.match(/<!DOCTYPE html>.*?<title>discord\.com \| ([^<]*)<\/title>/s)?.[1]
+	if (cloudflareErrorTitle) {
+		builder.addLine(
+			`\u26a0 Matrix event not delivered to Discord. Discord might be down right now. Cloudflare error: ${cloudflareErrorTitle}`,
+			`\u26a0 <strong>Matrix event not delivered to Discord</strong><br>Discord might be down right now. Cloudflare error: ${cloudflareErrorTitle}`
+		)
+	} else {
+		// What
+		const what = source === "Discord" ? "Bridged event from Discord not delivered" : "Matrix event not delivered to Discord"
+		builder.addLine(`\u26a0 ${what}`, `\u26a0 <strong>${what}</strong>`)
+
+		// Who
+		builder.addLine(`Event type: ${type}`)
+
+		// Why
+		builder.addLine(errorIntroLine)
+
+		// Where
+		const stack = stringifyErrorStack(e)
+		builder.addLine(`Error trace:\n${stack}`, `<details><summary>Error trace</summary><pre>${stack}</pre></details>`)
+
+		// How
+		builder.addLine("", `<details><summary>Original payload</summary><pre>${util.inspect(payload, false, 4, false)}</pre></details>`)
+	}
+
+	// Send
+	try {
+		await api.sendEvent(roomID, "m.room.message", {
+			...builder.get(),
 			"moe.cadence.ooye.error": {
-					source: "matrix",
-					payload: event
+				source: source.toLowerCase(),
+				payload
 			},
 			"m.mentions": {
 				user_ids: ["@cadence:cadence.moe"]
 			}
 		})
+	} catch (e) {}
+}
+
+function guard(type, fn) {
+	return async function(event, ...args) {
+		try {
+			return await fn(event, ...args)
+		} catch (e) {
+			await sendError(event.room_id, "Matrix", type, e, event)
 		}
 	}
 }
@@ -104,7 +179,7 @@ async function onRetryReactionAdd(reactionEvent) {
 	}
 
 	// Redact the error to stop people from executing multiple retries
-	api.redactEvent(roomID, event.event_id)
+	await api.redactEvent(roomID, event.event_id)
 }
 
 sync.addTemporaryListener(as, "type:m.room.message", guard("m.room.message",
@@ -114,6 +189,7 @@ sync.addTemporaryListener(as, "type:m.room.message", guard("m.room.message",
 async event => {
 	if (utils.eventSenderIsFromDiscord(event.sender)) return
 	const messageResponses = await sendEvent.sendEvent(event)
+	if (!messageResponses.length) return
 	if (event.type === "m.room.message" && event.content.msgtype === "m.text") {
 		// @ts-ignore
 		await matrixCommandHandler.execute(event)
@@ -263,7 +339,13 @@ async event => {
 
 	if (event.content.membership === "leave" || event.content.membership === "ban") {
 		// Member is gone
-		return db.prepare("DELETE FROM member_cache WHERE room_id = ? and mxid = ?").run(event.room_id, event.state_key)
+		db.prepare("DELETE FROM member_cache WHERE room_id = ? and mxid = ?").run(event.room_id, event.state_key)
+
+		// Unregister room's use as a direct chat if the bot itself left
+		const bot = `@${reg.sender_localpart}:${reg.ooye.server_name}`
+		if (event.state_key === bot) {
+			db.prepare("DELETE FROM direct WHERE room_id = ?").run(event.room_id)
+		}
 	}
 
 	const exists = select("channel_room", "room_id", {room_id: event.room_id}) ?? select("guild_space", "space_id", {space_id: event.room_id})
@@ -297,3 +379,6 @@ async event => {
 		db.prepare("UPDATE member_cache SET power_level = ? WHERE room_id = ? AND mxid = ?").run(newPower[mxid] || 0, event.room_id, mxid)
 	}
 }))
+
+module.exports.stringifyErrorStack = stringifyErrorStack
+module.exports.sendError = sendError

src/m2d/event-dispatcher.test.js

@@ -0,0 +1,23 @@
+// @ts-check
+
+const {test} = require("supertape")
+const {stringifyErrorStack} = require("./event-dispatcher")
+
+test("stringify error stack: works", t => {
+	function a() {
+		const e = new Error("message", {cause: new Error("inner")})
+		// @ts-ignore
+		e.prop = 2.1
+		throw e
+	}
+	try {
+		a()
+		t.fail("shouldn't get here")
+	} catch (e) {
+		const str = stringifyErrorStack(e)
+		t.match(str, /^Error: message$/m)
+		t.match(str, /^    at a \(.*event-dispatcher\.test\.js/m)
+		t.match(str, /^  \[cause\]: Error: inner$/m)
+		t.match(str, /^  \[prop\]: 2.1$/m)
+	}
+})

src/matrix/api.js

@@ -181,6 +181,23 @@ async function getFullHierarchy(roomID) {
 	return rooms
 }
 
+/**
+ * Like `getFullHierarchy` but reveals a page at a time through an async iterator.
+ * @param {string} roomID
+ */
+async function* generateFullHierarchy(roomID) {
+	/** @type {string | undefined} */
+	let nextBatch = undefined
+	do {
+		/** @type {Ty.HierarchyPagination<Ty.R.Hierarchy>} */
+		const res = await getHierarchy(roomID, {from: nextBatch})
+		for (const room of res.rooms) {
+			yield room
+		}
+		nextBatch = res.next_batch
+	} while (nextBatch)
+}
+
 /**
  * @param {string} roomID
  * @param {string} eventID
@@ -291,21 +308,33 @@ async function profileSetAvatarUrl(mxid, avatar_url) {
  * Set a user's power level within a room.
  * @param {string} roomID
  * @param {string} mxid
- * @param {number} power
+ * @param {number} newPower
  */
-async function setUserPower(roomID, mxid, power) {
+async function setUserPower(roomID, mxid, newPower) {
 	assert(roomID[0] === "!")
 	assert(mxid[0] === "@")
 	// Yes there's no shortcut https://github.com/matrix-org/matrix-appservice-bridge/blob/2334b0bae28a285a767fe7244dad59f5a5963037/src/components/intent.ts#L352
-	const powerLevels = await getStateEvent(roomID, "m.room.power_levels", "")
-	powerLevels.users = powerLevels.users || {}
-	if (power != null) {
-		powerLevels.users[mxid] = power
+	const power = await getStateEvent(roomID, "m.room.power_levels", "")
+	power.users = power.users || {}
+
+	// Check if it has really changed to avoid sending a useless state event
+	// (Can't diff kstate here because of (a) circular imports (b) kstate has special behaviour diffing power levels)
+	const oldPowerLevel = power.users?.[mxid] ?? power.users_default ?? 0
+	if (oldPowerLevel === newPower) return
+
+	// Bridge bot can't demote equal power users, so need to decide which user will send the event
+	const botPowerLevel = power.users?.[`@${reg.sender_localpart}:${reg.ooye.server_name}`] ?? power.users_default ?? 0
+	const eventSender = oldPowerLevel >= botPowerLevel ? mxid : undefined
+
+	// Update the event content
+	if (newPower == null || newPower === (power.users_default ?? 0)) {
+		delete power.users[mxid]
 	} else {
-		delete powerLevels.users[mxid]
+		power.users[mxid] = newPower
 	}
-	await sendState(roomID, "m.room.power_levels", "", powerLevels)
-	return powerLevels
+
+	await sendState(roomID, "m.room.power_levels", "", power, eventSender)
+	return power
 }
 
 /**
@@ -419,6 +448,14 @@ async function setPresence(data, mxid) {
 	await mreq.mreq("PUT", path(`/client/v3/presence/${mxid}/status`, mxid), data)
 }
 
+/**
+ * @param {string} mxid
+ * @returns {Promise<{displayname?: string, avatar_url?: string}>}
+ */
+function getProfile(mxid) {
+	return mreq.mreq("GET", `/client/v3/profile/${mxid}`)
+}
+
 module.exports.path = path
 module.exports.register = register
 module.exports.createRoom = createRoom
@@ -434,6 +471,7 @@ module.exports.getJoinedMembers = getJoinedMembers
 module.exports.getMembers = getMembers
 module.exports.getHierarchy = getHierarchy
 module.exports.getFullHierarchy = getFullHierarchy
+module.exports.generateFullHierarchy = generateFullHierarchy
 module.exports.getRelations = getRelations
 module.exports.getFullRelations = getFullRelations
 module.exports.sendState = sendState
@@ -452,3 +490,4 @@ module.exports.getAlias = getAlias
 module.exports.getAccountData = getAccountData
 module.exports.setAccountData = setAccountData
 module.exports.setPresence = setPresence
+module.exports.getProfile = getProfile

src/matrix/file.js

@@ -1,6 +1,9 @@
 // @ts-check
 
 const passthrough = require("../passthrough")
+const {reg, writeRegistration} = require("./read-registration.js")
+const Ty = require("../types")
+
 const {sync, db, select} = passthrough
 /** @type {import("./mreq")} */
 const mreq = sync.require("./mreq")
@@ -44,11 +47,8 @@ async function uploadDiscordFileToMxc(path) {
 		return existingFromDb
 	}
 
-	// Download from Discord
-	const promise = fetch(url, {}).then(async res => {
-		// Upload to Matrix
-		const root = await module.exports._actuallyUploadDiscordFileToMxc(urlNoExpiry, res)
-
+	// Download from Discord and upload to Matrix
+	const promise = module.exports._actuallyUploadDiscordFileToMxc(url).then(root => {
 		// Store relationship in database
 		db.prepare("INSERT INTO file (discord_url, mxc_url) VALUES (?, ?)").run(urlNoExpiry, root.content_uri)
 		inflight.delete(urlNoExpiry)
@@ -62,17 +62,31 @@ async function uploadDiscordFileToMxc(path) {
 
 /**
  * @param {string} url
- * @param {Response} res
+ * @returns {Promise<Ty.R.FileUploaded>}
  */
-async function _actuallyUploadDiscordFileToMxc(url, res) {
-	const body = res.body
-	/** @type {import("../types").R.FileUploaded} */
-	const root = await mreq.mreq("POST", "/media/v3/upload", body, {
+async function _actuallyUploadDiscordFileToMxc(url) {
+	const res = await fetch(url, {})
+	try {
+		/** @type {Ty.R.FileUploaded} */
+		const root = await mreq.mreq("POST", "/media/v3/upload", res.body, {
 			headers: {
 				"Content-Type": res.headers.get("content-type")
 			}
 		})
 		return root
+	} catch (e) {
+		if (e instanceof mreq.MatrixServerError && e.data.error?.includes("Content-Length") && !reg.ooye.content_length_workaround) {
+			reg.ooye.content_length_workaround = true
+			const root = await _actuallyUploadDiscordFileToMxc(url)
+			console.error("OOYE cannot stream uploads to Synapse. The `content_length_workaround` option"
+				+ "\nhas been activated in registration.yaml, which works around the problem, but"
+				+ "\nhalves the speed of bridging d->m files. A better way to resolve this problem"
+				+ "\nis to run an nginx reverse proxy to Synapse and re-run OOYE setup.")
+			writeRegistration(reg)
+			return root
+		}
+		throw e
+	}
 }
 
 function guildIcon(guild) {
@@ -84,8 +98,8 @@ function userAvatar(user) {
 }
 
 function memberAvatar(guildID, user, member) {
-	if (!member.avatar) return userAvatar(user)
-	return `/guilds/${guildID}/users/${user.id}/avatars/${member.avatar}.png?size=${IMAGE_SIZE}`
+	if (!member?.avatar) return userAvatar(user)
+	return `/guilds/${guildID}/users/${user.id}/avatars/${member?.avatar}.png?size=${IMAGE_SIZE}`
 }
 
 function emoji(emojiID, animated) {
@@ -95,18 +109,17 @@ function emoji(emojiID, animated) {
 }
 
 const stickerFormat = new Map([
-	[1, {label: "PNG", ext: "png", mime: "image/png"}],
-	[2, {label: "APNG", ext: "png", mime: "image/apng"}],
-	[3, {label: "LOTTIE", ext: "json", mime: "lottie"}],
-	[4, {label: "GIF", ext: "gif", mime: "image/gif"}]
+	[1, {label: "PNG", ext: "png", mime: "image/png", endpoint: "/stickers/"}],
+	[2, {label: "APNG", ext: "png", mime: "image/apng", endpoint: "/stickers/"}],
+	[3, {label: "LOTTIE", ext: "json", mime: "lottie", endpoint: "/stickers/"}],
+	[4, {label: "GIF", ext: "gif", mime: "image/gif", endpoint: "https://media.discordapp.net/stickers/"}]
 ])
 
 /** @param {{id: string, format_type: number}} sticker */
 function sticker(sticker) {
 	const format = stickerFormat.get(sticker.format_type)
 	if (!format) throw new Error(`No such format ${sticker.format_type} for sticker ${JSON.stringify(sticker)}`)
-	const ext = format.ext
-	return `/stickers/${sticker.id}.${ext}`
+	return `${format.endpoint}${sticker.id}.${format.ext}`
 }
 
 module.exports.DISCORD_IMAGES_BASE = DISCORD_IMAGES_BASE

src/matrix/matrix-command-handler.js

@@ -224,7 +224,7 @@ const commands = [{
 							.png()
 							.toBuffer({resolveWithObject: true})
 						console.log(`uploading emoji ${resizeOutput.data.length} bytes to :${e.name}:`)
-						const emoji = await discord.snow.guildAssets.createEmoji(guildID, {name: e.name, image: "data:image/png;base64," + resizeOutput.data.toString("base64")})
+						await discord.snow.assets.createGuildEmoji(guildID, {name: e.name, image: "data:image/png;base64," + resizeOutput.data.toString("base64")})
 					}
 					api.sendEvent(event.room_id, "m.room.message", {
 						...ctx,

src/matrix/mreq.js

@@ -1,12 +1,11 @@
 // @ts-check
 
-const mixin = require("@cloudrac3r/mixin-deep")
 const stream = require("stream")
 const streamWeb = require("stream/web")
-const getStream = require("get-stream")
-
-const {reg, writeRegistration} = require("./read-registration.js")
+const {buffer} = require("stream/consumers")
+const mixin = require("@cloudrac3r/mixin-deep")
 
+const {reg} = require("./read-registration.js")
 const baseUrl = `${reg.ooye.server_origin}/_matrix`
 
 class MatrixServerError extends Error {
@@ -19,20 +18,33 @@ class MatrixServerError extends Error {
 	}
 }
 
+/**
+ * @param {undefined | string | object | streamWeb.ReadableStream | stream.Readable} body
+ * @returns {Promise<string | streamWeb.ReadableStream | stream.Readable | Buffer>}
+ */
+async function _convertBody(body) {
+	if (body == undefined || Object.is(body.constructor, Object)) {
+		return JSON.stringify(body) // almost every POST request is going to follow this one
+	} else if (body instanceof stream.Readable && reg.ooye.content_length_workaround) {
+		return await buffer(body) // content length workaround is set, so convert to buffer. the buffer consumer accepts node streams.
+	} else if (body instanceof stream.Readable) {
+		return stream.Readable.toWeb(body) // native fetch can only consume web streams
+	} else if (body instanceof streamWeb.ReadableStream && reg.ooye.content_length_workaround) {
+		return await buffer(body) // content lenght workaround is set, so convert to buffer. the buffer consumer accepts async iterables, which web streams are.
+	}
+	return body
+}
+
+/* c8 ignore start */
+
 /**
  * @param {string} method
  * @param {string} url
- * @param {string | object | streamWeb.ReadableStream | stream.Readable} [body]
+ * @param {string | object | streamWeb.ReadableStream | stream.Readable} [bodyIn]
  * @param {any} [extra]
  */
-async function mreq(method, url, body, extra = {}) {
-	if (body == undefined || Object.is(body.constructor, Object)) {
-		body = JSON.stringify(body)
-	} else if (body instanceof stream.Readable && reg.ooye.content_length_workaround) {
-		body = await getStream.buffer(body)
-	} else if (body instanceof streamWeb.ReadableStream && reg.ooye.content_length_workaround) {
-		body = await stream.consumers.buffer(stream.Readable.fromWeb(body))
-	}
+async function mreq(method, url, bodyIn, extra = {}) {
+	const body = await _convertBody(bodyIn)
 
 	/** @type {RequestInit} */
 	const opts = mixin({
@@ -43,21 +55,11 @@ async function mreq(method, url, body, extra = {}) {
 		},
 		...(body && {duplex: "half"}), // https://github.com/octokit/request.js/pull/571/files
 	}, extra)
-	// console.log(baseUrl + url, opts)
+
 	const res = await fetch(baseUrl + url, opts)
 	const root = await res.json()
 
 	if (!res.ok || root.errcode) {
-		if (root.error?.includes("Content-Length") && !reg.ooye.content_length_workaround) {
-			reg.ooye.content_length_workaround = true
-			const root = await mreq(method, url, body, extra)
-			console.error("OOYE cannot stream uploads to Synapse. The `content_length_workaround` option"
-				+ "\nhas been activated in registration.yaml, which works around the problem, but"
-				+ "\nhalves the speed of bridging d->m files. A better way to resolve this problem"
-				+ "\nis to run an nginx reverse proxy to Synapse and re-run OOYE setup.")
-			writeRegistration(reg)
-			return root
-		}
 		delete opts.headers?.["Authorization"]
 		throw new MatrixServerError(root, {baseUrl, url, ...opts})
 	}
@@ -86,3 +88,4 @@ module.exports.MatrixServerError = MatrixServerError
 module.exports.baseUrl = baseUrl
 module.exports.mreq = mreq
 module.exports.withAccessToken = withAccessToken
+module.exports._convertBody = _convertBody

src/matrix/mreq.test.js

@@ -0,0 +1,47 @@
+// @ts-check
+
+const assert = require("assert")
+const stream = require("stream")
+const streamWeb = require("stream/web")
+const {buffer} = require("stream/consumers")
+const {test} = require("supertape")
+const {_convertBody} = require("./mreq")
+const {reg} = require("./read-registration")
+
+async function *generator() {
+	yield "a"
+	yield "b"
+}
+
+reg.ooye.content_length_workaround = false
+
+test("convert body: converts object to string", async t => {
+	t.equal(await _convertBody({a: "1"}), `{"a":"1"}`)
+})
+
+test("convert body: leaves undefined as undefined", async t => {
+	t.equal(await _convertBody(undefined), undefined)
+})
+
+test("convert body: leaves web readable as web readable", async t => {
+	const webReadable = stream.Readable.toWeb(stream.Readable.from(generator()))
+	t.equal(await _convertBody(webReadable), webReadable)
+})
+
+test("convert body: converts node readable to web readable (for native fetch upload)", async t => {
+	const readable = stream.Readable.from(generator())
+	const webReadable = await _convertBody(readable)
+	assert(webReadable instanceof streamWeb.ReadableStream)
+	t.deepEqual(await buffer(webReadable), Buffer.from("ab"))
+})
+
+test("convert body: converts node readable to buffer", async t => {
+	reg.ooye.content_length_workaround = true
+	const readable = stream.Readable.from(generator())
+	t.deepEqual(await _convertBody(readable), Buffer.from("ab"))
+})
+
+test("convert body: converts web readable to buffer", async t => {
+	const webReadable = stream.Readable.toWeb(stream.Readable.from(generator()))
+	t.deepEqual(await _convertBody(webReadable), Buffer.from("ab"))
+})

src/types.d.ts

@@ -29,7 +29,8 @@ export type AppServiceRegistrationConfig = {
 		include_user_id_in_mxid: boolean
 		invite: string[]
 		discord_origin?: string
-		discord_cdn_origin?: string
+		discord_cdn_origin?: string,
+		web_password: string
 	}
 	old_bridge?: {
 		as_token: string
@@ -166,6 +167,8 @@ export namespace Event {
 	export type M_Room_Message_File = {
 		msgtype: "m.file" | "m.image" | "m.video" | "m.audio"
 		body: string
+		format?: "org.matrix.custom.html"
+		formatted_body?: string
 		filename?: string
 		url: string
 		info?: any
@@ -183,6 +186,8 @@ export namespace Event {
 	export type M_Room_Message_Encrypted_File = {
 		msgtype: "m.file" | "m.image" | "m.video" | "m.audio"
 		body: string
+		format?: "org.matrix.custom.html"
+		formatted_body?: string
 		filename?: string
 		file: {
 			url: string

src/web/auth.js

@@ -23,10 +23,10 @@ async function getManagedGuilds(event) {
 
 /**
  * @param {h3.H3Event} event
- * @returns {ReturnType<typeof h3.useSession<{userID?: string, mxid?: string, managedGuilds?: string[], state?: string, selfService?: boolean}>>}
+ * @returns {ReturnType<typeof h3.useSession<{userID?: string, mxid?: string, managedGuilds?: string[], state?: string, selfService?: boolean, password?: string}>>}
  */
 function useSession(event) {
-	return h3.useSession(event, {password: reg.as_token})
+	return h3.useSession(event, {password: reg.as_token, maxAge: 365 * 24 * 60 * 60})
 }
 
 module.exports.getManagedGuilds = getManagedGuilds

src/web/pug/guild.pug

@@ -54,13 +54,13 @@ block body
   .s-page-title.mb24
     h1.s-page-title--header= guild.name
 
-  .d-flex.g16
+  .d-flex.g16(class="sm:fw-wrap")
     .fl-grow1
       h2.fs-headline1 Invite a Matrix user
 
-      form.d-grid.g-af-column.gy4.gx8.jc-start(method="post" action="/api/invite" hx-post="/api/invite" hx-trigger="submit" hx-swap="none" hx-on::after-request="if (event.detail.successful) this.reset()" hx-disabled-elt="input, button" hx-indicator="#invite-button")
+      form.d-grid.g-af-column.gy4.gx8.jc-start(method="post" action=rel("/api/invite") hx-post=rel("/api/invite") hx-trigger="submit" hx-swap="none" hx-on::after-request="if (event.detail.successful) this.reset()" hx-disabled-elt="input, button" hx-indicator="#invite-button")
         label.s-label(for="mxid") Matrix ID
-        input.fl-grow1.s-input.wmx3#mxid(name="mxid" required placeholder="@user:example.org" pattern="@([^:]+):([a-z0-9:\-]+\.[a-z0-9.:\-]+)")
+        input.fl-grow1.s-input.wmx3#mxid(name="mxid" required placeholder="@user:example.org" pattern="@([^:]+):([a-z0-9:\\-]+\\.[a-z0-9.:\\-]+)")
         label.s-label(for="permissions") Permissions
         .s-select
           select#permissions(name="permissions")
@@ -71,70 +71,16 @@ block body
         .grid--row-start2
           button.s-btn.s-btn__filled#invite-button Invite
     div
-      .s-card.d-flex.ai-center.jc-center(style="min-width: 130px; min-height: 130px;")
-        button.s-btn.s-btn__filled(hx-get=`/qr?guild_id=${guild_id}` hx-indicator="closest button" hx-swap="outerHTML" hx-disabled-elt="this") Show QR
+      .s-card.d-flex.ai-center.jc-center(style="min-width: 132px; min-height: 132px;")
+        button.s-btn(class=space_id ? "s-btn__muted" : "s-btn__filled" hx-get=rel(`/qr?guild_id=${guild_id}`) hx-indicator="closest button" hx-swap="outerHTML" hx-disabled-elt="this") Show QR
 
   if space_id
-    h2.mt48.fs-headline1 Matrix setup
-
-    h3.mt32.fs-category Linked channels
-
-    .s-card.bs-sm.p0
-      form.s-table-container(method="post" action="/api/unlink" hx-confirm="Do you want to unlink these channels?\nIt may take a moment to clean up Matrix resources.")
-        input(type="hidden" name="guild_id" value=guild_id)
-        table.s-table.s-table__bx-simple
-          each row in linkedChannelsWithDetails
-            tr
-              td.w40: +discord(row.channel)
-              td.p2: button.s-btn.s-btn__muted.s-btn__xs(name="channel_id" value=row.channel.id hx-post="/api/unlink" hx-trigger="click" hx-disabled-elt="this")!= icons.Icons.IconLinkSm
-              td: +matrix(row)
-          else
-            tr
-              td(colspan="3")
-                .s-empty-state No channels linked between Discord and Matrix yet...
-
-  h3.mt32.fs-category Auto-create
-  .s-card
-    form.d-flex.ai-center.g8
-      label.s-label.fl-grow1(for="autocreate")
-        | Create new Matrix rooms automatically
-        p.s-description If you want, OOYE can automatically create new Matrix rooms and link them when an unlinked Discord channel is spoken in.
-      - let value = !!select("guild_active", "autocreate", {guild_id}).pluck().get()
-      input(type="hidden" name="guild_id" value=guild_id)
-      input.s-toggle-switch.order-last#autocreate(name="autocreate" type="checkbox" hx-post="/api/autocreate" hx-indicator="#autocreate-loading" hx-disabled-elt="this" checked=value autocomplete="off")
-      #autocreate-loading
-
-  if space_id
-    h3.mt32.fs-category URL preview
-    .s-card
-      form.d-flex.ai-center.g8
-        label.s-label.fl-grow1(for="url-preview")
-          | Show Discord's URL previews on Matrix
-          p.s-description Shows info about links posted to chat. Discord's previews are generally better quality than Synapse's, especially for social media and videos.
-        - let value = !!select("guild_space", "url_preview", {guild_id}).pluck().get()
-        input(type="hidden" name="guild_id" value=guild_id)
-        input.s-toggle-switch.order-last#autocreate(name="url_preview" type="checkbox" hx-post="/api/url-preview" hx-indicator="#url-preview-loading" hx-disabled-elt="this" checked=value autocomplete="off")
-        #url-preview-loading
-
-    h3.mt32.fs-category Presence
-    .s-card
-      form.d-flex.ai-center.g8
-        label.s-label.fl-grow1(for="presence")
-          | Show online statuses on Matrix
-          p.s-description This might cause lag on really big Discord servers.
-        - value = !!select("guild_space", "presence", {guild_id}).pluck().get()
-        input(type="hidden" name="guild_id" value=guild_id)
-        input.s-toggle-switch.order-last#autocreate(name="presence" type="checkbox" hx-post="/api/presence" hx-indicator="#presence-loading" hx-disabled-elt="this" checked=value autocomplete="off")
-        #presence-loading
-
     h3.mt32.fs-category Privacy level
-    .s-card
-      form(hx-post="/api/privacy-level" hx-trigger="change" hx-indicator="#privacy-level-loading" hx-disabled-elt="input")
-        input(type="hidden" name="guild_id" value=guild_id)
-        .d-flex.ai-center.mb4
-          label.s-label.fl-grow1
-            | How people can join on Matrix
       span#privacy-level-loading
+    .s-card
+      form(hx-post=rel("/api/privacy-level") hx-trigger="change" hx-indicator="#privacy-level-loading" hx-disabled-elt="input")
+        input(type="hidden" name="guild_id" value=guild_id)
+
         .s-toggle-switch.s-toggle-switch__multiple.s-toggle-switch__incremental.d-grid.gx16.ai-center(style="grid-template-columns: auto 1fr")
           input(type="radio" name="privacy_level" value="directory" id="privacy-level-directory" checked=(privacy_level === 2))
           label.d-flex.gx8.jc-center.grid--row-start3(for="privacy-level-directory")
@@ -158,8 +104,58 @@ block body
           p.s-description.m0 Shareable invite links, like Discord
           p.s-description.m0 Publicly listed in directory, like Discord server discovery
 
+    h2.mt48.fs-headline1 Features
+    .s-card.d-grid.px0.g16
+      form.d-flex.ai-center.g16
+        #url-preview-loading.p8
+        - let value = !!select("guild_space", "url_preview", {guild_id}).pluck().get()
+        input(type="hidden" name="guild_id" value=guild_id)
+        input.s-toggle-switch#url-preview(name="url_preview" type="checkbox" hx-post=rel("/api/url-preview") hx-indicator="#url-preview-loading" hx-disabled-elt="this" checked=value autocomplete="off")
+        label.s-label.fl-grow1(for="url-preview")
+          | Show Discord's URL previews on Matrix
+          p.s-description Shows info about links posted to chat. Discord's previews are generally better quality than Synapse's, especially for social media and videos.
+
+      form.d-flex.ai-center.g16
+        #presence-loading.p8
+        - value = !!select("guild_space", "presence", {guild_id}).pluck().get()
+        input(type="hidden" name="guild_id" value=guild_id)
+        input.s-toggle-switch#presence(name="presence" type="checkbox" hx-post=rel("/api/presence") hx-indicator="#presence-loading" hx-disabled-elt="this" checked=value autocomplete="off")
+        label.s-label(for="presence")
+          | Show online statuses on Matrix
+          p.s-description This might cause lag on really big Discord servers.
+
+  if space_id
+    h2.mt48.fs-headline1 Channel setup
+
+    h3.mt32.fs-category Linked channels
+    .s-card.bs-sm.p0
+      form.s-table-container(method="post" action=rel("/api/unlink") hx-confirm="Do you want to unlink these channels?\nIt may take a moment to clean up Matrix resources.")
+        input(type="hidden" name="guild_id" value=guild_id)
+        table.s-table.s-table__bx-simple
+          each row in linkedChannelsWithDetails
+            tr
+              td.w40: +discord(row.channel)
+              td.p2: button.s-btn.s-btn__muted.s-btn__xs(name="channel_id" value=row.channel.id hx-post=rel("/api/unlink") hx-trigger="click" hx-disabled-elt="this")!= icons.Icons.IconLinkSm
+              td: +matrix(row)
+          else
+            tr
+              td(colspan="3")
+                .s-empty-state No channels linked between Discord and Matrix yet...
+
+  h3.fs-category.mt32 Auto-create
+  .s-card.d-grid.px0
+    form.d-flex.ai-center.g16
+      #autocreate-loading.p8
+      - let value = !!select("guild_active", "autocreate", {guild_id}).pluck().get()
+      input(type="hidden" name="guild_id" value=guild_id)
+      input.s-toggle-switch#autocreate(name="autocreate" type="checkbox" hx-post=rel("/api/autocreate") hx-indicator="#autocreate-loading" hx-disabled-elt="this" checked=value autocomplete="off")
+      label.s-label.fl-grow1(for="autocreate")
+        | Create new Matrix rooms automatically
+        p.s-description If you want, OOYE can automatically create new Matrix rooms and link them when an unlinked Discord channel is spoken in.
+
+  if space_id
     h3.mt32.fs-category Manually link channels
-    form.d-flex.g16.ai-start(hx-post="/api/link" hx-trigger="submit" hx-disabled-elt="input, button" hx-indicator="#link-button")
+    form.d-flex.g16.ai-start(hx-post=rel("/api/link") hx-trigger="submit" hx-disabled-elt="input, button" hx-indicator="#link-button")
       .fl-grow2.s-btn-group.fd-column.w40
         each channel in unlinkedChannels
           input.s-btn--radio(type="radio" name="discord" required id=channel.id value=channel.id)

src/web/pug/guild_access_denied.pug

@@ -6,7 +6,7 @@ block body
       != icons.Spots.SpotEmptyXL
       p You need to log in to manage your servers.
       .d-flex.jc-center.g8
-        a.s-btn.s-btn__icon.s-btn__blurple.s-btn__filled(href=rel("/oauth"))
+        a.s-btn.s-btn__icon.s-btn__featured.s-btn__filled(href=rel("/oauth"))
           != icons.Icons.IconDiscord
           = ` Log in with Discord`
         a.s-btn.s-btn__icon.s-btn__matrix.s-btn__filled(href=rel("/log-in-with-matrix"))

src/web/pug/guild_not_linked.pug

@@ -25,13 +25,13 @@ block body
 
   h3.mt32.fs-category Choose a space
 
-  form.s-card.bs-sm.p0.s-table-container.bar-md(method="post" action="/api/link-space")
+  form.s-card.bs-sm.p0.s-table-container.bar-md(method="post" action=rel("/api/link-space"))
     input(type="hidden" name="guild_id" value=guild_id)
     table.s-table.s-table__bx-simple
       each space in spaces
         tr
           td.p0: +space(space)
-          td: button.s-btn(name="space_id" value=space.room_id hx-post="/api/link-space" hx-trigger="click" hx-disabled-elt="this") Link with this space
+          td: button.s-btn(name="space_id" value=space.room_id hx-post=rel("/api/link-space") hx-trigger="click" hx-disabled-elt="this") Link with this space
       else
         if session.data.mxid
           tr
@@ -44,7 +44,7 @@ block body
 
   h3.mt48.fs-category Auto-create
   .s-card
-    form.d-flex.ai-center.g8(method="post" action="/api/autocreate" hx-post="/api/autocreate" hx-indicator="#easy-mode-button")
+    form.d-flex.ai-center.g8(method="post" action=rel("/api/autocreate") hx-post=rel("/api/autocreate") hx-indicator="#easy-mode-button")
       input(type="hidden" name="guild_id" value=guild_id)
       input(type="hidden" name="autocreate" value="true")
       label.s-label.fl-grow1

src/web/pug/home.pug

@@ -1,10 +1,27 @@
 extends includes/template.pug
 
 block body
+  - let locked = reg.ooye.web_password && reg.ooye.web_password !== session.data.password
+
+  if locked
+    aside.s-notice.s-notice__warning.p8
+      .d-flex.flex__center.jc-space-between.s-banner--container.g8(class="md:fw-wrap")
+        .d-flex.ai-center.g8
+          .flex--item!= icons.Icons.IconLock
+          p.m0 <strong>Private instance.</strong> You need the password to use this instance of Out Of Your Element.
+        form(method="post" action=rel("/api/password"))
+          input.s-input(placeholder="Enter password" name="password")
+
+    .h32
+
+    .s-page-title.mb24
+      h1.s-page-title--header Out Of Your Element
+
+  else
     .s-page-title.mb24
       h1.s-page-title--header Bridge a Discord server
 
-  .d-grid.g24.grid__2(class="sm:grid__1")
+    .d-grid.g24.grid__2.mb24(class="sm:grid__1")
       .s-card.bs-md.d-flex.fd-column
         h2 Easy mode
         p Add the bot to your Discord server.
@@ -22,3 +39,18 @@ block body
         a.s-btn.s-btn__outlined.s-btn__icon(href=rel("/oauth?action=add-self-service"))
           != icons.Icons.IconUnorderedList
           = ` Set up self-service`
+
+  .s-prose
+    h2 What is this?
+    p #[a(href="https://gitdab.com/cadence/out-of-your-element") Out Of Your Element] is a bridge between the Discord and Matrix chat apps. It lets people on both platforms chat with each other without needing to get everyone on the same app.
+    p Just chat like usual, and the bridge will forward messages back and forth between the two platforms, so everyone sees the whole conversation.
+    p All kinds of content are supported, including pictures, threads, emojis, and @mentions.
+    p It's really easy to set up, even if you only have Discord. Just add the bot to your server, and it'll make everything available on Matrix automatically.
+
+    if locked
+      h2 This is a private instance
+      p Anybody can run their own instance of the Out Of Your Element software. The person running this instance has made it private, so you can't add it to your server just yet. If you know who's in charge of #{reg.ooye.server_name}, ask them for the password.
+
+      h2 Run your own instance
+      p You can still use Out Of Your Element by running your own copy of the software, but this requires some technical skill.
+      p To get started, #[a(href="https://gitdab.com/cadence/out-of-your-element/src/branch/main/docs/get-started.md") check the installation instructions.]

src/web/pug/includes/hash.svg

@@ -1,46 +1 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
-<svg
-   aria-hidden="true"
-   class="svg-icon iconItalic"
-   width="18"
-   height="18"
-   viewBox="0 0 18 18"
-   version="1.1"
-   id="svg1"
-   sodipodi:docname="hash.svg"
-   inkscape:version="1.3.2 (091e20ef0f, 2023-11-25)"
-   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
-   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
-   xmlns="http://www.w3.org/2000/svg"
-   xmlns:svg="http://www.w3.org/2000/svg">
-  <defs
-     id="defs1" />
-  <path
-     style="opacity:1;stroke-width:24.2222;stroke-linecap:square;paint-order:stroke fill markers"
-     d="m 13.949463,2.0417087 c 0,0 0.664304,0.00704 0.854464,0.00134 0.19016,-0.0057 0.924873,0.2384962 0.57664,0.9863413 -0.288846,0.6203095 -5.045042,11.035358 -5.4783833,11.984378 -0.4333415,0.949021 -0.7881247,0.945761 -1.3553087,0.945761 -0.567184,0 -0.3175392,0 -0.734375,0 -0.4168358,0 -0.7985231,-0.467356 -0.5770328,-0.951217 C 7.4569576,14.524452 12.479729,3.5512928 12.725807,3.0070042 13.022379,2.3510304 13.336114,2.0361844 13.949463,2.0417087 Z"
-     id="path4"
-     sodipodi:nodetypes="czszzzzsc" />
-  <rect
-     style="opacity:1;stroke-width:27.7591;stroke-linecap:square;paint-order:stroke fill markers"
-     id="rect4"
-     width="11.987322"
-     height="2"
-     x="2.002677"
-     y="11.007812"
-     rx="1"
-     ry="1" />
-  <rect
-     style="opacity:1;stroke-width:27.7591;stroke-linecap:square;paint-order:stroke fill markers"
-     id="rect5"
-     width="11.987322"
-     height="2"
-     x="4.0100012"
-     y="5.007813"
-     rx="1"
-     ry="1" />
-  <path
-     style="opacity:1;stroke-width:24.2222;stroke-linecap:square;paint-order:stroke fill markers"
-     d="m 9.1764922,2.0417087 c 0,0 0.664304,0.00704 0.8544638,0.00134 0.19016,-0.0057 0.924873,0.2384962 0.57664,0.9863413 -0.288846,0.6203095 -5.0450418,11.035358 -5.4783831,11.984378 -0.4333415,0.949021 -0.7881247,0.945761 -1.3553087,0.945761 -0.567184,0 -0.3175392,0 -0.734375,0 -0.4168358,0 -0.7985231,-0.467356 -0.5770328,-0.951217 C 2.6839868,14.524452 7.7067582,3.5512928 7.9528362,3.0070042 8.2494082,2.3510304 8.5631432,2.0361844 9.1764922,2.0417087 Z"
-     id="path1"
-     sodipodi:nodetypes="czszzzzsc" />
-</svg>
+<svg fill="none" viewBox="0 0 16 16" height="16" width="16"><path stroke="currentcolor" stroke-width="2" d="m6.75 1-2.5 14m7.5-14-2.5 14M14 10.25H1m14-4.5H2"></path></svg>

src/web/pug/includes/template.pug

@@ -61,6 +61,9 @@ html(lang="en")
     <link rel="icon" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 80%22><text y=%22.83em%22 font-size=%2283%22>💬</text></svg>">
     meta(name="htmx-config" content='{"requestClass":"is-loading"}')
     style.
+      .s-prose a {
+        text-decoration: underline;
+      }
       .themed {
         --theme-base-primary-color-h: 266;
         --theme-base-primary-color-s: 53%;
@@ -76,8 +79,6 @@ html(lang="en")
       .s-btn__dropdown:has(+ :popover-open) {
         background-color: var(--theme-topbar-item-background-hover, var(--black-200)) !important;
       }
-    +define-theme("blurple", "236", "84%", "64%")
-    +define-themed-button("blurple", "blurple")
     +define-themed-button("matrix", "black")
   body.themed.theme-system
     header.s-topbar
@@ -91,11 +92,13 @@ html(lang="en")
               if !session.data.mxid
                 a.s-btn.s-btn__icon.s-btn__matrix.s-btn__outlined.as-center(href=rel("/log-in-with-matrix"))
                   != icons.Icons.IconSpeechBubble
-                  = ` Log in with Matrix`
+                  = ` Log in`
+                  span(class="sm:d-none")= ` with Matrix`
               if !session.data.userID
-                a.s-btn.s-btn__icon.s-btn__blurple.s-btn__outlined.as-center(href=rel("/oauth"))
+                a.s-btn.s-btn__icon.s-btn__featured.s-btn__outlined.as-center(href=rel("/oauth"))
                   != icons.Icons.IconDiscord
-                  = ` Log in with Discord`
+                  = ` Log in`
+                  span(class="sm:d-none")= ` with Discord`
               if guild_id && managed.has(guild_id) && discord.guilds.has(guild_id)
                 button.s-topbar--item.s-btn.s-btn__muted.s-btn__dropdown.pr32.bar0.s-user-card(popovertarget="guilds")
                   +guild(discord.guilds.get(guild_id))

src/web/pug/invite.pug

@@ -17,7 +17,7 @@ block body
       .fl-grow1
         h2.fs-headline1 Invite a Matrix user
 
-        form.d-flex.gy16.fd-column(method="post" action="/api/invite" hx-post="/api/invite" hx-indicator="#invite-button" hx-select="#ok" hx-target="#form-container")
+        form.d-flex.gy16.fd-column(method="post" action=rel("/api/invite") hx-post=rel("/api/invite") hx-indicator="#invite-button" hx-select="#ok" hx-target="#form-container")
           .d-flex.gy4.fd-column
             label.s-label(for="mxid") Matrix ID
             input.fl-grow1.s-input.wmx3#mxid(name="mxid" required placeholder="@user:example.org")

src/web/pug/log-in-with-matrix.pug

@@ -6,11 +6,11 @@ block body
 
   .d-flex.g16#form-container
     .fl-grow1
-      form.d-flex.gy16.fd-column(method="post" action="/api/log-in-with-matrix" hx-post="/api/log-in-with-matrix" hx-indicator="#log-in-button" hx-select="#ok" hx-target="#form-container")
+      form.d-flex.gy16.fd-column(method="post" action=rel("/api/log-in-with-matrix") hx-post=rel("/api/log-in-with-matrix") hx-indicator="#log-in-button" hx-select="#ok" hx-target="#form-container")
         if next
           input(type="hidden" name="next" value=next)
         .d-flex.gy4.fd-column
           label.s-label(for="mxid") Your Matrix ID
-          input.fl-grow1.s-input.wmx3#mxid(name="mxid" required placeholder="@user:example.org" pattern="@([^:]+):([a-z0-9:\-]+\.[a-z0-9.:\-]+)")
+          input.fl-grow1.s-input.wmx3#mxid(name="mxid" required placeholder="@user:example.org" pattern="@([^:]+):([a-z0-9:\\-]+\\.[a-z0-9.:\\-]+)")
         div
           button.s-btn.s-btn__github#log-in-button Continue with Matrix

src/web/routes/info.js

@@ -0,0 +1,65 @@
+// @ts-check
+
+const {z} = require("zod")
+const {defineEventHandler, getValidatedQuery, H3Event} = require("h3")
+const {as, from, sync, select} = require("../../passthrough")
+
+/** @type {import("../../m2d/converters/utils")} */
+const mUtils = sync.require("../../m2d/converters/utils")
+
+/**
+ * @param {H3Event} event
+ * @returns {import("../../matrix/api")}
+ */
+function getAPI(event) {
+	/* c8 ignore next */
+	return event.context.api || sync.require("../../matrix/api")
+}
+
+const schema = {
+	message: z.object({
+		message_id: z.string().regex(/^[0-9]+$/)
+	})
+}
+
+as.router.get("/api/message", defineEventHandler(async event => {
+	const api = getAPI(event)
+
+	const {message_id} = await getValidatedQuery(event, schema.message.parse)
+	const metadatas = from("event_message").join("message_channel", "message_id").join("channel_room", "channel_id").where({message_id})
+		.select("event_id", "event_type", "event_subtype", "part", "reaction_part", "room_id", "source").and("ORDER BY part ASC, reaction_part DESC").all()
+
+	if (metadatas.length === 0) {
+		return new Response("Message not found", {status: 404, statusText: "Not Found"})
+	}
+
+	const events = await Promise.all(metadatas.map(metadata =>
+		api.getEvent(metadata.room_id, metadata.event_id).then(raw => ({
+			metadata: Object.assign({sender: raw.sender}, metadata),
+			raw
+		}))
+	))
+
+	/* c8 ignore next */
+	const primary = events.find(e => e.metadata.part === 0) || events[0]
+	const mxid = primary.metadata.sender
+	const source = primary.metadata.source === 0 ? "matrix" : "discord"
+
+	let matrix_author = undefined
+	if (source === "matrix") {
+		matrix_author = select("member_cache", ["displayname", "avatar_url", "mxid"], {room_id: primary.metadata.room_id, mxid}).get()
+		if (!matrix_author) {
+			try {
+				matrix_author = await api.getProfile(mxid)
+			} catch (e) {
+				matrix_author = {}
+			}
+		}
+		if (!matrix_author.displayname) matrix_author.displayname = mxid
+		if (matrix_author.avatar_url) matrix_author.avatar_url = mUtils.getPublicUrlForMxc(matrix_author.avatar_url)
+		else matrix_author.avatar_url = null
+		matrix_author["mxid"] = mxid
+	}
+
+	return {source, matrix_author, events}
+}))

src/web/routes/info.test.js

@@ -0,0 +1,219 @@
+// @ts-check
+
+const assert = require("assert/strict")
+const {router, test} = require("../../../test/web")
+
+test("web info: returns 404 when message doesn't exist", async t => {
+	const res = await router.test("get", "/api/message?message_id=1")
+	assert(res instanceof Response)
+	t.equal(res.status, 404)
+})
+
+test("web info: returns data for a matrix message and profile", async t => {
+	let called = 0
+	const raw = {
+		type: "m.room.message",
+		room_id: "!qzDBLKlildpzrrOnFZ:cadence.moe",
+		sender: "@cadence:cadence.moe",
+		content: {
+			msgtype: "m.text",
+			body: "testing :heart_pink: :heart_pink: ",
+			format: "org.matrix.custom.html",
+			formatted_body: "testing <img data-mx-emoticon=\"\" src=\"mxc://cadence.moe/AyAhnRNjWyFhJYTRibYwQpvf\" alt=\":heart_pink:\" title=\":heart_pink:\" height=\"32\" vertical-align=\"middle\" /> <img data-mx-emoticon=\"\" src=\"mxc://cadence.moe/AyAhnRNjWyFhJYTRibYwQpvf\" alt=\":heart_pink:\" title=\":heart_pink:\" height=\"32\" vertical-align=\"middle\" />"
+		},
+		origin_server_ts: 1739312945302,
+		unsigned: {
+			membership: "join",
+			age: 10063702303
+		},
+		event_id: "$51gH61p_eJc2RylOdE2lAr4-ogP7dS0WJI62lCFzBvk",
+		user_id: "@cadence:cadence.moe",
+		age: 10063702303
+	}
+	const res = await router.test("get", "/api/message?message_id=1339000288144658482", {
+		api: {
+			// @ts-ignore - returning static data when method could be called with a different typescript generic
+			async getEvent(roomID, eventID) {
+				called++
+				t.equal(roomID, "!qzDBLKlildpzrrOnFZ:cadence.moe")
+				t.equal(eventID, "$51gH61p_eJc2RylOdE2lAr4-ogP7dS0WJI62lCFzBvk")
+				return raw
+			},
+			async getProfile(mxid) {
+				called++
+				t.equal(mxid, "@cadence:cadence.moe")
+				return {
+					displayname: "okay 🤍 yay 🤍"
+				}
+			}
+		}
+	})
+	t.deepEqual(res, {
+		source: "matrix",
+		matrix_author: {
+			displayname: "okay 🤍 yay 🤍",
+			avatar_url: null,
+			mxid: "@cadence:cadence.moe"
+		},
+		events: [{
+			metadata: {
+			  event_id: "$51gH61p_eJc2RylOdE2lAr4-ogP7dS0WJI62lCFzBvk",
+			  event_subtype: "m.text",
+			  event_type: "m.room.message",
+			  part: 0,
+			  reaction_part: 0,
+			  room_id: "!qzDBLKlildpzrrOnFZ:cadence.moe",
+			  sender: "@cadence:cadence.moe",
+			  source: 0
+			},
+			raw
+		}]
+	})
+	t.equal(called, 2)
+})
+
+test("web info: returns data for a matrix message without profile", async t => {
+	let called = 0
+	const raw = {
+		type: "m.room.message",
+		room_id: "!qzDBLKlildpzrrOnFZ:cadence.moe",
+		sender: "@cadence:cadence.moe",
+		content: {
+			msgtype: "m.text",
+			body: "testing :heart_pink: :heart_pink: ",
+			format: "org.matrix.custom.html",
+			formatted_body: "testing <img data-mx-emoticon=\"\" src=\"mxc://cadence.moe/AyAhnRNjWyFhJYTRibYwQpvf\" alt=\":heart_pink:\" title=\":heart_pink:\" height=\"32\" vertical-align=\"middle\" /> <img data-mx-emoticon=\"\" src=\"mxc://cadence.moe/AyAhnRNjWyFhJYTRibYwQpvf\" alt=\":heart_pink:\" title=\":heart_pink:\" height=\"32\" vertical-align=\"middle\" />"
+		},
+		origin_server_ts: 1739312945302,
+		unsigned: {
+			membership: "join",
+			age: 10063702303
+		},
+		event_id: "$51gH61p_eJc2RylOdE2lAr4-ogP7dS0WJI62lCFzBvk",
+		user_id: "@cadence:cadence.moe",
+		age: 10063702303
+	}
+	const res = await router.test("get", "/api/message?message_id=1339000288144658482", {
+		api: {
+			// @ts-ignore - returning static data when method could be called with a different typescript generic
+			async getEvent(roomID, eventID) {
+				called++
+				t.equal(roomID, "!qzDBLKlildpzrrOnFZ:cadence.moe")
+				t.equal(eventID, "$51gH61p_eJc2RylOdE2lAr4-ogP7dS0WJI62lCFzBvk")
+				return raw
+			}
+		}
+	})
+	t.deepEqual(res, {
+		source: "matrix",
+		matrix_author: {
+			displayname: "@cadence:cadence.moe",
+			avatar_url: null,
+			mxid: "@cadence:cadence.moe"
+		},
+		events: [{
+			metadata: {
+			  event_id: "$51gH61p_eJc2RylOdE2lAr4-ogP7dS0WJI62lCFzBvk",
+			  event_subtype: "m.text",
+			  event_type: "m.room.message",
+			  part: 0,
+			  reaction_part: 0,
+			  room_id: "!qzDBLKlildpzrrOnFZ:cadence.moe",
+			  sender: "@cadence:cadence.moe",
+			  source: 0
+			},
+			raw
+		}]
+	})
+	t.equal(called, 1)
+})
+
+test("web info: returns data for a discord message", async t => {
+	let called = 0
+	const raw1 = {
+		type: "m.room.message",
+		sender: "@_ooye_accavish:cadence.moe",
+		content: {
+			"m.mentions": {},
+			msgtype: "m.text",
+			body: "brony music mentioned on wikipedia's did you know and also unrelated cat pic"
+		},
+		origin_server_ts: 1749377203735,
+		unsigned: {
+			membership: "join",
+			age: 119
+		},
+		event_id: "$AfrB8hzXkDMvuoWjSZkDdFYomjInWH7jMBPkwQMN8AI",
+		room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
+	}
+	const raw2 = {
+		type: "m.room.message",
+		sender: "@_ooye_accavish:cadence.moe",
+		content: {
+			"m.mentions": {},
+			msgtype: "m.image",
+			url: "mxc://cadence.moe/ABOMymxHcpVeecHvmSIYmYXx",
+			external_url: "https://bridge.cadence.moe/download/discordcdn/112760669178241024/1381212840710504448/image.png",
+			body: "image.png",
+			filename: "image.png",
+			info: {
+				mimetype: "image/png",
+				w: 966,
+				h: 368,
+				size: 166060
+			}
+		},
+		origin_server_ts: 1749377203789,
+		unsigned: {
+			membership: "join",
+			age: 65
+		},
+		event_id: "$43baKEhJfD-RlsFQi0LB16Zxd8yMqp0HSVL00TDQOqM",
+		room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe"
+	}
+	const res = await router.test("get", "/api/message?message_id=1381212840957972480", {
+		api: {
+			// @ts-ignore - returning static data when method could be called with a different typescript generic
+			async getEvent(roomID, eventID) {
+				called++
+				t.equal(roomID, "!kLRqKKUQXcibIMtOpl:cadence.moe")
+				if (eventID === raw1.event_id) {
+					return raw1
+				} else {
+					assert(eventID === raw2.event_id)
+					return raw2
+				}
+			}
+		}
+	})
+	t.deepEqual(res, {
+		source: "discord",
+		matrix_author: undefined,
+		events: [{
+			metadata: {
+			  event_id: "$AfrB8hzXkDMvuoWjSZkDdFYomjInWH7jMBPkwQMN8AI",
+			  event_subtype: "m.text",
+			  event_type: "m.room.message",
+			  part: 0,
+			  reaction_part: 1,
+			  room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe",
+			  sender: "@_ooye_accavish:cadence.moe",
+			  source: 1
+			},
+			raw: raw1
+		}, {
+			metadata: {
+				event_id: "$43baKEhJfD-RlsFQi0LB16Zxd8yMqp0HSVL00TDQOqM",
+				event_subtype: "m.image",
+				event_type: "m.room.message",
+				part: 1,
+				reaction_part: 0,
+				room_id: "!kLRqKKUQXcibIMtOpl:cadence.moe",
+				sender: "@_ooye_accavish:cadence.moe",
+				source: 1
+			},
+			raw: raw2
+		}]
+	})
+	t.equal(called, 2)
+})

src/web/routes/link.js

@@ -89,12 +89,12 @@ as.router.post("/api/link-space", defineEventHandler(async event => {
 	try {
 		powerLevelsStateContent = await api.getStateEvent(spaceID, "m.room.power_levels", "")
 	} catch (e) {}
-	const selfPowerLevel = powerLevelsStateContent?.users?.[me] || powerLevelsStateContent?.users_default || 0
-	if (selfPowerLevel < (powerLevelsStateContent?.state_default || 50) || selfPowerLevel < 100) throw createError({status: 400, message: "Bad Request", data: "OOYE needs power level 100 (admin) in the target Matrix space"})
+	const selfPowerLevel = powerLevelsStateContent?.users?.[me] ?? powerLevelsStateContent?.users_default ?? 0
+	if (selfPowerLevel < (powerLevelsStateContent?.state_default ?? 50) || selfPowerLevel < 100) throw createError({status: 400, message: "Bad Request", data: "OOYE needs power level 100 (admin) in the target Matrix space"})
 
 	// Check inviting user is a moderator in the space
-	const invitingPowerLevel = powerLevelsStateContent?.users?.[session.data.mxid] || powerLevelsStateContent?.users_default || 0
-	if (invitingPowerLevel < (powerLevelsStateContent?.state_default || 50)) throw createError({status: 403, message: "Forbidden", data: `You need to be at least power level 50 (moderator) in the target Matrix space to set up OOYE, but you are currently power level ${invitingPowerLevel}.`})
+	const invitingPowerLevel = powerLevelsStateContent?.users?.[session.data.mxid] ?? powerLevelsStateContent?.users_default ?? 0
+	if (invitingPowerLevel < (powerLevelsStateContent?.state_default ?? 50)) throw createError({status: 403, message: "Forbidden", data: `You need to be at least power level 50 (moderator) in the target Matrix space to set up OOYE, but you are currently power level ${invitingPowerLevel}.`})
 
 	// Insert database entry
 	db.transaction(() => {
@@ -134,12 +134,14 @@ as.router.post("/api/link", defineEventHandler(async event => {
 	if (row) throw createError({status: 400, message: "Bad Request", data: `Channel ID ${row.channel_id} or room ID ${parsedBody.matrix} are already bridged and cannot be reused`})
 
 	// Check room is part of the guild's space
-	/** @type {Ty.Event.M_Space_Child?} */
-	let spaceChildEvent = null
-	try {
-		spaceChildEvent = await api.getStateEvent(spaceID, "m.space.child", parsedBody.matrix)
-	} catch (e) {}
-	if (!Array.isArray(spaceChildEvent?.via)) throw createError({status: 400, message: "Bad Request", data: "Matrix room needs to be part of the bridged space"})
+	let found = false
+	for await (const room of api.generateFullHierarchy(spaceID)) {
+		if (room.room_id === parsedBody.matrix && !room.room_type) {
+			found = true
+			break
+		}
+	}
+	if (!found) throw createError({status: 400, message: "Bad Request", data: "Matrix room needs to be part of the bridged space"})
 
 	// Check room exists and bridge is joined
 	try {
@@ -155,8 +157,8 @@ as.router.post("/api/link", defineEventHandler(async event => {
 	try {
 		powerLevelsStateContent = await api.getStateEvent(parsedBody.matrix, "m.room.power_levels", "")
 	} catch (e) {}
-	const selfPowerLevel = powerLevelsStateContent?.users?.[me] || powerLevelsStateContent?.users_default || 0
-	if (selfPowerLevel < (powerLevelsStateContent?.state_default || 50) || selfPowerLevel < 100) throw createError({status: 400, message: "Bad Request", data: "OOYE needs power level 100 (admin) in the target Matrix room"})
+	const selfPowerLevel = powerLevelsStateContent?.users?.[me] ?? powerLevelsStateContent?.users_default ?? 0
+	if (selfPowerLevel < (powerLevelsStateContent?.state_default ?? 50) || selfPowerLevel < 100) throw createError({status: 400, message: "Bad Request", data: "OOYE needs power level 100 (admin) in the target Matrix room"})
 
 	// Insert database entry, but keep the room's existing properties if they are set
 	const nick = await api.getStateEvent(parsedBody.matrix, "m.room.name", "").then(content => content.name || null).catch(() => null)

src/web/routes/link.test.js

@@ -233,13 +233,7 @@ test("web link space: successfully adds entry to database and loads page", async
 			mxid: "@cadence:cadence.moe"
 		},
 		api: {
-			async getStateEvent(roomID, type, key) {
-				return {}
-			},
-			async getMembers(roomID, membership) {
-				return {chunk: []}
-			},
-			async getFullHierarchy(roomID) {
+			async getFullHierarchy(spaceID) {
 				return []
 			}
 		}
@@ -344,7 +338,7 @@ test("web link room: checks the autocreate setting if the space doesn't exist ye
 	t.equal(called, 1)
 })
 
-test("web link room: check that room is part of space (event missing)", async t => {
+test("web link room: check that room is part of space (not in hierarchy)", async t => {
 	let called = 0
 	const [error] = await tryToCatch(() => router.test("post", "/api/link", {
 		sessionData: {
@@ -356,37 +350,9 @@ test("web link room: check that room is part of space (event missing)", async t
 			guild_id: "665289423482519565"
 		},
 		api: {
-			async getStateEvent(roomID, type, key) {
+			async *generateFullHierarchy(spaceID) {
 				called++
-				t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
-				t.equal(type, "m.space.child")
-				t.equal(key, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
-				throw new MatrixServerError({errcode: "M_NOT_FOUND", error: "what if I told you there was no such thing as a space"})
-			}
-		}
-	}))
-	t.equal(error.data, "Matrix room needs to be part of the bridged space")
-	t.equal(called, 1)
-})
-
-test("web link room: check that room is part of space (event empty)", async t => {
-	let called = 0
-	const [error] = await tryToCatch(() => router.test("post", "/api/link", {
-		sessionData: {
-			managedGuilds: ["665289423482519565"]
-		},
-		body: {
-			discord: "665310973967597573",
-			matrix: "!NDbIqNpJyPvfKRnNcr:cadence.moe",
-			guild_id: "665289423482519565"
-		},
-		api: {
-			async getStateEvent(roomID, type, key) {
-				called++
-				t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
-				t.equal(type, "m.space.child")
-				t.equal(key, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
-				return {}
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
 			}
 		}
 	}))
@@ -410,12 +376,16 @@ test("web link room: check that bridge can join room", async t => {
 				called++
 				throw new MatrixServerError({errcode: "M_FORBIDDEN", error: "not allowed to join I guess"})
 			},
-			async getStateEvent(roomID, type, key) {
+			async *generateFullHierarchy(spaceID) {
 				called++
-				t.equal(type, "m.space.child")
-				t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
-				t.equal(key, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
-				return {via: ["cadence.moe"]}
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				yield {
+					room_id: "!NDbIqNpJyPvfKRnNcr:cadence.moe",
+					children_state: {},
+					guest_can_join: false,
+					num_joined_members: 2
+				}
+				/* c8 ignore next */
 			}
 		}
 	}))
@@ -439,19 +409,25 @@ test("web link room: check that bridge has PL 100 in target room (event missing)
 				called++
 				return roomID
 			},
+			async *generateFullHierarchy(spaceID) {
+				called++
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				yield {
+					room_id: "!NDbIqNpJyPvfKRnNcr:cadence.moe",
+					children_state: {},
+					guest_can_join: false,
+					num_joined_members: 2
+				}
+				/* c8 ignore next */
+			},
 			async getStateEvent(roomID, type, key) {
 				called++
-				if (type === "m.space.child") {
-					t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
-					t.equal(key, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
-					return {via: ["cadence.moe"]}
-				} else if (type === "m.room.power_levels") {
 				t.equal(roomID, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
+				t.equal(type, "m.room.power_levels")
 				t.equal(key, "")
 				throw new MatrixServerError({errcode: "M_NOT_FOUND", error: "what if I told you there's no such thing as power levels"})
 			}
 		}
-		}
 	}))
 	t.equal(error.data, "OOYE needs power level 100 (admin) in the target Matrix room")
 	t.equal(called, 3)
@@ -473,19 +449,25 @@ test("web link room: check that bridge has PL 100 in target room (users default)
 				called++
 				return roomID
 			},
+			async *generateFullHierarchy(spaceID) {
+				called++
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				yield {
+					room_id: "!NDbIqNpJyPvfKRnNcr:cadence.moe",
+					children_state: {},
+					guest_can_join: false,
+					num_joined_members: 2
+				}
+				/* c8 ignore next */
+			},
 			async getStateEvent(roomID, type, key) {
 				called++
-				if (type === "m.space.child") {
-					t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
-					t.equal(key, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
-					return {via: ["cadence.moe"]}
-				} else if (type === "m.room.power_levels") {
 				t.equal(roomID, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
+				t.equal(type, "m.room.power_levels")
 				t.equal(key, "")
 				return {users_default: 50}
 			}
 		}
-		}
 	}))
 	t.equal(error.data, "OOYE needs power level 100 (admin) in the target Matrix room")
 	t.equal(called, 3)
@@ -507,17 +489,23 @@ test("web link room: successfully calls createRoom", async t => {
 				called++
 				return roomID
 			},
+			async *generateFullHierarchy(spaceID) {
+				called++
+				t.equal(spaceID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
+				yield {
+					room_id: "!NDbIqNpJyPvfKRnNcr:cadence.moe",
+					children_state: {},
+					guest_can_join: false,
+					num_joined_members: 2
+				}
+				/* c8 ignore next */
+			},
 			async getStateEvent(roomID, type, key) {
 				if (type === "m.room.power_levels") {
 					called++
 					t.equal(roomID, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
 					t.equal(key, "")
 					return {users: {"@_ooye_bot:cadence.moe": 100}}
-				} else if (type === "m.space.child") {
-					called++
-					t.equal(roomID, "!zTMspHVUBhFLLSdmnS:cadence.moe")
-					t.equal(key, "!NDbIqNpJyPvfKRnNcr:cadence.moe")
-					return {via: ["cadence.moe"]}
 				} else if (type === "m.room.name") {
 					called++
 					t.equal(roomID, "!NDbIqNpJyPvfKRnNcr:cadence.moe")

src/web/routes/log-in-with-matrix.js

@@ -5,7 +5,7 @@ const {randomUUID} = require("crypto")
 const {defineEventHandler, getValidatedQuery, sendRedirect, readValidatedBody, createError, getRequestHeader, H3Event} = require("h3")
 const {LRUCache} = require("lru-cache")
 
-const {as} = require("../../passthrough")
+const {as, db, select} = require("../../passthrough")
 const {reg} = require("../../matrix/read-registration")
 
 const {sync} = require("../../passthrough")
@@ -53,7 +53,7 @@ as.router.get("/log-in-with-matrix", defineEventHandler(async event => {
 	}
 
 	const userAgent = getRequestHeader(event, "User-Agent")
-	if (userAgent?.match(/bot/)) throw createError({status: 400, data: "Sorry URL previewer, you can't have this URL."})
+	if (userAgent?.match(/bot|matrix/)) throw createError({status: 400, data: "Sorry URL previewer, you can't have this URL."})
 
 	if (!validToken.has(token)) return sendRedirect(event, `${reg.ooye.bridge_origin}/log-in-with-matrix`, 302)
 
@@ -71,7 +71,6 @@ as.router.get("/log-in-with-matrix", defineEventHandler(async event => {
 as.router.post("/api/log-in-with-matrix", defineEventHandler(async event => {
 	const api = getAPI(event)
 	const {mxid, next} = await readValidatedBody(event, schema.form.parse)
-	let roomID = null
 
 	// Don't extend a duplicate invite for the same user
 	for (const alreadyInvited of validToken.values()) {
@@ -80,43 +79,32 @@ as.router.post("/api/log-in-with-matrix", defineEventHandler(async event => {
 		}
 	}
 
-	// See if we can reuse an existing room from account data
-	let directData = {}
-	try {
-		directData = await api.getAccountData("m.direct")
-	} catch (e) {}
-	const rooms = directData[mxid] || []
-	for (const candidate of rooms) {
+	// Check if we have an existing DM
+	let roomID = select("direct", "room_id", {mxid}).pluck().get()
+	if (roomID) {
 		// Check that the person is/still in the room
-		let member
 		try {
-			member = await api.getStateEvent(candidate, "m.room.member", mxid)
+			var member = await api.getStateEvent(roomID, "m.room.member", mxid)
 		} catch (e) {}
+
+		// Invite them back to the room if needed
 		if (!member || member.membership === "leave") {
-			// We can reinvite them back to the same room!
-			await api.inviteToRoom(candidate, mxid)
-			roomID = candidate
-		} else {
-			// Member is in this room
-			roomID = candidate
+			await api.inviteToRoom(roomID, mxid)
 		}
-		if (roomID) break	// no need to check other candidates
 	}
 
-	// No candidates available, create a new room and invite
-	if (!roomID) {
+	// No existing DM, create a new room and invite
+	else {
 		roomID = await api.createRoom({
 			invite: [mxid],
 			is_direct: true,
 			preset: "trusted_private_chat"
 		})
 		// Store the newly created room in account data (Matrix doesn't do this for us automatically, sigh...)
-		;(directData[mxid] ??= []).push(roomID)
-		await api.setAccountData("m.direct", directData)
+		db.prepare("REPLACE INTO direct (mxid, room_id) VALUES (?, ?)").run(mxid, roomID)
 	}
 
 	const token = randomUUID()
-	validToken.set(token, mxid)
 
 	console.log(`web log in requested for ${mxid}`)
 	const paramsObject = {token}
@@ -129,5 +117,7 @@ as.router.post("/api/log-in-with-matrix", defineEventHandler(async event => {
 		body
 	})
 
+	validToken.set(token, mxid)
+
 	return sendRedirect(event, "../ok?msg=Please check your inbox on Matrix!&spot=SpotMailXL", 302)
 }))

src/web/routes/log-in-with-matrix.test.js

@@ -9,7 +9,7 @@ const {MatrixServerError} = require("../../matrix/mreq")
 test("log in with matrix: shows web page with form on first request", async t => {
 	const html = await router.test("get", "/log-in-with-matrix", {
 	})
-	t.has(html, `hx-post="/api/log-in-with-matrix"`)
+	t.has(html, `hx-post="api/log-in-with-matrix"`)
 })
 
 // ***** second request *****
@@ -22,7 +22,7 @@ test("log in with matrix: checks if mxid format looks valid", async t => {
 			mxid: "x@cadence:cadence.moe"
 		}
 	}))
-	t.equal(error.data.issues[0].validation, "regex")
+	t.match(error.data.fieldErrors.mxid, /must match pattern/)
 })
 
 test("log in with matrix: checks if mxid domain format looks valid", async t => {
@@ -31,10 +31,10 @@ test("log in with matrix: checks if mxid domain format looks valid", async t =>
 			mxid: "@cadence:cadence."
 		}
 	}))
-	t.equal(error.data.issues[0].validation, "regex")
+	t.match(error.data.fieldErrors.mxid, /must match pattern/)
 })
 
-test("log in with matrix: sends message when there is no m.direct data", async t => {
+test("log in with matrix: sends message when there is no existing dm room", async t => {
 	const event = {}
 	let called = 0
 	await router.test("post", "/api/log-in-with-matrix", {
@@ -42,20 +42,10 @@ test("log in with matrix: sends message when there is no m.direct data", async t
 			mxid: "@cadence:cadence.moe"
 		},
 		api: {
-			async getAccountData(type) {
-				called++
-				t.equal(type, "m.direct")
-				throw new MatrixServerError({errcode: "M_NOT_FOUND"})
-			},
 			async createRoom() {
 				called++
 				return "!created:cadence.moe"
 			},
-			async setAccountData(type, content) {
-				called++
-				t.equal(type, "m.direct")
-				t.deepEqual(content, {"@cadence:cadence.moe": ["!created:cadence.moe"]})
-			},
 			async sendEvent(roomID, type, content) {
 				called++
 				t.equal(roomID, "!created:cadence.moe")
@@ -68,7 +58,7 @@ test("log in with matrix: sends message when there is no m.direct data", async t
 		event
 	})
 	t.match(event.node.res.getHeader("location"), /Please check your inbox on Matrix/)
-	t.equal(called, 4)
+	t.equal(called, 2)
 })
 
 test("log in with matrix: does not send another message when a log in is in progress", async t => {
@@ -82,7 +72,7 @@ test("log in with matrix: does not send another message when a log in is in prog
 	t.match(event.node.res.getHeader("location"), /We already sent you a link on Matrix/)
 })
 
-test("log in with matrix: reuses room from m.direct", async t => {
+test("log in with matrix: reuses room from direct", async t => {
 	const event = {}
 	let called = 0
 	await router.test("post", "/api/log-in-with-matrix", {
@@ -90,11 +80,6 @@ test("log in with matrix: reuses room from m.direct", async t => {
 			mxid: "@user1:example.org"
 		},
 		api: {
-			async getAccountData(type) {
-				called++
-				t.equal(type, "m.direct")
-				return {"@user1:example.org": ["!existing:cadence.moe"]}
-			},
 			async getStateEvent(roomID, type, key) {
 				called++
 				t.equal(roomID, "!existing:cadence.moe")
@@ -111,10 +96,10 @@ test("log in with matrix: reuses room from m.direct", async t => {
 		event
 	})
 	t.match(event.node.res.getHeader("location"), /Please check your inbox on Matrix/)
-	t.equal(called, 3)
+	t.equal(called, 2)
 })
 
-test("log in with matrix: reuses room from m.direct, reinviting if user has left", async t => {
+test("log in with matrix: reuses room from direct, reinviting if user has left", async t => {
 	const event = {}
 	let called = 0
 	await router.test("post", "/api/log-in-with-matrix", {
@@ -122,11 +107,6 @@ test("log in with matrix: reuses room from m.direct, reinviting if user has left
 			mxid: "@user2:example.org"
 		},
 		api: {
-			async getAccountData(type) {
-				called++
-				t.equal(type, "m.direct")
-				return {"@user2:example.org": ["!existing:cadence.moe"]}
-			},
 			async getStateEvent(roomID, type, key) {
 				called++
 				t.equal(roomID, "!existing:cadence.moe")
@@ -148,7 +128,7 @@ test("log in with matrix: reuses room from m.direct, reinviting if user has left
 		event
 	})
 	t.match(event.node.res.getHeader("location"), /Please check your inbox on Matrix/)
-	t.equal(called, 4)
+	t.equal(called, 3)
 })
 
 // ***** third request *****

src/web/routes/oauth.js

@@ -27,7 +27,7 @@ const schema = {
 	token: z.object({
 		token_type: z.string(),
 		access_token: z.string(),
-		expires_in: z.number({coerce: true}),
+		expires_in: z.coerce.number(),
 		refresh_token: z.string(),
 		scope: z.string()
 	})
@@ -37,6 +37,7 @@ as.router.get("/oauth", defineEventHandler(async event => {
 	const session = await auth.useSession(event)
 	let scope = "guilds"
 
+	if (!reg.ooye.web_password || reg.ooye.web_password === session.data.password) {
 		const parsedFirstQuery = await getValidatedQuery(event, schema.first.safeParse)
 		if (parsedFirstQuery.data?.action === "add") {
 			scope = "bot+guilds"
@@ -45,6 +46,7 @@ as.router.get("/oauth", defineEventHandler(async event => {
 			scope = "bot+guilds"
 			await session.update({selfService: true})
 		}
+	}
 
 	async function tryAgain() {
 		const newState = randomUUID()

src/web/routes/password.js

@@ -0,0 +1,21 @@
+// @ts-check
+
+const {z} = require("zod")
+const {defineEventHandler, readValidatedBody, sendRedirect} = require("h3")
+const {as, sync} = require("../../passthrough")
+
+/** @type {import("../auth")} */
+const auth = sync.require("../auth")
+
+const schema = {
+	password: z.object({
+		password: z.string()
+	})
+}
+
+as.router.post("/api/password", defineEventHandler(async event => {
+	const {password} = await readValidatedBody(event, schema.password.parse)
+	const session = await auth.useSession(event)
+	await session.update({password})
+	return sendRedirect(event, "../")
+}))

src/web/server.js

@@ -29,9 +29,11 @@ sync.require("./routes/download-matrix")
 sync.require("./routes/download-discord")
 sync.require("./routes/guild-settings")
 sync.require("./routes/guild")
+sync.require("./routes/info")
 sync.require("./routes/link")
-sync.require("./routes/oauth")
 sync.require("./routes/log-in-with-matrix")
+sync.require("./routes/oauth")
+sync.require("./routes/password")
 
 // Files
 

src/web/server.test.js

@@ -1,14 +1,14 @@
 // @ts-check
 
 const streamWeb = require("stream/web")
-const {test} = require("supertape")
+const {test} = require("../../test/web")
 const {router} = require("../../test/web")
 const assert = require("assert").strict
 
 require("./server")
 
 test("web server: can get home", async t => {
-	t.match(await router.test("get", "/", {}), /Add the bot to your Discord server./)
+	t.has(await router.test("get", "/", {}), /a bridge between the Discord and Matrix chat apps/)
 })
 
 test("web server: can get htmx", async t => {

test/ooye-test-data.sql

@@ -70,7 +70,10 @@ INSERT INTO message_channel (message_id, channel_id) VALUES
 ('1278002262400176128', '1100319550446252084'),
 ('1278001833876525057', '1100319550446252084'),
 ('1191567971970191490', '176333891320283136'),
-('1144874214311067708', '687028734322147344');
+('1144874214311067708', '687028734322147344'),
+('1339000288144658482', '176333891320283136'),
+('1381212840957972480', '112760669178241024'),
+('1401760355339862066', '112760669178241024');
 
 INSERT INTO event_message (event_id, event_type, event_subtype, message_id, part, reaction_part, source) VALUES
 ('$X16nfVks1wsrhq4E9SSLiqrf2N8KD0erD0scZG7U5xg', 'm.room.message', 'm.text', '1126786462646550579', 0, 0, 1),
@@ -110,7 +113,11 @@ INSERT INTO event_message (event_id, event_type, event_subtype, message_id, part
 ('$W1nsDhNIojWrcQOdnOD9RaEvrz2qyZErQoNhPRs1nK4', 'm.room.message', 'm.text', '1273743950028607530', 0, 0, 0),
 ('$UTqiL3Zj3FC4qldxRLggN1fhygpKl8sZ7XGY5f9MNbF', 'm.room.message', 'm.text', '1278002262400176128', 0, 0, 1),
 ('$aLVZyiC3HlOu-prCSIaXlQl68I8leUdnPFiCwkgn6qM', 'm.room.message', 'm.text', '1278001833876525057', 0, 0, 1),
-('$tBIT8mO7XTTCgIINyiAIy6M2MSoPAdJenRl_RLyYuaE', 'm.room.message', 'm.text', '1191567971970191490', 0, 0, 1);
+('$tBIT8mO7XTTCgIINyiAIy6M2MSoPAdJenRl_RLyYuaE', 'm.room.message', 'm.text', '1191567971970191490', 0, 0, 1),
+('$51gH61p_eJc2RylOdE2lAr4-ogP7dS0WJI62lCFzBvk', 'm.room.message', 'm.text', '1339000288144658482', 0, 0, 0),
+('$AfrB8hzXkDMvuoWjSZkDdFYomjInWH7jMBPkwQMN8AI', 'm.room.message', 'm.text', '1381212840957972480', 0, 1, 1),
+('$43baKEhJfD-RlsFQi0LB16Zxd8yMqp0HSVL00TDQOqM', 'm.room.message', 'm.image', '1381212840957972480', 1, 0, 1),
+('$7P2O_VTQNHvavX5zNJ35DV-dbJB1Ag80tGQP_JzGdhk', 'm.room.message', 'm.text', '1401760355339862066', 0, 0, 0);
 
 INSERT INTO file (discord_url, mxc_url) VALUES
 ('https://cdn.discordapp.com/attachments/497161332244742154/1124628646431297546/image.png', 'mxc://cadence.moe/qXoZktDqNtEGuOCZEADAMvhM'),
@@ -155,7 +162,8 @@ INSERT INTO member_cache (room_id, mxid, displayname, avatar_url, power_level) V
 ('!TqlyQmifxGUggEmdBN:cadence.moe', '@Milan:tchncs.de', 'Milan', NULL, 0),
 ('!TqlyQmifxGUggEmdBN:cadence.moe', '@ampflower:matrix.org', 'Ampflower 🌺', 'mxc://cadence.moe/PRfhXYBTOalvgQYtmCLeUXko', 0),
 ('!TqlyQmifxGUggEmdBN:cadence.moe', '@aflower:syndicated.gay', 'Rose', 'mxc://syndicated.gay/ZkBUPXCiXTjdJvONpLJmcbKP', 0),
-('!TqlyQmifxGUggEmdBN:cadence.moe', '@cadence:cadence.moe', 'cadence [they]', NULL, 0);
+('!TqlyQmifxGUggEmdBN:cadence.moe', '@cadence:cadence.moe', 'cadence [they]', NULL, 0),
+('!iSyXgNxQcEuXoXpsSn:pussthecat.org', '@austin:tchncs.de', 'Austin Huang', 'mxc://tchncs.de/090a2b5e07eed2f71e84edad5207221e6c8f8b8e', 0);
 
 INSERT INTO reaction (hashed_event_id, message_id, encoded_emoji) VALUES
 (5162930312280790092, '1141501302736695317', '%F0%9F%90%88');
@@ -166,10 +174,9 @@ INSERT INTO member_power (mxid, room_id, power_level) VALUES
 INSERT INTO lottie (sticker_id, mxc_url) VALUES
 ('860171525772279849', 'mxc://cadence.moe/ZtvvVbwMIdUZeovWVyGVFCeR');
 
-INSERT INTO auto_emoji (name, emoji_id, guild_id) VALUES
-('L1', '1144820033948762203', '529176156398682115'),
-('L2', '1144820084079087647', '529176156398682115'),
-('_',  '_',                   '529176156398682115');
+INSERT INTO auto_emoji (name, emoji_id) VALUES
+('L1', '1144820033948762203'),
+('L2', '1144820084079087647');
 
 INSERT INTO media_proxy (permitted_hash) VALUES
 (-429802515645771439),
@@ -181,4 +188,8 @@ INSERT INTO invite (mxid, room_id, type, name, avatar, topic) VALUES
 ('@cadence:cadence.moe', '!room:cadence.moe', NULL, 'some room', NULL, NULL),
 ('@rnl:cadence.moe', '!space:cadence.moe', NULL, 'somebody else''s space', NULL, NULL);
 
+INSERT INTO direct (mxid, room_id) VALUES
+('@user1:example.org', '!existing:cadence.moe'),
+('@user2:example.org', '!existing:cadence.moe');
+
 COMMIT;

test/test.js

@@ -29,6 +29,7 @@ reg.ooye.bridge_origin = "https://bridge.example.org"
 const sync = new HeatSync({watchFS: false})
 
 const discord = {
+	// @ts-ignore - ignore guilds, because my data dump is missing random properties
 	guilds: new Map([
 		[data.guild.general.id, data.guild.general],
 		[data.guild.fna.id, data.guild.fna],
@@ -42,6 +43,7 @@ const discord = {
 	application: {
 		id: "684280192553844747"
 	},
+	// @ts-ignore - ignore channels, because my data dump is missing random properties
 	channels: new Map([
 		[data.channel.general.id, data.channel.general],
 		[data.channel.updates.id, data.channel.updates],
@@ -126,10 +128,19 @@ file._actuallyUploadDiscordFileToMxc = function(url, res) { throw new Error(`Not
 
 	require("./addbot.test")
 	require("../src/db/orm.test")
+	require("../src/web/server.test")
+	require("../src/web/routes/download-discord.test")
+	require("../src/web/routes/download-matrix.test")
+	require("../src/web/routes/guild.test")
+	require("../src/web/routes/guild-settings.test")
+	require("../src/web/routes/info.test")
+	require("../src/web/routes/link.test")
+	require("../src/web/routes/log-in-with-matrix.test")
 	require("../src/discord/utils.test")
 	require("../src/matrix/kstate.test")
 	require("../src/matrix/api.test")
 	require("../src/matrix/file.test")
+	require("../src/matrix/mreq.test")
 	require("../src/matrix/read-registration.test")
 	require("../src/matrix/txnid.test")
 	require("../src/d2m/actions/create-room.test")
@@ -145,6 +156,7 @@ file._actuallyUploadDiscordFileToMxc = function(url, res) { throw new Error(`Not
 	require("../src/d2m/converters/remove-reaction.test")
 	require("../src/d2m/converters/thread-to-announcement.test")
 	require("../src/d2m/converters/user-to-mxid.test")
+	require("../src/m2d/event-dispatcher.test")
 	require("../src/m2d/converters/diff-pins.test")
 	require("../src/m2d/converters/event-to-message.test")
 	require("../src/m2d/converters/emoji.test")
@@ -155,11 +167,4 @@ file._actuallyUploadDiscordFileToMxc = function(url, res) { throw new Error(`Not
 	require("../src/discord/interactions/permissions.test")
 	require("../src/discord/interactions/privacy.test")
 	require("../src/discord/interactions/reactions.test")
-	require("../src/web/server.test")
-	require("../src/web/routes/download-discord.test")
-	require("../src/web/routes/download-matrix.test")
-	require("../src/web/routes/guild.test")
-	require("../src/web/routes/guild-settings.test")
-	require("../src/web/routes/link.test")
-	require("../src/web/routes/log-in-with-matrix.test")
 })()

test/web.js

@@ -5,6 +5,10 @@ const {SnowTransfer} = require("snowtransfer")
 const assert = require("assert").strict
 const domino = require("domino")
 const {extend} = require("supertape")
+const {reg} = require("../src/matrix/read-registration")
+
+const {AppService} = require("@cloudrac3r/in-your-element")
+const defaultAs = new AppService(reg)
 
 /**
  * @param {string} html
@@ -39,7 +43,7 @@ class Router {
 		for (const method of ["get", "post", "put", "patch", "delete"]) {
 			this[method] = function(url, handler) {
 				const key = `${method} ${url}`
-				this.routes.set(`${key}`, handler)
+				this.routes.set(key, handler)
 			}
 		}
 	}
@@ -49,7 +53,7 @@ class Router {
 	 * @param {string} inputUrl
 	 * @param {{event?: any, params?: any, body?: any, sessionData?: any, api?: Partial<import("../src/matrix/api")>, snow?: {[k in keyof SnowTransfer]?: Partial<SnowTransfer[k]>}, createRoom?: Partial<import("../src/d2m/actions/create-room")>, createSpace?: Partial<import("../src/d2m/actions/create-space")>, headers?: any}} [options]
 	 */
-	test(method, inputUrl, options = {}) {
+	async test(method, inputUrl, options = {}) {
 		const url = new URL(inputUrl, "http://a")
 		const key = `${method} ${options.route || url.pathname}`
 		/* c8 ignore next */
@@ -67,7 +71,8 @@ class Router {
 			req.headers["content-type"] = "application/json"
 		}
 
-		return this.routes.get(key)(Object.assign(event, {
+		try {
+			return await this.routes.get(key)(Object.assign(event, {
 				__is_event__: true,
 				method: method.toUpperCase(),
 				path: `${url.pathname}${url.search}`,
@@ -91,12 +96,17 @@ class Router {
 					}
 				}
 			}))
+		} catch (error) {
+			// Post-process error data
+			defaultAs.app.options.onError(error)
+			throw error
+		}
 	}
 }
 
 const router = new Router()
 
-passthrough.as = {router}
+passthrough.as = {router, on() {}, options: defaultAs.app.options}
 
 module.exports.router = router
 module.exports.test = test