continuwuity/src
Benjamin Lee 20a54aacd6 factor search tokenization out into a function
This ensures that the tokenization algorithm will remain in sync between
querying, indexing, and deindexing. The existing code had slightly
different behavior for querying, because it did not discard words with
>50 bytes. This was inconsequential, because >50 byte tokens are never
present in the index.

Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-06-12 14:04:47 -04:00
..
admin remove pfps/avatars and display names upon deactivation 2024-06-11 01:19:18 -04:00
api fix dropped events in search 2024-06-12 14:04:47 -04:00
core log client/remote IP address on various routes tracing calls 2024-06-10 21:04:51 -04:00
database Fix use-self 2024-06-09 20:06:50 +00:00
main Fix use-self 2024-06-09 20:06:50 +00:00
router log client/remote IP address on various routes tracing calls 2024-06-10 21:04:51 -04:00
service factor search tokenization out into a function 2024-06-12 14:04:47 -04:00