git-mirrors/continuwuity
1
0
Fork 0
mirror of https://forgejo.ellis.link/continuwuation/continuwuity.git synced 2025-06-28 00:54:51 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
2850 commits 37 branches 20 tags 19 MiB
Commit graph

76 commits

Author SHA1 Message Date
Jason Volk
865b5d7241 reorganize database crate. 
split database Cork into unit.

split database migrations from mod.rs

Signed-off-by: Jason Volk <jason@zemos.net>
 2024-04-09 15:26:45 -04:00
Jason Volk
fe91ce0601 add conf items for rocksdb repair and read-only modes. 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-04-09 15:26:45 -04:00
strawberry
f0a0704a93 slight adjustments, remove some explicit annotations 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-04-05 22:15:16 -04:00
Jason Volk
114324e26c remove write_lock around rocksdb 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-03-30 22:06:18 -04:00
strawberry
48d1a3af3c resolve the last few relevant pedantic clippy lints 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-30 22:06:18 -04:00
strawberry
868976a149 use chain_width 60 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-26 22:24:24 -04:00
Jason Volk
6b1933914d add file listing to database abstraction. 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-03-26 22:24:24 -04:00
strawberry
7bd56765ef fix some more pedantic clippy lints 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-26 22:24:24 -04:00
strawberry
101cb34f9a make rocksdb_recovery_mode a u8, document it 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
Jason Volk
331c0b37cd add conf item for alternate rocksdb recovery modes. 2024-03-23 01:49:27 -04:00
strawberry
3bc2af7d26 resolve and add even more pedantic clippy lints 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
strawberry
9d0b647911 resolve couple pedantic clippy lints, remove unnecessary qualifications 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
strawberry
9f7431c08f remove rocksdb prefix extractor 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
Jason Volk
3f60365cc6 add write buffer corking using rocksdb manual_wal_flush. 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-03-23 01:49:27 -04:00
strawberry
2ca357e44c use unwrap_or_default if timestamp conversion fails 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
strawberry
f9a60bf48b make database_backup_path a PathBuf 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
strawberry
f7e8054a00 fix lints 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
Jason Volk
fa942aedd7 add database backup with admin commands 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-03-23 01:49:27 -04:00
Jason Volk
e9d67ecb53 reconfigure and optimize rocksdb options. 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-23 01:49:27 -04:00
Jason Volk
544c38341b add sync() to db abstraction for fsync(2). 2024-03-23 01:49:27 -04:00
Jason Volk
d4cfee4e71 add rocksdb env to options. keep options in engine state. 
Signed-off-by: Jason Volk <jason@zemos.net>
 2024-03-23 01:49:27 -04:00
Jason Volk
6fe0ea05b8 add remove_batch with transaction to database abstraction. 
adjusted to make building sqlite happy again

Signed-off-by: Jason Volk <jason@zemos.net>
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-19 00:17:41 -04:00
strawberry
b4fa306a20 remove rocksdb optimize_level_style_compaction 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-19 00:17:41 -04:00
strawberry
a3c24bcc31 add rocksdb_bottommost_compression_level config option 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-16 00:09:48 -04:00
strawberry
9a9f7b9c54 add RocksDB rocksdb_compression_level and rocksdb_bottommost_compression 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-16 00:09:48 -04:00
Jason Volk
10336f9af6 use WriteBatchWithTransaction for batched insertions. 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-16 00:09:48 -04:00
strawberry
fcca352795 apply Read/Write/FlushOptions where available for future usage 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-16 00:09:48 -04:00
strawberry
20c089b7ed simplify/tune rocksdb options, config option for compression algo 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-16 00:09:48 -04:00
strawberry
0db3a43d1f add multi_get (multi_get_cf) for rocksdb 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-16 00:09:48 -04:00
strawberry
fee6cad778 s/rocksdb/rust_rocksdb, fix copy paste errors 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-10 13:03:28 -04:00
strawberry
d2060c8647 add config option for max RocksDB LOG files 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-08 12:51:21 -05:00
strawberry
7bfb86a851 set default rocksdb log level to error 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-08 12:51:21 -05:00
strawberry
7cbb0139f2 implement flush() and cleanup() for rocksdb 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-08 12:51:21 -05:00
strawberry
f419c64aca add rustfmt.toml, format entire codebase 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-08 12:51:21 -05:00
strawberry
90d90c46da use get_physical for only physical core count 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-08 12:51:21 -05:00
strawberry
0352ea7dda add rocksdb parallelism threads config option 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-08 12:51:21 -05:00
strawberry
c97483dbd3 fix rocksdb-specific lints 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-03 22:20:37 -05:00
strawberry
e2c7afe69c go through a ton of pedantic clippy lints 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-03 22:20:37 -05:00
strawberry
b26c593a2a adjust max rocksdb LOG files based on hard drive or SSD option 
keeps 3 LOG files at most if on hard drive, else keeps 20

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-03-03 22:20:37 -05:00
strawberry
82f10214b3 remove all unused lifetimes 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-02-28 13:56:19 -05:00
strawberry
24625e9659 resolve nightly rust warnings 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-02-26 00:40:15 -05:00
strawberry
36bc21193c oops forgot import 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-02-20 07:06:57 -05:00
strawberry
3d26210eac more useful database logging (compaction, time taken to load) 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-02-20 07:06:57 -05:00
strawberry
fae00e5136 remove rocksdb_max_open_files option 
default for RocksDB is -1 and conduwuit already raises the
soft and hard nofile limits at startup.

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-01-25 19:28:33 -05:00
strawberry
59c7f93656 fix more clippy lints (part 1) 
Signed-off-by: strawberry <strawberry@puppygock.gay>
 2024-01-17 14:26:26 -05:00
strawberry
5e641e2886 move default rocksdb logging from info to warn 
info is still logging useless database statistics and metrics that will
accumulate, less I/O activity.

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2023-12-25 17:07:24 -05:00
strawberry
d9d1ce3cb5 fix refering uninit static, use upstream rocksdb again, don't compile debug info 
even rust-analyzer themselves don't compile with debug info because it
makes cargo checks and builds in general a lot longer. helps speed
things up and we mainly care about panics/stacktraces and compiler
errors.

https://github.com/rust-lang/rust-analyzer/blob/master/Cargo.toml#L12-L15

upstream rust-rocksdb is active again finally

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2023-12-03 01:00:23 -05:00
strawberry
f62f641545 add rocksdb configurable options and tweaks, logging improvements, exp. room v11 support 
split out the spinning disk focused options into a configurable option, current
conduwuit users are NVMe/SSDs anyways so those options are just hindering performance.

rocksdb logging builds up overtime with no cleanup or anything, adds support for
configuring the amount of logging, size of files, log rotate, etc.
fixes https://gitlab.com/girlbossceo/conduwuit/-/issues/1

misc conduit logging improvements for help debugging issues and maybe a future feature

experimental Room V11 support from https://gitlab.com/famedly/conduit/-/merge_requests/562

Signed-off-by: strawberry <strawberry@puppygock.gay>
 2023-12-02 21:30:06 -05:00
girlbossceo
a3d219e401 tweak rocksdb, use direct io 
Signed-off-by: girlbossceo <june@girlboss.ceo>
 2023-10-21 12:18:41 -04:00
Charles Hall
9fb8498067
relax recovery mode 2023-07-26 15:32:36 -07:00