Commit Graph

339 Commits

Author SHA1 Message Date
perf3ct 4396ce312b
feat(tests): completely redo the test_helpers to actually be helpers and not hinderers 2025-08-01 21:21:15 +00:00
perf3ct aad6036b4c
feat(tests): bring back the test helpers 2025-08-01 20:34:42 +00:00
perf3ct 862c36aa72
feat(storage): further support the s3 storage backend 2025-08-01 17:57:09 +00:00
perf3ct 6624fc57fb
feat(queue): have the queue service use new storage service correctly 2025-08-01 17:33:59 +00:00
perf3ct abd55ef419
feat(storage): abstract storage to also support s3, along with local filesystem still 2025-08-01 04:33:08 +00:00
perf3ct 3ad0dd3600
feat(server): set up new storage service 2025-08-01 02:50:38 +00:00
perf3ct 68ceb1f9cb
feat(storage): implement s3 for storage 2025-08-01 00:27:13 +00:00
perfectra1n 0fbb106668 fix(tests): resolve typescript compilation and test compilation errors 2025-07-30 20:03:13 -07:00
perf3ct 32983c3fba feat(server): implement #106 for per-user watch directories 2025-07-31 00:10:10 +00:00
perf3ct e62e73a249 fix(tests): have the updated integration tests at least compile 2025-07-30 04:21:16 +00:00
perf3ct 7da99cd992 feat(server): implement websockets over sse 2025-07-30 02:04:44 +00:00
perf3ct d7a0a1f294 feat(server): do a *much* better job at determining file types thanks to infer rust package 2025-07-29 21:28:33 +00:00
perf3ct 96ea060450 fix(sync): fix issue where the webdav tasks weren't inserting source metadata into db 2025-07-29 21:27:40 +00:00
perf3ct e938ae3bd1 feat(debug): add some really...really noisy debugging for WebDAV URL paths
fadsf
2025-07-29 04:10:09 +00:00
perf3ct 8f1f502cc4 feat(tests): mom, take a picture, the tests pass 2025-07-29 02:28:39 +00:00
perf3ct b3e6630bd1 feat(tests): tests are mostly working now 2025-07-29 00:47:02 +00:00
perf3ct cfeb6c5c93 feat(tests): wrap the tests so that even if they fail, they still close their db connections 2025-07-28 18:15:08 +00:00
perf3ct c37014f924 feat(tests): work on resolving tests that don't pass given the large rewrite 2025-07-28 04:13:14 +00:00
perf3ct 319c1521c1 fix(labels): allow for nullable user_id on label fetch 2025-07-27 20:42:55 +00:00
perf3ct 24269ea513 feat(tests): resolve duplicated test coverage for webdav functionality 2025-07-27 20:36:54 +00:00
perf3ct 023d424293 feat(server/client): I have no words, hopefully this lesser abstraction and webdav tracking works now 2025-07-27 19:29:45 +00:00
perf3ct 2c0ef814d9 feat(client/server): implement better source sync output 2025-07-27 05:02:13 +00:00
perf3ct 1939c4aec3 feat(webdav): create dedicated module for managing the mess that is WebDAV XML responses 2025-07-26 23:37:09 +00:00
perf3ct b6c1d18d88 fix(db): try to increase the connection timeout to 30s to grab db connections? 2025-07-24 22:32:23 +00:00
perf3ct bb379f19dc fix(tests): resolve issues with compilation in tests 2025-07-24 17:02:38 +00:00
perf3ct 9699973196 feat(sync): create further comprehensive tests for testing our new "smart sync" functionality for managing directory etags 2025-07-22 20:32:59 +00:00
perf3ct 8c9d5dc12b feat(webdav): move to tracking all subdirectory etags 2025-07-22 19:59:25 +00:00
perf3ct 3c09e80d6d feat(webdav): also decrease log noise in webdav 2025-07-22 19:32:30 +00:00
perf3ct bf1fa0712f feat(ingestion): change noisy log-lines to debug instead of info 2025-07-22 19:31:09 +00:00
perf3ct 79d1d6ad19 fix(upload): resolve upload hanging with comprehensive error handling and timeouts 2025-07-22 03:46:54 +00:00
perf3ct 65f42c2cd7 fix(ocr): use proper failure reasons to avoid constraint violations in failed_documents table 2025-07-21 20:43:37 +00:00
perf3ct 0a2020d249 feat(tests): update default credentials for test db 2025-07-21 20:31:07 +00:00
perf3ct 95e5186b78 fix(ci): add missing languages to ci 2025-07-21 19:50:06 +00:00
perf3ct 6fcae1e752 fix(errors): swap the http response when deleting a user is restricted 2025-07-21 19:34:33 +00:00
perf3ct aac32ea64a feat(dev): let's just implement an entire error system while we're at it, since the tests are already broken 2025-07-20 03:38:13 +00:00
perf3ct 45ec99a031 feat(ocr): get rid of managing TESSDATA_PREFIX 2025-07-20 02:23:06 +00:00
perf3ct 1895fe6c5c feat(api): implement custom error type for document interactions 2025-07-19 22:28:39 +00:00
perf3ct 3e098dc085 fix(startup): also remove this now unused function check 2025-07-19 22:11:40 +00:00
perf3ct 1aac25631c feat(api): provide better error responses when manipulating documents 2025-07-19 22:10:28 +00:00
perf3ct 5a3c040942 fix(startup): remove annoying function check output from startup 2025-07-19 21:41:26 +00:00
perf3ct 29689fda4b feat(tests): try to fix the TESSDATA environment variable in tests, again 2025-07-18 21:15:16 +00:00
perf3ct 6b1a1eba14 feat(tests): resolve test issues to do with integration tests 2025-07-18 21:00:42 +00:00
perf3ct 7d89a711cb fix(tests): resolve issue in languageselector component for frontend tests 2025-07-18 20:39:41 +00:00
perf3ct 77db933b58 fix(tests): also fix the test utils 2025-07-18 20:03:38 +00:00
perf3ct 089d6c2853 fix(tests): resolve issues in unit tests due to dep changes 2025-07-17 16:09:10 +00:00
perf3ct 03bb4bf87d fix(client): resolve issues with showing user settings on debug pge 2025-07-15 16:20:30 +00:00
perf3ct ccc3bc2ce4 feat(ocr): use ocrmypdf and pdftotext to get OCR layer if it already exists 2025-07-15 15:59:29 +00:00
perf3ct 901942ae74 fix(client): also resolve missing thumbnails 2025-07-15 15:57:14 +00:00
perf3ct a3f33140ee feat(dev): drop pdf_extract in favor of ocrmypdf 2025-07-15 14:50:17 +00:00
perf3ct 862eb3217a fix(tests): resolve issues in integration tests for the new multiple ocr languages 2025-07-14 21:28:55 +00:00
perf3ct 7317fd5ebb Merge branch 'feat/multiple-ocr-languages' of https://github.com/readur/readur into feat/multiple-ocr-languages 2025-07-14 19:33:51 +00:00
perf3ct 849c9f91c7 feat(lang): update backend to support multiple languages at the same time during OCR 2025-07-14 19:33:43 +00:00
Jon Fuller f0e39d155e Merge branch 'main' into feat/multiple-ocr-languages 2025-07-14 11:29:46 -07:00
perf3ct 721a348888 feat(dev): update references of readur to newer versions 2025-07-14 17:17:24 +00:00
perf3ct 6165148e4d feat(ocr): gracefully handle problematic PDFs in all the ways, create tests so that it doesn't happen again 2025-07-14 16:36:32 +00:00
perf3ct 469ca29f5c fix(tests): resolve some broken e2e tests? 2025-07-14 01:39:30 +00:00
perf3ct 1bf4a66754 fix(tests): resolve issue in compilation of tests due to multiple ocr languages 2025-07-13 17:26:06 +00:00
perf3ct e6fd8424d2 fix(dev): merge main into feature 2025-07-13 17:15:59 +00:00
perf3ct ac49d8c2c9 fix(tests): fix urls used in test 2025-07-12 23:27:27 +00:00
perf3ct e736d485ee fix(webdav): fix incorrect concatenation logic when building URLs 2025-07-12 23:26:19 +00:00
perf3ct 3fbf941364 fix(webdav): make sure to normalize URL for double slashes, trailing slashes, etc. 2025-07-12 23:15:14 +00:00
perfectra1n 4a039eac82 feat(client): add new frontend page for admins to view running config settings 2025-07-12 14:06:09 -07:00
perfectra1n 9e143649d4 fix(upload): resolve issue with Axum not having config values set 2025-07-12 14:04:54 -07:00
perf3ct e16fd1d420 feat(server): rename queue-failed endpoint 2025-07-11 21:28:23 +00:00
perf3ct b31e1a672d feat(server): gracefully manage requeue requests for the same document 2025-07-11 21:27:12 +00:00
perf3ct 0b9b935334 fix(server): don't log usernames to log file, I guess 2025-07-11 20:05:56 +00:00
perf3ct 979078b5ac fix(tests): also resolve document struct in this unit test 2025-07-11 17:50:49 +00:00
perf3ct 8c90c5c3c3 feat(tests): do tests pass now? 2025-07-11 00:39:12 +00:00
perf3ct fb831e9624 feat(server): implement unit tests for source metadata extraction 2025-07-10 22:02:41 +00:00
perf3ct 305c6f1fb1 feat(server): show source metadata EVEN better 2025-07-10 21:51:30 +00:00
perf3ct ea43f79a90 feat(server): show source metadata better, and implement tests 2025-07-10 21:40:16 +00:00
perf3ct b7f1522b4a feat(server): updating the watcher.rs file to preserve source metadata 2025-07-10 21:18:08 +00:00
perf3ct 4521cc5ac6 feat(client): show new file metadata fields on the client 2025-07-10 21:07:35 +00:00
perf3ct 0465777890 feat(client): show more fields for Documents 2025-07-10 21:02:15 +00:00
perf3ct 96c47af2c0 feat(server/client): make sure that the documents endpoint isn't broken 2025-07-10 19:57:25 +00:00
perf3ct 438e79c441 fix(tests): no way, all the integration tests pass now 2025-07-10 01:38:55 +00:00
perf3ct 29800bdd1f fix(tests): resolve integration test response format 2025-07-09 20:10:36 +00:00
perf3ct 17f486a8b7 fix(server/client): rename document_id to id in DocumentUploadResponse, again 2025-07-09 01:40:50 +00:00
perf3ct f2a050458b fix(stats): create new get_queue_statistics function to avoid conflicts 2025-07-09 00:27:43 +00:00
perf3ct f0dc0669bd debug(tests): add some debug lines to see why CI is upset 2025-07-08 22:32:32 +00:00
perf3ct a6f2b6df09 fix(stats): try to fix the stats extraction, again 2025-07-08 21:18:21 +00:00
perf3ct e628b0d4d5 fix(server): resolve incorrect document failure titles 2025-07-08 20:24:52 +00:00
perf3ct 12bdeab503 fix(stats): try to fix stats export, again again again 2025-07-08 20:21:46 +00:00
perf3ct 2a59651fb9 fix(stats): try to fix stats export, again again 2025-07-08 20:16:33 +00:00
perf3ct a7e9f75eab fix(stats): try to fix stats export, again 2025-07-08 20:03:55 +00:00
perf3ct 03555ed756 fix(tests): fix the crazy metrics collection issue 2025-07-08 16:52:23 +00:00
perf3ct 58b8a71404 fix(tests): and resolve missing endpoint 2025-07-08 04:37:33 +00:00
perf3ct faf3299119 debug(tests): add some debug lines to see why CI is upset 2025-07-08 00:03:30 +00:00
perf3ct a4b9626616 fix(web_upload): resolve issue that caused files that were uploaded via the web, to not be added to the queue 2025-07-07 19:28:08 +00:00
perf3ct b356017484 feat(server): implement better error checking for sources 2025-07-07 19:10:45 +00:00
perf3ct c6089dd1b2 feat(webdav): resolve failing etag unit tests 2025-07-05 21:16:15 +00:00
perf3ct 26c618f984 feat(webdav): resolve failing etag unit tests 2025-07-05 19:47:21 +00:00
perf3ct 5b21c87675 feat(tests): move integration and unit tests to correct locations 2025-07-04 19:37:43 +00:00
perf3ct fbd7d561c3 fix(tests): binary and tests at least compile now 2025-07-04 19:07:53 +00:00
Jon Fuller ac6b4a522f Merge pull request #96 from readur/feat/deduplicate-test-utils-1
feat(tests): deduplicate test functionalities
2025-07-04 09:12:40 -07:00
perf3ct 545046509f fix(server): fix axum groups 2025-07-04 03:07:28 +00:00
perf3ct 922478d995 fix(server): resolve compilation errors due to splitting up the large files 2025-07-04 03:06:29 +00:00
perf3ct 497b34ce0a fix(server): resolve type issues and functions for compilation issues 2025-07-04 00:53:32 +00:00
perf3ct 51fb3a7e48 fix(tests): resolve broken test utils 2025-07-04 00:31:53 +00:00
perf3ct 0e84993afa fix(server): resolve import issues 2025-07-03 23:58:11 +00:00