Commit Graph

617 Commits

Author SHA1 Message Date
perf3ct 4f0497ba74
feat(tests): fix ocr_retry issues in tests 2025-07-02 21:48:01 +00:00
perf3ct dec4551fbd
feat(tests): fix ocr_retry issues in tests 2025-07-02 21:30:36 +00:00
perf3ct 86904205ed
feat(tests): fix ocr_retry issues in tests 2025-07-02 19:16:13 +00:00
perf3ct e15aa801c6
feat(tests): fix ocr_retry issues in tests 2025-07-02 18:48:26 +00:00
perf3ct 50e3f42484
feat(client): impelement retry all button on Document Management page 2025-07-02 18:26:22 +00:00
perf3ct dd963d0ecc
feat(server): allow also completed documents to be retried 2025-07-02 18:15:41 +00:00
perf3ct 8ed8701d5b
feat(server): implement DEBUG environment variable 2025-07-02 17:57:57 +00:00
Jon Fuller 09dab5c8c7
Merge pull request #87 from readur/feat/retry-functionality
feat(server/client): implement retry functionality for both successfu…
2025-07-02 09:57:54 -07:00
Jon Fuller f1bec120ea
Merge pull request #78 from readur/renovate/material-ui-monorepo
fix(deps): update material-ui monorepo to v7.2.0
2025-07-02 09:45:41 -07:00
perf3ct 34dc9a38b8
fix(tests): resolve broken integration test for ocr retry, and fix frontend tests 2025-07-02 16:39:41 +00:00
renovate[bot] 32e804043c
fix(deps): update material-ui monorepo to v7.2.0 2025-07-02 16:25:47 +00:00
Jon Fuller 90e186d49c
Merge pull request #88 from readur/renovate/tokio-1.x-lockfile
fix(deps): update rust crate tokio to v1.46.0
2025-07-02 09:24:31 -07:00
renovate[bot] adcc5101bb
fix(deps): update rust crate tokio to v1.46.0 2025-07-02 12:12:50 +00:00
perf3ct dadd1c3c0e
fix(server/client): try to resolve more tests for the new retry functionality 2025-07-02 04:52:44 +00:00
perf3ct 2d702f1c07
fix(server): resolve test for retry issues 2025-07-02 03:22:41 +00:00
perf3ct 0f3cb12c7a
fix(server): resolve NUMERIC db type and f64 rust type 2025-07-02 02:26:11 +00:00
perf3ct d6c0f412b5
feat(tests): create unit tests for retry 2025-07-02 02:13:58 +00:00
perf3ct 4e8c96a518
fix(client): resolve incorrect import on documentService 2025-07-02 00:21:36 +00:00
perf3ct c878a5a860
fix(tests): resolve broken frontend tests due to retry functionality 2025-07-02 00:18:21 +00:00
Jon Fuller 4975549e5f
Merge pull request #86 from readur/renovate/aws-sdk-rust-monorepo
fix(deps): update rust crate aws-sdk-s3 to v1.95.0
2025-07-01 17:14:35 -07:00
perf3ct 6af9e5e4a0
fix(tests): resolve using test config in ocr retry tests 2025-07-02 00:14:19 +00:00
perf3ct e4faf2cfd2
feat(server/client): implement retry functionality for both successful and failed documents 2025-07-02 00:06:47 +00:00
renovate[bot] 57b1d921a6
fix(deps): update rust crate aws-sdk-s3 to v1.95.0 2025-07-01 22:42:54 +00:00
Jon Fuller 6f16ddc207
Merge pull request #85 from readur/feat/greatly-improve-webdav-1
feat(webdav): track directory etags
2025-07-01 15:41:21 -07:00
perf3ct c766ecc52e
fix(tests): resolve broken test compilation 2025-07-01 22:14:53 +00:00
perf3ct c242a84326
feat(webdav): also fix the parser to include directories, and add tests 2025-07-01 22:03:06 +00:00
perf3ct 590cad3197
feat(tests): add unit tests for new webdav functionality 2025-07-01 21:39:31 +00:00
Jon Fuller aa3d0c3b72
Merge pull request #39 from readur/renovate/vite-7.x
chore(deps): update dependency vite to v7
2025-07-01 14:24:22 -07:00
Jon Fuller 4a7e79540b
Merge pull request #83 from readur/renovate/reqwest-0.x-lockfile
fix(deps): update rust crate reqwest to v0.12.22
2025-07-01 14:24:13 -07:00
Jon Fuller 73e5c20747
Merge pull request #84 from readur/renovate/aws-sdk-rust-monorepo
fix(deps): update rust crate aws-config to v1.8.1
2025-07-01 14:24:03 -07:00
perf3ct fdc240fa5b
feat(webdav): track directory etags
✅ Core Optimizations Implemented

  1. 📊 New Database Schema: Added webdav_directories table to track
directory ETags, file counts, and metadata
  2. 🔍 Smart Directory Checking: Before deep scans, check directory
ETags with lightweight Depth: 0 PROPFIND requests
  3. ΓÜí Skip Unchanged Directories: If directory ETag matches, skip the
entire deep scan
  4. 🗂️ N-Depth Subdirectory Tracking: Recursively track all
subdirectories found during scans
  5. 🎯 Individual Subdirectory Checks: When parent unchanged, check
each known subdirectory individually

  🚀 Performance Benefits

  Before: Every sync = Full Depth: infinity scan of entire directory
treeAfter:
  - First sync: Full scan + directory tracking setup
  - Subsequent syncs: Quick ETag checks → skip unchanged directories
entirely
  - Changed directories: Only scan the specific changed subdirectories

  📁 How It Works

  1. Initial Request: PROPFIND Depth: 0 on /Documents → get directory
ETag
  2. Database Check: Compare with stored ETag for /Documents
  3. If Unchanged: Check each known subdirectory (/Documents/2024,
/Documents/Archive) individually
  4. If Changed: Full recursive scan + update all directory tracking
data
2025-07-01 21:22:16 +00:00
renovate[bot] 9234323f9c
fix(deps): update rust crate aws-config to v1.8.1 2025-07-01 20:56:42 +00:00
Jon Fuller 1e655be542
Merge pull request #82 from readur/feat/merge-document-management-tabs
feat(client): combine the low confidence ocr tab with the bulk cleanup…
2025-07-01 13:55:10 -07:00
perf3ct a2ea446e16
feat(client): update swagger ui endpoints 2025-07-01 20:54:45 +00:00
perf3ct d2d7adb0ad
feat(client): update swagger ui endpoints 2025-07-01 19:00:29 +00:00
renovate[bot] 37f3b3be75
fix(deps): update rust crate reqwest to v0.12.22 2025-07-01 18:55:12 +00:00
Jon Fuller a88f387aeb
Merge branch 'main' into feat/multiple-ocr-languages 2025-07-01 11:53:42 -07:00
perf3ct 680cf5f526
feat(client): combine the low confience ocr tab with the bulk cleanup tab, and move debug menu 2025-07-01 18:52:28 +00:00
perf3ct a8a6047214
fix(client): try to fix the lint issues on the debug page 2025-07-01 02:25:24 +00:00
Jon Fuller 1695692729
Merge pull request #81 from readur/feat/debug-page
feat(debug/pdf): debug page actually works, add ocrmypdf
2025-06-30 18:09:59 -07:00
perf3ct 8deca105b1
fix(client): try to fix the lint issues on the debug page 2025-07-01 01:05:40 +00:00
perf3ct f7018575d8
feat(pdf): implement ocrmypdf to extract text from PDFs 2025-07-01 00:56:48 +00:00
Jon Fuller 59e80a1b92
Merge branch 'main' into feat/debug-page 2025-06-30 17:19:31 -07:00
perf3ct 2a1eeeda50
feat(debug): debug page actually works and does something 2025-07-01 00:15:48 +00:00
perf3ct f26ab1e367
fix(pdf): resolve PDF wordcount error 2025-07-01 00:10:49 +00:00
Jon Fuller 799fd88f41
Merge pull request #79 from readur/renovate/playwright-monorepo
chore(deps): update dependency @playwright/test to v1.53.2
2025-06-30 16:31:26 -07:00
Jon Fuller c8fa0b2ac7
Merge pull request #80 from readur/renovate/reqwest-0.x-lockfile
fix(deps): update rust crate reqwest to v0.12.21
2025-06-30 16:31:16 -07:00
Jon Fuller 75e6594f1a
Merge pull request #76 from readur/feat/wrangle-ignored-files
feat(server/client): wrangle ignored files and failed files
2025-06-30 16:31:00 -07:00
perf3ct dd90e48fd2
feat(server): mark documents with 0 words as failed, and fix webdav unit tests 2025-06-30 22:43:25 +00:00
perf3ct bf073132a1
fix(tests): fix documents tests 2025-06-30 21:56:21 +00:00