Commit Graph

462 Commits

Author SHA1 Message Date
perf3ct ffad8c4561 feat(tests): fix ocr_retry issues in tests 2025-07-02 21:30:36 +00:00
perf3ct ae33b25133 feat(tests): fix ocr_retry issues in tests 2025-07-02 19:16:13 +00:00
perf3ct 3c4e06fa77 feat(tests): fix ocr_retry issues in tests 2025-07-02 18:48:26 +00:00
perf3ct 8d27132e12 feat(client): impelement retry all button on Document Management page 2025-07-02 18:26:22 +00:00
perf3ct 8cea916abf feat(server): allow also completed documents to be retried 2025-07-02 18:15:41 +00:00
perf3ct 6bdd6f4a56 feat(server): implement DEBUG environment variable 2025-07-02 17:57:57 +00:00
Jon Fuller befd46ad76 Merge pull request #87 from readur/feat/retry-functionality
feat(server/client): implement retry functionality for both successfu…
2025-07-02 09:57:54 -07:00
Jon Fuller 64b91952bc Merge pull request #78 from readur/renovate/material-ui-monorepo
fix(deps): update material-ui monorepo to v7.2.0
2025-07-02 09:45:41 -07:00
perf3ct 9a7488f1d2 fix(tests): resolve broken integration test for ocr retry, and fix frontend tests 2025-07-02 16:39:41 +00:00
renovate[bot] 7de0796082 fix(deps): update material-ui monorepo to v7.2.0 2025-07-02 16:25:47 +00:00
Jon Fuller 372826de30 Merge pull request #88 from readur/renovate/tokio-1.x-lockfile
fix(deps): update rust crate tokio to v1.46.0
2025-07-02 09:24:31 -07:00
renovate[bot] 8fe1963d43 fix(deps): update rust crate tokio to v1.46.0 2025-07-02 12:12:50 +00:00
perf3ct 70ac57c3cd fix(server/client): try to resolve more tests for the new retry functionality 2025-07-02 04:52:44 +00:00
perf3ct a0420251fb fix(server): resolve test for retry issues 2025-07-02 03:22:41 +00:00
perf3ct 68aa492a96 fix(server): resolve NUMERIC db type and f64 rust type 2025-07-02 02:26:11 +00:00
perf3ct 05c04f242e feat(tests): create unit tests for retry 2025-07-02 02:13:58 +00:00
perf3ct 2006907d2f fix(client): resolve incorrect import on documentService 2025-07-02 00:21:36 +00:00
perf3ct 9004633d68 fix(tests): resolve broken frontend tests due to retry functionality 2025-07-02 00:18:21 +00:00
Jon Fuller 999cbbb965 Merge pull request #86 from readur/renovate/aws-sdk-rust-monorepo
fix(deps): update rust crate aws-sdk-s3 to v1.95.0
2025-07-01 17:14:35 -07:00
perf3ct 4279823268 fix(tests): resolve using test config in ocr retry tests 2025-07-02 00:14:19 +00:00
perf3ct d4b57d2ae0 feat(server/client): implement retry functionality for both successful and failed documents 2025-07-02 00:06:47 +00:00
renovate[bot] f0a25e2c77 fix(deps): update rust crate aws-sdk-s3 to v1.95.0 2025-07-01 22:42:54 +00:00
Jon Fuller 5c2c25b6e1 Merge pull request #85 from readur/feat/greatly-improve-webdav-1
feat(webdav): track directory etags
2025-07-01 15:41:21 -07:00
perf3ct 0e89a7a930 fix(tests): resolve broken test compilation 2025-07-01 22:14:53 +00:00
perf3ct a381cdd12c feat(webdav): also fix the parser to include directories, and add tests 2025-07-01 22:03:06 +00:00
perf3ct c1dbd06df2 feat(tests): add unit tests for new webdav functionality 2025-07-01 21:39:31 +00:00
Jon Fuller c164a4666f Merge pull request #39 from readur/renovate/vite-7.x
chore(deps): update dependency vite to v7
2025-07-01 14:24:22 -07:00
Jon Fuller 92b4ba7829 Merge pull request #83 from readur/renovate/reqwest-0.x-lockfile
fix(deps): update rust crate reqwest to v0.12.22
2025-07-01 14:24:13 -07:00
Jon Fuller fc510af644 Merge pull request #84 from readur/renovate/aws-sdk-rust-monorepo
fix(deps): update rust crate aws-config to v1.8.1
2025-07-01 14:24:03 -07:00
perf3ct 92b21350db feat(webdav): track directory etags
✅ Core Optimizations Implemented

  1. 📊 New Database Schema: Added webdav_directories table to track
directory ETags, file counts, and metadata
  2. 🔍 Smart Directory Checking: Before deep scans, check directory
ETags with lightweight Depth: 0 PROPFIND requests
  3. ΓÜí Skip Unchanged Directories: If directory ETag matches, skip the
entire deep scan
  4. 🗂️ N-Depth Subdirectory Tracking: Recursively track all
subdirectories found during scans
  5. 🎯 Individual Subdirectory Checks: When parent unchanged, check
each known subdirectory individually

  🚀 Performance Benefits

  Before: Every sync = Full Depth: infinity scan of entire directory
treeAfter:
  - First sync: Full scan + directory tracking setup
  - Subsequent syncs: Quick ETag checks → skip unchanged directories
entirely
  - Changed directories: Only scan the specific changed subdirectories

  📁 How It Works

  1. Initial Request: PROPFIND Depth: 0 on /Documents → get directory
ETag
  2. Database Check: Compare with stored ETag for /Documents
  3. If Unchanged: Check each known subdirectory (/Documents/2024,
/Documents/Archive) individually
  4. If Changed: Full recursive scan + update all directory tracking
data
2025-07-01 21:22:16 +00:00
renovate[bot] ba1cdfc9b7 fix(deps): update rust crate aws-config to v1.8.1 2025-07-01 20:56:42 +00:00
Jon Fuller 36dcbaed98 Merge pull request #82 from readur/feat/merge-document-management-tabs
feat(client): combine the low confidence ocr tab with the bulk cleanup…
2025-07-01 13:55:10 -07:00
perf3ct 6a23a407bf feat(client): update swagger ui endpoints 2025-07-01 20:54:45 +00:00
perf3ct c18fde6fc1 feat(client): update swagger ui endpoints 2025-07-01 19:00:29 +00:00
renovate[bot] 13a2b39bcd fix(deps): update rust crate reqwest to v0.12.22 2025-07-01 18:55:12 +00:00
perf3ct ee4841c027 feat(client): combine the low confience ocr tab with the bulk cleanup tab, and move debug menu 2025-07-01 18:52:28 +00:00
perf3ct 893ba331f5 fix(client): try to fix the lint issues on the debug page 2025-07-01 02:25:24 +00:00
Jon Fuller 83b046acc6 Merge pull request #81 from readur/feat/debug-page
feat(debug/pdf): debug page actually works, add ocrmypdf
2025-06-30 18:09:59 -07:00
perf3ct f22673bbc3 fix(client): try to fix the lint issues on the debug page 2025-07-01 01:05:40 +00:00
perf3ct df281f3b26 feat(pdf): implement ocrmypdf to extract text from PDFs 2025-07-01 00:56:48 +00:00
Jon Fuller 706e20f35c Merge branch 'main' into feat/debug-page 2025-06-30 17:19:31 -07:00
perf3ct 231f88f038 feat(debug): debug page actually works and does something 2025-07-01 00:15:48 +00:00
perf3ct 0052032772 fix(pdf): resolve PDF wordcount error 2025-07-01 00:10:49 +00:00
Jon Fuller b307cfd509 Merge pull request #79 from readur/renovate/playwright-monorepo
chore(deps): update dependency @playwright/test to v1.53.2
2025-06-30 16:31:26 -07:00
Jon Fuller 1fca815485 Merge pull request #80 from readur/renovate/reqwest-0.x-lockfile
fix(deps): update rust crate reqwest to v0.12.21
2025-06-30 16:31:16 -07:00
Jon Fuller c848e899f3 Merge pull request #76 from readur/feat/wrangle-ignored-files
feat(server/client): wrangle ignored files and failed files
2025-06-30 16:31:00 -07:00
perf3ct 830f9d0b38 feat(server): mark documents with 0 words as failed, and fix webdav unit tests 2025-06-30 22:43:25 +00:00
perf3ct 69279344cb fix(tests): fix documents tests 2025-06-30 21:56:21 +00:00
perf3ct 49c34aac5a fix(tests): update frontend tests for new FailedDocument view 2025-06-30 21:20:34 +00:00
renovate[bot] 918a1850cb fix(deps): update rust crate reqwest to v0.12.21 2025-06-30 20:13:56 +00:00