dernbu/lawnet-scraper
1
0
Fork 0
Code Issues 8 Pull requests Projects Releases Packages Wiki Activity Actions

[6] eLitigation ingestion #6

New issue
Closed
opened 2026-06-13 18:59:46 +02:00 by dernbu · 0 comments
dernbu commented 2026-06-13 18:59:46 +02:00
Owner
Copy link

Objective

Implement complete, resumable eLitigation discovery and artifact ingestion.

Scope

  • Implement listing, detail, and PDF clients.
  • Parse source HTML with lxml.
  • Implement ascending-date discovery, stabilization, checkpoints, and staged publication.
  • Preserve exact and unusual slugs and duplicate listing discrepancies.
  • Detect page-not-found, maintenance, and invalid PDF responses even when HTTP status is 200.
  • Store raw pages, judgment fragments, typed metadata, and PDFs.
  • Add fixture-backed tests for source edge cases.

Dependencies

Depends on #3 and #4.

Completion criteria

  • A mocked complete crawl publishes transactionally.
  • Invalid HTML and PDF responses are rejected correctly.
  • Interrupted work resumes.
  • Existing unavailable records remain visible.

Closure

Complete all child issues, or merge a feature-branch pull request that resolves this issue.

Detailed requirements: docs/plans/high-level-plan.md, Phase 6.

## Objective Implement complete, resumable eLitigation discovery and artifact ingestion. ## Scope - Implement listing, detail, and PDF clients. - Parse source HTML with `lxml`. - Implement ascending-date discovery, stabilization, checkpoints, and staged publication. - Preserve exact and unusual slugs and duplicate listing discrepancies. - Detect page-not-found, maintenance, and invalid PDF responses even when HTTP status is 200. - Store raw pages, judgment fragments, typed metadata, and PDFs. - Add fixture-backed tests for source edge cases. ## Dependencies Depends on #3 and #4. ## Completion criteria - A mocked complete crawl publishes transactionally. - Invalid HTML and PDF responses are rejected correctly. - Interrupted work resumes. - Existing unavailable records remain visible. ## Closure Complete all child issues, or merge a feature-branch pull request that resolves this issue. Detailed requirements: `docs/plans/high-level-plan.md`, Phase 6.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
temp
feat/53-rename-lawnet-routes
push-xrsnvuvuvsul
feat/37-bff-storage-frontend
l
feat/49-pg-bff-storage
push-zzvxlnkylqsp
feat/quick-and-dirty
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
dernbu/lawnet-scraper#6
No description provided.