[2.14] ingest.discovery_page #31

New issue

Closed

opened 2026-06-13 23:54:34 +02:00 by dernbu · 0 comments

dernbu commented

2026-06-13 23:54:34 +02:00

Owner

Parent

#2 — depends on [2.13]

Schema

CREATE TABLE ingest.discovery_page (
  id              bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  run_id          bigint NOT NULL REFERENCES ingest.discovery_run ON DELETE CASCADE,
  pass_number     integer NOT NULL,
  page_number     integer NOT NULL,
  raw_response    jsonb  NULL,
  parsed_count    integer NOT NULL DEFAULT 0,
  response_checksum text NULL,
  retrieved_at    timestamptz NOT NULL DEFAULT now()
);

Rules: Raw response is JSONB for LawNet and bytea/text for eLitigation (use jsonb with a note that eLitigation stores encoded HTML).

Done when

FK cascades on run delete
Composite index on (run_id, pass_number, page_number) for checkpoint queries

## Parent #2 — depends on [2.13] ## Schema ```sql CREATE TABLE ingest.discovery_page ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, run_id bigint NOT NULL REFERENCES ingest.discovery_run ON DELETE CASCADE, pass_number integer NOT NULL, page_number integer NOT NULL, raw_response jsonb NULL, parsed_count integer NOT NULL DEFAULT 0, response_checksum text NULL, retrieved_at timestamptz NOT NULL DEFAULT now() ); ``` **Rules:** Raw response is JSONB for LawNet and bytea/text for eLitigation (use `jsonb` with a note that eLitigation stores encoded HTML). ## Done when - FK cascades on run delete - Composite index on `(run_id, pass_number, page_number)` for checkpoint queries

dernbu closed this issue

2026-06-14 05:22:59 +02:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

dernbu/lawnet-scraper#31

No description provided.

Rows
Columns