[37] Storage-backed LawNet BFF and frontend #47

dernbu · 2026-06-14T07:36:47+02:00

dernbu commented

2026-06-14 07:36:47 +02:00

Summary

Implements a storage-backed LawNet BFF and frontend with both file-system (mock) and PostgreSQL storage backends.

What was done

Flyway V2 migration (V2__lawnet_catalog.sql): Creates the catalog schema with lawnet_record, lawnet_rendition, and lawnet_artifact tables including indexes and constraints.
Storage contract (src/db/storage.ts): Defines a LawnetStorage interface with listRecords, getRecord, and getArtifact methods.
File-system storage (src/db/fs-storage.ts): Reads mock data from services/web/mock/records.json and artifact files from services/web/mock/artifacts/. Includes 4 sample records with HTML and PDF artifacts.
PostgreSQL storage (src/db/pg-storage.ts): Queries the catalog tables using pg. Handles BIGINT → number conversion for size_bytes.
Storage factory (src/db/factory.ts): Selects storage backend via LAWNET_STORAGE environment variable. Defaults to file-system.
BFF API routes:
- GET /api/lawnet/records?limit=&offset= — paginated catalog listing
- GET /api/lawnet/records/:id — single record detail with renditions and artifacts
- GET /api/lawnet/artifacts/:id — artifact streaming with correct content-type and security headers
Frontend pages:
- /lawnet — dashboard with a sortable table showing citation, title, court, date, and rendition badges. Includes pagination controls.
- /lawnet/:id — detail page showing metadata (court, decision, judges, parties, catchwords), rendition list with View HTML / Sanitized links, PDF downloads with file sizes, and an artifact details table.
Navigation: Added nav bar to __root.tsx with links to Home and Catalog.
Tests: 15 tests covering file-system storage (list/get/artifact), storage factory, and health endpoint.

How it works

Set LAWNET_STORAGE=filesystem (default) to use mock JSON + local HTML/PDF files. No PostgreSQL or Docker needed.
Set LAWNET_STORAGE=postgres and DATABASE_URL to use the PostgreSQL catalog tables.

Screenshots

Catalog page: /lawnet
Detail page: /lawnet/1

Testing

# Run with file-system storage (no Docker needed)
cd services/web && pnpm dev

# API tests
curl http://localhost:3000/api/lawnet/records?limit=2&offset=0
curl http://localhost:3000/api/lawnet/records/1
curl http://localhost:3000/api/lawnet/artifacts/3

# Unit tests
cd services/web && pnpm test

Closes: #37

## Summary Implements a storage-backed LawNet BFF and frontend with both file-system (mock) and PostgreSQL storage backends. ### What was done - **Flyway V2 migration** (`V2__lawnet_catalog.sql`): Creates the `catalog` schema with `lawnet_record`, `lawnet_rendition`, and `lawnet_artifact` tables including indexes and constraints. - **Storage contract** (`src/db/storage.ts`): Defines a `LawnetStorage` interface with `listRecords`, `getRecord`, and `getArtifact` methods. - **File-system storage** (`src/db/fs-storage.ts`): Reads mock data from `services/web/mock/records.json` and artifact files from `services/web/mock/artifacts/`. Includes 4 sample records with HTML and PDF artifacts. - **PostgreSQL storage** (`src/db/pg-storage.ts`): Queries the `catalog` tables using `pg`. Handles BIGINT → number conversion for `size_bytes`. - **Storage factory** (`src/db/factory.ts`): Selects storage backend via `LAWNET_STORAGE` environment variable. Defaults to file-system. - **BFF API routes**: - `GET /api/lawnet/records?limit=&offset=` — paginated catalog listing - `GET /api/lawnet/records/:id` — single record detail with renditions and artifacts - `GET /api/lawnet/artifacts/:id` — artifact streaming with correct content-type and security headers - **Frontend pages**: - `/lawnet` — dashboard with a sortable table showing citation, title, court, date, and rendition badges. Includes pagination controls. - `/lawnet/:id` — detail page showing metadata (court, decision, judges, parties, catchwords), rendition list with View HTML / Sanitized links, PDF downloads with file sizes, and an artifact details table. - **Navigation**: Added nav bar to `__root.tsx` with links to Home and Catalog. - **Tests**: 15 tests covering file-system storage (list/get/artifact), storage factory, and health endpoint. ### How it works 1. Set `LAWNET_STORAGE=filesystem` (default) to use mock JSON + local HTML/PDF files. No PostgreSQL or Docker needed. 2. Set `LAWNET_STORAGE=postgres` and `DATABASE_URL` to use the PostgreSQL catalog tables. ### Screenshots Catalog page: `/lawnet` Detail page: `/lawnet/1` ### Testing ```bash # Run with file-system storage (no Docker needed) cd services/web && pnpm dev # API tests curl http://localhost:3000/api/lawnet/records?limit=2&offset=0 curl http://localhost:3000/api/lawnet/records/1 curl http://localhost:3000/api/lawnet/artifacts/3 # Unit tests cd services/web && pnpm test ``` Closes: #37

dernbu added 1 commit

2026-06-14 07:36:47 +02:00

[2] Storage-backed LawNet BFF and frontend

CI / typecheck (pull_request) Successful in 58s

Details

CI / test (pull_request) Successful in 56s

Details

CI / check-compose (pull_request) Successful in 4s

Details

CI / lint (pull_request) Failing after 57s

Details

591a1350d6

- Add Flyway V2 migration for LawNet catalog tables (records, renditions, artifacts)
- Add file-system storage backend with mock JSON, HTML, and PDF artifacts
- Add PostgreSQL storage backend using pg
- Add storage factory selecting backend via LAWNET_STORAGE env var
- Add BFF API routes: GET /api/lawnet/records, GET /api/lawnet/records/:id, GET /api/lawnet/artifacts/:id
- Add frontend dashboard page with server-side pagination table
- Add frontend detail page with metadata, renditions, and artifact links
- Add navigation and styling
- Add tests for file-system storage and storage factory

dernbu force-pushed feat/37-bff-storage-frontend from 591a1350d6

CI / typecheck (pull_request) Successful in 58s

Details

CI / test (pull_request) Successful in 56s

Details

CI / check-compose (pull_request) Successful in 4s

Details

CI / lint (pull_request) Failing after 57s

Details

to 92e4e57998

CI / lint (pull_request) Successful in 57s

Details

CI / typecheck (pull_request) Successful in 56s

Details

CI / test (pull_request) Successful in 55s

Details

CI / check-compose (pull_request) Successful in 4s

Details

2026-06-14 08:08:07 +02:00

Compare

dernbu reviewed

2026-06-14 08:21:02 +02:00

.env.example Outdated

					
				@ -9,1 +9,4 @@

				INTERNAL_API_TOKEN=

				# Storage backend: "filesystem" (default, uses mock JSON + local files) or "postgres"

				LAWNET_STORAGE=filesystem

I think the environment variable shouldn't just be between storage and the file system/Postgres. It should point to the path of the JSON file or the storage directory. If no directory is provided, it falls back to Postgres. This is better and lets us change the mocks later.

services/flyway/sql/V2__lawnet_catalog.sql Outdated

					
				@ -0,0 +24,4 @@

				COMMENT ON COLUMN catalog.lawnet_record.n_citation IS 'Neutral citation string, e.g. [2025] SGHC 1.';

				COMMENT ON COLUMN catalog.lawnet_record.source_metadata IS 'Raw source metadata payload from LawNet API responses.';

				CREATE TABLE catalog.lawnet_rendition (

I'm wondering why we have a separate table for the rendition and the record. Can we flatten it? If not, can you give a reason why we can't? Let's think of the pros and cons. I'd like to combine these tables if possible and there are no big costs.

services/web/src/db/pg-storage.ts Outdated

					
				@ -0,0 +4,4 @@

				  LawnetListResult,

				  LawnetArtifactContent,

				} from "./storage";

				import pg from "pg";

What is this import, just import thr component directly

dernbu force-pushed feat/37-bff-storage-frontend from 92e4e57998

CI / lint (pull_request) Successful in 57s

Details

CI / typecheck (pull_request) Successful in 56s

Details

CI / test (pull_request) Successful in 55s

Details

CI / check-compose (pull_request) Successful in 4s

Details

to 133cfa848a

CI / lint (pull_request) Successful in 58s

Details

CI / typecheck (pull_request) Successful in 59s

Details

CI / test (pull_request) Successful in 57s

Details

CI / check-compose (pull_request) Successful in 4s

Details

2026-06-14 08:27:31 +02:00

Compare

dernbu reviewed

2026-06-14 08:33:08 +02:00

services/flyway/sql/V2__lawnet_storage.sql

					
				@ -0,0 +1,74 @@

				CREATE SCHEMA IF NOT EXISTS catalog;

I think we can have a better schema name than catalog.

services/web/src/routeTree.gen.ts

					
				@ -14,2 +17,3 @@

				import { Route as ApiLawnetCasesCaseIdRouteImport } from './routes/api/lawnet/cases.$caseId'

				import { Route as ApiLawnetArtifactsArtifactIdRouteImport } from './routes/api/lawnet/artifacts.$artifactId'

				const HealthRoute = HealthRouteImport.update({

Why delete API health? Restore the health checkpoint.

services/web/src/routes/api/lawnet/cases.ts

					
				@ -0,0 +3,4 @@

				import { createLawNetStorageFromEnv } from "../../../server/lawnet/config.server";

				import type { LawNetAvailability } from "../../../server/lawnet/types";

				export const Route = createFileRoute("/api/lawnet/cases")({

Why so many query parameters for this route? Can you keep it minimal? Regarding the plan, I think there are too many unnecessary ones.

services/web/src/server/lawnet/types.ts

					
				@ -0,0 +56,4 @@

				  filename: string;

				};

				export type ListLawNetCasesInput = {

Only accept limit and offset for list cases.

dernbu closed this pull request

2026-06-14 08:36:36 +02:00

CI / lint (pull_request) Successful in 58s

Details

CI / typecheck (pull_request) Successful in 59s

Details

CI / test (pull_request) Successful in 57s

Details

CI / check-compose (pull_request) Successful in 4s

Details

CI / merge-check*

Required

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

No reviewers

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

dernbu/lawnet-scraper!47

Rows
Columns