Construction Timelapse — Session Memory

File này là nhật ký phát triển của dự án. Mỗi khi có thay đổi lớn, cập nhật phía dưới cùng (entry mới nhất).

Tổng quan dự án

Repo Gogs: https://git.k9tech.space/kingkong/timelapse-2
Token: a6224640fb6f576f6cba46f0e1646500c87226b6 (Personal Access Token)
Stack: Next.js 14 + NestJS + PostgreSQL (Drizzle) + Redis + Python
Package manager: Bun (KHÔNG dùng npm — workspace protocol không hoạt động với nexus registry)
Registry: Luôn thêm --registry https://registry.npmjs.org/ khi cài package

Kiến trúc hệ thống

MacOS (code) → push Gogs → webhook → Pi 4 (deploy)
                                          ↓
                    ┌─ postgres (docker, port 5432)
                    ├─ redis    (docker, port 6379)
                    ├─ api-server    (NestJS, port 3001)
                    ├─ web-dashboard (Next.js, port 3000)
                    ├─ worker        (Node, stub)
                    └─ simulator     (Python, tự động)

Simulator = container giả lập device (claim → approve → heartbeat + upload ảnh).

Workspace config

"workspaces": ["apps/api-server", "apps/web-dashboard", "apps/worker", "packages/*"]

device-agent và simulator là Python — KHÔNG đưa vào workspaces
Dùng @shared/types, không phải @shared-types

Các lỗi đã gặp và cách fix

#	Lỗi	Fix
1	`@shared-types` không hợp lệ	Đổi thành `@shared/types`
2	Python packages trong npm workspaces	Loại khỏi workspaces
3	`npm install` không hoạt động	`bun install --registry https://registry.npmjs.org/`
4	`bun add` với nexus registry	Luôn `--registry https://registry.npmjs.org/`
5	`db` không export được	Import từ `database.module.ts`
6	`experimentalDecorators` thiếu	Thêm vào `packages/config/tsconfig.node.json`
7	TSC output vào `src/`	Cleanup `.js/.d.ts/*.js.map` + fix outDir

Database (Drizzle Schema)

File: apps/api-server/src/db/schema.ts
16 tables: organizations, projects, users, memberships, sessions, magic_links, devices, device_heartbeats, captures, videos, video_jobs, commands, alert_rules, alerts, audit_logs, activity_logs
Enums: org_status, project_status, device_status (có pending), capture_status, video_status, alert_severity, alert_type, alert_state, user_role, command_result_status

Auth — Đã hoàn thành ✅

Feature	Chi tiết
Register/Login/Refresh/Logout	JWT + refresh token + bcrypt
Magic links	Email-based passwordless
Google OAuth	Stub (redirect + callback)
JWT guard	`@UseGuards(JwtAuthGuard)`
Dashboard auth flow	`storeTokens()`, `clearAuth()`, `apiFetch()`
Protected routes	`pages/_app.tsx` redirect if no token

RBAC — Đã hoàn thành ✅

Role	Quyền
`org_admin`	Full: org/project/member CRUD
`project_manager`	Tạo/sửa project
`viewer`	Read-only

API Endpoints (tổng hợp)

Auth (`/v1/auth/`)

POST register, POST login, POST refresh, POST logout, GET me, POST magic-link

Orgs (`/v1/orgs/`)

POST, GET, GET /:id/members, POST /:id/members, PATCH /:id/members/:userId, DELETE /:id/members/:userId

Projects (`/v1/projects/`)

POST, GET, GET /:id, PATCH /:id, DELETE /:id — đều có JwtAuthGuard

Devices (`/v1/devices/`)

Endpoint	Auth	Mô tả
`POST claim`	none	Device xin claim code
`GET claim/:code/status`	none	Poll trạng thái, nhận bootstrap API key
`GET pending`	JWT (org_admin)	List pending devices
`POST pending/:code/approve`	JWT (org_admin)	Approve + assign project
`POST pending/:code/approve-internal`	X-Internal-Key	Simulator tự approve
`POST pending/:code/reject`	JWT (org_admin)	Reject
`POST :deviceId/heartbeat`	X-API-Key	Device heartbeat (bcrypt verify)
`GET`	JWT	List all devices
`GET stats`	JWT	Dashboard stats
`GET :id`	JWT	Device detail
`GET :id/heartbeats`	JWT	Heartbeat history

Captures (`/v1/captures/`)

Endpoint	Auth	Mô tả
`POST upload`	X-API-Key	Upload ảnh (multipart), lưu local `/uploads/captures/`
`POST register`	X-API-Key	Đăng ký metadata capture
`GET project/:projectId`	JWT	List captures theo project
`GET device/:deviceId`	JWT	List captures theo device
`GET :id`	JWT	Chi tiết capture

Realtime (Socket.io `/realtime`)

Events: device.heartbeat, device.status.changed, capture.uploaded, alert.opened Dashboard subscribe theo project room: project:X

Device Provisioning Flow

Device (Pi+DSLR)                        Server API                       Dashboard
      │                                    │                                │
      │── POST /v1/devices/claim ──────────>│                                │
      |<─ { claimCode: "ABC123" } ──────────│                                │
      │                                    │── WS realtime ─────────────────>│ (new pending)
      │                                    │                                │
      │                                    │<── POST /v1/devices/pending/:code/approve
      │                                    │     (org_admin JWT)            │
      │── GET /v1/devices/claim/:code/status│                                │
      |<─ { status: approved, apiKey } ─────│                                │
      │                                    │                                │
      │── POST :id/heartbeat (X-API-Key) ──>│                                │
      │── POST /v1/captures/upload (file) ──>│                                │
      │                                    │── capture.uploaded WS ─────────>│

Simulator Container (`apps/simulator`)

Mục đích: Giả lập device kết nối thật vào server (POC mà không cần phần cứng thật).

Config: env var trong docker-compose.yml

SIM_SERVER_URL=http://api-server:3001
SIM_DEVICE_COUNT=2          # số device giả lập
SIM_CAPTURE_INTERVAL=30     # giây giữa mỗi capture
SIM_INTERNAL_KEY=...         # key để approve-internal endpoint

Luồng tự động:

Claim device → nhận claim code
Auto-approve qua /approve-internal (X-Internal-Key auth)
Poll /claim/:code/status → nhận bootstrap API key
Heartbeat loop (30s)
Capture loop (30s): Pillow tạo JPEG nhẹ (640×480, quality 60) → upload lên /v1/captures/upload

Files:

apps/simulator/simulator/main.py
apps/simulator/Dockerfile
apps/simulator/requirements.txt

Lưu ý: Cần ít nhất 1 project trong DB để simulator tự assign. Chạy seed trước.

Device Agent (Pi thật — `apps/device-agent`)

Chạy trên Pi 4 thật + DSLR:

agent/main.py — claim, poll, heartbeat, đọc thermal/disk metrics
systemd/timelapse-agent.service — systemd unit
scripts/install-agent.sh — Pi-only installer (kiểm tra EUID)
Config: /etc/timelapse/agent.env
Key lưu: /boot/timelapse/agent.key (chmod 600)
Metadata: /boot/timelapse/agent.json

Mock Device CLI (`apps/device-agent/mock-device-cli.py`)

Dùng cho test nhanh trên MacOS (không cần Docker):

# Claim device
python3 apps/device-agent/mock-device-cli.py claim \
  --server http://localhost:3001 --count 5

# Heartbeat cho seed devices
python3 apps/device-agent/mock-device-cli.py heartbeat \
  --server http://localhost:3001 --demo --interval 5

Gogs API

# List repos
curl -H "Authorization: token <TOKEN>" https://git.k9tech.space/api/v1/user/repos

# Create webhook
curl -X POST -H "Authorization: token <TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"type":"gogs","active":true,"config":{"url":"http://<pi>:9000/webhook","content_type":"json"},"events":["push"]}' \
  https://git.k9tech.space/api/v1/repos/<user>/<repo>/hooks

# Push với token
git remote set-url origin "https://<TOKEN>@git.k9tech.space/kingkong/timelapse-2.git"
git push -u origin main

Docker Compose (Pi 4)

docker compose up -d postgres redis
docker compose up -d --build api-server web-dashboard worker simulator

Volumes:

postgres_data — PostgreSQL data
redis_data — Redis data
timelapse_uploads — Captured images (gắn vào api-server /uploads/captures)

Env mới:

UPLOAD_DIR=/uploads/captures (api-server)
UPLOAD_BASE_URL (api-server — URL prefix cho ảnh, mặc định http://localhost:3001)
SIM_INTERNAL_KEY (api-server + simulator)

Cài đặt trên Pi 4

cd /opt/timelapse
git pull

# Rebuild
docker compose up -d --build

# Logs
docker compose logs -f simulator
docker compose logs -f api-server

# DB migration (cần chạy 1 lần sau khi schema thay đổi)
docker compose exec api-server bun run migrate

Lệnh nhanh (MacOS dev)

# Cài deps
bun install --registry https://registry.npmjs.org/

# Dev
cd apps/api-server && bun run dev
cd apps/web-dashboard && bun run dev

# Build verify
cd apps/api-server && bun run build
cd apps/web-dashboard && bun run build

# Test heartbeat
curl -X POST http://localhost:3001/v1/devices/test-device/heartbeat \
  -H "Content-Type: application/json" \
  -H "X-API-Key: dev-key-123" \
  -d '{"deviceId":"test-device","status":"online","storageFreeGb":50,"capturesToday":12,"firmwareVersion":"1.0.0"}'

TODO

DB migration + seed trên Pi (drizzle-kit push + tạo org/project seed)
Dashboard Pending Devices page — approve/reject/assign device trên UI
Worker thực — BullMQ capture queue processing (thumbnail, validate)
S3/MinIO thay local storage
Video generation — ghép ảnh thành video
Google OAuth thật
Alert rules engine
Dashboard captures gallery — xem ảnh đã upload
Pi 4 IP — cập nhật Gogs webhook URL

2026-03-26 — Device config form + image preview gallery

Backend:

Static file serving: GET /uploads/captures/* → file system (NestExpressApplication)
UPLOAD_BASE_URL env: prefix URL cho ảnh, mặc định http://localhost:3001
Captures API: mọi response đều có imageUrl field (full URL)
PATCH /v1/devices/:id/config — update device config (JWT auth)
UpdateDeviceConfigDto: captureInterval/resolution/quality/storage/heartbeat/nightMode/timezone

Dashboard (devices/[id].tsx):

Image preview gallery: lưới 10 cột, hover overlay hiện time/ISO/aperture
Lightbox: click thumbnail → xem ảnh lớn + metadata (time/res/size/exposure/ISO/aperture)
Auto-refresh captures mỗi 30s
Config editor: form cho 8 fields + toggle WiFi-only, Save button với trạng thái
Heartbeat timeline: bảng 10 dòng gần nhất (temp/storage/captures/network)
Device info bar: serial/firmware/lastseen/project
Commit: 290f089 feat: device config form + image preview gallery + static file serving

Trên Pi: rebuild để thấy ảnh placeholder từ simulator. Cần set UPLOAD_BASE_URL = URL thật của Pi trong .env:

UPLOAD_BASE_URL=http://<pi-ip>:3001

Nhật ký phát triển

2026-03-26 — Simulator container + Captures upload API

Thêm CapturesModule: POST /v1/captures/upload (multipart, lưu local /uploads/captures/), POST /v1/captures/register (metadata only), các GET endpoints (JWT auth)
Upload lưu vào /uploads/captures/{projectId}/{YYYY-MM-DD}/{id}.jpg, tính SHA256 checksum
Realtime event capture.uploaded emit khi upload thành công
Tạo apps/simulator/ — container Python tự động: claim → approve-internal → heartbeat + Pillow JPEG upload
Thêm POST /v1/devices/pending/:code/approve-internal endpoint (X-Internal-Key auth) cho simulator
docker-compose.yml: thêm timelapse_uploads volume, SIM_INTERNAL_KEY env, simulator service
api-server Dockerfile: tạo /uploads/captures directory
Commit: f3b3091 feat: add simulator container + captures upload API

2026-03-?? — Realtime WebSocket + Device Provisioning

Socket.io gateway tại src/realtime/gateways/realtime.gateway.ts
Events: device.heartbeat, device.status.changed, capture.uploaded, alert.opened
Dashboard subscribe theo project room: subscribe:project
Device provisioning: claim → approve → bootstrap API key (bcrypt hashed)
approveDeviceInternal() auto-assign first project nếu không có projectId
Commit: device provisioning flow

2026-03-?? — RBAC + Auth hoàn chỉnh

Register/Login/Refresh/Logout + JWT + bcrypt
Dashboard login/register/protected routes
Orgs CRUD + Membership management (org_admin/project_manager/viewer)
Projects CRUD với role enforcement
Commit: RBAC + dashboard auth

2026-03-?? — Device agent (Pi thật)

apps/device-agent/agent/main.py — claim, poll, heartbeat loop, thermal/disk metrics
systemd/timelapse-agent.service
scripts/install-agent.sh (Pi-only, kiểm tra EUID)
mock-device-cli.py — claim + heartbeat simulator cho MacOS dev
Commit: 515619f, 636ae53

2026-03-?? — Setup ban đầu

Scaffold NestJS api-server + Next.js dashboard + worker stub
Docker Compose (postgres, redis, 3 services)
Drizzle schema với 16 tables
Realtime module (stub)
Commit đầu tiên

memory.md 14 KB Permalink Verlauf Originalformat