Remove Dagster pipeline and redesign calibration flow UI
This commit is contained in:
Ruslan Bakiev
12
Dockerfile
12
Dockerfile
@@ -26,14 +26,14 @@ RUN pip install --no-cache-dir -r requirements.txt
|
|||||||
# Copy project files
|
# Copy project files
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
# Create data directory
|
# Create data/config directories
|
||||||
RUN mkdir -p data/dagster_home
|
RUN mkdir -p data jetson/config
|
||||||
|
|
||||||
# Add /app to Python path so 'src' module can be imported
|
# Add /app to Python path so 'src' module can be imported
|
||||||
ENV PYTHONPATH=/app:$PYTHONPATH
|
ENV PYTHONPATH=/app:$PYTHONPATH
|
||||||
|
|
||||||
# Expose ports for API (8000) and Dagster UI (3000)
|
# Expose web UI/API port
|
||||||
EXPOSE 8000 3000
|
EXPOSE 8080
|
||||||
|
|
||||||
# Default command - start Dagster webserver
|
# Default command - run Jetson referee web app
|
||||||
CMD ["dagster", "dev", "-m", "dagster_project", "--host", "0.0.0.0", "--port", "3000"]
|
CMD ["python3", "jetson/main.py", "--port", "8080"]
|
||||||
|
|||||||
183
README.md
183
README.md
@@ -1,151 +1,72 @@
|
|||||||
# 🎾 Pickle - Pickleball Ball Tracking
|
# Pickle Vision
|
||||||
|
|
||||||
Система трекинга пикабольного мяча с автоматической детекцией корта и преобразованием координат в метры.
|
Real-time referee system for pickleball with 2 CSI cameras on Jetson.
|
||||||
|
|
||||||
## Что делает
|
## Current Product Scope
|
||||||
|
|
||||||
1. **Детекция корта** - автоматический поиск 4 углов корта (Roboflow модель)
|
Three tabs in the web UI:
|
||||||
2. **Детекция мяча** - поиск мяча на каждом кадре (YOLO v8)
|
|
||||||
3. **Трансформация координат** - преобразование пикселей в метры (homography)
|
|
||||||
4. **Визуализация** - видео с траекторией, графики, тепловая карта
|
|
||||||
|
|
||||||
## Быстрый старт
|
1. `Camera` - live feeds from both cameras.
|
||||||
|
2. `Calibration` - step-by-step court calibration flow (ROI -> lines -> intersections -> template match -> camera pose).
|
||||||
|
3. `Trajectory` - 3D court + ball trajectory + VAR overlay.
|
||||||
|
|
||||||
|
Dagster pipeline has been removed from active project runtime.
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
### Local (Jetson)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Запуск
|
python3 jetson/main.py --port 8080
|
||||||
docker-compose up -d
|
|
||||||
|
|
||||||
# Открыть Dagster UI
|
|
||||||
open http://localhost:3000
|
|
||||||
|
|
||||||
# Запустить пайплайн
|
|
||||||
docker exec pickle-dagster dagster asset materialize --select '*' -m dagster_project
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Структура пайплайна
|
Open: `http://<jetson-ip>:8080`
|
||||||
|
|
||||||
```
|
### Docker
|
||||||
1. extract_video_frames → Извлекает 100 кадров (с 10-й секунды)
|
|
||||||
↓
|
|
||||||
2. detect_court_keypoints → Находит 4 угла корта
|
|
||||||
↓ ↓
|
|
||||||
3. detect_ball_positions ←┘ Детектит мяч на всех кадрах
|
|
||||||
↓
|
|
||||||
4. compute_2d_coordinates → Преобразует пиксели в метры
|
|
||||||
↓
|
|
||||||
5. visualize_trajectory → Создает визуализации
|
|
||||||
```
|
|
||||||
|
|
||||||
## Результаты
|
|
||||||
|
|
||||||
После выполнения пайплайна в `data/`:
|
|
||||||
|
|
||||||
- **extract_video_frames.json** - метаданные видео
|
|
||||||
- **detect_court_keypoints.json** - координаты углов корта
|
|
||||||
- **detect_ball_positions.json** - позиции мяча в пикселях
|
|
||||||
- **compute_2d_coordinates.json** - позиции мяча в метрах
|
|
||||||
- **visualization.mp4** - видео с траекторией, кортом и координатами
|
|
||||||
- **frames/** - извлеченные кадры
|
|
||||||
- **ball_detections/** - кадры с найденным мячом
|
|
||||||
- **court_detection_preview.jpg** - превью с найденными углами корта
|
|
||||||
|
|
||||||
## Структура проекта
|
|
||||||
|
|
||||||
```
|
|
||||||
pickle/
|
|
||||||
├── dagster_project/ # Dagster пайплайн
|
|
||||||
│ ├── assets/ # 5 asset'ов пайплайна
|
|
||||||
│ └── io_managers/ # JSON IO manager
|
|
||||||
├── src/ # Основной код
|
|
||||||
│ ├── ball_detector.py # YOLO детекция
|
|
||||||
│ ├── court_calibrator.py # Калибровка корта
|
|
||||||
│ ├── ball_tracker.py # Трекинг
|
|
||||||
│ └── video_processor.py # Обработка видео
|
|
||||||
├── data/ # Результаты выполнения
|
|
||||||
├── DJI_0017.MP4 # Видео для обработки
|
|
||||||
├── docker-compose.yml
|
|
||||||
└── Dockerfile
|
|
||||||
```
|
|
||||||
|
|
||||||
## Конфигурация
|
|
||||||
|
|
||||||
Параметры в `dagster_project/assets/`:
|
|
||||||
|
|
||||||
- **video_extraction.py** - `start_sec=10`, `num_frames=100`
|
|
||||||
- **ball_detection.py** - `confidence_threshold=0.3`, slicing 320x320
|
|
||||||
- **coordinate_transform.py** - корт 13.4м × 6.1м
|
|
||||||
|
|
||||||
## Модели
|
|
||||||
|
|
||||||
- **Корт**: `ping-pong-paddle-ai-with-images/pickleball-court-p3chl-7tufp` (Roboflow)
|
|
||||||
- **Мяч**: `pickleball-detection-1oqlw/1` (Roboflow) → fallback на YOLOv8n
|
|
||||||
|
|
||||||
## Требования
|
|
||||||
|
|
||||||
- Docker & Docker Compose
|
|
||||||
- 4GB+ RAM
|
|
||||||
- Видео файл `DJI_0017.MP4` в корне проекта
|
|
||||||
|
|
||||||
## Docker команды
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Билд и запуск
|
docker-compose up --build
|
||||||
docker-compose up --build -d
|
|
||||||
|
|
||||||
# Логи
|
|
||||||
docker-compose logs -f
|
|
||||||
|
|
||||||
# Остановка
|
|
||||||
docker-compose down
|
|
||||||
|
|
||||||
# Выполнить пайплайн
|
|
||||||
docker exec pickle-dagster dagster asset materialize --select '*' -m dagster_project
|
|
||||||
|
|
||||||
# Выполнить один asset
|
|
||||||
docker exec pickle-dagster dagster asset materialize --select 'detect_ball_positions' -m dagster_project
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Dagster UI
|
Open: `http://localhost:8080`
|
||||||
|
|
||||||
http://localhost:3000
|
## Calibration Flow
|
||||||
|
|
||||||
Показывает:
|
Calibration is launched from the `Calibration` tab with `Run Calibration Flow`.
|
||||||
- Граф зависимостей между assets
|
|
||||||
- Логи выполнения
|
|
||||||
- История запусков
|
|
||||||
- Метаданные результатов
|
|
||||||
|
|
||||||
## Формат данных
|
Per camera, UI shows each step status:
|
||||||
|
|
||||||
**compute_2d_coordinates.json**:
|
1. Court ROI (green mask)
|
||||||
```json
|
2. White line segments detection
|
||||||
[
|
3. Segment merge into court lines
|
||||||
{
|
4. Line intersections
|
||||||
"frame": 6,
|
5. Template point match
|
||||||
"timestamp": 0.2,
|
6. Camera pose solve (PnP)
|
||||||
"pixel_x": 1234.5,
|
7. Geometry overlay
|
||||||
"pixel_y": 678.9,
|
|
||||||
"x_m": 5.67,
|
Output is stored in:
|
||||||
"y_m": 2.34,
|
|
||||||
"confidence": 0.85
|
- `jetson/config/cam0_calibration.json`
|
||||||
}
|
- `jetson/config/cam1_calibration.json`
|
||||||
]
|
|
||||||
|
## Main Runtime Files
|
||||||
|
|
||||||
|
- `jetson/main.py` - dual-camera loop, calibration, detection, VAR events
|
||||||
|
- `src/web/app.py` - Flask API + tab endpoints
|
||||||
|
- `src/web/templates/index.html` - 3-tab UI
|
||||||
|
- `src/calibration/camera_calibrator.py` - camera geometry / projection
|
||||||
|
- `src/physics/trajectory.py` - trajectory model
|
||||||
|
- `src/physics/event_detector.py` - close-call logic
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
Install from `requirements.txt`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -r requirements.txt
|
||||||
```
|
```
|
||||||
|
|
||||||
## Производительность
|
## Notes
|
||||||
|
|
||||||
- Извлечение кадров: ~1 сек
|
- Calibration is required before trajectory/VAR logic becomes fully useful.
|
||||||
- Детекция корта: ~1 сек
|
- Each camera is calibrated independently against known half-court geometry.
|
||||||
- Детекция мяча: ~6 сек (100 кадров, ~15 FPS)
|
|
||||||
- Трансформация координат: <1 сек
|
|
||||||
- Визуализация: ~1 сек
|
|
||||||
|
|
||||||
**Итого**: ~10 секунд на 100 кадров видео
|
|
||||||
|
|
||||||
## Стоимость
|
|
||||||
|
|
||||||
**$0** - всё работает локально в Docker, без облачных API
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
MIT
|
|
||||||
|
|||||||
@@ -1,32 +0,0 @@
|
|||||||
"""
|
|
||||||
Dagster project for pickleball ball tracking with 3D coordinates
|
|
||||||
"""
|
|
||||||
|
|
||||||
from dagster import Definitions
|
|
||||||
from dagster_project.assets import (
|
|
||||||
extract_video_frames,
|
|
||||||
detect_court_keypoints,
|
|
||||||
detect_ball_positions,
|
|
||||||
visualize_ball_on_court,
|
|
||||||
detect_net,
|
|
||||||
calibrate_camera_3d,
|
|
||||||
compute_ball_3d_coordinates,
|
|
||||||
create_interactive_viewer
|
|
||||||
)
|
|
||||||
from dagster_project.io_managers.json_io_manager import json_io_manager
|
|
||||||
|
|
||||||
defs = Definitions(
|
|
||||||
assets=[
|
|
||||||
extract_video_frames,
|
|
||||||
detect_court_keypoints,
|
|
||||||
detect_ball_positions,
|
|
||||||
visualize_ball_on_court,
|
|
||||||
detect_net,
|
|
||||||
calibrate_camera_3d,
|
|
||||||
compute_ball_3d_coordinates,
|
|
||||||
create_interactive_viewer
|
|
||||||
],
|
|
||||||
resources={
|
|
||||||
"json_io_manager": json_io_manager
|
|
||||||
}
|
|
||||||
)
|
|
||||||
@@ -1,21 +0,0 @@
|
|||||||
"""Dagster assets for pickleball tracking pipeline"""
|
|
||||||
|
|
||||||
from dagster_project.assets.video_extraction import extract_video_frames
|
|
||||||
from dagster_project.assets.court_detection import detect_court_keypoints
|
|
||||||
from dagster_project.assets.ball_detection import detect_ball_positions
|
|
||||||
from dagster_project.assets.visualization import visualize_ball_on_court
|
|
||||||
from dagster_project.assets.net_detection import detect_net
|
|
||||||
from dagster_project.assets.camera_calibration import calibrate_camera_3d
|
|
||||||
from dagster_project.assets.coordinate_transform_3d import compute_ball_3d_coordinates
|
|
||||||
from dagster_project.assets.interactive_viewer import create_interactive_viewer
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"extract_video_frames",
|
|
||||||
"detect_court_keypoints",
|
|
||||||
"detect_ball_positions",
|
|
||||||
"visualize_ball_on_court",
|
|
||||||
"detect_net",
|
|
||||||
"calibrate_camera_3d",
|
|
||||||
"compute_ball_3d_coordinates",
|
|
||||||
"create_interactive_viewer"
|
|
||||||
]
|
|
||||||
@@ -1,181 +0,0 @@
|
|||||||
"""Asset 3: Detect ball positions using YOLO"""
|
|
||||||
|
|
||||||
import sys
|
|
||||||
import cv2
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict, List, Optional
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
from tqdm import tqdm
|
|
||||||
|
|
||||||
# Add src to path to import existing ball detector
|
|
||||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="yolo",
|
|
||||||
description="Detect ball positions on all frames using YOLO"
|
|
||||||
)
|
|
||||||
def detect_ball_positions(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
extract_video_frames: Dict
|
|
||||||
) -> List[Dict]:
|
|
||||||
"""
|
|
||||||
Detect ball positions on all extracted frames
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- extract_video_frames: metadata from frame extraction
|
|
||||||
- data/frames/*.jpg: all extracted frames
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- data/detect_ball_positions.json
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
List of dicts with:
|
|
||||||
- frame: frame number
|
|
||||||
- x: pixel x coordinate (or None if not detected)
|
|
||||||
- y: pixel y coordinate (or None if not detected)
|
|
||||||
- confidence: detection confidence (0-1)
|
|
||||||
- diameter_px: estimated ball diameter in pixels
|
|
||||||
"""
|
|
||||||
from src.ball_detector import BallDetector
|
|
||||||
|
|
||||||
frames_dir = Path(extract_video_frames['frames_dir'])
|
|
||||||
num_frames = extract_video_frames['num_frames']
|
|
||||||
|
|
||||||
context.log.info(f"Initializing YOLO ball detector...")
|
|
||||||
|
|
||||||
# Initialize detector
|
|
||||||
detector = BallDetector(
|
|
||||||
model_id="pickleball-moving-ball/5",
|
|
||||||
confidence_threshold=0.3, # Lower threshold to catch more detections
|
|
||||||
slice_enabled=False # Disable slicing for faster Hosted API inference
|
|
||||||
)
|
|
||||||
|
|
||||||
context.log.info(f"Processing {num_frames} frames for ball detection...")
|
|
||||||
|
|
||||||
detections = []
|
|
||||||
frames_with_ball = 0
|
|
||||||
|
|
||||||
for i in tqdm(range(num_frames), desc="Detecting ball"):
|
|
||||||
frame_path = frames_dir / f"frame_{i:04d}.jpg"
|
|
||||||
|
|
||||||
if not frame_path.exists():
|
|
||||||
context.log.warning(f"Frame {i} not found: {frame_path}")
|
|
||||||
detections.append({
|
|
||||||
"frame": i,
|
|
||||||
"x": None,
|
|
||||||
"y": None,
|
|
||||||
"confidence": 0.0,
|
|
||||||
"diameter_px": None,
|
|
||||||
"bbox": None
|
|
||||||
})
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Load frame
|
|
||||||
frame = cv2.imread(str(frame_path))
|
|
||||||
|
|
||||||
# Detect ball
|
|
||||||
results = detector.detect(frame)
|
|
||||||
|
|
||||||
if results and len(results) > 0:
|
|
||||||
# Take highest confidence detection
|
|
||||||
ball = results[0]
|
|
||||||
|
|
||||||
# Calculate diameter from bbox
|
|
||||||
bbox = ball.get('bbox')
|
|
||||||
diameter_px = None
|
|
||||||
if bbox:
|
|
||||||
width = bbox[2] - bbox[0]
|
|
||||||
height = bbox[3] - bbox[1]
|
|
||||||
diameter_px = (width + height) / 2
|
|
||||||
|
|
||||||
detections.append({
|
|
||||||
"frame": i,
|
|
||||||
"x": float(ball['center'][0]),
|
|
||||||
"y": float(ball['center'][1]),
|
|
||||||
"confidence": float(ball['confidence']),
|
|
||||||
"diameter_px": float(diameter_px) if diameter_px else None,
|
|
||||||
"bbox": [float(b) for b in bbox] if bbox else None
|
|
||||||
})
|
|
||||||
|
|
||||||
frames_with_ball += 1
|
|
||||||
else:
|
|
||||||
# No detection
|
|
||||||
detections.append({
|
|
||||||
"frame": i,
|
|
||||||
"x": None,
|
|
||||||
"y": None,
|
|
||||||
"confidence": 0.0,
|
|
||||||
"diameter_px": None,
|
|
||||||
"bbox": None
|
|
||||||
})
|
|
||||||
|
|
||||||
# Log progress every 20 frames
|
|
||||||
if (i + 1) % 20 == 0:
|
|
||||||
detection_rate = frames_with_ball / (i + 1) * 100
|
|
||||||
context.log.info(f"Processed {i + 1}/{num_frames} frames. Detection rate: {detection_rate:.1f}%")
|
|
||||||
|
|
||||||
detection_rate = frames_with_ball / num_frames * 100
|
|
||||||
context.log.info(f"✓ Ball detected in {frames_with_ball}/{num_frames} frames ({detection_rate:.1f}%)")
|
|
||||||
|
|
||||||
# Save ALL detection images
|
|
||||||
_save_detection_preview(context, frames_dir, detections, num_preview=999)
|
|
||||||
|
|
||||||
return detections
|
|
||||||
|
|
||||||
|
|
||||||
def _save_detection_preview(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
frames_dir: Path,
|
|
||||||
detections: List[Dict],
|
|
||||||
num_preview: int = 5
|
|
||||||
):
|
|
||||||
"""Save preview images showing ball detections"""
|
|
||||||
run_id = context.run_id
|
|
||||||
preview_dir = Path(f"data/{run_id}/ball_detections")
|
|
||||||
preview_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
# Find first N frames with detections
|
|
||||||
detected_frames = [d for d in detections if d['x'] is not None][:num_preview]
|
|
||||||
|
|
||||||
for detection in detected_frames:
|
|
||||||
frame_num = detection['frame']
|
|
||||||
frame_path = frames_dir / f"frame_{frame_num:04d}.jpg"
|
|
||||||
|
|
||||||
if not frame_path.exists():
|
|
||||||
continue
|
|
||||||
|
|
||||||
frame = cv2.imread(str(frame_path))
|
|
||||||
|
|
||||||
# Draw ball
|
|
||||||
x, y = int(detection['x']), int(detection['y'])
|
|
||||||
cv2.circle(frame, (x, y), 8, (0, 0, 255), -1)
|
|
||||||
|
|
||||||
# Draw bbox if available
|
|
||||||
if detection['bbox']:
|
|
||||||
bbox = detection['bbox']
|
|
||||||
cv2.rectangle(
|
|
||||||
frame,
|
|
||||||
(int(bbox[0]), int(bbox[1])),
|
|
||||||
(int(bbox[2]), int(bbox[3])),
|
|
||||||
(0, 255, 0),
|
|
||||||
2
|
|
||||||
)
|
|
||||||
|
|
||||||
# Draw confidence
|
|
||||||
cv2.putText(
|
|
||||||
frame,
|
|
||||||
f"Conf: {detection['confidence']:.2f}",
|
|
||||||
(x + 15, y - 10),
|
|
||||||
cv2.FONT_HERSHEY_SIMPLEX,
|
|
||||||
0.6,
|
|
||||||
(255, 255, 255),
|
|
||||||
2
|
|
||||||
)
|
|
||||||
|
|
||||||
# Save preview
|
|
||||||
preview_path = preview_dir / f"detection_frame_{frame_num:04d}.jpg"
|
|
||||||
cv2.imwrite(str(preview_path), frame)
|
|
||||||
|
|
||||||
context.log.info(f"Saved {len(detected_frames)} preview images to {preview_dir}")
|
|
||||||
@@ -1,143 +0,0 @@
|
|||||||
"""Asset: Calibrate camera using court corners and net position"""
|
|
||||||
|
|
||||||
import cv2
|
|
||||||
import numpy as np
|
|
||||||
from typing import Dict
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="opencv",
|
|
||||||
description="Calibrate camera using cv2.solvePnP with court corners and net"
|
|
||||||
)
|
|
||||||
def calibrate_camera_3d(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
detect_court_keypoints: Dict,
|
|
||||||
detect_net: Dict
|
|
||||||
) -> Dict:
|
|
||||||
"""
|
|
||||||
Calibrate camera to get 3D pose using known 3D↔2D point correspondences
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- detect_court_keypoints: 4 court corners in pixels
|
|
||||||
- detect_net: 4 net corners in pixels
|
|
||||||
|
|
||||||
Known 3D points:
|
|
||||||
- Court: 13.4m × 6.1m rectangle at Z=0
|
|
||||||
- Net: height 0.914m at middle of court (Y=3.05m)
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
Camera calibration parameters
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict with:
|
|
||||||
- camera_matrix: [[fx, 0, cx], [0, fy, cy], [0, 0, 1]]
|
|
||||||
- rotation_vector: [rx, ry, rz]
|
|
||||||
- translation_vector: [tx, ty, tz]
|
|
||||||
- rotation_matrix: 3x3 matrix
|
|
||||||
- reprojection_error: RMS error in pixels
|
|
||||||
- calibrated: success flag
|
|
||||||
"""
|
|
||||||
# Get 2D points (pixels)
|
|
||||||
court_corners = np.array(detect_court_keypoints['corners_pixel'], dtype=np.float32)
|
|
||||||
net_corners = np.array(detect_net['net_corners_pixel'], dtype=np.float32)
|
|
||||||
|
|
||||||
# Define 3D points (meters) in world coordinates
|
|
||||||
# Court corners (Z=0, on ground)
|
|
||||||
court_3d = np.array([
|
|
||||||
[0, 0, 0], # TL
|
|
||||||
[13.4, 0, 0], # TR
|
|
||||||
[13.4, 6.1, 0], # BR
|
|
||||||
[0, 6.1, 0] # BL
|
|
||||||
], dtype=np.float32)
|
|
||||||
|
|
||||||
# Net endpoints (2 points)
|
|
||||||
# Net is at Y=3.05m (middle of 6.1m court width)
|
|
||||||
# We have 2 endpoints: left and right side at top of net
|
|
||||||
net_3d = np.array([
|
|
||||||
[0, 3.05, 0.914], # Left endpoint (top of net)
|
|
||||||
[13.4, 3.05, 0.914], # Right endpoint (top of net)
|
|
||||||
], dtype=np.float32)
|
|
||||||
|
|
||||||
# Combine all 3D and 2D points
|
|
||||||
object_points = np.vstack([court_3d, net_3d]) # 6 points total (4 court + 2 net)
|
|
||||||
image_points = np.vstack([court_corners, net_corners])
|
|
||||||
|
|
||||||
context.log.info(f"Calibrating with {len(object_points)} point correspondences")
|
|
||||||
context.log.info(f"3D points shape: {object_points.shape}")
|
|
||||||
context.log.info(f"2D points shape: {image_points.shape}")
|
|
||||||
|
|
||||||
# Initial camera matrix estimate
|
|
||||||
# Assume principal point at image center
|
|
||||||
w = detect_court_keypoints['frame_width']
|
|
||||||
h = detect_court_keypoints['frame_height']
|
|
||||||
cx = w / 2
|
|
||||||
cy = h / 2
|
|
||||||
# Estimate focal length (typical for drone/action cameras)
|
|
||||||
focal_length = max(w, h) # Initial guess
|
|
||||||
|
|
||||||
camera_matrix = np.array([
|
|
||||||
[focal_length, 0, cx],
|
|
||||||
[0, focal_length, cy],
|
|
||||||
[0, 0, 1]
|
|
||||||
], dtype=np.float32)
|
|
||||||
|
|
||||||
# No lens distortion (assume corrected or minimal)
|
|
||||||
dist_coeffs = None
|
|
||||||
|
|
||||||
# Solve PnP to get camera pose
|
|
||||||
try:
|
|
||||||
success, rotation_vec, translation_vec = cv2.solvePnP(
|
|
||||||
object_points,
|
|
||||||
image_points,
|
|
||||||
camera_matrix,
|
|
||||||
dist_coeffs,
|
|
||||||
flags=cv2.SOLVEPNP_ITERATIVE
|
|
||||||
)
|
|
||||||
|
|
||||||
if not success:
|
|
||||||
context.log.error("cv2.solvePnP failed")
|
|
||||||
return {
|
|
||||||
"calibrated": False,
|
|
||||||
"error": "solvePnP failed"
|
|
||||||
}
|
|
||||||
|
|
||||||
# Convert rotation vector to rotation matrix
|
|
||||||
rotation_matrix, _ = cv2.Rodrigues(rotation_vec)
|
|
||||||
|
|
||||||
# Calculate reprojection error
|
|
||||||
projected_points, _ = cv2.projectPoints(
|
|
||||||
object_points,
|
|
||||||
rotation_vec,
|
|
||||||
translation_vec,
|
|
||||||
camera_matrix,
|
|
||||||
dist_coeffs
|
|
||||||
)
|
|
||||||
projected_points = projected_points.reshape(-1, 2)
|
|
||||||
reprojection_error = np.sqrt(np.mean((image_points - projected_points) ** 2))
|
|
||||||
|
|
||||||
context.log.info(f"✓ Camera calibration successful")
|
|
||||||
context.log.info(f" Reprojection error: {reprojection_error:.2f} pixels")
|
|
||||||
context.log.info(f" Focal length: {focal_length:.1f}")
|
|
||||||
context.log.info(f" Rotation vector: {rotation_vec.flatten()}")
|
|
||||||
context.log.info(f" Translation vector: {translation_vec.flatten()}")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"camera_matrix": camera_matrix.tolist(),
|
|
||||||
"rotation_vector": rotation_vec.flatten().tolist(),
|
|
||||||
"translation_vector": translation_vec.flatten().tolist(),
|
|
||||||
"rotation_matrix": rotation_matrix.tolist(),
|
|
||||||
"reprojection_error": float(reprojection_error),
|
|
||||||
"focal_length": float(focal_length),
|
|
||||||
"principal_point": [float(cx), float(cy)],
|
|
||||||
"image_size": [w, h],
|
|
||||||
"calibrated": True
|
|
||||||
}
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
context.log.error(f"Error during camera calibration: {e}")
|
|
||||||
return {
|
|
||||||
"calibrated": False,
|
|
||||||
"error": str(e)
|
|
||||||
}
|
|
||||||
@@ -1,130 +0,0 @@
|
|||||||
"""Asset 4: Transform pixel coordinates to real-world 2D coordinates"""
|
|
||||||
|
|
||||||
import sys
|
|
||||||
import numpy as np
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict, List
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
|
|
||||||
# Add src to path to import existing calibrator
|
|
||||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="transform",
|
|
||||||
description="Transform pixel coordinates to real-world 2D court coordinates using homography"
|
|
||||||
)
|
|
||||||
def compute_2d_coordinates(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
detect_court_keypoints: Dict,
|
|
||||||
detect_ball_positions: List[Dict],
|
|
||||||
extract_video_frames: Dict
|
|
||||||
) -> List[Dict]:
|
|
||||||
"""
|
|
||||||
Transform ball pixel coordinates to real-world court coordinates
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- detect_court_keypoints: court corners in pixels
|
|
||||||
- detect_ball_positions: ball detections in pixels
|
|
||||||
- extract_video_frames: video metadata (FPS)
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- data/compute_2d_coordinates.json
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
List of dicts with:
|
|
||||||
- frame: frame number
|
|
||||||
- timestamp: time in seconds
|
|
||||||
- pixel_x, pixel_y: pixel coordinates
|
|
||||||
- x_m, y_m: real-world coordinates in meters
|
|
||||||
- confidence: detection confidence
|
|
||||||
"""
|
|
||||||
from src.court_calibrator import CourtCalibrator
|
|
||||||
|
|
||||||
context.log.info("Initializing court calibrator...")
|
|
||||||
|
|
||||||
calibrator = CourtCalibrator(
|
|
||||||
court_width_m=detect_court_keypoints['court_width_m'],
|
|
||||||
court_length_m=detect_court_keypoints['court_length_m']
|
|
||||||
)
|
|
||||||
|
|
||||||
# Calibrate using detected corners
|
|
||||||
corners = detect_court_keypoints['corners_pixel']
|
|
||||||
context.log.info(f"Calibrating with corners: {corners}")
|
|
||||||
|
|
||||||
success = calibrator.calibrate_manual(corners)
|
|
||||||
|
|
||||||
if not success:
|
|
||||||
raise RuntimeError("Court calibration failed. Check corner points.")
|
|
||||||
|
|
||||||
context.log.info("✓ Court calibration successful")
|
|
||||||
|
|
||||||
# Transform all ball positions
|
|
||||||
fps = extract_video_frames['fps']
|
|
||||||
trajectory = []
|
|
||||||
|
|
||||||
detected_count = 0
|
|
||||||
for detection in detect_ball_positions:
|
|
||||||
frame_num = detection['frame']
|
|
||||||
timestamp = frame_num / fps
|
|
||||||
|
|
||||||
if detection['x'] is not None and detection['y'] is not None:
|
|
||||||
# Transform pixel → meters
|
|
||||||
pixel_coords = [detection['x'], detection['y']]
|
|
||||||
real_coords = calibrator.pixel_to_real(pixel_coords)
|
|
||||||
|
|
||||||
if real_coords is not None:
|
|
||||||
trajectory.append({
|
|
||||||
"frame": frame_num,
|
|
||||||
"timestamp": round(timestamp, 3),
|
|
||||||
"pixel_x": round(detection['x'], 2),
|
|
||||||
"pixel_y": round(detection['y'], 2),
|
|
||||||
"x_m": round(float(real_coords[0]), 3),
|
|
||||||
"y_m": round(float(real_coords[1]), 3),
|
|
||||||
"confidence": round(detection['confidence'], 3)
|
|
||||||
})
|
|
||||||
detected_count += 1
|
|
||||||
else:
|
|
||||||
# Transformation failed (point outside court?)
|
|
||||||
trajectory.append({
|
|
||||||
"frame": frame_num,
|
|
||||||
"timestamp": round(timestamp, 3),
|
|
||||||
"pixel_x": round(detection['x'], 2),
|
|
||||||
"pixel_y": round(detection['y'], 2),
|
|
||||||
"x_m": None,
|
|
||||||
"y_m": None,
|
|
||||||
"confidence": round(detection['confidence'], 3)
|
|
||||||
})
|
|
||||||
else:
|
|
||||||
# No detection
|
|
||||||
trajectory.append({
|
|
||||||
"frame": frame_num,
|
|
||||||
"timestamp": round(timestamp, 3),
|
|
||||||
"pixel_x": None,
|
|
||||||
"pixel_y": None,
|
|
||||||
"x_m": None,
|
|
||||||
"y_m": None,
|
|
||||||
"confidence": 0.0
|
|
||||||
})
|
|
||||||
|
|
||||||
# Log progress
|
|
||||||
if (frame_num + 1) % 20 == 0:
|
|
||||||
context.log.info(f"Transformed {frame_num + 1}/{len(detect_ball_positions)} positions")
|
|
||||||
|
|
||||||
transform_rate = detected_count / len(detect_ball_positions) * 100
|
|
||||||
context.log.info(f"✓ Transformed {detected_count}/{len(detect_ball_positions)} positions ({transform_rate:.1f}%)")
|
|
||||||
|
|
||||||
# Calculate statistics
|
|
||||||
valid_positions = [t for t in trajectory if t['x_m'] is not None]
|
|
||||||
|
|
||||||
if valid_positions:
|
|
||||||
x_coords = [t['x_m'] for t in valid_positions]
|
|
||||||
y_coords = [t['y_m'] for t in valid_positions]
|
|
||||||
|
|
||||||
context.log.info(f"Position statistics:")
|
|
||||||
context.log.info(f" X range: {min(x_coords):.2f}m to {max(x_coords):.2f}m")
|
|
||||||
context.log.info(f" Y range: {min(y_coords):.2f}m to {max(y_coords):.2f}m")
|
|
||||||
context.log.info(f" Court dimensions: {calibrator.court_length}m x {calibrator.court_width}m")
|
|
||||||
|
|
||||||
return trajectory
|
|
||||||
@@ -1,190 +0,0 @@
|
|||||||
"""Asset: Transform ball positions to 3D coordinates (X, Y, Z) in meters"""
|
|
||||||
|
|
||||||
import cv2
|
|
||||||
import numpy as np
|
|
||||||
from typing import Dict, List
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="opencv",
|
|
||||||
description="Compute 3D ball coordinates (X, Y, Z) using camera calibration"
|
|
||||||
)
|
|
||||||
def compute_ball_3d_coordinates(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
detect_court_keypoints: Dict,
|
|
||||||
detect_ball_positions: List[Dict],
|
|
||||||
calibrate_camera_3d: Dict
|
|
||||||
) -> List[Dict]:
|
|
||||||
"""
|
|
||||||
Transform ball pixel coordinates to 3D world coordinates (meters)
|
|
||||||
|
|
||||||
Strategy:
|
|
||||||
- If ball on ground: use homography (Z=0)
|
|
||||||
- If ball in air: use ray casting + bbox size estimation (Z>0)
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- detect_court_keypoints: court corners
|
|
||||||
- detect_ball_positions: ball positions in pixels
|
|
||||||
- calibrate_camera_3d: camera calibration
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
List of dicts with 3D coordinates for each frame:
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"frame": 0,
|
|
||||||
"x_m": float or null, # X position on court (0-13.4m)
|
|
||||||
"y_m": float or null, # Y position on court (0-6.1m)
|
|
||||||
"z_m": float or null, # Z height above court (meters)
|
|
||||||
"on_ground": bool, # True if ball touching court
|
|
||||||
"confidence": float # Detection confidence
|
|
||||||
},
|
|
||||||
...
|
|
||||||
]
|
|
||||||
"""
|
|
||||||
if not calibrate_camera_3d.get('calibrated'):
|
|
||||||
context.log.error("Camera not calibrated, cannot compute 3D coordinates")
|
|
||||||
return [{"frame": i, "x_m": None, "y_m": None, "z_m": None, "on_ground": False}
|
|
||||||
for i in range(len(detect_ball_positions))]
|
|
||||||
|
|
||||||
# Extract calibration parameters
|
|
||||||
camera_matrix = np.array(calibrate_camera_3d['camera_matrix'], dtype=np.float32)
|
|
||||||
rotation_vec = np.array(calibrate_camera_3d['rotation_vector'], dtype=np.float32)
|
|
||||||
translation_vec = np.array(calibrate_camera_3d['translation_vector'], dtype=np.float32)
|
|
||||||
rotation_matrix = np.array(calibrate_camera_3d['rotation_matrix'], dtype=np.float32)
|
|
||||||
|
|
||||||
fx, fy = camera_matrix[0, 0], camera_matrix[1, 1]
|
|
||||||
cx, cy = camera_matrix[0, 2], camera_matrix[1, 2]
|
|
||||||
focal_length = (fx + fy) / 2
|
|
||||||
|
|
||||||
# Build homography for ground plane (Z=0)
|
|
||||||
court_corners_pixel = np.array(detect_court_keypoints['corners_pixel'], dtype=np.float32)
|
|
||||||
court_corners_meters = np.array([
|
|
||||||
[0, 0],
|
|
||||||
[13.4, 0],
|
|
||||||
[13.4, 6.1],
|
|
||||||
[0, 6.1]
|
|
||||||
], dtype=np.float32)
|
|
||||||
|
|
||||||
homography_matrix = cv2.getPerspectiveTransform(court_corners_pixel, court_corners_meters)
|
|
||||||
|
|
||||||
# Camera position in world coordinates
|
|
||||||
camera_position = -rotation_matrix.T @ translation_vec.reshape(3, 1)
|
|
||||||
|
|
||||||
context.log.info(f"Processing {len(detect_ball_positions)} frames for 3D coordinate transformation")
|
|
||||||
|
|
||||||
results = []
|
|
||||||
|
|
||||||
for i, ball_det in enumerate(detect_ball_positions):
|
|
||||||
if ball_det['x'] is None:
|
|
||||||
# No ball detected
|
|
||||||
results.append({
|
|
||||||
"frame": i,
|
|
||||||
"x_m": None,
|
|
||||||
"y_m": None,
|
|
||||||
"z_m": None,
|
|
||||||
"on_ground": False,
|
|
||||||
"confidence": 0.0
|
|
||||||
})
|
|
||||||
continue
|
|
||||||
|
|
||||||
ball_x = ball_det['x']
|
|
||||||
ball_y = ball_det['y']
|
|
||||||
ball_diameter = ball_det['diameter_px']
|
|
||||||
bbox = ball_det['bbox']
|
|
||||||
confidence = ball_det['confidence']
|
|
||||||
|
|
||||||
# Strategy 1: Try ground plane projection (assume Z=0)
|
|
||||||
ball_point_2d = np.array([[ball_x, ball_y]], dtype=np.float32)
|
|
||||||
ball_ground = cv2.perspectiveTransform(ball_point_2d.reshape(-1, 1, 2), homography_matrix)
|
|
||||||
x_ground, y_ground = ball_ground[0][0]
|
|
||||||
|
|
||||||
# Check if ball is likely on ground by comparing bbox size
|
|
||||||
# If ball bbox is large → likely on ground
|
|
||||||
# Simple heuristic: if diameter > threshold, assume on ground
|
|
||||||
on_ground_threshold = 30 # pixels - tune this based on typical ball size on ground
|
|
||||||
|
|
||||||
if ball_diameter and ball_diameter > on_ground_threshold:
|
|
||||||
# Ball likely on ground
|
|
||||||
results.append({
|
|
||||||
"frame": i,
|
|
||||||
"x_m": float(x_ground),
|
|
||||||
"y_m": float(y_ground),
|
|
||||||
"z_m": 0.0,
|
|
||||||
"on_ground": True,
|
|
||||||
"confidence": float(confidence)
|
|
||||||
})
|
|
||||||
else:
|
|
||||||
# Ball likely in air - use ray casting
|
|
||||||
try:
|
|
||||||
# Unproject 2D point to 3D ray
|
|
||||||
point_2d_normalized = np.array([
|
|
||||||
(ball_x - cx) / fx,
|
|
||||||
(ball_y - cy) / fy,
|
|
||||||
1.0
|
|
||||||
])
|
|
||||||
|
|
||||||
# Ray direction in camera coordinates
|
|
||||||
ray_camera = point_2d_normalized / np.linalg.norm(point_2d_normalized)
|
|
||||||
|
|
||||||
# Transform ray to world coordinates
|
|
||||||
ray_world = rotation_matrix.T @ ray_camera
|
|
||||||
|
|
||||||
# Estimate distance using ball size
|
|
||||||
# Real pickleball diameter: 74mm = 0.074m
|
|
||||||
ball_diameter_real = 0.074 # meters
|
|
||||||
ball_diameter_pixels = ball_diameter if ball_diameter else 20 # fallback to 20px
|
|
||||||
|
|
||||||
# Distance formula: D = (real_size * focal_length) / pixel_size
|
|
||||||
distance = (ball_diameter_real * focal_length) / ball_diameter_pixels
|
|
||||||
|
|
||||||
# Compute 3D point along ray
|
|
||||||
ball_3d = camera_position.flatten() + ray_world * distance
|
|
||||||
|
|
||||||
x_3d, y_3d, z_3d = ball_3d
|
|
||||||
|
|
||||||
# Sanity checks
|
|
||||||
if z_3d < 0 or z_3d > 10: # Height should be 0-10m
|
|
||||||
# Invalid, fallback to ground projection
|
|
||||||
results.append({
|
|
||||||
"frame": i,
|
|
||||||
"x_m": float(x_ground),
|
|
||||||
"y_m": float(y_ground),
|
|
||||||
"z_m": 0.0,
|
|
||||||
"on_ground": True,
|
|
||||||
"confidence": float(confidence)
|
|
||||||
})
|
|
||||||
else:
|
|
||||||
results.append({
|
|
||||||
"frame": i,
|
|
||||||
"x_m": float(x_3d),
|
|
||||||
"y_m": float(y_3d),
|
|
||||||
"z_m": float(z_3d),
|
|
||||||
"on_ground": False,
|
|
||||||
"confidence": float(confidence)
|
|
||||||
})
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
context.log.warning(f"Frame {i}: Error computing 3D coords, using ground projection: {e}")
|
|
||||||
results.append({
|
|
||||||
"frame": i,
|
|
||||||
"x_m": float(x_ground),
|
|
||||||
"y_m": float(y_ground),
|
|
||||||
"z_m": 0.0,
|
|
||||||
"on_ground": True,
|
|
||||||
"confidence": float(confidence)
|
|
||||||
})
|
|
||||||
|
|
||||||
if (i + 1) % 20 == 0:
|
|
||||||
context.log.info(f"Processed {i + 1}/{len(detect_ball_positions)} frames")
|
|
||||||
|
|
||||||
# Statistics
|
|
||||||
detected_count = sum(1 for r in results if r['x_m'] is not None)
|
|
||||||
on_ground_count = sum(1 for r in results if r.get('on_ground', False))
|
|
||||||
in_air_count = detected_count - on_ground_count
|
|
||||||
|
|
||||||
context.log.info(f"✓ Computed 3D coordinates: {detected_count} detections")
|
|
||||||
context.log.info(f" On ground: {on_ground_count}, In air: {in_air_count}")
|
|
||||||
|
|
||||||
return results
|
|
||||||
@@ -1,277 +0,0 @@
|
|||||||
"""Asset 2: Detect court keypoints using Roboflow Hosted API"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import cv2
|
|
||||||
import numpy as np
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict, List
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
from inference_sdk import InferenceHTTPClient
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="roboflow",
|
|
||||||
description="Detect pickleball court corners using Roboflow keypoint detection model"
|
|
||||||
)
|
|
||||||
def detect_court_keypoints(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
extract_video_frames: Dict
|
|
||||||
) -> Dict:
|
|
||||||
"""
|
|
||||||
Detect court keypoints from first frame using Roboflow model
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- extract_video_frames: metadata from frame extraction
|
|
||||||
- data/frames/frame_0000.jpg: first frame
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- data/detect_court_keypoints.json
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict with:
|
|
||||||
- corners_pixel: list of 4 corner coordinates [[x,y], ...]
|
|
||||||
- court_width_m: court width in meters (6.1)
|
|
||||||
- court_length_m: court length in meters (13.4)
|
|
||||||
- keypoints: all detected keypoints
|
|
||||||
"""
|
|
||||||
from inference import get_model
|
|
||||||
|
|
||||||
frames_dir = Path(extract_video_frames['frames_dir'])
|
|
||||||
first_frame_path = frames_dir / "frame_0000.jpg"
|
|
||||||
|
|
||||||
context.log.info(f"Loading first frame: {first_frame_path}")
|
|
||||||
|
|
||||||
if not first_frame_path.exists():
|
|
||||||
raise FileNotFoundError(f"First frame not found: {first_frame_path}")
|
|
||||||
|
|
||||||
# Load frame
|
|
||||||
frame = cv2.imread(str(first_frame_path))
|
|
||||||
h, w = frame.shape[:2]
|
|
||||||
context.log.info(f"Frame dimensions: {w}x{h}")
|
|
||||||
|
|
||||||
# Get API key
|
|
||||||
api_key = os.getenv("ROBOFLOW_API_KEY")
|
|
||||||
if not api_key:
|
|
||||||
context.log.warning("ROBOFLOW_API_KEY not set, using estimated corners")
|
|
||||||
corners = _estimate_court_corners(w, h)
|
|
||||||
else:
|
|
||||||
# Try to detect court using Roboflow Hosted API
|
|
||||||
try:
|
|
||||||
context.log.info("Detecting court using Roboflow Hosted API...")
|
|
||||||
|
|
||||||
client = InferenceHTTPClient(
|
|
||||||
api_url="https://serverless.roboflow.com",
|
|
||||||
api_key=api_key
|
|
||||||
)
|
|
||||||
|
|
||||||
result = client.infer(str(first_frame_path), model_id="pickleball-court-cfyv4/1")
|
|
||||||
|
|
||||||
# Extract keypoints from result
|
|
||||||
all_points = []
|
|
||||||
if result and 'predictions' in result and len(result['predictions']) > 0:
|
|
||||||
pred = result['predictions'][0]
|
|
||||||
if 'points' in pred and len(pred['points']) >= 4:
|
|
||||||
# Модель возвращает много points (линии корта)
|
|
||||||
all_points = [[p['x'], p['y']] for p in pred['points']]
|
|
||||||
context.log.info(f"✓ Detected {len(all_points)} keypoints from court lines")
|
|
||||||
|
|
||||||
# Находим 4 угла для калибровки (но не для визуализации)
|
|
||||||
corners = _extract_court_corners_from_points(all_points, w, h)
|
|
||||||
context.log.info(f"✓ Extracted 4 corners from keypoints")
|
|
||||||
else:
|
|
||||||
context.log.warning("No keypoints in prediction, using estimated corners")
|
|
||||||
corners = _estimate_court_corners(w, h)
|
|
||||||
else:
|
|
||||||
context.log.warning("No predictions from model, using estimated corners")
|
|
||||||
corners = _estimate_court_corners(w, h)
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
context.log.warning(f"Court detection failed: {e}. Using estimated corners.")
|
|
||||||
corners = _estimate_court_corners(w, h)
|
|
||||||
|
|
||||||
context.log.info(f"Court corners: {corners}")
|
|
||||||
|
|
||||||
# Save visualization - рисуем ВСЕ точки и линии от модели
|
|
||||||
vis_frame = frame.copy()
|
|
||||||
|
|
||||||
# Рисуем все точки от модели
|
|
||||||
if len(all_points) > 0:
|
|
||||||
context.log.info(f"Drawing {len(all_points)} keypoints on visualization")
|
|
||||||
|
|
||||||
# Рисуем все точки
|
|
||||||
for i, point in enumerate(all_points):
|
|
||||||
x, y = int(point[0]), int(point[1])
|
|
||||||
cv2.circle(vis_frame, (x, y), 5, (0, 255, 0), -1)
|
|
||||||
# Подписываем каждую точку номером
|
|
||||||
cv2.putText(
|
|
||||||
vis_frame,
|
|
||||||
str(i),
|
|
||||||
(x + 8, y),
|
|
||||||
cv2.FONT_HERSHEY_SIMPLEX,
|
|
||||||
0.4,
|
|
||||||
(255, 255, 0),
|
|
||||||
1
|
|
||||||
)
|
|
||||||
|
|
||||||
# Соединяем все соседние точки линиями
|
|
||||||
for i in range(len(all_points) - 1):
|
|
||||||
p1 = tuple(map(int, all_points[i]))
|
|
||||||
p2 = tuple(map(int, all_points[i + 1]))
|
|
||||||
cv2.line(vis_frame, p1, p2, (0, 255, 0), 2)
|
|
||||||
|
|
||||||
# Save visualization with run_id
|
|
||||||
run_id = context.run_id
|
|
||||||
vis_path = Path(f"data/{run_id}/court_detection_preview.jpg")
|
|
||||||
cv2.imwrite(str(vis_path), vis_frame)
|
|
||||||
context.log.info(f"Saved court visualization to {vis_path}")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"corners_pixel": corners,
|
|
||||||
"court_width_m": 6.1,
|
|
||||||
"court_length_m": 13.4,
|
|
||||||
"frame_width": w,
|
|
||||||
"frame_height": h
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _estimate_court_corners(width: int, height: int) -> List[List[float]]:
|
|
||||||
"""
|
|
||||||
Estimate court corners based on typical DJI camera position
|
|
||||||
(camera in corner at angle)
|
|
||||||
|
|
||||||
Returns corners in order: [TL, TR, BR, BL]
|
|
||||||
"""
|
|
||||||
# Assume court takes up ~80% of frame with perspective
|
|
||||||
margin_x = width * 0.05
|
|
||||||
margin_y = height * 0.1
|
|
||||||
|
|
||||||
# Perspective: far edge narrower than near edge
|
|
||||||
return [
|
|
||||||
[margin_x + width * 0.1, margin_y], # Top-left (far)
|
|
||||||
[width - margin_x - width * 0.1, margin_y], # Top-right (far)
|
|
||||||
[width - margin_x, height - margin_y], # Bottom-right (near)
|
|
||||||
[margin_x, height - margin_y] # Bottom-left (near)
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_court_corners_from_points(points: List[List[float]], width: int, height: int) -> List[List[float]]:
|
|
||||||
"""
|
|
||||||
Extract 4 court corners from many detected points (court lines)
|
|
||||||
|
|
||||||
Strategy:
|
|
||||||
1. Build convex hull from all points
|
|
||||||
2. Classify hull points into 4 sides (left, right, top, bottom)
|
|
||||||
3. Fit line for each side using linear regression
|
|
||||||
4. Find 4 corners as intersections of fitted lines
|
|
||||||
|
|
||||||
This works even if one corner is not visible on frame (extrapolation)
|
|
||||||
"""
|
|
||||||
if len(points) < 4:
|
|
||||||
return _estimate_court_corners(width, height)
|
|
||||||
|
|
||||||
# Build convex hull from all points
|
|
||||||
points_array = np.array(points, dtype=np.float32)
|
|
||||||
hull = cv2.convexHull(points_array)
|
|
||||||
hull_points = np.array([p[0] for p in hull], dtype=np.float32)
|
|
||||||
|
|
||||||
# Classify hull points into 4 sides
|
|
||||||
# Strategy: sort hull points by angle from centroid, then split into 4 groups
|
|
||||||
center = hull_points.mean(axis=0)
|
|
||||||
|
|
||||||
# Calculate angle for each point relative to center
|
|
||||||
angles = np.arctan2(hull_points[:, 1] - center[1], hull_points[:, 0] - center[0])
|
|
||||||
|
|
||||||
# Sort points by angle
|
|
||||||
sorted_indices = np.argsort(angles)
|
|
||||||
sorted_points = hull_points[sorted_indices]
|
|
||||||
|
|
||||||
# Split into 4 groups (4 sides)
|
|
||||||
n = len(sorted_points)
|
|
||||||
quarter = n // 4
|
|
||||||
|
|
||||||
side1 = sorted_points[0:quarter]
|
|
||||||
side2 = sorted_points[quarter:2*quarter]
|
|
||||||
side3 = sorted_points[2*quarter:3*quarter]
|
|
||||||
side4 = sorted_points[3*quarter:]
|
|
||||||
|
|
||||||
# Fit lines for each side using cv2.fitLine
|
|
||||||
def fit_line_coefficients(pts):
|
|
||||||
if len(pts) < 2:
|
|
||||||
return None
|
|
||||||
# cv2.fitLine returns (vx, vy, x0, y0) - direction vector and point on line
|
|
||||||
line = cv2.fitLine(pts, cv2.DIST_L2, 0, 0.01, 0.01)
|
|
||||||
vx, vy, x0, y0 = line[0][0], line[1][0], line[2][0], line[3][0]
|
|
||||||
# Convert to line equation: y = mx + b or vertical line x = c
|
|
||||||
if abs(vx) < 1e-6: # Vertical line
|
|
||||||
return ('vertical', x0)
|
|
||||||
m = vy / vx
|
|
||||||
b = y0 - m * x0
|
|
||||||
return ('normal', m, b)
|
|
||||||
|
|
||||||
line1 = fit_line_coefficients(side1)
|
|
||||||
line2 = fit_line_coefficients(side2)
|
|
||||||
line3 = fit_line_coefficients(side3)
|
|
||||||
line4 = fit_line_coefficients(side4)
|
|
||||||
|
|
||||||
lines = [line1, line2, line3, line4]
|
|
||||||
|
|
||||||
# Find intersections between adjacent sides
|
|
||||||
def line_intersection(line_a, line_b):
|
|
||||||
if line_a is None or line_b is None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
# Handle vertical lines
|
|
||||||
if line_a[0] == 'vertical' and line_b[0] == 'vertical':
|
|
||||||
return None
|
|
||||||
elif line_a[0] == 'vertical':
|
|
||||||
x = line_a[1]
|
|
||||||
m2, b2 = line_b[1], line_b[2]
|
|
||||||
y = m2 * x + b2
|
|
||||||
return [float(x), float(y)]
|
|
||||||
elif line_b[0] == 'vertical':
|
|
||||||
x = line_b[1]
|
|
||||||
m1, b1 = line_a[1], line_a[2]
|
|
||||||
y = m1 * x + b1
|
|
||||||
return [float(x), float(y)]
|
|
||||||
else:
|
|
||||||
m1, b1 = line_a[1], line_a[2]
|
|
||||||
m2, b2 = line_b[1], line_b[2]
|
|
||||||
|
|
||||||
if abs(m1 - m2) < 1e-6: # Parallel lines
|
|
||||||
return None
|
|
||||||
|
|
||||||
x = (b2 - b1) / (m1 - m2)
|
|
||||||
y = m1 * x + b1
|
|
||||||
return [float(x), float(y)]
|
|
||||||
|
|
||||||
# Find 4 corners as intersections
|
|
||||||
corners = []
|
|
||||||
for i in range(4):
|
|
||||||
next_i = (i + 1) % 4
|
|
||||||
corner = line_intersection(lines[i], lines[next_i])
|
|
||||||
if corner:
|
|
||||||
corners.append(corner)
|
|
||||||
|
|
||||||
# If we got 4 corners, return them
|
|
||||||
if len(corners) == 4:
|
|
||||||
return corners
|
|
||||||
|
|
||||||
# Fallback: use convex hull extreme points
|
|
||||||
tl = hull_points[np.argmin(hull_points[:, 0] + hull_points[:, 1])].tolist()
|
|
||||||
tr = hull_points[np.argmax(hull_points[:, 0] - hull_points[:, 1])].tolist()
|
|
||||||
br = hull_points[np.argmax(hull_points[:, 0] + hull_points[:, 1])].tolist()
|
|
||||||
bl = hull_points[np.argmin(hull_points[:, 0] - hull_points[:, 1])].tolist()
|
|
||||||
|
|
||||||
return [tl, tr, br, bl]
|
|
||||||
|
|
||||||
|
|
||||||
def _extract_court_corners(keypoints: List[Dict], width: int, height: int) -> List[List[float]]:
|
|
||||||
"""
|
|
||||||
Extract 4 court corners from detected keypoints (old function for compatibility)
|
|
||||||
"""
|
|
||||||
if len(keypoints) < 4:
|
|
||||||
return _estimate_court_corners(width, height)
|
|
||||||
|
|
||||||
points = [[kp['x'], kp['y']] for kp in keypoints]
|
|
||||||
return _extract_court_corners_from_points(points, width, height)
|
|
||||||
@@ -1,665 +0,0 @@
|
|||||||
"""Asset: Create interactive HTML viewer for frame-by-frame ball tracking"""
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict, List
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="html",
|
|
||||||
description="Create interactive HTML viewer with frame + 3D court visualization"
|
|
||||||
)
|
|
||||||
def create_interactive_viewer(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
extract_video_frames: Dict,
|
|
||||||
compute_ball_3d_coordinates: List[Dict]
|
|
||||||
) -> Dict:
|
|
||||||
"""
|
|
||||||
Create interactive HTML viewer showing:
|
|
||||||
- Left: Original video frame
|
|
||||||
- Right: Interactive 3D court (Three.js - rotatable with mouse)
|
|
||||||
- Controls: Prev/Next buttons + slider
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- data/{run_id}/viewer/index.html
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict with viewer_path
|
|
||||||
"""
|
|
||||||
run_id = context.run_id
|
|
||||||
frames_dir = Path(extract_video_frames['frames_dir'])
|
|
||||||
|
|
||||||
# Create viewer directory
|
|
||||||
viewer_dir = Path(f"data/{run_id}/viewer")
|
|
||||||
viewer_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
# Copy frames to viewer directory
|
|
||||||
import shutil
|
|
||||||
viewer_frames_dir = viewer_dir / "frames"
|
|
||||||
viewer_frames_dir.mkdir(exist_ok=True)
|
|
||||||
|
|
||||||
# Filter frames with detections
|
|
||||||
frames_with_ball = [f for f in compute_ball_3d_coordinates if f['x_m'] is not None]
|
|
||||||
|
|
||||||
context.log.info(f"Creating viewer for {len(frames_with_ball)} frames with ball detections")
|
|
||||||
|
|
||||||
# Copy only frames with detections
|
|
||||||
for frame_data in frames_with_ball:
|
|
||||||
frame_num = frame_data['frame']
|
|
||||||
src = frames_dir / f"frame_{frame_num:04d}.jpg"
|
|
||||||
dst = viewer_frames_dir / f"frame_{frame_num:04d}.jpg"
|
|
||||||
if src.exists():
|
|
||||||
shutil.copy2(src, dst)
|
|
||||||
|
|
||||||
context.log.info(f"Copied {len(frames_with_ball)} frames to viewer directory")
|
|
||||||
|
|
||||||
# Generate HTML
|
|
||||||
html_content = _generate_html(frames_with_ball, run_id)
|
|
||||||
|
|
||||||
# Save HTML
|
|
||||||
html_path = viewer_dir / "index.html"
|
|
||||||
with open(html_path, 'w') as f:
|
|
||||||
f.write(html_content)
|
|
||||||
|
|
||||||
context.log.info(f"✓ Interactive viewer created: {html_path}")
|
|
||||||
context.log.info(f" Open in browser: file://{html_path.absolute()}")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"viewer_path": str(html_path),
|
|
||||||
"num_frames": len(frames_with_ball)
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _generate_html(frames_data: List[Dict], run_id: str) -> str:
|
|
||||||
"""Generate HTML with Three.js for real 3D visualization"""
|
|
||||||
|
|
||||||
# Convert frames data to JSON
|
|
||||||
import json
|
|
||||||
frames_json = json.dumps(frames_data, indent=2)
|
|
||||||
|
|
||||||
html = f"""<!DOCTYPE html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="UTF-8">
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
||||||
<title>Ball Tracking 3D Viewer - Run {run_id[:8]}</title>
|
|
||||||
<style>
|
|
||||||
* {{
|
|
||||||
margin: 0;
|
|
||||||
padding: 0;
|
|
||||||
box-sizing: border-box;
|
|
||||||
}}
|
|
||||||
|
|
||||||
body {{
|
|
||||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif;
|
|
||||||
background: #1a1a1a;
|
|
||||||
color: #ffffff;
|
|
||||||
padding: 20px;
|
|
||||||
overflow-x: hidden;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.container {{
|
|
||||||
max-width: 1900px;
|
|
||||||
margin: 0 auto;
|
|
||||||
}}
|
|
||||||
|
|
||||||
h1 {{
|
|
||||||
text-align: center;
|
|
||||||
margin-bottom: 25px;
|
|
||||||
font-size: 28px;
|
|
||||||
color: #00ff88;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.viewer {{
|
|
||||||
display: grid;
|
|
||||||
grid-template-columns: 1fr 1fr;
|
|
||||||
gap: 20px;
|
|
||||||
margin-bottom: 20px;
|
|
||||||
height: 600px;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.panel {{
|
|
||||||
background: #2a2a2a;
|
|
||||||
border-radius: 8px;
|
|
||||||
padding: 20px;
|
|
||||||
box-shadow: 0 4px 6px rgba(0,0,0,0.3);
|
|
||||||
display: flex;
|
|
||||||
flex-direction: column;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.panel h2 {{
|
|
||||||
margin-bottom: 15px;
|
|
||||||
font-size: 18px;
|
|
||||||
color: #00ff88;
|
|
||||||
}}
|
|
||||||
|
|
||||||
#frameImage {{
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
object-fit: contain;
|
|
||||||
border-radius: 4px;
|
|
||||||
}}
|
|
||||||
|
|
||||||
#canvas3d {{
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
border-radius: 4px;
|
|
||||||
display: block;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.controls {{
|
|
||||||
background: #2a2a2a;
|
|
||||||
border-radius: 8px;
|
|
||||||
padding: 20px;
|
|
||||||
box-shadow: 0 4px 6px rgba(0,0,0,0.3);
|
|
||||||
}}
|
|
||||||
|
|
||||||
.controls-row {{
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 15px;
|
|
||||||
margin-bottom: 15px;
|
|
||||||
}}
|
|
||||||
|
|
||||||
button {{
|
|
||||||
background: #00ff88;
|
|
||||||
color: #1a1a1a;
|
|
||||||
border: none;
|
|
||||||
padding: 10px 24px;
|
|
||||||
border-radius: 4px;
|
|
||||||
font-size: 16px;
|
|
||||||
font-weight: 600;
|
|
||||||
cursor: pointer;
|
|
||||||
transition: all 0.2s;
|
|
||||||
}}
|
|
||||||
|
|
||||||
button:hover {{
|
|
||||||
background: #00cc6a;
|
|
||||||
transform: translateY(-1px);
|
|
||||||
}}
|
|
||||||
|
|
||||||
button:active {{
|
|
||||||
transform: translateY(0);
|
|
||||||
}}
|
|
||||||
|
|
||||||
button:disabled {{
|
|
||||||
background: #444;
|
|
||||||
color: #888;
|
|
||||||
cursor: not-allowed;
|
|
||||||
transform: none;
|
|
||||||
}}
|
|
||||||
|
|
||||||
input[type="range"] {{
|
|
||||||
flex: 1;
|
|
||||||
height: 6px;
|
|
||||||
background: #444;
|
|
||||||
border-radius: 3px;
|
|
||||||
outline: none;
|
|
||||||
-webkit-appearance: none;
|
|
||||||
}}
|
|
||||||
|
|
||||||
input[type="range"]::-webkit-slider-thumb {{
|
|
||||||
-webkit-appearance: none;
|
|
||||||
width: 18px;
|
|
||||||
height: 18px;
|
|
||||||
background: #00ff88;
|
|
||||||
border-radius: 50%;
|
|
||||||
cursor: pointer;
|
|
||||||
}}
|
|
||||||
|
|
||||||
input[type="range"]::-moz-range-thumb {{
|
|
||||||
width: 18px;
|
|
||||||
height: 18px;
|
|
||||||
background: #00ff88;
|
|
||||||
border-radius: 50%;
|
|
||||||
cursor: pointer;
|
|
||||||
border: none;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.info {{
|
|
||||||
display: grid;
|
|
||||||
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
|
|
||||||
gap: 15px;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.info-item {{
|
|
||||||
background: #1a1a1a;
|
|
||||||
padding: 12px;
|
|
||||||
border-radius: 4px;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.info-label {{
|
|
||||||
font-size: 12px;
|
|
||||||
color: #888;
|
|
||||||
margin-bottom: 4px;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.info-value {{
|
|
||||||
font-size: 20px;
|
|
||||||
font-weight: 600;
|
|
||||||
color: #00ff88;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.kbd {{
|
|
||||||
display: inline-block;
|
|
||||||
padding: 3px 8px;
|
|
||||||
background: #444;
|
|
||||||
border-radius: 3px;
|
|
||||||
font-family: monospace;
|
|
||||||
font-size: 14px;
|
|
||||||
margin: 0 4px;
|
|
||||||
}}
|
|
||||||
|
|
||||||
.help {{
|
|
||||||
margin-top: 15px;
|
|
||||||
padding: 10px;
|
|
||||||
background: #1a1a1a;
|
|
||||||
border-radius: 4px;
|
|
||||||
font-size: 14px;
|
|
||||||
color: #888;
|
|
||||||
}}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<div class="container">
|
|
||||||
<h1>🎾 Pickleball Ball Tracking 3D Viewer</h1>
|
|
||||||
|
|
||||||
<div class="viewer">
|
|
||||||
<div class="panel">
|
|
||||||
<h2>📹 Video Frame</h2>
|
|
||||||
<img id="frameImage" alt="Video frame">
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="panel">
|
|
||||||
<h2>🗺️ Interactive 3D Court (drag to rotate, scroll to zoom)</h2>
|
|
||||||
<div id="canvas3d"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="controls">
|
|
||||||
<div class="controls-row">
|
|
||||||
<button id="prevBtn">← Prev</button>
|
|
||||||
<input type="range" id="frameSlider" min="0" max="0" value="0">
|
|
||||||
<button id="nextBtn">Next →</button>
|
|
||||||
<button id="playBtn">▶ Play</button>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="info">
|
|
||||||
<div class="info-item">
|
|
||||||
<div class="info-label">Frame</div>
|
|
||||||
<div class="info-value" id="frameNum">-</div>
|
|
||||||
</div>
|
|
||||||
<div class="info-item">
|
|
||||||
<div class="info-label">X Position (m)</div>
|
|
||||||
<div class="info-value" id="posX">-</div>
|
|
||||||
</div>
|
|
||||||
<div class="info-item">
|
|
||||||
<div class="info-label">Y Position (m)</div>
|
|
||||||
<div class="info-value" id="posY">-</div>
|
|
||||||
</div>
|
|
||||||
<div class="info-item">
|
|
||||||
<div class="info-label">Z Height (m)</div>
|
|
||||||
<div class="info-value" id="posZ">-</div>
|
|
||||||
</div>
|
|
||||||
<div class="info-item">
|
|
||||||
<div class="info-label">Confidence</div>
|
|
||||||
<div class="info-value" id="confidence">-</div>
|
|
||||||
</div>
|
|
||||||
<div class="info-item">
|
|
||||||
<div class="info-label">On Ground</div>
|
|
||||||
<div class="info-value" id="onGround">-</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="help">
|
|
||||||
💡 <strong>Controls:</strong>
|
|
||||||
<span class="kbd">←/→</span> Navigate frames
|
|
||||||
<span class="kbd">Space</span> Play/Pause
|
|
||||||
<span class="kbd">Mouse drag</span> Rotate 3D view
|
|
||||||
<span class="kbd">Mouse wheel</span> Zoom
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Three.js from CDN -->
|
|
||||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
|
|
||||||
<script>
|
|
||||||
// Frame data from Python
|
|
||||||
const framesData = {frames_json};
|
|
||||||
|
|
||||||
let currentIndex = 0;
|
|
||||||
let isPlaying = false;
|
|
||||||
let playInterval = null;
|
|
||||||
|
|
||||||
// DOM elements
|
|
||||||
const frameImage = document.getElementById('frameImage');
|
|
||||||
const canvas3dContainer = document.getElementById('canvas3d');
|
|
||||||
const prevBtn = document.getElementById('prevBtn');
|
|
||||||
const nextBtn = document.getElementById('nextBtn');
|
|
||||||
const playBtn = document.getElementById('playBtn');
|
|
||||||
const frameSlider = document.getElementById('frameSlider');
|
|
||||||
const frameNum = document.getElementById('frameNum');
|
|
||||||
const posX = document.getElementById('posX');
|
|
||||||
const posY = document.getElementById('posY');
|
|
||||||
const posZ = document.getElementById('posZ');
|
|
||||||
const confidence = document.getElementById('confidence');
|
|
||||||
const onGround = document.getElementById('onGround');
|
|
||||||
|
|
||||||
// Initialize
|
|
||||||
frameSlider.max = framesData.length - 1;
|
|
||||||
|
|
||||||
// Setup Three.js scene
|
|
||||||
const scene = new THREE.Scene();
|
|
||||||
scene.background = new THREE.Color(0x1a1a1a);
|
|
||||||
|
|
||||||
const camera = new THREE.PerspectiveCamera(
|
|
||||||
60,
|
|
||||||
canvas3dContainer.clientWidth / canvas3dContainer.clientHeight,
|
|
||||||
0.1,
|
|
||||||
1000
|
|
||||||
);
|
|
||||||
camera.position.set(15, 12, 15);
|
|
||||||
camera.lookAt(6.7, 3, 0);
|
|
||||||
|
|
||||||
const renderer = new THREE.WebGLRenderer({{ antialias: true }});
|
|
||||||
renderer.setSize(canvas3dContainer.clientWidth, canvas3dContainer.clientHeight);
|
|
||||||
canvas3dContainer.appendChild(renderer.domElement);
|
|
||||||
|
|
||||||
// Lighting
|
|
||||||
const ambientLight = new THREE.AmbientLight(0xffffff, 0.6);
|
|
||||||
scene.add(ambientLight);
|
|
||||||
|
|
||||||
const directionalLight = new THREE.DirectionalLight(0xffffff, 0.8);
|
|
||||||
directionalLight.position.set(10, 20, 10);
|
|
||||||
scene.add(directionalLight);
|
|
||||||
|
|
||||||
// Grid helper
|
|
||||||
const gridHelper = new THREE.GridHelper(20, 20, 0x444444, 0x222222);
|
|
||||||
scene.add(gridHelper);
|
|
||||||
|
|
||||||
// Court dimensions
|
|
||||||
const courtLength = 13.4;
|
|
||||||
const courtWidth = 6.1;
|
|
||||||
const netHeight = 0.914;
|
|
||||||
|
|
||||||
// Court floor (green)
|
|
||||||
const courtGeometry = new THREE.PlaneGeometry(courtLength, courtWidth);
|
|
||||||
const courtMaterial = new THREE.MeshStandardMaterial({{
|
|
||||||
color: 0x00aa44,
|
|
||||||
side: THREE.DoubleSide,
|
|
||||||
roughness: 0.8
|
|
||||||
}});
|
|
||||||
const court = new THREE.Mesh(courtGeometry, courtMaterial);
|
|
||||||
court.rotation.x = -Math.PI / 2;
|
|
||||||
court.position.set(courtLength / 2, 0, courtWidth / 2);
|
|
||||||
scene.add(court);
|
|
||||||
|
|
||||||
// Court boundaries (white lines)
|
|
||||||
const boundaryMaterial = new THREE.LineBasicMaterial({{ color: 0xffffff, linewidth: 3 }});
|
|
||||||
const boundaryPoints = [
|
|
||||||
new THREE.Vector3(0, 0.01, 0),
|
|
||||||
new THREE.Vector3(courtLength, 0.01, 0),
|
|
||||||
new THREE.Vector3(courtLength, 0.01, courtWidth),
|
|
||||||
new THREE.Vector3(0, 0.01, courtWidth),
|
|
||||||
new THREE.Vector3(0, 0.01, 0)
|
|
||||||
];
|
|
||||||
const boundaryGeometry = new THREE.BufferGeometry().setFromPoints(boundaryPoints);
|
|
||||||
const boundary = new THREE.Line(boundaryGeometry, boundaryMaterial);
|
|
||||||
scene.add(boundary);
|
|
||||||
|
|
||||||
// Net (mesh) - positioned at middle of court LENGTH (X=6.7m), spanning full WIDTH (Z axis)
|
|
||||||
const netGeometry = new THREE.BoxGeometry(0.05, netHeight, courtWidth);
|
|
||||||
const netMaterial = new THREE.MeshStandardMaterial({{
|
|
||||||
color: 0xcccccc,
|
|
||||||
transparent: true,
|
|
||||||
opacity: 0.7,
|
|
||||||
side: THREE.DoubleSide
|
|
||||||
}});
|
|
||||||
const net = new THREE.Mesh(netGeometry, netMaterial);
|
|
||||||
net.position.set(courtLength / 2, netHeight / 2, courtWidth / 2);
|
|
||||||
scene.add(net);
|
|
||||||
|
|
||||||
// Net poles at both ends of the net (Z=0 and Z=courtWidth)
|
|
||||||
const poleGeometry = new THREE.CylinderGeometry(0.05, 0.05, netHeight, 8);
|
|
||||||
const poleMaterial = new THREE.MeshStandardMaterial({{ color: 0x666666 }});
|
|
||||||
|
|
||||||
const pole1 = new THREE.Mesh(poleGeometry, poleMaterial);
|
|
||||||
pole1.position.set(courtLength / 2, netHeight / 2, 0);
|
|
||||||
scene.add(pole1);
|
|
||||||
|
|
||||||
const pole2 = new THREE.Mesh(poleGeometry, poleMaterial);
|
|
||||||
pole2.position.set(courtLength / 2, netHeight / 2, courtWidth);
|
|
||||||
scene.add(pole2);
|
|
||||||
|
|
||||||
// Ball (sphere)
|
|
||||||
let ballMesh = null;
|
|
||||||
let ballShadow = null;
|
|
||||||
let ballLine = null;
|
|
||||||
|
|
||||||
function createBall() {{
|
|
||||||
// Remove old ball if exists
|
|
||||||
if (ballMesh) scene.remove(ballMesh);
|
|
||||||
if (ballShadow) scene.remove(ballShadow);
|
|
||||||
if (ballLine) scene.remove(ballLine);
|
|
||||||
|
|
||||||
// Create ball
|
|
||||||
const ballGeometry = new THREE.SphereGeometry(0.074 / 2, 32, 32);
|
|
||||||
const ballMaterial = new THREE.MeshStandardMaterial({{
|
|
||||||
color: 0xffff00,
|
|
||||||
emissive: 0xff4444,
|
|
||||||
emissiveIntensity: 0.3,
|
|
||||||
metalness: 0.5,
|
|
||||||
roughness: 0.3
|
|
||||||
}});
|
|
||||||
ballMesh = new THREE.Mesh(ballGeometry, ballMaterial);
|
|
||||||
|
|
||||||
// Shadow (circle on ground)
|
|
||||||
const shadowGeometry = new THREE.CircleGeometry(0.1, 32);
|
|
||||||
const shadowMaterial = new THREE.MeshBasicMaterial({{
|
|
||||||
color: 0x000000,
|
|
||||||
transparent: true,
|
|
||||||
opacity: 0.3,
|
|
||||||
side: THREE.DoubleSide
|
|
||||||
}});
|
|
||||||
ballShadow = new THREE.Mesh(shadowGeometry, shadowMaterial);
|
|
||||||
ballShadow.rotation.x = -Math.PI / 2;
|
|
||||||
|
|
||||||
scene.add(ballMesh);
|
|
||||||
scene.add(ballShadow);
|
|
||||||
}}
|
|
||||||
|
|
||||||
function updateBall(x, y, z) {{
|
|
||||||
if (!ballMesh) createBall();
|
|
||||||
|
|
||||||
ballMesh.position.set(x, z, y);
|
|
||||||
ballShadow.position.set(x, 0.01, y);
|
|
||||||
|
|
||||||
// Draw line from shadow to ball if in air
|
|
||||||
if (z > 0.1) {{
|
|
||||||
if (ballLine) scene.remove(ballLine);
|
|
||||||
|
|
||||||
const lineMaterial = new THREE.LineBasicMaterial({{
|
|
||||||
color: 0xffffff,
|
|
||||||
transparent: true,
|
|
||||||
opacity: 0.3
|
|
||||||
}});
|
|
||||||
const linePoints = [
|
|
||||||
new THREE.Vector3(x, z, y),
|
|
||||||
new THREE.Vector3(x, 0, y)
|
|
||||||
];
|
|
||||||
const lineGeometry = new THREE.BufferGeometry().setFromPoints(linePoints);
|
|
||||||
ballLine = new THREE.Line(lineGeometry, lineMaterial);
|
|
||||||
scene.add(ballLine);
|
|
||||||
}} else {{
|
|
||||||
if (ballLine) {{
|
|
||||||
scene.remove(ballLine);
|
|
||||||
ballLine = null;
|
|
||||||
}}
|
|
||||||
}}
|
|
||||||
}}
|
|
||||||
|
|
||||||
// Mouse controls for camera
|
|
||||||
let isDragging = false;
|
|
||||||
let previousMousePosition = {{ x: 0, y: 0 }};
|
|
||||||
let cameraAngle = {{ theta: Math.PI / 4, phi: Math.PI / 3 }};
|
|
||||||
let cameraDistance = 22;
|
|
||||||
|
|
||||||
canvas3dContainer.addEventListener('mousedown', (e) => {{
|
|
||||||
isDragging = true;
|
|
||||||
previousMousePosition = {{ x: e.clientX, y: e.clientY }};
|
|
||||||
}});
|
|
||||||
|
|
||||||
document.addEventListener('mouseup', () => {{
|
|
||||||
isDragging = false;
|
|
||||||
}});
|
|
||||||
|
|
||||||
canvas3dContainer.addEventListener('mousemove', (e) => {{
|
|
||||||
if (!isDragging) return;
|
|
||||||
|
|
||||||
const deltaX = e.clientX - previousMousePosition.x;
|
|
||||||
const deltaY = e.clientY - previousMousePosition.y;
|
|
||||||
|
|
||||||
cameraAngle.theta += deltaX * 0.01;
|
|
||||||
cameraAngle.phi = Math.max(0.1, Math.min(Math.PI / 2 - 0.1, cameraAngle.phi - deltaY * 0.01));
|
|
||||||
|
|
||||||
previousMousePosition = {{ x: e.clientX, y: e.clientY }};
|
|
||||||
|
|
||||||
updateCameraPosition();
|
|
||||||
}});
|
|
||||||
|
|
||||||
canvas3dContainer.addEventListener('wheel', (e) => {{
|
|
||||||
e.preventDefault();
|
|
||||||
cameraDistance += e.deltaY * 0.01;
|
|
||||||
cameraDistance = Math.max(5, Math.min(50, cameraDistance));
|
|
||||||
updateCameraPosition();
|
|
||||||
}});
|
|
||||||
|
|
||||||
function updateCameraPosition() {{
|
|
||||||
const centerX = courtLength / 2;
|
|
||||||
const centerZ = courtWidth / 2;
|
|
||||||
|
|
||||||
camera.position.x = centerX + cameraDistance * Math.sin(cameraAngle.phi) * Math.cos(cameraAngle.theta);
|
|
||||||
camera.position.y = cameraDistance * Math.cos(cameraAngle.phi);
|
|
||||||
camera.position.z = centerZ + cameraDistance * Math.sin(cameraAngle.phi) * Math.sin(cameraAngle.theta);
|
|
||||||
|
|
||||||
camera.lookAt(centerX, 0, centerZ);
|
|
||||||
}}
|
|
||||||
|
|
||||||
// Handle window resize
|
|
||||||
window.addEventListener('resize', () => {{
|
|
||||||
camera.aspect = canvas3dContainer.clientWidth / canvas3dContainer.clientHeight;
|
|
||||||
camera.updateProjectionMatrix();
|
|
||||||
renderer.setSize(canvas3dContainer.clientWidth, canvas3dContainer.clientHeight);
|
|
||||||
}});
|
|
||||||
|
|
||||||
// Animation loop
|
|
||||||
function animate() {{
|
|
||||||
requestAnimationFrame(animate);
|
|
||||||
renderer.render(scene, camera);
|
|
||||||
}}
|
|
||||||
animate();
|
|
||||||
|
|
||||||
// Render current frame
|
|
||||||
function render() {{
|
|
||||||
if (framesData.length === 0) return;
|
|
||||||
|
|
||||||
const frame = framesData[currentIndex];
|
|
||||||
|
|
||||||
// Update image - frames are in ./frames/ directory relative to HTML
|
|
||||||
const framePath = `frames/frame_${{String(frame.frame).padStart(4, '0')}}.jpg`;
|
|
||||||
frameImage.src = framePath;
|
|
||||||
|
|
||||||
// Update info
|
|
||||||
frameNum.textContent = `${{currentIndex + 1}} / ${{framesData.length}} (Frame #${{frame.frame}})`;
|
|
||||||
posX.textContent = frame.x_m !== null ? frame.x_m.toFixed(2) : 'N/A';
|
|
||||||
posY.textContent = frame.y_m !== null ? frame.y_m.toFixed(2) : 'N/A';
|
|
||||||
posZ.textContent = frame.z_m !== null ? frame.z_m.toFixed(2) : 'N/A';
|
|
||||||
confidence.textContent = frame.confidence.toFixed(2);
|
|
||||||
onGround.textContent = frame.on_ground ? '✓ Yes' : '✗ No';
|
|
||||||
|
|
||||||
// Update slider
|
|
||||||
frameSlider.value = currentIndex;
|
|
||||||
|
|
||||||
// Update buttons
|
|
||||||
prevBtn.disabled = currentIndex === 0;
|
|
||||||
nextBtn.disabled = currentIndex === framesData.length - 1;
|
|
||||||
|
|
||||||
// Update ball position in 3D
|
|
||||||
if (frame.x_m !== null && frame.y_m !== null && frame.z_m !== null) {{
|
|
||||||
updateBall(frame.x_m, frame.y_m, frame.z_m);
|
|
||||||
}}
|
|
||||||
}}
|
|
||||||
|
|
||||||
// Navigation functions
|
|
||||||
function nextFrame() {{
|
|
||||||
if (currentIndex < framesData.length - 1) {{
|
|
||||||
currentIndex++;
|
|
||||||
render();
|
|
||||||
}}
|
|
||||||
}}
|
|
||||||
|
|
||||||
function prevFrame() {{
|
|
||||||
if (currentIndex > 0) {{
|
|
||||||
currentIndex--;
|
|
||||||
render();
|
|
||||||
}}
|
|
||||||
}}
|
|
||||||
|
|
||||||
function gotoFrame(index) {{
|
|
||||||
currentIndex = Math.max(0, Math.min(index, framesData.length - 1));
|
|
||||||
render();
|
|
||||||
}}
|
|
||||||
|
|
||||||
function togglePlay() {{
|
|
||||||
if (isPlaying) {{
|
|
||||||
clearInterval(playInterval);
|
|
||||||
isPlaying = false;
|
|
||||||
playBtn.textContent = '▶ Play';
|
|
||||||
}} else {{
|
|
||||||
isPlaying = true;
|
|
||||||
playBtn.textContent = '⏸ Pause';
|
|
||||||
playInterval = setInterval(() => {{
|
|
||||||
if (currentIndex < framesData.length - 1) {{
|
|
||||||
nextFrame();
|
|
||||||
}} else {{
|
|
||||||
clearInterval(playInterval);
|
|
||||||
isPlaying = false;
|
|
||||||
playBtn.textContent = '▶ Play';
|
|
||||||
currentIndex = 0;
|
|
||||||
render();
|
|
||||||
}}
|
|
||||||
}}, 100); // 10 fps
|
|
||||||
}}
|
|
||||||
}}
|
|
||||||
|
|
||||||
// Event listeners
|
|
||||||
prevBtn.addEventListener('click', prevFrame);
|
|
||||||
nextBtn.addEventListener('click', nextFrame);
|
|
||||||
playBtn.addEventListener('click', togglePlay);
|
|
||||||
frameSlider.addEventListener('input', (e) => gotoFrame(parseInt(e.target.value)));
|
|
||||||
|
|
||||||
// Keyboard shortcuts
|
|
||||||
document.addEventListener('keydown', (e) => {{
|
|
||||||
switch(e.key) {{
|
|
||||||
case 'ArrowLeft':
|
|
||||||
prevFrame();
|
|
||||||
break;
|
|
||||||
case 'ArrowRight':
|
|
||||||
nextFrame();
|
|
||||||
break;
|
|
||||||
case ' ':
|
|
||||||
e.preventDefault();
|
|
||||||
togglePlay();
|
|
||||||
break;
|
|
||||||
}}
|
|
||||||
}});
|
|
||||||
|
|
||||||
// Initial render
|
|
||||||
render();
|
|
||||||
</script>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
"""
|
|
||||||
|
|
||||||
return html
|
|
||||||
@@ -1,72 +0,0 @@
|
|||||||
"""Asset: Detect tennis/pickleball net using Roboflow"""
|
|
||||||
|
|
||||||
import os
|
|
||||||
import cv2
|
|
||||||
import numpy as np
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
from inference_sdk import InferenceHTTPClient
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="roboflow",
|
|
||||||
description="Detect pickleball/tennis net using Roboflow model"
|
|
||||||
)
|
|
||||||
def detect_net(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
extract_video_frames: Dict,
|
|
||||||
detect_court_keypoints: Dict
|
|
||||||
) -> Dict:
|
|
||||||
"""
|
|
||||||
Detect net on first frame using Roboflow model
|
|
||||||
|
|
||||||
NO FALLBACKS - if model doesn't detect net, this will fail
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- extract_video_frames: frame metadata
|
|
||||||
- detect_court_keypoints: court corners (for visualization)
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- data/{run_id}/net_detection_preview.jpg: visualization
|
|
||||||
- JSON with net detection results
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict with net detection data
|
|
||||||
"""
|
|
||||||
run_id = context.run_id
|
|
||||||
frames_dir = Path(extract_video_frames['frames_dir'])
|
|
||||||
first_frame_path = frames_dir / "frame_0000.jpg"
|
|
||||||
|
|
||||||
context.log.info(f"Loading first frame: {first_frame_path}")
|
|
||||||
|
|
||||||
# Load frame
|
|
||||||
frame = cv2.imread(str(first_frame_path))
|
|
||||||
h, w = frame.shape[:2]
|
|
||||||
context.log.info(f"Frame dimensions: {w}x{h}")
|
|
||||||
|
|
||||||
# Get API key
|
|
||||||
api_key = os.getenv("ROBOFLOW_API_KEY")
|
|
||||||
if not api_key:
|
|
||||||
raise ValueError("ROBOFLOW_API_KEY environment variable is not set")
|
|
||||||
|
|
||||||
context.log.info("Detecting net using Roboflow model...")
|
|
||||||
|
|
||||||
client = InferenceHTTPClient(
|
|
||||||
api_url="https://serverless.roboflow.com",
|
|
||||||
api_key=api_key
|
|
||||||
)
|
|
||||||
|
|
||||||
# Call Roboflow model - MODEL_ID WILL BE PROVIDED BY USER
|
|
||||||
# Placeholder - user will provide correct model
|
|
||||||
model_id = "MODEL_ID_PLACEHOLDER"
|
|
||||||
|
|
||||||
result = client.infer(str(first_frame_path), model_id=model_id)
|
|
||||||
|
|
||||||
context.log.info(f"Roboflow response: {result}")
|
|
||||||
|
|
||||||
# TODO: Parse result based on actual model output format
|
|
||||||
# User will provide correct model and we'll update parsing logic
|
|
||||||
|
|
||||||
raise NotImplementedError("Waiting for correct Roboflow model from user")
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
"""Asset 1: Extract frames from video"""
|
|
||||||
|
|
||||||
import cv2
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="opencv",
|
|
||||||
description="Extract frames from video starting at specified second"
|
|
||||||
)
|
|
||||||
def extract_video_frames(context: AssetExecutionContext) -> Dict:
|
|
||||||
"""
|
|
||||||
Extract frames from DJI_0017.MP4 video
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- DJI_0017.MP4 (video file in root directory)
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- data/frames/frame_XXXX.jpg (100 frames)
|
|
||||||
- data/extract_video_frames.json (metadata)
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict with:
|
|
||||||
- frames_dir: path to frames directory
|
|
||||||
- num_frames: number of extracted frames
|
|
||||||
- fps: video FPS
|
|
||||||
- start_frame: starting frame number
|
|
||||||
"""
|
|
||||||
# Configuration
|
|
||||||
video_path = "DJI_0017.MP4"
|
|
||||||
start_sec = 10
|
|
||||||
num_frames = 100
|
|
||||||
|
|
||||||
context.log.info(f"Opening video: {video_path}")
|
|
||||||
cap = cv2.VideoCapture(video_path)
|
|
||||||
|
|
||||||
if not cap.isOpened():
|
|
||||||
raise RuntimeError(f"Could not open video: {video_path}")
|
|
||||||
|
|
||||||
fps = cap.get(cv2.CAP_PROP_FPS)
|
|
||||||
total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
|
|
||||||
start_frame = int(start_sec * fps)
|
|
||||||
|
|
||||||
context.log.info(f"Video info: {total_frames} frames, {fps} FPS")
|
|
||||||
context.log.info(f"Extracting {num_frames} frames starting from frame {start_frame} ({start_sec}s)")
|
|
||||||
|
|
||||||
# Create output directory with run_id
|
|
||||||
run_id = context.run_id
|
|
||||||
frames_dir = Path(f"data/{run_id}/frames")
|
|
||||||
frames_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
# Set starting position
|
|
||||||
cap.set(cv2.CAP_PROP_POS_FRAMES, start_frame)
|
|
||||||
|
|
||||||
# Extract frames
|
|
||||||
extracted = 0
|
|
||||||
for i in range(num_frames):
|
|
||||||
ret, frame = cap.read()
|
|
||||||
if not ret:
|
|
||||||
context.log.warning(f"Could not read frame {i}. Stopping extraction.")
|
|
||||||
break
|
|
||||||
|
|
||||||
frame_path = frames_dir / f"frame_{i:04d}.jpg"
|
|
||||||
cv2.imwrite(str(frame_path), frame)
|
|
||||||
extracted += 1
|
|
||||||
|
|
||||||
if (i + 1) % 20 == 0:
|
|
||||||
context.log.info(f"Extracted {i + 1}/{num_frames} frames")
|
|
||||||
|
|
||||||
cap.release()
|
|
||||||
|
|
||||||
context.log.info(f"✓ Extracted {extracted} frames to {frames_dir}")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"frames_dir": str(frames_dir),
|
|
||||||
"num_frames": extracted,
|
|
||||||
"fps": fps,
|
|
||||||
"start_frame": start_frame,
|
|
||||||
"start_sec": start_sec
|
|
||||||
}
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
"""Asset: Draw court polygon with 4 corners"""
|
|
||||||
|
|
||||||
import cv2
|
|
||||||
import numpy as np
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Dict
|
|
||||||
from dagster import asset, AssetExecutionContext
|
|
||||||
|
|
||||||
|
|
||||||
@asset(
|
|
||||||
io_manager_key="json_io_manager",
|
|
||||||
compute_kind="opencv",
|
|
||||||
description="Draw court polygon with 4 corners on first frame"
|
|
||||||
)
|
|
||||||
def visualize_ball_on_court(
|
|
||||||
context: AssetExecutionContext,
|
|
||||||
extract_video_frames: Dict,
|
|
||||||
detect_court_keypoints: Dict
|
|
||||||
) -> Dict:
|
|
||||||
"""
|
|
||||||
Draw court polygon (4 corners) on first frame
|
|
||||||
|
|
||||||
Inputs:
|
|
||||||
- extract_video_frames: frame metadata
|
|
||||||
- detect_court_keypoints: 4 court corners with perspective
|
|
||||||
|
|
||||||
Outputs:
|
|
||||||
- One image: data/{run_id}/court_polygon.jpg
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict with:
|
|
||||||
- image_path: path to saved image
|
|
||||||
"""
|
|
||||||
run_id = context.run_id
|
|
||||||
frames_dir = Path(extract_video_frames['frames_dir'])
|
|
||||||
|
|
||||||
# Load first frame
|
|
||||||
first_frame_path = frames_dir / "frame_0000.jpg"
|
|
||||||
context.log.info(f"Loading first frame: {first_frame_path}")
|
|
||||||
|
|
||||||
if not first_frame_path.exists():
|
|
||||||
raise FileNotFoundError(f"First frame not found: {first_frame_path}")
|
|
||||||
|
|
||||||
frame = cv2.imread(str(first_frame_path))
|
|
||||||
if frame is None:
|
|
||||||
raise RuntimeError(f"Failed to load frame: {first_frame_path}")
|
|
||||||
|
|
||||||
# Get court corners (4 points with perspective)
|
|
||||||
corners = detect_court_keypoints['corners_pixel']
|
|
||||||
court_polygon = np.array(corners, dtype=np.int32)
|
|
||||||
|
|
||||||
context.log.info(f"Drawing court polygon with 4 corners: {corners}")
|
|
||||||
|
|
||||||
# Draw court polygon (4 corners with perspective)
|
|
||||||
cv2.polylines(
|
|
||||||
frame,
|
|
||||||
[court_polygon],
|
|
||||||
isClosed=True,
|
|
||||||
color=(0, 255, 0), # Green
|
|
||||||
thickness=3
|
|
||||||
)
|
|
||||||
|
|
||||||
# Draw court corners as circles
|
|
||||||
for i, corner in enumerate(corners):
|
|
||||||
cv2.circle(
|
|
||||||
frame,
|
|
||||||
(int(corner[0]), int(corner[1])),
|
|
||||||
8,
|
|
||||||
(0, 255, 255), # Yellow
|
|
||||||
-1
|
|
||||||
)
|
|
||||||
# Label corners
|
|
||||||
cv2.putText(
|
|
||||||
frame,
|
|
||||||
str(i),
|
|
||||||
(int(corner[0]) + 12, int(corner[1])),
|
|
||||||
cv2.FONT_HERSHEY_SIMPLEX,
|
|
||||||
0.6,
|
|
||||||
(0, 255, 255),
|
|
||||||
2
|
|
||||||
)
|
|
||||||
|
|
||||||
# Save image
|
|
||||||
output_path = Path(f"data/{run_id}/court_polygon.jpg")
|
|
||||||
cv2.imwrite(str(output_path), frame)
|
|
||||||
context.log.info(f"✓ Saved court polygon visualization to {output_path}")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"image_path": str(output_path)
|
|
||||||
}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
"""IO Managers for Dagster assets"""
|
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
"""JSON IO Manager for storing asset outputs as JSON files"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
from dagster import IOManager, io_manager, OutputContext, InputContext
|
|
||||||
|
|
||||||
|
|
||||||
class JSONIOManager(IOManager):
|
|
||||||
"""IO Manager that stores outputs as JSON files in data/{run_id}/ directory"""
|
|
||||||
|
|
||||||
def __init__(self, base_path: str = "data"):
|
|
||||||
self.base_path = Path(base_path)
|
|
||||||
self.base_path.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
def _get_path(self, context) -> Path:
|
|
||||||
"""Get file path for asset with run_id subdirectory"""
|
|
||||||
asset_name = context.asset_key.path[-1]
|
|
||||||
|
|
||||||
# For InputContext, try upstream run_id first, fallback to finding latest
|
|
||||||
if isinstance(context, InputContext):
|
|
||||||
try:
|
|
||||||
run_id = context.upstream_output.run_id
|
|
||||||
except:
|
|
||||||
# If upstream run_id not available, find the most recent run directory
|
|
||||||
# that contains this asset (for partial re-runs)
|
|
||||||
run_dirs = sorted([d for d in self.base_path.iterdir() if d.is_dir()],
|
|
||||||
key=lambda d: d.stat().st_mtime, reverse=True)
|
|
||||||
for run_dir in run_dirs:
|
|
||||||
potential_path = run_dir / f"{asset_name}.json"
|
|
||||||
if potential_path.exists():
|
|
||||||
return potential_path
|
|
||||||
# If not found, use the latest run_dir
|
|
||||||
run_id = run_dirs[0].name if run_dirs else "unknown"
|
|
||||||
else:
|
|
||||||
run_id = context.run_id
|
|
||||||
|
|
||||||
# Create run-specific directory
|
|
||||||
run_dir = self.base_path / run_id
|
|
||||||
run_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
return run_dir / f"{asset_name}.json"
|
|
||||||
|
|
||||||
def handle_output(self, context: OutputContext, obj: Any):
|
|
||||||
"""Save asset output to JSON file"""
|
|
||||||
file_path = self._get_path(context)
|
|
||||||
|
|
||||||
with open(file_path, 'w') as f:
|
|
||||||
json.dump(obj, f, indent=2)
|
|
||||||
|
|
||||||
context.log.info(f"Saved {context.asset_key.path[-1]} to {file_path}")
|
|
||||||
|
|
||||||
def load_input(self, context: InputContext) -> Any:
|
|
||||||
"""Load asset input from JSON file"""
|
|
||||||
file_path = self._get_path(context)
|
|
||||||
|
|
||||||
if not file_path.exists():
|
|
||||||
raise FileNotFoundError(f"Asset output not found: {file_path}")
|
|
||||||
|
|
||||||
with open(file_path, 'r') as f:
|
|
||||||
obj = json.load(f)
|
|
||||||
|
|
||||||
context.log.info(f"Loaded {context.asset_key.path[-1]} from {file_path}")
|
|
||||||
return obj
|
|
||||||
|
|
||||||
|
|
||||||
@io_manager
|
|
||||||
def json_io_manager():
|
|
||||||
"""Factory for JSON IO Manager"""
|
|
||||||
return JSONIOManager(base_path="data")
|
|
||||||
@@ -1,34 +1,20 @@
|
|||||||
version: '3.8'
|
version: '3.8'
|
||||||
|
|
||||||
services:
|
services:
|
||||||
dagster:
|
pickle-vision:
|
||||||
build: .
|
build: .
|
||||||
container_name: pickle-dagster
|
container_name: pickle-vision
|
||||||
ports:
|
ports:
|
||||||
- "3000:3000"
|
- "8080:8080"
|
||||||
volumes:
|
volumes:
|
||||||
# Mount data directory for pipeline outputs (frames, detections, JSON)
|
# Runtime outputs and configs
|
||||||
- ./data:/app/data
|
- ./data:/app/data
|
||||||
# Mount dagster_home for Dagster metadata (history, logs, storage)
|
- ./jetson/config:/app/jetson/config
|
||||||
- ./dagster_home:/app/dagster_home
|
|
||||||
# Mount models directory
|
|
||||||
- ./models:/app/models
|
- ./models:/app/models
|
||||||
# Mount video file
|
|
||||||
- ./DJI_0017.MP4:/app/DJI_0017.MP4
|
|
||||||
# Mount source code for hot reload
|
|
||||||
- ./dagster_project:/app/dagster_project
|
|
||||||
- ./src:/app/src
|
- ./src:/app/src
|
||||||
|
- ./jetson:/app/jetson
|
||||||
environment:
|
environment:
|
||||||
- PYTHONUNBUFFERED=1
|
- PYTHONUNBUFFERED=1
|
||||||
- DAGSTER_HOME=/app/dagster_home
|
|
||||||
- ROBOFLOW_API_KEY=JxrPOJZjb5lwHw0pnxey
|
- ROBOFLOW_API_KEY=JxrPOJZjb5lwHw0pnxey
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
command: dagster dev -m dagster_project --host 0.0.0.0 --port 3000
|
command: python3 jetson/main.py --port 8080
|
||||||
|
|
||||||
# Optional: Redis for Celery (if you want to add it later)
|
|
||||||
# redis:
|
|
||||||
# image: redis:7-alpine
|
|
||||||
# container_name: pickle-redis
|
|
||||||
# ports:
|
|
||||||
# - "6379:6379"
|
|
||||||
# restart: unless-stopped
|
|
||||||
|
|||||||
159
jetson/main.py
159
jetson/main.py
@@ -34,6 +34,19 @@ _cam_readers = {}
|
|||||||
_args = None
|
_args = None
|
||||||
|
|
||||||
|
|
||||||
|
def _init_calibration_steps():
|
||||||
|
"""Initialize ordered calibration step status map for UI."""
|
||||||
|
return {
|
||||||
|
'green_mask': {'status': 'pending', 'detail': 'Not started'},
|
||||||
|
'line_segments': {'status': 'pending', 'detail': 'Not started'},
|
||||||
|
'merged_lines': {'status': 'pending', 'detail': 'Not started'},
|
||||||
|
'intersections': {'status': 'pending', 'detail': 'Not started'},
|
||||||
|
'template_match': {'status': 'pending', 'detail': 'Not started'},
|
||||||
|
'camera_pose': {'status': 'pending', 'detail': 'Not started'},
|
||||||
|
'overlay': {'status': 'pending', 'detail': 'Not started'},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def auto_calibrate():
|
def auto_calibrate():
|
||||||
"""Calibrate cameras by detecting court line intersections and matching
|
"""Calibrate cameras by detecting court line intersections and matching
|
||||||
them to the known pickleball court template.
|
them to the known pickleball court template.
|
||||||
@@ -51,9 +64,21 @@ def auto_calibrate():
|
|||||||
results = {}
|
results = {}
|
||||||
|
|
||||||
for sensor_id, reader in _cam_readers.items():
|
for sensor_id, reader in _cam_readers.items():
|
||||||
|
steps = _init_calibration_steps()
|
||||||
frame = reader.grab()
|
frame = reader.grab()
|
||||||
if frame is None:
|
if frame is None:
|
||||||
results[str(sensor_id)] = {'ok': False, 'error': 'No frame available'}
|
steps['green_mask'] = {'status': 'fail', 'detail': 'No frame available'}
|
||||||
|
steps['line_segments'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['merged_lines'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['intersections'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['template_match'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['camera_pose'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['overlay'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
results[str(sensor_id)] = {
|
||||||
|
'ok': False,
|
||||||
|
'error': f'CAM {sensor_id}: No frame available',
|
||||||
|
'steps': steps,
|
||||||
|
}
|
||||||
continue
|
continue
|
||||||
|
|
||||||
h, w = frame.shape[:2]
|
h, w = frame.shape[:2]
|
||||||
@@ -68,6 +93,13 @@ def auto_calibrate():
|
|||||||
|
|
||||||
green_pct = np.count_nonzero(green_mask) / (w * h) * 100
|
green_pct = np.count_nonzero(green_mask) / (w * h) * 100
|
||||||
if green_pct < 5:
|
if green_pct < 5:
|
||||||
|
steps['green_mask'] = {'status': 'fail', 'detail': f'Green area too small: {green_pct:.1f}%'}
|
||||||
|
steps['line_segments'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['merged_lines'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['intersections'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['template_match'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['camera_pose'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
|
steps['overlay'] = {'status': 'fail', 'detail': 'Calibration aborted'}
|
||||||
cv2.putText(debug_frame, f"FAILED: Green area too small ({green_pct:.0f}%)",
|
cv2.putText(debug_frame, f"FAILED: Green area too small ({green_pct:.0f}%)",
|
||||||
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
|
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
|
||||||
_, jpeg = cv2.imencode('.jpg', debug_frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
|
_, jpeg = cv2.imencode('.jpg', debug_frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
|
||||||
@@ -75,13 +107,19 @@ def auto_calibrate():
|
|||||||
'ok': False,
|
'ok': False,
|
||||||
'error': f'CAM {sensor_id}: Green area too small ({green_pct:.0f}%)',
|
'error': f'CAM {sensor_id}: Green area too small ({green_pct:.0f}%)',
|
||||||
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
||||||
|
'steps': steps,
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
|
steps['green_mask'] = {'status': 'ok', 'detail': f'Green area: {green_pct:.1f}%'}
|
||||||
|
|
||||||
# Step 2: Detect white line segments on court
|
# Step 2: Detect white line segments on court
|
||||||
white_segments = _detect_white_lines_on_court(frame, green_mask)
|
white_segments = _detect_white_lines_on_court(frame, green_mask)
|
||||||
for x1, y1, x2, y2 in white_segments:
|
for x1, y1, x2, y2 in white_segments:
|
||||||
cv2.line(debug_frame, (x1, y1), (x2, y2), (255, 255, 0), 1)
|
cv2.line(debug_frame, (x1, y1), (x2, y2), (255, 255, 0), 1)
|
||||||
|
if white_segments:
|
||||||
|
steps['line_segments'] = {'status': 'ok', 'detail': f'{len(white_segments)} segments detected'}
|
||||||
|
else:
|
||||||
|
steps['line_segments'] = {'status': 'fail', 'detail': '0 segments detected'}
|
||||||
|
|
||||||
# Step 3: Merge into distinct lines
|
# Step 3: Merge into distinct lines
|
||||||
merged = _merge_line_segments(white_segments)
|
merged = _merge_line_segments(white_segments)
|
||||||
@@ -89,28 +127,55 @@ def auto_calibrate():
|
|||||||
p1, p2 = m['p1'], m['p2']
|
p1, p2 = m['p1'], m['p2']
|
||||||
cv2.line(debug_frame, (int(p1[0]), int(p1[1])),
|
cv2.line(debug_frame, (int(p1[0]), int(p1[1])),
|
||||||
(int(p2[0]), int(p2[1])), (0, 165, 255), 2)
|
(int(p2[0]), int(p2[1])), (0, 165, 255), 2)
|
||||||
|
if merged:
|
||||||
|
steps['merged_lines'] = {'status': 'ok', 'detail': f'{len(merged)} merged lines'}
|
||||||
|
else:
|
||||||
|
steps['merged_lines'] = {'status': 'fail', 'detail': '0 merged lines'}
|
||||||
|
|
||||||
# Step 4: Find intersections
|
# Step 4: Find intersections
|
||||||
intersections = _find_line_intersections(merged, w, h)
|
intersections = _find_line_intersections(merged, w, h)
|
||||||
for ix, iy in intersections:
|
for ix, iy in intersections:
|
||||||
cv2.circle(debug_frame, (int(ix), int(iy)), 6, (0, 0, 255), -1)
|
cv2.circle(debug_frame, (int(ix), int(iy)), 6, (0, 0, 255), -1)
|
||||||
|
if intersections:
|
||||||
|
steps['intersections'] = {'status': 'ok', 'detail': f'{len(intersections)} intersections'}
|
||||||
|
else:
|
||||||
|
steps['intersections'] = {'status': 'fail', 'detail': '0 intersections'}
|
||||||
|
|
||||||
cv2.putText(debug_frame,
|
cv2.putText(debug_frame,
|
||||||
f"{len(white_segments)} segs -> {len(merged)} lines -> {len(intersections)} pts",
|
f"{len(white_segments)} segs -> {len(merged)} lines -> {len(intersections)} pts",
|
||||||
(10, h - 15), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 255), 1)
|
(10, h - 15), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 255), 1)
|
||||||
|
|
||||||
|
template = get_half_court_intersections(side)
|
||||||
|
quad = _find_green_quad(green_mask, w * h)
|
||||||
|
|
||||||
if len(intersections) < 4:
|
if len(intersections) < 4:
|
||||||
# Fallback: try green quad + both mappings
|
# Fallback: try green quad + both mappings
|
||||||
quad = _find_green_quad(green_mask, w * h)
|
|
||||||
if quad is not None:
|
if quad is not None:
|
||||||
cal, cam_pos, mapping_info = _calibrate_from_quad(quad, side, w, h)
|
cal, cam_pos, mapping_info = _calibrate_from_quad(quad, side, w, h)
|
||||||
if cal is not None:
|
if cal is not None:
|
||||||
|
steps['template_match'] = {
|
||||||
|
'status': 'ok',
|
||||||
|
'detail': 'Low intersections, used quad fallback mapping'
|
||||||
|
}
|
||||||
_finalize_calibration(
|
_finalize_calibration(
|
||||||
cal, cam_pos, sensor_id, side, debug_frame, w, h,
|
cal=cal,
|
||||||
results, mapping_info, len(intersections), len(merged)
|
cam_pos=cam_pos,
|
||||||
|
sensor_id=sensor_id,
|
||||||
|
side=side,
|
||||||
|
debug_frame=debug_frame,
|
||||||
|
results=results,
|
||||||
|
method_info=mapping_info,
|
||||||
|
steps=steps,
|
||||||
|
n_segments=len(white_segments),
|
||||||
|
n_lines=len(merged),
|
||||||
|
n_intersections=len(intersections),
|
||||||
|
n_points_matched=0,
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
steps['template_match'] = {'status': 'fail', 'detail': 'Not enough intersections for template fit'}
|
||||||
|
steps['camera_pose'] = {'status': 'fail', 'detail': 'solvePnP skipped'}
|
||||||
|
steps['overlay'] = {'status': 'fail', 'detail': 'No calibration output'}
|
||||||
cv2.putText(debug_frame, f"FAILED: Need 4+ intersections, got {len(intersections)}",
|
cv2.putText(debug_frame, f"FAILED: Need 4+ intersections, got {len(intersections)}",
|
||||||
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
|
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
|
||||||
_, jpeg = cv2.imencode('.jpg', debug_frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
|
_, jpeg = cv2.imencode('.jpg', debug_frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
|
||||||
@@ -118,14 +183,11 @@ def auto_calibrate():
|
|||||||
'ok': False,
|
'ok': False,
|
||||||
'error': f'CAM {sensor_id}: {len(intersections)} intersections (need 4+) from {len(merged)} lines',
|
'error': f'CAM {sensor_id}: {len(intersections)} intersections (need 4+) from {len(merged)} lines',
|
||||||
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
||||||
|
'steps': steps,
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
|
|
||||||
# Step 5: Match intersections to court template
|
# Step 5: Match intersections to court template
|
||||||
template = get_half_court_intersections(side)
|
|
||||||
|
|
||||||
# Try matching with green quad seeded homography
|
|
||||||
quad = _find_green_quad(green_mask, w * h)
|
|
||||||
match = _match_intersections_to_template(
|
match = _match_intersections_to_template(
|
||||||
intersections, template, side, quad, w, h
|
intersections, template, side, quad, w, h
|
||||||
)
|
)
|
||||||
@@ -135,12 +197,29 @@ def auto_calibrate():
|
|||||||
if quad is not None:
|
if quad is not None:
|
||||||
cal, cam_pos, mapping_info = _calibrate_from_quad(quad, side, w, h)
|
cal, cam_pos, mapping_info = _calibrate_from_quad(quad, side, w, h)
|
||||||
if cal is not None:
|
if cal is not None:
|
||||||
|
steps['template_match'] = {
|
||||||
|
'status': 'ok',
|
||||||
|
'detail': 'Template match weak, used quad fallback mapping'
|
||||||
|
}
|
||||||
_finalize_calibration(
|
_finalize_calibration(
|
||||||
cal, cam_pos, sensor_id, side, debug_frame, w, h,
|
cal=cal,
|
||||||
results, mapping_info, len(intersections), len(merged)
|
cam_pos=cam_pos,
|
||||||
|
sensor_id=sensor_id,
|
||||||
|
side=side,
|
||||||
|
debug_frame=debug_frame,
|
||||||
|
results=results,
|
||||||
|
method_info=mapping_info,
|
||||||
|
steps=steps,
|
||||||
|
n_segments=len(white_segments),
|
||||||
|
n_lines=len(merged),
|
||||||
|
n_intersections=len(intersections),
|
||||||
|
n_points_matched=0,
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
steps['template_match'] = {'status': 'fail', 'detail': 'Could not match intersections to template'}
|
||||||
|
steps['camera_pose'] = {'status': 'fail', 'detail': 'solvePnP skipped'}
|
||||||
|
steps['overlay'] = {'status': 'fail', 'detail': 'No calibration output'}
|
||||||
cv2.putText(debug_frame, "FAILED: Could not match court pattern",
|
cv2.putText(debug_frame, "FAILED: Could not match court pattern",
|
||||||
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
|
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
|
||||||
_, jpeg = cv2.imencode('.jpg', debug_frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
|
_, jpeg = cv2.imencode('.jpg', debug_frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
|
||||||
@@ -148,9 +227,16 @@ def auto_calibrate():
|
|||||||
'ok': False,
|
'ok': False,
|
||||||
'error': f'CAM {sensor_id}: Could not match court pattern',
|
'error': f'CAM {sensor_id}: Could not match court pattern',
|
||||||
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
||||||
|
'steps': steps,
|
||||||
}
|
}
|
||||||
continue
|
continue
|
||||||
|
|
||||||
|
n_matched = len(match['image_points'])
|
||||||
|
steps['template_match'] = {
|
||||||
|
'status': 'ok',
|
||||||
|
'detail': f'Matched {n_matched}/{len(template)} template points'
|
||||||
|
}
|
||||||
|
|
||||||
# Draw matched points
|
# Draw matched points
|
||||||
for name, (px, py) in match['image_points'].items():
|
for name, (px, py) in match['image_points'].items():
|
||||||
cv2.circle(debug_frame, (int(px), int(py)), 10, (0, 255, 0), 2)
|
cv2.circle(debug_frame, (int(px), int(py)), 10, (0, 255, 0), 2)
|
||||||
@@ -162,8 +248,22 @@ def auto_calibrate():
|
|||||||
pts_3d = np.array(list(match['world_points'].values()), dtype=np.float32)
|
pts_3d = np.array(list(match['world_points'].values()), dtype=np.float32)
|
||||||
|
|
||||||
cal = CameraCalibrator()
|
cal = CameraCalibrator()
|
||||||
cal.calibrate(pts_2d, pts_3d, w, h)
|
try:
|
||||||
|
cal.calibrate(pts_2d, pts_3d, w, h)
|
||||||
|
except RuntimeError as exc:
|
||||||
|
steps['camera_pose'] = {'status': 'fail', 'detail': str(exc)}
|
||||||
|
steps['overlay'] = {'status': 'fail', 'detail': 'No calibration output'}
|
||||||
|
_, jpeg = cv2.imencode('.jpg', debug_frame, [cv2.IMWRITE_JPEG_QUALITY, 85])
|
||||||
|
results[str(sensor_id)] = {
|
||||||
|
'ok': False,
|
||||||
|
'error': f'CAM {sensor_id}: solvePnP failed ({exc})',
|
||||||
|
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
||||||
|
'steps': steps,
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
|
||||||
cam_pos = (-cal.rotation_matrix.T @ cal.translation_vec).flatten()
|
cam_pos = (-cal.rotation_matrix.T @ cal.translation_vec).flatten()
|
||||||
|
method_info = f"{n_matched} intersections matched"
|
||||||
|
|
||||||
# Sanity check
|
# Sanity check
|
||||||
if cam_pos[2] < 0 or cam_pos[2] > 10:
|
if cam_pos[2] < 0 or cam_pos[2] > 10:
|
||||||
@@ -173,19 +273,39 @@ def auto_calibrate():
|
|||||||
cal2, cam_pos2, info = _calibrate_from_quad(quad, side, w, h)
|
cal2, cam_pos2, info = _calibrate_from_quad(quad, side, w, h)
|
||||||
if cal2 is not None:
|
if cal2 is not None:
|
||||||
cal, cam_pos = cal2, cam_pos2
|
cal, cam_pos = cal2, cam_pos2
|
||||||
|
method_info = f"{method_info}; fallback={info}"
|
||||||
|
|
||||||
n_matched = len(match['image_points'])
|
|
||||||
_finalize_calibration(
|
_finalize_calibration(
|
||||||
cal, cam_pos, sensor_id, side, debug_frame, w, h,
|
cal=cal,
|
||||||
results, f"{n_matched} intersections matched", len(intersections), len(merged)
|
cam_pos=cam_pos,
|
||||||
|
sensor_id=sensor_id,
|
||||||
|
side=side,
|
||||||
|
debug_frame=debug_frame,
|
||||||
|
results=results,
|
||||||
|
method_info=method_info,
|
||||||
|
steps=steps,
|
||||||
|
n_segments=len(white_segments),
|
||||||
|
n_lines=len(merged),
|
||||||
|
n_intersections=len(intersections),
|
||||||
|
n_points_matched=n_matched,
|
||||||
)
|
)
|
||||||
|
|
||||||
return results
|
return results
|
||||||
|
|
||||||
|
|
||||||
def _finalize_calibration(cal, cam_pos, sensor_id, side, debug_frame, w, h,
|
def _finalize_calibration(cal, cam_pos, sensor_id, side, debug_frame,
|
||||||
results, method_info, n_intersections, n_lines):
|
results, method_info, steps, n_segments, n_lines,
|
||||||
|
n_intersections, n_points_matched):
|
||||||
"""Save calibration and build result dict."""
|
"""Save calibration and build result dict."""
|
||||||
|
steps['camera_pose'] = {
|
||||||
|
'status': 'ok',
|
||||||
|
'detail': f'Camera pose solved: X={cam_pos[0]:.2f}, Y={cam_pos[1]:.2f}, Z={cam_pos[2]:.2f}m'
|
||||||
|
}
|
||||||
|
steps['overlay'] = {
|
||||||
|
'status': 'ok',
|
||||||
|
'detail': f'Geometry projected with {n_points_matched} matched points'
|
||||||
|
}
|
||||||
|
|
||||||
cv2.putText(debug_frame,
|
cv2.putText(debug_frame,
|
||||||
f"CAM{sensor_id}: X={cam_pos[0]:.2f} Y={cam_pos[1]:.2f} Z={cam_pos[2]:.2f}m",
|
f"CAM{sensor_id}: X={cam_pos[0]:.2f} Y={cam_pos[1]:.2f} Z={cam_pos[2]:.2f}m",
|
||||||
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2)
|
(10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2)
|
||||||
@@ -208,7 +328,12 @@ def _finalize_calibration(cal, cam_pos, sensor_id, side, debug_frame, w, h,
|
|||||||
'camera_position': cam_pos.tolist(),
|
'camera_position': cam_pos.tolist(),
|
||||||
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
'debug_image': base64.b64encode(jpeg.tobytes()).decode('ascii'),
|
||||||
'matched_lines_3d': matched_lines_3d,
|
'matched_lines_3d': matched_lines_3d,
|
||||||
'points_matched': n_intersections,
|
'points_matched': n_points_matched,
|
||||||
|
'intersections_detected': n_intersections,
|
||||||
|
'lines_detected': n_lines,
|
||||||
|
'segments_detected': n_segments,
|
||||||
|
'method': method_info,
|
||||||
|
'steps': steps,
|
||||||
}
|
}
|
||||||
print(f"[CAM {sensor_id}] Calibrated! Camera at "
|
print(f"[CAM {sensor_id}] Calibrated! Camera at "
|
||||||
f"({cam_pos[0]:.2f}, {cam_pos[1]:.2f}, {cam_pos[2]:.2f}) via {method_info}")
|
f"({cam_pos[0]:.2f}, {cam_pos[1]:.2f}, {cam_pos[2]:.2f}) via {method_info}")
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
|
|||||||
[project]
|
[project]
|
||||||
name = "pickle-ball-tracking"
|
name = "pickle-ball-tracking"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
description = "Pickleball ball tracking system using YOLO and Dagster"
|
description = "Pickleball referee system with dual-camera calibration and ball tracking"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"ultralytics>=8.0.0",
|
"ultralytics>=8.0.0",
|
||||||
@@ -14,8 +14,6 @@ dependencies = [
|
|||||||
"supervision>=0.16.0",
|
"supervision>=0.16.0",
|
||||||
"opencv-python>=4.8.0",
|
"opencv-python>=4.8.0",
|
||||||
"numpy>=1.24.0",
|
"numpy>=1.24.0",
|
||||||
"dagster>=1.5.0",
|
|
||||||
"dagster-webserver>=1.5.0",
|
|
||||||
"matplotlib>=3.8.0",
|
"matplotlib>=3.8.0",
|
||||||
"fastapi>=0.104.0",
|
"fastapi>=0.104.0",
|
||||||
"uvicorn>=0.24.0",
|
"uvicorn>=0.24.0",
|
||||||
@@ -25,9 +23,6 @@ dependencies = [
|
|||||||
"tqdm>=4.66.0",
|
"tqdm>=4.66.0",
|
||||||
]
|
]
|
||||||
|
|
||||||
[tool.dagster]
|
|
||||||
module_name = "dagster_project"
|
|
||||||
|
|
||||||
[tool.setuptools.packages.find]
|
[tool.setuptools.packages.find]
|
||||||
where = ["."]
|
where = ["."]
|
||||||
include = ["dagster_project*", "src*"]
|
include = ["src*"]
|
||||||
|
|||||||
@@ -16,10 +16,6 @@ pydantic>=2.0.0
|
|||||||
# celery>=5.3.0
|
# celery>=5.3.0
|
||||||
# redis>=5.0.0
|
# redis>=5.0.0
|
||||||
|
|
||||||
# Dagster
|
|
||||||
dagster>=1.5.0
|
|
||||||
dagster-webserver>=1.5.0
|
|
||||||
|
|
||||||
# Roboflow Hosted Inference
|
# Roboflow Hosted Inference
|
||||||
inference-sdk>=0.9.0
|
inference-sdk>=0.9.0
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""
|
"""
|
||||||
3D camera calibration using court geometry.
|
3D camera calibration using court geometry.
|
||||||
Extracted from dagster_project/assets/camera_calibration.py and coordinate_transform_3d.py.
|
Standalone runtime module for calibration and 3D projection.
|
||||||
|
|
||||||
For our setup: two cameras mounted at the net center, looking at opposite halves.
|
For our setup: two cameras mounted at the net center, looking at opposite halves.
|
||||||
Each camera gets its own calibration.
|
Each camera gets its own calibration.
|
||||||
|
|||||||
@@ -12,17 +12,13 @@
|
|||||||
background: #0a0a1a;
|
background: #0a0a1a;
|
||||||
color: #e0e0e0;
|
color: #e0e0e0;
|
||||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
|
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', system-ui, sans-serif;
|
||||||
overflow: hidden;
|
min-height: 100vh;
|
||||||
height: 100vh;
|
overflow-x: hidden;
|
||||||
|
overflow-y: auto;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Header */
|
/* Header */
|
||||||
.header {
|
.header {
|
||||||
position: fixed;
|
|
||||||
top: 0;
|
|
||||||
left: 0;
|
|
||||||
right: 0;
|
|
||||||
z-index: 10;
|
|
||||||
display: flex;
|
display: flex;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
justify-content: space-between;
|
justify-content: space-between;
|
||||||
@@ -68,7 +64,7 @@
|
|||||||
gap: 8px;
|
gap: 8px;
|
||||||
padding: 8px;
|
padding: 8px;
|
||||||
justify-content: center;
|
justify-content: center;
|
||||||
height: calc(100vh - 48px);
|
align-items: flex-start;
|
||||||
}
|
}
|
||||||
.cam-box {
|
.cam-box {
|
||||||
flex: 1;
|
flex: 1;
|
||||||
@@ -104,10 +100,10 @@
|
|||||||
|
|
||||||
/* Full-height tab layout */
|
/* Full-height tab layout */
|
||||||
.tab-content {
|
.tab-content {
|
||||||
padding-top: 48px;
|
padding: 8px 12px 12px;
|
||||||
}
|
}
|
||||||
.tab-full {
|
.tab-full {
|
||||||
height: calc(100vh - 48px - 100px);
|
min-height: 420px;
|
||||||
}
|
}
|
||||||
.tab-full .viewport-3d {
|
.tab-full .viewport-3d {
|
||||||
width: 100%;
|
width: 100%;
|
||||||
@@ -115,11 +111,6 @@
|
|||||||
border-top: none;
|
border-top: none;
|
||||||
}
|
}
|
||||||
.bottom-bar {
|
.bottom-bar {
|
||||||
position: fixed;
|
|
||||||
bottom: 0;
|
|
||||||
left: 0;
|
|
||||||
right: 0;
|
|
||||||
z-index: 10;
|
|
||||||
display: flex;
|
display: flex;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
justify-content: center;
|
justify-content: center;
|
||||||
@@ -127,7 +118,7 @@
|
|||||||
padding: 4px 8px;
|
padding: 4px 8px;
|
||||||
background: #0d0d20;
|
background: #0d0d20;
|
||||||
border-top: 1px solid #222;
|
border-top: 1px solid #222;
|
||||||
height: 100px;
|
min-height: 100px;
|
||||||
}
|
}
|
||||||
.bottom-card {
|
.bottom-card {
|
||||||
width: 120px;
|
width: 120px;
|
||||||
@@ -260,10 +251,9 @@
|
|||||||
/* Event banner */
|
/* Event banner */
|
||||||
.event-banner {
|
.event-banner {
|
||||||
display: none;
|
display: none;
|
||||||
position: fixed;
|
position: relative;
|
||||||
top: 60px;
|
margin: 8px auto;
|
||||||
left: 50%;
|
width: fit-content;
|
||||||
transform: translateX(-50%);
|
|
||||||
background: rgba(255, 60, 60, 0.95);
|
background: rgba(255, 60, 60, 0.95);
|
||||||
color: white;
|
color: white;
|
||||||
padding: 12px 32px;
|
padding: 12px 32px;
|
||||||
@@ -293,42 +283,111 @@
|
|||||||
.info-item .label { color: #666; }
|
.info-item .label { color: #666; }
|
||||||
.info-item .value { color: #4ecca3; font-weight: 600; margin-left: 4px; }
|
.info-item .value { color: #4ecca3; font-weight: 600; margin-left: 4px; }
|
||||||
|
|
||||||
/* Calibration split layout */
|
/* Calibration flow layout (scrollable) */
|
||||||
.cal-split {
|
.calibration-page {
|
||||||
display: flex;
|
|
||||||
height: calc(100vh - 48px);
|
|
||||||
}
|
|
||||||
.cal-left {
|
|
||||||
flex: 1;
|
|
||||||
min-width: 0;
|
|
||||||
}
|
|
||||||
.cal-left .viewport-3d {
|
|
||||||
width: 100%;
|
|
||||||
height: 100%;
|
|
||||||
border-top: none;
|
|
||||||
}
|
|
||||||
.cal-right {
|
|
||||||
width: 360px;
|
|
||||||
flex-shrink: 0;
|
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
border-left: 1px solid #2a2a4a;
|
gap: 12px;
|
||||||
|
}
|
||||||
|
.cal-cameras {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr 1fr;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
.cal-cam-card {
|
||||||
|
position: relative;
|
||||||
|
border: 1px solid #2a2a4a;
|
||||||
|
border-radius: 6px;
|
||||||
|
overflow: hidden;
|
||||||
background: #0d0d20;
|
background: #0d0d20;
|
||||||
overflow-y: auto;
|
}
|
||||||
|
.cal-cam-card img {
|
||||||
|
width: 100%;
|
||||||
|
display: block;
|
||||||
|
}
|
||||||
|
.cal-cam-card .cam-label {
|
||||||
|
z-index: 1;
|
||||||
}
|
}
|
||||||
.cal-controls {
|
.cal-controls {
|
||||||
padding: 16px;
|
padding: 16px;
|
||||||
border-bottom: 1px solid #2a2a4a;
|
border: 1px solid #2a2a4a;
|
||||||
|
border-radius: 6px;
|
||||||
|
background: #0d0d20;
|
||||||
}
|
}
|
||||||
.cal-debug-images {
|
.flow-grid {
|
||||||
flex: 1;
|
display: grid;
|
||||||
|
grid-template-columns: 1fr 1fr;
|
||||||
|
gap: 12px;
|
||||||
|
}
|
||||||
|
.flow-card {
|
||||||
|
border: 1px solid #2a2a4a;
|
||||||
|
border-radius: 6px;
|
||||||
|
background: #0d0d20;
|
||||||
|
padding: 12px;
|
||||||
|
}
|
||||||
|
.flow-title {
|
||||||
|
font-size: 12px;
|
||||||
|
color: #aaa;
|
||||||
|
margin-bottom: 8px;
|
||||||
|
letter-spacing: 0.4px;
|
||||||
|
text-transform: uppercase;
|
||||||
|
font-weight: 700;
|
||||||
|
}
|
||||||
|
.flow-steps {
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
flex-direction: column;
|
||||||
|
gap: 6px;
|
||||||
|
}
|
||||||
|
.flow-step {
|
||||||
|
border: 1px solid #2a2a4a;
|
||||||
|
border-radius: 4px;
|
||||||
|
padding: 6px 8px;
|
||||||
|
background: #10102a;
|
||||||
|
font-size: 12px;
|
||||||
|
}
|
||||||
|
.flow-step .step-head {
|
||||||
|
display: flex;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
.flow-step .step-name {
|
||||||
|
color: #ddd;
|
||||||
|
font-weight: 600;
|
||||||
|
}
|
||||||
|
.flow-step .step-status {
|
||||||
|
color: #888;
|
||||||
|
font-size: 11px;
|
||||||
|
text-transform: uppercase;
|
||||||
|
}
|
||||||
|
.flow-step .step-detail {
|
||||||
|
margin-top: 4px;
|
||||||
|
color: #8e8eb0;
|
||||||
|
font-size: 11px;
|
||||||
|
}
|
||||||
|
.flow-step.ok .step-status { color: #4ecca3; }
|
||||||
|
.flow-step.fail .step-status { color: #ff6666; }
|
||||||
|
.flow-step.pending .step-status { color: #aaa; }
|
||||||
|
.calibration-3d-wrap {
|
||||||
|
border: 1px solid #2a2a4a;
|
||||||
|
border-radius: 6px;
|
||||||
|
overflow: hidden;
|
||||||
|
background: #0d0d20;
|
||||||
|
}
|
||||||
|
.calibration-3d-wrap .viewport-3d {
|
||||||
|
border-top: none;
|
||||||
|
}
|
||||||
|
.cal-debug-images {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: 1fr 1fr;
|
||||||
gap: 8px;
|
gap: 8px;
|
||||||
padding: 8px;
|
|
||||||
}
|
}
|
||||||
.cal-debug-card {
|
.cal-debug-card {
|
||||||
position: relative;
|
position: relative;
|
||||||
|
border: 1px solid #2a2a4a;
|
||||||
|
border-radius: 6px;
|
||||||
|
overflow: hidden;
|
||||||
|
background: #0d0d20;
|
||||||
|
padding: 8px;
|
||||||
}
|
}
|
||||||
.cal-debug-card img {
|
.cal-debug-card img {
|
||||||
width: 100%;
|
width: 100%;
|
||||||
@@ -374,6 +433,28 @@
|
|||||||
max-height: 95vh;
|
max-height: 95vh;
|
||||||
object-fit: contain;
|
object-fit: contain;
|
||||||
}
|
}
|
||||||
|
@media (max-width: 980px) {
|
||||||
|
.header {
|
||||||
|
flex-direction: column;
|
||||||
|
align-items: flex-start;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
.status-bar {
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
.cameras {
|
||||||
|
flex-direction: column;
|
||||||
|
}
|
||||||
|
.cam-box {
|
||||||
|
max-width: 100%;
|
||||||
|
}
|
||||||
|
.cal-cameras,
|
||||||
|
.flow-grid,
|
||||||
|
.cal-debug-images {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
}
|
||||||
|
}
|
||||||
</style>
|
</style>
|
||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
@@ -408,41 +489,66 @@
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Tab 2: Calibration — split: 3D left, debug right -->
|
<!-- Tab 2: Calibration — full scroll flow -->
|
||||||
<div class="tab-content" id="tab-calibration">
|
<div class="tab-content" id="tab-calibration">
|
||||||
<div class="cal-split">
|
<div class="calibration-page">
|
||||||
<div class="cal-left">
|
<div class="cal-cameras">
|
||||||
|
<div class="cal-cam-card">
|
||||||
|
<div class="cam-label">CAM 1 LIVE</div>
|
||||||
|
<img id="cal-cam1" alt="Camera 1 live">
|
||||||
|
</div>
|
||||||
|
<div class="cal-cam-card">
|
||||||
|
<div class="cam-label">CAM 0 LIVE</div>
|
||||||
|
<img id="cal-cam0" alt="Camera 0 live">
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="cal-controls">
|
||||||
|
<button class="btn-calibrate" id="btnCalibrate" onclick="doCalibrate()" style="padding:8px 24px;font-size:14px">Run Calibration Flow</button>
|
||||||
|
<div class="calibrate-status" id="calStatus" style="margin-top:8px">
|
||||||
|
<span id="calStatusText">Not calibrated</span>
|
||||||
|
</div>
|
||||||
|
<div id="calError" style="color:#ff4444;font-size:11px;word-break:break-all;display:none;margin-top:8px"></div>
|
||||||
|
<div id="calPositions" style="display:none;margin-top:8px;font-size:11px">
|
||||||
|
<div id="calPos0" style="color:#4ecca3"></div>
|
||||||
|
<div id="calPos1" style="color:#ff88cc"></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="flow-grid">
|
||||||
|
<div class="flow-card">
|
||||||
|
<div class="flow-title">CAM 0 Calibration Steps</div>
|
||||||
|
<div id="calFlowCam0" class="flow-steps"></div>
|
||||||
|
</div>
|
||||||
|
<div class="flow-card">
|
||||||
|
<div class="flow-title">CAM 1 Calibration Steps</div>
|
||||||
|
<div id="calFlowCam1" class="flow-steps"></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="cal-debug-images">
|
||||||
|
<div class="cal-debug-card">
|
||||||
|
<div class="cal-debug-label">CAM 0 RESULT</div>
|
||||||
|
<img id="calDebugImg0" alt="CAM 0 calibration result">
|
||||||
|
</div>
|
||||||
|
<div class="cal-debug-card">
|
||||||
|
<div class="cal-debug-label">CAM 1 RESULT</div>
|
||||||
|
<img id="calDebugImg1" alt="CAM 1 calibration result">
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="calibration-3d-wrap">
|
||||||
|
<div class="flow-title" style="padding:12px 12px 0;margin-bottom:0">3D Reconstruction Preview</div>
|
||||||
<div class="viewport-3d" id="calibration-3d"></div>
|
<div class="viewport-3d" id="calibration-3d"></div>
|
||||||
</div>
|
</div>
|
||||||
<div class="cal-right">
|
<div class="flow-card">
|
||||||
<div class="cal-controls">
|
<div class="flow-title">Workflow Legend</div>
|
||||||
<button class="btn-calibrate" id="btnCalibrate" onclick="doCalibrate()" style="padding:8px 24px;font-size:14px">Calibrate</button>
|
<div style="font-size:12px;color:#aaa;line-height:1.5">
|
||||||
<div class="calibrate-status" id="calStatus">
|
1) detect court ROI → 2) detect line segments → 3) merge lines → 4) get intersections →
|
||||||
<span id="calStatusText">Not calibrated</span>
|
5) match to template points → 6) solve camera pose → 7) overlay reconstructed geometry.
|
||||||
</div>
|
Green steps are confirmed, red steps failed, gray steps are pending/skipped.
|
||||||
<div id="calError" style="color:#ff4444;font-size:11px;word-break:break-all;display:none;margin-top:4px"></div>
|
|
||||||
<div id="calPositions" style="display:none;margin-top:6px;font-size:11px">
|
|
||||||
<div id="calPos0" style="color:#4ecca3"></div>
|
|
||||||
<div id="calPos1" style="color:#ff88cc"></div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="cal-debug-images">
|
|
||||||
<div class="cal-debug-card">
|
|
||||||
<div class="cal-debug-label">CAM 0</div>
|
|
||||||
<img id="calDebugImg0" alt="CAM 0">
|
|
||||||
<img id="cal-cam0" class="cal-live" alt="CAM 0 live">
|
|
||||||
</div>
|
|
||||||
<div class="cal-debug-card">
|
|
||||||
<div class="cal-debug-label">CAM 1</div>
|
|
||||||
<img id="calDebugImg1" alt="CAM 1">
|
|
||||||
<img id="cal-cam1" class="cal-live" alt="CAM 1 live">
|
|
||||||
</div>
|
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Tab 3: Trajectory — 3D main, cameras small bottom center -->
|
<!-- Tab 3: Trajectory — 3D main + camera strip -->
|
||||||
<div class="tab-content" id="tab-trajectory">
|
<div class="tab-content" id="tab-trajectory">
|
||||||
<div class="tab-full">
|
<div class="tab-full">
|
||||||
<div class="viewport-3d" id="trajectory-3d"></div>
|
<div class="viewport-3d" id="trajectory-3d"></div>
|
||||||
@@ -502,17 +608,78 @@ window.addEventListener('popstate', function() {
|
|||||||
if (activeTab !== 'camera') switchTab(activeTab);
|
if (activeTab !== 'camera') switchTab(activeTab);
|
||||||
|
|
||||||
// ===================== Calibration =====================
|
// ===================== Calibration =====================
|
||||||
|
var CALIBRATION_STEP_ORDER = [
|
||||||
|
'green_mask',
|
||||||
|
'line_segments',
|
||||||
|
'merged_lines',
|
||||||
|
'intersections',
|
||||||
|
'template_match',
|
||||||
|
'camera_pose',
|
||||||
|
'overlay'
|
||||||
|
];
|
||||||
|
|
||||||
|
var CALIBRATION_STEP_LABELS = {
|
||||||
|
green_mask: '1. Court ROI',
|
||||||
|
line_segments: '2. White Segments',
|
||||||
|
merged_lines: '3. Merge Lines',
|
||||||
|
intersections: '4. Intersections',
|
||||||
|
template_match: '5. Template Match',
|
||||||
|
camera_pose: '6. Camera Pose',
|
||||||
|
overlay: '7. Geometry Overlay'
|
||||||
|
};
|
||||||
|
|
||||||
|
function renderCalibrationSteps(containerId, steps, fallbackError) {
|
||||||
|
var el = document.getElementById(containerId);
|
||||||
|
if (!el) return;
|
||||||
|
|
||||||
|
var html = '';
|
||||||
|
for (var i = 0; i < CALIBRATION_STEP_ORDER.length; i++) {
|
||||||
|
var key = CALIBRATION_STEP_ORDER[i];
|
||||||
|
var item = steps && steps[key] ? steps[key] : { status: 'pending', detail: 'No data yet' };
|
||||||
|
var status = item.status || 'pending';
|
||||||
|
var detail = item.detail || '';
|
||||||
|
html += '<div class="flow-step ' + status + '">';
|
||||||
|
html += ' <div class="step-head">';
|
||||||
|
html += ' <div class="step-name">' + CALIBRATION_STEP_LABELS[key] + '</div>';
|
||||||
|
html += ' <div class="step-status">' + status + '</div>';
|
||||||
|
html += ' </div>';
|
||||||
|
html += ' <div class="step-detail">' + detail + '</div>';
|
||||||
|
html += '</div>';
|
||||||
|
}
|
||||||
|
|
||||||
|
if (fallbackError) {
|
||||||
|
html += '<div class="flow-step fail">';
|
||||||
|
html += ' <div class="step-head"><div class="step-name">Error</div><div class="step-status">fail</div></div>';
|
||||||
|
html += ' <div class="step-detail">' + fallbackError + '</div>';
|
||||||
|
html += '</div>';
|
||||||
|
}
|
||||||
|
|
||||||
|
el.innerHTML = html;
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateCalibrationFlow(result) {
|
||||||
|
var cam0 = result && result['0'] ? result['0'] : null;
|
||||||
|
var cam1 = result && result['1'] ? result['1'] : null;
|
||||||
|
renderCalibrationSteps('calFlowCam0', cam0 ? cam0.steps : null, cam0 && cam0.error ? cam0.error : '');
|
||||||
|
renderCalibrationSteps('calFlowCam1', cam1 ? cam1.steps : null, cam1 && cam1.error ? cam1.error : '');
|
||||||
|
}
|
||||||
|
|
||||||
function doCalibrate() {
|
function doCalibrate() {
|
||||||
var btn = document.getElementById('btnCalibrate');
|
var btn = document.getElementById('btnCalibrate');
|
||||||
var errEl = document.getElementById('calError');
|
var errEl = document.getElementById('calError');
|
||||||
btn.disabled = true;
|
btn.disabled = true;
|
||||||
btn.textContent = 'Calibrating...';
|
btn.textContent = 'Calibrating...';
|
||||||
errEl.style.display = 'none';
|
errEl.style.display = 'none';
|
||||||
|
updateCalibrationFlow({
|
||||||
|
'0': { steps: { green_mask: { status: 'pending', detail: 'Running calibration flow...' } } },
|
||||||
|
'1': { steps: { green_mask: { status: 'pending', detail: 'Running calibration flow...' } } }
|
||||||
|
});
|
||||||
|
|
||||||
fetch('/api/calibration/trigger', { method: 'POST' })
|
fetch('/api/calibration/trigger', { method: 'POST' })
|
||||||
.then(function(r) { return r.json(); })
|
.then(function(r) { return r.json(); })
|
||||||
.then(function(data) {
|
.then(function(data) {
|
||||||
btn.disabled = false;
|
btn.disabled = false;
|
||||||
|
updateCalibrationFlow(data.result || null);
|
||||||
|
|
||||||
// Show debug images from calibration
|
// Show debug images from calibration
|
||||||
if (data.result) {
|
if (data.result) {
|
||||||
@@ -526,7 +693,7 @@ function doCalibrate() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (data.ok) {
|
if (data.ok) {
|
||||||
btn.textContent = 'Re-calibrate';
|
btn.textContent = 'Re-run Calibration Flow';
|
||||||
errEl.style.display = 'none';
|
errEl.style.display = 'none';
|
||||||
updateCalibrationStatus();
|
updateCalibrationStatus();
|
||||||
|
|
||||||
@@ -557,7 +724,7 @@ function doCalibrate() {
|
|||||||
// Highlight detected court lines on 3D scene
|
// Highlight detected court lines on 3D scene
|
||||||
if (data.result) highlightDetectedLines(data.result);
|
if (data.result) highlightDetectedLines(data.result);
|
||||||
} else {
|
} else {
|
||||||
btn.textContent = 'Calibrate';
|
btn.textContent = 'Run Calibration Flow';
|
||||||
// Show errors
|
// Show errors
|
||||||
var errors = [];
|
var errors = [];
|
||||||
if (data.result) {
|
if (data.result) {
|
||||||
@@ -572,9 +739,13 @@ function doCalibrate() {
|
|||||||
})
|
})
|
||||||
.catch(function(e) {
|
.catch(function(e) {
|
||||||
btn.disabled = false;
|
btn.disabled = false;
|
||||||
btn.textContent = 'Calibrate';
|
btn.textContent = 'Run Calibration Flow';
|
||||||
errEl.textContent = String(e);
|
errEl.textContent = String(e);
|
||||||
errEl.style.display = 'block';
|
errEl.style.display = 'block';
|
||||||
|
updateCalibrationFlow({
|
||||||
|
'0': { error: String(e) },
|
||||||
|
'1': { error: String(e) }
|
||||||
|
});
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -614,6 +785,7 @@ function updateCalibrationStatus() {
|
|||||||
}
|
}
|
||||||
// Check on load and periodically
|
// Check on load and periodically
|
||||||
updateCalibrationStatus();
|
updateCalibrationStatus();
|
||||||
|
updateCalibrationFlow(null);
|
||||||
setInterval(updateCalibrationStatus, 3000);
|
setInterval(updateCalibrationStatus, 3000);
|
||||||
|
|
||||||
var cameraMeshes = [];
|
var cameraMeshes = [];
|
||||||
|
|||||||
Reference in New Issue
Block a user