←Back to ProjectsPROJECT // SAMDEF

SAMDEF

Edge Computer Vision System & Satellite Imagery Detection Pipeline

Overview

A high-performance computer vision system built in Rust and CUDA for large-scale satellite imagery (GeoTIFF) processing. Features an asynchronous multi-stage GPU pipeline, spatial grid NMS for deduplication, and a native GPUI (Vulkan) GIS desktop viewer delivering 0.20s/km² inference throughput.

GPU Throughput
0.20 s/km²
Benchmarked Area
227.61 km²
GPU Utilization
85–95% (Sustained)
VRAM Footprint
< 1.5 GB
SAMDEF Visual Overview
Click to zoom & inspect
SAMDEF Interface & TelemetryClick to expand
Application Screenshots & Telemetry Views7 Views
Click any image to zoom & inspect details
Native GPUI GIS Desktop Viewer: Real-time multi-class vector detection (1,256 active targets) & live radar minimap
Zoom
Native GPUI GIS Desktop Viewer: Real-time multi-class vector detection (1,256 active targets) & live radar minimap
Instant Class Soloing: Isolating 469 building footprints with zero GPU texture re-uploads & synchronized radar blips
Zoom
Instant Class Soloing: Isolating 469 building footprints with zero GPU texture re-uploads & synchronized radar blips
Industrial & Storage Depot Scene: Multi-class GeoTIFF overlays with 0.31 m/px GSD telemetry
Zoom
Industrial & Storage Depot Scene: Multi-class GeoTIFF overlays with 0.31 m/px GSD telemetry
1600% Sub-Pixel Magnification & Pinned Target Inspector HUD: Ground dimensions (8.6m x 7.7m) & WGS-84 coordinates
Zoom
1600% Sub-Pixel Magnification & Pinned Target Inspector HUD: Ground dimensions (8.6m x 7.7m) & WGS-84 coordinates
Target Detection Table & Filtering: Isolated heavy vehicle telemetry with one-click GeoJSON & CSV exports
Zoom
Target Detection Table & Filtering: Isolated heavy vehicle telemetry with one-click GeoJSON & CSV exports
One-Click Table Target Focus: Instant jump to highway vehicle detections (#2132) with live radar frustum tracking
Zoom
One-Click Table Target Focus: Instant jump to highway vehicle detections (#2132) with live radar frustum tracking
Interactive Geodesic Haversine Ruler: Measuring true WGS-84 ground distance & forward azimuth (121.0 m | 243.7°)
Zoom
Interactive Geodesic Haversine Ruler: Measuring true WGS-84 ground distance & forward azimuth (121.0 m | 243.7°)
Technologies:
RustCUDA 13ONNX Runtime (ort)YOLO26s (FP16)GPUI (Vulkan)Zenoh P2PPostgreSQLPostGISGDALCrossbeamRayonTokio

THE PROBLEM

Processing high-resolution satellite imagery (typically 3,000×3,000 to 15,000×15,000 multi-spectral GeoTIFFs) presents several key technical challenges:

1. Host Memory Spikes on Large Rasters:

Attempting to read and decompress large multi-spectral GeoTIFFs directly into contiguous host memory can quickly consume tens of gigabytes of RAM, causing severe swap thrashing and out-of-memory errors.

2. GPU Idle Time in Sequential Pipelines:

In a sequential loop (read -> tile on CPU -> upload -> run inference -> parse), the GPU sits idle while waiting for disk I/O and CPU tensor conversions, resulting in low hardware utilization.

3. Duplicate & Cut-Off Detections Along Tile Seams:

Slicing large satellite rasters into smaller tiles bisects objects (such as vessels, aircraft, and large buildings) across tile seams. Without boundary overlap buffers and spatial deduplication, models produce duplicate and cut-off bounding boxes.

4. Preserving Real-World Earth Coordinates:

Standard detection models output local pixel coordinates (e.g., x=450, y=320). Operators and GIS analysts require exact geospatial coordinates (WGS84 Latitude/Longitude, DMS, and ground footprint metrics) for mapping and operational use.

5. Desktop Viewer GPU Memory Overhead:

Rendering ultra-high-resolution rasters (e.g. 3,433×2,642 GeoTIFFs) with thousands of interactive detections requires careful graphics pipeline design. Generating raster overlays per filter toggle consumed ~41 MB of heap per click and caused UI stutter.

6. Bandwidth Limits for Cloud Processing:

Uploading gigabyte-scale satellite rasters over slow network uplinks introduces severe latency and high bandwidth costs. The workflow requires reliable local, air-gapped on-premise execution.

THE SOLUTION

SAMDEF (Satellite Multi-Class Detection & Geospatial Framework) is a local, air-gapped, high-throughput system built in Rust and CUDA. It handles raster ingestion, virtual tiling, GPU tensor inference, spatial deduplication, database persistence, and native GPU-accelerated GIS visualization locally on edge hardware.

Core Architecture Highlights:

•Pipelined Execution: A decoupled 3-stage thread pipeline using bounded Crossbeam channels that overlaps CPU preprocessing, GPU tensor inference, and async disk/network I/O to sustain 85–95% GPU core utilization.
•Virtual Tiling: Reads raster blocks directly via GDAL C-bindings in 896×896 windows with 20% stride overlap, keeping host memory strictly bounded.
•Geodetic Affine Projection: Maps localized pixel bounding boxes back into global geographic coordinates using GDAL 6-parameter GeoTIFF affine transform matrices.
•Spatial Grid NMS: Fast deduplication using spatial hash-buckets, eliminating tile-boundary duplicates efficiently.
•Native GPUI GIS Desktop Viewer (gui_gpui): A GPU-accelerated desktop application built with Rust and GPUI (Vulkan/Blade) delivering 5-layer immediate-mode rendering, sub-pixel cursor-invariant pan/zoom, class soloing, live radar minimap, geodesic Haversine ruler, target inspector HUD, tabular detection drawer, and GeoJSON/CSV/Markdown exports with zero GPU texture re-uploads.
•Binary Messaging Fabric: Uses Zenoh in peer-to-peer mode for low-overhead binary streaming between detection workers and database persistence.

HIGH-LEVEL SYSTEM ARCHITECTURE

+-------------------------------------------------------------------------------------------------+
|                                    SAMDEF SYSTEM ARCHITECTURE                                   |
+-------------------------------------------------------------------------------------------------+

   [ High-Resolution GeoTIFFs (Planar Config 2) ]
                          │
                          ▼
   ┌─────────────────────────────────────────────────────────────┐
   │                   DETECTOR MICROSERVICE                     │
   │  ┌───────────────────────────┐     ┌─────────────────────┐  │
   │  │ Work-Stealing Thread Pool │────▶│ Crossbeam Channel   │  │
   │  │ (GDAL Virtual Tiler)      │     │ (Bounded Task Queue)│  │
   │  └───────────────────────────┘     └──────────┬──────────┘  │
   │                                               │             │
   │  ┌────────────────────────────────────────────▼──────────┐  │
   │  │ Decoupled 3-Stage Pipeline:                           │  │
   │  │  Stage 1: Rayon CPU Preprocessor (Array4<f16>)        │  │
   │  │  Stage 2: Dedicated GPU Runner (CUDA Session Loop)    │  │
   │  │  Stage 3: Spatial Grid NMS & Geodetic Coordinate Map  │  │
   │  └────────────────────────────────────────────┬──────────┘  │
   └───────────────────────────────────────────────┼─────────────┘
                                                   │
                        ┌──────────────────────────┴──────────────────────────┐
                        │                                                     │
                        ▼ (Zenoh P2P Binary Bus: bincode)                     ▼ (Structured JSON / GeoJSON)
         ┌──────────────────────────────┐                      ┌──────────────────────────────┐
         │         DB PROCESSOR         │                      │   NATIVE GIS VIEWER (GPUI)   │
         │  - Async Tokio Zenoh Listener│                      │  - Built with Rust & GPUI    │
         │  - Batched SQL Inserts       │                      │  - 5-Pass Vulkan Paint Layers│
         │  - Spatial GIST Indexing     │                      │  - Zero Texture Re-uploads   │
         │  - Connection Pool Management│                      │  - Sub-Pixel Zoom & Geodesy  │
         └──────────────┬───────────────┘                      │  - Radar, HUD & GIS Exports  │
                        │                                      └──────────────────────────────┘
                        ▼
         ┌──────────────────────────────┐
         │     PostgreSQL Database      │
         │  (Local Geospatial Storage)  │
         └──────────────────────────────┘

SYSTEM ARCHITECTURE & DEEP DIVE

1. The Detector Microservice (apps_deploy/detector)

•Work-Stealing Producer Thread Pool (modules/io/producer.rs):

Scans the input directory for GeoTIFF files and spawns PRODUCER_PARALLELISM persistent worker threads. Each worker pulls image paths from a concurrent work queue, opens rasters using GDAL C-bindings, and slices them into fixed 896×896 RGB tiles with 20% stride overlap (179-pixel buffer) along horizontal and vertical axes.

•Decoupled 3-Stage Consumer Pipeline (modules/io/consumer.rs):

The consumer architecture is decoupled into 3 dedicated OS threads connected by bounded Crossbeam channels (gpu_tx: bounded(4), post_tx: bounded(64)): - Stage 1 (OS Thread A - CPU Preprocessor & Batch Accumulator): Receives raw tile bytes from the input channel, bundles them into batches (optimal batch size = 2), and uses Rayon data parallelism (par_iter) across CPU cores to normalize pixels and transpose HWC to CHW planar format: normalized_val = (raw_byte / 255.0 - mean) / std. Ready Array4<f16> tensors are forwarded to the dedicated GPU channel in sub-milliseconds. - Stage 2 (OS Thread B - Dedicated GPU Inference Runner): A persistent OS thread holding the CUDA Session in ONNX Runtime (ort 2.0.0-rc.13) with native CUDA 13 support. Operates as a pure GPU execution loop without blocking file I/O. Dequeues ready tensors and immediately executes session.run() on the RTX 4060 GPU, forwarding raw output tensors to Stage 3. - Stage 3 (OS Thread C - Postprocessing, NMS & Tokio Async Output): Runs on the main Tokio runtime thread. Parses bounding box coordinates, shifts them by spatial tile offsets, tracks per-file completion in a HashMap, runs Spatial Grid NMS, and spawns background Tokio JoinSet tasks to write JSON and publish via Zenoh.

•Concurrency Advantage:

- Thread A prepares Batch N+1 on CPU cores. - Thread B executes Batch N on the RTX 4060 GPU simultaneously. - Thread C saves Batch N-1 to disk and network simultaneously. - Bounded channels provide automatic backpressure: Thread 1 buffers up to 4 preprocessed batches in memory. When Thread 2 finishes a batch, the next tensor is pulled in 0 microseconds without waiting for CPU math. If CPU preprocessing outpaces GPU execution, gpu_tx.send() pauses Thread 1, preventing unbounded memory growth in RAM.

•Geodetic Affine Projection:

Every detected pixel coordinate is mathematically projected onto the Earth's surface using the GeoTIFF affine transformation matrix: X_geo = GeoTransform[0] + X_pixel * GeoTransform[1] + Y_pixel * GeoTransform[2] Y_geo = GeoTransform[3] + X_pixel * GeoTransform[4] + Y_pixel * GeoTransform[5]

•Spatial Grid NMS:

Standard Non-Maximum Suppression scales at O(N²), causing latency spikes when tens of thousands of detections occur across satellite scenes. SAMDEF partitions detections into spatial hash-grid buckets, restricting IoU evaluation strictly to adjacent spatial cells. Boundary duplicates are suppressed in sub-millisecond time.

2. Native GIS Desktop Viewer (apps_deploy/gui_gpui)

•Planar Configuration 2 & Chunky BGRA GeoTIFF Strip Decoder (src/io/tiff_loader.rs):

Satellite imagery frequently uses Planar Configuration 2 (separate color planes: RRR... GGG... BBB...) alongside interleaved RGB/RGBA/Grayscale strips. SAMDEF's custom decoder inspects TIFF header tags, streams separate strips across color planes, packs pixels directly into zero-copy BGRA8 memory wrapped in gpui::RenderImage, and simultaneously generates a downsampled BGRA thumbnail for the interactive radar minimap.

•5-Layer Ordered GPU Scene Pipeline (src/viewer/render.rs & src/viewer/overlays.rs):

Built on GPUI's Vulkan/Blade immediate-mode compositor, the viewport separates primitives across five sequential window.paint_layer passes to guarantee strict z-ordering while preserving GPU instanced batching: - Layer 1 (PolychromeSprite): Renders the full-resolution GeoTIFF raster texture uploaded once to VRAM (plus optional luminance B/W inspection mode). - Layer 2 (Quad Batch): Instanced hollow neon vector bounding boxes (4 edge quads per visible target) with corner brackets on hovered/pinned targets and 100% transparent interiors. - Layer 3 & 4 (Quad + MonochromeSprite): Crisp GPU-shaped class and confidence glyph badges (window.text_system().shape_line). - Layer 5 (PathBuilder + Quad + MonochromeSprite): Interactive geodesic ruler line, crosshair endpoints, and metric telemetry pill. - Changing class filters, Solo isolation, or confidence thresholds triggers ZERO memory allocations and zero GPU texture re-uploads (< 0.1 ms).

•Sub-Pixel Cursor-Invariant Zoom & WGS-84 Geodesy (src/viewer/state.rs & src/geo.rs):

Implements smooth exp() scroll-wheel magnification (5% to 4000% plus 1:1 native pixel mode) anchored at the exact sub-pixel cursor coordinate. Projects cursor pixels via the GDAL 6-parameter affine matrix into live WGS-84 Decimal Degrees, Degrees-Minutes-Seconds (DMS), estimated Ground Sample Distance (m/px), target ground footprints (m × m, m²), and Haversine great-circle distance & forward azimuth (0°–360°).

•Inspection HUD, Detection Table & Multi-Format Exports (src/ui/ & src/io/exporter.rs):

Features client-side Wayland/X11 window controls, instant dataset quick-search, an interactive radar minimap with neon target blips and draggable cyan viewport frustum, an occluded Target Inspector HUD card, a collapsible Target Detection Table with one-click target focus, and offline exporters for RFC 7946 GeoJSON (FeatureCollection), CSV spreadsheets, and Markdown summary reports.

3. Distributed Messaging & Database Processor (apps_deploy/db_processor)

•Zenoh Peer-to-Peer Bus (modules/writer/zenoh_subscriber.rs):

Subscribes to the topic pattern satellite/detections/**. Detection payloads published by the detector are deserialized from binary bincode into structured DetectionPayload objects.

•Persistent Geospatial Storage (modules/writer/queries.rs):

Connects to PostgreSQL using SQLx connection pooling. Converts pixel coordinates to WGS84 longitude/latitude via affine transforms and executes batched bulk inserts into detections (with PostGIS Point and Polygon geometry columns) and detections_pixels (with raw pixel coordinates), backed by spatial GIST indexing for rapid geospatial queries.

4. Training & Data Preparation Pipeline (apps_training/)

•Ingestor (apps_training/ingestor): High-throughput Rust microservice for automated dataset balancing, label format conversion, and GeoTIFF tile preparation.
•YOLO Model Trainer (apps_training/yolo_model_trainer): Python/PyTorch pipeline training YOLOv26s models on multi-spectral satellite imagery, featuring automated ONNX export with FP16 precision optimization.

TESTING, VERIFICATION & BENCHMARKS

Operational Inference Timing Benchmark (YOLO26s, FP16):

•Dataset: 281 high-resolution satellite scenes (~3,000×3,000 pixels each, 0.3m GSD)
•Total Geographic Area: 227.61 km²
•Hardware Testbed: Intel Core i9-13900HX (24 cores, 32 threads), 48 GB DDR5 RAM, NVIDIA GeForce RTX 4060 (8 GB VRAM, CUDA 13)
ProviderModelBatch SizeTotal Time (sec)Time per km² (sec)Hardware Profile
GPU (CUDA)YOLO26s FP16244.590.20RTX 4060, full FP16 acceleration, 85–95% core load
CPU (Host)YOLO26s FP164999.004.39i9-13900HX, FP16 emulated, 22.4x slower

Multi-Class Validation Metrics (127,500 Instances across 1,289 Satellite Images):

ClassImagesInstancesPrecision (P)Recall (R)mAP50mAP50-95
all12891275000.5500.4270.4420.241
Light-Vehicle697501910.7130.7830.7700.349
Boxy-Truck54248510.5120.4020.4250.227
Long-Trucks50248200.4730.3500.3630.180
Small-Boat847480.6290.6830.6500.331
Large-Ship803150.4790.3020.3600.193
Fixed-Wing742000.8720.7140.8030.462
Helicopter450.2770.2000.1970.122
Building783606800.5830.6550.6350.328
Storage-Tank784070.7190.3690.4680.271
Railway245860.7600.6910.7120.428
Engineering-Machinery26414330.5370.4260.4570.260
Tower-Pylon941600.8020.3880.4520.295

GIS Viewer Rendering & Memory Performance Comparison:

Performance MetricPrevious Raster Overlay ApproachCurrent GPUI Dynamic Vector Pipeline
Filter / Solo Toggle Latency150 ms – 450 ms (CPU raster + GPU upload)< 0.1 ms (instantaneous)
Heap Allocation on Filter Click~41 MB per toggle0 bytes (zero-allocation)
GPU Texture Re-uploadsRequired on every toggle/slider move0 re-uploads
Scene Composition8 stacked raster textures1 static BGRA RenderImage + 5 ordered Vulkan paint layers
Visual AcuityRaster blur & pixelation at high zoomCrisp vector quads & GPU-shaped glyph badges up to 4000% zoom
PROJECT // SAMDEF[ Back to Projects ↑ ]