SAMDEF
Edge Computer Vision System & Satellite Imagery Detection Pipeline
A high-performance computer vision system built in Rust and CUDA for large-scale satellite imagery (GeoTIFF) processing. Features an asynchronous multi-stage GPU pipeline, spatial grid NMS for deduplication, and a native GPUI (Vulkan) GIS desktop viewer delivering 0.20s/km² inference throughput.








THE PROBLEM
Processing high-resolution satellite imagery (typically 3,000×3,000 to 15,000×15,000 multi-spectral GeoTIFFs) presents several key technical challenges:
1. Host Memory Spikes on Large Rasters:
Attempting to read and decompress large multi-spectral GeoTIFFs directly into contiguous host memory can quickly consume tens of gigabytes of RAM, causing severe swap thrashing and out-of-memory errors.
2. GPU Idle Time in Sequential Pipelines:
In a sequential loop (read -> tile on CPU -> upload -> run inference -> parse), the GPU sits idle while waiting for disk I/O and CPU tensor conversions, resulting in low hardware utilization.
3. Duplicate & Cut-Off Detections Along Tile Seams:
Slicing large satellite rasters into smaller tiles bisects objects (such as vessels, aircraft, and large buildings) across tile seams. Without boundary overlap buffers and spatial deduplication, models produce duplicate and cut-off bounding boxes.
4. Preserving Real-World Earth Coordinates:
Standard detection models output local pixel coordinates (e.g., x=450, y=320). Operators and GIS analysts require exact geospatial coordinates (WGS84 Latitude/Longitude, DMS, and ground footprint metrics) for mapping and operational use.
5. Desktop Viewer GPU Memory Overhead:
Rendering ultra-high-resolution rasters (e.g. 3,433×2,642 GeoTIFFs) with thousands of interactive detections requires careful graphics pipeline design. Generating raster overlays per filter toggle consumed ~41 MB of heap per click and caused UI stutter.
6. Bandwidth Limits for Cloud Processing:
Uploading gigabyte-scale satellite rasters over slow network uplinks introduces severe latency and high bandwidth costs. The workflow requires reliable local, air-gapped on-premise execution.
THE SOLUTION
SAMDEF (Satellite Multi-Class Detection & Geospatial Framework) is a local, air-gapped, high-throughput system built in Rust and CUDA. It handles raster ingestion, virtual tiling, GPU tensor inference, spatial deduplication, database persistence, and native GPU-accelerated GIS visualization locally on edge hardware.
Core Architecture Highlights:
gui_gpui): A GPU-accelerated desktop application built with Rust and GPUI (Vulkan/Blade) delivering 5-layer immediate-mode rendering, sub-pixel cursor-invariant pan/zoom, class soloing, live radar minimap, geodesic Haversine ruler, target inspector HUD, tabular detection drawer, and GeoJSON/CSV/Markdown exports with zero GPU texture re-uploads.HIGH-LEVEL SYSTEM ARCHITECTURE
+-------------------------------------------------------------------------------------------------+
| SAMDEF SYSTEM ARCHITECTURE |
+-------------------------------------------------------------------------------------------------+
[ High-Resolution GeoTIFFs (Planar Config 2) ]
│
▼
┌─────────────────────────────────────────────────────────────┐
│ DETECTOR MICROSERVICE │
│ ┌───────────────────────────┐ ┌─────────────────────┐ │
│ │ Work-Stealing Thread Pool │────▶│ Crossbeam Channel │ │
│ │ (GDAL Virtual Tiler) │ │ (Bounded Task Queue)│ │
│ └───────────────────────────┘ └──────────┬──────────┘ │
│ │ │
│ ┌────────────────────────────────────────────▼──────────┐ │
│ │ Decoupled 3-Stage Pipeline: │ │
│ │ Stage 1: Rayon CPU Preprocessor (Array4<f16>) │ │
│ │ Stage 2: Dedicated GPU Runner (CUDA Session Loop) │ │
│ │ Stage 3: Spatial Grid NMS & Geodetic Coordinate Map │ │
│ └────────────────────────────────────────────┬──────────┘ │
└───────────────────────────────────────────────┼─────────────┘
│
┌──────────────────────────┴──────────────────────────┐
│ │
▼ (Zenoh P2P Binary Bus: bincode) ▼ (Structured JSON / GeoJSON)
┌──────────────────────────────┐ ┌──────────────────────────────┐
│ DB PROCESSOR │ │ NATIVE GIS VIEWER (GPUI) │
│ - Async Tokio Zenoh Listener│ │ - Built with Rust & GPUI │
│ - Batched SQL Inserts │ │ - 5-Pass Vulkan Paint Layers│
│ - Spatial GIST Indexing │ │ - Zero Texture Re-uploads │
│ - Connection Pool Management│ │ - Sub-Pixel Zoom & Geodesy │
└──────────────┬───────────────┘ │ - Radar, HUD & GIS Exports │
│ └──────────────────────────────┘
▼
┌──────────────────────────────┐
│ PostgreSQL Database │
│ (Local Geospatial Storage) │
└──────────────────────────────┘
SYSTEM ARCHITECTURE & DEEP DIVE
1. The Detector Microservice (apps_deploy/detector)
modules/io/producer.rs):Scans the input directory for GeoTIFF files and spawns PRODUCER_PARALLELISM persistent worker threads. Each worker pulls image paths from a concurrent work queue, opens rasters using GDAL C-bindings, and slices them into fixed 896×896 RGB tiles with 20% stride overlap (179-pixel buffer) along horizontal and vertical axes.
modules/io/consumer.rs):The consumer architecture is decoupled into 3 dedicated OS threads connected by bounded Crossbeam channels (gpu_tx: bounded(4), post_tx: bounded(64)): - Stage 1 (OS Thread A - CPU Preprocessor & Batch Accumulator): Receives raw tile bytes from the input channel, bundles them into batches (optimal batch size = 2), and uses Rayon data parallelism (par_iter) across CPU cores to normalize pixels and transpose HWC to CHW planar format: normalized_val = (raw_byte / 255.0 - mean) / std. Ready Array4<f16> tensors are forwarded to the dedicated GPU channel in sub-milliseconds. - Stage 2 (OS Thread B - Dedicated GPU Inference Runner): A persistent OS thread holding the CUDA Session in ONNX Runtime (ort 2.0.0-rc.13) with native CUDA 13 support. Operates as a pure GPU execution loop without blocking file I/O. Dequeues ready tensors and immediately executes session.run() on the RTX 4060 GPU, forwarding raw output tensors to Stage 3. - Stage 3 (OS Thread C - Postprocessing, NMS & Tokio Async Output): Runs on the main Tokio runtime thread. Parses bounding box coordinates, shifts them by spatial tile offsets, tracks per-file completion in a HashMap, runs Spatial Grid NMS, and spawns background Tokio JoinSet tasks to write JSON and publish via Zenoh.
- Thread A prepares Batch N+1 on CPU cores. - Thread B executes Batch N on the RTX 4060 GPU simultaneously. - Thread C saves Batch N-1 to disk and network simultaneously. - Bounded channels provide automatic backpressure: Thread 1 buffers up to 4 preprocessed batches in memory. When Thread 2 finishes a batch, the next tensor is pulled in 0 microseconds without waiting for CPU math. If CPU preprocessing outpaces GPU execution, gpu_tx.send() pauses Thread 1, preventing unbounded memory growth in RAM.
Every detected pixel coordinate is mathematically projected onto the Earth's surface using the GeoTIFF affine transformation matrix: X_geo = GeoTransform[0] + X_pixel * GeoTransform[1] + Y_pixel * GeoTransform[2] Y_geo = GeoTransform[3] + X_pixel * GeoTransform[4] + Y_pixel * GeoTransform[5]
Standard Non-Maximum Suppression scales at O(N²), causing latency spikes when tens of thousands of detections occur across satellite scenes. SAMDEF partitions detections into spatial hash-grid buckets, restricting IoU evaluation strictly to adjacent spatial cells. Boundary duplicates are suppressed in sub-millisecond time.
2. Native GIS Desktop Viewer (apps_deploy/gui_gpui)
src/io/tiff_loader.rs):Satellite imagery frequently uses Planar Configuration 2 (separate color planes: RRR... GGG... BBB...) alongside interleaved RGB/RGBA/Grayscale strips. SAMDEF's custom decoder inspects TIFF header tags, streams separate strips across color planes, packs pixels directly into zero-copy BGRA8 memory wrapped in gpui::RenderImage, and simultaneously generates a downsampled BGRA thumbnail for the interactive radar minimap.
src/viewer/render.rs & src/viewer/overlays.rs):Built on GPUI's Vulkan/Blade immediate-mode compositor, the viewport separates primitives across five sequential window.paint_layer passes to guarantee strict z-ordering while preserving GPU instanced batching: - Layer 1 (PolychromeSprite): Renders the full-resolution GeoTIFF raster texture uploaded once to VRAM (plus optional luminance B/W inspection mode). - Layer 2 (Quad Batch): Instanced hollow neon vector bounding boxes (4 edge quads per visible target) with corner brackets on hovered/pinned targets and 100% transparent interiors. - Layer 3 & 4 (Quad + MonochromeSprite): Crisp GPU-shaped class and confidence glyph badges (window.text_system().shape_line). - Layer 5 (PathBuilder + Quad + MonochromeSprite): Interactive geodesic ruler line, crosshair endpoints, and metric telemetry pill. - Changing class filters, Solo isolation, or confidence thresholds triggers ZERO memory allocations and zero GPU texture re-uploads (< 0.1 ms).
src/viewer/state.rs & src/geo.rs):Implements smooth exp() scroll-wheel magnification (5% to 4000% plus 1:1 native pixel mode) anchored at the exact sub-pixel cursor coordinate. Projects cursor pixels via the GDAL 6-parameter affine matrix into live WGS-84 Decimal Degrees, Degrees-Minutes-Seconds (DMS), estimated Ground Sample Distance (m/px), target ground footprints (m × m, m²), and Haversine great-circle distance & forward azimuth (0°–360°).
src/ui/ & src/io/exporter.rs):Features client-side Wayland/X11 window controls, instant dataset quick-search, an interactive radar minimap with neon target blips and draggable cyan viewport frustum, an occluded Target Inspector HUD card, a collapsible Target Detection Table with one-click target focus, and offline exporters for RFC 7946 GeoJSON (FeatureCollection), CSV spreadsheets, and Markdown summary reports.
3. Distributed Messaging & Database Processor (apps_deploy/db_processor)
modules/writer/zenoh_subscriber.rs):Subscribes to the topic pattern satellite/detections/**. Detection payloads published by the detector are deserialized from binary bincode into structured DetectionPayload objects.
modules/writer/queries.rs):Connects to PostgreSQL using SQLx connection pooling. Converts pixel coordinates to WGS84 longitude/latitude via affine transforms and executes batched bulk inserts into detections (with PostGIS Point and Polygon geometry columns) and detections_pixels (with raw pixel coordinates), backed by spatial GIST indexing for rapid geospatial queries.
4. Training & Data Preparation Pipeline (apps_training/)
apps_training/ingestor): High-throughput Rust microservice for automated dataset balancing, label format conversion, and GeoTIFF tile preparation.apps_training/yolo_model_trainer): Python/PyTorch pipeline training YOLOv26s models on multi-spectral satellite imagery, featuring automated ONNX export with FP16 precision optimization.TESTING, VERIFICATION & BENCHMARKS
Operational Inference Timing Benchmark (YOLO26s, FP16):
| Provider | Model | Batch Size | Total Time (sec) | Time per km² (sec) | Hardware Profile |
|---|---|---|---|---|---|
| GPU (CUDA) | YOLO26s FP16 | 2 | 44.59 | 0.20 | RTX 4060, full FP16 acceleration, 85–95% core load |
| CPU (Host) | YOLO26s FP16 | 4 | 999.00 | 4.39 | i9-13900HX, FP16 emulated, 22.4x slower |
Multi-Class Validation Metrics (127,500 Instances across 1,289 Satellite Images):
| Class | Images | Instances | Precision (P) | Recall (R) | mAP50 | mAP50-95 |
|---|---|---|---|---|---|---|
| all | 1289 | 127500 | 0.550 | 0.427 | 0.442 | 0.241 |
| Light-Vehicle | 697 | 50191 | 0.713 | 0.783 | 0.770 | 0.349 |
| Boxy-Truck | 542 | 4851 | 0.512 | 0.402 | 0.425 | 0.227 |
| Long-Trucks | 502 | 4820 | 0.473 | 0.350 | 0.363 | 0.180 |
| Small-Boat | 84 | 748 | 0.629 | 0.683 | 0.650 | 0.331 |
| Large-Ship | 80 | 315 | 0.479 | 0.302 | 0.360 | 0.193 |
| Fixed-Wing | 74 | 200 | 0.872 | 0.714 | 0.803 | 0.462 |
| Helicopter | 4 | 5 | 0.277 | 0.200 | 0.197 | 0.122 |
| Building | 783 | 60680 | 0.583 | 0.655 | 0.635 | 0.328 |
| Storage-Tank | 78 | 407 | 0.719 | 0.369 | 0.468 | 0.271 |
| Railway | 24 | 586 | 0.760 | 0.691 | 0.712 | 0.428 |
| Engineering-Machinery | 264 | 1433 | 0.537 | 0.426 | 0.457 | 0.260 |
| Tower-Pylon | 94 | 160 | 0.802 | 0.388 | 0.452 | 0.295 |
GIS Viewer Rendering & Memory Performance Comparison:
| Performance Metric | Previous Raster Overlay Approach | Current GPUI Dynamic Vector Pipeline |
|---|---|---|
| Filter / Solo Toggle Latency | 150 ms – 450 ms (CPU raster + GPU upload) | < 0.1 ms (instantaneous) |
| Heap Allocation on Filter Click | ~41 MB per toggle | 0 bytes (zero-allocation) |
| GPU Texture Re-uploads | Required on every toggle/slider move | 0 re-uploads |
| Scene Composition | 8 stacked raster textures | 1 static BGRA RenderImage + 5 ordered Vulkan paint layers |
| Visual Acuity | Raster blur & pixelation at high zoom | Crisp vector quads & GPU-shaped glyph badges up to 4000% zoom |