SOCIAL NAVIGATION SYSTEM β DATA FLOW
Click any node for details Β· Abolghasem Esmaeily Β· Idiap / KTH
PERCEPTION
TRACKING & REASONING
OUTPUT GENERATION
NAVIGATION
RGB-D Camera
Hardware Sensor
π
YOLO
Node 1 β Detection & ID
π
Kalman Filter Tracker
Identity Persistence Engine
π§
Gemini Flash 2.0
Social Scene Analysis
πΊοΈ
Costmap Gen
analyse_loop @ 10Hz
π€
Nav2 Stack
Path Planning
/camera/color/image_raw
/detected_humans
/annotated_image
self.trackers[]
.social_edges{}
/social_obstacles
PoseArray
sensor_msgs/Image
PointCloud2
KF DATA ASSOCIATION β WHO IS WHO
1. Predict all trackers β 2. For each detection, find nearest tracker (<5m gate)
3. Update matched tracker with Kalman gain β 4. Unmatched detection β new tracker
5. Trackers with no update for 3.0s β removed (TTL expiry)
Sensor Input
Detection
Tracking
VLM Reasoning
Output
Navigation