Skip to content
View Nishant-ZFYII's full-sized avatar

Block or report Nishant-ZFYII

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
Nishant-ZFYII/README.md

Nishant Pushparaju

Robotics engineer building the full sensor-to-deployment stack — embedded firmware through edge ML inference, with research on perception under sensor failure.

arXiv Hugging Face OpenVINO PRs LinkedIn Portfolio

MS Mechatronics, Robotics, and Automation Engineering at NYU Tandon (May 2026). First author on an arXiv preprint under peer review at IEEE CASE 2026 and IEEE Robotics and Automation Letters. Production C++17 perception pipeline shipped end-to-end with Docker, docs site, and 162 + 31 tests. Six compute platforms in production code, from 8-core Parallax Propeller cogs written from datasheets to NVIDIA GPU clusters I co-architected. Based in New York City. F-1 / STEM-OPT eligible.


Featured work

terra-perceive — accumulated BEV map and full perception dashboard

terra-perceive
Production C++17 perception pipeline for off-road / construction-site autonomy. From-scratch sector RANSAC, LiDAR-inertial SLAM with Lie groups + on-manifold IMU pre-integration, factor-graph LM, Scan Context, SORT + IMM cascade, probabilistic traversability, 1D CBF safety supervisor. 0.577 m ATE on RELLIS-3D Seq 00 (within 13% of Cartographer). 162 C++ + 31 Python tests, Docker repro in 45 s. M1–M13 shipped. Blog · solo.
V9 distilled depth student running on Jetson Orin Nano, six-panel comparison

ml_inference · arXiv 2603.28890
Bootstrap depth fusion under hardware sensor failure: the surviving valid ToF pixels calibrate a learned monocular depth student to metric scale at runtime. Distilled student at 218 FPS / 4.6 ms / 2.7 GB on Jetson Orin Nano. Costmap obstacle coverage +55% (corridor), +110% (LILocBench) over LiDAR alone. 9/10 closed-loop navigation success matching ground-truth-depth baseline. Under peer review at IEEE CASE 2026 / RA-L.
NCHSB threading a tight-space gap between a mid-corridor obstacle and the wall

NCHSB
ROS 2 Humble high-speed Ackermann racing platform — modified Traxxas Maxx 4S, custom URDF resolving the four-bar-linkage closed-loop problem, custom C++ Nav2 GlobalPlanner plugin, CVXPY QP racing-line optimizer, Nav2 MPPI tuned for high-speed Ackermann, SLAM Toolbox + EKF retuned away from differential-drive defaults. Emergency-MPC fallback with Human > Object > Wall ethical cost hierarchy. 3-person team (first author).
Unitree Go2 walking policy trained with PPO at 4096 envs

Unitree Go2 RL — Isaac Lab PPO
Walking policy trained in NVIDIA Isaac Lab. Linear velocity tracking 88.0 / angular 47.2 against course targets of 48 / 24 — nearly 2×. Raibert heuristic gait clock, bonus actuator-friction randomization for sim-to-real robustness, RayCaster height-scanner terrain extension. Trained at 4096 parallel environments, 500 epochs in ~12 min on the same NYU AWS ParallelCluster I co-architected. Physical Go2 walked one minute without falling on first deployment. 3-person team.
ChoiRbot 3-robot heterogeneous swarm coordinating across Parallax Propeller, Arduino, ESP32, and Raspberry Pi 5

ChoiRbot swarm
3-robot heterogeneous swarm: two physical bots on radically different MCUs — Arduino Nano 33 BLE Sense + 8-core Parallax Propeller P8X32A bridged through an ESP32 BLE-UART — coordinated through a Raspberry Pi 5 central station running ROS 2 Jazzy. Solo-built firmware on all four platforms; the Propeller multi-cog C was written from datasheets directly. 0.31% EKF position error. Sim-to-real story: Lyapunov controller failed under real motor friction, retuned to PID. Co-implemented Distributed Simplex task assignment and CBF safety filter.
AI/ML Quants VIP visualization pending

Quantum Geometric Intelligence Lab
Two semesters in NYU Finance and Risk Engineering on commodity forecasting. Sem 1: co-authored survey, solo-ablated CNN / CNN+attention / RCNN+GRU / RCNN+attention on multivariate wheat futures. Sem 2: pivoted from regression to classification reformulation (buy/sell/hold signals) and fused FRED-MD macro + FinBERT NLP sentiment + Google Earth Engine NDVI satellite into a unified input space feeding RCNN-with-skip, RCNN-with-attention, BiGRU.

Headline claims

Result Detail Evidence
arXiv 2603.28890 under peer review Bootstrap perception under hardware depth failure, IEEE CASE / RA-L arXiv · HF models
218 FPS distilled depth student on Jetson Orin Nano EfficientViT-B1, TensorRT FP16, 4.6 ms / 2.7 GB ml_inference
+55% / +110% costmap obstacle coverage Corridor / LILocBench, over LiDAR-only baseline arXiv §IV-B / §IV-C
162 C++ + 31 Python tests in a from-scratch C++17 perception stack Docker reproducible in 45 s, M1–M13 shipped terra-perceive
0.577 m ATE from-scratch LiDAR-inertial SLAM RELLIS-3D Seq 00, within 13% of Google Cartographer terra-perceive M8
88.0 / 47.2 velocity tracking on Unitree Go2 PPO Vs course targets 48 / 24 — nearly 2× rob6323_go2_project
4096-environment PPO training in ~12 min on AWS HPC NYU ParallelCluster co-architected with NYU IT RTS same cluster used for both build and train
0.31% EKF position error on Parallax Propeller multi-cog C Written from datasheets, no library ChoiRbot Mate
3 merged PRs to Intel OpenVINO C++ / Node.js / Python language bindings PR list

Sensor-to-deployment stack

I write production code on every layer of the autonomy stack — six distinct compute platforms in shipped projects, from datasheet-level firmware to GPU-cluster training.

Layer Platform Where it shipped
HPC NVIDIA GPU clusters · CUDA · SLURM · Isaac Lab Go2 PPO @ 4096 envs · SDXL Turbo serving · terra-perceive ablations on NYU Greene
Edge GPU Jetson Orin Nano 8 GB · TensorRT FP16 Distilled depth student @ 218 FPS · arXiv 2603.28890
Workstation / Linux / Docker x86_64 · ROS 2 Humble · Eigen3 · CMake terra-perceive · NCHSB simulation stack
Single-board Raspberry Pi 5 · Ubuntu 24.04 · ROS 2 Jazzy ChoiRbot central station · ArUco vision · BLE bridge orchestration
Microcontroller (Cortex-M4 + Tensilica) ESP32-WROOM-32 · Arduino Nano 33 BLE Sense BLE-UART bridge · GATT command pipeline · H-bridge motor control
Microcontroller (multi-cog) Parallax Propeller P8X32A · 8 cores @ 80 MHz · multi-cog C ChoiRbot Mate firmware, written from Parallax datasheets directly — no library

Research

Pushparaju, N., Mattam, V., Arab, A. (2026). Bootstrap Perception Under Hardware Depth Failure for Indoor Robot Navigation. arXiv preprint 2603.28890 — under peer review at IEEE CASE 2026 and IEEE Robotics and Automation Letters.

A self-referential failure-aware sensing hierarchy: the surviving valid ToF pixels (up to 78% invalid in the worst-case reflective-corridor segment) calibrate a learned monocular depth estimator (DA3-Small student, distilled from DA3-Large) to metric scale at runtime — the system fills its own gaps without external data. LiDAR remains the geometric anchor; hardware ToF depth is preserved where valid; learned depth enters only where ToF has dropped out; semantics refine inflation only when needed. Counter-intuitive bonus result: learned depth reduces costmap flicker (0.61% → 0.43% normalized occupancy jitter) by stabilizing intermittent ToF dropouts.

arXiv · Hugging Face models (vortex-depth-v5-general, vortex-depth-v6-pretrained, vortex-depth-v9-corridor) · Code under double-blind review embargo until publication.

Pushparaju, N. et al. (2025). Survey of Commodity Price Forecasting. NYU Quantum Geometric Intelligence Lab — 5-author internal lab deliverable. My sections: data sourcing (futures + USDA WASDE + horticulture panels), feature engineering (continuous-futures construction, stationarity transforms, lag selection via differential evolution and ACF/PACF), CNN architectures for time series, and attention-based RNN/CNN variants with gradient-flow analysis. PDF in repo.


Open source

3 merged pull requests to OpenVINO — Intel's production AI inference toolkit — across C++, Node.js, and Python language bindings. Notable: import_model(tensor) JavaScript API addition; segment_mean_csr converter implementation. ~10 PRs in flight across language bindings.

Other public activity: GPU-accelerated NYC taxi data analysis (1.1B records via NVIDIA RAPIDS / cuGraph); modified DBA-Fusion to support stereo cameras; fork of openxrlab/xrdslam for deep-learning SLAM exploration.


Tech stack

Robotics & SLAM — ROS 2 (Humble / Jazzy / Foxy), Nav2 (MPPI, Smac Hybrid A*, behavior trees, lifecycle nodes), SLAM Toolbox, robot_localization EKF, ros2_control, ackermann_steering_controller, URDF / Xacro, Gazebo Harmonic / Fortress, NVIDIA Isaac Lab, KISS-ICP, IMU pre-integration on SO(3), Scan Context loop closure, g2o factor graphs, SuperPoint + SuperGlue, CosPlace, ORB-SLAM3, Hybrid A* / Reeds-Shepp planning.

Machine learning & edge ML — PyTorch, TensorFlow, knowledge distillation (EfficientViT-B1 student from DA3-Large), PPO reinforcement learning, time-series ML with leakage-free TimeSeriesSplit CV, multimodal feature fusion (FRED-MD macro + FinBERT NLP + GEE NDVI satellite), TensorRT FP16 / INT8, ONNX, OpenVINO (3 merged PRs), Apptainer / Singularity for HPC inference.

Embedded firmware (six platforms) — Parallax Propeller P8X32A multi-cog C written from datasheets · Arduino Nano 33 BLE Sense (BLE GATT, encoders, PWM, H-bridge) · ESP32-WROOM-32 (BLE-UART bridges) · Raspberry Pi 5 (Ubuntu 24.04 + ROS 2 Jazzy) · Jetson Orin Nano (TensorRT inference) · NVIDIA GPU clusters (CUDA-aware Slurm jobs).

Cloud & HPC — AWS ParallelCluster co-architected with NYU IT Research Technology Services, CloudFormation IaC, Slurm (interactive + batch), FSx for Lustre, Apptainer SIF containers (deliberate choice over Docker for HPC cgroup integration), Cloudflare Tunnels for zero-trust networking, IP-restricted security groups + IAM hardening. Built the cluster, then used it to train the Go2 PPO policy at 4096 envs and serve an SDXL Turbo image-generation pipeline within a $2,000 AWS budget.

Math & optimization — Eigen3, CVXPY, qpSWIFT, OSQP, Distributed Simplex, Control Barrier Functions, EKF, factor-graph optimization (g2o; GTSAM exposure), Lie groups SO(3) / SE(3).

Languages — C++17, Python 3, C, MATLAB, R, JavaScript (Node.js), SQL, Bash, LaTeX, KUKA KRL.

Industrial robotics — KUKA KR-6, KUKA KRL programming, mastering / tool calibration / base calibration, Schmalz high-flow vacuum end-effectors, offline programming in KUKA Sim Pro and RoboDK.

Data engineering (Cognizant / Sanofi, 1 year 8 months) — Informatica PowerCenter, Oracle 11g/12c, MySQL, Cognos BI, Power BI — multi-region (APAC / EMEA / AMER) ETL with reports informing quarterly business decisions for Sanofi senior leadership under GxP-adjacent pharmaceutical data governance.


Currently

May 2026: finishing the MS, shipping nuScenes cross-domain validation (M14) for terra-perceive, IEEE CASE peer-review cycle for the arXiv preprint, wrapping the Sem 2 commodity-classification report at the VIP lab, queuing trt-bench (C++ TensorRT benchmarking utility) as the next public artifact. Looking for robotics engineering · perception engineering · edge-ML deployment · robotics systems integration roles in the United States. Strongest fit in AV deployment teams, defense robotics, warehouse / logistics autonomy, drone autonomy, and edge-ML inference infrastructure. EAD active July 2026; STEM-OPT eligible through ~2029.


Education

New York University, Tandon School of Engineering — M.S. Mechatronics, Robotics, and Automation Engineering (Sep 2024 – May 2026). MS project advised by Prof. Aliasghar Arab (NYU MAE). Coursework: Robot Perception · Reinforcement Learning for Robotics · Networked Robotics Systems / Cooperative Control / Swarming · Advanced Mechatronics · Linear Algebra · Foundations of Robotics · Optimization Techniques · Data-Driven Decision Making · SQP Optimization for Robotics.

SRM Institute of Science and Technology — B.Tech Mechatronics, Robotics, and Automation Engineering (Aug 2017 – Jun 2021). Capstone "Automatic Handling of Cut Textiles" in collaboration with Institut für Textiltechnik (ITA), RWTH Aachen University — pick-and-place automation of limp denim using a KUKA KR-6 with Schmalz high-flow vacuum gripper.


Get in touch

Pinned Loading

  1. rob6323_go2_project rob6323_go2_project Public

    Forked from machines-in-motion/rob6323_go2_project

    Code for the Go2 project of the RL class rob6323.

    Python 1

  2. ml_inference ml_inference Public

    Python 1

  3. NCHSB NCHSB Public

    ROS2 ackerman bot.

    Python

  4. terra-perceive terra-perceive Public

    From-scratch C++17 LiDAR + camera perception — sector RANSAC, LiDAR-inertial SLAM, MOT, probabilistic traversability, CBF safety. 13 engineering deep-dives.

    C++

  5. aojedao/AdvancedMechatronics aojedao/AdvancedMechatronics Public

    Repository for the GY 6103 Advanced Mechatronics course at NYU

    Python 1

  6. AI_ML_Quants_VIP AI_ML_Quants_VIP Public

    VIP lab for Commodities price prediction using AI/ML.

    Jupyter Notebook