Skip to content
View vaibhav3000's full-sized avatar

Highlights

  • Pro

Block or report vaibhav3000

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
vaibhav3000/README.md

Vaibhav Mahore

AI/ML Engineer · B.Tech Mathematics and Computing, IISc Bangalore (graduating 2027)

I build AI systems that have to prove they work: models checked against their own math,
LLM evaluation that fails the build on a regression, and agents that finish only when the tests pass.

Open to 2027 full-time roles and internships: AI/ML Engineer · ML Engineer · Applied AI · AI Automation


Portfolio Resume (PDF) LinkedIn Email LeetCode


Highlights

Result Where
10.7x lower latency and 42.6x less GPU memory than a Transformer Ericsson internship, Mamba wireless model on AWS A10G
Groundedness collapse (0.57 → 0.00) blocked in CI on every push AIRE, LLM/RAG evaluation engine
8/8 adversarial tool calls rejected, 0 failed tool calls repo-engineer, verified coding agent
98.3% parity accuracy where baselines stay at chance (≤56.3%) s4-to-mamba, Mamba-3 reproduced in PyTorch
Top 100 national finalist Amazon ML Challenge 2026

Featured projects

LLM evaluation RAG CI quality gate · Python, pytest, GitHub Actions, Gemini API

Trace-based evaluation for LLM and RAG systems: run a system once, store validated traces, then score them, compare versions and attribute every regression to the cases that caused it. The aire-gate CLI turns a comparison into a CI check.

The repo's own CI plants a "fluent but uncited" version and the gate blocks it on every push: groundedness 0.57 → 0.00 and ungrounded failures +22, even though retrieval recall rose 0.73 → 0.91. A live Gemini 2.5 Flash run raised abstention 0 → 1.0 at 15 s per case, and the report prices that trade-off instead of hiding it.

Code · Live demo · Report (PDF) · Case study

agents tool calling sandboxing · Python, pytest, Gemini API

A coding agent where the LLM only proposes JSON tool calls. A deterministic state machine (plan → act → observe → verify) validates every call against a schema, path-jails it to the workspace and applies AUTO/ASK/DENY permissions. A task is done only when the tests pass.

4/4 bug-fix, test-writing and refactoring tasks with 0 failed tool calls, 2/2 with a live Gemini planner on the same runtime, and 8/8 path-escape, command-injection and malformed calls rejected.

Code · Live demo · Report (PDF) · Case study

deep learning state space models GPU benchmarking · PyTorch

Minimal S4D, Mamba, Mamba-2 (chunked SSD) and Mamba-3 in pure PyTorch, with 22 tests checking that the parallel and recurrent forms agree, and training benchmarks from 1K to 32K tokens on a 6 GB laptop GPU.

Reproduced the Mamba-3 state-tracking result: complex transitions solve parity at 98.3% while real-transition SSMs and a Transformer stay at chance (≤56.3%). On selective copying, LTI S4D collapses to 6% while selective SSMs reach 67%.

Code · Live demo · Report (PDF) · Case study


Experience

AI/ML Research Intern · Ericsson India · Bengaluru · May 2026 - Jul 2026
Benchmarked WiMamba (a Mamba wireless model) against a Transformer on AWS A10G GPUs: 10.7x lower latency (30 ms vs 327 ms) and 42.6x less GPU memory. Evaluated transfer to an unseen geography (+16.6% composite score) and parameter-efficient fine-tuning that kept 98.5% of full fine-tuning performance with 75% of parameters frozen.

Machine Learning Engineer · Alignerr and micro1 · Remote · May 2025 - Aug 2025
Wrote reasoning traces and tool-use data for LLM post-training (Alignerr), and built model evaluation, validation and data-preprocessing workflows in Python with TensorFlow and scikit-learn (micro1).


Achievements and certifications

  • Amazon ML Challenge 2026: Top 100 national finalist; led Team AutoML.
  • Oracle Cloud Infrastructure Certified Professional: Generative AI and Data Science

Tech stack

Languages
Python C++ SQL Bash

ML and deep learning
PyTorch TensorFlow scikit-learn Hugging Face NumPy Pandas

LLM systems
RAG LLM evaluation and regression testing tracing tool calling agent runtimes Gemini / OpenAI APIs

Engineering and MLOps
GitHub Actions Docker pytest Playwright AWS Linux Git


Beyond the code

Gojo Satoru lifting his blindfold, dark cinematic scene from Jujutsu Kaisen Super Saiyan aura meme: football coach powering up with Goku-style energy

"...throughout heaven and earth, I alone verify the output."

Sci-Tech Coordinator for Rhapsody Fest at IISc: ran three inter-college gaming tournaments, a coding contest with 200+ teams and a 500+ member Discord community. Off the clock it's Jujutsu Kaisen and Dragon Ball Z. The Gojo in me trusts the process; the engineer in me checks the logs anyway.

Hiring for an AI/ML, applied AI or automation role? Email me or see the portfolio.

Pinned Loading

  1. aire aire Public

    Replayable evaluation and regression testing engine for LLM and RAG systems - trace-based metrics, failure taxonomy, deterministic and live-LLM modes

    Python

  2. repo-engineer repo-engineer Public

    Verified tool-using coding agent with sandboxed execution, explicit permission policies, replanning, and test-based verification

    Python

  3. s4-to-mamba s4-to-mamba Public

    First-principles study of efficient sequence models from S4D through Mamba-3, with equivalence tests and 1K-32K long-sequence benchmarks

    Python

  4. multi-agent-system multi-agent-system Public

    Python

  5. nimbus nimbus Public

    Python

  6. portfolio portfolio Public

    Personal portfolio: AI/ML engineer. Next.js static export deployed to GitHub Pages

    TypeScript