user_image

Avni Kothari

Senior Applied Scientist - AI Safety

I am an Applied Scientist at Salesforce on the Responsible AI team with 5 years of software and ML engineering experience.

My AI research, presented at ICLR, AAAI, ICML, & NeurIPS, focuses on: 1) improving transparency and alignment in high-stakes AI models, 2) ensuring fairness by detecting scenarios of preclusion & 3) ensuring that AI benefits all individuals by creating LLM tools for those in resource-constrained settings. Below you can find my research with corresponding open-source software packages.

I graduated from my Master's in Computer Science at UC San Diego where I was named a DeepMind Fellow. Outside of work you can usually find me biking or enjoying the outdoors.

My CV can be found here.

Don't hesitate to reach out if you want to learn more about me or my work!

News

  • August 2025

    Deeply humbled to hear many of LLM Evals expert Shreya Shankar's favorite topics are covered in our most recent paper!

  • March 2025

    Check out new my blogpost about Constitutional Classifiers!

Industry Experience

  • Senior Applied Scientist, Responsible AI; Salesforce - San Francisco, CA

    January 2026 – Present

  • AI/ML Engineer; University of California, San Francisco - San Francisco, CA

    September 2023 – December 2025

    Architected and deployed clinical AI systems to save social workers 300+ hours/ week, identify patients at risk for readmission, and create simple, interpretable models from multimodal patient data

    Drove end-to-end architecture of a GPT 4o powered clinical summarization pilot to condense patient charts (20K+ tokens) into actionable summaries, co-developed with 5+ clinicians to improve patient care

    Established and led human and automated annotation pipelines with active learning to ensure 90%+ accuracy on 1000+ GPT 4o clinical summaries

    Designed and deployed a scalable ETL pipeline to process health record data, enabling ML training and evaluation for 3000+ patient records and 30000+ patient visits

    Engineered, deployed, and evaluated a custom readmission risk prediction model, adopted by 10+ clinics, with 12% higher accuracy than the general model

    Created and presented a hybrid LLM-empirical framework to reconcile clinical LLM hallucinations, achieving 40% interpretability improvement (validated by clinical experts) while matching top predictive performance and enabling production deployment

    Developed and authored a publication for a scalable agentic pipeline using GPT-4 with statistical learning to generate interpretable clinical features from multimodal data, outperforming black-box models while maintaining interpretability for decision support

  • ML Engineer; University of California, SD; CA

    June 2022 – Mar 2024

    Developed a model-agnostic bias auditing framework surfacing bias in up to 30% of lending datasets across 5+ models and 3+ datasets with 25+ real-world constraints

    Collaborated with a lending expert to define real-world constraints on lending datasets to identify individuals subject to preclusion

    Authored a publication that received an ICLR Spotlight (Top 5% among submissions)

  • Software Engineer; Edovo - Chicago, IL

    Jan 2020 – May 2021

    Led the re-architecture and deployment of a scalable educational platform, handling the full lifecycle from leading product requirement sessions to engineering a data backend

    Architected, tested, and deployed a content platform using Elasticsearch to handle 700K+ requests per day to thousands of users

    Led 10+ requirement gathering sessions with Product owners to re-build a platform

    Created a data pipeline and job to merge 4B rows of user event data in PostgreSQL

  • Lead Software Engineer; 8th Light - Chicago, IL

    Aug 2017 – March 2019

    Built high-performance technical systems from inception to production, including load testing platforms, API integrations, and patient-facing healthcare applications

    Implemented and deployed a scalable load testing platform simulating 1000+ RPS

    Deployed API integrations to sync 1000+ interactions/ minute in different timezones

    Developed a diabetes management iOS app to connect 60+ patients with diabetic nurses

    Mentored peers and residents through pair programming sessions and code reviews

Papers

Skills

    Languages

  • Python
  • TypeScript
  • Java
  • Elixir
  • Clojure

    • AI/ML

  • GenAI (GPT 4o, Claude Sonnet 4, Cohere Command A)
  • Langchain
  • HuggingFace
  • PyTorch
  • LLM Evaluations
  • Benchmarking LLMs
  • Scikit-Learn

    • MLOps

  • ML Pipelines and ETL
  • CI/CD
  • AWS (EC2, S3, Terraform)
  • Elasticsearch
  • Docker

Research Software

  • bc-llm

    June 2024

    Responsible AI software that enhances model transparency and safety through a multimodal LLM–Bayesian framework, allowing for the creation of simple and interpretable models.

  • reachml

    June 2022

    Created a model-agnostic fairness and safety audit to identify scenarios of preclusion

Education

Poster Presentations

  • NeurIPS Workshop Statistical Foundations of LLMs and Foundation Models

    December 2024

  • ICML Workshop on Data-centric Machine Learning Research

    July 2023

  • ICML Workshop on Spurious Correlations, Invariance and Stability

    July 2023

  • ICML Workshop on Artificial Intelligence & Human Computer Interaction

    July 2023

Teaching Experience

  • Fall 2022

    TA for DSC 291 - Interpretability and Explainability in Machine Learning

  • May 2011 – August 2013

    Differential Calculus Tutor

Service

  • Vision 1948 (May 2023 – Present)
  • PenPal for the Incarcerated (September 2020 – Present)
  • Warren Community Garden (August 2021 – Present)
  • UCSF AI4All (July 2024 – July 2024)
  • The Recyclery (August 2018 – May 2021)