Zichen "Charlie" Zhang

About Me 👋

I worked as the first full-time hire at Mira (formerly known as Halo), an AI smart glasses startup in San Francisco. We raised $6.6M from General Catalyst, Naval Ravikant, Pillar VC, Soma Capital, and Village Global. I joined at pre-seed and helped scale us to 1,000+ sales.

On the technical side, I built real-time speech-to-text pipelines with WebRTC voice activity detection, deployed self-hosted speech recognition models using TorchServe, and architected the FastAPI backend handling hundreds of concurrent WebSocket connections. I also built our agentic memory system – basically the infrastructure that lets the glasses listen, remember, and actually be useful.

Before that, I worked at the AI Lab at Supercell – the folks behind Brawl Stars, Clash Royale, and Clash of Clans. I built AI-powered NPC dialogue systems that generate branching conversation trees on the fly using LLMs. The NPCs have their own personalities, moods that shift based on context, and can spin up emergent storylines as players interact with them. I built the FastAPI backend with Google GenAI SDK, deployed it serverless on AWS Lambda, and integrated everything with Unity C# game clients so conversations happen in real-time as you play.

For undergrad, I studied at the University of Michigan, majoring in Computer Science. I was affiliated to the Honors Program at the College of Literature, Science, and the Arts (LSA).

🔗 You can find my social media accounts here at my Linktree.

📢 Feel free to drop me an email at charliezhang325@gmail.com. I’m always open to chats and feedbacks.

🎨 View my Chinese calligraphy and photography works here!

Interests

Multimodal LLM Agents in Software Systems
Powering Games with AI-Generated Dynamic NPC Moods and Storylines

Education

B.S. in Computer Science, 2022-2025
University of Michigan--Ann Arbor

Research Projects

* denotes equal contribution

Learning Pushing Dynamics for Arbitrary 2D Rigid Bodies

Art Boyarov*, Zichen Zhang*.

We study the problem of learning the pushing dynamics of arbitrary 2D rigid bodies, developing neural network models trained on simulated data collected with a Franka Panda robot. By comparing a shallow MLP to a deeper point-cloud-inspired network, we show that the deeper model better captures the complex motion dynamics of different 2D shapes. Using the learned models in a Model Predictive Path Integral (MPPI) controller, we successfully achieve closed-loop pushing and obstacle avoidance across diverse 2D rigid bodies.

Learning Pushing Dynamics for Arbitrary 2D Rigid Bodies

Simpler is Better: Finding the Best Reward Function in Long Chain-of-Thought Reinforcement Learning for Small Language Models

Luning Wang*, Zichen Zhang*, Junkuan Liu*.

We study three types of reward functions — normal, cosine, and dynamic — for long chain-of-thought reinforcement learning in Small Language Models, and find that the simple normal reward consistently outperforms more complex designs, suggesting that simpler rewards are good enough for eliciting reasoning in smaller models.

<em>Simpler is Better:</em> Finding the Best Reward Function in Long Chain-of-Thought Reinforcement Learning for Small Language Models

VTMo: Unified Visuo-Tactile Transformer Encoder with Mixture-of-Modality-Experts

Zichen Zhang, Peihao Li, Yuan Cheng.

We introduce VTMo, a modular Vision-Touch Transformer encoder that unifies dual-encoder flexibility with fusion-encoder accuracy through a shared self-attention mechanism and modality-specific or cross-modal experts. VTMo supports image-only, touch-only, and vision-touch fusion tasks, offering versatility for speed or accuracy. Our method achieves competitive performance on the Image-to-Touch Retrieval task while reducing training time and computational complexity.

VTMo: Unified Visuo-Tactile Transformer Encoder with Mixture-of-Modality-Experts

Babysitting a Small Language Model through One-Step Tree-of-Thoughts Knowledge Distillation

Anurag Renduchintala*, Adi Mahesh*, Zichen Zhang*, Zimo Si*, Shangjun Meng*, Samuel Fang*.

We introduce the One-Step Tree-of-Thoughts framework, a simplified prompting method that distills multi-step reasoning into a single structured prompt, and demonstrates how knowledge distillation can transfer this reasoning capability from Large Language Models to Small Language Models with much less parameters, enabling significant improvements reasoning performance, beating GPT-4o and GPT-4, as shown on the model's performance on Game of 24.

Babysitting a Small Language Model through One-Step Tree-of-Thoughts Knowledge Distillation

MIA-Sort: Multiplex Chromatin Interaction Analysis by Efficiently Sorting Chromatin Complexes

Zichen Zhang, Minji Kim.

MIA-Sort is a Python bioinformatics tool for efficiently extracting and sorting chromatin complexes from large datasets like Hi-C and Pore-C, enabling researchers to analyze chromatin loops, stripes, jets, and hubs to study loop extrusion.

MIA-Sort: Multiplex Chromatin Interaction Analysis by Efficiently Sorting Chromatin Complexes

Engineering Projects

Builds & Products (* denotes equal contribution)

GenHint: An AI Coding Assistant that Teaches How to Code

🏆 Honorable Mention Best Developer Tool 🏆. MHacks 17 Hackathon, 2024

Zichen Zhang*, Peihao Li*, Tongyuan Miao*, Yuchen Huang*.

Unlike conventional AI coding assistants like GitHub Copilot, GenHint does not give you codes directly. Instead, it generates code templates with "TODO" comments and explains each subproblem for you. It's powered by Llama 3 70B hosted on Groq. Published as a VS Code Extension and easy to download, it will be a great companion for your coding workflow.

GenHint: An AI Coding Assistant that Teaches How to Code

Collage: Pinterest for College Advising

🌟 700+ active users since launch 🌟

Max Feldman*, Alex Wang*, Zichen Zhang*, Jaden Sun*, Kali Francisco, Nate Bennett, Leo Choi, Tanishka Nalawade, Joe Tang, Asher Katz, Eden McCullough, Dasha Skalitzky.

Accessible via your Google Education account, Collage is an AI-powered educational discovery engine for the University of Michigan that personalizes academic advising and scheduling and connects UMich students with their classes, advisors, and peers. Collage features a Pinterest-like course catalog tailored to individual academic interests and career goals, an AI advisor to assist with scheduling decisions, and a social networking system for sharing schedules and connecting with peers.

Blogs

Zichen "Charlie" Zhang

Jan 22, 2025 2 min read

My Takes on Building Simple AI Chatbots in Web Applications

Have you ever thought about building a chatbot on the web that can tailor its responses using the data you’ve collected?

Zichen "Charlie" Zhang

Nov 1, 2024 0 min read

UMich EECS 482 Operating System Notes

General programming patterns for writing multi-threaded programs and implementing synchronization primitives with low-level hardware

Zichen "Charlie" Zhang

May 19, 2024 61 min read

A Deep Learning Tutorial Written for Everyone

A detailed introduction to the most common deep learning model architectures and theories, including Transformer, BERT, GAN, Meta Learning, Adversarial Learning, Reinforcement Learning, Network Compression, Anomaly Detection, etc.

Awards

Honorable Mention Best Developer Tool

MHacks 2024 Sep 2024

One of the largest hackathons in the U.S., attracting over 550 students from leading universities in North America.

See certificate

University Honors

University of Michigan May 2024

Received for 5 terms: Fall 2022, Winter 2023, Fall 2023, Winter 2024, Fall 2024

See certificate

James B. Angell Scholar

University of Michigan Mar 2024

“A” record for two or more consecutive terms. Received for two years, 2024 and 2025

See certificate

William J. Branstrom Freshman Prize

University of Michigan Mar 2023

First-term freshmen who rank in the upper 5%

See certificate

Fun Facts

My Chinese name is 张紫宸. In Chinese, 'Zi' (紫) means 'purple,' which traditionally represents nobility, elegance, and auspiciousness. 'Chen' (宸) refers to a palace or the residence of an emperor, symbolizing dignity and grandeur. Together, the name 'Zichen' conveys a sense of royalty, grace, and aspiration for greatness.

I was born and raised in Zhangjiagang, a small city close to Suzhou, China. I studied at the Jiangsu Provincial Liangfeng Senior Middle School (梁丰高中) International Department (now named International Academy) in China before transferring to Rochester Adams High School in Rochester Hills, Michigan. Go Highlanders!