Building a Streaming Chatbot with Vercel AI SDK: What I Learned

I wanted my website to be more than a static page. Instead of just listing projects and links, I wanted visitors to be able to ask questions and get immediate answers, something that feels closer to a real conversation.

At first, I considered building a chatbot from scratch. But once you get into it, things get complicated quickly: HTTP streaming, managing partial responses, message state, loading indicators, and error handling. That's a lot of infrastructure for a portfolio project.

So I used Vercel AI SDK.

Why Streaming Actually Matters

Without streaming, chatbots feel slow and awkward. You send a message, wait a couple of seconds, and then the full response appears all at once. Even if the response is fast, the experience feels broken.

With streaming, users see the response appear as it's being generated. The first words show up almost immediately, which makes the interaction feel responsive and natural. Even if the full response takes a second or two, users feel engaged instead of waiting.

What the SDK Gave Me (and What It Didn't)

The biggest win with Vercel AI SDK is how much complexity it removes:

• Streaming responses without manually handling HTTP streams
• Message and loading state handled via useChat
• Clean integration with OpenAI
• A simple, type-safe API

That said, it's not "magic." You still have to think about:

• Costs (LLM calls aren't free)
• Rate limiting (I capped it at 10 chats per IP per day)
• Context size (long conversations get expensive and slow)

The SDK makes the hard parts manageable, but the architectural decisions are still yours.

When I Wouldn't Use This

I wouldn't use a streaming chatbot for everything:

• If the content is static, a FAQ is simpler and cheaper
• For high-traffic sites, API costs add up quickly
• For sensitive data, sending prompts to third-party LLMs isn't acceptable

But for a portfolio with moderate traffic? It's a great fit.

Final Thoughts

Using Vercel AI SDK let me focus on the experience instead of building streaming infrastructure from scratch. It turned what could've been weeks of work into something approachable, while still forcing me to think about real-world trade-offs.

For me, it wasn't about adding "AI" to a portfolio, it was about improving how people interact with it. And in that context, streaming made all the difference.