Building an AI Poker Coaching App
As a mathematician, AI practitioner, and avid poker player, it was inevitable that I would build an AI poker app. I can’t help it!
I was walking down the strip through the 110-degree heat between my two favorite poker rooms. It was day 10 of the World Series of Poker(WSOP) — the few months of the year when the best players from all over the world converge on Las Vegas and compete in many black-to-back, long, expensive, serious tournament events.
I had traveled out to Las Vegas for the WSOP myself, and just never left. I live here now. And I’ve learned the hard way that those same pro players in town for the WSOP will also try their luck in the local poker rooms at the casinos on the strip. So my bankroll was feeling a bit thin after running into so many pros all over what I now thought of as my home circuit. I missed the good old days before the WSOP was in town when my favorite poker rooms were full of inexperienced players (🐟) looking to try their luck at poker. That’s when I was making money (🦈). So as I trudged between poker rooms, I was searching for something — the best tournament structure in town.
Choosing the right poker tournament involves analyzing a lot of factors — the payout structure, blinds and antes, starting stack, levels, number of players and their skill level, and so much more. The probability of cashing (reaching a top rank where you receive a payout or are “in the money”) also depends on the player’s skill, which can be influenced by various factors such as diet, exercise, sleep, and play style.
I’d been tracking my results (they looked great before the WSOP came to town). But I wanted a single database to record my tournament performance as well as all the metadata I was using for game selection.
I also had a pretty mathematical equation floating around in my head when I thought about tournament selection. This equation modeled my personal style and skill level, as well as additional variables that I might be able to control. But I needed to bring that equation to life programmatically so I could run it against new tournaments I’m considering.
And then I thought- wouldn’t it be cool to have an AI chatbot to tell all my bad beat stories to?
I didn’t just want to dump my hand analysis into ChatGPT; I wanted an AI assistant that was a poker expert to offer personalized advice based on my data and skill model. Then it could dynamically adjust the coaching advice to help me improve my game and fix my leaks over time.
So I built it 🤓.
Meet EVIE — the Expected Value Intelligence Engine.
This app does not automate poker. Far from it. It is meant to work as a pre- and post-game coach, helping you track outcomes and improve your live game by chatting through your results. The AI Poker Coach is an AI system that utilizes a combination of historical data and past chat conversations to develop an extensive model of the player’s style, abilities, and areas for improvement. Then, it tailors personalized coaching for each tournament before they enter it, helping us choose the right fields to enter and devise a winning strategy for each one.
🏛️ Code Architecture
For this project, I assembled an open-source and lightweight tech stack that let me move fast while building reliably. With the constant launch of new AI tools for developers, it can feel overwhelming to keep up. But I’ve learned that the best way to understand new tools or AI methods truly is to get hands-on and build something real. The lessons I’ve learned by integrating these tools directly into a functioning system, rather than just reading docs or watching demos, have helped me better understand what they can and can’t do, and how they fit into the overall project.
All these tools are either open-source or require minimal investment to use today.
Cursor AI (AI-Powered IDE)
- Cursor is an AI-assisted code editor designed to enhance developer productivity through features like code completion, natural language commands, AI chat integration, and deep codebase understanding. I used Cursor extensively throughout this project and learned the importance of collaborative iteration with AI. Read more about AI-assisted coding best practices here: https://statistician-in-stilettos.medium.com/best-practices-i-learned-for-ai-assisted-coding-70ff7359d403
MongoDB (Tabular Database for analytics)
- MongoDB is a flexible NoSQL database designed for scalability and modern application needs. In this project, I use MongoDB to store structured/tabular data, making it easy to handle queries, updates, and persistence for non-vector information like user profiles, settings, or metadata.
Weaviate (Vector Database for embeddings)
- Weaviate is an open-source vector database that supports semantic search using embeddings. I use Weaviate to store semantic embeddings of chat history for the retrieval-augmented generation (RAG) chat. This is essential for building the context-aware LLM chat results.
Comet Opik (LLM Observability & Evaluation)
- I implemented a custom observability strategy using Opik (via Comet) to track cost, log traces, and evaluate performance using LLM-as-a-judge metrics.
Streamlit (App Deployment Framework)
- Streamlit is a Python-based framework for building and deploying interactive web apps without needing front-end expertise. I used Streamlit to quickly deploy the application and create a clean, interactive user interface for testing and sharing the tool.
The software architecture consists of 3 core modules: Math Module, Data Layer, and RAG Chat Layer. There are also simple analytics tools and a pretty UI to bring it all to life. The goal of this modular architecture is clarity and maintainability by design. Let’s dive into each module and learn what it does.
📊 Data
The user can enter data from past tournaments, and this data is stored in MongoDB. This allows the poker coach database to serve as a system of record for a player’s tournament performance, collecting a wide range of variables, including both tabular and free-text notes, which can be used to correlate performance in a mathematical model.
✖️ Math Module
One of the core capabilities of the AI Poker Coach is the ability to mathematically model a player’s expected value (profit) from playing a specific tournament. I wanted to implement my own mathematical equation for this because I had some ideas on how best to model this.
Modeling a player’s ability to cash in the i-th place in a poker tournament is tricky because not only is it influenced by skill, variance, and the tournament type, but it will also vary from casino to casino.
I approached defining the model by first designing the equations with pen and paper. I selected an equation that was the most suitable way to model the expected results from poker tournaments. I identified this equation through my knowledge of statistical modeling, and a few hours of looking sideways at data examples until I went crosseyed, drawing equations on my window and erasing them, and talking to myself extensively.
I ultimately landed on this function that I believe elegantly captures the patterns in the data and the environments I aimed to model.
I won’t bore you with the math, but for a deep dive on my model, you can refer to the paper here (link coming soon).
This model helps me understand my abilities and identify where I need to make adjustments for different tournaments. Notice that the flexible shape of the beta distribution models poker player skill variance well.
🔵 Left-skewed: Skilled player likely to finish near the top.
🔴 Right-skewed: Unskilled player more likely finish low.
🟢 Uniform: random/unskilled player.
🟣 Bell-shaped: consistent average performer.
The driving purpose of building this software was to implement this mathematical equation in a way I could interact with it over time and use it to improve my play. Therefore, the majority of the work developing this app was focused on this module. The chat interface that leverages the LLM as simply a “UI” to this equation, as well as a data collection tool to help personalize the output of the coaching over time.
💬 Chat Interface
In the chat layer, an LRM (a kind of LLM called a Large Reasoning Model) will review the output of the equation as well as interpret patterns in past chat histories to make optimal recommendations to the player.
The AI chatbot does not do advanced mathematical calculations. Instead, it will review the output of the equation to make optimal recommendations to the player, and deliver these recommendations in a chat interface. So I like to think of the chat as a kind of interface to the equation.
Note that I would never use an LRM or LLM for the purpose the expected value calculation or model defined above serves. I do not think we should ask an LLM the questions I am asking in this equation, as this is not a language task. Instead, we need to solve for those answers mathematically. This is because LLMs are from a family of models designed for NLP (Natural Language Processing) and are mathematically constricted to perform well on language data, so I use LLMs only for language tasks like conversations. Pure mathematics, however, cannot be replaced. So it was imperative that I implement the mathematical module separately and only allow the LLM to interpret the output, and not perform any mathematical analysis. The LLM here serves as a mechanism to discuss the outcome of this equation with the user.
Model selection:
Another concept that is important to understand mathematically to avoid misconceptions is LLM “reasoning”. Recently, the concept of LLM reasoning was introduced, and there has been a surge of developing LRMs that generate detailed intermediate thought processes before arriving at a final answer. (such as DeepSeek-R1)
These models are popularly tested on benchmark datasets of mathematical problems, with the goal of “proving” the LRMs can reason mathematically.
Apple’s research team bravely debunked and explained the reasoning myth in the 2025 paper, The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. I like these models a lot, but I don’t like them for solving math equations. These models have been shown to mimic strategic thinking. So I’ve powered the LLM Chat component of my app with a reasoning model so it can interpret and discuss complex scenarios verbally.
🔍 Pro tip: Use a smaller cheap model like o3-mini for debugging and switch on gpt-4-turbo for reasoning for live.
Prompt engineering:
The prompt is designed so this model will output useful advice on tournament and cash strategy, and help you analyze hands and improve your play.
🔍 Pro tip: An easy hack to jumpstart your prompt engineering — ask AI to write the prompt for you. AI actually crafts great prompts!
RAG:
The AI chat is enhanced with personalized user data through a RAG architecture. This RAG system transforms the AI coach from a generic chatbot into a truly personalized poker mentor that grows with your game!
🔍 Observability
I implemented a sophisticated LLM eval approach to monitor the cost and quality of my app to ensure a positive user experience using Opik. As I’m building and experimenting with different models, prompts, and architectures, I use Opik to version and track different iterations. I also set up monitoring for cost and token use so I don’t get any surprise bills and can manage and anticipate costs as I scale this out to more users.
In addition to monitoring costs, I designed LLM-as-a-Jude metrics to protect my app against misuse. I have bespoke customized metrics to identify and block off-topic discussions in the app.
Conclusion
As a next step, I’ll deploy and share my Poker Coach AI with a few local poker players so they can use it in the real world and provide customer feedback before I iterate more on the feature set. I started simple and shipped an MVP so I could start getting feedback. Then, I’ll let that feedback shape the next steps. I’m creating tech debt and have a ton of “next steps” in my messy notes. These ideas are fluid and may change based on what I’m learning as we use the app.
This project isn’t finished yet, but that doesn’t stop me from sharing it! I’ve demoed it to several audiences and learned so much from their insights. Never be afraid to showcase unfinished work — if you manage expectations, you’ll get invaluable feedback.
All content in this article is copyright Claire Longo. All IP belongs to Claire Longo unless otherwise stated.
