I've spent the last year building AI agents and working with CEOs doing the same. Most of what I built, I threw out. Here are the 10 rules that survived.
I've been building agents across my coaching practice, content production, and business operations — and the biggest lesson is that the hard part isn't the AI. It's codifying what actually matters in your work and organizing it so agents can use it. In this episode, I break down 10 rules for building agents that are genuinely useful — from structuring your data and building context that compounds, to choosing the right model for the right job, mixing deterministic software with LLMs, and managing agents the way you'd manage a high-performing team. No hype, no theory — just what I've learned from doing it every day. If you've been curious about agents but haven't started yet, this is your playbook.
I built my first AI agent and notion, and I was proud of myself. Everything was prepared behind the scenes. For the next day at 9:00 PM I went and worked out. I was on the Peloton and I got a notification on my phone that said, your agent needs your feedback. And in that moment, I had my personal aha that we're living in the future.
And we're managing agents now instead of managing people. Over the past 12 months, I've been working with these tools nonstop and building agents, and also working with CEOs and companies that are working in this new way. If you're not building and managing agents, you're already behind, but it's not because you don't know how to, it's because you haven't started.
I have 10 principles that I've learned from my experience that you can apply today to practice this new skill that's incredibly important for everybody going forward.
Number one, put all your data in one place. Relatively organized, not perfect, but all in one place. On the personal side, think about notion. HubSpot, Google Drive for each type of content, have one location for it, and on the business side. You will have many systems, but you can downstream it into a snowflake or a Databricks and have a data lake.
That's one repository. It's important for many reasons. I'll give you one important one. When you're building these AI systems, there's a technique called rag retrieval augmented generation, which is really powerful one, done right. For example, if I'm preparing for a workshop and I want an example of a case study that fixed a very particular learning outcome, and I put.
A prompt into my favorite model, it will gimme a response, but with rag, I can put a prompt into my favorite model, and then it will automatically go into my data repository, find all of the content that's relevant to my search query, and then put into the model my prompt and 10 pieces of information that it was able to find from my history.
Of other things I've done and examples that worked well and feedback, et cetera, so that the response is 10 times better. That's when it gets really powerful. But it only works if all your data's in one place. Okay.
Number two, , build context that compounds the most valuable thing that you can put into a prompt is rich context with all of the history and information and not tremendous amounts. If you have too much, if. It won't fit, or the motto will just burp. But if you have just enough and it's rich and dense with information, your output is so much more powerful.
I'll give you an example. A lot of my work is executive coaching. I have transcripts that I transcribe from every coaching call. I record these in granola. I have them automatically pushed to notion, and then I'll have an agent create a dossier for every client, and every time there's a new meeting, it will take the dossier and then it'll take the latest transcript and say, take this transcript.
Get all of the rich information and important topics out of it, and then update the dossier. So at any given point in time, that dossier is, let's say two to 10 pages of really rich context of the whole history, of all of our conversations, the most important topics, the stakeholders this person has, the challenges they have, the goals that they have, everything that I need to be able to dive into a conversation.
And be super productive. And now with an agent, that context document is both useful for me, but can go into any prompt for anything that I wanna ask AI about to make the answer 10 times better. And so build context that automatically compounds over time. You don't wanna be hunting and finding stuff and then manually pasting it every single time for each topic.
Or project or client you want one context rich place that compounds naturally over time so it gets better and better and better because the results will get better.
Number three, design for rapidly swapping models. If we look at this layer cake of the agent stack, now you have your data at the bottom. Then you have context, which is taking the data, extracting key information and storing documents that are reusable automatically by your agents. Then you have the model, which is running the prompt along with the context and maybe using RAG to go across your whole database as well.
And at the very top, you have orchestration of determining what question to send to the model and when. And that could be you manually, or it could be an agent behind the scenes autonomously doing it every 15 minutes, every evening, every time you take an action, et cetera. And it's important to understand how each of these pieces fit together, and you have to commit to your data layer and invest a lot in your context layer.
But once you have that part right, that's what most people are missing because a lot of times the hard part is not understanding the model or this thing or that thing. It's being able to codify what actually matters. And organize your information so it turns into context. Once you get there, then you can swap any of the models back and forth and take any model and dump all of this great context into it.
If you look at Gemini Gems or Chat, GPT, custom GPTs or Claude Projects, all of these are just tools to be able to put documents in one place That creates context and then it runs rag across those documents every time you put a prompt in. But you can just do this yourself.
Number four, use the right model for the right problem. One concept we'll introduce here is the context window. So first of all, a token is the unit of economics. In LLM World, every piece of information, let's say you have a document, will get chunked into little pieces, and those little pieces are tokens.
On average, a token is four characters. Which is 75% of a word. It varies a lot depending on the situation and the technique, but just consider it one piece and each piece has meaning. So you can see if apple is similar to banana, but different than dog, as an example. And so you have these tokens and your input is everything that you put into the prompt and it gets broken into a certain number of tokens, and then the output.
Also gets broken into a certain number of tokens, and if you have a reasoning model, you have interim results where the reasoning model is thinking through it one step at a time. Opus 4.6 will do this a lot, as an example from Claude and those. Interim examples, plus the output is your total output tokens, and then you add your input, and that's the total tokens of the total context window.
And so if you have a 1 million token context window, it means you can have about 750,000 words across all three of those. And that's important to consider because sometimes you won't be able to. Stuff, all of the context that you want into a small and inex expensive model, and you need to use the big model, but you don't have to use the big model for everything because it's overkill and very expensive, and it will turn into a situation like you're at the arcades, like a little kid just putting quarters in the machine.
I've been up late at night working with Claude and it constantly wanted more tokens so use the right model for the right job.
Number five, use deterministic software mixed with LLMs. LLMs are great at producing creative information about pulling information together, about reasoning about all of this stuff that was never possible with code in the past, but there are times when old fashioned code that has really strict rules.
If this, then do that. And it's deterministic, meaning with specific inputs, you'll always get the same output versus probabilistic. LLMs are probabilistic, meaning you provide inputs and the same exact inputs will result in different results at different moments, and there's probabilities of different outcomes.
Deterministic software is really good at moving data and. Transforming data and doing certain number calculations, et cetera. For me, for example, when I move data from granola to Notion, I use deterministic software to do that, I use a platform called make.com that matches it all up, moves the data over, changes a couple things, but it's always exactly the same and I know nothing gets dropped.
If I have an LLM do that, first of all, it's expensive and unnecessary, and second of all. I've noticed that sometimes it summarizes or sometimes it truncates, or sometimes it changes something. It tries to be smart, and there are times when you don't want it to be smart. You just want it to be reliable and do exactly what you ask it to do.
And in that case, you're better off mixing old fashioned software. The nice thing is there are a lot of tools for this, and you can ask the LLMs to write the code to run the old fashioned software, but there are many cases where keeping it simple is better. And so don't ask the LLMs to do things that they're not good at.
That you could do as something that's much more specific.
Number six, orchestrate based on frequency and value. So this is where automation and the agentic piece comes in. There's a difference between writing a skill or a script for what you want AI to do and then how frequently it does it and how autonomously it does it. So cloud skills is a great place.
Perplexity computer is another example where you can write a very specific skill and have AI run that playbook. If you will. And so I have an example here of a cloud skill that I've written to go into HubSpot, automatically pull all my pipeline data and give me a report about how I'm doing. This is how many coaching clients you have that are active.
This is how long they'll be on with their current contract. This is your MRR, this is your a RR. This is what the pipeline looks like. Gives me a great view of what's going on with my business. And all I have to do is tell Claude to run this skill. And that's in a markdown file, which is basically just a rich text document, if you will.
And you can use that markdown file in a lot of different platforms. So investing in defining what you want them to do is really important. And then there's, how frequently do you run it? So for the pipeline report, I don't really need that more than once a week, and I'm fine to just go in and say, run my pipeline report.
I can automate that, have it come out every Sunday morning. That's fun. It's cool, but only saves me a few seconds. On the flip side, I have another agent that runs and looks through all my transcripts from yesterday. Says Keith, you have this privileged opportunity to talk to all of these interesting entrepreneurs and founders and executives that are working on AI at these different companies, and we're gonna look through all of the transcripts and find fun ideas for you to post videos about or create content or write blog posts or emails, and it takes every idea and puts it into a notion database for me, and I can just go into the Notion database.
And see all the latest ideas and it will remind me, oh yeah, that's interesting. I did have a conversation about this topic. That would be a great thing to pull forward in an anonymous way and share more publicly. And because of that, I'll miss those pieces of information if I don't have it running continuously.
So that running continuously provides a ton of value to me. Because I get to things I never would've gotten to before. And that's what one of the best unlocks about agents is. It's not just doing something 10% faster or 20% faster. It's doing the things you actually never would've gotten to. And that's a big unlock.
But don't do it for everything. Just automate the things where it's a lot of value and you're gonna miss something otherwise.
Number seven, keep agents narrowly focused, and it's because of the context window. So you probably have a lot of different things that you do. I'll take my interests. I love cooking.
I love fitness. I love ai. I love coaching. I love teaching. I like travel. There's a bunch of different things that will come up in my life that I have ai. For that I built agents for. And at the end of the day these things are constantly playing games to squeeze as much into the context window as they can.
Claude will run rag on your past transcripts and your past conversations to try to cherry pick information that's really relevant to what you're talking about so that it can know more about you as it goes through, but it still can only fit so much into the context window. So if I had one agent that went across all of the domains of interest for me.
It would just blow up, meaning it dilutes because it would drop pieces of information that would be really important for that topic. 'cause it's trying to fit all of the topics into one context window. But if you keep each agent focused and say, this focuses on marketing and this focuses on revenue and this focuses on this thing, then you can fit a lot more in the context window for that narrow job.
And each of them will be 10 times better literally, than trying to have one agent do everything. So keep one. Agent one, job, one set of context. Narrow focus, 10 times better results.
Number eight. This is not hands off. Get ready to iterate. This is about rolling up your sleeves, getting your hands dirty, and just diving in that process will teach you a what you're actually trying to accomplish in life. Because codifying what you're trying to accomplish is actually half the battle.
And what a lot of people are missing. And then the other portion is using these new tools to make it much more powerful or much faster or much easier, and you're only gonna learn by diving in and doing it. And so I'll leave you with some really clear takeaways so that you can tinker with it because that iteration process is going to get you there, but you can't do it unless you're just knee deep in it and you just get started.
And it won't happen easily. It will have some bugs. It will have some issues. You'll have to adjust things. Sometimes it will produce terrible results, but if you refine it and every step gets one little bit better, it can totally change your life.
Number nine. Managing agents is a new skill. There's a Venn diagram. Some of it is the same as managing humans, and some of it is new. Some of it is different, but at the end of the day, it's about making sure that you prepare and serve up the right context. It's about being very clear about what your goals and objectives are, what you want it to accomplish, and then it's about providing feedback as you go.
It's setting the bar for quality. Saying, no, that's not good enough for this specific reason. Let's get better. No, that's not good enough for this specific reason. Let's get better. And then documenting that each time in the agent's protocol or a skill file, et cetera, so that every single time it gets better.
And ultimately you have something that's reliable that is still probabilistic. It'll behave in different ways, but it will produce in general, excellent results for what you're looking for. And this clarity of communication, when I grew up as a product manager. It would take you years to get practice in communicating with engineers and writing documents because you would provide this communication, and then they would come back and have questions, and you'd communicate and they'd have questions, and then they would ship code weeks later, months later, you'd ship a product six months later, and this feedback cycle took so long.
That a lot of the communication and a lot of the leadership skills, a lot of the repetitions just took a while calendar time, and now those repetitions are so condensed that you can spend a few weekends cranking on a project like this and learn so much just from putting a prompt in and getting bad results and learning how to do it better and better and better, and learning how to provide the right context at the right time and getting it to compound and then orchestrate in the right way.
It's up to you how fast you wanna learn these things, and it is a learning curve and it's moving as we go. And so it's not like you're learning how to do something and that field is stable. It's learning how to do something in a field that's moving. And so. It's a leadership skill. It's important for the future.
Some of it's the same as managing humans. Some of it's different, but it's something all of us need to learn, and the value of it is very, very high, and the risk of not having it is also tremendous.
Bringing it home. Number 10, and this is a tough one to swallow. Half of what you build is disposable, and there are two reasons for it. The first reason is that you learn as you go. You're just learning about what you actually need, and you don't know until you build some things and you use 'em, and then finally you get this really, really crisp picture and you might need to throw out some of the stuff you already did.
And then the other reason is because the tools are changing so fast A year ago. I built Agentic tools to help me with my coaching workflow to help me with my ideas, but they didn't exist in the world, and so I was using GitHub copilot and shipping code and using ai, and then Cursor came out and I was using Cursor, and then Cloud Code replaced that.
But ultimately, I built each layer. I built my own data layer. In fire store, which is a Google database. Then I built a context layer on top of that in these documents, and I built my own vectors and semantic search, and then I built the orchestration layer on top of that to fire and trigger certain things at certain times and certain moments and run every day behind the scenes.
And then I had the models that I was pulling in via API and then one by one. It was well. I don't really need this data layer And then my context layer will be notion. It has such a great API. I'll just move things in and outta notion.
Some of that was just learning and some of it was new tools, and then when the orchestration came out and there were cloud skills and there was perplexity computer and now there's open claw. You have these orchestration abilities to trigger these things at specific moments in time and run autonomously behind the scenes without you taking any action.
And I don't need any of that, so I just deleted and throughout the whole app, I don't need any of it now, and I just spent a lot of time on it. But the time that I spent on it was useful because I learned and I learned a, what I needed to do and how to codify my life and my work, and that was a huge portion.
Of the time and the energy and the value. And then also I learned a lot about these tools and how they actually operate and what works and what doesn't. And so now I can operate much faster as I move from one tool to the next. And I truly believe that if you get in the weeds, even though you have this mindset that you're gonna throw half of it out, and you know that's the case six months from now, the value of learning is so high.
That it's worth it. And if that's the mindset you have, it's gonna be really fun. And I'll leave you some really specific takeaways of just go to cloud skills and build a skill. If you haven't done it, it's a markdown file. And guess what? They will walk you through it. You go into cloud and say, I want to build a skill.
Pick one particular thing, preparing for your day tomorrow, checking your email for you. Go in and go to the connectors tab at Google Drive at Gmail. Add whatever the tools are that you're using. Or do it in Notion. Notion has agents. There's an agent tool right in the left side bar. It will also walk you through building your first agent or use perplexity computer, or if you wanna be a little adventurous, use open claw.
But pick one data repository that you have where all your data's in one place, and then build a context layer on top of that and it can start as one document, this is my context document, and then build an agent or a skill on top of that where you can run a particular workflow. And at the end of the day, it's just the data you already have.
It's a notion document or some other document that contains context and then it's a skill that can take particular actions and it's orchestration that goes off at particular times. And then you do that once end to end and you can do it a single day in a couple hours. It's so worth the time. And from there you get the momentum going and you'll be able to truly learn how to run agents.
And so these are my 10 principles. I hope they help you think about this, maybe avoid some of the problems that a lot of people have and realize it's really not that bad. But you just gotta get started and you gotta put your energy into the right thing. So go ahead, go figure it out. Use your data, build your context layer, find the orchestration that works for you and change your life.
One email a week. No fluff, just lessons from the field.
