Common Pitfalls of Using AI, ML, and LLMs with Suyash
Joining us in the Bay Area this Summer, Suyash brought his expertise in AI and ML to the students in our 2025 Bay Area Entrepreneurship Program.
Understanding the common challenges that AI poses can help you make better decisions about incorporating AI into your product or service. It can also save you some heartache by navigating this newer landscape of AI.
This article is the final piece in a three-part series by Suyash, exploring how startups can use AI effectively and navigate its challenges. Read the rest of the series here:
Should I Use AI in My Startup?
ML is not a visual thinker
When incorporating Machine Learning into your product, the foundational truth is that computers cannot *see*. Computers are incredibly fast at performing calculations. With the right algorithms, it can process things like pixels of an image in a way that humans are incapable of.
Tasks that humans are really good at, such as “find cell C4 on a spreadsheet,” can become a nightmare for ML systems. This deceptively simple task causes you to bang your head up against the wall when trying to get a system to do something easy for a 7-year-old to do.
So instead of using ML as a hammer and every problem as a nail, using ML to do tasks that are difficult for humans (finding patterns in large sets), while also using more traditional algorithms, can allow you to use the strength of both tools to create even better products.
LLMs are inefficient
The way that LLMs work is, by nature, inefficient. They have a wild number of parameters that they use to generate a lot of text. If you are incorporating LLMs into your product, even in the back end, know that it will use a lot of computing power and will need to generate a lot of text, meaning that the more complex the task you are doing, the slower it will be.
Further, you will most likely be utilizing a cloud provider and costs can easily skyrocket, especially if part of the system encounters bugs that make it even more inefficient.
If you can achieve the same outcomes through a traditional algorithm, you can create a far cheaper and more efficient solution, which may ultimately be better for your company and its customers.
AI shouldn’t be used the same across all organizations
If you are just starting a business, using AI can help you get to market quicker and test iterations much faster than ever before. This will have tradeoffs, but when used correctly, this can be a huge advantage.
If you are running an established company with a complex system, you shouldn’t destroy your code base just to move to a new tool. Instead, finding opportunities for using ML in a wise way that enhances your customers’ experiences is the right approach.
**LLMs have no memory**
When you send a prompt or your user sends a prompt, that is being sent to a status model that will process the query. However, there is no memory of how it was used in the past.
To overcome this, systems have utilized LLMs to summarize previous prompts, thereby sending context with the new prompt. Over time, this becomes unwieldy, so the system begins taking shortcuts to stay efficient. However, missing information can be critical to providing an expected answer.
This becomes even more difficult when sending things like code, which is already very compressed. How do you summarize 20,000 lines of efficient code?
If you are using AI to help you code, as your codebase grows, it will degrade over time. There are methodologies to minimize the effect – you can get better and smarter at this, but there are limitations that you will eventually run up against.
Good practice is good practice, even in AI
Understanding how code functions, even at a more theoretical level, is a critical step if you are planning on using AI efficiently and well. Practicing good habits for code should also extend to your interaction with AI.
A great example of this is adding a debugging section as you vibe-code, allowing you to process and understand what a system is doing to help diagnose and solve issues.
Or, as you ask AI to help you code in a more traditional console, engineering the prompts to follow best practices and organizing your code well, among other best practices, will help you maintain a better codebase to build upon.
AI has high potential for risk and reward
AI tools have experienced significant growth in the past few years, unlocking new areas of products and services. These tools are only going to become more sophisticated and essential. By understanding some potential pitfalls, you can avoid some possible challenges.
Perhaps more importantly, if you can learn how to leverage these tools well, you can build in ways that will give you an advantage in the marketplace.
