Bringing clarity to the AI ​​mess

Machine Learning


AI feels powerful, but most teams struggle because they can’t define the intelligence they actually need. However, there are ways to address this challenge.

For those working in AI, the dizzying speed of growth is a constant struggle. The speed of this field never slows down, and staying up to date with the latest information becomes a daily task.

I think I can share a personal experience that may help you understand how this kind of speed affects us all. Let’s say that back in 2003, someone bought a bicycle, registered it, and fixed a license plate with a white background and yellow font. A week later, the government decided to change the rules. License plates must have black letters on a white background. Two weeks later, the rules will change again. This constant change led some people, at some point, to decide to put all possible formats on one license plate so that it would technically cover whatever rules came along next. This was a common problem for people who got their driver’s license between 2003 and 2005.

I went through this confusion myself when I got my license in 2005. And this is exactly what developing AI applications feels like today.

If you’re an AI developer, you probably already know the biggest pain points. The field is changing so rapidly that even a short break can quickly put you behind. When you return to your team after a short vacation, within minutes someone says, “You’re outdated, something has changed.” This is how fast stacks, algorithms, and platforms evolve.

Transformation itself is not new. The industry has seen the birth of the internet, cloud computing, mobility, and even the early stages of AI. What is different about this phase, however, is that the scope of the AI ​​is not defined. Leaders in various industries use the phrase “everywhere, everything.” Unlike the cloud and the internet, where there are clear functional boundaries, AI doesn’t have clear boxes. What exactly should AI do for your business? Where does it stop? How do I define scope?

This lack of definition is the fundamental problem. AI is everywhere, but organizations don’t know what “everywhere” means.

To solve this, I work on four principles to help bring clarity: Define, Target, Scale, and Grow.

define

The first principle is to define the level of intelligence required in your solution. Not every problem requires an LLM or deep learning model, so you need to be careful here. This principle can be handled at three levels.

Good old AI (GOFAI)

At the heart of GOFAI is a rules-based system. The logic is in the code. If your business rules change, just edit the rules. GOFAI is still very useful and practical and is the right answer for many use cases. Everything doesn’t have to be complicated.

machine learning

Use ML when your system needs to learn from patterns rather than rules. This is where training data, prediction, supervised and unsupervised learning come into play.

complex AI

This includes deep learning, dynamic models, advanced architectures, and LLM. AI needs to evolve, but setting limits is equally important. At each stage, simply continue to evaluate the question, “Is the current level of complexity strictly necessary to solve the customer’s problem?” Often the honest answer is “no.”

Therefore, developers must stop and choose the minimum intelligence level that meets their requirements. Nothing more. Nothing more.

target

The second principle is to address business problems in the right way. AI application development is similar to traditional SDLC, but with one major difference. Software development involves designing, coding, testing, deploying, and maintaining. But AI adds a permanent and necessary final step: refinement. This is different from iterative development. Improvement is a continuous loop built into the system itself. Feedback from customers and users should be regularly collected, analyzed, and fed back into the model. Without improvements, AI products will quickly degrade.

This means two things:

  • Feedback mechanisms are non-functional requirements. It should be present in the code or as a direct touchpoint with the customer.
  • AI is no longer an isolated IT offering, so developers need a top-down view from business case to implementation. It affects not only the software but also the business model.

Previously, we didn’t revisit our business case frequently. That’s what AI has to do. Needs evolve. The model will drift. The direction of the use case will change. Developers can no longer focus solely on implementation. You need to understand the charter, the model, and the end-to-end improvement cycle.

scale

The third principle is about scaling. Traditionally, we designed modules based on functionality. Module 1 performs function A. Module 2 performs function B. Testing followed the same pattern.

However, today’s AI development requires a shift from a functional mindset to service-level development. why? This is because every module now calls external services (LLMs, APIs, cloud platforms) many times within a single workflow instead of once. We no longer live in a world where only one or two modules are externally connected. Now all modules do.

This is where microservices become essential. If your organization currently uses GPT and switches to a different provider next year, how will your system adapt? Will you redo the entire codebase if the model is upgraded?

Microservices allow changes in one service to flow throughout your organization without having to rewrite everything. This is the ability to think in terms of services rather than features.

Cloud providers are also evolving. Previously, we used Infrastructure as a Service. Currently, we rely heavily on platform-as-a-service (APIs, ML services, LLM endpoints) models. Therefore, our architecture must match the service mindset.

growing up

The fourth principle is growth. To move forward responsibly, we need to understand where we stand. Gartner’s AI maturity model describes five levels:

1. consciousness: Employees understand the basics of AI.

2. active: Teams experiment with POCs, hackathons, and simple use cases.

3. In operation: AI improves the efficiency of internal operations. This is true for most organizations today.

4. product: AI provides customers with products that are accurate and reliable.

5. sentinel: A fully autonomous decision-making system without human intervention. Autonomous driving is one example.

Importantly, organizations do not move through these levels sequentially. You cannot start Level 2 after finishing Level 1. Instead, activities across all levels must be performed in parallel. For example, companies can run operational AI systems while creating responsible AI awareness programs in parallel. Growth is not linear. It’s layered.

  • Increase awareness with company-wide training programs, expert lectures, AI meetings, and more.
  • Active exploration comes from hackathons and simple POCs.
  • Operational impact typically comes through a central team or center of excellence that identifies and implements use cases.
  • Product-level reliability comes from more powerful compute, streamlined pipelines, and model governance.
  • The Sentinel level is a long-term vision and represents the highest form of autonomous intelligence.

So why has AI suddenly reached a tipping point? We’ve been using different types of AI since the early 1900s. The term itself appeared in 1842 and was officially coined in 1956. We’ve seen expert systems, rule-based systems, games like Pac-Man and Chess, and early healthcare applications. But the real acceleration started around 2011 with the rise of recommendation systems and machine learning and deep learning.

Today’s inflection point is driven by four distinct trends.

data explosion

We collect data from our phones, vehicles, applications, and sensors, both consciously and unconsciously. We now have enough data to experiment with, and our algorithms can also generate synthetic data.

cloud computing

In the past, computing meant connecting to a mainframe in a separate location, often with delays and limitations. Now, you can instantly get 4GB, 8GB, or 32GB of GPU power with just a few clicks. Cloud platforms have made high-end computing accessible to all developers.

Improved algorithm

New AI models emerge all the time. But the real reason for this speed is that everything is open source. This is no longer under the control of the company. Developers around the world are contributing to global progress.

open post

The open source ecosystem will accelerate this industry. All improvements will be available to everyone. This is why it feels like the field is moving at lightning speed.

AI is everywhere. AI is everything. But unless we define a box for ourselves, we end up feeling overwhelmed.

No organization progresses linearly, and no developer can afford to work without understanding business and refinement cycles. AI requires a different way of thinking that is structured, strategic, and continuously evolving.


This article is based on the session “Beyond Automation – Strategic AI Integration with GenAI and LLM” by Pradeeba P., Thoughtworks Delivery Manager, at AI DevCon in Bangalore. This article was written and supervised by Apurva Sen, senior journalist at EFY Group.





Source link