AI Decision Models: Distinguishing Them from LLMs

AI decision models have recently attracted significant attention as a distinct approach within the field. Jev, developed by TypeSafe AI, is one of the most discussed examples in this category. Rather than generating free-form text, Jev produces structured choices, scores, and confidence estimates.

Although traditional AI models have long been used for predictions and classifications, Jev focuses specifically on creating structured decisions that software can utilize directly. This method offers an alternative pathway for building AI applications, particularly when a system requires the ability to evaluate information and determine subsequent actions.

How does Jev work?

In a standard LLM workflow, you submit a prompt and receive generated text in return. Even when you request JSON output, the application typically needs to parse the response and validate its contents before it can be used effectively.

Jev operates differently. You supply context, define the specific questions to be answered, and specify the expected data type for the answers. Jev then returns structured results that your application can process immediately without additional parsing.

Jev supports three primary question types:

• Choice: Selects an option from a predefined list, such as billing, technical support, or sales.
• Score: Rates an item against an ordered scale, such as low, medium, or high priority.
• Yes or no: Estimates the probability that a statement is true, for instance, whether a support ticket requires human intervention.

You can pose multiple questions regarding the same input within a single request. For example, a support application could classify a ticket, assess its urgency, and determine if it needs human review all at once. Jev returns these results in a structured format, enabling the application to determine the next steps.

Decision models vs. LLMs

LLMs and decision models serve distinct purposes. An LLM is capable of explaining decisions, generating content, writing code, or reasoning through complex tasks. In contrast, Jev is designed for focused decision-making with clearly defined outputs.

The key distinction is not that LLMs are incapable of making decisions; they can indeed classify information and return structured data. However, Jev is purpose-built for this specific task, eliminating the need to generate a free-form response that an application must subsequently interpret.

This makes it particularly useful for repeated, well-defined decisions. For tasks requiring detailed explanations, creative output, or complex multi-step reasoning, an LLM remains the more suitable tool.

What are confidence scores?

A key feature of Jev is that its outputs include probabilities and confidence information. These values assist developers in assessing the strength of the model’s preference for a particular answer.

For instance, a customer support application might query whether a ticket should be escalated. If the model indicates a high probability that escalation is necessary, the application could route the ticket to a senior support team. Conversely, if the result is uncertain, it could flag the ticket for human review.

However, high confidence does not guarantee correctness. A model may be highly confident yet still make an incorrect decision. Developers should test results against real-world examples and select thresholds that align with the potential consequences of each decision.

It is also crucial to distinguish between the probability of an answer and a separate confidence score. The precise meaning depends on the output type. Applications should adhere to the model's documentation rather than assuming every returned numerical value represents the same metric.

Where can decision models be useful?

Classification and routing

Many applications require sorting incoming information before processing it. A support platform, for example, might classify requests by topic and direct them to the appropriate team. Similarly, a business application could categorize documents, emails, or customer feedback.

Jev can return a category from a predefined list, allowing the application to route the item efficiently without parsing a generated paragraph.

Scoring and evaluation

Decision models can also evaluate information against a defined scale. For example, an application could rate the urgency of a support ticket, score a document for relevance, or assess whether a response meets specific criteria.

These results can be used to rank items, prioritize work, or identify cases requiring further review. The quality of these decisions still depends on the model and how well the criteria describe the task.

AI agents and tool selection

AI agents use models to determine which actions to take. An agent might select a tool, classify a request, or determine if a task requires additional information.

While a general-purpose LLM can make these decisions as part of its broader reasoning, a decision model can handle a narrower choice before the agent takes action.

For example, a custom workflow using Hermes Agent could leverage Jev to classify an incoming task before passing it to the agent. It could also assist in determining which workflow should handle a request. This would require a custom integration rather than relying on assumed native support.

Decision models should not be relied upon as standalone security controls. If an agent can execute commands or modify files, permissions and deterministic rules must enforce which actions are permitted. While a model's classification can inform this process, it should not replace these safeguards.

Alternatives to Jev

Jev is part of a broader category of models designed to produce structured decisions. OpenJev and Laya are two other projects worth exploring when comparing different approaches.

OpenJev

OpenJev is an open project related to the Jev approach. It is worth examining if you wish to explore an implementation outside of a hosted service. Before using it, verify its current availability, hardware requirements, supported features, and license terms.

Laya

Laya is an open-source decision model from Convai Innovations. It focuses on tasks such as choosing from predefined options, rating information, and answering yes-or-no questions. It may be relevant if you want to evaluate a model that can run in your own environment.

These projects should be compared based on actual performance, not just their stated capabilities. Use consistent test data and measure accuracy, latency, resource requirements, and how well their confidence estimates align with real outcomes.

Can you run decision models locally?

Running a model locally provides greater control over the software environment and data processing location. However, deployment options vary by project. Jev is available via an API, while open-source alternatives may offer options for running inference on your own hardware.

Hardware requirements depend on model size, memory usage, and throughput. Some models can run on a CPU, while others benefit from a GPU, especially when handling high volumes of requests.

If you want to experiment with local AI models, DaDesktop provides cloud desktops with dedicated GPU options. You can use them for development and testing without purchasing a high-end GPU for your local computer.

Learn more about running local LLMs or explore AI agent environments.

When should you use a decision model?

Decision models are useful when an application needs a defined answer that can trigger a specific next step. They can complement LLMs by handling classification, scoring, and routing while a generative model manages language and more complex reasoning.

The optimal choice depends on your specific application. Test the model against your own data, compare it with conventional classifiers and LLMs, and measure accuracy, latency, and cost. For critical decisions, include human review and appropriate safeguards where necessary.

Want to discuss your use case? Contact us and we'll point you in the right direction.