r/Clojure • u/yogthos • Sep 24 '26

Introducing Lev

https://yogthos.net/posts/2026-09-24-introducing-lev.html
44 Upvotes

91% Upvoted

9 comments sorted by

14

u/Illustrious_Car344 Sep 24 '26

The hype behind this proprietary model is so strange, I see as many people question why anyone even cares and nobody can give an answer. 

17

u/yogthos Sep 24 '26 edited Sep 24 '26

Classifier models like Jev and openBERT are pretty useful because they're efficient and can be highly accurate in specific domains. They're best suited for bounded decision tasks requiring calibrated judgments rather than text generation. Scenarios where you have a fixed set of options to pick from like determining which team should handle a support ticket are a good fit for this. While Jev gets a lot of hype, you can easily do a lot of what it does using a small local model and still get subsecond results with it. And that's basically what I illustrate with Lev.

1

u/rcorrear Sep 25 '26

What I’ve found missing from them is that once you have identified your options you still have to pass it through an LLM to generate some sort of structured data, that is unless you want to stay in the pure string world. IMO these two would be killer if they could extract structured outputs at the same time the classification is done.

→ More replies (1)

12

u/BeautifulSynch Sep 25 '26

A lot of the people who are actually building orchestrations and harnesses (said as someone in that situation) have been heavily struggling with getting LLMs to give cheap, fast, parse-able results to allow proper neurosymbolic reasoning (eg classifications, scoring, LLM-as-Judge metrics, data analysis, anomaly-detection, etc).

- You can do it, but it’s a bit unnatural to the model without fine-tuning, and you can’t ‘fire and forget’ the infra and prompts you used to get format compliance (unlike with well-designed deterministic systems, where the module-API usually screens off the internals), in case a change elsewhere in your system makes the LLM deviate again.

I haven’t used Jev/Laya/etc in a ‘preprod’ test yet, but they seem to be built to solve that problem out of the box, which is interesting and worth looking into.

7

u/ares623 Sep 24 '26

gonna have to rename Astroturfing to Jevving.

but I do kinda see the appeal. A general purpose classifier that used to be out of reach for most engineers/teams does sound enticing. I'm sure a lot of us have been thinking of little features/tools that would have otherwise been blocked due to having to (somewhat reliably) classify natural language.

1

u/carlosomar2 Sep 26 '26

Turns out that using opus or astra to decide if you customer ask should go to the agent for billing or the agent for lending becomes very expensive quickly. It's also slow as hell under load and makes mistakes. Jev can fix all of that potentially

1

u/yogthos Sep 26 '26

What's even better is that it turns out that a local model running on the CPU might be able to handle a lot of these cases.

2

u/lgstein Sep 26 '26

Presenting a user with two buttons can fix that, too.