Benchmarks

👋 Sign in for the ability to sort posts by relevant, latest, or top.

techaiwire

Sep 19

Laya is a 421M open-weights answer to Jev

#llm #openweights #inference #benchmarks

4 min read

Qasim Parray

Sep 19

Efficiency Hallucination: Every Model Rewrote Code That Couldn't Get Faster

#llm #aicodingagents #codeoptimization #benchmarks

7 min read

techaiwire

Sep 14

DeepMind agents blew the whistle on cheating agents

#aiagents #aisafety #benchmarks #google

4 min read

Cole Halton

Sep 14

Giving a coding agent more time barely helps

#agenticcoding #benchmarks #evaldesign #ai

2 min read

Cole Halton

Sep 13

The render said clean, the file said broken: why AI visual QA lies

#aireview #evaldesign #benchmarks #independentverification

2 min read

Peremptory

Sep 7

Claude Formalized Fermat in 11 Days. The Math Isn't New.

#anthropic #research #claude #benchmarks

3 min read

DaisukeYoda

Sep 6

13 of 14 Models Write Messier Code Than the Human Who Fixed the Same Bug

#python #ai #codequality #benchmarks

6 min read

techaiwire

Sep 8

GPT-6 Astra scores 95% on one robot task, 10% on another

#benchmarks #openai #anthropic #robotics

4 min read

JaviMaligno

Sep 16

Benchmaxing: Winning the Exam Is Not Doing Better Work

#ai #evaluation #claude #benchmarks

8 min read

Rickesh T N

Sep 5

Temperature 0 is not reproducible. I measured 30 percent of my output changing between identical runs.

#llm #benchmarks #localllm #reproducibility

3 min read

Rickesh T N

Sep 5

Our 4B beat Claude Opus on a 440K-token corpus. Then it came last on the public benchmark.

#llm #benchmarks #localllm #privateai

4 min read

Rickesh T N

Sep 5

A 4B on a 6GB laptop matched frontier-model accuracy on aggregation — except when the answer is a number

#llm #benchmarks #localllm #agents

4 min read

Rickesh T N

Sep 5

Your agent truncates the corpus and answers anyway. Two harnesses, and a router that picks between them.

#llm #agents #localllm #benchmarks

5 min read

Peremptory

Aug 21

Frontier Models Hit a Wall on Research Thinking

#research #benchmarks #aidevelopment

2 min read

Seth Wheeler

Aug 18

Ranking Language Models by How Well They Spot Liars

#llm #measurement #benchmarks #statistics

9 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.