All posts

The Most Useful AI Test Might Be a Tuesday

OpenAI’s LifeVal asks a useful question: can AI help with the ordinary tasks that make up a day? The scores are interesting—but they need an asterisk.

The easiest way to talk about AI progress is with a scoreboard: more answers right, fewer seconds, higher benchmark scores. But if the job is helping someone get through an ordinary Tuesday, the scoreboard is only a proxy. The real question is whether a person can move from “I need help” to a safe, useful next step.

That is the idea behind OpenAI’s November 2 announcement of LifeVal, an evaluation built around practical everyday tasks. The company says it drew on the American Time Use Survey and the Multinational Time Use Study, then added a demographically representative survey of 3,995 respondents to identify where people wanted help. The listed areas include caregiving, health, education, household administration, personal finances, shopping, and government services. OpenAI’s LifeVal announcement says its latest models score highly on the example workflows it shows, with displayed scores ranging from 90.4% to 100%.

A recipe is not dinner

Imagine asking for help with dinner. A polished recipe is nice. But if it ignores who is eating, what is already in the fridge, the budget, or how much time is left, it has not helped much. A useful assistant has to keep the constraints together, turn them into an actionable plan, maybe find what is missing, and leave the human able to check decisions before they matter. The value is in the whole route, not just the first clever sentence.

A better question than “How smart is it?”

LifeVal points at a real gap in typical tests: knowledge and fluent conversation are not the same as completing a messy task. A kindergarten search after a move involves location, family priorities, and enrollment details. A meal plan after a diagnosis calls for caution, usable choices, and a clear point where professional advice matters. The examples OpenAI publishes make evaluation feel closer to the user’s goal.

To me, the useful unit is not a prompt or an answer but a task with constraints and an outcome someone can inspect. That changes what “better” means. A model that can name the right steps is not necessarily the one that helps a person complete them.

Keep the asterisk next to the score

High numbers are interesting, but they need careful reading. These are tasks selected for OpenAI’s evaluation and results the company reports about its own models; they are not independent confirmation that AI can reliably manage the same problem for every family, in every country, with real consequences. A percentage on the page cannot, on its own, tell me whether a workflow stays reliable when details change, whether the answer is safe, or whether people complete the task outside test conditions.

What I would want the next test to show

Tests that matter to ordinary people should say more than whether the final answer sounds plausible. Did the assistant respect the budget, location, timing, and accessibility constraints? Did it flag uncertainty? Did it stop for a human decision before spending money, sharing information, or giving high-stakes advice? Could another evaluator review the steps, not just the final prose? These are not complaints about LifeVal; they are the questions I would want to see alongside any score.

That is why I like the direction of LifeVal, with an asterisk. It asks us to measure AI against the chores and decisions that take up real life, not only clever questions in a test set. The next challenge is showing that a good result on a test task stays useful when life adds the missing detail. If AI can make the next step easier without hiding the hard choices, that is the sort of progress I care about. Anyway, a good Tuesday is a better demo than a perfect-sounding answer.