Andrej Karpathy coined the phrase in February 2025, and the definition he gave it was an act of surrender. You describe what you want, you accept what comes back, and you forget that the code even exists. The word he chose was vibes, and the whole appeal of the word is that it promises you do not have to know anything.

A team at ETH Zurich has now run the first preregistered test of what actually predicts whether you are any good at it. Sverrir Thorgeirsson, Theo Weidmann and Zhendong Su put a hundred university students in a controlled lab, gave each of them an expository writing test, a computer science achievement test, a general reasoning test, and forty five minutes on a purpose built platform where they could prompt an LLM and watch the rendered application change but could never look at the source. Then a blinded grader scored what they built.

The finding is not that vibes are enough. The finding is that the two skills we all assume are doing the work are both real, both measurable, and both much smaller than the folklore says. And the one that holds up under scrutiny is the one that has nothing to do with prose.

The design is unusually clean, so it is worth being specific about it. Participants were recruited through ETH's Decision Science Laboratory from the ETH and University of Zurich student pools. A hundred students, 56 women and 44 men, mean age 25.0. Everyone had completed at least an introductory CS course, had used LLMs for programming before, and met C1 English. Each session ran an hour and forty five minutes for 55 Swiss francs.

Writing was measured with a task built for the study by two writing instructors from the ETH and UZH language centers. Write 300 to 450 words about one technical concept from your own studies, for a college educated non expert, in twenty minutes. Two blinded graders scored each essay against a twenty item rubric covering paragraph logic, sentence clarity, cross sentence cohesion and vocabulary precision. Inter rater reliability landed at 0.731 after one round of recalibration.

CS achievement was a twelve item subset of the SCS1, a validated language independent instrument that tests definitional knowledge, code tracing and code completion in pseudocode. General reasoning was the ICAR16, sixteen items of matrix reasoning, letter series, verbal reasoning and three dimensional rotation.

The vibe coding tasks were selected by an eight expert consensus panel from a pool of candidates, scored on interdisciplinarity, authenticity, challenge, purpose clarity and cultural fairness. The three survivors were an exam scheduler, a meal planner and a course registration platform. Fifteen minutes each. The model behind the platform was Claude Sonnet 4.

Writing skill and CS achievement were essentially uncorrelated with each other in this sample. r = 0.126, p = 0.213. Control for general reasoning and it goes to almost exactly zero, at r = -0.032.

Two independent doors into the same room. The student who writes a clean paragraph and the student who can trace a while loop are, in this cohort, not the same student. Which is why both variables kept earning their keep in the joint model rather than cannibalizing each other, and why the honest curricular implication is not to pick one.

The authors ran a mediation analysis to test whether writing skill works through prompt quality. An independent expert, blind to everything else, graded each participant's full prompt sequence against a rubric designed by people who had never seen the prompt data. Writing predicted prompt quality at 0.35. Prompt quality predicted performance at 0.43. The direct path from writing to performance dropped from 0.29 to a non significant 0.14 once prompt quality entered, with the indirect effect accounting for around 52 percent of the total.

That is a tidy story and it is probably partly true. It is also the weakest link in the paper. I ran the claim set past two frontier models with authorship stripped, and both independently flagged the same thing. The prompts being graded are the prompts that produced the artifact being graded. When your mediator is causally upstream of your outcome by construction, a strong correlation between them is not news. The r = 0.479 between prompt quality and performance is the largest number in the paper and the least surprising one.

The mediation is worth reading as a description of the pathway rather than as evidence for it.

Self reported frequency of LLM use correlated negatively with vibe coding performance at r = -0.258, p = 0.010. It also correlated negatively with writing skill at r = -0.282, p = 0.005. It correlated with CS achievement at r = 0.001, which is as close to nothing as a number gets.

The authors offer three readings and decline to choose between them. LLMs are degrading students' ability to express themselves. Or weaker writers reach for LLMs more often. Or both at once. Their own exploratory work rules out the flattering explanation that heavy users were simply better at working the tool, since heavy use bought no CS advantage at all.

Every part of this is self reported on a five point Likert item in a study that was not designed to test it. It is the single most quotable number in the paper and the one I would least want to build an argument on.

Writing is probably underestimated here. Writing scores had a mean of 0.72 and a standard deviation of 0.11 on a zero to one scale. This was a C1 English cohort at one of the better technical universities in Europe. Restricted range attenuates correlations. Run the same study across a population with a real spread of writing ability and the writing coefficient very likely gets bigger.

Writing's independent effect is probably overestimated here. The instrument used to control it away, the ICAR16, had a Cronbach's alpha of 0.625, below the conventional 0.7 threshold and the least reliable of the three measures. Measurement error in a covariate means you do not fully remove its variance, so the partial correlation of 0.186 retains some of the confound it was supposed to strip out. The true independent contribution of writing may be lower still.

Those two do not cancel. They mean the interval around writing is wider than one decimal place of a correlation makes it look.

On the writing side, partly, and with a distinction that matters. Chantel Prat's group at Washington found in 2020 that natural language aptitude was the strongest single predictor of how fast adults learned Python, with numeracy explaining an average of 2 percent. Follow up work from the same lab found that two language aptitude subskills predicted Python and Java acquisition even after controlling for fluid intelligence, which is the control that the Zurich writing measure did not survive.

The distinction is that Prat measured aptitude for acquiring a language. Thorgeirsson measured the ability to produce clear expository prose. Those are cousins, not twins, and the second is the one that vibe coding actually asks you to perform.

On the CS side the corroboration is stronger and more specific. Lucchetti and colleagues, in a 2025 NAACL paper with the excellent title Substance Beats Style, tested two competing explanations for why beginners fail at prompting. Hypothesis one was that they lack technical vocabulary. Hypothesis two was that they do not understand how much information the model needs. They ran a causal intervention on the vocabulary and found it barely mattered. Substituting imprecise terms for precise ones had little effect. What predicted success was the information content of the prompt.

Which is a finding about knowing what to say, not about knowing how to say it. It sits squarely on the CS achievement side of the ledger, and it is the closest thing in this literature to an independent replication of the Zurich result.

None of this speaks to whether the workflow is any good, and the best evidence there is genuinely unsettled. METR's randomized trial of experienced open source developers, run February to June 2025, found that allowing AI tools made them 19 percent slower on their own repositories, while the same developers estimated afterward that they had been sped up by 20 percent.

METR ran it again with late 2025 tooling and, in February 2026, published a post explaining why they are abandoning the design. Developers refused to participate in the no AI arm. Between 30 and 50 percent of those who did participate admitted to withholding tasks they did not want to be assigned to do by hand. Their raw estimate flipped to a speedup, around 18 percent for the returning developers and 4 percent for new recruits, with confidence intervals crossing zero in both cases. They consider their own number a lower bound and no longer trust the instrument.

So the honest state of play is that we now have a decent estimate of which skills predict individual variation in this workflow, and a poor one of whether the workflow beats the alternative. The Zurich study tells you who will be better at it than their classmate. It does not tell you whether any of them should be doing it.

Karpathy's formulation involved forgetting that the code even exists. Google's own documentation now distinguishes pure vibe coding, where you never look, from the version where you read what came back and refine it. This study deliberately measured the pure form, because it is the only version with a clean boundary around it.

And under those conditions, the strongest individual predictor was formal training in how machines represent and move through a problem, worth roughly double the writing measure. The preregistered hypothesis that writing would matter most on the task where the model had no context to fall back on did not hold, at 0.239 against 0.245 for the context rich comparison, which is a null result the authors report plainly and to their credit.

What the study measures is the capacity to specify a system completely to something that shares none of your assumptions. That is requirements engineering with a faster feedback loop. The IEEE standard for it is forty years old and says requirements should be stated simply and be easy to understand. The NASA systems engineering handbook has an appendix on how to write a good requirement that warns against indefinite pronouns.

Karpathy's word describes the part of the process you are allowed to skip. It does not describe the part that determines whether you succeed.