Trending topics
#
Bonk Eco continues to show strength amid $USELESS rally
#
Pump.fun to raise $1B token sale, traders speculating on airdrop
Yaron (Ron) Minsky
Occasional OCaml programmer. Host of @signalsthreads.
@yminsky@discuss.systems
@yminsky.bsky.social
https://t.co/kiUGRvWOO2
Awesome that they did this study. It demonstrates things that anyone using the models for serious engineering work could see, but is invisible in the public data.
Mar 11, 00:34
How do benchmarks map to real-world capabilities? To study this, we hired 4 maintainers of repos used in SWE-bench Verified to review agent code.
Of agent PRs that passed SWE-bench’s grader, maintainers would merge ~half. This holds accounting for noise in maintainer decisions.
118
Top
Ranking
Favorites