@Talor_A Huh! I ran it on pi and val town's townie agent and it has been super smart, not making mistakes, great at tool calls. Maybe it's particular good at val town's toolset? Dunno!
Notes
- @HeyGarrison @cognition to have independent benchmarks, you need a public api (or at least that makes it easier) would be cool if they released one! i'm sure it's on the roadmap
- @kettlecorn i'd imagine aws with bedrock is near instant i imagine @modal is great for this too ditto for hosted @CloudflareDev models i've been thinking about this a lot. as the ttft+tps approaches 0 and 2k respectively, the bottleneck moves to latency exciting times!
- @cognition inference api?
- @warianoguerra @ValDotTown Would be cool to see the mocked val town! In theory we should have a version of this we maintain for use cases like this, local development
- Open weight models have advantages Obviously: cost + self-hostable. Truly ours. Closed models are rented But massively overlooked: seeing the reasoning tokens 1. Feels faster. Instant feedback 2. Feels smarter. When it makes mistakes, you see why. Legibility = intelligence https://x.com/stevekrouse/status/2084399338592080025
- @jamwt congrats!!!
- @DerVonDenBergen Helpful feature request!
- Townie is now 10x cheaper (again!) Deepseek v4 flash 0731 is about as smart as Opus and Fable for most tasks, but dirt cheap The hour of Townie that cost you $100 in Fable or $50 in Opus or $10 in GLM 5.2, now costs you $1.40 in Deepseek Flash
- just had my first stacked pr moment on github and it was magical the year is 2026
- @ericwithcoffee Totally! Report back with what you find
- @nbbaier Yep
- @ericwithcoffee i haven't used it *that* extensively, so it's an overstatement probably but it's close enough that for many tasks it doesn't matter
- @nbbaier i don't think so, i have zdr and no training turned on
- @nfarina I'm currently using it in pi and openrouter, i think fireworks and together mostly, whoever has zdr and no training
- @nfarina you're missing out
- deepseek 4 flash 0731 is christmas come early 100x price reduction vs fable for close-enough-not-to-matter intelligence imo (for most coding tasks) what cost $100 now costs $1. so cool. https://x.com/stevekrouse/status/2083964123067666816
- @aidenybai programming flow state is not dead, just hibernating. it will rise again
- @HeatherRasley You can but it's 4 or 5 shots and apparently overkill
- @ankrgyl it's the job of agents to get out of our way, to surface to us ideas, tradeoffs, designs, and let us push and prod and mold them, as we learn and build our taste and unfold our creations as we go the future is not, as many suppose, for us to better specify what we want up front…
Text may be partial · Read original
- @ankrgyl it's more the opposite I think
- to test for rabies, wait ten days, see if the dog dies, like an old wives tale also an actual thing i was told in 2026 luckily, there hasn't been a case of a rabid dog in NY in 60 years so if I get rabies it'll be front page news Florida man gets rabies in prospect park
- What cost $10 on June 1, cost $0.10 on July 31 Not precisely true, but what it feels like What a time to be alive
- @rileytomasek What about duplicated abstractions vs using an existing pattern? Seems like that also matters
- @glcst why are they a bad idea? seems like they make us more secure
- @davidmeh We have a Devin agent that takes a first look, but it's mostly a lot of hard work
- since april, we've been inundated with bug bounty reports at val town – some very valid, and many more total slop https://x.com/stevekrouse/status/2083598708537454753/photo/1
- I love reading the reasoning traces from deepseek flash Seems crazy to go back to frontier models that do all this work & thinking that you have no ability to inspect
- @hipotluck Not by myself. I'd need access to AI But that's a benchmark computers have long been winning at!
- @MilksandMatcha wait is sol on cerebras out yet???
- going to be in sf weds & thurs next week - who's around??
- you're thinking about evals, i'm thinking about eval()
- @CharlieMolthrop @benadam11 the original name! i think is still own the domain
- val town + chatgpt plugin = 🔥 https://x.com/stevekrouse/status/2083253031596912941/video/1
- @__drewface thanks! kinda a fun game
- After calling out frontier models as dumb, @CharlieMolthrop challenged me to come up with an eval to prove it Here's a very minimal one: "What's the smartest deepseek model?" Both Claude Code Fable 5 Max and ChatGPT Sol 5.6 Extra High got the answer wrong. Fable 5 even doubled down on its wrongness after I gave it a hint! (Photo evidence in next tweet) Seems like LLMs are bad at searching around for stuff that happened yesterday and disambiguating that against historical info
- https://x.com/stevekrouse/status/2083245630034526631/photo/1
- @__drewface Pedantic is good. There should be a Turing Test Rule book! 1. I dunno, would have to play a bunch to see 2. It should be decisive like chess, go, etc 3. n/a 4. Defeat the best human 5. don't understand 6. Text seems simplest, like normal LLM chat, web app is fine 7. Double blind…
Text may be partial · Read original
- @__drewface I think the point at which models win forever is AGI
- @__drewface I think clearing the bar of fooling me is equivalent to AGI indistinguishable from human intelligence

