OpenAI Models Dominate Structured Code Edit Benchmark(blog.mentat.ai)5 points·by biobootloader·3 ปีที่แล้ว·1 commentsblog.mentat.aiOpenAI Models Dominate Structured Code Edit Benchmarkhttps://blog.mentat.ai/there-is-only-one-model1 commentsPost comment[–]granawkins·3 ปีที่แล้วreplyI saw the same thing with Mailogy.I only tested variants of gpt-3.5 and -4 but got ~50% invalid syntax errors with 3.5, and virtually none with 4.
I only tested variants of gpt-3.5 and -4 but got ~50% invalid syntax errors with 3.5, and virtually none with 4.