Very insightful! I found the section on killing papers to be a helpful reminder. As a Ph.D. student, this can be particularly challenging as your environment expects somewhat steady progress (annual reviews, advisor meetings, etc.), and you're encouraged to finish papers rather than starting over.
Very interesting topic. I also wonder why other signs of AI writing, such as negative parallelism ("It's not just X, it's Y"), are preferred by the models.
Also, I wrote a small extension that automatically replaces ChatGPT responses with em dashes with alternative punctuation marks: https://github.com/nckclrk/rm-em-dashes
I agree, it kind of reminds me of this paper that shows LLMs, just like humans, will preferentially remember information, which can lead to biased outputs.
The best way to identify the values LLMs hold is not to give them a survey with questions of the form "I believe in the value of justice. Agree or disagree?"
Instead you need to present scenarios which put these beliefs into effect. This is similar to how humans may claim to value sustainability but that doesn't mean when making a purchasing decision they will always opt for an eco-friendly option.
I'm interested if this might be a more robust method to resolve conflicting information across retrieved documents. Instead of having the LLM reason over which source to trust, the solution is to incorporate the knowledge into parametric memory and see how this combination of perspectives produces a final response.
This might also be of interest: https://karpathy.github.io/2016/09/07/phd/