For me, the paper contained many gems. Besides the superficial alignment hypothesis and its consequences on the fine-tuning dataset, Figure 7 about instruction alignment vs conversation alignment and Figure 9 about the positive correlation of the perplexity number with the quality score (i.e. negative correlation of the perplexity based model quality and with the response based quality) were very insightful.
What I missed: How does the superficial alignment hypothesis related to model size (they only investigate disjoint aspects on 7B vs 65B llama models). Since the paper focuses on data quality, I would have expected an annotation guideline.
What I missed: How does the superficial alignment hypothesis related to model size (they only investigate disjoint aspects on 7B vs 65B llama models). Since the paper focuses on data quality, I would have expected an annotation guideline.
Still, I think the paper is an excellent read.