EMNLP 2023, NeurIPS 2023 Primer

구글 리서쳐 Sebastian Ruder 가 정리한 EMNLP와 NeurIPS 학회 내용입니다. 현재의 어느정도 검증된 LLM 트렌드에 대해서 알 수 있어서 좋을것 같습니다 :) ## EMNLP 2023 1. Instruction-tuned LMs and LLMs are everywhere. Similar to earlier years where BERT was ubiquitous, instruction-tuned language models (LMs) and large language models (LLMs) are used in almost every paper. 2. Evaluation based on LLMs is increasingly common. While some papers employ automatic evaluation based on GPT-4, new metrics that are proposed are based on LLMs in zero-shot prompted or fine-tuned settings. 3. Prompt usage is getting more creative. Beyond a standard prompt template, prompts are getting increasingly complex and specialized to the desired setting. Techniques such as chain-of-thought prompting are common tools. 4. Multilinguality is increasingly popular. I came across a substantial number of papers studying multilingual settings, which indicates that LLMs are still limited in non-English settings and that making LLMs more multilingual is an important direction. EMNLP 에 대한 자세한 내용은 아래를 참고 부탁드립니다. https://nlpnewsletter.substack.com/p/emnlp-2023-primer?utm_source=post-email-title&publication_id=1178062&post_id=139296734&utm_campaign=email-post-title&isFreemail=true&r=1ebg8s&utm_medium=email ## NeurIPS 2023 1. Most NLP work at NeurIPS is related to large language models (LLMs). While there are some papers that do not employ LLMs or use a different setting (see Suhr & Artzi below, for instance), papers still presented their contributions in the context of LLMs. 2. Synthetic setups to analyze LLM properties are becoming more common. This is because it is computationally prohibitive to run many different pre-training experiments. Investigated properties range from the emergence of in-context learning and learning using global statistics to chain-of-thought reasoning. 3. Aligning models based on human preferences received a lot of attention. Papers particularly focused on improving RLHF and studying alignment to specific personality traits and beliefs. 4. A comprehensive understanding of in-context learning still remains elusive. Papers studied different aspects of in-context learning such as whether it persists during training and using a Bayesian perspective. 5. Reasoning is still challenging with current models. Papers focused on improving performance on various types of reasoning tasks including pragmatic, graph-based, algorithmic, compositional, and planning-based reasoning. 6. External tools are increasingly used to improve LLMs’ reasoning abilities. These range from external verifiers to code execution modules. NeurIPS 에 대한 자세한 내용은 아래를 참고 부탁드립니다. https://nlpnewsletter.substack.com/p/neurips-2023-primer?utm_source=post-email-title&publication_id=1178062&post_id=138865653&utm_campaign=email-post-title&isFreemail=true&r=1ebg8s&utm_medium=email