# 📄 In-context Learning

## Paper List

<div style="line-height:0.2em;">


[**LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation**](https://arxiv.org/abs/2406.12832) （**2024.06.18**）

<font color="gray">Seyedarmin Azizi, Souvik Kundu, M. Pedram </font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**The Impact of Initialization on LoRA Finetuning Dynamics**](https://arxiv.org/abs/2406.08447) （**2024.06.12**）

<font color="gray">Soufiane Hayou, Nikhil Ghosh, Bin Yu </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-3-red)

---

[**An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models**](https://arxiv.org/abs/2406.05130) （**2024.06.07**）

<font color="gray">Xiongtao Zhou, Jie He, Yuhua Ke, Guangyao Zhu, V'ictor Guti'errez-Basulto, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-1-red)

---

[**Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning**](https://arxiv.org/abs/2406.02547) （**2024.06.04**）

<font color="gray">Alex Jinpeng Wang, Linjie Li, Yiqi Lin, Min Li, Lijuan Wang, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-6-blue)](https://github.com/showlab/VisInContext)

---

[**Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks**](https://arxiv.org/abs/2406.02550) （**2024.06.04**）

<font color="gray">Tianyu He, Darshil Doshi, Aritra Das, Andrey Gromov </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-2-red)  [![](https://img.shields.io/badge/Github%20Stars-3-blue)](https://github.com/ablghtianyi/ICL_Modular_Arithmetic)

---

[**Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models**](https://doi.org/10.48550/arXiv.2405.17915) （**2024.05.28**）

<font color="gray">Longze Chen, Ziqiang Liu, Wanwei He, Yunshui Li, Run Luo, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-1-green)  [![](https://img.shields.io/badge/Github%20Stars-39-blue)](https://github.com/October2001/ProLong)

---

[**Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion**](https://doi.org/10.48550/arXiv.2405.11464) （**2024.05.19**）

<font color="gray">Pengxiang Lan, Enneng Yang, Yuting Liu, Guibing Guo, Linying Jiang, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning**](https://doi.org/10.48550/arXiv.2405.11446) （**2024.05.19**）

<font color="gray">Sanchit Sinha, Yuguang Yue, Victor Soto, Mayank Kulkarni, Jianhua Lu, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning**](https://doi.org/10.48550/arXiv.2404.16807) （**2024.04.25**）

<font color="gray">Tianhui Zhang, Bei Peng, D. Bollegala .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-1-green)

---

[**Stronger Random Baselines for In-Context Learning**](https://doi.org/10.48550/arXiv.2404.13020) （**2024.04.19**）

<font color="gray">Gregory Yauney, David M. Mimno .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-1-blue)](https://github.com/gyauney/max-random-baseline)

---

[**Towards Reliable Latent Knowledge Estimation in LLMs: In-Context Learning vs. Prompting Based Factual Knowledge Extraction**](https://doi.org/10.48550/arXiv.2404.12957) （**2024.04.19**）

<font color="gray">Qinyuan Wu, Mohammad Aflah Khan, Soumi Das, Vedant Nanda, Bishwamittra Ghosh, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-1-green)

---

[**Point-In-Context: Understanding Point Cloud via In-Context Learning**](https://doi.org/10.48550/arXiv.2404.12352) （**2024.04.18**）

<font color="gray">Mengyuan Liu, Zhongbin Fang, Xia Li, Joachim Buhmann, Xiangtai Li, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-61-blue)](https://github.com/fanglaosi/point-in-context)

---

[**AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models**](https://arxiv.org/abs/2403.13269) （**2024.03.20**）

<font color="gray">Zeyu Liu, Souvik Kundu, Anni Li, Junrui Wan, Lianghao Jiang, etc </font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Towards Multimodal In-Context Learning for Vision & Language Models**](https://doi.org/10.48550/arXiv.2403.12736) （**2024.03.19**）

<font color="gray">Sivan Doveh, Shaked Perek, M. J. Mirza, Amit Alfassy, Assaf Arbelle, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-3-green)

---

[**ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models**](https://arxiv.org/abs/2403.09583) （**2024.03.14**）

<font color="gray">Runyu Ma, Jelle Luijkx, Zlatan Ajanovic, Jens Kober </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-6-red)

---

[**Attention Prompt Tuning: Parameter-efficient Adaptation of Pre-trained Models for Spatiotemporal Modeling**](https://arxiv.org/abs/2403.06978) （**2024.03.11**）

<font color="gray">W. G. C. Bandara, Vishal M. Patel </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-2-red)  [![](https://img.shields.io/badge/Github%20Stars-11-blue)](https://github.com/wgcban/apt)

---

[**Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought**](https://arxiv.org/abs/2403.05518) （**2024.03.08**）

<font color="gray">James Chua, Edward Rees, Hunar Batra, Samuel R. Bowman, Julian Michael, etc </font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Securing Reliability: A Brief Overview on Enhancing In-Context Learning for Foundation Models**](https://arxiv.org/abs/2402.17671) （**2024.02.27**）

<font color="gray">Yunpeng Huang, Yaonan Gu, Jingwei Xu, Zhihong Zhu, Zhaorun Chen, etc </font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**GISTEmbed: Guided In-sample Selection of Training Negatives for Text Embedding Fine-tuning**](https://doi.org/10.48550/arXiv.2402.16829) （**2024.02.26**）

<font color="gray">Aivin V. Solatorio .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-1-green)  [![](https://img.shields.io/badge/Github%20Stars-27-blue)](https://github.com/avsolatorio/gistembed)

---

[**DiffuCOMET: Contextual Commonsense Knowledge Diffusion**](https://arxiv.org/abs/2402.17011) （**2024.02.26**）

<font color="gray">Silin Gao, Mete Ismayilzada, Mengjie Zhao, Hiromi Wakaki, Yuki Mitsufuji, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-2-red)  [![](https://img.shields.io/badge/Github%20Stars-2-blue)](https://github.com/silin159/diffucomet)

---

[**Long-Context Language Modeling with Parallel Context Encoding**](https://arxiv.org/abs/2402.16617) （**2024.02.26**）

<font color="gray">Howard Yen, Tianyu Gao, Danqi Chen </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-15-red)  [![](https://img.shields.io/badge/Github%20Stars-116-blue)](https://github.com/princeton-nlp/cepe)

---

[**Training Nonlinear Transformers for Efficient In-Context Learning: A Theoretical Learning and Generalization Analysis**](https://arxiv.org/abs/2402.15607) （**2024.02.23**）

<font color="gray">Hongkang Li, Meng Wang, Songtao Lu, Xiaodong Cui, Pin-Yu Chen </font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Models**](https://arxiv.org/abs/2402.15637) （**2024.02.23**）

<font color="gray">Yanzheng Xiang, Hanqi Yan, Lin Gui, Yulan He </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-2-red)  [![](https://img.shields.io/badge/Github%20Stars-3-blue)](https://github.com/xyzcs/infoac)

---

[**In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization**](https://arxiv.org/abs/2402.14951) （**2024.02.22**）

<font color="gray">Ruiqi Zhang, Jingfeng Wu, Peter L. Bartlett </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-1-red)

---

[**Unlocking Instructive In-Context Learning with Tabular Prompting for Relational Triple Extraction**](https://arxiv.org/abs/2402.13741) （**2024.02.21**）

<font color="gray">Guozheng Li, Wenjun Ke, Peng Wang, Zijie Xu, Ke Ji, etc </font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive**](https://arxiv.org/abs/2402.13228) （**2024.02.20**）

<font color="gray">Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-25-red)  [![](https://img.shields.io/badge/Github%20Stars-45-blue)](https://github.com/abacusai/smaug)

---

[**Feedback Loops With Language Models Drive In-Context Reward Hacking**](https://doi.org/10.48550/arXiv.2402.06627) （**2024.02.09**）

<font color="gray">Alexander Pan, Erik Jones, Meena Jagadeesan, Jacob Steinhardt .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-5-blue)](https://github.com/aypan17/llm-feedback)

---

[**On the Convergence of Zeroth-Order Federated Tuning in Large Language Models**](https://doi.org/10.48550/arXiv.2402.05926) （**2024.02.08**）

<font color="gray">Zhenqing Ling, Daoyuan Chen, Liuyi Yao, Yaliang Li, Ying Shen .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**EmojiCrypt: Prompt Encryption for Secure Communication with Large Language Models**](https://doi.org/10.48550/arXiv.2402.05868) （**2024.02.08**）

<font color="gray">Guo Lin, Wenyue Hua, Yongfeng Zhang .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-12-blue)](https://github.com/agiresearch/emojicrypt)

---

[**Large Language Model Meets Graph Neural Network in Knowledge Distillation**](https://doi.org/10.48550/arXiv.2402.05894) （**2024.02.08**）

<font color="gray">Shengxiang Hu, Guobing Zou, Song Yang, Yanglan Gan, Bofeng Zhang, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model**](https://arxiv.org/abs/2312.11370) （**2023.12.18**）

<font color="gray">Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-12-red)  [![](https://img.shields.io/badge/Github%20Stars-106-blue)](https://github.com/pipilurj/g-llava)

---

[**A mathematical perspective on Transformers**](https://arxiv.org/abs/2312.10794) （**2023.12.17**）

<font color="gray">Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, Philippe Rigollet </font>

![](https://img.shields.io/badge/Citations-1-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-42-red)  [![](https://img.shields.io/badge/Github%20Stars-23-blue)](https://github.com/borjang/2023-transformers-rotf)

---

[**Mitigating Label Bias in Machine Learning: Fairness through Confident Learning**](https://doi.org/10.48550/arXiv.2312.08749) （**2023.12.14**）

<font color="gray">Yixuan Zhang, Boyu Li, Zenan Ling, Feng Zhou .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Control Risk for Potential Misuse of Artificial Intelligence in Science**](https://arxiv.org/abs/2312.06632) （**2023.12.11**）

<font color="gray">Jiyan He, Weitao Feng, Yaosen Min, Jingwei Yi, Kunsheng Tang, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-7-red)  [![](https://img.shields.io/badge/Github%20Stars-11-blue)](https://github.com/scimt/scimt-benchmark)

---

[**WonderJourney: Going from Anywhere to Everywhere**](https://arxiv.org/abs/2312.03884) （**2023.12.06**）

<font color="gray">Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-31-red)

---

[**Minimizing Factual Inconsistency and Hallucination in Large Language Models**](https://doi.org/10.48550/arXiv.2311.13878) （**2023.11.23**）

<font color="gray">I. Muneeswaran, Shreya Saxena, Siva Prasad, M. V. S. Prakash, Advaith Shankar, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents**](https://doi.org/10.48550/arXiv.2311.11797) （**2023.11.20**）

<font color="gray">Zhuosheng Zhang, Yao Yao, Aston Zhang, Xiangru Tang, Xinbei Ma, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-316-blue)](https://github.com/zoeyyao27/cot-igniting-agent)

---

[**An Embodied Generalist Agent in 3D World**](https://doi.org/10.48550/arXiv.2311.12871) （**2023.11.18**）

<font color="gray">Jiangyong Huang, Silong Yong, Xiaojian Ma, Xiongkun Linghu, Puhao Li, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-1-green)  [![](https://img.shields.io/badge/Github%20Stars-282-blue)](https://github.com/embodied-generalist/embodied-generalist)

---

[**MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning**](https://doi.org/10.48550/arXiv.2311.10537) （**2023.11.16**）

<font color="gray">Xiangru Tang, Anni Zou, Zhuosheng Zhang, Yilun Zhao, Xingyao Zhang, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-188-blue)](https://github.com/gersteinlab/medagents)

---

[**Towards Verifiable Text Generation with Symbolic References**](https://doi.org/10.48550/arXiv.2311.09188) （**2023.11.15**）

<font color="gray">Lucas Torroba Hennigen, Zejiang Shen, Aniruddha Nrusimha, Bernhard Gapp, David Sontag, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Learning skillful medium-range global weather forecasting.**](https://doi.org/10.1126/science.adi2336) （**2023.11.14**）

<font color="gray">Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, etc .  - 【Science】</font>

![](https://img.shields.io/badge/Citations-8-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-549-red)

---

[**u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model**](https://doi.org/10.48550/arXiv.2311.05348) （**2023.11.09**）

<font color="gray">Jinjin Xu, Liwu Xu, Yuzhe Yang, Xiang Li, Yanchun Xie, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-2-green)  [![](https://img.shields.io/badge/Github%20Stars-120-blue)](https://github.com/OPPOMKLab/u-LLaVA)

---

[**Levels of AGI: Operationalizing Progress on the Path to AGI**](https://doi.org/10.48550/arXiv.2311.02462) （**2023.11.04**）

<font color="gray">Meredith Ringel Morris, Jascha Narain Sohl-Dickstein, Noah Fiedel, T. Warkentin, Allan Dafoe, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-1-green)

---

[**Which Examples to Annotate for In-Context Learning? Towards Effective and Efficient Selection**](https://doi.org/10.48550/arXiv.2310.20046) （**2023.10.30**）

<font color="gray">Costas Mavromatis, Balasubramaniam Srinivasan, Zhengyuan Shen, Jiani Zhang, H. Rangwala, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-5-green)  [![](https://img.shields.io/badge/Github%20Stars-15-blue)](https://github.com/amazon-science/adaptive-in-context-learning)

---

[**CodeFusion: A Pre-trained Diffusion Model for Code Generation**](https://arxiv.org/abs/2310.17680) （**2023.10.26**）

<font color="gray">Mukul Singh, J. Cambronero, Sumit Gulwani, Vu Le, Carina Negreanu, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-39-red)

---

[**SuperHF: Supervised Iterative Learning from Human Feedback**](https://arxiv.org/abs/2310.16763) （**2023.10.25**）

<font color="gray">Gabriel Mukobi, Peter Chatain, Su Fong, Robert Windesheim, Gitta Kutyniok, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-6-red)  [![](https://img.shields.io/badge/Github%20Stars-9-blue)](https://github.com/openfeedback/superhf)

---

[**In-Context Learning Creates Task Vectors**](https://doi.org/10.48550/arXiv.2310.15916) （**2023.10.24**）

<font color="gray">Roee Hendel, Mor Geva, Amir Globerson .  - 【Conference on Empirical Methods in Natural Language Processing】</font>

![](https://img.shields.io/badge/Citations-25-green)

---

[**Woodpecker: Hallucination Correction for Multimodal Large Language Models**](https://arxiv.org/abs/2310.16045) （**2023.10.24**）

<font color="gray">Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-47-red)  [![](https://img.shields.io/badge/Github%20Stars-572-blue)](https://github.com/bradyfu/woodpecker)

---

[**In-context Learning with Transformer Is Really Equivalent to a Contrastive Learning Pattern**](https://doi.org/10.48550/arXiv.2310.13220) （**2023.10.20**）

<font color="gray">Ruifeng Ren, Yong Liu .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**MemGPT: Towards LLMs as Operating Systems**](https://arxiv.org/abs/2310.08560) （**2023.10.12**）

<font color="gray">Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-120-red)  [![](https://img.shields.io/badge/Github%20Stars-10.8k-blue)](https://github.com/cpacker/memgpt)

---

[**LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models**](https://doi.org/10.48550/arXiv.2309.12307) （**2023.09.21**）

<font color="gray">Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-2.5k-blue)](https://github.com/dvlab-research/longlora)

---

[**Adapting Large Language Models via Reading Comprehension**](https://doi.org/10.48550/arXiv.2309.09530) （**2023.09.18**）

<font color="gray">Daixuan Cheng, Shaohan Huang, Furu Wei .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-3.4k-blue)](https://github.com/microsoft/lmops)

---

[**Giraffe: Adventures in Expanding Context Lengths in LLMs**](https://doi.org/10.48550/arXiv.2308.10882) （**2023.08.21**）

<font color="gray">Arka Pal, Deep Karkhanis, Manley Roberts, S. Dooley, Arvind Sundararajan, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-567-blue)](https://github.com/abacusai/long-context)

---

[**Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval**](https://doi.org/10.48550/arXiv.2308.07648) （**2023.08.15**）

<font color="gray">Chaorui Deng, Qi Chen, Pengda Qin, Dave Zhenyu Chen, Qi Wu .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-24-blue)](https://github.com/bladewaltz1/promptswitch)

---

[**Exploring the Intersection of Large Language Models and Agent-Based Modeling via Prompt Engineering**](https://doi.org/10.48550/arXiv.2308.07411) （**2023.08.14**）

<font color="gray">Edward Junprung .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**PromptCARE: Prompt Copyright Protection by Watermark Injection and Verification**](https://doi.org/10.48550/arXiv.2308.02816) （**2023.08.05**）

<font color="gray">Hongwei Yao, Jian Lou, Kui Ren, Zhan Qin .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-1-green)  [![](https://img.shields.io/badge/Github%20Stars-22-blue)](https://github.com/grasses/promptcare)

---

[**Learning to Retrieve In-Context Examples for Large Language Models**](https://arxiv.org/abs/2307.07164) （**2023.07.14**）

<font color="gray">Liang Wang, Nan Yang, Furu Wei </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-64-red)  [![](https://img.shields.io/badge/Github%20Stars-3.4k-blue)](https://github.com/microsoft/lmops)

---

[**Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models**](https://doi.org/10.48550/arXiv.2307.03762) （**2023.07.07**）

<font color="gray">Yuxi Ma, Chi Zhang, Song-Chun Zhu .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-3-green)

---

[**Understanding In-Context Learning via Supportive Pretraining Data**](https://doi.org/10.48550/arXiv.2306.15091) （**2023.06.26**）

<font color="gray">Xiaochuang Han, Daniel Simig, Todor Mihaylov, Yulia Tsvetkov, Asli Celikyilmaz, etc .  - 【Annual Meeting of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-8-green)

---

[**Schema-learning and rebinding as mechanisms of in-context learning and emergence**](https://doi.org/10.48550/arXiv.2307.01201) （**2023.06.16**）

<font color="gray">Siva K. Swaminathan, A. Dedieu, Rajkumar Vasudeva Raju, M. Shanahan, M. Lázaro-Gredilla, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models**](https://doi.org/10.48550/arXiv.2306.01311) （**2023.06.02**）

<font color="gray">Masoud Monajatipoor, Liunian Harold Li, Mozhdeh Rouhsedaghat, Lin F. Yang, Kai-Wei Chang .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Measuring and Mitigating Constraint Violations of In-Context Learning for Utterance-to-API Semantic Parsing**](https://arxiv.org/abs/2305.15338) （**2023.05.24**）

<font color="gray">Shufan Wang, Sebastien Jean, Sailik Sengupta, James Gung, Nikolaos Pappas, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-8-red)

---

[**OverPrompt: Enhancing ChatGPT Capabilities through an Efficient In-Context Learning Approach**](https://arxiv.org/abs/2305.14973) （**2023.05.24**）

<font color="gray">Jiazheng Li, Runcong Zhao, Yulan He, Lin Gui </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-1-red)

---

[**Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations**](https://doi.org/10.48550/arXiv.2305.15035) （**2023.05.24**）

<font color="gray">Wei-Lin Chen, Cheng-Kuang Wu, Hsin-Hsi Chen .  - 【Conference on Empirical Methods in Natural Language Processing】</font>

![](https://img.shields.io/badge/Citations-3-green)

---

[**Adversarial Demonstration Attacks on Large Language Models**](https://arxiv.org/abs/2305.14950) （**2023.05.24**）

<font color="gray">Jiongxiao Wang, Zichen Liu, Keun Hee Park, Muhao Chen, Chaowei Xiao </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-19-red)

---

[**Frugal Prompting for Dialog Models**](https://arxiv.org/abs/2305.14919) （**2023.05.24**）

<font color="gray">Bishal Santra, Sakya Basak, Abhinandan De, Manish Gupta, Pawan Goyal </font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-1-blue)](https://github.com/bsantraigi/frugal-prompting)

---

[**Coverage-based Example Selection for In-Context Learning**](https://arxiv.org/abs/2305.14907) （**2023.05.24**）

<font color="gray">Shivanshu Gupta, Sameer Singh, Matt Gardner </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-13-red)  [![](https://img.shields.io/badge/Github%20Stars-10-blue)](https://github.com/shivanshu-gupta/icl-coverage)

---

[**SummIt: Iterative Text Summarization via ChatGPT**](https://arxiv.org/abs/2305.14835) （**2023.05.24**）

<font color="gray">Haopeng Zhang, Xiao Liu, Jiawei Zhang </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-42-red)  [![](https://img.shields.io/badge/Github%20Stars-2-blue)](https://github.com/hpzhang94/summ_it)

---

[**Exploring Diverse In-Context Configurations for Image Captioning**](https://arxiv.org/abs/2305.14800) （**2023.05.24**）

<font color="gray">Xu Yang, Yongliang Wu, Mingzhuo Yang, Haokun Chen </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-14-red)  [![](https://img.shields.io/badge/Github%20Stars-27-blue)](https://github.com/yongliang-wu/explorecfg)

---

[**Mastering the ABCDs of Complex Questions: Answer-Based Claim Decomposition for Fine-grained Self-Evaluation**](https://arxiv.org/abs/2305.14750) （**2023.05.24**）

<font color="gray">Nishant Balepur, Jie Huang, Samraj Moorjani, Hari Sundaram, Kevin Chen-Chuan Chang </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-5-red)

---

[**In-Context Demonstration Selection with Cross Entropy Difference**](https://arxiv.org/abs/2305.14726) （**2023.05.24**）

<font color="gray">Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-10-red)  [![](https://img.shields.io/badge/Github%20Stars-3.4k-blue)](https://github.com/microsoft/lmops)

---

[**ExpertPrompting: Instructing Large Language Models to be Distinguished Experts**](https://arxiv.org/abs/2305.14688) （**2023.05.24**）

<font color="gray">Benfeng Xu, An Yang, Junyang Lin, Quang Wang, Chang Zhou, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-36-red)  [![](https://img.shields.io/badge/Github%20Stars-292-blue)](https://github.com/ofa-sys/expertllama)

---

[**Active Learning Principles for In-Context Learning with Large Language Models**](https://arxiv.org/abs/2305.14264) （**2023.05.23**）

<font color="gray">Katerina Margatina, Timo Schick, Nikolaos Aletras, Jane Dwivedi-Yu </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-33-red)

---

[**Skill-Based Few-Shot Selection for In-Context Learning**](https://arxiv.org/abs/2305.14210) （**2023.05.23**）

<font color="gray">Shengnan An, Bo Zhou, Zeqi Lin, Qiang Fu, Bei Chen, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-20-red)

---

[**Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning**](https://arxiv.org/abs/2305.14160) （**2023.05.23**）

<font color="gray">Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-65-red)  [![](https://img.shields.io/badge/Github%20Stars-127-blue)](https://github.com/lancopku/label-words-are-anchors)

---

[**Make a Choice! Knowledge Base Question Answering with In-Context Learning**](https://arxiv.org/abs/2305.13972) （**2023.05.23**）

<font color="gray">Chuanyuan Tan, Yuehe Chen, Wenbiao Shao, Wenliang Chen </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-12-red)

---

[**Concept-aware Training Improves In-context Learning Ability of Language Models**](https://arxiv.org/abs/2305.13775) （**2023.05.23**）

<font color="gray">Michal vStef'anik, Marek Kadlvc'ik </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-4-red)

---

[**RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learning**](https://arxiv.org/abs/2305.14502) （**2023.05.23**）

<font color="gray">Alexander Scarlatos, Andrew Lan </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-8-red)  [![](https://img.shields.io/badge/Github%20Stars-134-blue)](https://github.com/lupantech/promptpg)

---

[**Can We Edit Factual Knowledge by In-Context Learning?**](https://arxiv.org/abs/2305.12740) （**2023.05.22**）

<font color="gray">Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-33-red)  [![](https://img.shields.io/badge/Github%20Stars-37-blue)](https://github.com/zce1112zslx/ike)

---

[**Iterative Forward Tuning Boosts In-context Learning in Language Models**](https://arxiv.org/abs/2305.13016) （**2023.05.22**）

<font color="gray">Jiaxi Yang, Binyuan Hui, Min Yang, Binhua Li, Fei Huang, etc </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-17-red)

---

[**Explaining How Transformers Use Context to Build Predictions**](https://arxiv.org/abs/2305.12535) （**2023.05.21**）

<font color="gray">Javier Ferrando, Gerard I. Gállego, Ioannis Tsiamas, M. Costa-jussà </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-22-red)  [![](https://img.shields.io/badge/Github%20Stars-10-blue)](https://github.com/mt-upc/logit-explanations)

---

[**Few-Shot Dialogue Summarization via Skeleton-Assisted Prompt Transfer**](https://arxiv.org/abs/2305.12077) （**2023.05.20**）

<font color="gray">Kaige Xie, Tong Yu, Haoliang Wang, Junda Wu, Handong Zhao, etc </font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning**](https://arxiv.org/abs/2305.12295) （**2023.05.20**）

<font color="gray">Liangming Pan, Alon Albalak, Xinyi Wang, William Yang Wang </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-74-red)  [![](https://img.shields.io/badge/Github%20Stars-201-blue)](https://github.com/teacherpeterpan/logic-llm)

---

[**RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thought**](https://doi.org/10.48550/arXiv.2305.11499) （**2023.05.19**）

<font color="gray">Tianci Xue, Ziqi Wang, Zhenhailong Wang, Chi Han, Pengfei Yu, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Do Models Really Learn to Follow Instructions? An Empirical Study of Instruction Tuning**](https://doi.org/10.48550/arXiv.2305.11383) （**2023.05.19**）

<font color="gray">Po-Nien Kung, Nanyun Peng .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**AutoTrial: Prompting Language Models for Clinical Trial Design**](https://doi.org/10.48550/arXiv.2305.11366) （**2023.05.19**）

<font color="gray">Zifeng Wang, Cao Xiao, Jimeng Sun .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Efficient Prompting via Dynamic In-Context Learning**](https://doi.org/10.48550/arXiv.2305.11170) （**2023.05.18**）

<font color="gray">Wangchunshu Zhou, Yuchen Jiang, Ryan Cotterell, Mrinmaya Sachan .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Generalized Planning in PDDL Domains with Pretrained Large Language Models**](https://doi.org/10.48550/arXiv.2305.11014) （**2023.05.18**）

<font color="gray">Tom Silver, Soham Dan, Kavitha Srinivas, J. Tenenbaum, L. Kaelbling, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-38-blue)](https://github.com/tomsilver/llm-genplan)

---

[**Discriminative Diffusion Models as Few-shot Vision and Language Learners**](https://doi.org/10.48550/arXiv.2305.10722) （**2023.05.18**）

<font color="gray">Xuehai He, Weixi Feng, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, etc .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Temporal Knowledge Graph Forecasting Without Knowledge Using In-Context Learning**](https://doi.org/10.48550/arXiv.2305.10613) （**2023.05.17**）

<font color="gray">Dong-Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, J. Pujara .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-16-blue)](https://github.com/usc-isi-i2/isi-tkg-icl)

---

[**What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning**](https://doi.org/10.48550/arXiv.2305.09731) （**2023.05.16**）

<font color="gray">Jane Pan, Tianyu Gao, Howard Chen, Danqi Chen .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-3-green)  [![](https://img.shields.io/badge/Github%20Stars-21-blue)](https://github.com/princeton-nlp/whaticllearns)

---

[**Enriching language models with graph-based context information to better understand textual data**](https://doi.org/10.48550/arXiv.2305.11070) （**2023.05.10**）

<font color="gray">Albert Roethel, M. Ganzha, Anna Wr'oblewska .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts**](https://doi.org/10.1145/3544548.3581388) （**2023.04.19**）

<font color="gray">J. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, Qiang Yang .  - 【International Conference on Human Factors in Computing Systems】</font>

![](https://img.shields.io/badge/Citations-24-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-202-red)

---

[**SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models**](https://doi.org/10.48550/arXiv.2303.10464) （**2023.03.18**）

<font color="gray">Vithursan Thangarasa, Abhay Gupta, William Marshall, Tianda Li, Kevin Leong, etc .  - 【Conference on Uncertainty in Artificial Intelligence】</font>

![](https://img.shields.io/badge/Citations-3-green)

---

[**Larger language models do in-context learning differently**](https://doi.org/10.48550/arXiv.2303.03846) （**2023.03.07**）

<font color="gray">Jerry W. Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-2-green)

---

[**Language Model Crossover: Variation through Few-Shot Prompting**](https://doi.org/10.48550/arXiv.2302.12170) （**2023.02.23**）

<font color="gray">Elliot Meyerson, M. Nelson, Herbie Bradley, Arash Moradi, Amy K. Hoover, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-1-green)  [![](https://img.shields.io/badge/Github%20Stars-666-blue)](https://github.com/carperai/openelm)

---

[**How Does In-Context Learning Help Prompt Tuning?**](https://doi.org/10.48550/arXiv.2302.11521) （**2023.02.22**）

<font color="gray">Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, Mohit Iyyer .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Compositional Exemplars for In-context Learning**](https://doi.org/10.48550/arXiv.2302.05698) （**2023.02.11**）

<font color="gray">Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, Lingpeng Kong .  - 【International Conference on Machine Learning】</font>

![](https://img.shields.io/badge/Citations-43-green)  [![](https://img.shields.io/badge/Github%20Stars-90-blue)](https://github.com/hkunlp/icl-ceil)

---

[**PLACES: Prompting Language Models for Social Conversation Synthesis**](https://doi.org/10.48550/arXiv.2302.03269) （**2023.02.07**）

<font color="gray">Maximillian Chen, A. Papangelis, Chenyang Tao, Seokhwan Kim, Andrew Rosenbaum, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-1-green)  [![](https://img.shields.io/badge/Github%20Stars-12-blue)](https://github.com/alexa/places)

---

[**Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning**](https://doi.org/10.48550/arXiv.2301.11916) （**2023.01.27**）

<font color="gray">Xinyi Wang, Wanrong Zhu, William Yang Wang .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-3-green)

---

[**Transformers as Algorithms: Generalization and Stability in In-context Learning**](https://arxiv.org/abs/2301.07067) （**2023.01.17**）

<font color="gray">Yingcong Li, M. E. Ildiz, Dimitris Papailiopoulos, Samet Oymak </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-58-red)  [![](https://img.shields.io/badge/Github%20Stars-7-blue)](https://github.com/yingcong-li/transformers-as-algorithms)

---

[**OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization**](https://doi.org/10.48550/arXiv.2212.12017) （**2022.12.22**）

<font color="gray">S. Iyer, Xiaojuan Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-11-green)  [![](https://img.shields.io/badge/Github%20Stars-34-blue)](https://github.com/tanyuqian/cappy)

---

[**Prompt-Augmented Linear Probing: Scaling Beyond The Limit of Few-shot In-Context Learners**](https://doi.org/10.48550/arXiv.2212.10873) （**2022.12.21**）

<font color="gray">Hyunsoo Cho, Hyuhng Joon Kim, Junyeob Kim, Sang-Woo Lee, Sang-goo Lee, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-2-green)

---

[**In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models**](https://doi.org/10.48550/arXiv.2212.10670) （**2022.12.20**）

<font color="gray">Yukun Huang, Yanda Chen, Zhou Yu, K. McKeown .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-0-green)

---

[**Self-adaptive In-context Learning**](https://doi.org/10.48550/arXiv.2212.10375) （**2022.12.20**）

<font color="gray">Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, Lingpeng Kong .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-3-green)

---

[**Is GPT-3 a Good Data Annotator?**](https://doi.org/10.48550/arXiv.2212.10450) （**2022.12.20**）

<font color="gray">Bosheng Ding, Chengwei Qin, Linlin Liu, Lidong Bing, Shafiq R. Joty, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-3-green)  [![](https://img.shields.io/badge/Github%20Stars-6-blue)](https://github.com/damo-nlp-sg/llm-data-annotator)

---

[**Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters**](https://doi.org/10.48550/arXiv.2212.10001) （**2022.12.20**）

<font color="gray">Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-7-green)  [![](https://img.shields.io/badge/Github%20Stars-75-blue)](https://github.com/sunlab-osu/understanding-cot)

---

[**Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers**](https://doi.org/10.48550/arXiv.2212.10559) （**2022.12.20**）

<font color="gray">Damai Dai, Yutao Sun, Li Dong, Y. Hao, Zhifang Sui, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-10-green)

---

[**Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations**](https://doi.org/10.48550/arXiv.2212.09865) （**2022.12.19**）

<font color="gray">Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi .  - 【Annual Meeting of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-29-green)  [![](https://img.shields.io/badge/Github%20Stars-9-blue)](https://github.com/alrope123/z-icl)

---

[**Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale**](https://doi.org/10.48550/arXiv.2212.09095) （**2022.12.18**）

<font color="gray">Hritik Bansal, Karthik Gopalakrishnan, Saket Dingliwal, S. Bodapati, Katrin Kirchhoff, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-1-green)  [![](https://img.shields.io/badge/Github%20Stars-22-blue)](https://github.com/amazon-science/llm-interpret)

---

[**Transformers learn in-context by gradient descent**](https://doi.org/10.48550/arXiv.2212.07677) （**2022.12.15**）

<font color="gray">J. Oswald, Eyvind Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-13-green)  [![](https://img.shields.io/badge/Github%20Stars-295-blue)](https://github.com/google-research/self-organising-systems)

---

[**Structured Prompting: Scaling In-Context Learning to 1, 000 Examples**](https://doi.org/10.48550/arXiv.2212.06713) （**2022.12.13**）

<font color="gray">Y. Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-2-green)  [![](https://img.shields.io/badge/Github%20Stars-3.4k-blue)](https://github.com/microsoft/lmops)

---

[**Diverse Demonstrations Improve In-context Compositional Generalization**](https://doi.org/10.48550/arXiv.2212.06800) （**2022.12.13**）

<font color="gray">Itay Levy, Ben Bogin, Jonathan Berant .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-3-green)  [![](https://img.shields.io/badge/Github%20Stars-8-blue)](https://github.com/itayle/diverse-demonstrations)

---

[**What learning algorithm is in-context learning? Investigations with linear models**](https://doi.org/10.48550/arXiv.2211.15661) （**2022.11.28**）

<font color="gray">Ekin Akyürek, D. Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-16-green)

---

[**What learning algorithm is in-context learning? Investigations with linear models**](https://doi.org/10.48550/arXiv.2211.15661) （**2022.11.28**）

<font color="gray">Ekin Akyürek, D. Schuurmans, Jacob Andreas, Tengyu Ma, Denny Zhou .  - 【International Conference on Learning Representations】</font>

![](https://img.shields.io/badge/Citations-197-green)

---

[**Complementary Explanations for Effective In-Context Learning**](https://doi.org/10.48550/arXiv.2211.13892) （**2022.11.25**）

<font color="gray">Xi Ye, Srini Iyer, Asli Celikyilmaz, V. Stoyanov, Greg Durrett, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-5-green)

---

[**Complementary Explanations for Effective In-Context Learning**](https://doi.org/10.48550/arXiv.2211.13892) （**2022.11.25**）

<font color="gray">Xi Ye, Srini Iyer, Asli Celikyilmaz, Ves Stoyanov, Greg Durrett, etc .  - 【Annual Meeting of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-41-green)

---

[**Large Language Models with Controllable Working Memory**](https://doi.org/10.48550/arXiv.2211.05110) （**2022.11.09**）

<font color="gray">Daliang Li, A. Rawat, M. Zaheer, Xin Wang, M. Lukasik, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-3-green)

---

[**Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning**](https://doi.org/10.48550/arXiv.2211.03044) （**2022.11.06**）

<font color="gray">Yu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang, T. Abdelzaher, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-2-green)  [![](https://img.shields.io/badge/Github%20Stars-35-blue)](https://github.com/yumeng5/fewgen)

---

[**Large Language Models Are Human-Level Prompt Engineers**](https://doi.org/10.48550/arXiv.2211.01910) （**2022.11.03**）

<font color="gray">Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-21-green)  [![](https://img.shields.io/badge/Github%20Stars-1.0k-blue)](https://github.com/keirp/automatic_prompt_engineer)

---

[**ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback**](https://doi.org/10.48550/arXiv.2210.12329) （**2022.10.22**）

<font color="gray">Jiacheng Ye, Jiahui Gao, Jiangtao Feng, Zhiyong Wu, Tao Yu, etc .  - 【Conference on Empirical Methods in Natural Language Processing】</font>

![](https://img.shields.io/badge/Citations-4-green)  [![](https://img.shields.io/badge/Github%20Stars-21-blue)](https://github.com/hkunlp/progen)

---

[**Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them**](https://doi.org/10.48550/arXiv.2210.09261) （**2022.10.17**）

<font color="gray">Mirac Suzgun, Nathan Scales, Nathanael Scharli, Sebastian Gehrmann, Yi Tay, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-29-green)  [![](https://img.shields.io/badge/Github%20Stars-388-blue)](https://github.com/suzgunmirac/big-bench-hard)

---

[**In-context Learning and Induction Heads**](https://doi.org/10.48550/arXiv.2209.11895) （**2022.09.24**）

<font color="gray">Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-34-green)

---

[**Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generation**](https://doi.org/10.48550/arXiv.2209.11000) （**2022.09.22**）

<font color="gray">Xingdi Yuan, Tong Wang, Yen-Hsiang Wang, Emery Fine, Rania Abdelghani, etc .  - 【Annual Meeting of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-7-green)

---

[**On the Relation between Sensitivity and Accuracy in In-context Learning**](https://doi.org/10.48550/arXiv.2209.07661) （**2022.09.16**）

<font color="gray">Yanda Chen, Chen Zhao, Zhou Yu, K. McKeown, He He .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-9-green)

---

[**What Can Transformers Learn In-Context? A Case Study of Simple Function Classes**](https://doi.org/10.48550/arXiv.2208.01066) （**2022.08.01**）

<font color="gray">Shivam Garg, Dimitris Tsipras, Percy Liang, G. Valiant .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-27-green)  [![](https://img.shields.io/badge/Github%20Stars-172-blue)](https://github.com/dtsip/in-context-learning)

---

[**Rationale-Augmented Ensembles in Language Models**](https://doi.org/10.48550/arXiv.2207.00747) （**2022.07.02**）

<font color="gray">Xuezhi Wang, Jason Wei, D. Schuurmans, Quoc Le, E. Chi, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-26-green)

---

[**Self-Generated In-Context Learning: Leveraging Auto-regressive Language Models as a Demonstration Generator**](https://doi.org/10.48550/arXiv.2206.08082) （**2022.06.16**）

<font color="gray">Hyuhng Joon Kim, Hyunsoo Cho, Junyeob Kim, Taeuk Kim, Kang Min Yoo, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-1-green)

---

[**Instruction Induction: From Few Examples to Natural Language Task Descriptions**](https://doi.org/10.48550/arXiv.2205.10782) （**2022.05.22**）

<font color="gray">Or Honovich, Uri Shaham, Samuel R. Bowman, Omer Levy .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-8-green)  [![](https://img.shields.io/badge/Github%20Stars-57-blue)](https://github.com/orhonovich/instruction-induction)

---

[**Prototypical Calibration for Few-shot Learning of Language Models**](https://doi.org/10.48550/arXiv.2205.10183) （**2022.05.20**）

<font color="gray">Zhixiong Han, Y. Hao, Li Dong, Yutao Sun, Furu Wei .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-4-green)  [![](https://img.shields.io/badge/Github%20Stars-5-blue)](https://github.com/zihanwangki/x-tc)

---

[**Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning**](https://doi.org/10.48550/arXiv.2205.05638) （**2022.05.11**）

<font color="gray">Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-62-green)  [![](https://img.shields.io/badge/Github%20Stars-418-blue)](https://github.com/r-three/t-few)

---

[**Improving In-Context Few-Shot Learning via Self-Supervised Training**](https://doi.org/10.48550/arXiv.2205.01703) （**2022.05.03**）

<font color="gray">Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, etc .  - 【North American Chapter of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-6-green)

---

[**On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model**](https://doi.org/10.48550/arXiv.2204.13509) （**2022.04.28**）

<font color="gray">Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, Hyoungseok Kim, etc .  - 【North American Chapter of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-14-green)

---

[**Data Distributional Properties Drive Emergent In-Context Learning in Transformers**](https://doi.org/10.48550/arXiv.2205.05055) （**2022.04.22**）

<font color="gray">Stephanie C. Y. Chan, Adam Santoro, Andrew Kyle Lampinen, Jane X. Wang, Aaditya K Singh, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-20-green)  [![](https://img.shields.io/badge/Github%20Stars-74-blue)](https://github.com/deepmind/emergent_in_context_learning)

---

[**Can language models learn from explanations in context?**](https://doi.org/10.48550/arXiv.2204.02329) （**2022.04.05**）

<font color="gray">Andrew Kyle Lampinen, I. Dasgupta, Stephanie C. Y. Chan, Kory Matthewson, Michael Henry Tessler, etc .  - 【Conference on Empirical Methods in Natural Language Processing】</font>

![](https://img.shields.io/badge/Citations-61-green)

---

[**Self-Consistency Improves Chain of Thought Reasoning in Language Models**](https://doi.org/10.48550/arXiv.2203.11171) （**2022.03.21**）

<font color="gray">Xuezhi Wang, Jason Wei, D. Schuurmans, Quoc Le, E. Chi, etc .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-133-green)  [![](https://img.shields.io/badge/Github%20Stars-898-blue)](https://github.com/lastmile-ai/aiconfig/tree/main/cookbooks/Multi-LLM-Consistency)

---

[**Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?**](https://arxiv.org/abs/2202.12837) （**2022.02.25**）

<font color="gray">Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, M. Lewis, etc .  - 【Conference on Empirical Methods in Natural Language Processing】</font>

![](https://img.shields.io/badge/Citations-130-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-472-red)  [![](https://img.shields.io/badge/Github%20Stars-164-blue)](https://github.com/alrope123/rethinking-demonstrations)

---

[**Co-training Improves Prompt-based Learning for Large Language Models**](https://arxiv.org/abs/2202.00828) （**2022.02.02**）

<font color="gray">Hunter Lang, Monica Agrawal, Yoon Kim, D. Sontag .  - 【International Conference on Machine Learning】</font>

![](https://img.shields.io/badge/Citations-8-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-54-red)  [![](https://img.shields.io/badge/Github%20Stars-16-blue)](https://github.com/clinicalml/cotrain-prompting)

---

[**Black-Box Tuning for Language-Model-as-a-Service**](https://arxiv.org/abs/2201.03514) （**2022.01.10**）

<font color="gray">Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, Xipeng Qiu .  - 【International Conference on Machine Learning】</font>

![](https://img.shields.io/badge/Citations-31-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-113-red)  [![](https://img.shields.io/badge/Github%20Stars-254-blue)](https://github.com/txsun1997/black-box-tuning)

---

[**Learning To Retrieve Prompts for In-Context Learning**](https://doi.org/10.18653/v1/2022.naacl-main.191) （**2021.12.16**）

<font color="gray">Ohad Rubin, Jonathan Herzig, Jonathan Berant .  - 【North American Chapter of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-85-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-178-red)  [![](https://img.shields.io/badge/Github%20Stars-56-blue)](https://github.com/ohadrubin/epr)

---

[**True Few-Shot Learning with Prompts—A Real-World Perspective**](https://doi.org/10.1162/tacl_a_00485) （**2021.11.26**）

<font color="gray">Timo Schick, Hinrich Schütze .  - 【Transactions of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-14-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-83-red)

---

[**An Explanation of In-context Learning as Implicit Bayesian Inference**](https://arxiv.org/abs/2111.02080) （**2021.11.03**）

<font color="gray">Sang Michael Xie, Aditi Raghunathan, Percy Liang, Tengyu Ma .  - 【International Conference on Learning Representations】</font>

![](https://img.shields.io/badge/Citations-59-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-218-red)  [![](https://img.shields.io/badge/Github%20Stars-93-blue)](https://github.com/p-lambda/incontext-learning)

---

[**MetaICL: Learning to Learn In Context**](https://doi.org/10.18653/v1/2022.naacl-main.201) （**2021.10.29**）

<font color="gray">Sewon Min, M. Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi .  - 【North American Chapter of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-82-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-202-red)  [![](https://img.shields.io/badge/Github%20Stars-244-blue)](https://github.com/facebookresearch/metaicl)

---

[**Meta-learning via Language Model In-context Tuning**](https://doi.org/10.18653/v1/2022.acl-long.53) （**2021.10.15**）

<font color="gray">Yanda Chen, Ruiqi Zhong, Sheng Zha, G. Karypis, He He .  - 【Annual Meeting of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-32-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-91-red)  [![](https://img.shields.io/badge/Github%20Stars-20-blue)](https://github.com/yandachen/in-context-tuning)

---

[**Multitask Prompted Training Enables Zero-Shot Task Generalization**](https://arxiv.org/abs/2110.08207) （**2021.10.15**）

<font color="gray">Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, etc .  - 【International Conference on Learning Representations】</font>

![](https://img.shields.io/badge/Citations-387-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-653-red)  [![](https://img.shields.io/badge/Github%20Stars-2.6k-blue)](https://github.com/bigscience-workshop/promptsource)

---

[**The Inductive Bias of In-Context Learning: Rethinking Pretraining Example Design**](https://arxiv.org/abs/2110.04541) （**2021.10.09**）

<font color="gray">Yoav Levine, Noam Wies, Daniel Jannai, D. Navon, Yedid Hoshen, etc .  - 【International Conference on Learning Representations】</font>

![](https://img.shields.io/badge/Citations-14-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-19-red)

---

[**Reframing Instructional Prompts to GPTk’s Language**](https://doi.org/10.18653/v1/2022.findings-acl.50) （**2021.09.16**）

<font color="gray">Swaroop Mishra, Daniel Khashabi, Chitta Baral, Yejin Choi, Hannaneh Hajishirzi .  - 【Findings】</font>

![](https://img.shields.io/badge/Citations-59-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-109-red)

---

[**Noisy Channel Language Model Prompting for Few-Shot Text Classification**](https://doi.org/10.18653/v1/2022.acl-long.365) （**2021.08.09**）

<font color="gray">Sewon Min, Michael Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer .  - 【Annual Meeting of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-70-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-201-red)  [![](https://img.shields.io/badge/Github%20Stars-128-blue)](https://github.com/shmsw25/Channel-LM-Prompting)

---

[**Multimodal Few-Shot Learning with Frozen Language Models**](https://arxiv.org/abs/2106.13884) （**2021.06.25**）

<font color="gray">Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, S. Eslami, Oriol Vinyals, etc .  - 【Neural Information Processing Systems】</font>

![](https://img.shields.io/badge/Citations-173-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-531-red)

---

[**Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity**](https://doi.org/10.18653/v1/2022.acl-long.556) （**2021.04.18**）

<font color="gray">Yao Lu, Max Bartolo, Alastair Moore, S. Riedel, Pontus Stenetorp .  - 【Annual Meeting of the Association for Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-170-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-310-red)  [![](https://img.shields.io/badge/Github%20Stars-451-blue)](https://github.com/RUCAIBox/LLMBox)

---

[**Calibrate Before Use: Improving Few-Shot Performance of Language Models**](https://arxiv.org/abs/2102.09690) （**2021.02.19**）

<font color="gray">Tony Zhao, Eric Wallace, Shi Feng, D. Klein, Sameer Singh .  - 【International Conference on Machine Learning】</font>

![](https://img.shields.io/badge/Citations-281-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-435-red)  [![](https://img.shields.io/badge/Github%20Stars-329-blue)](https://github.com/tonyzhaozh/few-shot-learning)

---

[**What Makes Good In-Context Examples for GPT-3?**](https://doi.org/10.18653/v1/2022.deelio-1.10) （**2021.01.17**）

<font color="gray">Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, L. Carin, etc .  - 【Workshop on Knowledge Extraction and Integration for Deep Learning Architectures; Deep Learning Inside Out】</font>

![](https://img.shields.io/badge/Citations-176-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-381-red)  [![](https://img.shields.io/badge/Github%20Stars-14.0k-blue)](https://github.com/stanfordnlp/dsp)

---

[**Few-Shot Text Generation with Pattern-Exploiting Training**](https://arxiv.org/abs/2012.11926) （**2020.12.22**）

<font color="gray">Timo Schick, Hinrich Schütze .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-59-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-114-red)  [![](https://img.shields.io/badge/Github%20Stars-1.6k-blue)](https://github.com/timoschick/pet)

---

[**InContext: A mobile application for the improvement of learning strategies at University**](https://doi.org/10.3916/c64-2020-10) （**2020.06.01**）

<font color="gray">Claudia-A. Lerma-Noriega, María-L. Flores-Palacios, Genaro Rebolledo-Méndez </font>

![](https://img.shields.io/badge/Citations-0-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-85-red)

---

[**Language Models are Few-Shot Learners**](https://arxiv.org/abs/2005.14165) （**2020.05.28**）

<font color="gray">Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, J. Kaplan, etc .  - 【Neural Information Processing Systems】</font>

![](https://img.shields.io/badge/Citations-8637-green)  ![](https://img.shields.io/badge/Mendeley%20Readers-10.3k-red)  [![](https://img.shields.io/badge/Github%20Stars-15.6k-blue)](https://github.com/openai/gpt-3)

---

[**Does GPT-3 Generate Empathetic Dialogues? A Novel In-Context Example Selection Method and Automatic Evaluation Metric for Empathetic Dialogue Generation**](https://api.semanticscholar.org/f3e54291e235df8ca91d3c83697c392155ed584d) 

<font color="gray">Young-Jun Lee, Chae-Gyun Lim, Ho-Jin Choi .  - 【International Conference on Computational Linguistics】</font>

![](https://img.shields.io/badge/Citations-0-green)  [![](https://img.shields.io/badge/Github%20Stars-19-blue)](https://github.com/passing2961/empgpt-3)

---

[**Transformers as Algorithms: Generalization and Implicit Model Selection in In-context Learning**](https://doi.org/10.48550/arXiv.2301.07067) 

<font color="gray">Yingcong Li, M. E. Ildiz, Dimitris Papailiopoulos, Samet Oymak .  - 【ArXiv】</font>

![](https://img.shields.io/badge/Citations-4-green)

---

[**Search-in-the-Chain: Towards the Accurate, Credible and Traceable Content Generation for Complex Knowledge-intensive Tasks**](https://doi.org/10.48550/arXiv.2304.14732) 

<font color="gray">Shicheng Xu, Liang Pang, Huawei Shen, Xueqi Cheng, Tat-Seng Chua .  - 【arXiv.org】</font>

![](https://img.shields.io/badge/Citations-0-green)


</div>

# CONTINUE...