Out-of-Distribution Generalization: Challenges and Paper List

Builder & Current Maintainer: Jiashuo Liu(Page)

Credit to THU-TAI Group

Challenges

NICO CHALLENGE 2022 is open, a new image recognition competition aims to provide a gold standard test for OOD image recognition, which is a vital problem in making AI technology trustworthy. Welcome to join and try via this link.

Paper List

We have summarized the main branches of works for Out-of-Distribution(OOD) Generalization problem, which are classified according to the research focus, including unsupervised representation learning, supervised learning models and optimization methods. For more details, please refer to our survey on OOD generalization(paper).

Branch 1: Unsupervised Representation Learning

1.1 Disentangeled Representation Learning

beta-VAE: Learning basic visual concepts with a constrained variational framework. Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner(ICLR2017)(paper)
Disentangling by factorising. Hyunjik Kim, Andriy Mnih(ICML2018)(papper)
Challenging common assumptions in the unsupervised learning of disentangled representations. Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Ratsch, Sylvain Gelly, Bernhard Scholkopf, Olivier Bachem(ICML2019)(paper)
Structure by Architecture: Disentangled Representations without Regularization. Felix Leeb, Guilia Lanzillotta, Yashas Annadani, Michel Besserve, Stefan Bauer, Bernhard Scholkopf (paper)
On disentangled representations learned from correlated data. Frederik Trauble, Elliot Creager, Niki Kilbertus, Francesco Locatello, Andrea Dittadi, Anirudh Goyal, Bernhard Scholkopf, Stefan Bauer(ICML2021)(paper)
On the transfer of disentangled representations in realistic settings. Andrea Dittadi, Frederik Trauble, Francesco Locatello, Manuel Wuthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, Bernhard Scholkopf(ICLR2021)(paper)
Self-Supervised Learning Disentangled Group Representation as Feature. Tan Wang, Zhongqi Yue, Jianqiang Huang, Qianru Sun, Hanwang Zhang(NeurIPS2021) (paper) (code)

1.2 Causal Representation Learning

CausalVAE: disentangled representation learning via neural structural causal models. Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, Jun Wang(paper)
Disentangled generative causal representation learning. Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, Tong Zhang(2020)(paper)
Variational autoencoders and nonlinear ica: A unifying framework. Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, Aapo Hyvarinen(AISTATS2020)(paper) (deleted in paper)
Toward causal representation learning. Bernhard Scholkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, Yoshua Bengio(Proc. IEEE, 2021)(paper)

Existing Surveys

Representation learning: A review and new perspectives. Yoshua Bengio, Aaron C. Courville, Pascal Vincent(TPAMI2013)(paper)

Bracnch 2: Supervised Learning Models

2.1 Causal Learning

Identification of causal effects using instrumental variables. Hei Chan, Manabu Kuroki(AISTATS2010).
Causal inference by using invariant prediction: identification and confidence intervals. Jonas Peters, Peter Buhlmann, Nicolai Meinshausen(JRSS)(paper)
Anchor regression: heterogeneous data meets causality. Dominik Rothenhäusler, Nicolai Meinshausen, Peter Buhlmann, Jonas Peters(paper)
Invariant Causal Prediction for Sequential Data. Niklas Pfister, Peter Buhlmann, Jonas Peters(paper)
Invariant Causal Prediction for Nonlinear Models. Christina Heinze-Deml, Jonas Peters, Nicolai Meinshausen(paper)
Active Invariant Causal Prediction: Experiment Selection through Stability. Juan Gamella, Christina Heinze-Deml(NeurIPS2020).(paper)
Regularizing towards Causal Invariance: Linear Models with Proxies. Michael Oberst, Nikolaj Thams, Jonas Peters, David A. Sontag(ICML2021)(paper)
Counterfactual Normalization: Proactively Addressing Dataset Shift and Improving Reliability Using Causal Mechanisms. Adarsh Subbaswamy, Suchi Saria(2018).(paper)
A Universal Hierarchy of Shift-Stable Distributions and the Tradeoff Between Stability and Performance. Adarsh Subbaswamy, Bryant Chen, Suchi Saria(paper)
Preventing Failures Due to Dataset Shift: Learning Predictive Models That Transport. Adarsh Subbaswamy, Peter Schulam, Suchi Saria(AISTATS2019)(paper)
Towards causality-aware predictions in static anticausal machine learning tasks: the linear structural causal model case. Elias Chaibub Neto(NeurIPS2020 Workshop)(paper)
I-SPEC: An End-to-End Framework for Learning Transportable, Shift-Stable Models. Adarsh Subbaswamy, Suchi Saria(2020)(paper)

Invariant Learning

Invariant Risk Minimization. Martin Arjovsky, Leon Bottou, Ishaan Gulrajani, David Lopez-Paz(2019)(paper)
Invariant Risk Minimization Games. Kartik Ahuja, Karthikeyan Shanmugam, Kush R. Varshney, Amit Dhurandhar(ICML2020)(paper)
Domain extrapolation via regret minimization. Wengong Jin, Regina Barzilay, Tommi S. Jaakkola(2020)
Risk variance penalization: From distributional robustness to causality. Chuanlong Xie, Fei Chen, Yue Liu, Zhenguo Li(2020)
Empirical or invariant risk minimization? a sample complexity perspective. Kartik Ahuja, Jun Wang, Amit Dhurandhar, Karthikeyan Shanmugam, Kush R. Varshney(ICLR2021)(paper)
Does Invariant Risk Minimization Capture Invariance? Pritish Kamath, Akilesh Tangella, Danica J. Sutherland, Nathan Srebro(AISTATS2021)(paper)
The risks of invariant risk minimization. Elan Rosenfeld, Pradeep Kumar Ravikumar, Andrej Risteski(ICLR2021)(paper)
Out-of-distribution generalization via risk extrapolation (REx). David Krueger, Ethan Caballero, Jorn-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, Aaron C. Courville(ICML2021)(paper)
Domain Generalization using Causal Matching. Divyat Mahajan, Shruti Tople, Amit Sharma(ICML2021)(paper)
Causal Attention for Unbiased Visual Recognition. Tan Wang, Chang Zhou, Qianru Sun, Hanwang Zhang(ICCV2021)(paper)
Environment Inference for Invariant Learning. Elliot Creager, Jorn-Henrik Jacobsen, Richard S. Zemel(ICML2021)(paper)
Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization. Kartik Ahuja, Ethan Caballero, Dinghuai Zhang, Yoshua Bengio, Ioannis Mitliagkas, Irina Rish(NeurIPS2021)(paper)
Nonlinear invariant risk minimization: A causal approach. Chaochao Lu, Yuhuai Wu, Jose Miguel Hernández-Lobato, Bernhard Scholkopf(paper)

2.2 Domain Generalization

A theory of learning from different domains. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, Jennifer Wortman Vaughan(Mach. Learn. 2010)(paper)
Deeper, broader and artier domain generalization. Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M. Hospedales(ICCV2017)(paper)

Representation

Domain generalization via invariant feature representation. Krikamol Muandet, David Balduzzi, Bernhard Scholkopf(ICML2013)(paper)
Deep domain confusion: Maximizing for domain invariance. Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, Trevor Darrell(2014)(paper)
Unsupervised domain adaptation by backpropagation. Yaroslav Ganin, Victor S. Lempitsky(ICML2015)(paper)
Batch normalization: Accelerating deep network training by reducing internal covariate shift. Sergey Ioffe, Christian Szegedy(ICML2015)(paper)
Domain generalization based on transfer component analysis. Thomas Grubinger, Adriana Birlutiu, Holger Schoner, Thomas Natschläger, Tom Heskes(IWANN2015)
Learning attributes equals multi-source domain generalization. Chuang Gan, Tianbao Yang, Boqing Gong(CVPR2016)(paper)
Robust domain generalisation by enforcing distribution invariance. Sarah M. Erfani, Mahsa Baktashmotlagh, Masud Moshtaghi, Vinh Nguyen, Christopher Leckie, James Bailey, Kotagiri Ramamohanarao(IJCAI2016)
Deep coral: Correlation alignment for deep domain adaptation. Baochen Sun, Kate Saenko(ECCV Workshop 2016)(paper)
Return of frustratingly easy domain adaptation. Baochen Sun, Jiashi Feng, Kate Saenko(AAAI2016)(paper)
Unified deep supervised domain adaptation and generalization. Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, Gianfranco Doretto(ICCV2017)(paper)
Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis. Dmitry Ulyanov, Andrea Vedaldi, Victor S. Lempitsky(CVPR2017)(paper)
Arbitrary style transfer in real-time with adaptive instance normalization. Xun Huang, Serge J. Belongie(ICCV2017)(paper)
Scatter component analysis: A unified framework for domain adaptation and domain generalization. Muhammad Ghifary, David Balduzzi, W. Bastiaan Kleijn, Mengjie Zhang(TPAMI2017)
Batch-instance normalization for adaptively style-invariant neural networks. Hyeonseob Nam, Hyo-Eun Kim(NeurIPS2018)(paper)
Generalizing across domains via cross-gradient training. Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, Sunita Sarawagi(ICLR2018)(paper)
Visual domain adaptation with manifold embedded distribution alignment. Jindong Wang, Wenjie Feng, Yiqiang Chen, Han Yu, Meiyu Huang, Philip S. Yu(MM2018)(paper)
Synthetic to real adaptation with generative correlation alignment networks. Xingchao Peng, Kate Saenko(WACV2018)(paper)
Two at once: Enhancing learning and generalization capacities via IBN-Net. Xingang Pan, Ping Luo, Jianping Shi, Xiaoou Tang(ECCV2018)(paper)
Domain generalization via multidomain discriminant analysis. Shoubo Hu, Kun Zhang, Zhitang Chen, Laiwan Chan(UAI2019)(paper)
Moment matching for multi-source domain adaptation. Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, Bo Wang(ICCV2019)(paper)
Adversarial target-invariant representation learning for domain generalization. Isabela Albuquerque, João Monteiro, Tiago H. Falk, Ioannis Mitliagkas(ECMLPKDD2019)(paper)
Domain generalization with adversarial feature learning. Haoliang Li, Sinno Jialin Pan, Shiqi Wang, Alex C. Kot(CVPR 2018)
Dlow: Domain flow for adaptation and generalization. Rui Gong, Wen Li, Yuhua Chen, Luc Van Gool(CVPR2019)(paper)
Multi-adversarial discriminative deep domain generalization for face presentation attack detection. Rui Shao, Xiangyuan Lan, Jiawei Li, Pong C. Yuen(CVPR2019)
Correlation-aware adversarial domain adaptation and generalization Mohammad Mahfujur Rahman, Clinton Fookes, Mahsa Baktashmotlagh, Sridha Sridharan(PR2020)(paper)
Unseen target stance detection with adversarial domain generalization. Zhen Wang, Qiansheng Wang, Chengguo Lv, Xue Cao, Guohong Fu(IJCNN2020)(paper)
Single-Side domain generalization for face anti-spoofing. Yunpei Jia, Jie Zhang, Shiguang Shan, Xilin Chen(CVPR2020)(paper)
Domain generalization via entropy regularization. Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, Dacheng Tao(NeurIPS2020)
Style normalization and restitution for generalizable person re-identification. Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen, Li Zhang(CVPR2020)(paper)
Transfer learning with dynamic distribution adaptation. Jindong Wang, Yiqiang Chen, Wenjie Feng, Han Yu, Meiyu Huang, Qiang Yang(TIST 2020)(paper)
Style Normalization and Restitution for Domain Generalization and Adaptation. Xin Jin, Cuiling Lan, Wenjun Zeng, Zhibo Chen(2021)(paper)
Domain generalization by marginal transfer learning. Gilles Blanchard, Aniket Anand Deshmukh, urun Dogan, Gyemin Lee, Clayton Scott(JMLR2021)(paper)
Learn to expect the unexpected: Probably approximately correct domain generalization. Vikas K. Garg, Adam Tauman Kalai, Katrina Ligett, Zhiwei Steven Wu(AISTATS2021)(paper)
Domain adversarial neural networks for domain generalization: When it works and how to improve. Anthony Sicilia, Xingchen Zhao, Seong Jae Hwang(2021)(paper)

Training Strategy

Feature space independent semi-supervised domain adaptation via kernel matching. Min Xiao, Yuhong Guo(TPAMI2015)
Unsupervised learning of visual representations by solving jigsaw puzzles. Mehdi Noroozi, Paolo Favaro(ECCV 2016)(paper)
Model-agnostic meta-learning for fast adaptation of deep networks. Chelsea Finn, Pieter Abbeel, Sergey Levine(ICML2017)(paper)
Domain randomization for transferring deep neural networks from simulation to the real world. Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, Pieter Abbeel(IROS2017)(paper)
Learning to generalize: Meta-learning for domain generalization. Da Li, Yongxin Yang, Yi-Zhe Song, Timothy M. Hospedales(AAAI2018)(paper)
Domain generalization with domain-specific aggregation modules. Antonio D'Innocente, Barbara Caputo(2018)(paper)
Deep domain generalization with structured low-rank constraint. Zhengming Ding, Yun Fu(TIP2018)
Best sources forward: domain generalization through source-specific nets. Massimiliano Mancini, Samuel Rota Bulò, Barbara Caputo, Elisa Ricci(ICIP2018)(paper)
Metareg: Towards domain generalization using meta-regularization. Yogesh Balaji, Swami Sankaranarayanan, Rama Chellappa(NeurIPS2018)
Generalizing to unseen domains via adversarial data augmentation. Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John C. Duchi, Vittorio Murino, Silvio Savarese(NeurIPS2018)(paper)
Sim-to-real transfer of robotic control with dynamics randomization. Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, Pieter Abbeel(ICRA2018)(paper)
Training deep networks with synthetic data: Bridging the reality gap by domain randomization. Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, Stan Birchfield(CVPR Workshop2018)(paper)
Domain randomization for scene-specific car detection and pose estimation. Rawal Khirodkar, Donghyun Yoo, Kris M. Kitani(WACV2019)(paper)
Multi-component image translation for deep domain generalization. Mohammad Mahfujur Rahman, Clinton Fookes, Mahsa Baktashmotlagh, Sridha Sridharan(WACV2019)(paper)
Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data. Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto L. Sangiovanni-Vincentelli, Kurt Keutzer, Boqing Gong(ICCV2019)(paper)
Structured domain randomization: Bridging the reality gap by context-aware synthetic data. Aayush Prakash, Shaad Boochoon, Mark Brophy, David Acuna, Eric Cameracci, Gavriel State, Omer Shapira, Stan Birchfield(ICRA2019)(paper)
Feature-critic networks for heterogeneous domain generalization. Yiying Li, Yongxin Yang, Wei Zhou, Timothy M. Hospedales(ICML2019)(paper)
Domain generalization by solving jigsaw puzzles. Fabio Maria Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, Tatiana Tommasi(CVPR2019)(paper)
Domain generalization via model-agnostic learning of semantic features. Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, Ben Glocker(NeurIPS2019)(paper)
Domain generalization with optimal transport and metric learning. Fan Zhou, Zhuqing Jiang, Changjian Shui, Boyu Wang, Brahim Chaib-draa(2020)(paper)
Deep semisupervised domain generalization network for rotary machinery fault diagnosis under variable speed. Yixiao Liao, Ruyi Huang, Jipu Li, Zhuyun Chen, Weihua Li( IEEE TIM2020)
Unsupervised domain adaptation through self-supervision. Fei Pan, Inkyu Shin, François Rameau, Seokju Lee, In So Kweon(CVPR2020)(paper)
Shape-aware meta-learning for generalizing prostate MRI segmentation to unseen domains. Quande Liu, Qi Dou, Pheng-Ann Heng(MICCAI2020)(paper)
Learning to learn with variational information bottleneck for domain generalization. Ying-Jun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees G. M. Snoek, Ling Shao(ECCV2020)(paper)
Learning to generalize unseen domains via memory-based multi-source meta-learning for person re-identification. Yuyang Zhao, Zhun Zhong, Fengxiang Yang, Zhiming Luo, Yaojin Lin, Shaozi Li, Nicu Sebe(CVPR2021)(paper)
Meta Batch-Instance Normalization for Generalizable Person Re-Identification. Seokeon Choi, Taekyung Kim, Minki Jeong, Hyoungseob Park, Changick Kim(CVPR2021)(paper)
Dofe: Domain-oriented feature embedding for generalizable fundus image segmentation on unseen datasets. Shujun Wang, Lequan Yu, Kang Li, Xin Yang, Chi-Wing Fu, Pheng-Ann Heng(TMI2020)(paper)
Batch normalization embeddings for deep domain generalization. Mattia Segù, Alessio Tonioni, Federico Tombari(2020)(paper)
Generalized convolutional forest networks for domain generalization and visual recognition. Jongbin Ryu, Gitaek Kwon, Ming-Hsuan Yang, Jongwoo Lim(ICLR2020)
Episodic training for domain generalization. Bincheng Huang, Si Chen, Fan Zhou, Cheng Zhang, Feng Zhang(ICCSIP2020)(paper)
Self-challenging improves cross-domain generalization. Zeyi Huang, Haohan Wang, Eric P. Xing, Dong Huang(ECCV2020)(paper)
Learning to learn single domain generalization. Fengchun Qiao, Long Zhao, Xi Peng(CVPR2020)(paper)
Learning to generate novel domains for domain generalization. Kaiyang Zhou, Yongxin Yang, Timothy M. Hospedales, Tao Xiang(ECCV2020)(paper)
Towards Unsupervised Domain Generalization. Xingxuan Zhang, Linjun Zhou, Renzhe Xu, Peng Cui, Zheyan Shen, Haoxin Liu(CVPR2022)(paper)
Frustratingly simple domain generalization via image stylization. Nathan Somavarapu, Chih-Yao Ma, Zsolt Kira(2020)(paper)
Deep domain-adversarial image generation for domain generalisation. Kaiyang Zhou, Yongxin Yang, Timothy M. Hospedales, Tao Xiang(AAAI2020)(paper)
Metanorm: Learning to normalize few-shot batches across domains. Ying-Jun Du, Xiantong Zhen, Ling Shao, Cees G. M. Snoek(ICLR2021)
Frustratingly easy semi-supervised domain adaptation
Staining invariant features for improving generalization of deep convolutional neural networks in computational pathology

Existing Surveys

A survey on transfer learning. Sinno Jialin Pan, Qiang Yang(TKDE2010)(paper)
Meta-learning in neural networks: A survey. Timothy M. Hospedales, Antreas Antoniou, Paul Micaelli, Amos J. Storkey(2020)(paper)
Generalizing to Unseen Domains: A Survey on Domain Generalization. Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin(IJCAI2021)(paper)
Domain generalization in Vision: A survey. Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, Chen Change Loy(2021)(paper)

2.3 Stable Learning

Stable Learning Establishes Some Common Ground Between Causal Inference and Machine Learning. Peng Cui and Susan Athey. Nature Machine Intelligence, 2022.(paper)
Causally Regularized Learning with Agnostic Data Selection Bias. Zheyan Shen, Peng Cui, Kun Kuang, Bo Li, Peixuan Chen(MM2018)(paper)
Stable Prediction across Unknown Environments. Kun Kuang, Peng Cui, Susan Athey, Ruoxuan Xiong and Bo Li(KDD2018)(paper)
Stable Prediction with Model Misspecification and Agnostic Distribution Shift. Kun Kuang, Ruoxuan Xiong, Peng Cui, Susan Athey, Bo Li(AAAI2020)(paper)
Stable Learning via Sample Reweighting. Zheyan Shen, Peng Cui, Tong Zhang, Kun Kuang(AAAI2020)(paper)
Stable Learning via Differentiated Variable Decorrelation. Zheyean Shen, Peng Cui, Jiashuo Liu, Tong Zhang, Bo Li and Zhitang Chen(KDD2020)(paper)
Decorrelated Clustering with Data Selection Bias. Xiao Wang, Shaohua Fan, Kun Kuang, Chuan Shi, Jiawei Liu, Bai Wang(IJCAI2020)(paper)
DeVLBert: Learning Deconfounded Visio-Linguistic Representations. Shengyu Zhang, Tan Jiang, Tan Wang, Kun Kuang, Zhou Zhao, Jianke Zhu, Jin Yu, Hongxia Yang, Fei Wu(MM2020)(paper)
Meta-Learning Causal Feature Selection for Stable Prediction. Zhaoquan Yuan, Xiao Peng, Xiao Wu, Bing-kun Bao, Changsheng Xu(ICME 2021)(paper)
Deep Stable Learning for Out-Of-Distribution Generalization. Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, Zheyan Shen(CVPR2021)(paper)
Towards Domain Generalization in Object Detection. Xingxuan Zhang, Zekai Xu, Renzhe Xu, Jiashuo Liu, Peng Cui, Weitao Wan, Chong Sun, and Chen Li(paper)
Balance-Subsampled Stable Prediction. Kun Kuang, Hengtao Zhang, Fei Wu, Yueting Zhuang, Aijun Zhang (paper)
A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization. Renzhe Xu, Xingxuan Zhang, Zheyan Shen, Tong Zhang, Peng Cui (ICML 2022) (paper)

Existing Surveys

Invariance, causality and robustness. Peter Buhlmann(paper)

Branch 3: Optimization

3.1 Distributionally Robust Optimization

Robust Regression and Lasso. Huan Xu, Constantine Caramanis, Shie Mannor:(NeurIPS2008)(paper)
Distributionally Robust Optimization Under Moment Uncertainty with Application to Data-Driven Problems. Erick Delage, Yinyu Ye(OR2010)
A Unified Robust Regression Model for Lasso-like Algorithms. Wenzhuo Yang, Huan Xu(ICML2013)
Distributionally Robust Logistic Regression. Soroosh Shafieezadeh-Abadeh, Peyman Mohajerin Esfahani, Daniel Kuhn(NeurIPS2015)(paper)
Stochastic Gradient Methods for Distributionally Robust Optimization with f-divergences. Hongseok Namkoong, John C. Duchi(NeurIPS2016)
Wasserstein distributional robustness and regularization in statistical learning. Rui Gao, Xi Chen, Anton J. Kleywegt(2017)
Characterization of the equivalence of robustification and regularization in linear and matrix regression. Dimitris Bertsimas, Martin S. Copenhaver(EJOR2018)(paper)
Learning models with uniform performance via distributionally robust optimization. John C. Duchi, Hongseok Namkoong(2018)(paper)
Certifying Some Distributional Robustness with Principled Adversarial Training. Aman Sinha, Hongseok Namkoong, John C. Duchi(ICLR2018)(paper)
A robust learning approach for regression models based on distributionally robust optimization. Ruidi Chen, Ioannis Ch. Paschalidis(JMLR2018)
Data-driven chance constrained stochastic program. Ruiwei Jiang, Yongpei Guan(Math. Program. 2018)
Data-driven robust optimization. Dimitris Bertsimas, Vishal Gupta, Nathan Kallus(Math. Program.2018)(paper)
Does distributionally robust supervised learning give robust classifiers? Weihua Hu, Gang Niu, Issei Sato, Masashi Sugiyama(ICML2018)(paper)
Causality from a distributional robustness point of view. Nicolai Meinshausen(DSW2018)
Regularization via Mass Transportation. Soroosh Shafieezadeh-Abadeh, Daniel Kuhn, Peyman Mohajerin Esfahani(JMLR2019)(paper)
Data-driven optimal transport cost selection for distributionally robust optimization. Jose H. Blanchet, Yang Kang, Karthyek R. A. Murthy, Fan Zhang(WSC2019)
Incorporating Unlabeled Data into Distributionally Robust Learning. Charlie Frogner, Sebastian Claici, Edward Chien, Justin Solomon(2019)(paper)
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization. Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, Percy Liang(ICLR2020)(paper)
Stable Adversarial Learning under Distributional Shifts. Jiashuo Liu, Zheyan Shen, Peng Cui, Linjun Zhou, Kun Kuang, Bo Li, Yishi Lin(AAAI2021)(paper)
On Distributionally Robust Optimization and Data Rebalancing. Agnieszka Słowik, Leon Bottou (AISTATS 2022)(paper)
Robust Solutions to Least-Squares Problems with Uncertain Data.
Statistics of robust optimization: A generalized empirical likelihood approach. John Duchi, Peter Glynn, Hongseok Namkoong (paper)
Distributionally robust losses against mixture covariate shifts.
Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. Peyman Mohajerin Esfahani, Daniel Kuhn
Regression shrinkage and selection via the lasso: a retrospective.

Existing Surveys

Distributionally robust optimization: A review. Hamed Rahimian, Sanjay Mehrotra(2019)(paper)

3.2 Invariance-Based Optimization

Invariant models for causal transfer learning. Mateo Rojas-Carulla, Bernhard Scholkopf, Richard E. Turner, Jonas Peters(JMLR2018)(paper)
Invariant Rationalization. Shiyu Chang, Yang Zhang, Mo Yu, Tommi S. Jaakkola(ICML2020)(paper)
Out-of-Distribution Generalization with Maximal Invariant Predictor. Masanori Koyama, Shoichiro Yamaguchi(2020)
Heterogeneous Risk Minimization. Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li, Zheyan Shen(ICML2021)(paper)
Can Subnetwork Structure be the Key to Out-of-Distribution Generalization? Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang, Aaron Courville(ICML2021)(paper)
Kernelized Heterogeneous Risk Minimization. Jiashuo Liu, Zheyuan Hu, Peng Cui, Bo Li, Zheyan Shen(NeurIPS2021)(paper)