In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. This packages offers two modes of computation to calculate the influence In, Mei, S. and Zhu, X. calculations, which could potentially be 10s of thousands. << But keep in mind that some of the key concepts in this course, such as directional derivatives or Hessian-vector products, might not be so straightforward to use in some frameworks. affecting everything else. Understanding Black-box Predictions via Influence Functions - YouTube AboutPressCopyrightContact usCreatorsAdvertiseDevelopersTermsPrivacyPolicy & SafetyHow YouTube worksTest new features 2022. While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Here, we plot I up,loss against variants that are missing these terms and show that they are necessary for picking up the truly inuential training points. test images, the helpfulness is ordered by average helpfulness to the Understanding black-box predictions via influence functions. Negative momentum for improved game dynamics. Liu, Y., Jiang, S., and Liao, S. Efficient approximation of cross-validation for kernel methods using Bouligand influence function. To scale up influence functions to modern machine learning Often we want to identify an influential group of training samples in a particular test prediction. Deep inside convolutional networks: Visualising image classification models and saliency maps. That can increase prediction accuracy, reduce Reconciling modern machine-learning practice and the classical bias-variance tradeoff. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. When testing for a single test image, you can then Which optimization techniques are useful at which batch sizes? x\Y#7r~_}2;4,>Fvv,ZduwYTUQP }#&uD,spdv9#?Kft&e&LS 5[^od7Z5qg(]}{__+3"Bej,wofUl)u*l$m}FX6S/7?wfYwoF4{Hmf83%TF#}{c}w( kMf*bLQ?C}?J2l1jy)>$"^4Rtg+$4Ld{}Q8k|iaL_@8v To run the tests, further requirements are: You can either install this package directly through pip: Calculating the influence of the individual samples of your training dataset In. Understanding black-box predictions via influence functions. ImageNet large scale visual recognition challenge. Deep learning via hessian-free optimization. Influence functions are a classic technique from robust statistics to identify the training points most responsible for a given prediction. above, keeping the grad_zs only makes sense if they can be loaded faster/ Not just a black box: Learning important features through propagating activation differences. ICML 2017 best paperStanfordPang Wei KohPercy liang, x_{test} y_{test} label x_{test} , n z_1z_n z_i=(x_i,y_i) L(z,\theta) z \theta , \hat{\theta}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta), z z \epsilon ERM, \hat{\theta}_{\epsilon,z}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta)+\epsilon L(z,\theta), influence function, \mathcal{I}_{up,params}(z)={\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0}=-H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}), H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta) Hessien, \begin{equation} \begin{aligned} \mathcal{I}_{up,loss}(z,z_{test})&=\frac{dL(z_{test},\hat\theta_{\epsilon,z})}{d\epsilon}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T {\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T\mathcal{I}_{up,params}(z)\\&=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta) \end{aligned} \end{equation}, lossNLPer, influence function, logistic regression p(y|x)=\sigma (y \theta^Tx) \sigma sigmoid z_{test} loss z \mathcal{I}_{up,loss}(z,z_{test}) , -y_{test}y \cdot \sigma(-y_{test}\theta^Tx_{test}) \cdot \sigma(-y\theta^Tx) \cdot x^{T}_{test} H^{-1}_{\hat\theta}x, \sigma(-y\theta^Tx) outlieroutlier, x^{T}_{test} x H^{-1}_{\hat\theta} Hessian \mathcal{I}_{up,loss}(z,z_{test}) resistencevariation, \mathcal{I}_{up,loss}(z,z_{test})=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta), Hessian H_{\hat\theta} O(np^2+p^3) n p z_i , conjugate gradientstochastic estimationHessian-vector productsHVP H_{\hat\theta} s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta) \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta) , H_{\hat\theta}^{-1}v=argmin_{t}\frac{1}{2}t^TH_{\hat\theta}t-v^Tt, HVPCG O(np) , H^{-1} , (I-H)^i,i=1,2,\dots,n H 1 j , S_j=\frac{I-(I-H)^j}{I-(I-H)}=\frac{I-(I-H)^j}{H}, \lim_{j \to \infty}S_j z_i \nabla_\theta^{2} L(z_i,\hat\theta) H , HVP S_i S_i \cdot \nabla_\theta L(z_{test},\hat\theta) , NMIST H loss , ImageNetInceptionRBF SVM, RBF SVMRBF SVM, InceptionInception, Inception, , Inception591/60059133557%, check \mathcal{I}_{up,loss}(z_i,z_i) z_i , 10% \mathcal{I}_{up,loss}(z_i,z_i) , H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta), s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta), \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta), S_i \cdot \nabla_\theta L(z_{test},\hat\theta). As a result, the practical success of neural nets has outpaced our ability to understand how they work. However, in a lower Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. Most weeks we will be targeting 2 hours of class time, but we have extra time allocated in case presentations run over. Time permitting, we'll also consider the limit of infinite depth. ( , ) Inception, . Disentangled graph convolutional networks. >> The canonical example in machine learning is hyperparameter optimization. Riemannian metrics for neural networks I: Feed-forward networks. Loss non-convex, quadratic loss . To scale up influence functions to modern machine learning settings, How can we explain the predictions of a black-box model? Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. Understanding black-box predictions via influence functions While influence estimates align well with leave-one-out. Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. Hopefully this understanding will let us improve the algorithms. Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., and Suh, J. Modeltracker: Redesigning performance analysis tools for machine learning. Interacting with predictions: Visual inspection of black-box machine learning models. The project proposal is due on Feb 17, and is primarily a way for us to give you feedback on your project idea. In. James Tu, Yangjun Ruan, and Jonah Philion. If you have questions, please contact Pang Wei Koh (pangwei@cs.stanford.edu). To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. We motivate second-order optimization of neural nets from several perspectives: minimizing second-order Taylor approximations, preconditioning, invariance, and proximal optimization. We'll consider the heavy ball method and why the Nesterov Accelerated Gradient can further speed up convergence. Borys Bryndak, Sergio Casas, and Sean Segal. Here are the materials: For the Colab notebook and paper presentation, you will form a group of 2-3 and pick one paper from a list. Check if you have access through your login credentials or your institution to get full access on this article. In. Often we want to identify an influential group of training samples in a particular test prediction for a given We study the task of hardness amplification which transforms a hard function into a harder one. ? Overview Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. Class will be held synchronously online every week, including lectures and occasionally tutorials. This site last compiled Wed, 08 Feb 2023 10:43:27 +0000. Abstract. 2019. Stochastic Optimization and Scaling [Slides]. Pearlmutter, B. most harmful. How can we explain the predictions of a black-box model? Metrics give a local notion of distance on a manifold. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through . Requirements chainer v3: It uses FunctionHook. Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W. A theory of learning from different domains. Gradient descent on neural networks typically occurs on the edge of stability. Understanding Black-box Predictions via Inuence Functions 2. calculates the grad_z values for all images first and saves them to disk. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. The final report is due April 7. non-convex non-differentialble . prediction outcome of the processed test samples. We'll see how to efficiently compute with them using Jacobian-vector products. In this paper, we use influence functions a classic technique from robust statistics to trace a . Wei, B., Hu, Y., and Fung, W. Generalized leverage and its applications. Appendix: Understanding Black-box Predictions via Inuence Functions Pang Wei Koh1Percy Liang1 Deriving the inuence functionIup,params For completeness, we provide a standard derivation of theinuence functionIup,params in the context of loss minimiza-tion (M-estimation). There are various full-featured deep learning frameworks built on top of JAX and designed to resemble other frameworks you might be familiar with, such as PyTorch or Keras. No description, website, or topics provided. Strack, B., DeShazo, J. P., Gennings, C., Olmo, J. L., Ventura, S., Cios, K. J., and Clore, J. N. Impact of HbA1c measurement on hospital readmission rates: analysis of 70,000 clinical database patient records. An evaluation of the human-interpretability of explanation. The mechanics of n-player differentiable games. Understanding black-box predictions via influence functions Computing methodologies Machine learning Recommendations On second-order group influence functions for black-box predictions With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Yuwen Xiong, Andrew Liao, and Jingkang Wang. functions. ": Explaining the predictions of any classifier. This leads to an important optimization tool called the natural gradient. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. How can we explain the predictions of a black-box model? In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. This Differentiable Games (Lecture by Guodong Zhang) [Slides]. calculate which training images had the largest result on the classification Automatically creates outdir folder to prevent runtime error, Merge branch 'expectopatronum-update-readme', Understanding Black-box Predictions via Influence Functions, import it as a package after it's in your, Combined, the original paper suggests that. Either way, if the network architecture is itself optimizing something, then the outer training procedure is wrestling with the issues discussed in this course, whether we like it or not. We'll use linear regression to understand two neural net training phenomena: why it's a good idea to normalize the inputs, and the double descent phenomenon whereby increasing dimensionality can reduce overfitting. Theano: A Python framework for fast computation of mathematical expressions. We'll consider the two most common techniques for bilevel optimization: implicit differentiation, and unrolling. In this paper, we use influence functions a classic technique from robust statistics to trace a models prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. can take significant amounts of disk space (100s of GBs) but with a fast SSD Helpful is a list of numbers, which are the IDs of the training data samples the first approximation in s_test and once to combine with the s_test The marking scheme is as follows: The problem set will give you a chance to practice the content of the first three lectures, and will be due on Feb 10. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. There are several neural net libraries built on top of JAX. For a point z and parameters 2 , let L(z; ) be the loss, and let1 n P n i=1L(z The next figure shows the same but for a different model, DenseNet-100/12. The Agarwal, N., Bullins, B., and Hazan, E. Second order stochastic optimization in linear time. We have a reproducible, executable, and Dockerized version of these scripts on Codalab. This will naturally lead into next week's topic, which applies similar ideas to a different but related dynamical system. Please download or close your previous search result export first before starting a new bulk export. config is a dict which contains the parameters used to calculate the Thomas, W. and Cook, R. D. Assessing influence on predictions from generalized linear models. On robustness properties of convex risk minimization methods for pattern recognition. In, Martens, J. We'll consider two models of stochastic optimization which make vastly different predictions about convergence behavior: the noisy quadratic model, and the interpolation regime. The security of latent Dirichlet allocation. This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. How can we explain the predictions of a black-box model? In order to have any hope of understanding the solutions it comes up with, we need to understand the problems. Infinite Limits and Overparameterization [Slides]. Rethinking the Inception architecture for computer vision. Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. Three mechanisms of weight decay regularization. Proc 34th Int Conf on Machine Learning, p.1885-1894. on the final predictions is straight forward. Optimizing neural networks with Kronecker-factored approximate curvature. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. kept in RAM than calculating them on-the-fly. samples for each test data sample. Using machine teaching to identify optimal training-set attacks on machine learners. Validations 4. A tag already exists with the provided branch name. Often we want to identify an influential group of training samples in a particular test prediction for a given machine learning model. The first mode is called calc_img_wise, during which the two More details can be found in the project handout. Cook, R. D. and Weisberg, S. Characterizations of an empirical influence function for detecting influential cases in regression. 10 0 obj Understanding Black-box Predictions via Influence Functions Proceedings of the 34th International Conference on Machine Learning . A. Kansagara, D., Englander, H., Salanitro, A., Kagen, D., Theobald, C., Freeman, M., and Kripalani, S. Risk prediction models for hospital readmission: a systematic review. Simonyan, K., Vedaldi, A., and Zisserman, A. thereby identifying training points most responsible for a given prediction. Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function The second mode is called calc_all_grad_then_test and If there are n samples, it can be interpreted as 1/n. Understanding the Representation and Computation of Multilayer Perceptrons: A Case Study in Speech Recognition. Imagenet classification with deep convolutional neural networks. You can get the default config by calling ptif.get_default_config(). Please try again. Despite its simplicity, linear regression provides a surprising amount of insight into neural net training. ordered by helpfulness. , . A. P. Nakkiran, B. Neyshabur, and H. Sedghi. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. In many cases, they have far more than enough parameters to memorize the data, so why do they generalize well? For this class, we'll use Python and the JAX deep learning framework. Assignments for the course include one problem set, a paper presentation, and a final project. Understanding black-box predictions via influence functions. How can we explain the predictions of a black-box model? Cook, R. D. Detection of influential observation in linear regression. ordered by harmfulness. ; Liang, Percy. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. NIPS, p.1097-1105. Biggio, B., Nelson, B., and Laskov, P. Support vector machines under adversarial label noise. logistic regression p (y|x)=\sigma (y \theta^Tx) \sigma . This is the case because grad_z has to be calculated twice, once for Your file of search results citations is now ready. PVANet: Lightweight Deep Neural Networks for Real-time Object Detection. Understanding Black-box Predictions via Influence Functions. initial value of the Hessian during the s_test calculation, this is The details of the assignment are here. The more recent Neural Tangent Kernel gives an elegant way to understand gradient descent dynamics in function space. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. In this paper, we use influence functions --- a classic technique from robust statistics --- Are you sure you want to create this branch? Data poisoning attacks on factorization-based collaborative filtering. below is divided into parameters affecting the calculation and parameters I am grateful to my supervisor Tasnim Azad Abir sir, for his . In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. , Hessian-vector . S. L. Smith, B. Dherin, D. Barrett, and S. De. Influence functions help you to debug the results of your deep learning model In, Metsis, V., Androutsopoulos, I., and Paliouras, G. Spam filtering with naive Bayes - which naive Bayes? Understanding Black-box Predictions via Influence Functions ICML2017 3 (influence function) 4 ICML 2017 best paperStanfordPang Wei KohCourseraStanfordNIPS 2019influence functionPercy Liang11Michael Jordan, , \hat{\theta}_{\epsilon, z} \stackrel{\text { def }}{=} \arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L(z, \theta), \left.\mathcal{I}_{\text {up, params }}(z) \stackrel{\text { def }}{=} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0}=-H_{\tilde{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}), , loss, \begin{aligned} \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) &\left.\stackrel{\text { def }}{=} \frac{d L\left(z_{\text {test }}, \hat{\theta}_{\epsilon, z}\right)}{d \epsilon}\right|_{\epsilon=0} \\ &=\left.\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, \varepsilon=-1/n , z=(x,y) \\ z_{\delta} \stackrel{\text { def }}{=}(x+\delta, y), \hat{\theta}_{\epsilon, z_{\delta},-z} \stackrel{\text { def }}{=}\arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L\left(z_{\delta}, \theta\right)-\epsilon L(z, \theta), \begin{aligned}\left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} &=\mathcal{I}_{\text {up params }}\left(z_{\delta}\right)-\mathcal{I}_{\text {up, params }}(z) \\ &=-H_{\hat{\theta}}^{-1}\left(\nabla_{\theta} L(z_{\delta}, \hat{\theta})-\nabla_{\theta} L(z, \hat{\theta})\right) \end{aligned}, \varepsilon \delta \deltaloss, \left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} \approx-H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \hat{\theta}_{z_{i},-z}-\hat{\theta} \approx-\frac{1}{n} H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \begin{aligned} \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top} &\left.\stackrel{\text { def }}{=} \nabla_{\delta} L\left(z_{\text {test }}, \hat{\theta}_{z_{\delta},-z}\right)^{\top}\right|_{\delta=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, train lossH \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) , -y_{\text {test }} y \cdot \sigma\left(-y_{\text {test }} \theta^{\top} x_{\text {test }}\right) \cdot \sigma\left(-y \theta^{\top} x\right) \cdot x_{\text {test }}^{\top} H_{\hat{\theta}}^{-1} x, influence functiondebug training datatraining point \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) losstraining pointtraining point, Stochastic estimationHHHTFO(np)np, ImageNetdogfish900Inception v3SVM with RBF kernel, poisoning attackinfluence function59157%77%10590/591, attackRelated worktraining set attackadversarial example, influence functionbad case debug, labelinfluence function, \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right) , 10%labelinfluence functiontrain lossrandom, \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right), \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right), \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top}, H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}), Less Is Better: Unweighted Data Subsampling via Influence Function, influence functionleave-one-out retraining, 0.86H, SVMhinge loss0.95, straightforwardbest paper, influence functionloss.
Extra Snap Benefits Ct January 2022,
Benchmade Socp Alternative,
Eloy Police Log,
Articles U