Author Description

As machine learning (ML) systems become ubiquitous, it is critically important to ensure that they are secure against adversaries. This is the focus of the recently developing sub-field of adversarial machine learning, which aims to analyze and defend ML systems. In this thesis, we uncover the crucial role that data geometry plays in adversarial ML. We show that it helps craft effective attacks against real-world ML systems, develop defenses that are robust to adaptive attacks and is instrumental in deriving fundamental bounds on the robustness of ML systems. Our focus is mainly on evasion attacks carried out using adversarial examples, which are maliciously modified inputs that cause catastrophic failures of ML systems at test time. The \\emph{first part of the thesis} deals with black-box attacks on ML systems. These are attacks carried out by adversaries with only query access to the systems under attack. Nevertheless, we show that these are as pernicious as attacks with full knowledge of the system, demonstrating that adversarial examples do indeed represent a serious threat to deployed ML systems. We use data geometry to increase the query efficiency of these attacks and leverage this to carry out the first effective attack on a commercially deployed ML system in an ethical manner. The \\emph{second part of the thesis} considers the use of dimensionality reduction to defend against evasion attacks. These defenses are effective against a variety of attacks, crucially including those with full knowledge of the defense. We use Principal Component Analysis to carry out this dimensionality reduction and also propose a variant of it known as anti-whitening, both of which improve the security-utility trade-off for ML systems. The \\emph{third part of the thesis} steps away from the attack-defense arms race to develop fundamental limits on learning in the presence of evasion attacks. Our first result uses the underlying geometry of the data and the theory of optimal tr