Come hang with the bad boys of natural language processing (NLP)! Jack Morris joins Daniel and Chris to talk about TextAttack, a Python framework for adversarial attacks, data augmentation, and model training in NLP. TextAttack will improve your understanding of your NLP models, so come prepared to rumble with your own adversarial attacks!
Jack Morris: Yeah, absolutely. It might help for me to talk real quickly about that systemized [unintelligible 00:32:25.03] the components, and then I can explain the most common use cases… Because obviously, you can pull out any one of the components and use them for your own purposes. So one thing that we really focused on in TextAttack is trying to make it work out of the box. For example, those counterfitted word embeddings, instead of going to this website, downloading it, unzipping it, moving it, finding out how to load all the data, you just import TextAttack and do “textattack.the-class” and just initialize it and it will download everything for you… Which I think is really cool.
If you guys know about Hugging Face Transformers - a lot of the TextAttack stuff is built around transformers and tokenizers, and now this dataset loading library called nlp, which I’m very grateful for… We kind of tried to follow the same model. So instead of having all these files you manipulate yourself, you pretty much just reuse other people’s, and it saves a lot of time.
The easiest or probably most common way that I would imagine people use TextAttack down the line is for things like that, for embeddings. Or another very common thing is sentence encodings, which is something I mentioned at the beginning of this talk. There’s so many different methods for taking a sentence and encoding it into a fixed-length vector; whether they’re very effective or not is a question, but they’re useful in a lot of situations…
So one thing TextAttack has done is just sort of abstracted them into classes that work by themselves, so you could just – for example, if you were doing some project… I don’t know, you wanted to look at a bunch of Airbnb reviews and cluster them based on which ones were similar, you could just import TextAttack and then just call [unintelligible 00:34:13.05] and then give it the list, and it would just do it for you, which I think is pretty valuable.
I’ll tell you what the components are very quickly. There’s four, and we have our own names for them, which I think increases the learning curve a little bit… But there’s some benefits, I think, to having around terminology. So it’s all based around this idea of the NLP attack as a system, which is taking the text input, looking for changes you can make to it, making sure those changes are acceptable, and then whenever you have decided you fool the model, you stop.
The first component would be what we call the transformation, which is taking an input and changing some of the words or characters. One transformation would be substituting words with their counterfitted word embedding neighbors. Then once you do that transformation step, there’s also this idea of a constraint, which is trying to make sure you didn’t make any mistakes.
A common constraint is use a sentence encoder. A popular one is called the Universal Sentence Encoder, which is by some folks at Google… And you encode the original input and now your potential adversarial, and make sure that the sentence encoder also says they’re very similar. It’s basically like a sanity check to make sure you didn’t change the meaning, or change too many characters, if that’s what you decide…
And then there’s two other components. So we had the transformation and the constraints… And you have to define your notion of whether you fooled the model or not. A common thing would just be change the classification output, or change the classification output to a specific class. Those would both be examples of what we call the goal function.
I think a really cool one that I wanna explore more in the future is with sequence to sequence models, like a machine translation model. Your goal might be to take the original output translation and change as many characters as possible.
[00:36:14.08] Say you’re translating a sentence into French; you would have your original translation, and if you could substitute a word from the input with a synonym, and then it produced a translation that was totally different, even just in terms of characters, or its Blue score, that would be pretty telling, and probably very bad for your translation system… So that would be another goal function, would be trying to minimize the Blue score.
And then the last component is called the Search method. That’s basically like if you have the input and you have all these transformations, how do you decide which one to keep? Which is important, because if you just tried all the combinations – I mean, if you have an input of ten words and each word has 50 neighbors, you end up with 50 times 50 times 50 possible substitutions that you might wanna combine… So the space grows exponentially very quickly, so you have to come up with some sort of greedy, or approximate heuristics for doing that. That’s what we call a Search method.
So you can combine those four things into an attack, in NLP what we call an attack, which is just a search for adversarial examples that meet the constraints and fool the model as defined by the goal function. But there’s some really cool other things that come off of that. A big one that I’ve been talking to people about recently is data augmentation, which is also a very under-researched field in NLP; it’s another thing that is pretty commonplace in vision, and almost everyone does it… You know, if you wanna train a state of the art vision model on CIFAR-10, or ImageNet, or some other dataset, you’re gonna do some sort of augmentation to change and increase the size of your dataset.
With TextAttack, if you have this transformation which can find maybe semantics-preserving changes to your input, and you could add on constraints, which make sure that they preserve semantics, then you can end up with some pretty good tools for data augmentation, just from those two things. And since we’re trying to implement more components, that would hopefully grow the list of potential augmentation modules as well. So yeah, that’s something I’m really excited about, just data augmentation.