Practical AI Ethics

Welcome to another Fully Connected episode, where Daniel and I keep you fully connected with everything that’s happening in the AI community. We’re gonna take some time to discuss the latest AI news, and we’re gonna dig into learning resources to help you level up on your machine learning game.

My name’s Chris Benson, I’m a principal AI strategist at Lockheed Martin, and with me as always is Daniel Whitenack, data scientist with SIL International. How’s it going today, Daniel?

It’s going pretty good. It’s been an interesting week. I went to visit a couple of family members, and of course, traveling during Covid time is somewhat interesting, so… Navigating those waters and trying to stay safe. It’s been an interesting week. What about you?

Same here. I’ve been trying to kind of reassess everything. I’m in Georgia, as long-time listeners will know… And definitely our numbers in terms of transmission are going through the roof on Covid.

Yeah, it’s kind of all across the South at this point, I think…

It really is. Big questions about like “How we’re doing schooling going forward?” in different local governments and such, trying to figure out what they’re gonna do… So it’s interesting times. This whole year has been just quite a crazy year, in a lot of different ways.

Yeah, and of course, still a lot of things going on in our country related to politics, and also justice for marginalized groups, and…

Absolutely.

…various injustices that are trying to be righted… There’s a mix of encouraging things, really sad things, confusing things and stressful things, all at once…

Yeah.

Which I guess does interface with what we had thought about talking about today. I don’t know if you wanna get into that a little bit, about what your thoughts were in terms of what to dive into related to AI today.

I will… Yeah, I will. And I know I’ve alluded to it in passing on a number of episodes, but one of the things I do in my own job is I lead AI ethics for the company that I work at, which is Lockheed Martin. And this show is not about specifically Lockheed Martin. I may mention it a little bit in terms of things that have been approved for public release by our communications department, but that’s gonna be very little.

I really wanted to talk about the fact that I spent a lot of time focusing on the details of implementing AI ethics, both internally and externally, in our own operations, and it is a topic that I get asked about more and more often these days, in terms of people looking for guidance on how to do that in their own organizations. And in my own organization we are fairly far along that process relative to most other organizations out there, I think… And the feedback that I’ve gotten is such.

[04:27] So I wanted to really kind of start with what most people that have delved into this topic are probably most familiar with, which is principles that are being adopted by various organizations, kind of explore what some of those look like, but then talk beyond just principles of AI ethics, and talk about how that affects your organization and your operations when you’re trying to take the principles that you have chosen and move them through your organization from end to end, so that they become part of what you are. I thought that would be a fun thing to talk about. Are you up for it?

Yeah, I’m definitely up for it. It’s probably not hard to convince people to think about AI ethics in terms of some of the things that we’ve seen in our world of recent times, especially usage of things like facial recognition and other things, and by governments and other institutions that are trying to discriminate against certain groups, or something like that… So that’s sort of a very – AI ethics comes up very easily in that conversation. I think that there may be though some people out there that are practitioners, and maybe let’s say they’re using AI to optimize their infrastructure, or they’re using AI to plan their marketing campaign, or something like that.

It might be a little bit less clear why AI ethics and the statement about your principles around AI ethics and how you approach AI is relevant to those scenarios. I don’t know if you have any thoughts on that before we just jump in… Like, if you’re doing AI, why it may be important that you start to consider some of the ethics behind how you approach your AI development, regardless of the specific use case, or are there some motivations that everyone should be thinking about?

Yeah, I think so. What we are calling AI today, which in a practical sense is probably the area – you have a lot of dispute, obviously, on what AI is out there. And every organization, and really every person that’s in this field kind of has their own definition. So that muddies the water a little bit. I think if there is probably one area that most people would agree is AI as we record this in 2020, it’d probably be the field of deep learning and the other adjacent fields that are inclusive or tied to that.

I think one of the notable aspects of that is that they are non-deterministic systems. That means that you’re not guaranteed that a given input is always going to yield exactly the same output. You can have variability in there. And as a matter of fact, obviously when we train a neural network, we introduce randomness on purpose, for purpose of getting to a better model, so it doesn’t run into some of the technical issues with training… Which is kind of outside the scope of this conversation.

So I think given that you have these capabilities, but it’s in the early days, and we’ve already talked in previous episodes about how AI will evolve probably fairly dramatic in the years to come… But as we’re looking at it right now, today AI as it’s mostly deployed it’s not aware of right and wrong. It’s not aware of values that you’d want to have govern your actions and your thinking. You don’t have a framework that ties ethical considerations, moral considerations, value considerations into the output of an inference.

[08:04] And so you could think of AI ethics as a new field, which probably in the future will stop being this thing onto itself and become part of artificial intelligence development, just like in other fields they have ethical considerations that are part of the core field itself. Right now we’re building that from scratch, and so we’re kind of talking about it as a thing onto itself. That’s really why we’re having to do that, is to ensure that we get the outcomes that we are shooting for, that we are anticipating, versus the unexpected outcomes that can cause real problems. We’ve certainly seen quite a few of those over the last few years.

Yeah, I think that there’s definitely things that are unexpected in terms of how AI behaves, that a problem set might not in and of itself appear like there’s ethical issues with a certain problem you’re trying to solve with AI, but they can crop up in an unexpected way. I gave the example of optimizing infrastructure - while you may not be discriminating based on race or gender when you’re optimizing infrastructure, there are certain things about the way that we handle infrastructure as a company that are maybe energy-related, and that sort of thing, that might have some implications on the world at large… Whereas there’s other scenarios, especially if you’re doing advertising campaigns online for hiring at your company, or something like that. That of course is something that is very fraught with ethical and fairness issues in terms of how you’re presenting job postings to people, and all of those things…

So I think there’s certain problems that the ethical issues are more obvious, and certain problems where they’re less obvious… But I guess having principles sat down for how you approach AI problems at your company - the purpose of having that set of principles, from your perspective, is it to help tease out what ethical issues might be at play in a certain problem set? What’s the real purpose behind having a set of principles that you adhere to as a group of AI developers? What’s the main advantage of that, and how that might play out?

Yeah, I think principles have been where the majority of the conversation has been so far. There have been a lot of press releases with major organizations adopting and publishing their principles… Those really are intended to explicitly establish what your values as an organization are, and the kinds of things that your organization is thinking about and cares about to ensure that if you’re operating the way you think you should be, then you should be in full compliance with what those principles are. And if you operate in such a way, either on a regular basis or by exception, where you feel one or more of those principles that you’ve adopted are violated, then it’s a sign that you’re moving outside that zone of normalcy for you…

So really the principles are trying to capture that organizational ethic… But it’s not enough. You’re gonna have to take those principles and push them down through your operations in a meaningful way, so that you can execute. Otherwise you end up having a set of principles that sound very wonderful, but don’t have a meaningful connection with what you’re doing day-to-day. So it’s very important that you start with that, but then you find your way down below that. But nonetheless, we should start with principles today, because there’s been a lot of thinking in that area, certainly.

[12:03] Sure. So if you’re an organization that’s using AI, what’s a good way to start developing a set of principles or guidelines that guide your AI development team, or your AI development department, or whatever it is…? What’s maybe a good way to put a flag on the ground and get started?

Well, I think what a lot of organizations have done is to collect a representative group of people from their organization, that represent all of the different functions that that organization tries to address in its operations. There could be product teams, and service teams, there could be representation from your legal team, or your HR team… All of these have a unique voice, and so you’ll bring these people together, typically, and you will have a set of conversations on looking at operations; you’ll tend to look at use cases and say “What does this mean for us? How do we use technology? How do we think we’re using AI? What do we expect to be doing in the future with AI, as far as what we understand it to be today?” and you’ll collect a set of cases to go through and to dive into that…

And from that process, and bringing all those different perspectives - and diversity is so important in this process… That you start kind of working together to agree on a set of things to address. Use cases is a good word for it; scenarios, if you will. It can be at a functional level, it might be an internal thing, like “How does AI apply to an HR system?” or it might be “How does it apply to the services and the products that we offer our customers, and how does it affect our customers?”

In doing that, you really say “This is what we do”, and you start identifying what would you feel right about doing with your customers and with your employees and such, and you’re really focusing on a set of values that will describe what a good process, a good operation looks like there. The one that you want. The outcome that you would like to achieve. And you have to narrow those down to a set of specific words, that are the principles, with some kind of definition around those, that describe what it means to you and your organization. Some version of that is what most organizations that we’ve seen have addressed.

Yeah… I think maybe I naturally struggle with this process a little bit, because it seems a little bit to me like - you know, when people develop vision or mission statements for their organization; it seems to me like a similar sort of process, and I have trouble with that, maybe not surprisingly, because named this podcast Practical AI… And I try to be a very practical person, and so a lot of times those seem so far from practicalities that I struggle understanding how they play out in everyday scenarios. I don’t know if that makes sense.

No, it does. And I think that a lot of people feel that way. In particular, given the fact that AI practitioners tend to be very down-to-earth people; they’re engineers, they’re data scientists, they’re people who work with the tangible side of technology to achieve something… AI principles can feel very wishy-washy, so that’s one of the challenges. But ultimately, to get those people satisfied, you’re gonna have to start with those principles, you’re gonna have to figure out “What does that mean for your own policies?” The policies that govern how you operate. How you’re gonna govern that once you reach that point. How do you know that you’re in compliance with what you’ve decided that you want to do and govern? And then what kind of tooling and workflow integration are you going to implement to ensure that all of that happens and that it can be done by people who are not necessarily experts at ethical AI.

[15:55] So Chris, you were getting into some details about how companies can start getting their AI principles together, and also how to connect that in a practical sense to day-to-day workflows… I was wondering if we could maybe take an example. Let’s say that fairness in terms of “We’re a marketing company”, or a hiring company, and fairness in terms of gender and race is something that is important to us, and we have some principle around that… Where might you go from there in terms of making sure that an ethical principle actually filters down to the people that are doing the development and building products?

It seems like maybe that’s a place where a lot of people get hung up.

If you’re talking about fairness being one, and by way of example, Google has five principles, and one of those principles is fairness that they’ve chosen as one of the descriptive words for how they’re operating… So really in the beginning it’s deceptively simple; you think “That’s fairness. I know what fairness is.” Well, not everyone does, and not everyone actually will agree on that, and that’s pretty true all the way across. There’s a lot of ways to define it.

One of the first ways that you might do that is if your organization - and this tends to happen probably with larger organizations more often - they may have a formal description about what fairness is. If it’s a large company in particular, they may have an ethics department at the company. Or if not, maybe a smaller company, the HR department will take this kind of thing on. And being able to tie the principles that you’re adopting for AI back to what you already believe, and particularly what you’ve already documented, is really important. Because otherwise you end up having these AI principles that are kind of this beast onto themselves, that are separate from everything else. But it’s pretty crucial that you end up with principles that apply directly to your organization in a very organic way, and fit in with what you already have.

Yeah, I agree with that. I think maybe this is a tension that I often feel when I get into these vision and mission statement conversations… And maybe these conversations are similar around AI ethics and principles; there’s also a tension between who you are now, and who you want to be as a company, and kind of casting that vision out. And all of the things you’re doing now - maybe they aren’t consistent with the company that you envision. You’re wanting to put these in place partly to push people to do “good” development work… But then also they have to be integrated in to the sort of natural philosophy and values that already exist within your organization, otherwise it’s gonna be hard for a company to implement then, I think…

So I think that there is that tension. You can’t sort of from the top down view just impose a new set of AI principles that just kind of shock and are totally coming out of leftfield to everyone in the company… But at the same time, you want to put some things in place to maybe push people to do certain things in a different way. I don’t know if you’ve seen that sort of tension play out, but…

Yeah, I have… It’s kind of that aspirational tension of where you’re at today vs. what you have been thinking you wanna go to… And when you’re first designing your AI principles - and I say “design” on purpose, because you are designing your way into it, you don’t just pick them - it really has to feel authentic, not only to your own teams, but also it needs to feel authentic to the people that your organization will interact with.

So without that sense of authenticity, like the principles are in alignment with existing policies - you know, we’ve talked about fairness as that example that Google has, and it’s notable that that is one that you would typically find. Fairness was also selected, for instance, by Microsoft, as one of theirs. And as you’re doing that, you need to ensure that whatever HR policies, for instance, that you have around fairness of what your employees might do internally, that that is reflected… Because with AI being an enabling mechanism, an enabling tool that you’re typically going to apply to existing things that you’re trying to do to enhance them, the output from your AI needs to comply with what your idea of fairness is. And fairness is mostly the same (there’s a general idea), but it might mean a little bit of something different to Google from Microsoft, at least in terms of their official policies.

[20:35] So it feels very fluffy, I know, at this level, but if you don’t go through this, then when you get to the more – pushing it down to the practical levels, you’re gonna struggle. This really has to represent your organization’s values within a diverse context it needs to be as real to your HR person as it is to your salesperson, as it is to your engineer. So all of those people need to be able to touch those principles and say “I know what that means to me in the context of my organization.”

So in starting out with getting a set of AI principles in place for your organization, since I’m mainly involved in practical development work, I tend to take the viewpoint that – I think it was Kelsey Hightower in a talk that I saw, I always remember him saying that “Good developers copy, and great developers paste…” So looking at some examples that people have already put a lot of work into out there I think is a good thing.

While we were talking, I was looking up some papers on the subject… There’s a paper from a group at the Chinese Academy of Sciences, and they took sets of AI principles from Academic organizations, and governments, and industry organizations including Google and Microsoft and IBM and others, and they did a bit of analysis on them. So they did some Word2vec analysis on them. It looks like kind of categorized the statements into a set of topics that occurs very frequently in these subjects, or in these statements of principles… So that’s actually very helpful to me, because I think looking at what people are valuing in their principles might be a good point to get you thinking.

So these topics include Humanity, and that has to do with keywords around being beneficial for humanity, and human-centered, and human-friendly. A second topic is Collaboration, having to do with things like partnership and cooperation and dialogue… There is a Share topic, which has to do with share and equality, inequity, inequality topics.

There’s a Fairness topic, which has to do with bias, discrimination and prejudice. A Transparency topic, having to do with things like explainability and audit and tracing. A Privacy topic, a Security topic, a Safety topic, having to do with controlling risk in human control… An Accountability topic, and then a topic around artificial general intelligence, or super-intelligence, and that sort of thing. So I don’t know, is that consistent with what you’ve seen in various statements of AI principles?

I think so. I mean, we see so much commonality across different organizations’ principles that I think it is fair to stand on the shoulders of giants, if you will, and benefit from their own work. I think there has to be a point where if you just adopt and move it in, it doesn’t have a lot of meaning for your organization… So there’s a point where you’re applying it to your own internal policies, your own way of doing things - you have to customize it there to your own organization, so that it is meaningful… Because otherwise it’s just a set of words written down, and – you know, once you’ve implemented these, you should be able to have a common meaning through your organization, that understands it, and from even the same word (like we mentioned “fairness”), from one organization to another, it may mean slightly different things to them based on what their worldview is as an organization, and the employees that are working there, trying to affect what their operations are… So you can start by borrowing from others and stealing from others, if you will. I think that’s perfectly reasonable. But at some point you do have to make them your own.

[24:21] A good starting point would be to take some of those bigger topics… Let’s say privacy and security across a lot of these statements, I think. But I guess what you’re saying is that in privacy, you know, in my organization, SIL International - we have to decide “How does privacy impact the things we do with AI?” and how could the things that we’re developing that are AI-driven influence privacy in positive or negative ways for our specific users or customers or that sort of thing. Is that kind of what you’re saying?

I think so. It’s interesting, one of the things that I’ve found is that choice of words maters. If you look across a number of different principles as outlined by different organizations, they’ll have different numbers of principles, they’ll have different words that they use to describe… And it may be that one organization uses the word “privacy” and another organization does not; that doesn’t mean that privacy is absent from their principles, but it may mean that they’ve taken the principles that they believe to be privacy and divided them into maybe two other of their principles, because they see that as two subsets. It’s really dependent upon how you see the world and how you believe you’re interacting with it, and the language that you use to do that, beyond just AI ethics. The language of your own operations.

Yeah, and I guess depending on the industry that you work in, some of these might be more important than others. Like if you’re an actual company that works in the HR/hiring space, then some of these are gonna become extremely important… Whereas if you’re a software as a service company, or a developer tools company or something like that, then other of these might be immediately important.

A lot of these intersect with people no matter what company they’re working in. A lot of these, when I read them, I think could be interpreted – like, depending on who’s reading them, they could be interpreted in different ways… How do you make sure that what you’re going after with your AI principles actually trickles down to the work that’s going on in your AI development teams? Is there tooling around that, or is it mostly an education thing, or what’s your thought there?

It’s gonna be both, but what you have to start doing is you have to connect on the assumption that your operations, whether internal or external, are government by some (essentially) policies internally. You really have to connect the principles to the policies that you have, because those policies are representing the reality, or at least an ideal reality the organization is trying to achieve. And so until you can map principles to the policies that you already have in place and figure out if you have gaps and need to create new policies or change existing policies, that’s the first step - saying “I understand how the principles that we’ve said identify our values map out to the policies that govern our operations and our functions.”

So that’s really the first step. And then you’ll follow that up with “How do we know that we are in compliance then? What’s the mechanism and how do we govern that? If something goes awry with the way that we’re thinking about this, how does our organization handle that and process that?” Like you would have in anything having not to do with AI ethics - if you have something in your operations that is not compliant with your expectation, how do you handle that? You need that for this as well, and then you need to connect that to the people that are doing the actual work. It needs to fit into the workflow, and it needs to fit into the tooling… And if it doesn’t, or if that’s not an easy thing to do, then you need to figure out what that delta is and figure out “Do I need new tools? Do I need to adjust existing tools?” which has a lot of variability based on “Are they vendor tools that we’re paying for and we’re limited to the features that the vendor offers us? Is it stuff that we’ve written for ourselves? How do we know that and how do we get these to filter in at the workflow level in terms of being productive?”

It’s quite a long process, actually. I would say it starts with principles, and that’s what most people think of now… But once you’ve done your principles, you’ve basically just gotten to the starting line.

So Chris, as a practitioner, when I hear you say the word “governance” and that sort of thing, immediately I have these feelings inside of me that are just sort of naturally negative…

[laughs] I’m not surprised. A lot of us do.

Yeah, I think it’s sort of natural for a developer to have that feeling, because it’s like “Oh, governance…” Or a natural first thought might be that governance in some way, whether that be over certain data, and access to data thing, or whether it be ensuring compliance to certain things - that naturally means slow, I think. To most developers it’s like “Oh, this is going to complicate my workflow. It’s not gonna allow me to get anything done, and it’s just gonna be a slow process.”

So I do think that there is some validity to that, in that in order to check some of the things that should be checked in terms of an AI application, you’re gonna have to spend some time on those things though… But at the same time - I forget which guest it was; I’ll actually have to look through our transcripts to figure out who it was… And maybe it was just a conversation I had at a conference, but the idea was that doing good data science, or doing good AI development, and “good” meaning in an ethical sense; so being an ethical data scientist or being an ethical AI developer actually works to your benefit, development-wise, in the long run.

[32:16] One example of that is thinking about – you know, when I read that list of topics in that paper that I was mentioning, one of them had to do with auditing, and tracing, and that sort of thing. So basically understanding “This data was used to train this model, which output this prediction, at this time, based on this data from my user.” So all of that sort of auditable and traceable - there’s that data lineage.

So if you put the work in – and it’s extra time… But if you put the work into putting tooling, whether that’s just some good logging, or whether that’s some great – you know, there’s several products now that allow you to trace experiments, and that sort of thing… But if you put the work in to implement those solutions, then actually it does help you in the long run, because as you’re doing your own training, being able to trace what hyper-parameters you selected and trained on before, and what data you used before to get these certain numbers, it actually helps elevate your future work, because you have a better understanding of what you did in the past, and you can track things much better over time.

So I think the argument is that, you know - yes, it’s faster to just spin up a notebook and do the thing and export the model and just copy-paste it over to somewhere and run it somewhere, in a totally non-traceable manner… But even outside of the ethical side of things, you’re kind of shooting yourself in the foot at some point if you plan on doing that over and over again. So there is some of this argument that, you know, it’s slower to implement some of these things, but once they’re implemented, I think you can have confidence that actually doing good or ethical work will actually work to your benefit development-wise as well, in many cases.

I also think of a scenario… When I teach, I often – and I think I mentioned this on the show maybe a time or two… Like, if you’re developing a self-driving car in Sweden, and you make a great product and then you wanna export it internationally, and you ship your car over to Australia, and the first thing it sees is a kangaroo, and it runs off the road and kills its passengers - that could have been solved by thinking about the biases in your data, and the target markets that you’re going after, and what’s represented in your dataset, and all of that… So these sorts of things can work to your benefit, development and product-wise, over time.

I totally agree with that. And I actually wanna point at a distinction… I know at the very beginning of when you were talking, you said “governance”. I would actually argue, having listened to you just now, that much of what you were talking about was really more on the compliance side. The two go together, but how do you know that as you’re going through something, that you are in compliance with what those values are, without slowing you down? I think I would start with the fact that this is a great place where you can use, in a lot of cases, technology to help you get there… And then on the process side, the governance side of being able to handle when things don’t come out as you expect. And you can go back and do that analysis.

When you were talking about tooling helping you, and being able to know what data was used when, as an example, what came to mind as you were saying that was we’ve had Pachyderm on the show, and if I recall, that sounds like some of the capability in the file system on that solution… So when you were implementing AI ethics, that might or might - there may be others as well, but that might be one of your ways of trying to automate some of the abilities to being compliant, so that you are not necessarily just slowing yourself down with a lot of manual intervention with this. And you have a governance process, so that when unexpected outcomes do arise - going back to your kangaroo example - you can kind of go back and figure out “What went wrong? How did it deviate specifically from what you were trying to do?”

[36:25] I know for me I look at – we’ve talked about a couple of industry principles… I tend to work a lot with the U.S. Department of Defense principles, which are responsible, equitable, traceable, reliable and governable, and there’s verbiage with each of those that our listeners can go look at… But from the standpoint of your kangaroo example, you can say “Well, right off the bat, that would at least be a violation of responsible and reliable”, and there’s specific language that tells us what those two terms mean in this context.

So as you’re looking at that problem that you’ve just described, then you go back and you start tracing that back - and traceable happens to be another one of the five, in terms of “What goes back to allow you to assess what went wrong? Why did that model give you an outcome in that situation that was not as expected?” And that’s what governance is right there. That is going back and saying “How do we fix the problems that we’re gonna have?”, because we certainly will have those. This is why you go through this whole process, it’s to ensure that as you go forward, things do get better and better. And better means “Outcomes that you expect”. I don’t know if I answered everything that you posed to me at that one…

Yeah, it’s good points. A lot of what I’m thinking of with this is sort of the trickle-down… And I think part of that was motivated by some of the teaching I’ve done on the subject. Oftentimes a question – when I’m entering into this sort of discussion, a question comes up about “Well, if including a sensitive feature in my model (gender, or whatever it is) improves the model performance in terms of the accuracy on my test set, or whatever it is, why would I not include that feature in the data? Or why would I be careful about how I treat that feature in my data?” Isn’t this sort of accuracy what I’m most concerned about?

It might be. The answer is “maybe” on that. Go ahead and finish, because I didn’t mean to cut you off, but I’m gonna come back to that.

I agree with you, “maybe”, but I think also that – so there’s a paper that I was reading before our conversation today on the role and limits of principles in AI ethics towards a focus on tensions from a group at Cambridge… And they were talking about some of these natural tensions or trade-offs when we’re thinking about ethics, and one of those was this using data to improve the quality and efficiency of services, versus respecting privacy and autonomy of individuals. And there’s other of these tensions, but I think what it comes down to for me is that as AI practitioners, the things that drive that sort of performance and accuracy for us is data that has been generated in the real world. We parameterize our models based on observations that have happened and been recorded in data from the real world; and the bottom line is that our real world is broken, in many ways.

There are very bad things that are happening in our real world, there is over-representation and under-representation that happens in our world, and depending even on just where you get the data, or how you gather it - you know, you’re never going to have the ideal dataset. So really what it comes down to for me is “Are you trying to get the best performance you can on your sample of test data?”, which is not representative of the whole world anyway. Or are you trying to ethically create a model that performs very well, but also at the same time does not discriminate or treat unfairly or use data in an unethical way. I think that those things have to be paired together. From my perspective, it’s not right to just take the performance at the expense of considering those other things.

[40:20] I would agree with that… It’s interesting, we’ve used privacy as our example a bunch of times, and in some contexts you’re looking and saying “Well, if I had a performance capability that I can add in, but I do that at the expense of privacy, do I wanna do that?” That is exactly why you want to have those ethical principles in place. And while on first blush that may seem self-evident to most of our listeners and to us as we sit here – but I’ll give you an example of where it may not be as evident, or at least you have to think about it just a little bit. There are cases where privacy is not necessarily one of the things that you’re trying to achieve.

An example of that - and this comes from obviously the larger world in which I work - is if you have an intelligent satellite that does intelligent surveillance and reconnaissance, and its purpose is to go over to a foreign nation and get as much intelligence as you can, that is very distinctly not something that is putting privacy as one of your values. You’re basically [unintelligible 00:41:19.02] So in that case, in that particular example that you gave us, the performance versus privacy, performance wins every time because of the nature of what you’re trying to do, and the governing principles that you’re using.

But if you were to say “I work for one of the large tech firms (Google, Microsoft, Amazon, something like that) and for performance am I willing to give up privacy?”, well even if you personally were willing to give up privacy, your customers probably aren’t going to agree with you… And to do something like that would probably be devastating to your business, particularly if you get caught doing something that your customers are not okay with… And we’ve seen examples of that many times. Facebook is another one that comes to mind as I talk through that.

So you’re literally looking at the same set of value judgments, but coming out in very different ways, depending on how you’re choosing to arrange those values in your principles. So the point I’m making is that having those principles really matters, and the context of what your organization does with those principles really matters, and you can have very, very different outcomes based on those.

Yup. I guess one thing I should say is we’ll put links to some of the main industry AI principles that Chris and I have reviewed in preparation for this show. We’ll put a link to those in our show notes, along with the various papers that we’re discussing.

One other paper that drew my attention was talking about “Principles alone cannot guarantee ethical AI.” And now that we’re kind of getting down into how this plays out, I think that that was a very interesting paper that maybe people should take a look at. It appeared in Nature… And it’s talking about where we might fall short in terms of the AI principle that are coming out. One of those might be this sort of trickle-down effect, and creating sustainable pathways to impact an accountability in terms of ethical AI, which I think comes down to more of an implementation thing… But they also kind of bring up this interesting idea which I’ve also heard other people talking about in terms of AI, and I don’t know if I fully developed a thought process around - but they talk about licensing AI practitioners similar to like a doctor would have a license, or something, because their profession is very risky in terms of what they do… Or maybe a pilot license in terms of when you’re flying people around, there’s an inherent risk in that… And there are risks in terms of the AI that we developed, especially if we’re talking about cars that are autonomous, or maybe systems that are implemented in airplanes, or whether you get insurance or not, and that’s driven by certain algorithms… I don’t know if you have any thought about that…

[44:16] I do.

It kind of scares me a little bit, because I don’t know if I want to get a license, but I get where they’re coming from. There’s a lot of people that have put a lot more thought into this than myself.

Yeah, so I think I would note that we are still in the very early days of artificial intelligence… And the world in which we’re operating right now is largely devoid of legal frameworks and regulatory frameworks. Now, before people start objecting and naming their favorite one, there are some, and I’m acknowledging that… But compared to other areas that may be regulated, we are still trying to figure out what that means.

National and local laws, and international law is trailing far behind where the technical capability is. So I think that is one of the conversations that we need to have in the years ahead, is figuring out how to have the appropriate regulatory - whether it’s licensing, or whether it is other constructs that you may do. Our legal and regulatory frameworks are supposed to reflect our values in the world, and the various things that we have to do to keep people safe and protect certain rights… And we have largely not done that. Not only with AI, but with lots of different technologies in recent years. So there’s a conversation and an adjustment that really needs to be had in the days ahead, and I think that’s gonna be pretty crucial.

It’s funny, I know on this show we are either AI practitioners, or we’re at least minimally - by tuning in, you’re an AI enthusiast, in most cases. You care about this topic and want to do that, so it makes sense for us, our collective community here, to want to engage in this. But what I would actually urge you to do is because AI impacts so many people, and ultimately most of the people in the world can be impacted in various ways, you really need to bring the people into the conversation that aren’t necessarily as familiar with it as all of us are… Because it does impact them; they have a right to have a voice in it, and I think all of us – it’s incumbent upon us to explain these issues objectively, as scientist you may say, ourselves, to make sure that the general public understands what the implications of these things are. So we haven’t really in a meaningful way started that conversation yet. I’m hoping to see that in the next few years.

Yeah, definitely. I’m hoping for similar things. As we close out here this episode - of course, in these Fully Connected episodes with just Chris and myself we always like to share some learning resources… And maybe Chris has one that he wants to share, I’m not sure…

I do, actually.

Yeah. One that I wanted to share, which I’ve mentioned a couple of times on this show - it’s just a very practical piece of tooling, which gets to maybe some of these, but not all of the ideas that we talked about, but at least gets started, is the project from Driven Data called Deon… Which is an ethics checklist for data scientists. What I like about this is it’s just not a static checklist, it’s an actual Python project where you can embed a checklist in your Jupyter notebooks or in your Python code repository, and actually have that there right with your code where you’re developing your project… And it’s fully customizable; they have a sort of default one that you can use, but you can customize it for your own company’s ethics checklist, so that actually as you develop these things you can implement them in an ethics checklist, and embed that right in your project, so that every project - even if certain things in the checklist aren’t relevant to a certain project, you can at least check them off that you’ve thought about them or considered them in completing the project.

[48:08] That sounds like a good one. I know you’ve already noted that we’re gonna put links to a lot of these principles that various organizations have published out there… From a learning resource - and I’m gonna suggest folks do what I have done in quite a bit of detail, is go to half a dozen different published principles that are out there. Go to the Googles, Microsofts and others like that, and read what they’ve done and compare it, analyze between the two. Have them up on the screen at the same time, with their explanations, and try to discern why they might have chosen the verbiage and the particular ways of describing it that each has done. You’ll find a lot of commonality, and you’ll find some very distinct differences.

If you do that, you might throw in the Department of Defense one, because it has a different thing that it’s trying to achieve from a commercial entity. So it allows you to see a diversity of thought, that has a lot of common points, but allows you to kind of put some critical thinking around it. And then as a second step, after you’ve done that, choose what you think would be good principles for your organization and try to do a little mind exercise where you think about how does this work in your own organization? How do you get it from the principles into your own organization’s approach to work, and the policies that govern that, through governance and the compliance that you’re trying to implement, into the tools and the workflow integrations that your organization is engaged in. How do you get from that fluffy high-level all the way into something that directly affects your customers or whoever your organization is working for? So just going through that process is really educational.

And then there’s another one that I wanted to suggest… Not long ago, on episode #85, we had Stuart Russell as a guest on the show. He’s a Berkeley professor; he’s a legend in the AI community and has been doing this for decades in its various forms… And he’s written several editions of “Artificial Intelligence: A Modern Approach”, which is one of the key textbooks in this industry as it’s evolved over time…

He has also more recently written a book that’s intended for general audience called Human Compatible. Read that book, because it talks about some of the same issues and his own thoughts on how he would address it, and what he thinks is important on that. I know in that episode I thought that was a fascinating conversation. I thoroughly enjoyed the conversation with professor Russell. He really addresses AI that avoids harmful, unintended consequences and offers a path forward towards a future in which humans can safely rely on provably beneficial AI. And at the end of the day that’s what you’re trying to do with AI ethics; that is the outcome you’re trying to get to. So I recommend the book, and I hope that it will get people started on this path.

Excellent. Yeah, we’ll put that link in our show notes. Please let us know in our community online your thoughts around AI ethics and things that have been useful for you. You can find us on Changelog.com/community, and in Slack and LinkedIn and Twitter. Looking forward to hearing from you. Thanks for bringing your thoughts around this today, Chris. I really enjoyed the conversation, and I’m looking forward to talking to you next week.

It was good. Thanks a lot, Daniel.

Bye-bye.