Biases in AI, helping veterans get jobs in software, open science

So we’re joined by Camille Eddy with Girl STEM Stars. Camille, you opened up OSCON this morning, talking about cultural bias in AI - how we recognize it, how we deal with it… Give us just a quick synopsis, a re-run - don’t go over the whole thing - of your keynote and what you’re here to talk about.

Yeah, sure. Thank you. Happy to be here. This morning I got to talk a little bit about how we reflect on our own biases and how that is propagated into the technology that we produce; the importance of recognizing that AI has made mistakes in the past based on those biases, based on things that we can’t possibly know not to do, like faux pas… Like, categorizing black people as gorillas - that’s a really bad thing, right?

It’s really bad, right? Yeah.

So talking about those mistakes from the past, biases - but that we can’t fix them without reflecting on those… And then different things, like explainable AI is seeking to come in and understand why algorithms make the decisions that they make, and the importance of having more technology like that prevalent in the future of machine learning and AI in general.

We’ll definitely dive into those. Let’s hear a little bit about your story, how you came to be keynoting at OSCON.

Yeah, so I have been a really fortunate student, really, to be able to go about and do a lot of different things. I started in Idaho, that’s where I’m getting my bachelor’s degree; when I was in Idaho, I of course dealt with a lot of biases. I’m African-American, so one of the very few black people there… And it changed my story a little bit, and how I dealt with the people around me, and also what kind of opportunities I got.

As a student, I kind of came into the understanding that life isn’t the same, no matter how much you want it to be, and we all have our own biases. I started thinking about that more, and I was given the opportunity to actually talk about my experience.

From that talk, one champion in one place at my school, someone invited me to go to San Francisco and talk. That’s where talk kind of evolved; that was maybe over a year ago. So then I just kept going, because I have a mantra as a student - my mantra is there’s three rules.

[04:07] I like this…

You say yes to every new opportunity, you don’t do anything twice, and then you always make your accomplishments visible. So through that, I said yes to speaking in San Francisco, even though I’d never been in San Francisco; I was okay with that. And then I also make sure I keep changing… So now I’m at OSCON, which is just how it happened.

What was the last one, again? The last point…

Make my accomplishments visible.

Well, I’m happy and sad about this mantra. I’m happy because you said yes to us, but I’m sad because now this is the last time we’re gonna talk to you…

That’s right, you can’t do anything twice. [laughter]

Oh, we’ll just have to talk about something else next time.

I was gonna disagree as well, because I was like, “You have to come back.” [laughter]

I can do something else.

Okay, so you’ll break that rule. That’s the loose rule.

Well, I’ll come back and–

New conversation.

A new conversation with y’all. Yes.

New conversation. Okay, gotcha. Interesting.

So one thing that you mentioned this morning is about how representation matters and how you saw – well, just tell the story about the African-American astronaut.

Sure. So when I was 12, I was trying to figure out what I wanted to do, because that’s what 12-year-olds do… And I was home-schooled, and my mom gave us a lesson at some point about Mae Jemison, who was the first female astronaut for NASA… And I thought that was dope. But when she gave me the lesson of the first black female astronaut at NASA, Mae Jemison, that’s when I connected and said “Oh, I actually identify with her, and I want to be an astronaut.”

So having that representation with someone who directly and strongly identifies with me made the difference in my choices. So from there, my mom was like, “Okay, so you’re gonna have to do your own research, what you have to do to become an astronaut.” That’s where I saw that to become an astronaut you have to be a scientist, a doctor or an engineer, and I chose engineering, because I felt like that fit the best.

Then I eventually ended up at Idaho and I got some informational lessons in high school, did Space Camp, which was really cool, and I was like “I’m really into this”, and that’s why I settled on engineering.

Cool. So that brought you to engineering. Is the dream alive, the astronaut dream, or you just settled in engineering?

No, so the thing about becoming an astronaut is you can’t become an astronaut when you’re like 20, because you have to become the best in your field. So I feel like I’m on the way to becoming an astronaut, but it’s not gonna happen until I’m like way older, so… I’m young, in my twenties now.

Can you reflect maybe on some experiences you’ve had where representation was there and not there for you? So how you felt when representation wasn’t present, how did you feel about maybe exploring a role or being invited? And maybe the flipside of both of those.

Sure. Yeah, so the thing about people like Mae Jemison, who was the first black female astronaut - she did a first, right? And so in Idaho I did a first, as well; I was the first black female student to lead my students’ NASA research team. But one of the things that were different there was I had a lot of amazing mentors, other people who did it first… Like, Barbara Morgan was my mentor while I was in Idaho; she’s the first female teacher to space; she is a former astronaut as well. But I didn’t have any other black females to be my mentor… So people who have gone before me and said “Oh, I see how your journey is being different than everyone else around you, and this is why, and this is how you deal with it.”

So I was able to push through that, I was a first NASA research student; I did undergrad research, I led the team, which was, again, a first… And it was hard to have certain conversations with people. You know, you get into those eeky conversations about “You know, I don’t understand why you just blew up at me in the middle of the room, and didn’t see the fact that you didn’t blow up at anybody else… But it was when I spoke that you blew up.” Little things like that, that are just more cultural, that you wouldn’t necessarily question…

But they add up over time.

[07:58] But it adds up over time. Yeah, adding up in a way – there are other questions like “Where do you come from? What country are you from?” things that bother you, that they don’t realize, and they also don’t realize that it happens to me like ten times a day. So it’s like, yeah, you might think you’re the first one to ask me this, but I get this ALL the time. So just paying attention to little things that bother you and understanding my relation to that.

Then when I came and did some internships out of the state I did find black people to be my mentors… And they helped me realize that I’m not the only one, I’m not crazy, and that there are other ways to encourage the conversation.

Not being crazy is pretty important.

Yeah, it’s really important for your sanity!

I’m always looking for somebody to validate my feelings to some degree, like “Am I crazy…?” I ask Jerod this all the time, “Dude, am I crazy?”

Yeah, I don’t always say no. Let’s get back to the topic at hand, which is the cultural biases in general in AI… This is something that we’ve discussed a handful of times, because we focus on these things for Practical AI… But machine learning specifically, because you’re training a computer by example, like “Here is a set of data…” I’ve heard them describe machine learning as “a bag of bias.” Like, you’re basically taking everything that you gather and say “Here, learn this, and then I’m going to reuse the results that you’re learning based on…” - so it perpetuates a history.

It’s very perspective-driven, right? What you feed it is a perspective. It’s essentially its own bubble, so to speak…

So I guess the question becomes “How do we fight that? How do we deal with that?” What’s your take on the topic?

I think part of the problem is there’s humans in the loop, right? We’re basically helping AI codify our experiences and then represent that again… But sometimes what happens – we as humans have feedback all the time. We’ll butt up against something, say something wrong, and then we’re like “Oh, that was wrong. I need to correct that.” But AI and machine learning doesn’t always have that feedback in the loop, so it’s really important to figure out a way - and there are different ways to do it - to provide feedback.

I know Microsoft and Facebook have come out with their own bias toolkits for their artificial intelligence, that they said that it was very important for them to add.

The other thing I like to talk about is explainable AI, or ex-AI, which seeks to understand why an algorithm makes the decisions it makes… So instead of having AI be a black box, it becomes more transparent.

Another place to go look on the web for ideas around understanding why the biases exist and how to look at them is to look at the idea of transparency in AI.

I see. So that means displaying to the end user why – let’s say a recommendation engine for an example, because that’s one place that we see machine learning applied a lot… You know, why this particular thing is being recommended to me by Amazon - because it is based on a model, and it will actually just say “Based on this, this and this.” Is that what you’re saying?

Yeah, that’s a good example. Let’s do another one - another similar example is Facebook… Why do I see what’s in my Facebook feed? Is it because someone liked it, is it because I know commented on it? Is it because I’ve liked this thing before? Is it because somewhere back in the day I liked this particular page, and maybe I wanna unlike it, or is it because it’s based on my geographic location? Google also does this - they have like 21 or more different points that they look at when you do a search, like where you are located, what have you searched before, what kind of things have you bought, what are these socks and where are they coming from?

And it’s not just the end user that needs to know this, it’s also the developer. I think Facebook is a really good example of this again… Because the developers have some of those tools and information, they just don’t let us see it, which is up to them… But to release some more of those tools and have more transparency I think would help bring us along a little bit more.

From an end user’s perspective, I can say that I can trust AI more if I know the transparency point of it. If I know why you’re telling me this is important to me, that I can confirm whether that’s true or not, and help it even shape its future recommendations for me… Because if it’s inaccurate, I want it to know. So if everybody can somehow influence that – are you advocating for that? Or how do we shape those kinds of biases inside if AI in that case?

[12:19] Yeah, I think one of the things to think about is – this conversation has been happening for a long time, it’s just not coming into prevalence… And this is kind of what happens when you engineer products in a box, basically; when you engineer in one lab–

In a research lab.

Yeah, when you engineer in a research lab…

…with perfect conditions…

One set of scientists or engineers are like “Oh, this is great! This works for us! This helps our narrative. Our lives can work well with this…” So that’s another point that I make in my talk, is not only have we not been having everyone in the room when we’re developing it, we don’t have a lot of users in the room, different users to test this product.

And then on top of that, the whole world isn’t online yet. There are large groups of population, of human beings in other parts of the world that don’t have the access to the internet that we have, and if we’re making all these decisions based off press products or conversations we’re seeing online, we actually are missing a big part of that conversation… Because not everyone’s online.

So it’s about being transparent, being able to see those ideas and being able to control it, but then also about continuing to get that conversation pushed, deeper ingrained into our processes of how we develop our technology.

Who’s in control of this transparency? Who are the gatekeepers of the black box being transparent?

It’s literally every single person that walks into a startup, that founds a company - the non-technical founders, too - they’re all involved in those conversations. We’re developing - especially we’re at an open source conference - these things and building on top of it, on top of it, and we’re creating the legacy technology of the future… So literally, these should be the conversations we’re having the first time we put up an idea on the whiteboard… Like, “Okay, who’s not at the table? Who do we see not represented?” It’s really the individual people.

Larry Page, Sergey, even Tim O’Reilly - they were all individuals at one point, looking at their business models, looking at their ideas, and so it’s on that level. If you seek or aspire to be any type of entrepreneur, leader, manager, just someone in the room engineering a piece of code, you should be having this conversation, or thinking about it, or getting more people to talk about it.

So what’s the practical way to make it transparent then? It can be at the table, but how do you actually implement transparency?

So a couple different things… You have explainable AI, so making sure that you’re making it visible to other people… Just turning it inside out, your development process. We’re watching Facebook do this now, we’re watching them do “This ad is paid for by…” That’s a really good example. Really small, but it’s leading in the right direction.

Another version would be like when I go on my Twitter and I look at my Twitter analytics, I can see who’s liking my posts, who’s commenting on them, and also the impression footprint that it has… So maybe opening that up just a little bit more, like past just the idea of impressions.

I think to get that done you have to be able to convince the people who are making the product or the business decisions - or be one of those people - that this is valuable, this is worth their effort. So that starts with conversations, that starts with grassroots efforts…

And then also the reflection that we haven’t done it yet; we haven’t done half of the work that’s necessary yet. Not just to define the problem, but to create products, especially out-of-the-box products. They’re not existing yet… Except for those couple of toolkits that I mentioned, that don’t necessarily serve all of the ideas that I’m talking about, for Microsoft and Facebook… We haven’t been talking about this in an actionable way long enough, and really, at the end of the day, I’m a mechanical engineer, right? I’m here to help dip people’s toes in the water, and hopefully, as we start talking about it more, there’s really cool books, like The Algorithms of Oppression, that really lays it out… It lays out that case use about why it would be helpful to your business model to do it. And then also just more conversations with people who are using it and not finding a great experience.

[16:30] One great example - I got back to this all the time - is Instagram ads specifically, and the visceral reaction that Instagram users have had to those ads, so much so that people believe that Instagram/Facebook is listening to their conversations… Because the ads were getting good enough to where they will suggest something to you that you don’t think you’ve google-searched, you don’t think you’ve put into Amazon… You think you were just talking about it with your friend or your significant other, and all of a sudden they’re advertising it to you, and you have no idea why…

So people are convinced that Facebook is listening to them, like actually turning on their microphone. In fact, there was a great Reply All episode all about that, about whether or not it’s actually happening… And it’s not. They’re not doing that. But they’re applying AI and different other fuzzy techniques in order to make their ads so good - they’re getting very well targeted - that it creeps you out because you don’t know how they came to that distinction.

Now, if I go to Instagram’s head of marketing and say “Your ads are actually making people despise you and your advertisers because they’re so creepy, but if you were transparent about how you came to those conclusions…”, if here’s your ad and I think “Oh, how do they know I even needed toothpaste…?”

“We listened to your conversation…”

Yeah, “We got this because we were listening to you.” No, if they actually said “This ad is based on these things that we know about you”, then I would look at that and say “Oh, okay, that makes sense.”

That’s my point. I could appreciate you serving me in that way. I should be interested in that, but it might also creep me out, like “Stop knowing that stuff about me.”

Right, and maybe you even opt out, but my point is that’s the business case in that particular sense. Your ads will be more effective with the transparency added, because they’re actually being counter-active in their current form.

Yeah, I think so too. I think the transparency piece is a huge component. Netflix does this somewhat well… “We’ve recommended this because you watched X.”

“…because you liked this, or because you watched…”

And so they just give me one title, but I know who was in it, what the subject matter was, what genre of movie was it, what was its PG rating - was it R, or PG13? I can deduce all those things myself and do my own research, because I got at least the one thing they tracked me on to recommend this.

That’s why you never let your kids watch on your profile…

That’s true.

…because they’ll watch one episode of a cartoon, and then Netflix will be like “Oh, you must love cartoons”, and then all the recommendations are kids shows.

What’s also interesting is the use of just an IP address, and not a profile… Because there’s things that happen in a household or behind one single IP that doesn’t reflect every person behind that IP.

That’s true

So I may go and search “overstock”, or some brand for a new couch, or some sort of decoration, and that may be a present for my wife, and now she may know, because she’s getting advertised from her favorite brands, or something like that… Like, it’s kind of revealing. I want my secrets to be my secrets, so that I can reveal them on my own terms, you know what I mean?

[laughs]

You know, the way you could probably attack that directly is to start looking at recommender engines/algorithms (I think that’s what they’re called) and being like “Okay, this recommender algorithm… I’m gonna go back to see what its training model looks like, I’ll go back and read the papers, and then I’m gonna present a really specific argument to whoever made that.”

Is there a case where the recommender engine, as you say, is deemed somewhat proprietary, considering maybe the thought leadership of like “Here’s what we can connect to make assumptions?”

Yeah, I think Facebook has a big recommender engine. That stuff is definitely proprietary. That’s why we can’t see it. That’s part of the problem, that’s the wall there… It’s proprietary, we can’t tell you, we don’t wanna tell you, and therefore you don’t have the levers. If it was built on a more open source platform, then we probably would be able to go in there and finagle with it.

[20:11] In that case, we have to switch it, make it inside out, and say “Hey, now we really do want these.” Make the business case to the people; holler, scream, shout, pull hair, do all those things.

Open source for the win.

Yes!

So tell us what’s next for you? Where are you headed next?

Well, my main goal is to finish school, so I can get out… I love learning, but I don’t like academics, so I’m really excited to finish.

What’s the distinction there?

The distinction is being in the real world. I took a gap year in the middle of my academics, just because I felt like I needed it, and it has taught me a lot about where engineering is going, where machine learning is going… And even being at this conference, I was able to do that because I was on the break and I was getting involved and being aware of the conversations being had… I don’t know, I feel like – I’m probably trying to grow up too early, but yeah, I’m definitely trying to get out of school soon.

Other than that, I’ve been writing a lot and just making community and finding stakeholders – stakeholders/champions is what I should say… Champions, people who are also on the same path, and talking. So I’m just trying to create conversations, I think that’s really important.

What were some of the things you did on your gap year to kind of feed into that insight? Because I’m sure there’s somebody listening thinking “Hey, I should probably do that. What should I do?”

Yeah! So I volunteered a lot… For example, Girl STEM Stars was one of the places I volunteered. I’m on their board, and I help organize groups of girls from like 6-7 girls, between middle school and high school ages, and they went on tech days at companies. I would bring them to Google, and we’d have a whole tech day and they’d learn about coding. Some of them it was their first time they had ever coded.

Similar, I did a cybersecurity camp just last weekend at Berkeley, where it was also for middle school to high school kids. I’ve spent 20 hours a day with them – maybe not with them, but getting them ready, getting the materials ready, teaching them cybersecurity for the first time. They heard from cybersecurity professionals, as well as women in cybersecurity… So doing a lot of that.

But in that time - you know, the people you bring in and talk to these kids are the people I’m networking with. I’m rubbing elbows with some really cool people.

And then also just learning more… Right now I’m working on autonomous cars, which is completely different from the other robots I was doing in the past… So learning about a whole new system of technology and being aware of how it’s coming to the conversation and the importance there, because that’s a whole other big conversation… Immersion is just really helpful, which I can’t necessarily do in academics, because you’re doing a lot of different classes…

Well, you’re immersed in academics. [laughs]

Yeah, immersed in academics is not really what I wanna be, so I’m really looking forward to finishing up and getting back to being immersed in these other really cool technologies that are popping up.

So if we have any young girls that are listening to this show, or anybody that would benefit from Girl STEM Stars - is that it…?

Yeah.

What’s the first step to getting involved? Either as a mentor, or somebody to actually attend.

Sure. Yeah, and this goes not just for Girl STEM Stars, but if you’re interested in any type of organizing or volunteering across the country - just send me a ping on Twitter, @nikkymill. GirlSTEMStars.org is also open. But yeah, if you really wanna volunteer, we could totally use you. Just come down and send me a ping.

Nice.

Cool. This was a blast. Thanks, Camille!

Thank you, Camille.

Thank you!

Pleasure.

So Jerome, you do some pretty cool stuff for veterans, man.

Roger that. Well, thank you. I don’t know if I do cool things for veterans. I feel like it’s important work, but thanks nonetheless.

What exactly do you do?

What we do at Vets Who Code is we teach veterans how to program; we do this remotely (100% remote) and we focus solely on open source technologies, so React, JavaScript, NodeJS… Those are the main points of education, while doing a deep dive in computer science fundamentals.

So is this while they’re still in active duty, or are they in National Guard, Reserves - what’s their engagement currently with the military.

We usually don’t care. We look for the type of veteran that’s looking for a job. The average veteran that comes to our program is within a year from leaving service, or within six months of being out. By doing that, we can focus on people that are more serious, versus those that are maybe looking for a hobby… Because you know, I’m spending approximately 26 weeks out of my life educating, so I wanna make sure that people who are getting the fruits of this labor are serious about it.

Yeah, legit. I was just thinking, having ETS myself out of the military at one point, all the process they have, as you leave… All the different briefings you’ve gotta do, all the ceremony involved in exiting the military in an honorable status, that that would be a great time to mention, “Hey, there’s Vets Who Code, and as you look to new opportunities, there’s this opportunity for you.”

In my opinion - I’m not an expert - I think it’s well before that. If you look at the current tech hiring process, the current situation we’re in, if you’re gonna actually look at technology as a viable sector to transition in, you need to be focusing on that like six months to a year before you get out, simply because it will take you 3-4 months just in building relationships and making sure your portfolio is right, your GitHub is correct, you’re building relationships in your community based upon where you want to live, where you want to move, you’ve dwindled down all of the recruiter soup that you’re going to get and find two or three recruiters that are actually gonna focus on knowing your strengths and your weaknesses and building that relationship with them. So I will argue a year to six months before you hit that transition button.

Yeah, it’s a tough position in any soldier’s life, regardless of what they did at their service. It could have been a three-year foreign engagement, they could have been deployed, they could have just been on a base… Either way, transitioning out of and back into civilian life once you’ve been through the process of being in the military is an experience nonetheless.

Yes, it’s hard. It doesn’t matter if you did 4 years or 20 years; that transition from that community to back in the civilian life is shocking, to say the least.

Is this a free program for veterans? How does it work? What’s the charge?

Yes, we don’t charge veterans a dime. It’s all about finding the veterans that have the most promise. Usually, the average veteran that comes to our program - they’re stuck, they have been trying to learn how to code, they’ve hit a brick wall… Like there’s so much stuff on the internet and you don’t know which direction to go, and that’s our job.

[28:18] We not only point them in the right directions, we provide a curriculum for them to go through. As they get more advanced, we supply a mentor, and these are the processes that we do; then we start helping them with the process of prepping for a job. We help them with interview prep, resume, looking at your portfolio, looking at your GitHub, looking at your LinkedIn, looking at how you present yourself when it comes to your resume - all of these things that come into play, and we make sure that everybody who’s telling you advice, they’ve walked that road.

I’m at CBSI. Our primary CTO - he is at USAA, and then we have our CDO who’s at USA Today. These guys - we’re all people at big companies, so we take that time – you know, this is what you need to do, this is what we’re seeing, this is how we would change… You know, keeping that feedback loop open.

What’s the lifespan of that relationship?

It varies. On average, we do 14 weeks so I would say at least half a year to a year veterans are staying in contact. We have some veterans that since 2014 they are always in contact, they’re forever fans… It’s really weird.

Well, if you haven’t got a job… I mean…

It’s really cool, because it’s my way of creating the type of community that I like, people who are goofy and serious… I like the work hard/play hard types; we are gonna finish this project, but I wanna play Cards Against Humanity after this, too. We have a hard deadline for Cards Against Humanity Let’s go! These are the types of veterans that I find.

You mentioned finding the ones that are serious… How do you judge that? How do you formalize that?

Copy that. We have a three-prong process; we went from two to three. The primary phase is we put everybody in a Facebook group, we have the pre-work on GitHub, so they can look at the pre-work on the readme and go through this. Until they finish that pre-work, they don’t get interviewed. Those who complete the pre-work, they get an interview.

The first phase of the interview is always with me. I wanna make sure everybody that wants to be in Vets Who Code has done their prerequisites, talks to me face-to-face… Like, let’s go ahead and zoom it up, chat, so I can get a feel for you. I treat programming like people treat boxing - if you could do anything else and make money, you should do it, because programming is a forever job. You’re never gonna stop learning, you’re never gonna know it all. You’re gonna be the stupidest person in the room at least once a week. So if you have an ego, you might not like this…

Check it at the door.

Check it at the door, yeah.

If you’re one of those guys that think this is like college… Like you’re gonna go to college, get this degree, and then you’re gonna stop - this is not for you, this is not your bag. That’s the first thing.

Then after that we have a technical interview; another person, that way there is no bias. So I don’t handle all the interview phases either. We have a technical face, where Noel - he goes through their GitHub, he starts asking questions, seeing where they are on the technical aspects, so what things they’ve done outside of us… Because we’re always sharing other things. That’s the real gotcha - we wanna see if you’re hungry. Like I said, programming is like boxing - you have to be hungry for it, you have to want it.

This is an analogy I’ve never heard.

As a person who is in the military, box, and does programming, I box in my spare time to release stress, so…

[31:56] You see the parallels.

I see the parallels all the time. You have to be hungry, you have to want it, you have to show up every day. There’s never a day or a time where you’re like “I’ve achieved it all”, because there’s always somebody right behind you who’s gonna know just as much as you.

Absolutely.

Programming and boxing - it’s literally the same. Complacency kills.

Interesting.

That’s some military word there for you… Complacency kills.

Yeah, for sure.

It’s an interesting focus, actually, on veterans. I mean, what do you see – I guess you’re kind of biased because you’ve been through the military, but I’m thinking like how this might be for non-military to military. The mindset of the person, the change there. Can you maybe describe the mindset of somebody who’s been through the military, served the country, been through the training?

Yeah. First and foremost, I am not one of those veterans that had like a technical job in the military. I was security forces. I carried an M4 carbine and a 9 mm to work every day. It was nothing about computers in my job at all. So that’s the first thing I let people know, “You’re talking to one of those veterans that didn’t fit the criteria.”

Secondly, everything that we do in the military these days are a lot of procedures that we do on the tech side, they’re just different names. You guys have Agile, in the military there’s Rapid Deployment Procedures. We have components, and we have fire teams; the fire team is nothing but a component of an entire squad. So these are practices that are already ingrained in the military - that is also ingrained in software. The process of being able to read boring, dry, death by PowerPoint style documentation - that is the first thing you learn in the military, is death by PowerPoint… “Oh my goodness, this is 1,000 pages of useless junk, and I’m gonna be tested on it.” Programming - just like that.

1,000 pages of useless junk. Programming. [laughter]

Well, to get rank you do have to study some interesting things. You’ve gotta go before board, you’ve gotta come present it… But it comes with knowledge, and that knowledge is gained by you, not by somebody handing it to you. You’ve gotta want it; it’s part of the boxing thing - you’ve gotta chase it.

Same with the military - on the job training, learning by doing… That’s how you learn in the military.

Yeah, OJT.

Yeah, OJT. You go through basic, then you turn around and then you go to your training school, and then they send you to your base, and then your base teaches you how to do it the way they wanna do it. So you come here with a base set of skills, you meet the metric, meet the requirement, but then they’re like “Alright, keep this, keep this, throw this away, keep this, keep this. I don’t like that. Keep this; we might keep that.” That’s the same way when you go to your first company - “Do this, do this, do this. That’s cool. I don’t like that. We’re not gonna do it that way.” And pretty much that’s how you start your first week of a software job. They look at the things you have, see what they like, tell you what they’re gonna fire, tell you what they’re gonna add, and then move on.

What hooked you about software? I mean, you also like boxing, but the way you’re describing these things - they’re very harsh… They’re hard, difficult.

But I like it because it’s hard and difficult.

Okay. That’s what I’m asking.

That’s the best part. I like…

…a challenge.

I guess Daniel Cormier says it best… He’s the current UFC light-heavyweight and heavyweight champion. He says “Embrace the suck.” That’s something you hear like on a wrestling match, like NCAA - all these guys say that, “Embrace the suck”, and that’s what it is… I’m embracing the suck of software for the reward that it gives, like being able to have the type of lifestyle I want, being able to meet crazy cool people.

There are people that I know today that four years ago I was in awe of. I’ve turned my heroes into my peers. That is cray. [laughter] You can’t put a price on that type of experience. That’s what helps me get up at 04:30 in the morning, and start focusing on making myself a better person.

[35:52] Yeah. “Embrace the suck” reminds me of a saying – a distinction that I’ve heard lots of times is like there’s good suck and there’s bad suck. This is the good suck, and that’s the one you have to embrace… Like “Yeah, this is hard, this is harsh, this sucks”, but you know what’s at the end of the road is good. There’s all that stuff that sucked that’s like, just get that out of your life.

And that’s the strength of military life. If you were ever deployed, if you’ve ever heard like “Hurry up and wait”, you’ve been in that world where you’re waiting for six hours, and then somebody comes out like “Hey, we’ve gotta hurry up and knock this out.” I’m like “Really?” We have to move in 15 minutes; we’ve gotta move 40 people in 15 minutes. We’ve been here six hours, they didn’t say anything to us, but now in these 15 minutes we have to move everybody. Okay, that’s cool… Embracing the suck; that’s the military life. If you’ve ever been on deployment, you have to embrace the suck. It’s like, it’s 120 degrees out here and everybody hates us, but you know, we’re gonna go home soon, so we’re just gonna embrace the suck and move on.

[laughs]

Oh, my gosh… I’ve heard another one too, it’s good training. Anytime you’ve done something, you’re like “That was terrible. Why do we do that?” “Good training. Just get over it. Good training.”

[laughter] Yeah, it was training.

That’s right. That’s similar to what we were saying recently about decisions that we’ve made with Changelog, or in business… It’s like, you go down a path and you realize it’s the wrong path, and maybe you’re six months down the road and you’re like “Well, we’ve gotta go back to where we were”, and it’s like “Well, that sucked… That was a waste of time, money and effort”, and then we always say “Well, education.”

“Now we know.”

Like, could we have learned that otherwise?

Yeah, it’s just good education, good experience.

That’s good training.

It’s good training.

I like that.

Good training.

One thing I like about this is I’m learning lots of cool sayings.

Oh, I have a million militarisms.

Give me some other ones, c’mon. On the spot.

I don’t know… I don’t know, it’s PG…

Keep in family-friendly.

Yeah, like I said, I’m trying to keep it…

Alright, you’ll share with me the other ones later.

Yeah. See, now I’m brain farting, right? It happens in the moment; you’re like, “Oh, there you go. Bam!”

Let’s talk about maybe those active duty/military men and women who were out there serving our country, or they’re transitioning out and they’re looking for that opportunity… They’re listening to this podcast, or somebody who knows one is listening to this podcast - how do they reach out? What’s the first step?

Well, the first step is always go to vetswhocode.io. We have our application form on there. Once you apply, I’ll pretty much always email them, asking them to have a Facebook group. Some people just go straight to the Facebook group, but I always email them personally and say “Hey, do you have Facebook? Here’s our Facebook group. Join it, so we can start the application process with you.”

They’re in there with the pre-work, they’re talking to other veterans… It’s a way to make sure that everybody has a fair metric that we can at least start off of… And not only that, it’s a way for them to meet other people who are interested in this stuff. You know, it’s better to embrace the suck with a group of people than to embrace it by yourself. Misery loves company, that’s why everybody misses the military days… “Oh, those were the good, old days.” No, they weren’t, but you made some good friends, because…

They were the best worst days.

It sounds like high school.

Yeah, so they were terrible…

Yeah, it’s like freshman year in high school. That’s how your entire military career is, like freshman in high school. Everybody wants to kick your butt… It’s awful.

It’s awful. And then you get out and you’re like, “Remember how good that was?”

Yeah, “Man, I miss those days…” [laughter]

“I miss those days…”

So how many people have you put through this program?

Right now we’ve done over 100 people; they’ve gotten jobs, in 14 states. We’ve had people who are working here, people who are working in Seattle… We actually had a veteran who started two weeks ago in Microsoft, so that was pretty cool…

Awesome.

This is a big deal.

This is our first cohort that we’ve gotten 100% success rate with…

I was gonna ask what your placement rate is.

Usually, it’s around 95%-97%…

That’s amazing.

But that’s because we’re very hyper-focused. The way I look at it is like “Listen, you’re not paying for this, and I have the real-world experience, so listen to me so I can help you, or don’t listen to me and don’t be around.”

Hit the road, yeah.

Don’t waste my time.

[39:48] People don’t really like that style, but I’m doing this because I remember every day how hard it was with the transition… So I’m here to make your transition easy, so you don’t have to go through what I went through.

Alright. Was there somebody there for you?

No. The transition process when I got out of the military was trash, on top of trash, on top of trash.

Isolating, I would say… My experience was isolating.

Yeah. Well, the military transitions to help get you off their books; the military transition isn’t about you getting acclimated into civilian society. It’s like, “Alright, you don’t wanna be a part of our team anymore? Bye!” Like some straight up, “Bye Felicia”, type move! Like, “Okay, what will you do?”, so you have to figure out these things…

It’s so weird too, going back into civilian life, man…

Yeah, it is, because it’s just a whole dichotomy. We can get into this – that’s a whole podcast if we get into that.

What about the placements for people? If we’ve got people listening to this podcast that are at places where they’re looking for good programmers, how do they reach out?

There’s a contact form on VetsWhoCode.io. You can just fill that out. Everything goes into our chat ops; my phone is buzzing right now because people are hitting me up… So pretty much in real-time, on like Slack, and they’ll apply. So basically, what happens is you go to the contact form, you fill it out, I’ll reach out to you and we’ll start conversations.

Well, I mean for the companies… Same deal?

Yeah, same deal. We start the conversation, because I wanna make sure you’re a good fit. We’ve had companies come in and being like “We love what you’re doing; we have colleagues who have hired your people. Would you mind doing Java?” and I’m like, “No… You don’t understand how hard it is if I’m not actually there in front of that veteran to be able to get their machine prepped to do Java and Java Spring Boot well. We have to control the install phase. That’s why we chose JavaScript; the ease of use of being able to get that veteran from not having a dev environment to having a dev environment is super easy in JavaScript, versus more stable languages… It’s like, “Okay, it’s very difficult to do that, so let’s work on this, and then as they get interested, they’ll be able to have this base of knowledge, then they can build on it.”

We had a veteran right now - he last week started his first day of work at J.P. Morgan as Angular and Java Spring Boot developer. We don’t teach Angular, we don’t teach Java Spring Boot, but he was able to get that job because of the deep knowledge base he got with us, and then being able to go and venture out on his spare time outside of class, with Java. I was like, “Alright, that’s awesome! I don’t care what you do, as long as you’re programming. Cool! You’re building. Never stop, dudes.”

That’s another thing that programming has in common with boxing - you stop for a week and you pick it back up, and you will feel it.

You’ve gotta do it every day.

Yup.

Anything else to share, Jerome?

Follow @VetsWhoCode on Twitter. If you’re looking for any good React developers, any good JavaScript engineers, reach out to us. I am always looking for people who like to hire good people.

Cool. It was good seeing you, man. Thank you.

Thanks.

Thank you.

Alright, so we’re joined by Abby Cabunoc Mayes, Working Open, Practice Lead at the Mozilla Foundation… That’s a mouthful.

I’m sorry… Yes, yes it is.

It sounds like an important thing. Tell us what that means.

Yeah, it means I care a lot about how to work open, working openly, and about how to do that past just open source… So doing that in open science project, with civic tech, with the government. Are you writing curriculum - how do you do that openly? So just helping people do that.

Okay. So it’s not just open source, but it includes open source.

It does include open source, and my background is coming from open source.

Okay. But also open science, documentation…

Yeah.

Okay. How did you get into that and how did you end up at Mozilla, doing this work?

I have a background in bio-informatics - just computer science, applied to biology. So I was writing software for scientists at this cancer research institute, and the longer I was in Academia, the more often you notice how people maybe fudge their data a little bit, or hide their datasets…

Like to come up with certain results?

Yeah, just so that they can get that really cool results and it gets published in Nature, and their career goes – and just how you get forward in science.

Oh… That stinks.

It does. That’s when I really got into open science… Because it’s like, well, we really should be doing this so we can have the best innovation, so we can help more people.

Okay… What percentage of closed science, or not open science, when you experience this – just give me somewhere to look at… Like, are 20% of people doing this, 60%? Is it pervasive?

So the lab I was in - I should clarify that - they were great; they did not do this… But you hear about it a lot. With collaborators, you’re really scared about getting scooped, so they’d hide their data as much as they could, or maybe like just play around with the P value, to see what you can do to make the results show the story that you want, where the data itself doesn’t really do that…

It sounds like a statistician!

Yeah, a little bit…

That’s important though, because in history you have like Einstein, who is remembered, but then the persons who actually had some theory of relativity before Einstein doesn’t get the same credit, because Einstein was the one who was –

Yeah.

A lot of times you’ll have that dual invention, and one person is just the one who gets all the credit.

So it makes sense to be secretive to some degree with your data and your research.

Yeah, it does.

Secret is okay, but tweaking it to fit your results…

Okay, let’s not go there…

Right.

I think one of my former colleagues, Greg Wilson - he mined a lot of these research papers and looked at what the P value was, and there was a huge spike right before what we consider significant, in research… So a lot of people got their P value just right below that, the statistical significance…

What’s the P value again?

It’s the statistical significance of the results, so it shows that there is correlation there.

Gotcha.

Right. It’s key. It’s P.

[47:56] It’s P.

Okay.

The P is key.

So you said open…?

Yeah, so that’s when I got really interested in open source, because my lab was really writing open source software for these researchers… But then open science, generally; I was like, “Yeah, this is really important.” A lot of people are really scared about – like, there’s something wrong with the research system if a lot of people are just hiding their data.

So then Mozilla Science started, so that’s why I joined Mozilla. And since then, my role has shifted to be less about just open science - I still do quite a bit of open science - and about working open across [48:26]

Tell us a little bit about Mozilla, because… You know, intellectually, I know Mozilla does a lot of stuff, but instinctively, I think Firefox, and then that’s pretty much where my brain stops. So tell us - Mozilla Science is not a thing I’ve heard of… Tell us some of the other stuff Mozilla is into, and how your work affects everything.

Yeah, so at Mozilla our mission is to ensure that the internet is a global public resource, open and accessible to all, and we do that through products like Firefox, but also through movement-building, and working with different communities. Mozilla Science is one of those communities that they’re working with, but also government civic tech - working with that.

We’ve released this internet health report, so it’s like “What is hurting and what is helping the internet?” We look at things like how open is the internet, how private and secure are we on there, how inclusive is it, what’s web literacy like, who can actually make a change online?

We do a lot of things like that, and then there’s the Mozilla Festival every year in London, in October…

MozFest.

MozFest.

Oh yeah, that’s where it’s at!

Yeah, it’s pretty great. So it’s all of that put together - all these different communities who really rely on the internet and really wanna make sure that it stays healthy, and they’re there to really meet about that, brainstorm, make cool things…

It’s sort of the underpinning of the Mozilla brand too, to be open…

Yeah, yeah.

So you’re kind of like on the core mission of Mozilla at large.

Yeah, and I think our main goal is internet health, but then how we’re doing that is through openness… So either building products openly, or by rallying the community to build something open.

What does your day-to-day look like when you’re trying to educate, lead, mentor, build a movement? What does that look like in a tactical sense?

It’s a lot of email… [laughter]

Very hard work.

A lot of emails.

Yeah, a lot of emails.

A lot of emails, but also a lot of video calls. I spend a lot of time just meeting with people online. We’ll have big conference calls for trainings…

So what kind of people are you meeting with?

All sorts of people. I run Mozilla Open Leaders, which is this mentorship and training program around how to work open… So people come with their projects, whether it’s an open science project, or an open data project, or some civic tech project… So I meet with a lot of people who are running cool open projects from all these different fields, and just tell them “This is how you work open properly.” Not that I know everything, but…

So what does that look like, working open properly?

Yeah, working open properly… I think a lot of people forget to strategize around how to work open, or plan to do this, so a big thing at Mozilla now is open by design, instead of open by default. I think because open is part of our DNA at Mozilla, we often forget why we do things openly… So by default everything’s just online, but then it’s not making that impact we really want.

I think if you’re doing open well, at the beginning you’re thinking about “Who do you wanna work with? How are you gonna engage them? What’s the value exchange gonna be? How are you gonna bring them from being a user to a contributor, to maybe a project lead?” Thinking all these pathways through, and then writing the documentation to make sure that they know how it all happens, and providing that support to people and mentoring them as they go through your project.

So that’s a broader view… It’s different for each project, what that looks like… But yeah.

Interesting. So take one of your projects, maybe even the Open Leadership project - I’m not sure how your projects break out - and then describe to us how that was designed to have a specific goal, or a specific end in mind.

Yeah, so this work started when I was a part of Mozilla Science, and what we are trying to do – we just took a bunch of developers who were interested in science, and put them in touch with a bunch of scientists… But then we realized that scientists weren’t great at working open; they’d have their project and they were like “Oh, I don’t need you yet. Just wait…”

[52:08] It’s probably like cultural – not clashes, but just differences.

Yeah. So then I started writing these little guidebooks… It’s like, “Here’s how you can open your work a little bit, so that you can benefit from all these developers that want to help you, that care about science.” Then we realized these guides are really helpful for just anyone running an open project, so we started doing that.

Also, my husband - at the time, he was running this startup accelerator in Toronto, and the way he modeled it was like a three-month thing, and I just took a lot of the ideas there… So we’d meet with them at the beginning, we’d have them set goals and figure out what they’re gonna do over those three months, we’d work with them regularly… Yeah, so it was modeled after a startup accelerator, this mentorship program.

Okay… Is it working?

I think so. [laughter] And what really excites me about this program is that – so I designed it so that people can come back and become a mentor after they’ve gone through it… So about 50% of the people that have gone through have come back and mentored other people. So we’re showing people how to work open in a way that they get really excited and wanna help other people do that. That’s what I think is the essence of building a movement.

Is a lot of this inside Mozilla only, or is it sort of like Mozilla and external?

The people in the program are from everywhere.

Okay.

Yeah. It’s Mozilla only that’s running it right now, but I’m trying to – actually, we’re partnering with a few people, so they can run their own versions in their organization or maybe in their language, or in their city. We’ve done that a few times, but I’m trying to make it more forkable, so that people can just run this program wherever they are.

So is that like a face-to-face thing? You mentioned you do a lot of video calls and a lot of emails, and stuff… Is any of this distributed, or is it all sort of collocated?

So the first iteration we had was a two-day event, where we ran the training at the beginning, and then we just followed up with mentorship afterwards. Then we realized, “Oh, we could do this all online”, where we just meet every week online, do a little bit of training, and then alternate weeks you do one-on-one mentorship.

You mentioned guides… Are those guides open at all, or available to people?

Yes. There’s the Open Leadership Training Series, it’s on GitHub. You can edit it, you can remix it… Yeah.

Nice.

How weird would it be if she said no to that question?

It would have been really weird.

[laughs]

Like, “Actually, they’re proprietary, offline.”

They’re copyright.

They’re locked in a vault… Underneath the pillow!

“No PRs here…”

So what are you doing here at OSCON, what are you trying to talk about? I mean, obviously, the open stuff, but is there specific – because this is about open source specifically, so we’re software people… What’s your message here, and who are you talking to?

Yeah, this is my first year at OSCON, this is really exciting. I gave a talk yesterday on open as a competitive advantage, and that was really about that “open by design”, instead of “open by default.” So like, what choices can you make to be really strategic about what you open? And like, are you opening something to increase your adoption, by giving away for free, or are you trying to lower your operating costs? There’s different reasons why you can open different parts…

Looking at those reasons, is there ever a decision workflow wherein it’s like, “You know what, don’t open this…” Or is open always better, in your opinion?

I don’t think open is always better. Some of the people in the mentorship program - they’re a bigger organization, and just telling them to make everything open is a little bit too much… It’s like, how can you start, what are little things you can do to start opening things up? And it might be just like getting ideas from your community, like “What features do you want us to do?” and let people suggest them and vote on it. I think LEGO does something like that, where you can suggest which kit you want.

Right, yeah.

Yeah, so it’s like a tiny way you can open it.

Get people involved.

When you say “advantage”, it makes feel it’s like a growth hack of some sort… Like, you’re doing it as a hack to an alternative way, and somehow you’re gonna get better benefits from – as an advantage, so to speak… Can you speak to that? The advantages of being open, the growth hack part of it, like how you would be better off that way, the growth potential…

[56:08] Yeah. I think there’s a lot of advantages to working open, but I think by working open and letting people see what you’re doing, and inviting people to join in, you can – well, with science, you get the best ideas, you get the best innovation that way, and it’s not just one person trying to figure out how to solve this problem, but you have hundreds of people trying to solve it.

Yeah, once you get de-facto trust, right?

Yeah, and people see how it’s done… And if they can see all the data there, then they know that you didn’t tweak it, and that you didn’t hide parts of it. So that’s a huge advantage. Just that buy-in from the community, and also that good will is usually pretty nice.

We see that with a lot of infrastructure companies that are open source, but they’re also startups or small businesses, or big businesses, and we always ask them “Why are you open source? What’s the point for you?” or “Where are you coming from?” and a lot of the times it’s about trust, because they just think “This is table stakes. You don’t have to trust that we’re doing things with your stuff, you can see what we’re doing with your stuff.” So on the data side, it makes a lot of sense of like “My research is legit. Here’s everything.” Or if there’s a problem, like, “There it is, in line 37. Help me fix it.”

With software companies, a lot of the times the advantages - we don’t have to prove to anybody that we’re trustworthy. I mean, you still have to, to a certain degree, but “Here’s our proof right here. It’s right there, out in the open.” It’s another example.

Or even just listening to people. I think that’s a nice, open interaction that can build trust, even if they can’t see – or if they already trust what you’re building… But knowing that you’re hearing what they’re saying, and making changes - I think that’s really important.

In regards to building a movement, or starting a movement, a lot of people - and I’ve had this in the past… Open source things, or write openly, or publish openly… Into the void.

[laughs]

And so they want interaction, they want other people’s ideas, they want contributors, they want all these things, and yet, there is a disconnect, or sometimes there’s just too much noise and you can’t even get your voice heard… With Mozilla, you guys have allowed voice in the community, so it’s probably established that these things are going to have interaction, and stuff… But if you’re just starting from square one, do you have to advise people on how to build that movement, how to kickstart the interactions?

Yeah, definitely, and I think something people forget to do is really have that concise mission, or concise vision around what they’re doing, so people can understand it right away. We do try to amplify – like you said, we do have big platforms, we do try to amplify everyone that we trust or that’s coming through the program, to help give them that head start. But if they have a very confusing mission statement that people don’t get, it’s gonna be a bit harder for them to gather that community.

We do a couple exercises around solidifying what you’re doing and your messaging there, but then also once you start getting a couple people interested, how do you keep that momentum going, how do you follow up with them, and really find out why they came, so that you can give them the kind of value that they want coming out of this. Did they come because they wanna learn JavaScript? Then help make that a great learning experience for them. Or did they come because they really wanna help take down the browser monopoly? Then really give them that opportunity… Back to Firefox, yeah…

That’s a movement right there.

Yeah, yeah.

[59:27] You’re also part of the Journal for Open Source - is that right?

Yeah, the Journal for Open Source Software.

What is that about?

So in Academia, a lot of times if you write open source software, you write software for science, you can’t cite the software directly… You have to write a paper about your software and get that published, and then you get more citations, and then you can make the argument that you need more funding for your software. It’s a little roundabout, so the Journal of Open Source Software makes it really easy for you to publish a paper on your software. So they just take the readme, and then we have a review process, which is similar to – just like “Are you following best practices with your software?” and then it generates this paper that’s all online, and it mints it with a DOI (digital object identifier), so that people can cite it in their real academic papers… And then you can say “Oh look, these ten people published about my software, used my software to do their analysis. I need funding.”

So it makes it a little easier to make software and science more sustainable… And it’s a little hack between – because right now you can’t cite software directly, so it’s just like… Adding that little step.

Well, that’s what I was saying - it almost seems like it’d be more useful for papers…

Yeah.

…but it’s for open source software.

Yeah. I think you can sort of cite software directly, but not everyone thinks that’s a good idea. [laughter] We’re trying to make that happen.

It’s sort of frowned upon.

Yeah,

So you say it’s kind of a hack - is it a first step, or is it the end goal…? It seems like the kind of thing that would be generally useful for all sorts of research.

Yeah, yeah. I think the end goal would be you can just cite software directly. So you can get a DOI for your software, so you could cite it… Just a lot of publications out there are like “That’s not a real paper.” So this is just a way to make software a little bit more visible.

Okay. Because citations are super-important in Academia.

Yes. That is how you move forward in your career.

It’s like, that’s your street cred right there.

“I’ve been cited… They agree.” It’s a concur.

What’s that?

It’s a concur.

“I’ve been cited.”

Yeah, like “I concur. I agree with you.”

Although sometimes people cite it because they disagree…

It’s more like…

Notoriety, than it is…

It’s like, “We used your research.”

Right.

It’s similar with page rank, how it works with links. The more times you’re linked, the more influential that you are.

That makes sense.

Sometimes they’re agreeing/disagreeing, but you’re obviously a part of that conversation.

Yeah, they consider the disagree part of that, but yeah, definitely. It just shows that you have some influence; whether it’s negative or positive is to be seen by the reader.

It’s like how many stars you have on GitHub… No, it’s not. Let’s not start there…

[laughs]

Yeah, Arfon Smith started the Journal of Open Source Software. I hope I did a good job explaining what that was… I’m sorry, Arfon, if I didn’t…

I think you did. I got it. But we do know Arfon, we’ve had him on RFC, so I was somewhat familiar with this…

I wasn’t… Not at all. Brand new today.

That’s where he works now.

I didn’t know that either.

You learn something new every day.

I miss you, bro. Good to see you!

What’s up, Arfon? Shout-out.

The last time I talked to him he was traveling everywhere. He was like a vagabond with a family.

He travels quite a lot, yeah.

We interviewed him from a – was it a bus?

RV…

It was an RV…

Parked outside of a Starbucks, in Canada somewhere.

That’s right.

That’s not surprising, yeah.

It was pretty wild.

That was fun.

[laughs]

Cool. Abby, anything else you’d like to talk about?

I think that was… Yeah, that was good. Nothing comes to mind.

Adam, anything else?

Nothing from me.

Alright. Well, thanks for all the work that you’re doing.

It’s a real pleasure being here. Thank you so much for having me.

Thank you, Abby.

Thank you.