Kris Brandow: Part 2: Storytelling

When you step on stage you become a storyteller. Just because you’re not acting doesn’t mean you can’t employ the same techniques that have been used for centuries to captivate your audience and tell great stories. Before the stage though, you’re storytelling on the page, weaving the narrative of your talk through a proposal. But how do you achieve this?

First, note that your proposal’s audience differs from that of your talk. The former consists of a group of Go community veterans, who read hundreds of proposals to organize an exciting and intellectually stimulating program. The latter is a group of Gophers looking to learn and grow. The story you tell in your proposal to captivate your first audience differs from the story you’ll tell on stage for your second.

When I started crafting my first proposal I searched for articles and blog posts to aid in the process. I easily found resources that guided structure, but few on how to tell an engaging story. Through this guide I aim to fill that gap, helping you tell your stories in a captivating way for both the review committee and the audience at GopherCon.

Before I dive in, I want to emphasize that while this advice may help increase the odds of selection, it will not guarantee acceptance of your proposal.

How To Think About Your Proposal

When starting something new I use analogies to make connections between what I’m doing and what I’ve done. As an author, I connected that the storytelling techniques I use in my prose translate to the writing done for a proposal and a talk.

Listening to one person talk for 25 minutes can be difficult if the speaker is not telling an interesting story; this is even more true when it’s 45 minutes. Similarly, when reviewing hundreds of proposals on a tight deadline, it’s difficult to engage with each proposal individually, unless the material pulls the reader in and excites them. The key component in both situations is time: an abundance in your talk and a scarcity in your proposal. Techniques like suspense and foreshadowing are handy in your talk, but not so much in your proposal, where you need to get straight to the point. In the latter, opt for concise but exciting language, where each word can serve a purpose.

[36:01] Another element to consider is competition. At the maximum, there are four options for attendees at the conference: the three session tracks and a hallway track. That gives you an odds of 25%. Contrast this with the proposal review process, where your odds are closer to 10%, and your opportunity to hold the audience’s attention is far shorter. Holding that attention requires not just writing in a concise manner, but also adding elements to your proposal that set it apart from the rest, and this is where storytelling elements are helpful.

Combining these two elements, we arrive at an important piece of advice: don’t bury the lede. The structure of your proposal does not need to match that of your talk. Let’s use an analogy. Your talk is like a book; you want to pull the audience through all the pages, even those treacherous middle ones, through to the end. You make that task more difficult if you front load all of the interesting information. Your proposal though? That’s more like a newspaper article; you want to put all the interesting information upfront, and then extrapolate over the remainder of the space. In your proposal, that end space is where you discuss elements like foreshadowing and suspense, which can be conveyed either directly in prose, or indirectly through an outline. This is also where you can discuss elements like timing, context, and why you’re the best person to give this talk.

Finally, while there are a variety of formats that conference talks come in, the structure of your proposal is static. You want to address the same requirements and criteria I laid out in part one, while ensuring that you include as much context (including storytelling elements) as you can compactly fit.

At the end of the day, storytelling is about the way you use words and less about the structure used to contain them. Whenever you’re storytelling, think about your audience and cater your story to them. Craft it based on the time you have, the structure the audience is expecting, and the other things competing for your audience’s attention.

Justin-Casimir Braun: So there’s two things I would respond to that. One is that I hope we’ll have more discussions around transparency around these systems. I think that’s a precondition for anything else. And for that to happen, there is an argument that needs to be dispelled. And that argument is that making these systems public allows people to game them. One, I think it’s really, really hard, and there’s some very good academic research that shows how hard it would be. And two - well, these systems operate essentially like bylaws, right? They’re essentially administrative guidelines encoded in a model file for how a decision was being made in some bureaucracy. And I think it’s really hard to make the case that such guidelines should be secret.

And so yeah, I think we need to have a discussion and make the case proactively that transparency in this space, and encouraging people to learn how they work is a good thing, and encouraging people to game those systems is probably a good thing, because that means you’re probably closer to abiding by the law. And if you can game the systems, then maybe they aren’t very good. That’s the first thing I want to say.

[41:45] The second one is that most of the systems we’ve looked at are pretty terrible, in most ways. I think they don’t work very well. They either use features that are absolutely terrible, or have training data construction that is really problematic, or have disparate impacts on various groups. Almost every single system we’ve looked at so far has one or multiple of these features. But there are some systems that maybe are better. And it’s possible, I think, if you think very seriously about how you do each of these steps - the feature selection, the training data, and then constructing the model, and then evaluate for bias, and then potentially retrain/reweigh your training data, and so on, maybe it’s possible to get to a better place. Technically, it certainly is. And I think then you get to a different set of questions. And I hope that the conversation at some point can move beyond kind of the gross incompetence in the way which we’re showcasing across the board, but can move to a place where we can discuss “Okay, let’s take this best-case scenario. We have a system that doesn’t have obvious bias and so on, that was constructed carefully. Should we do this? Is it a good idea? Is a machine making the decision removing something inherently kind of valuable from the type of interaction? Is the machine actually more explainable than a human is? And is that a good thing? Is it equal treatment, because everybody’s being scored by the exact same system and not by individual caseworkers? Or is it not equal treatment, because the tool contains a decision tree based model, and so different people are based on different characteristics?”

How do we think about systems that include some level of probabilistic assessment? Is that something that we think an administrative position should do? And then of course, we can also have the maybe fun for some people discussions around like which fairness definition is the best, whether we should seek to minimize or equalize false positive rates across different groups, and so on.

I think there’s a bunch of really important questions that society has to grapple with here, but I don’t think we’re there quite yet in most cases. And so long as we aren’t, I think Gabriel and I will have plenty of work showcasing incompetence, and all that stuff. But I hope that at some point we can move beyond that.

Daniel Whitenack: Yeah, yeah, sure. I appreciate that. So yeah, Prediction Guard is what I’ve been working on since about Christmas time-ish… Originally, I was thinking a lot about large language model evaluation and model selection, but it’s kind of morphed into something else. What I’ve realized is that there’s this market pressure, there’s internal company pressure for people to implement these kind of generative AI version of models into their workflows, because enterprises realize the benefits that they could have. But in practice, when they go and they go from Chat GPT, they type in something, it’s amazing, and then liek “How do we do this in our enterprise, where we have maybe rules around data privacy, or compliance issues? And also, we want to automate things maybe, or we want to do data extraction… But I just get text vomit out of these models. What do I do with that? I just get unstructured text; how do I build a robust system out of inconsistent text vomit?”

So Prediction Guard is really focused on those two things. One is kind of compliance and running kind of state of the art AI models in a compliant way, and then layering on top of that layers of control for structuring output, and validating output. So some people might be familiar with projects Guardrails, or Guidance, or these things… So we’ve integrated some of the best of those things into the platform, plus some ways to easily do self-consistency checks, and factuality checks, and other things on top of large language model output.

David Yakobovitch: I think it could be a combination of both. I hope it’s more that we as developers reorient ourselves and our technology. Think about if you are a developer building a product - you don’t want your product to break; so whatever the user input is, you’re designing it so that if that’s a number, the number gets fed into the system. But if it’s text, the text gets converted to a number, so it still gets fed into the system.

I think just like data scientists and engineers today who are developing their APIs and their systems not to break based on inputs, the same thing should be that “Are we thinking about humans first in these systems?”

We’re starting to see AI guidelines. I know the European Union recently launched their own AI ethics standards; that came out a few months ago. There’s similar initiatives going on in the U.S. as well on ethics, and about integrating these systems for all. But it’s really about having the conversation and then making sure you take action.

Just like you mentioned about the languages, I had the opportunity to sit down on the panel, an advisory session with one of the leading candidates for the New York City mayoral election in 2021, and that exact topic you brought up, we brought up as well.

You mentioned 6,000-7,000+ languages, and in fact in New York City over 20% of the people do not speak English. 10% of that is about 800,000 of the eight million people in New York. So if you take some of those languages, just the top ten other languages that are not English or Spanish, it is about 25% of the population. That’s enough to have enough votes to win an election. Not that the goal should be just to win an election, but it should be serving all constituents. And when you’re in an accessible city like New York City, it should not just be translating services for eight languages, but how about at least 100 of them or more?

I think when we start thinking about processes, we need to do a lot more competitive intelligence, a lot of research on who our constituents are, and then to best serve them. And if that means rolling out a feature in tranches, that’s totally fine. But as long as you have the goals there and you’re thinking about accessibility on the onset, it’s paving you in the right direction.

Seth Vargo: Yeah, you know… I do some sketch diagrams, I have some hanging on the wall. But you know, let’s say you’re not buying what I’m saying, and you wanna try it out for yourself. Zero risk, you can launch an in-the-cloud Vault environment with an interactive tutorial right from Vault’s website.

VaultProject.io - there’s a big ol’ button, really hard to miss, it says “Launch Interactive Tutorial.” If you’re not driving, you can actually pull out your phone and do it now, it works on mobile. It will walk you through some really common Vault commands, and it actually launches your own Vault instance in the background. HashiCorp funds that, so that people can try out Vault zero risk, no installation; all you need is a modern browser with some JavaScript enabled.

Once you’ve bought into the idea - because you will fall in love with it, trust me - the next thing you wanna do is actually download it. Before you put it in production, you’re gonna wanna download it locally.

[55:51] I run it on a Mac locally; I know that’s very common for software engineers. So you can head on over to that same website, VaultProject.io, download the Mac binary - you’re gonna want the 64-bit one, because if you have a 32-bit Mac I feel bad for you.

Download the 64-bit one. That’s gonna give you a single static binary that you can run, it’s just a Vault. That single static binary is the client and the server - it can run as the client, or the server, or both. If you wanna ever try something out locally - and I still do this myself; there are times when I forget an API or I wanna see what a response is - most of our tools have this thing called -dev flag (dev is short for development). So locally you can just spin up a Vault server by running vault server -dev. That will give you a fully ready to go, ready to accept request Vault server that you can do some sample commands against. The same thing that you are running in the cloud, but it’s all local on your laptop. And you can really abuse that; you can do a lot more to that than you can do to the cloud instance. You can [unintelligible 00:56:45.26] around with different configuration, you can see all the different log output.

Then, once you’re satisfied with that, you’re ready to put it in production. From there, there’s actually a number of techniques. You have to make a few decisions. If you’re a tiny startup, it might be best just o start with a single Vault instance. Don’t worry about high availability, especially if this is a beta test. Just get it in production, make sure you’re comfortable using it. It’s senseless to invest a ton of time and energy into a tool that you’re not a hundred percent sold on yet. If you’re in a cloud environment or bare metal, spin up a server, use your automation tooling (Chef, Puppet, Bash) typing things in the terminal, and install it.

The tools are really easy to install. We have a server called the HashiCorp Releases Service; if you head on over to releases.hashicorp.com, you can browse any version of any product we’ve ever published and you can download it right to a machine. It’s a single static binary, the same that you downloaded for your Mac earlier. So you put it on the server and then you run it. Maybe you use SystemD or init, or Upstart or whatever your init system is. You have it running on the system, and from there you can hit it publicly. You might give it a public IP or put it within your private subnet, and you can start addressing it from other applications and really see how it behaves. Give it a trial, but you’re not ready to invest yet. You’re just kind of really feeling the waters out in a production or a staging environment.

Once you’re sold on it, you’re gonna wanna automate the process of standing up a Vault cluster, and you’re gonna wanna move to a highly available environment. And again, this is all completely open source, so you don’t need to pay for Pro to have high availability. High availability is built right into Vault. What you do is you spin up multiple Vault clusters, and Terraform is a great tool for doing this. You spin up these Vault servers, you connect them to each other; they all talk to each other, they do a leader election algorithm, one of them gets the leader, the other one goes into standby mode, and from there you can make requests against Vault. If one of those servers goes down - there’s an outage, or someone pulls the plug or it just dies, the other two will pick up the work. So you run in that high availability mode, which is something that’s really important, as this becomes a crucial piece of the secret management, the credential management in the organization.

So that’s the best way to get started. That’s my recommended path. You should try it on the internet before you download it, you should download it locally before you put it on a staging server, and then once you’re ready into production, we have a couple guides that published that are kind of like best architecture practices.

Once it’s running in the staging environment - that step before production - I think it’s important that the organization takes a step back and thinks about “How are we gonna manage this thing? Who’s gonna be responsible for it? Who’s gonna be setting up a policy? What’s the backup strategy?” Because that’s ultimately gonna be important. Systems fail, it happens all the time, so “How do we plan for that? What’s our recovery strategy?”

Vault’s internals are also gonna become important at that point. I’ve talked a lot about it here on the podcast, but we have a whole section of the documentation online that’s devoted to Vault’s internals - the architecture, the high availability model, the security model, the threat model, how you do telemetry; are you pushing things into Graphite and monitoring? The process of key rotation, online upgrades, the whole nine years - those are things that whoever is responsible for managing the Vault cluster, they’re gonna wanna familiarize themselves with those things before you move into a production scenario.

Those are details that if you are just getting started, you don’t need. You can read them if you want to, but they’re kind of dry, they’re very academic in nature, but they are crucial if you’re gonna run this in production. You should really understand the internals and the architecture and how it’s working, so that if something does go wrong, you have a better picture of how information is flowing through this system.

Rob Barnes: So in terms of backups, we have a functionality built in there to take snapshots, for example, depending on – so the way you’ve got to think about Vault is dependent on your storage backend, you can think of Vault’s entire architecture as kind of decoupled. You have the place where your secrets are stored, so that’s where all the secrets are stored, encrypted there. And then you have Vault itself, which is the thing that you interact with and the thing that does all the encryption, decryption and secrets management for you, and it interfaces with the storage backend.

Now, typically, we used to recommend Consul as a storage backend. We now have integrated storage backend in Vault, so now it’s not as decoupled as we used to be. You never used to have kind of – you used to have Consul, obviously, as a storage backend, and you’d have to manage that additional thing. It’s another system. And if Vault is just the business value you’re trying to extract, then no one really wants to manage this additional system just to get their secrets management, right? Which is where the whole argument of integrated storage comes in.

So in terms of that, it is a clustered approach, it is highly available. If you do lose a node, there are other nodes to take its place, there are leadership elections, so on and so forth; there is replication of the secrets between the nodes as well. And then it will just come down to implementing a good, fault-tolerant design for your Vault cluster. You’d have them in different kind of fault domains, and so on and so forth.

[42:22] So that’s kind of the first part. In terms of things like upgrades and general operational overheads, I’m not gonna lie to people out there, there’s a lot of things to do. So if you’re running it on a VM, for example, then you have to think about patching the underlying operating system (that is your responsibility), and protecting the underlying operating system as well. In general, I think if I remember correctly from the Vault hardening guide, you wouldn’t even have SSH to that. So you need to think about what your consumption patterns are there. Are you going to have a thing where you kind of throw away a VM and deploy a new one with Vault, and it joins the cluster? And so on and so forth.

And then you have upgrades of Vault itself. Now, it depends on how you approach this. If you are someone that stays on top of upgrades, so when a new Vault comes out, you upgrade in some kind of development environment, you test out the application, you make sure it all works, and then you kind of roll it up to the rest of your environments, then upgrades can be, dare I say, straightforward. But honestly speaking, as someone who worked with many organizations who were hosting Vault, I don’t think I came across a single organization that were on top of upgrades.

So what will happen is they are on a specific version, then something will happen one day where they run into an issue, and they think “Oh, it’s a bug.” And maybe it is a bug, right? How do you fix that? “Oh yeah, we’ve fixed that already. It’s in this version here.” Okay, you need to upgrade to that version. How do you get through all of these minor, intermediate and major versions to kind of do that upgrade there?

Sometimes the upgrade path is not so clear to understand. Do you have to go to the next version, and then the next version, and then the next version? Or can you do a one-hop jump to your target version? And sometimes the reason why you upgrade is not even because there’s an issue, it’s because there’s new functionality and you want to utilize it to make your lives a bit easier.

So if I’m being honest, from my point of view, I don’t want to manage all of that stuff there. I don’t care enough for that type of operations. In fact, if someone will do it for me, I’d much prefer that, because all the things like your SLAs for downtime and so on and so forth become your responsibility and accountability… Whereas you have professionals who can look after these things here for you, and it becomes their responsibility and accountability. And that sounds super-sweet to me. Let me just focus on building the thing that makes the business profits, it builds a small market share, rather than thinking about things like Vault. That’s not really what I want to do. I’m just an application developer. It turns out I need to encrypt some stuff, it turns out I need to access a few secrets to get to different parts of our platform… Yeah, I just need to know where to get that information, or my application needs to know where to get that information, and let’s go. Let’s go and build greatness.

That’s why when Rosemary talks about the best way to get started is kind of HCP. It’s actually interesting, because I remember years ago that was one of my startup ideas was HCP. I was like, “Oh, imagine if I could just run people’s Vaults for them. That’d be super-cool.” And you know, I never really got down to the drawing board or figuring out what that would look like, and so on and so forth. And I’ll tell you what, the people here are HashiCorp have done such a tremendous job. A far, far better job than I ever could have imagined. But there’s a reason why I came up with that idea, is because honestly, managing platforms - there’s a lot of things to do. So if there’s one less thing that you can manage, especially when it underpins your entire security, I wouldn’t see why you wouldn’t want to offset that.

Paige Bailey: [19:56] That is a great, great question. I personally prefer – hm, let me back-track… So if you have a very, very large model, you should probably be training it using TensorFlow Core, and then exporting it as a saved model to TensorFlow.js. The friendly 10 lines of code syntax that I showed you before - that works to export as a saved model.

If you have a small machine learning problem, it’s fine to train it in TensorFlow.js. You might not see as high of accuracy as you would expect from a TensorFlow Core model, but it will still be good enough to solve the task. And then also the slowness factor… So if you train in a browser, using TensorFlow.js, it will probably be a bit slower to train than if you use TensorFlow Core… And especially for large-scale models. So the small-scale stuff - it will still train in a reasonable amount of time; the large-scale stuff - it might be 10 to 15 times slower.

I think that there’s a benchmark on the website as well, to kind of give an idea… Yeah, let me send a link over to that guide. So the idea is that, for the most part, you can train lots of models using TensorFlow.js. They’re creating a models repo… Let me pull up the link for that one as well. But the idea for that is sort of a model garden where, again, you don’t have to know everything in the world about neural networks and machine learning in order to implement and to use the things that researchers have created. So that link right there is a link to tfjs models on GitHub. You can see mobilenet, so classifying images with labels, posenet, so the real-time pose detection that you can see in the browser, object detection, speech commands, k-nearest neighbors, and they’re growing out this repo pretty substantially and very, very quickly. You can use off-the-shelf, state of the art models without having to understand the machine learning internals.

Wes Bos: [40:12] Yeah, program bore. And they’re like, “Well thanks, but that doesn’t help me, because I don’t know what to build.” I was lucky enough that I’m always curious and I always have ideas of stuff I wanna build, but some people are not that way, or they’re just sort of sitting there and they need somebody to guide them through it. So I came up with this JavaScript 30, which is just 30 projects that are between 10 minutes and 30 minutes long.

It’s totally all over the place - it’s from webcam stuff, to speech detection, to creating speech, to just doing basic DOM stuff, understanding how event listeners work, and clicks, and ES6 and what not… And people seem to really like it, because it’s just a great way of – I don’t know, it’s kind of a neat way to learn modern JavaScript.