Importance and Purpose of Keynotes: Nick Walsh discusses the significance of keynotes in DevRel, explaining they aren't just promotional but play a crucial role in aligning a company's messaging, building brand preference, and educating and inspiring both external and internal audiences.
Effective Keynote Elements: Keynotes should make audiences feel informed, inspired, confident, curious, and excited about the technology. This involves storytelling, visuals, and structuring the narrative to resonate with developers specifically.
Developer-Centric Storytelling: Walsh highlights the importance of crafting technical stories that align with developers' experiences, using data-driven demonstrations, clear architecture diagrams, and relatable presenters to foster a genuine connection.
Audience Awareness and Cultural Sensitivity: Effective global communication in keynotes requires sensitivity to regional differences in humor, examples, and references, making collaboration with local teams essential.
Continuous Improvement and Measurement: He emphasizes the importance of feedback and data to refine keynote content. Measuring audience response and reassessing assumptions ensure keynotes remain impactful and relevant.
(00:00) all right uh thank you for coming out uh hope you're enjoying de Al con my name is Nick Walsh I'm extremely fortunate to have what I think is honestly one of the coolest jobs in the world I get to work on Keynotes demos and lots of other really fun strategic projects at AWS and while no two days or Keynotes are the same I've gotten a unique perspective as someone who starts on day one with what do we want to talk about what do we want to show to ultimately owning what shows up on stage so today I'm going to talk a
(00:26) little bit about a topic that's near and dear to me Keynotes but I think there's a lot of interesting lessons that I'll share that could be applicable to your technical content no matter what it is that you work on so I'm sure we've all seen a keynote from that company before quick show of hands who's seen a keynote before you've probably seen one at this event too although looks very different so in each of our heads we can probably think of the good ones the bad ones and most of them probably mundane somewhere
(00:48) in the middle but today I have a question I want to ask all of you and that's how would you do things differently a few years ago this was the exact same question I found myself being asked through a series of what I'll only describe as series of uh Happy unfortunate accidents I was faced with the scariest proposition of all a blank sheet of paper the pen and a deadline for a keynote our company was set to deliver and with a bit of existential dread and some soul searching I found the North Star and it was this I was
(01:14) going to write the Keynotes that I wanted to see as a developer and longtime user of our products I had lots of opinions about how we could show up better for our customers and as someone in Devil this is our superpower collectively the ability to know the customer not because we've spent years researching and studying them but because that's us we are the customer today I'd like to walk through a number of the questions that have followed since this day and share a number of those lessons hopefully a number of them
(01:39) will be helpful for you so the first is why do Keynotes matter what are they what do they do who are they for well some of these may seem really obvious like the external ones Keynotes are a primary vehicle to engender a preference for your brand educate your audience on technical Concepts establish thought leadership entertain and Inspire your viewers showcase your value like de and sustainability highlight the customers and Inspire for what's possible but less lesser known is the internal benefits of some of these
(02:07) Keynotes they align your key messaging as a company they set the tone for how you're going to speak to your audience both in the field on the big stage and Beyond provides recognition for internal teams and more and when I compiled this list I figured doesn't this sound awfully familiar in developer relations these are a lot of the core vectors that we use in divining our work and figuring out whether it's successful and so the more I thought about it you Keynotes are really just another way and one of the
(02:32) primary ways your company shows up for developers when people ask me what I do yes I make Keynotes and my day-to-day involves lots of different things but what I really tell them and it needs no explanation in this room is that I'm a developer Advocate that just sits in a very different seat now a days at other companies a Super Bowl or an ad might be their big thing but for developers it's how we get up on a stage how we tell a story and how we show them what they can build with our products so with this we
(02:58) want to ask ourselves what makes for a good keynote is it the turtleneck sweaters is it the leather jackets they may look nice but really these are not the things that make things great instead for my money I'd say that a good keynote is defined by how you make the audience feel and there's a few different emotions that I think are really critical for uh us to land informed about the latest technology news announcements inspired seeing examples of what's come before and what may be possible for them confident that
(03:26) in choosing your company or product that they'll have the tools support and more necessary to make use of these things curious as to what the art of the possible might be and excited about what the future will bring when all of these things come together ultimately if you're able to land each of these things these emotions your audience will walk away from you get what your business wants which is awareness perception lift organic evangelism purchase and long long-term loyalty and Trust the ways you
(03:52) measure each of these things is a conversation to itself I'm not going to get into it but if you're looking at things in this way I think you're on the right track so if we know that we want to drive these tions how can we do that well you want to tell a story very few textbooks or even documentation ever made people feel and while informational they didn't reinvent the ways that people would think about things that's what stories did there are a lot of stories that exist in the world and
(04:15) there's lots of breakdowns on how well- defined the formats are for telling these successfully but at their core a story is just a vessel for delivering a series of related events or details and what happens between each of the book ends is up to you but I think we all know what makes for a bad story too long uninteresting Bad characters is unsatisfying punchline but what about good stories well there are an awful lot of examples we can look towards for inspiration here movies plays TV shows music concerts bedtime stories fables we
(04:41) don't have to reinvent the wheel and we don't have to look far good storytelling is what takes the individual components the pieces of dialogue The Script the visuals the participants the actors the stage and it elevates them to be more than the sum of their parts pair away the genre the language the format they're all just different ways of telling stories and while each of them are made in very different ways they have a certain set of common tools that they use to tell these stories effectively but we're not telling
(05:07) stories for just anybody right we're telling stories for developers the group that's apparently allergic to marketing potentially even divinely discontent the kind that wants to go to war over Vim versus emac so how do we tell stories for developers when I think about this I think back to a quote from Rob Spectre that I heard in 2016 and it's this if there's one thing that I've learned it's that developers regardless of skill level regardless of where they work in the world and regardless of the company that they work
(05:32) at they're all turned on by the same stuff they're all Bonafide technologists that are genuinely enthusiastic about learning new things R Spectre said that back at develon in 2016 in a talk that he gave about scaling developer relations at twio it was true then and it's as true as ever now developers technologists Builders whatever you want to call our core audience they're not homogeneous but they do have a set of shared experiences and tendencies by virtue of what they and we do for work so as technology companies these are
(05:59) some of the the tools we have in our toolbx to tell stories for them some of these are more cut and dry like launches hey we have this new thing here's what it does it's really exciting for these reasons some of it is broader like thought leadership where is this space going how should I be thinking about this case studies what problems did others face and how did they solve them customer testimonials architecture diagrams demos and more the list goes on and when I think about how we assemble these tools I think of these as part of
(06:25) the anatomy of a keynote here's what most of us know as Anatomy there's this uh visual here of the human body with a whole bunch of labels I think I counted about 36 of them labeled and it's comprehensive it's probably not exhaustive but it's got a lot of detail but it doesn't tell us a whole lot about how we would build something like this but if we look at the components on the last slide we see some commonalities some of those parts serve different purposes they may be adjacent or
(06:51) complimentary to one another they form these different systems skeletal nervous circulatory and Beyond and interestingly notice that these systems don't live in discret SE IED parts of the body they're often um interspersed throughout and embedded and this needs to be the case so that uh you can have their function applied broadly so what does this have to do with Keynotes well many of you look at this and you see a two-hour presentation from the audience or on YouTube wherever you're watching but
(07:17) when I look at this this is what I see now if you'll humor me for a second I would say the Keynotes are not that much different from the anatomy slide I showed you before here's all the slides it can be overwhelming but you'd equally be overwhelmed if I was to show you a few 100 frames of a storyboard from your favorite movie but there's a method to the madness here and that's what I want you to be able to take away the first is that like any good set of stories you have some sort of narrative structure an
(07:40) intro and outro three sort of big sections here from this we start with some of our big set pieces these are the launches in this case from this keynote for some people in the audience this is all they're there for they just want to hear the news and it might be their highlight of the entire event so make them pop visually pacing and Beyond but launches themselves don't stand on their own you need system around them just like the body if all you want to do is inform people you could reach out with some
(08:04) patch notes or Roundup blog but that's not what they're what that's not what we are there for uh next momentum and stats audience are technically minded they don't want hand waving developers don't think in words like a lot or many they think in orders of magnitude and performance metrics so what does scale look like this is where we dig in and try to look at the stats and the numbers and find ways to be able to demonstrate these Concepts like how our authentication engine serves over half a
(08:26) billion requests per second or that each day customers launch over 60 million virtual servers on Amazon E2 these are mind mindboggling numbers but they're way more impactful in terms of engendering that trust for your brand and inspiring what you want to get across to your developers next is customer use cases some of the best Showcases of what our products are capable of are the things customers are currently doing with them this does two things one provide social proof hey other people are using this thing but two it helps the customer
(08:54) triangulate between the problem they have the tool or problem we're talking about and how they may use it to solve their problem and the benefits that they'll get from it next technical Graphics architecture diagrams and more developers see around corners and know that these are tools they need to build with they're going to dive deep on them so show them how these tools will fit in their toolbox it's one of the first things they'll want to look for when they want to learn more about your
(09:15) product or when they use it especially at scale so don't leave it chance and include it at the front next service service category product messaging this is the interstitial tissue and Exposition in your talk you can't assume people know everything about your services you live in breathing but if you know your audience odds are some of them may be new to your world and even power users ones that go really deep may not may be familiar with one part of your ecosystem but not others so take them along for
(09:39) the ride make sure you're not leaving anybody uh leaving anybody behind next technical thought leadership statements things like direction we see the industry going takeaways you want to drive the user to I don't love the term technical thought leadership I feel like thought leadership generally gets a bad rep because of the ways it's misused but when done well this can be some of the most enlightening content in your keynote but it has to be done right right what are the ways you're thinking
(10:01) about a technal technological Trend that motivates how we as a company are thinking about building products like in AI these large language models are only getting even bigger more expensive to train so how does that motivate how we're going to be building products this is something the audience wants to hear or what is the direction we see this space going so that we can help developers skate to where the puck will be not to where it is today like in the Advent of microservices creating mental Frameworks
(10:25) like many systems Loosely coupled helping them rationalize how they would use these things and it's not just about checking the boxes for each of these independent things I just went through the flow and composition really does matter in a keynote we're in the business of a 90 plus minute cohesive show nobody would enjoy a movie that's designed to have a different scene for a different type of person where the rest of the scenes bored them or didn't bring them along for the ride and similarly
(10:47) scenes running overly long or extra extraneous scenes that didn't add to the plot they take you out of the moment so we want these wow moments distributed throughout to keep people's attention here are the ones we aimed for but you want to start Strong finish strong and keep the pacing going well throughout there's a lot of other things that don't fall into the buckets of what I've made here a lot of things like narrative garnish and tips and tricks uh but ultimately those are more how you say
(11:10) these things rather than what you're saying themselves and this is the way that I look at a 2hour presentation and so with each of these systems I hope you'll see they need to be embedded throughout holistically you would never have one of these on their own at least not at the scale of a keynote and in the same way you would be silly to think of constraining Bones nerves or muscles to a part of the body powerful story is told by leaning on each of these techniques at the right times and the right proportions to have
(11:31) the right impact I think this auditing process we did it retroactively it could be really powerful for assessing your own work identifying some gaps and thinking about ways you can do even better for your developers and for your customers so here are some of the tenets I've developed that have steered me well over the years disclaimer these are just the ones that I and my team have come to know and they're constant work in progress there are certainly some that I don't have time for here and you may
(11:55) know better in which case let's talk afterwards I'm excited to hear the first is knowing your audience I'm making this zero index because it's the most important thing on the list if someone comes to me and asks what they should talk about in a keynote or how they're going to talk about it my first question is who are you writing for this because the same content presented in different places at different times will be wildly varied in its Effectiveness so who's your audience are they 100 level expert practitioners the
(12:21) 400 level this wildly dictates what you can expect from them to a lesser extent types of assumptions you can make about their knowledge where is your audience we talk a lot lot about localization in devell often on big picture things like language localization and content and docs but I think a lesser expressed Nuance is the difference in Geographic soot technical Trends things like stances on data residency latest JavaScript web framework dour in the US may not be the case somewhere else in the world and even the types of use
(12:48) cases that are seen positively or negatively in region can be a make or break in terms of your content resonating in region so pay mind to it and ask your local teams lastly temporality a talk always occurs at one very particular moment in time while the talk can live on for a while and Evergreen content is really really important it's important to understand what's going on in the broader technology space and customers lives to effectively tailor the content to that moment the content a customer may have
(13:12) wanted to see six months ago may not be the content they want to see today especially in an area where technolog is moving very very quickly next up context is needed for people to care it's easy to assume that the audience can connect the dots and know as much as you do but you need to recognize that you live and breathe your company's products far more than anybody else ever will so here's one of these examples of how you can tell stories it's the three act structure one of the many forms of narrative development that
(13:36) I mentioned before let's take a look at using this in the context of launching a new product first the status quo what's the lay of the land today day and life of a developer in your domain second what are the unsolved challenges with current tools developers working on this in this area struggle with X and Y third the hero arrives this is your launch this is your product that's why today I'm excited to announce Amazon elastic Problem Solver and and there the hero comes to save the day fourth optionally
(14:02) depending on the product how it works you may want to show it in action it's really really easy and this uh this model Works basically for launching any product it's not the only way to do things but it is certainly a way that uh is reliable and and will resonate with your users but it goes beyond just a launch or any one particular topic what about the context of your company your mission and your entire set of products it's probably no Surpise to folks here that AWS has a lot of products I think
(14:25) the count is over 300 uh and making sense of it all can definitely be a challenge so people frequently ask us why do we have so many and so when we were thinking about this problem and how to communicate it uh we came up with this mental model that complex systems evolved from simpler systems we reached out to some of the Primitives from engineering and physics with an overview of simple machines it would be impossible to build every car or complex machine the customers would want because of the variety of needs the customers
(14:51) would have but ultimately by providing the right Primitives we would enable customers to create what they need in as performative a way as possible allowing the breath and freedom of things customers would want to build with the depth of efficiency and Innovation from our concentrated effort on a small Set uh or growing set of powerful Primitives next simplify the complex this takes a lot of different forms analogies visuals mental models when it comes to computer science in our world there's no shortage of technically
(15:19) accurate but incomprehensible explanations for Concepts Mastery comes from being able to carve out Simplicity without losing accuracy here's one uh it's concept called eraser coding and this is the Wikipedia page for read Solomon encoding it's a form of eraser coding used for data storage and transmission it's a really lowlevel sort of concept there's a whole lot of math here formulas a funky line graph for the log axis it's a lot to digest and honestly kind of gross so for the sake of the story we were
(15:46) telling we really only needed the audience to understand the macro concept A lot of the detail was extraneous and what's really key here is that there's an objects that can be broken down into chunks and then reassembled from subsets of that so we sketch it out and then it just really starts to come together when we really get into animation without even having more T track the relationships between these entities start to become far more clear it's just simple geometry and movement I made this animation in PowerPoint but
(16:11) this goes a long way in making these complex dense topics easy for people to understand or take another mechanism Jokes which can work really well for lightening the mood on otherwise dry subject matter one of my favorites is this idea that these llms large language models are a bit like swiss cheese they have areas of information information density areas of information sparity which are the gaps and these huc hallucinations are when uh llms are trying to make predictions in the areas of of sparsity so we can use other
(16:39) techniques like fine tuning and guard rails to plug the gaps in the cheese and steer the model search space away from the areas of sparsity thus improving the way it can work for us it's been hilarious for me to see this full bleed image of AI generated cheese across five screens in the javet center but it gets the audien's attention it gives them a laugh and it helps them understand a concept that's understandably quite difficult to grasp next show don't tell it's easy to say that a product experience is easy but
(17:05) it's lazy at best and lying at worst if a user closes their eyes can they understand what what that product looks like from your words alone would they imagine what you built probably not at the end of the day most of what we build on as developers are experiential products so get up there on stage and show it these can take a lot of different forms live coding demos an animated explainer a snazzy Sizzle reel showing all the different uh ways you could interact with products but they're all ways of telling your audience look
(17:33) you don't need to trust me here's what we've built for you and we're showing you that we know how you'll want to use it and that it matches up how you show up here also matters a ton Executives and mainstage key presenters can certainly deliver demos but there are certainly times where seeing someone who's closer to the end user Persona like a developer like a developer Advocate a business analyst what have you uh makes it much easier for the audience to see themselves in the shoes of the Persona on stage and often times
(17:59) can be delivered more authentically next make it personal the best stories are the ones that are unique to the speaker I think we all remember the first application we built at works or the time we broke prod or we tried learning a new programming language these probably evoke very visceral memories not all of all not all of which are great but they make us feel and that's something that's very real from your past and while many of these Keynotes are put together by corporations they're always told by a
(18:24) person a Storyteller things like caring deeply about the system performance on a talk I with writing since the very beginning that executive was able to think back to some of the earliest days of AWS and find an email from Jeff Bezos asking them about how performance was looking for beta customers of AWS in 2006 when we're talking about motivation for technology making a real impact in the world another executive I work with real um realized that his move into computer science and technology was largely driven by his motivation to
(18:52) scale his impact he previously worked in radiology and he realized that by working on computer systems he would be able to help far more people than he would otherwise these stories can't really be faked you definitely don't want to do it as Devall practitioners we know when something feels authentic and when it doesn't and this is really important as our audiences are hyper sensitive to this sort of thing the stories are there you may have to dig but they make all the difference in landing what you want to
(19:16) say next people often ask the question how long should it be and my answer for Keynotes is actually the same answer I have for anything else it should be as long as it needs to be and no longer it's easy to run long but it's not respectful of your audience's time selfishly if you run long-winded you run the risk of people disconnecting and tuning out for the things they do tune into you risk diluting the impact of those statements you're trying to make there's no fixed amount or magic Silver
(19:40) Bullet everyone always wants the answer the slight of hand here is in being really really honest with yourself and doing rigorous self assessment of your content to make sure that you know what you're trying to accomplish and how long you really need eliminating everything else that is extraneous to that and making it really clear you can always dive deeper elsewhere you can't regain your audience's attention or trust next think big it may be cliche but the technology we're representing is likely changing the world in one way or
(20:05) another so that way we show up should probably be similarly interesting or grandiose one of my favorite examples of this is this AWS snowmobile launch from reinvent in 2016 uh it's a niche service not tons of customers but it shows the scale of the problems we work on and lengths we will go to solve this for customers uh this this snowmobile is basically a massive truck that helps exfiltrate data from your on- premise says data store and migrate to the cloud more quickly because at certain scales the speed of data the bandwidth of data
(20:34) going down the highway on a truck is faster than the highest Uplink you could have over the Internet so your company may not have an endtoend data Trucking with an 18wheeler as a service but I implore you to think what's your company's examples of this of going the extra mile it may not even be a product at all I said before the customers will always surprise you on what they do and we heard that in the last talk too um this photo was taken on a cell phone inside the International Space Station where our Edge device service is being
(20:58) used for remote compute um I love this because there's no editing there's no like Photoshop in this it's just a photo of a customer taking a picture that's already doing something really cool uh and these are my favorite types of stories to be able to share with the world and in another case where something is supposed to be nearly indestructible uh why not show that why not take a flamethrower to it uh and show it living to tell the tale these are the ways that we make these Concepts more real for our customers and more
(21:24) exciting so next and at risk of sounding like a skeptic is to question everything thing um I never wrote Keynotes for anyone else before I entered this job I never even wrote talks for anyone else before I entered this job so if you don't understand how something works question helps you to get more knowledge uh I think day-to-day I ask way more questions and do way more learning than I do writing or explaining but it's one of those measure twice cut one sort of scenarios questions can lead you to new
(21:47) heights break you out of the mold that came before um and this is something I truly believe is necessary to do things that make history um questions like why do we do things a certain way should we continue doing it like this or that crazy idea comes to you in the shower sometimes the most powerful question of all is one as simple is why not why not drive a truck out on stage in the middle of the keynote Hall why not put full-bleed photos of cheese to explain AI Concepts why not put a developer advocate in charge of this keynote thing
(22:13) the Ingenuity of developers and are entire industry can't be understated we live in an era where the computers in our pockets were just the thing of sci-fi just a decade or two ago nobody knows for certain where this whole thing leads so lean in don't be afraid to experiment and think differently don't do things just because it's how it's been done before so here's my list of tenants this list has certainly evolved over time but I can stand by each of these as things that will help improve your content if
(22:35) there was one thing you take away from this talk hopefully it's these so thank you um it was a real pleasure to get to share these stories I'll be here uh for the rest of the event my team is also hiring if you're interested in hearing more about Keynotes AWS or chatting about our work I have lots of fun stories to share um I'll be around at the event I think I have a little extra time also could do Q&A yeah Q&A anyone have question questions all right I'm coming for you that's a great presentation and um I
(23:15) had a question for you with doing presentations do you try to stay away from pop culture stuff and memes does it feel dated or is it a good way to connect with people wondering what your opinion is yeah great question um pop culture and I think one of the things that I have learned in this role particularly at such a global company is that you start to uh become more sensitive to things that may be funny or like a good match for yourself but then you can be ignorant to how it may not resonate as resonate as well
(23:46) internationally to a broader audience and so um I think when you're trying to make cultural references in particular memes are just one example of that I would try to do that in a like hyper localized fashion like if I'm working working at a talk that's delivered at one event in one region but broadly I feel like sometimes there's a better way to do it in a in a more accessible broader context that may even be less moment in time as an example I'll give one example of that actually um we had a
(24:15) talk we were trying to conceptualize how much data was being generated in the world and the need to store it and process it and all of this and so the number was was something very large on this scale of exabytes and so we were like okay like how do we conceptualize this number well I took like the latest run-ofthe-mill like data center hard drive size and said okay how many of those would it take and then if you stacked them in a tower it would be like some amount of like uh like 1.
(24:41) 5 times the moon in back or something like that which feels very visceral people can kind of conceptualize that and I said okay well what if we localize this to a whole bunch of different countries and I just kind of sheepishly went on Wikipedia found the tallest buildings in every country assumed people would all know what they were and after putting these drafts in front of some of my uh uh teammates globally they said oh well you may not realize this but this building is kind of riddled with Scandal or this this building it's very tall but
(25:05) it's in the middle of nowhere and nobody knows it and so this is just one of these examples where even with the best intentions um and something that may not be feeling as particular as like even a meme uh you may you you don't know what you don't know and so that's where I've really like one developed my Spidey Sense for what types of content I should do in making those references but also just being really cognizant of working with local teams because you'll never know better than they will
(25:30) thank you um thank you for your presentation that was very insightful um I do have a question about like if you want to get into doing Keynotes like do you have any you knew I was going to ask that you're laughing no it's um like how did I know that this question was going to come up um I'm curious if you have any recommendations on like how somebody goes about actually getting into speaking at Keynotes like there's usually not really cfps that are aligned for that it's usually like selected and
(25:59) handpicked so yeah it's a great question so like how do you get into kyotes well uh I'll start with the caveat that before I did this job I don't think it quite existed as a full-time role in the way that it does like the role was kind of created and invented I think in the same way that developer relations is just this intersection of like Communications uh technical software development marketing um strategic communic like there just all of these different things if you work at a company which I think any of us do that
(26:26) that perform all of these tasks find what your highest value your highest leverage Communication channel is for me at my company that happens to be Keynotes right you may not be giving Keynotes in on massive stages in the same way but the ability to have someone that can be a single threaded owner or at least a feedback provider which is a great foot in the door to be able to then hold the pen on some of these things is the best way I would go about it I don't think there's many people that would say no to another well
(26:49) positioned voice in the room to help guide that and if your companies working on particular things like a demo or a launch of a product those are great individual work streams that ladder up to a keynote that you could be able to start getting FaceTime with Executives and and a body of work behind you that ultimately can build your trust internally at the organization which is largely what happened for me I joked it was a series of happy accidents but it was just a whole lot of different uh things I had worked on uh with
(27:14) visibility that um ultimately resulted in me being in the right place at the right time any other qu any other questions now we got one that was great thank you um I've got a question when you're presenting a keynote that you expect is going to be good but it bombs how do you get the feedback like how do you know why and what you should be doing differently unless it's never bombed in which case ignore my question no I I wish I could say I've uh had ideas that were uh only good uh that's definitely not the case
(27:55) um thankfully most of them get shot down in the writer room before we get on stage but uh one thing I didn't talk too much about is measurement um yes it's easy to think like as developer Advocates we are our customer but we're there's always some degree of separation and so you want to measure you want to assess and reassess your assumptions about your customers and your content so there's qualitative and quantitative ways of doing this but the short answer is it's an and not an or we do surveys
(28:18) where we ask people to rate quantitatively what they felt about the keynote distributed on vectors like the speaker and the content and the visuals so that we can actually start to separate the way they feel about it um but separately um the the qualitative responses what are people saying on social media what are people saying out in the wild in the hallway when you talk to them um what are some of our most invested customers or our community Heroes saying you take all of this and holistically analyze it to understand
(28:44) one whether they liked it because no matter how much you think you know you need to ask but two why they felt that way um the things that they really like those are things we're going to want to do more of and the things that they didn't well that's a lesson that we hopefully learn and then don't repeat again again um so just a whole lot of ear to the ground measurement um is the way we do it but it's holistic it's not like we put into a formula and it tells us how much time we should spend on
(29:07) different things anyone else okay let's give it up for Nick thank you everybody [Applause]
