A discussion about getting ready in the morning, negotiating a used car purchase, and selecting the best AirBnB place to stay at help frame a conversation about the decision theoretic principal known as the Value of Information equation.
Data Skeptic
[MINI] Value of Information
[ Music ] >> The Data Skeptic Podcast is a weekly show featuring conversations about skepticism, critical thinking, and data science. >> Welcome back to another mini episode of the Data Skeptic Podcast. I'm here as always with my co-host Linda. >> Howdy. >> So Linda, I've been wondering, do you have any big decisions you need to make in the near future? >> My big decisions on a daily basis revolve around, what am I going to eat for breakfast, and what am I going to wear to work? >> What are some of the useful things that help you decide what to wear to work? >> Well, for me, when I go to work, I work with people who are older and they dress pretty much business casual. So what determines what I wear is the social aspect, which is what is everyone else wearing? >> But you won't know what they're wearing until you get there. >> Well, I obviously knew when I interviewed and when they gave me a tour. And it's obviously if on the first time you go to work, you probably want to dress the most conservative and safe because you don't know what everyone else is wearing. And then once you know what everyone else is wearing, then you can make decisions based on that. And then as you have more data, as you collect, you know, say a month's worth of what everyone else is wearing and what's appropriate, then you have a lot more data to go off of. >> What else goes into helping you decide on more of a day-to-day basis? >> What to wear? What the temperature is like? For me, I parked at a remote watt and then a shuttle buses us in. So what that means is I either have to wait for the shuttle or I have to be willing to walk, you know, eight minutes to my workplace. So for me, it's important that I'm comfortable. And so I think about what kind of shoes to wear with my outfit so I make sure it matches. >> So the thing that varies day-to-day then is the temperature. >> Well, temperature, whether my clothes are dirty or clean, whether or not the clothes fit. For those of you who don't know, I have a small closet, however I manage to pack it with a lot of clothes. So it may be a few years before I try on the same clothing and then when I try it on, it may not fit. >> Can I put a picture of your closet in the show notes? >> No. I'm very embarrassed by my closet because it's also pack or clothes and not very organized. >> All right. So knowing the weather does help you decide. What if you couldn't know the weather for some reason? Could you still make a decision? >> Yeah. >> All right, and if you then let's say we're told the weather, would you make a better or a worse decision? >> Well, if I don't know the weather, then I mean, if I care enough, I may look at the past history of the weather, like make a projection myself. Also, we live in Los Angeles, which is Southern California. So the weather doesn't vary as much as it does on the east coast. For example, the east coast could be freezing one day, hot the next. >> Good point. Let's just assume you were in New York for the week on business. >> Okay. So let's say I didn't know what the weather was going to be. But I'm in New York, so that means I'm traveling. I have a six number of outfits I can wear, and I can't use appreciating information. Like I just arrived in New York that day, is that what you're saying? >> Correct. >> Well, if I wanted to strategize what to wear, I'd probably just wear the most middle ground outfit as possible. Gotcha. So if it's really cold, well, you didn't wear the warmest. And if it's really hot, you didn't wear the coolest, but you're in both the situations least likely to be unhappy, I guess, right? >> Right. >> And what if you did know the weather, what it was going to be? How would your decision making change? >> Well, let's say I knew it was going to be hot. I would probably pick the lightest clothing I had. And then if I knew it was going to be cold, I would bring my sweater, scar, whatever accessories I had. >> So that new information, you could say, informs your decision? >> Yeah. >> And you would make a decision you're happier with, given that information I would assume, right? >> Sure, assuming the information is accurate. >> Now, would you be totally unhappy, completely miserable, worst situation you ever been in if you didn't have the information? >> I mean, that would only be in the extremist cases. For example, I just assumed a t-shirt and jeans would be okay, then a snowstorm happened. >> Yeah, so you're likely to be happier if you have that information and make your decision than if you have to decide without that information. That means the information's valuable to you. How valuable would it be? What would you be willing to pay to know the weather? >> I can't imagine putting a price tag out on it because it's free. >> Yeah, that's true. >> And it's always been free. And then so I'm actually not sure what the consequence would be. >> I would have to know what the consequence was. >> Yeah, so my analogy is breaking down a little bit because weather data is effectively free. But there are other situations where it's not. If you were thinking about buying a used car and not from a dealership just from like a random person, and you need to negotiate the price, right? If you don't know the condition of the car very well, you can still decide to buy it or not buy it. But if you, let's say, took it to a mechanic and got an inspection and an appraisal and that sort of thing, that's information. That helps you make a more effective decision about whether or not you want to buy the car. And in that case, you'd have to pay that person who performed that service for you. So in general, there's a nice little formula. And I know it's hard to do formulas on an audio podcast. But everyone pay attention for just a few seconds. It's not a fancy one. There's no integrals or derivatives. It's an easy one. We can all apply in our everyday lives. If you want to know the value of information, it's the happiness, or I'm going to call it utility, expected utility. You derive from the decision you would make given the information, minus the expected utility you would make from making that decision without the information, minus the cost of acquiring the information. >> Yeah, that's what I said. I said I have to know what it's like if I don't know the weather. I don't need to know the consequence. So this is a handy little formula. And it works in simple situations and more complicated ones. Now there's another challenge there. And let's go back to the weather. How reliable do you find the weather service to be? >> I mean, there's obviously a range of error. I don't know what the range is. It depends on each of the specific properties. You're having humidity or versus the temperature range. But let's just talk about the temperature, because that's probably what people care about most. I mean, nowadays, at least in LA, I'm not measuring it, but versus my car thermometer, how it feels, and the temperature I see that they predict seems accurate within five degrees. >> Yeah, I'd say that's about right. Meteorology is a pretty advanced science, from what I understand. So in that case, but it's still not perfect. So when assessing value of information, you also have to consider how much you trust the information you're going to get, because rarely can we get access to ground truth. So most likely, the information you're going to get has some amount of noise or imperfection in it. So when you do a calculation like this, the way in which you decide the utility you would garner from the decision with the information needs to also include the fact that the information can be slightly flawed. But there's a neat boundary condition here, in that if hypothetically, you could become omnipotent and get access to perfect information, that's the maximum value. No information can be more valuable than perfect information. On the other side of the spectrum, let's go back to your wardrobe decision making. Let's say I was willing to tell you, you're going to get ready in the morning kind of early, and the stock market has not opened yet. What if I could tell you and guarantee it was going to be correct, what the opening price of IBM shares will be? How valuable is this information for you, for the purposes of picking out your wardrobe? Over the wardrobe, I don't care. So that information has zero value? No, unless you could say there's a correlation between the cost and the weather. Fair enough. Let's assume that IBM stock is not at all cold in here with the weather. Good point though, clearly you're learning something from participating in these podcasts. I thought that's where you're heading. I thought you were going to tell me, what if I told you the weather's correlated with these stocks? No. What would be the value? The point I'm trying to make is that if the data doesn't influence your decision, in that context, that information has zero value to you. So if there's some information that's available to you, it has some value between zero and the equivalent of what perfect information would tell you. So assessing whether or not you want to incur the cost of getting information and that cost could come in a couple of ways, in the case of buying a used car, you might have the cost of paying the mechanic to inspect it and make sure it's not a lemon. In the case of dressing while you're in your hotel room away on business, the cost might be your time. In figuring out how do you get your phone onto the hotel Wi-Fi and then take a few minutes going to the website that you want to check your weather on, if the cost of some information is more expensive than the improvement you would get from a more informed decision, it might not be worth it. You might just want to stick with the sighting in ignorance. One last example. There is an activity that I think you enjoy doing this. A lot of people will probably be familiar with it. It's deciding whether or not you want to get a hotel or use Airbnb or let's just say you've committed to Airbnb. Now that can be a good decision or a bad decision. If you commit on just the price alone, you can do that or you can get new information. Now the cost is your investment of time. And then new information comes in the form of looking at photos, reading reviews, maybe doing Google Street View, this sort of thing. Have you found yourself, I know you didn't know the formula before, but using something like it to assess whether or not you want to track down new information to inform your choice about what Airbnb location you might want to stay in or might not. Well, in this case, I think the limiting factor is time. Definitely, the cost is time. And how much you care? Yep. Because I mean, some cities, they are already pretty nice and most neighborhoods are pretty nice, or whichever neighborhood you chose to stay in is a good bet to be nice because they've already stayed there before. So then there is just kind of a baseline of how dirty of a hotel room or how clean that you're willing to put up with. So depending where it falls, I would only continue looking if, for example, one of the hotels or Airbnb's I saw looked extremely dirty or something, or it took away where it made me uncomfortable where I felt like I had to look. Otherwise, I just couldn't live with that decision. So it sounds like you're saying that you can feel trustworthy in certain situations and make a sort of uninformed decision and be comfortable with it. Maybe you're not going to find the best possible place, but you could be a decision you'll be happy with. And that the cost, in this case of time, in pursuing added information, you don't think will allow you to make a significantly more informed decision in order to justify the cost of time. Yeah. I think what he's trying to do is get me to book all the hotels in the future. And I object, object. I book hotels. Some duds. When was a dud? Austin. That doesn't count. Well, thank you again for joining me, Linda. I hope this was informative for both you and my listening audience. I really love the value of information equation. And I find myself talking about it a lot, because I think it's a really useful heuristic for people to apply in figuring out what's useful and how to inform good decisions. We'll see you next time. Goodbye. Thanks for listening to the Data Skeptic podcast. Show notes and more information are available at www.dataskeptic.com. You could follow the show on Twitter at Data Skeptic. If you enjoy the program, please leave us a review on iTunes or Stitcher. A review is the greatest way to show your support. [MUSIC PLAYING] [END PLAYBACK] [MUSIC PLAYING] [END PLAYBACK] [END PLAYBACK] [END PLAYBACK] [END PLAYBACK] [END PLAYBACK] [END PLAYBACK] [END PLAYBACK] [BLANK_AUDIO]