A dinner party at Data Skeptic HQ helps teach the uses of fractional factorial design for studying 2-way interactions.
Data Skeptic
[MINI] Fractional Factorial Design
(upbeat music) - Data skeptic mini episodes provide high level descriptions of key concepts related to data science and skepticism. - Today's topic is fractional factorial design. - So Linda, we're having a little dinner party here, huh? Well, who's here with us? - I have my sister and Julian. You guys are sort of on the ambient mic. - Hey. - Thank you guys for coming. So you know how we talk about cornbread a lot in the show after it's- - You do, you talk about cornbread. - Well, we get a lot of response and your recipe has been very requested. - Yeah. - So I wanted to do some variance on it and I thought like maybe it should have an extra ingredient. So I thought up three new ingredients to put into it and I want to see if they're any good, right? If I've improved your recipe or not. - Okay, so it's an experiment? - Yeah, now there's the possibility that secret ingredients could mix, right? Like do you like ice cream? - Sure. - How about guacamole? - Sure. - Would you like some ice cream with a scuba guacamole on top? - No, but ice cream and avocado sounds nice. - Yeah, I can see that. But even though you like the things separately, together it's a negative. So that's what's called a confounding factor or an interaction. And that's a two-way interaction. There are two ways that are between two variables. There are also three ways, but it's really rare. Do these individual ingredients, do they make the recipe better? And do they in combinations conflict with one another? And we want to study these confounding factors. So we have three secret ingredients and we're gonna check for two-level interactions. So we call that a three-factor, 'cause there's three variables, right? Could have secret ingredient one, two, or three. - Okay. - The question we're all gonna ask is we sample is, does this recipe impress you? Now what do you think it means to be impressed by a recipe? - It means memorable and I want to eat it again. - Yeah, exactly. So like it stands out. Here's one thing that could happen. Perhaps because we have two guests here that we invited over for dinner, they might feel like socially obliged to say that everything's amazing, right? Like how would it be to go to someone's house and say like, this tastes terrible? - Yeah. - Have you ever done that? - No. - Really? I feel like if anyone I know would do that, you would do that. - Nope. - Okay. - Food is sacred. (laughs) There's certain like social impetus, you have to be polite. So they might say they're impressed by everything. In which case, you would actually say they're actually impressed by the defining relationship as if any cornbread is just good cornbread. Give me any kind of it. But they might say some are impressive and some are not. In which case we want to know, oh, is it a particular ingredient they like? Like every time ingredient A shows up. So I could tell them like, hey, next time you make cornbread, you want to put this in it? Or maybe they'll like two of the ingredients, but not together. Like maybe they would like ice cream and maybe they'd like guacamole, but not a bowl of both. Fractional factorial design is a way of setting up an experiment so that you can figure out not only which factors people like or have an effect. And in this case, the effect is being impressed. But also check if there's a relationship between the factors. And we can do that by only eating eight different versions. All right, so should I go get the eight versions? - Oh yeah. - Okay. - I'm very hungry now. - All right, let's go. - The three secret ingredients that I chose that my guests do not know about are honey, peanut butter, and grilled onions. So since those factors can either be present or not, that's two combinations to the third power since there's three of them, eight different cornbreads we're going to taste. And we're going to try and use fractional factorial design here to figure out which of those ingredients if any, impress my dinner guests. - We might not like because you get different combos. - I mean, visually they all look pretty similar. You can see like specks though. - I don't know what kind of method it is though. I don't know either. But peanut, honestly, I feel like they're all really good. - Well, that would be a disappointing result for the show. (laughs) But what it would mean is that it could mean that, yeah, either I pick three things that are all coincidentally good and good together, or that is to say their goodness is non-interacting. They're independently good. Or it could mean that you simply enjoy what's called the defining relationship like cornbread, plain, and simple. - I've just finished all of them. - So I don't really have any favorites, actually, 'cause I thought they were so good. I would have to think about what my favorite... - Really? - I think maybe the walnut ones, and the ones like three or four. I think I like those. Those are really good, or you go with the really sweet one, which is number seven. They were all too strictly good, so that would make sense 'cause I love cornbread. - But should I put you down for three and four? - Yeah, you could say three and four. - Okay, so Linda likes three and four. Whenever either you use ready, also we have some appetizers here. These are homemade whole wheat crackers with a bacon, mustard, sauce, and pickled, what do you call these, pickled, radishes. - Did you make the crackers? - Yeah, yeah, yeah, I made the crackers. - So Kim and Julien, so wait. - I feel like I'm in Chicago. - Are you asking them to speak into the mic, or? - Yes, they have a mic, either one of them can comment them. If any of these at all were impressive. - My top three, and I definitely think they were biased. They were definitely more in the sweeter side. I think there was a roasted onion or roasted element that was more savory that I loved. So my top three are number eight, the last one. I wrote that, I just love this. It's just roasted and sweet, and I definitely think it was savory. And then I really enjoyed number seven too. It was kind of sweet, and it was nutty. Maybe there was applesauce, I'm not sure, molasses. It was just, it was unctuous, I liked it. And then number two, which had, I think, like a roasted onion, and it was just delicious. Those are my top favorites. - They were all really good, so I can see why you said that. - Yeah, they were delicious. I definitely tended towards the more, whatever that roasted element was. - I feel like you like sweet and savory. - I do, I like you mixed. - So let's see, I kind of figured out like maybe the three flavors could be like a peanut, or kind of a nutty element. There was definitely some caramelized onion and one of these in a couple of these, which is something that I really liked. And like some kind of sugar or sweetener was towards the end there, '78. I did like number two, which is like this caramelized onion and chive is very savory and like good, but it didn't have that nutty flavor that some of the other ones had. Number three was good, and it did have that nutty, onion flavor, it could have been like a peanut or something. And I did like number eight, which was kind of the sweet caramelized onion flavor one. So those were the ones that I chose. - All right, so Julian basically guessed it very closely. The three secret ingredients were peanut butter, honey and caramelized onions. So that is what was in combinations in each of these. So interestingly enough, Linda, you liked your two picks, your top two were peanut butter and onions, as well as honey, peanut butter and onions. So this would imply that you are experiencing a two way interaction. You prefer when peanut butter and onions are present, you're indifferent to honey. You didn't pick out peanut butter alone or onions alone. So we would conclude that you have a taste for a two way interaction. Perhaps we call that a more complex flavor is what you like. - Wow. (laughs) And I'm betting Kim picked the ones that had more honey, 'cause I thought she picked a really sweet one. - Super savory, super sweet. - Interestingly, Kim has a similar interaction. She liked the honey alone, the onion alone and the honey with onions, which would imply that you didn't care for the peanut butter, you liked the first two ingredients, but your appreciation of them appears to be independent. There does not seem to be a two way interaction as there was with Linda, because you also liked those ingredients independently. So that is what fractional factorial design helps expose. - Does that mean I'll just eat anything? (laughs) - Is that what that means? - That is it. - Inconclusive, but certainly anything with onions, caramelized onions are honey. And then, let's see, Julian picked interestingly, his three choices, or top three, I guess it was, were onions, honey and onions, and all three together. So the study here would conclude that onions were, the caramelized onions were the driving factor in your decision making, and that you had no two way interactions with the other ingredients. And actually, you might have, because honey and onions, you liked peanut butter, so you could have a two way interaction with honey. We don't know about peanut butter, 'cause we don't have a three way interaction to measure in this, 'cause it's a resolution three experiment. Being able to suss out whether, as Linda did, you like two things together, or as Kim did, you like two things independently, that is what fractional factorial design does. - Wow. - This is really interesting how Kyle can get a feel for what our preferences are in this cornbread taste, based on our top three or four cornbreads, 'cause I feel like all of us had a distinct pattern, like Kim was like, oh, I just like this alone, or these two, sweet and savory, that was the best for Kim. - That's true, that's very interesting. - That's super interesting, I explain how the way I cook. - I like to mix. - I like how I mix all the way. - I think you're a super taster. - No. - No? - I don't think I am, I just can taste like one flavor, you know, and I thought mix was like, I don't know, I just, I like the nutty. I think I was just drawn to nuttiness, and the sweet was too sweet, that's all. - All right, so I don't think we did proper introductions at the beginning, really. We said something, we didn't talk about you guys' company, and the fact that you redid the data skeptic logo. - That's right, it was fun. You're a great client, you gave us good inspiration. - All right, so everyone, you know, lots of people have commented, they're like, in fact, the funniest thing was someone on Twitter, once I put up the new logo you guys made, they were like, oh, do you do this full time now? Which, I don't know if that's a commentary on how bad the logo was before, or how good it is now, or if that's a two-way interaction, I'm not sure. - I love how you use it, it's looking good. - Yeah, yeah, so thank you guys. - Something to any concluding thoughts you have here on a little dinner party? - Wow, cornbread is really, really good with peanut butter, honey, and caramelized onions. - Actually, technically, to your results, it's not good with honey, it's only good with peanut butter or onions. - Ah, I guess I didn't like the honey. - You were indifferent, it was independent for you. - Aw. - Well, thank you guys for joining us, and thank you as always, Linda, and until next time, I want to remind everyone to keep thinking skeptically of and with data. Good night. - For more on this episode, visit data skeptic.com. If you enjoyed the show, please give us a review on iTunes or Stitcher. (upbeat music) [BLANK_AUDIO]