Sampling is a pretty intuitive concept. To explain it, I’ll flashback to 10-year-old me, drunk and in a back alley casino.

I lean over the roulette wheel, clutching my head as I see my life savings taken away. How did it come to this? I’ve rolled a 00 three out of the ten times I’ve rolled this wheel of my life, where there are a total of 37 notches. I’m unable to leave, desperately clinging on to ten years of my savings fitted into the chip rack.

I try distracting myself with numbers. What the probability of rolling my three 00’s? Well there should be a 137{37} chance of a 0000⁠, and so 3637{37} chance of not rolling it. Multiplying by the permutations gave me an unlikely result of (103)(137)3(137)7=0.002{3}\left({37}\right)^3\left({37}\right)^7=0.002.

My mood worsened. I should never have emptied my entire piggy bank into chips. Am I doomed to waste my life at the ripe age of 10?

No! I will not simply take this as a lesson about gambling. My purpose is noble, and my heart is true. Luck cannot be so out of my favor. After half of the people leave, drudgingly, with empty pockets, followed by the other half which ripped their pockets from their tear-soaked hands, I slither my way back to the roulette table. With not even the managers left, I set my notebook on the rim of the wheel. Pulling out a synthetic ivory ball from my pocket, I spin the wheel 200 times, recording the results.

I wrote down 5 times 00 was rolled, 97 times green was rolled, and 98 times red was rolled. Now I did some arithmetic expecting a rigged game and daydreaming about a very angry complaint to the Washington State Gambling Commission. 5200=0.025=137{200} = 0.025={37}⁠, 97200=0.485=1837{200} = 0.485={37}⁠, 98200=0.49=1837{200} = 0.49={37}.

\@\\#

The point still stands though. Sampling takes the values of a random variable and generalizes to describe the random variable as a whole. Where in our case the random variable was the final notch our ball rolls on.

This might sound light right? WRONG! What if we want to sample a really disgusting random variable XX which has a probability density function of f:(−2⁠,2)→R>0f:(-2,2)\rightarrow _{>0}: f(x)=e−x4(12cos⁡2(x)+3sin⁡2(x))f(x)=e^{-x^4}\left({2}\cos^2(x) + 3\sin^2(x)\right) images/ugly distribution.png

Remember, when we sample, we want to compile a list of values of XX (which are the values on the x-axis) with a frequency that matches the density function.

Hmmm, this is difficult. To sample, as always, we’ll have to go back to playing games. Consider creating a dart board behind this distribution, which has a length 44⁠, and a height 1.2131.213: images/ugly dartboard.png

Now let’s situate ourselves with a few hundred darts standing before the board. Due to how intoxicated we are, every throw lands at a uniformly random location on the rectangle. Like the monkeys we are, we begin throwing. Every time it lands above the distribution the dart bounces off and we try again. If it lands under the distribution the dart stays there. At the end we write down the x value of all the darts. Now notice that a part with a thicker distribution has a proportionally higher chance of being sampled than a thinner part, so thus we sampled the graph!

Ok, now you might be looking for some real-world examples since you’ve only heard one about a drunk ten-year-old hitting up the casino (which is pretty relatable). One that comes to mind is that people often prefer discreet models (ie. histograms) to continuous ones. To do this, we can sample the density function a lot of times and create a histogram at intervals of XX values with a height based on the number of times XX was sampled in that interval.

Enjoy the darts!... and statistics too!