Looking for answers, I fired up my computer and got to work. I would measure how often specific numbers came up whenever I rolled certain numbers of dice. The Law of Large Numbers would ensure that whatever results I got would be roughly the same if someone else repeated this investigation; to this end, I rolled two through twenty-four dice 50,000 times each and recorded how often each value came up.
Here is a sample of the results:
Looking at this data alone, we see one of the earliest things I noticed: the most frequently occurring value is usually a multiple of six. This is because six is the largest value the dice can give you, meaning that it takes very few sixes to exceed the other matched sets.
When we look at larger numbers of dice, we see another phenomenon as well:
These charts exclude values that didn't appear in the results even if it was mathematically possible for that value to appear. But note which type of value is extremely unlikely to appear: a prime number. Prime numbers have only two factors -- itself and 1. In OVA, this means that the only way to get a prime number result that isn't already on the dice (2, 3, and 5) is to get a result where a set of ones decides the roll. Ones are the lowest number the dice can give you, so it takes an awful lot of ones to exceed the other matched sets. Even if enough ones appeared to make a prime number, it would likely get exceeded by sets of the other numbers and thus won't decide the roll's value. Furthermore, a prime number not already on the dice and larger than the number of dice being rolled cannot appear at all.
Another interesting case is semiprimes, or numbers with exactly two prime factors. Excluding six and nine, semiprimes in which one of the factors is either two or three are very uncommon, since two and three not only need the correct number of occurrences in a roll, but also the largest matched set (semiprimes with five, on the other hand, occur frequently since five's larger size allows it to decide the roll more easily.)
Which brings me to my next point. As I looked at the frequencies, I noticed a very large flaw in my old difficulty chart: it used rarely-occurring values as difficulty benchmarks.
The values 14 and 22 stand out. Both are semiprimes, with 14 factoring out to 2*7 and 22 factoring out to 2*11. Not only must these values be arrived at with twos, but the set of twos must exceed all the other sets. As a result, 14 occurs very infrequently and 22 almost never appears at all. This explains why the probabilities for 22 and 24 are nearly identical -- beating 22 is not easier than beating 24 since 22 is so rare that using that value won't improve your chances above beating 24. When rolling less than eleven dice, getting 22 isn't even mathematically possible, so the chances of beating 22 and 24 are absolutely identical when rolling four through ten dice.
Noticing how uncommon some of the benchmarks were, I came up with new difficulty numbers that reflect far more commonly occurring values (all percentages are rounded to the nearest whole number.)
With the exception of 9 and 35, the values that occur between the difficulty numbers are very uncommon; you can estimate the probability of difficulties in between by checking the probability of the larger neighbor (for example, if a 21 turns up, you can estimate its probability by checking the chances of beating 24.) If you are simply declaring difficulties per the rules, these are my recommended numbers to use. You'll also find that these numbers turn up a lot when you roll the dice.
Dissecting OVA's dice-rolling mechanics has helped me become a better Game Master, since it allowed me to see probabilities much more clearly and take the right actions at the table to keep the game interesting. Below, I have provided a link to the Excel spreadsheet where I recorded the frequencies as well as the Python source code of the program I used to find them and derive the new difficulty chart. I've also included the spreadsheet for the difficulty charts. My hope is that my work will also help other OVA Game Masters.
Roll Frequency Data
Difficulty and opposed roll charts
Python source code of program that was used to make all of the tables
But looking at it again, the drop isn’t THAT exorbitant. I think the odds of 4 (not shown here) is like 80%, 6 40%, and 8 close to 10%. So it’s actually a smoother progression for unskilled characters. It’s just that difficult remains at least a little difficult for increasing amounts of dice.