Statistical Significance

[Redi]

acetyl salicylic acid is the aspirin just FYI

And acetic acid is what I get when I leave my wine open to the air

[KyrosKrane]

And acetic acid is what I get when I leave my wine open to the air

... isn't that a rather expensive method of getting vinegar?

[Ishwar]

I am pretty certain that befoer you add the effect of having skill over the trivial, the succes chance is 96%. Back when i develloped the succes rate theory I got mya hands on a 200k data point sample that analysed the frequnecy of failure streaks. Succes rate in that smaple was extremely close to 96% (this predates the concept of getting better chanche by having skill high above trivial). The original data iin this thread just confirms that. Unfortunatly this finding never made it into the FAQ on succes rates.

Note that a 95% confidence interval means there is 5% chance you are wrong. I am aware that this is the confidence level used many places, where repeating the experiment lots of times is expensive. But for something like this I would use a higher confidence level if I were to show anything.

[Konjure]

I did my own program to analyze chance to success, streaks-propability, gain-skill-chance (pass1/pass2) paying obnoxious detail to truncating/decimals used in various places, as a hobby project. (And atleast you can't get it to show 2.58% chance to gain skill from 299 to 300 when pass2 is said to cap it to 2.5. )

All this is was kinda neat, but I realized in the end it doesn't really reflect how EQ works, as I don't know how their random numbergenerator works (and a few other details), so it's mostly of academical value.

For example... if you have a RNG that gives you a value from 0 to 65536 and use modulo 100 in order to get an integer value from 0 to 99 (to add +1 and compare to your successchance), you will get an average slightly below 50 instead of 50 because 65536 isn't evenly divided with 100.

That might be one explanation to succeeding more than expected. But then again - random number generators are hard to make, so there are probably other explanations.

[Lofwyr44]

People are reading into this way too much. 4000 combines is an extremely insufficient data sample to try to calculate such a sensitive statistical figure.

Remember, a coin has 50/50 shot of coming up heads or tails. So if you flip it 10 times, you should get 5 heads, and 5 tails. However, because of the statistical bell curve, this only happens roughly 2 out of 3 attempts. That 3rd run of 10 flips, it wont be 50/50. This doesnt change the chance, its just normal statistical varience.

His numbers dont show anything, except that theres no detrimental effects to having higher skill, higher skill mods, or higher level mastery aa. People just percieve it, and blame those things, because clearly its the games fault they fail. Theres a whole psychological term called 'the vividness effect' which delves into why this happens. Its the psychological reason the 1 time oddball things stand out, why testimonials work in advertising, etc.

Everyone remembers the one super lucky, or disasterously unlucky streak. And if it happens right after you gain something new, supersition can quickly assume theres a link. Ngreth disproved that to be exactly what it is, superstition.

If a larger sample size was run, under consistent conditions, then you could apply detailed statistical analysis against the game engine to determine what is actually happening vs what should be happening. But given that it should be 95, and its in the 94-97 range, Id say things appear to be working well as intended. For real this time =p

[Yakk]

People are reading into this way too much. 4000 combines is an extremely insufficient data sample to try to calculate such a sensitive statistical figure.

You do not know what you are talking about.

I can explain exactly how you don't know what you are talking about, but it would add more math noise to this thread. And give the moderator an even bigger headache!

You can draw statistical results from a 4000 sample, and Ngreth's sample is statistically significantly different from a 4000 sample of a 95% success rate trial.

That is why I did the math. Had I not done the math, I could not say that.

If you want more information, send me a PM and I'll toss it at you.

[Lofwyr44]

You can draw statistical results from a 4000 sample, and Ngreth's sample is statistically significantly different from a 4000 sample of a 95% success rate trial.

That is certainly correct, but thats not what I was talking about. You *can* draw a statisitcal analysis from any sized sample. The sample itself is insufficiently large to extrapolate from, to draw conclusions about the larger picture from. It may have variance to other samples its size, but theyre all too small to use as examples of how the engine works overall.

You can say I dont know what Im talking about, all you do is prove it about yourself. The engine takes many more combines than that to hit an accurate distribution, as has been evidenced several times on the boards in the past, which is why there are streaks for players even over very large stretches.

The: "If a larger sample size was run, under consistent conditions, then you could apply detailed statistical analysis against the game engine to determine what is actually happening vs what should be happening" at the end of my post was rather relavent, please dont ignore my own quantifications of my claims.

[Yakk]

Continued in a PM.

[Rwauld]

/sigh.

/pushes the I BELIEVE button.

RNG is fickle, I will fail easy stuff when I really can't afford it (when I have the least materials available.)

When it doesn't matter, I will succeed on stuff I should be failing.

/shrug

[Xodar]

One thousand isn't enough to even out the random number generator. I have a retentive habit of tracking my combines per skill up on paper. 245 pottery to 249 pottery today took me 37, 1, 70, and 1 combines. I have seen runs of 3 digit combines per skill up, interspersed with single digits. It's safe to say that Ngreth proved what he intended to as far as a signifigant reduction in success rate within the margin of error.

On that note, is anyone familiar with hoe eq's random number generator is handled?

[tuis]

SOE has a blind monkey stuck in a rotating drum, of which the inside is covered in numbered buttons. Rumor has it that it is one of the devs.

It is also rumored that the poor thing died years ago, but as long as there is something in there to keep hitting the buttons and no one willing to open the drum, the RNG lives on!

[Lothay]

While the persons involved have taken their discussion to Private Messages, I still believe this thread should be closed.

As such, I'm locking it. People who wish to argue the math of the RNG can beat that dead horse in private messages.