Proving 'Success have a greater chance of giving a skillup'

[calidorn2]

Best skill ups

Used to be fairly common knowledge that your best skill ups were in the range of 10-20 points above your current level.
ie if you are at 100 make items that triv between 110-120.
Always made sense to me but I know of no statistical proof.

[PinyonTreedotter]

kiztent wrote:

I don't know how to calculate a standard deviation, but it looks like there is a correlation in this small run.

What you need to do is set it up as a contingency table. When you do that (combining characters), the Chi-square value is 7.84, much larger than the tabled value for p=0.05 of 3.84 (in fact, the p value is less than 0.01).

Again, it demonstrates that you are more likely to have a skill increase with a success than a failure. This doesn't mean you won't get skill increases with failure -- particularly if you are working on a low-success item, such as Solstice Robes for tailoring. I got (and continue to get) a lot of skill ups with failures.

I should have mentioned in my earlier post that I was using Fisher's Exact test, in case anyone wants to look it up.

[Xorshaik]

Let me see if I am analyzing the data from kiztent correctly. (Maybe I'm trying to do statistics for dummies... I had engineering statistics and passed by virtue of the class average, just like everyone else who couldn't follow the instructor)

When looking at all successes, percent chance of a skillup coming from a combine is 32.8% overall.

When looking at all failures, percent chance of a skillup coming from a combine is 20.3% overall.

Overall, 24.7% percent chance of getting a skillup on any combine.

There were more failures than successes by nearly a 2 to 1 margin (232 to 128)

The likelihood of skillup was, however, greater for success than failure.

The likelihood of skillup on success was also greater than likelihood overall, and the likelihood of skillup on failure was lesser than likelihood overall.

These conclusions hold for each individual trial and for the overall average.

Am I reaching the right conclusion based on the data in concluding that success is more likely to produce skillup?

It would seem this should be a repeatable experiment and doing so should only reduce the error, I wouldn't know if the margin for error means this could be a not yet sound conclusion based on not a large enough sample. (I think there's a big problem because the percent chance of a success yielding skillup within the given data ranges from 37.5% and 23.7%)

[kiztent]

Am I reaching the right conclusion based on the data in concluding that success is more likely to produce skillup?

Yes. I believe the p=.01 means we are 99 percent certain, based on this small sample, that there is some significance to the fact that there are more skill ups on successes versus failures, as a percent. We can't really say WHAT that increased chance to skill up is, but we can be pretty certain there is one.

It would seem this should be a repeatable experiment and doing so should only reduce the error, I wouldn't know if the margin for error means this could be a not yet sound conclusion based on not a large enough sample. (I think there's a big problem because the percent chance of a success yielding skillup within the given data ranges from 37.5% and 23.7%)

There is a problem, but we are dealing with very small samples, statistically speaking.

The reason I chose this particular scenario is because of the spread of data. Given that most people quote skill ups on success when they have 1/20 to skill up and 1/20 to fail, that means you would see one skill up on failure per 400 combines. So getting, say, 0 skill ups on success on 800 combines (where you'd expect 2) isn't a big deal. So, I always questioned the results people threw out when they tracked skilling up in the hell levels (post 180).

By choosing a sample where I am 1/3 to succeed and 1/4 to skill up, you would expect to see 1 skill up on success per 12 combines, and 2 skill ups on failure per 12 combines. This would make the results statistically significant for smaller sample sizes.

I think anyway. It's been :shock: 15 years since my college statistics class.

Anyway, by calculating certainties (p=.05, p=.01) it can tell us how reliable the results are, which puts a handle on our intuition of "hm, there's a lot of spread in the samples, I wonder how accurate this is?"

[Boleslav]

I would look at the data Kiztent supplied this way.

Assume that the average rate of skilling up for this recipe is very close to the sample average for all three runs combined. There
were 89 skillups out of 360 combines, so I will use a 25% skillup rate.

Then I would test each of the runs on success to see if the skillup rate was higher than .25 and all the failures to see if the skillup
rate was lower. You have to be careful when calculating probabilities in this situation. If you are using the probability of getting 18 successes out of 48 combines you will get a small number. But! We are interested in getting AT LEAST 18 successes, so we have to include the probability of getting 19, or 20 ect…

Run #1
What is the probability of getting 18 or more skillups from 48 successful combines if the underlying skillup rate is 25%?
Chance of getting 18 = [48!/(18!*(48-18 )]*.25^18*(1-.25)^(48-18 ) = .019
Chance of getting 19 = [48!/(19!*(48-19)!)]*.25^19*(1-.25)^(48-19) = .001 (etc…)

Chance of getting 18 or more on successes = .0374 (3.74%)

What is the probability of getting 14 or fewer skillups from 72 failed combines if the underlying skillup rate is 25%?

Chance of getting 14 or fewer on failures = .171 (17.1%)

Run #2

Chance of getting 15 or more on successes = .085 (8.5%)
Chance of getting 14 or fewer on failures = .092 (9.2%)

Run #3

Chance of getting 9 or more on successes = .635 (63.5%)
Chance of getting 19 or fewer on failures = .407 (40.7%)

Overall results

Chance of getting 42 or more skillups on 128 successful combines = .029 (2.9%)
Chance of getting 47 or fewer skillups on 232 failed combines = .0617 (6.17%)

Analysis

The data suggest that you are more likely to get a skillup from a success than from a failure. If you calculate the ratio of the
success rates (.328125 for successes and .2025 for failures) it shows a 62% increase in the chances to gain a skillup from a
success rather than a failure.

However, there is great variability within this data set. The third set of data shows an almost equal distribution of skillups from
successes or failures. So while it looks like we have detected a trend, we must be careful. Fully one third of our data does not
support the existence of a trend. I suspect that you do get a better chance to skillup on a success than a failure, but like
everything else in EQ, there will be wild swings in the data. I would like to see this experiment repeated another 50 times and
crunch the data from that pool before trying to say anything definitive the size of the difference in skillup rates.

Boleslav Forgehammer (the math geek)
Palaidn of Brell in his 60th Campaign
E'ci - Desitny Awaits

[kiztent]

I would like to see this experiment repeated another 50 times and crunch the data from that pool before trying to say anything definitive the size of the difference in skillup rates.

Yes, please. It only took me an hour and 90pp to do my three runs. Plus I got heady Kiolas!

[Zeralenn]

Taking the Kiola Challenge - preliminary findings

I took the Kiola Challenge, although I actually ended up doing three experiments in a way, not one.

I made two successive toons, both level 1 Dark Elf mages with 134 INT, just like in Kiz's experiments. Both trained 1 point in Brewing. There were no differences in the two "brewbots" other than names and gender (my first brewbot was female and I got a lot of guys stopping to chat with her at the brew barrel. So I made my second brewbot male and nobody bugged him).

I still think this experiment needs repeating several times, but here is what I got from this round.

Toon1 made Heady Kiolas only from levels 1 to 46. I carefully recorded each combine for successes, failures, and skillups. Instead of stopping at 120 combines, as Kiz did, I went all the way to level 46. So I not only have data for those first 120 combines, but I have data from all the combines that took him to 46.

The results for the first 120 combines (number - skill points - percent chance to skill)

Number of successes: 46, 13 skillups, .28
Number of failures: 72, 13 skillups, .18

However, the more interesting stuff starts to happen when you look at the entire picture -- the 191 skillups it took to get to 46.

Toon 2 did a different experiment. Instead of doing Heady Kiola from 1-46, she took a different skilling path: Bog Juice to 21, Malted Milk to 26, Root Beer to 31, and then Heady Kiola to 46.

A statistics friend of mine is going to take the spreadsheet I did to apply some regressions and chart the data I got, but here are some interesting broad things I found.

Toon1 (Kiola only) completed 191 combines, 1-46.
Toon 2 (Mixed) completed 131 combines, 1-46.

Toon1 (Kiola) averaged 4.3 combines per skillup.
Toon2 (Mixed) averaged 2.9 combines per skillup.

Here's the fascinating thing:

Toon1 and Toon2 had exactly the same rate of SUCCESSES per skillup. Both averaged 1.8 successes per skillup.

What made these two paths stand apart from one another was NOT in how many times one of them succeeded. Stastically, they succeed at the same rate. What stands them apart is the rates at which they FAIL. Toon1 averaged 4.3 combines per skillup at a success rate of 1.8, or 37%. But Toon2 averaged 2.9 combines per skillup at a success rate of 1.8, or a 62%.

If it is true that more success = faster skilling, then it also stands to reason that staying closer to trivials and using geerloks to help ensure success will support faster skilling, not necessarily by ensuring successes, but by minimizing failures. The success rates for both paths were absolutely even. But it was actually fewer failures helped speed the path; the mixed path took 68% of the time taken by the kiola path.

Perhaps a picky distinction, but I think one that is important to understand.

There is obviously a complex relationship here and perhaps further analysis by a trained statistician may help clarify some of the relationships. I hope to have more data on these experiments. I actually hope to repeat both experiments soon.

...Zera

[Palarran]

I'm currently powerlevelling an enchanter on a second account to be used strictly as a tradeskill mule. The original intent was to simply get to 49 to enchant mithril, but I may go ahead and get him jewelcraft mastery 3. (Yes, I'm aware that I need 36 AA's and level 59...I don't expect it to take all that long, especially once he hits 51 to get that bonus for mobs near or above his level being killed. My main is a 65 druid with plenty of practice charm soloing in PoP.) Current jewelcraft skill is 0.

If I think of it, I'll let you know when I have JCM 3, and someone can tell me what the conditions should be when I practice jewelcraft. Please don't make it too expensive, I have a tailoring habit to support.

[Boleslav]

Actually, your data appear to corroborate Kiztent's data to me. Looking at the Kiola toon, it was clear that there was a larger proportion of skillups on successful combines. The averages are a bit different, but the idea is the same.

I like the second toon you did too. It gives a piece of data that indicates you can skill up on fewer combines by keeping trivials close to your current skill level. I will try to find the experiment I referred to above (JC from 0 to 100).

Boleslav Forgehammer
Paladin of Brell in his 60th Campaign
E'ci - Destiny Awaits

[Greymantle]

Negative

Well i don't have hard and fast numbers but, I completed my drive to get my brewing up to master level on the weekend. While i did not keep track of the numbers i did note the sucsess's to skill -ups. I was making fetid essence wich trivials out at 122 i belive. In ALL of the combines i made i got 2 count them, TWO skill increases on a sucsess. All otheres on fails. This was done not all at once but in a series of runs, last run of 100 atempts got me to 101, and only one of the skill ups was on a sucsess.

[Zeralenn]

More experiments

I am in the process of repeating the brewing experiments I did before, with a few additional things thrown in. So far, however, my current experiments are corroborating the data posted here so far.

In my original experiment, I used 2 Dark Elf level 1 agnostic mages with an INT of 134. Experiment 1 did 1-46 on Heady Kiola alone, and Experiment 2 did 1-46 with a smoother skill path using several items with no more than a 20-point trivial jump at any time. I recorded every combine on worksheets that tracked the current skill level, trivial of the item attempted, whether it succeeded, and whether there was a skillup. Without careful recording of actual data, you can't really make any meaningful conclusions.

I have four more experiments planned and am almost finished with the first one. They will all be Level 1 Dark Elf agnostic mages with an INT of 134.

Experiment 3 is doing Heady Kiola 1-46, then doing Fetid Essence 46-122. I am finished with the Heady Kiola part and so far the numbers are very much in line with my original experiment -- details to come.

Experiment 4 will follow the same smoother path from 1-46 as before: bog juice, malted milk, root beer, then heady kiola. Then, Experiment 4 will go on to do Soda Water to 58, fermented yarrow to 74, white wine to 82, paeala tannin to 102, then fetid essence to 122 -- again, never more than 20 points from trivial.

Experiment 5 will be a repeat of Experiment 3, but with a geerlok equipped. Experiment 6 will be a repeat of Experiment 4 with a geerlok equipped. I want to do an iteration of the experiment with a geerlok because I want to examine this question: If you get more skillups on successes than on failures, and a geerlok increases your chance of succeeding, does a geerlok actually contribute to faster skilling?

The theory has always been that a geerlok was for succeeding, not for skilling. But if we eventually accept the idea that more success = faster skilling, then a geerlok WOULD contribute to faster skilling.

Perhaps.

We shall see ...

...Zera

PS. I am not planning an iteration of these experiments with intelligence buffs or equipped intelligence items. It has been proven elsewhere that higher intelligence (or wisdom, or strength in the case of smithing) contributes to faster skilling. Besides, I'm already getting carpal tunnel syndrome just from THESE experiments...

[kiztent]

Re: More experiments

Experiment 4 will follow the same smoother path from 1-46 as before: bog juice, malted milk, root beer, then heady kiola. Then, Experiment 4 will go on to do Soda Water to 58, fermented yarrow to 74, white wine to 82, paeala tannin to 102, then fetid essence to 122 -- again, never more than 20 points from trivial.

I'm guessing you never made fermented yarrow before.

It's a bit pricey to spend on an experiment (26pp per attempt), and with the insane yields, you're talking about a lot of fermented yarrow kicking around.

Ale is 68 trivial, might be saner to use on the experiment.

[Quesci]

Assuming that all of this end up proving that skill ups are more likely on successes than on failures, this begs another question:

How much INT/WIS can one give up to balance the benefit of a Geerlok?

When I wield a Geerlok, I am losing 25 INT (wizard epic). I have always followed the formula of using my epic when I wanted skill points, and the Geerlok when I wanted successes. The main reason that I got my pottery trophy early was so that I could have the +% modifier and still have max INT.

Test: Run a skill set with a toon with 100 INT + Geerlok.
Run a skill set with a toon with 100 INT and no Geerlok.
Compare
Run a skill set with a toon with 110 INT and no Geerlok.
Run a skill set with a toon with 120 INT and no Geerlok.
Run a skill set with a toon with 130 INT and no Geerlok.
Compare

[Zeralenn]

Hmm, you're right -- fermented yarrow out the window, then.

There are certainly other toon-specific experiments that could be run, but I am trying to answer some general and broadly-applicable questions. The intelligence vs. geerlok question is another one, of course, but I'll let someone else do that experiment. I've already set up myself for hours of work to prove points.

Just say a novena for my wrists, please...

...Zera

PS. If you see toons from Innoruuk with wine names, such as Chardonnay, Zinfandel, or Malbec on the serverwide channel that's one of my brew-bots working on these experiments.

[Zeralenn]

Report Number 2: Experiments 3-4 complete through level 46

This is an interim report on my continued experiments into the question of whether increasing success rates increases skillups, and whether choosing a path closer to trivials actually results in faster progress.

To remind folks, the experiment uses creating level 1 characters, Dark Elven mages, Agnostic, with 134 INT and the rest of the points into CHA. I also use a few charisma enhancing items just to improve the prices at which they can buy ingredients. I train 1 point in Brewing.

I use brewing for these experiments because it is relatively cheap. The general principles should be more widely applicable.

Exp 1 and Exp 3 made only Heady Kiola from 1-46. This experiment is the (Kiola) experiment.

Exp 2 and Exp 4 made four items to 46 - bog juice to 21, malted milk to 26, root beer to 31, and
heady kiola to 46. This experiment is the (Mixed) experiment.

--------------------------------------------------------------

The original experiment counted only the first 120 combines. So, to support that experiment, here is the data for the first 120 combines for these four experiments:

The results for the first 120 combines (number - skill points - percent chance to skill)

Exp 1 (Kiola only)
Number of successes: 46, 13 skillups, .28
Number of failures: 72, 13 skillups, .18

Exp 3 (Kiola only)
Number of successes: 39, 17 skillups, .43
Number of failures: 81, 13 skillups, .16


Exp 2 (Mixed)
Number of successes: 72, 31 skillups, .43
Number of failures: 48, 6 skillups, .12

Exp 4 (Mixed)
Number of successes: 63, 26 skillups, .42
Number of failures: 47, 13 skillups, .28

In each case, the percentage chance to skill was greater on skillups than on failures.

--------------------------------------------------------------

However, the more interesting stuff starts to happen when you look at the entire picture -- the skillups it took to get to 46.


Exp 1 (Kiola) completed 191 combines, 1-46.
Exp 3 (Kiola) completed 176 combines, 1-46.
Average number between the two: 184

Exp 2 (Mixed) completed 131 combines, 1-46.
Exp 4 (Mixed) completed 146 combines, 1-46.
Average number between the two: 139

Although Exp 3 (Kiola) had a slightly better run than Exp 1, and Exp 4 had a slightly worse run than Exp 2, it is still clear that staying closer to trivials results in fewer combines.

If you look at the numbers of average combines per level by level, you will see something interesting:

1. The two Mixed experiments generally did much better on skillups in the combines from levels 1-35, averaging 2 combines per skillup in Exp 2 and 3 combines in Exp 4. The two Kiola experiments ranged over 4 combines per skillup each in these level ranges -- 4.2 in Exp 1 4.3 in Exp 3.

2. The level 35-46 skillups, within 10 levels of trivial, is where things get more interesting. All four experiments were working on Heady Kiola at this time. However, in general, the mixed group did significantly worse making Heady Kiola than the Kiola-only group did at this level range.

Exp 1 (Kiola) 3.9 combines/skillup, 35-46
Exp 3 (Kiola) 2.6 combines/skillup, 35-46

Exp 2 (Mixed) 4.6 combines/skillup, 35-46
Exp 4 (Mixed) 4.7 combines/skillup, 35-46

Is there perhaps a hidden role of "practice makes perfect"? Only repeats of this experiment will tell.

-----------------------------------------

Successes and failures per skillup stayed remarkably consistent:

Success rates per skillup:

Exp 1 (Kiola) 1.8
Exp 3 (Kiola) 1.7

Exp 2 (Mixed) 1.8
Exp 4 (Mixed) 1.7

Failure rates per skillup:

Exp 1 (Kiola) 2.6
Exp 3 (Kiola) 2.4

Exp 2 (Mixed) 1.5
Exp 4 (Mixed) 1.5

Again, like the first time I did this experiment, what is remarkable here is that both paths seem to have very similar success rates -- the rates between the two types of experiments differ by only .1.

Exp 1 (Kiola) averaged 4.3 combines per level with 1.8 success (37%)
Exp 3 (Kiola) averaged 3.8 combines per level with 1.7 success (44%)

Exp 2 (Mixed) averaged 2.9 combines per level with 1.8 success (62%)
Exp 4 (Mixed) averaged 3.4 combines per level with 1.7 success (50%)

Even though Exp 3 had a better run and Exp 4 had a worse run, it is still clear that what differentiates the one-item skillup with longer trivial jumps vs. the mixed, smoother trivial path is NOT the rate of success, but the rate of failure.

-----------------------------------------

Preliminary report: I have actually run Exp 3 all the way out to level 122, using Heady Kiola to 46 and Fetid Essence to 122. The only thing I will say about that at this point is that it took 660 combines to reach level 122. I am currently engaging in extending Exp 4 to 122 using a trivial path with no more than a 20-point trivial jump at any time, and so far she is far, far ahead of Exp 3 in terms of progress. But that is for my next report.

...Zera