How to calculate the tradeskill difficulty

[Suani]

The skillup formula, when S<=200 and skill<=190, is
P= S*(200-skill) / (Y*F*100*200) where S is the adjusted best relevant stat; the difficulty Y is 2, 3 or 4; F is 1 on a sucessful skillup or 2 on a failed skillup; and skill is your skill when you make the attempt. Y was left for us to find out.

In looking for ways to estimate it, I came across the negative binomial distribution. It give the probablilty P of r successes and x failures when the probability of success is p for one attempt. (I use the Excel docs' definition of r).

Using Excel notation, the general formula is P= negbinomdist(failures, successes, p) and the formula for one success is
P= p * (1-p)^x.

Our situation is more complex because p is different when skilling up on successful combines c from skilling up on failed combines f. So we can't write P= p * (1-p)^(c+f-1). It's still worth it to find a way to simplify failures to a single number to take advantage of what is known about the negative binomial distribution.

We know that the chance of skilling on a sucessful combine is twice that of skilling up on a failure, so let's say that the number of attempts for a single skillup is c/2+f+0.5. (The analysis works better with the +0.5; don't ask me why.)

If we set things up so that p=1/Y, we can calculate the following table for the percentage of occurance of attempts when Y is 2, 3 or 4. (The number of attempts can go to infinity, but I accumulated all the attempts greater or equal to 10 into one line). We can test for the closest match between the distribution of our skillup attempts and these theoretical distributions. I used the Chi2 test. I also noticed from these distributions that the median plus 1 is an estimate of Y.

[B]Trials	Y=2	Y=3	Y=4[/B]
1	50%	33%	25%
2	25%	22%	19%
3	13%	15%	14%
4	6%	10%	11%
5	3%	7%	8%
6	2%	4%	6%
7	1%	3%	4%
8	0%	2%	3%
9	0%	1%	3%
10	0%	3%	8%

We can also simplify the formulas for the negative binomial statistics.

The average number of failures r*(1-p)/p simplifies to Y-1. In other words, Y is equal to the number of attempts when p=1/Y.
Every skillup gives an estimate of Y which we can then average.

The variance r*(1-p)/p^2 simplifies to Y*(Y-1). I used the F ratio to test the difference between results and theory. (The F here is different from the F in the skillup formula; it's just the name of that statistic in the litterature.)

We now need to adjust the skillup data so that p=1/Y. From the skillup formula, we see that p=1/Y when S=100, F=1 and skill=0. So we adjust each individual skillup data to get
Yi = ((c/2+f) +0.5) * (S/100) * ((200-skill)/200) when the skillup came on a combine; and
Yi = ((c/2+f)/2 +0.5) * (S/100) * ((200-skill)/200) when the skillup came on a failed combine.

We have to round those fractional frequencies and we have to set the minmum frequency to 1.

We're working with so few data points that hell runs can distort the results. The best solution is more data, but I did a version of the mean and of the analysis of variance with the Yi capped at 10. I think this is a better compromise than excluding the hell runs altogether.

I got the following results so far:
Baking 3
Brewing 3
Fletching 4
Jewelcraft 4 <- new estimate
Pottery 4
Smithing 2
Tailoring 2

Data for the level-capped skills is not as easy to get. With the technique I used to gather data (explained here), I can get only 4 skillups per character.

I attached the spreadsheet I used to this message. To switch tradeskills, just select the one you want from the drop-down list on the "Analysis" sheet. Trivial levels are included although this analysis does not need them.

Edit:
I added data from a monk on Live and from four magicians with S=150 to the data section of the spreadsheet.

I also changed the estimate for jewelcraft to 4.

I analyse the new data in a following thread

[Suani]

I gathered new data from four magicians on Test who did the first phase of the abysmal freebie tradeskill quest.

I made sure that their S was always 150 by adjusting their /testbuffme equipment so that intelligence was 165 for all tradeskills except smithing and fletching; for the latter, I adjusted intelligence to 150.

I chose S=150 because the skillups take too many attempts with stupid pcs. It still took 2 hours from character creation to questing the adventure stone to skilling up to 31 in all tradeskills even when I streamlined the process as best I could.

Because those four characters had the same max stat and were skilling the same range, I can compare the results of all tradeskills directly; I can also merge the data of all four. I only adjusted the data so that the skill was equivalent for each skillup. (number of attempts * (200-skill)/200). I then calculated the percentage of skillups that came when F=1 (sucessful combines) and the percentage of skillups that came when F=2 (failed combines).

I wanted to know if the results I got were unusual so I simulated data for a million 4-player-character groups with the same characteristics as my magicians. I used the following SAS code

* SAS program Attempts;
* created 2004.05.18;
*****************************************************************;
*;
* Let SAS know where to look for files;
*libname findy 'c:\tradeskills';
*options fmtsearch= (findy);

data generator;
trivial= 31; /* 1st freebie abysmal sea tradeskill quest trivial */
S= 150;
*do S= 100, 150; /* maximum adjusted stat */
  do Y= 2 to 4; /* difficulty */
    do i= 1 to 1000000; /* a million groups of people */
      made= 0; /* number of successful combines */
      failed= 0; /* number of failed combines */
      som= 0; /* number of skillups on a successful combine */
      sof= 0; /* number of skillups on a failed combine */
      do j= 1 to 4; /* groups of 4 people */
        do skill= 0 to 30; /* skillup from 0 to 31 */
          done= 0;
          success= (skill-trivial+66)/100; /* when trivial<=66 */
          adjust= (200-skill)/200;
          do until (done eq 1);
            F= 1+ (ranuni(0) < success);
            if f eq 1 then made+adjust;
            else failed+adjust;
            if ranuni(0) lt s/(y*f)/100 then do;
              if ranuni(0) ge skill/200 then do;
                if f eq 1 then som+1;
                else sof+1;
                done=1;
              end;
            end;
          end;
        end;
      end;
      pctsom= round(100*som/made,1); /* % success on combines */
      pctsof= round(100*sof/failed,1); /* % success on failed */
      output;
    end;
  end;
*end;
run;

ods html body= "c:\tradeskills\attempts BODY.html"
      contents= "c:\tradeskills\attempts TOC.html"
      frame= "c:\tradeskills\attempts FRAME.html";

proc means data=generator mean p1 p5 p10 Q1 median Q3 p90 p95 p99 std skew;
   class y ;
   var pctsom pctsof;
run;

proc freq data=generator;
  tables (pctsom pctsof) * y /nopercent nocol norow;
run;

ods html close;

I got these results


The results are clear when F=1, but there is so much overlap between difficulty levels when F=2 that it's difficult to come to a conclusion. Still, I think the results from successful combines are enough to confirm the first estimates, except for jewelcraft which now looks like a 4.

Some estimates would benefit from more data, but I'm all tradeskilled out at the moment . In particular, the results for smithing when F=2 are unusual. Pottery results (from the full dataset in the spreadsheet linked to my previous message) are a little vague also. Finally, we were specifically told that tailoring is 3. My results don't support this at all. I leave all those worries to others while I return to the game .