Announcement

Collapse
No announcement yet.

Proposed Experiment to Test the Eternal Question: success vs. Failure

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by Moraganth
    Ultimately to test this, all anyone needs to do is a bunch of combines.. and record the number of combines, failures, successes, and wether the skillups came on failure or success.
    Eh. In theory, that would be right. The problem is that in practice, your numbers would get heavily skewed by how people skill up.

    Consider JC. If you use Gumkak's guide, you'll always be making something very close to trivial, and fails will be quite low. The vast majority of your skillups will come on successes merely because you will have far more successes then fails.

    Now consider brewing. The common advice for getting started in brewing is to jump directly into Fetid Essence, which is trivial 122. In the first half of your combines, you'll have very few successes indeed, but you'll still be gaining skillups -- in other words, the majority of your skillups would be on fails. The same would apply to baking if you jump directly into Fish Rolls, then on to Patty Melts.

    On the other hand, both baking and brewing have a number of interim recipes you could try, which would raise your success rate and hence, your rate of skillups on success. Likewise, with JC, you could just pick a particular combine and always make that, assuming cost is not an issue. That would likely reduce your success rate.

    This is the main reason folks say that the testing environment has to be controlled. If we standardize on one toon with particular stats, who does only certain combines in a certain order, we eliminate the variables. This would allow us to focus on the data.

    Side note: I've mentioned this before in other threads, but I think it got overlooked. All the test suggestions I've seen say to create a toon and skill it up from zero to X. This does get us some data to work with, but no one has yet checked whether any conclusions derived from this data are applicable to the higher end of tradeskills, particularly the so-called hell levels. It's been repeatedly established that the number of combines per skillup increases rather dramatically as you get past 190 or so. As we gather data from the low-end tests, we have to remember to keep testing the higher-end too, to see whether our conclusions hold across the board.
    Sir KyrosKrane Sylvanblade
    Master Artisan (300 + GM Trophy in all) of Luclin (Veeshan)
    Master Fisherman (200) and possibly Drunk (2xx + 20%), not sober enough to tell!
    Lightbringer, Redeemer, and Valiant servant of Erollisi Marr

    Comment


    • #17
      right.. i see the way it could be skewed number wise depending on how far off trivial it's made... but i'm not sure that would matter using comparisons.. mind you, i never was a math major.

      if you did baking.. fishrolls, then patty melts.. the lower skill levels would be largely fails, with few successes.. but that percentage should carry over to skillups, in either way.

      If 95% of your combines are failures, no difference would show 95% of your skillups on fails. If successes are better for skillups, then you'd expect more than 5% of the skilups on successes.

      If that does work statistically, then everyone's runs could be added together, reguardless of what they're skilling up on, and at what levels, to give a big enough sampling to clean out any streakyness.

      ... and yes, it's late.. did that make any sense?

      Comment


      • #18
        I agree that 0->15 may not speak to 189->190.

        But I do hope to do enough controlled tests (I may have to do them myself and maybe on a different server than I normally use -- anyone want to let me some bows and some arrow cash?) so that a 0->15 theory can be ruled out or maintained.

        I do not think we can do a standardized experiment on 189->190 withone some SERIOUS sugar daddying from a research grant.

        Step 1: Skill your smith to 188.
        Step 2: Get INT to 255
        Step 3: Go back to Sugar Daddy for another 20K to replenish bank account
        etc...

        It could be done (people could, for example, share the INT suit on a given server), but to establish it on 0->1, 1->2, etc. will take a lot less time and if it does NOT help from 0->1, it's ahrd to imagine that it does help from 188->189. Though it might.
        Andyhre playing Guiscard, 78th-level Ranger, E`ci (Tunare)
        Master Artisan (2100 Club), Wielder of the Fully Functional Artisan's Charm, Proud carrier of the 8th shawl


        with occasion to call upon Gnomedeguerre, 16th-level Wizard, Master Tinker, E`ci (Tunare)


        and in shouting range of Vassl Ofguiscard, 73rd-level Enchanter, GM Jewelcrafter, E`ci (Tunare)

        Comment


        • #19
          I did the tests Cynix is referring to.

          I did two types of tests. The test I thought was more robust didn't seem to catch people's attention, while the simpler test did.

          Test one (simple test)

          I needed to raise one character's JC anyway, so I picked JC. The two characters had equal stats. One made Gold-Amber combines (trivial 102) from 0 – 100, while the other used the closest recipe to trivial from 0 – 100. The closest to trivial character took a shade over 20% fewer combines to get from 0 – 100 than the Gold-Amber character.

          This simple test seemed to convince a lot of people, perhaps because it went right to the bottom line. I think the next test I describe is much better, and easier to repeat without worrying about standardizing characters as much.

          Test two (with lots of data sent in by traders)

          I had people send me skillup reports for their runs in different skills. For each skillup, the recipe was noted and it was recorded if it occurred on a success or a failure. This test was a bit more complex looking, because I used the probabilities from the success rate formula (recently derived at the time) to weight the results. This would get around the argument of "Of course there were more skillups on successes than failures… there were more successes. It worked like this:

          You would start by calculating the expected success rate for the recipe at the skill level of the character making the recipe. If the skillup took place on a successful combine, you would keep that value. If the skillup took place on a failed combine, you would put in the probability minus 1 (this would be a negative number). For example, if you were making a recipe with a 35% chance of success, you would enter 0.35 if the skillup came on a success and –0.65 if the skillup came on a failure.

          You would update the expected success rate for each skill point, and then take a sum of the numbers you entered. If you really got a skillup as frequently on a success as on a failure, then the sum of that column should be ZERO. Consistently the sum was positive though.

          This test really convinced me. Trade after trade, sample after sample, the results were positive. I'll see if I can dig some data out of the vault and post it.

          Again, to do this test you just need to keep track of the recipe being used, and whether each skillup came on a success or a failure. Then make yourself a spreadsheet to calculate the expected success rate and to make the positive and negative entries to be summed up.

          Boleslav Forgehammer
          Paladin of Brell in his 65th Campaign
          E'ci – Sacred Destiny

          Comment

          Working...
          X