Announcement

Collapse
No announcement yet.

Data mining / downloadable db

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Data mining / downloadable db

    I'm looking to do some investigation of tribute values on tradeskill items, with a view to advising people on how to maximise their tradeskill efforts towards producting tribute. To this end I'd be wanting to do some quite extensive querying of the eqtc database. I've read this thread and others that seemed relevant, so I know your general position is against static copies of the db existing. How would you feel about doing something like Lucy does with her spell database - that is, making available clearly timestamped downloadable copies of all or part of the backend db? I realise your db is orders of magnitude larger than a spell list. Really I just want to avoid making n queries for every EQ item there is...

    Or you could just ask me not to, and I won't
    Nichola Smith
    Archon of Erollisi Marr
    Tunare

  • #2
    It would be possible to have code (assuming PHP since the boards are in php as well as the main site), make a pdf version of the database with a timestamp saying when it was "up to date". The problem with this is the amount of information that has to be compiled to do this project. The server would be taxed to make several copies at once (and it will happen). The resources would eat up bandwidth as well as processor cycles. I for one would love to have one clear concise PDF of the database, but it would require a LOT of work and server resources. Since the site uses a lot of both at current time I would highly doubt this project would go anywhere.

    Comment


    • #3
      Nichola,

      The idea of a downloadable version of the site has been suggested a few times, and each time Ngreth (or one of the other mods) shoots it down for two reasons. The download would be out of date literal seconds after it's generated, and it would horribly increase the bandwidth used by the site.

      Have you visited the EQEcon.com site? I subscribed to their service recently, and they seem pretty solid.

      Since they already have all the item info (I'm assuming they're somehow getting the info the same as Lucy and Magelo do), they also have the tribute value of the item stored there. I sent in a suggestion that they allow us to search by items by the ratio of bazaar price to tribute points. This lets you find the best "bargains" on your particular server, using actual prices. That might be something for you to look at.

      Another alternative would be for EQTraders to add the Tribute value to the database. However, this would require extensive work, both in modifying the database and in manually keying in all the values. There's also the possibility that tribute values will vary in the future, making manual entry burdensome at best.

      Just some random thoughts to chew on.
      Sir KyrosKrane Sylvanblade
      Master Artisan (300 + GM Trophy in all) of Luclin (Veeshan)
      Master Fisherman (200) and possibly Drunk (2xx + 20%), not sober enough to tell!
      Lightbringer, Redeemer, and Valiant servant of Erollisi Marr

      Comment


      • #4
        Lucy now also lists Tribute Values on items.
        Angelsyn Whitewings, Cleric of Tunare for 66! Seasons.
        Grandmistress Smith - 300, Grandmistress Tailor - 300, Potter - 300, Jeweler - 300, Brewer - 200, Baker - 200, Fletcher - 200, Fisherwoman - 169
        Keyne Falconer, Paladin of Erollisi Marr for 66 Seasons.
        Grandmistress Baker - 300, Grandmistress Blacksmith - 300, Potter - 200, Brewer - 139, Tailor - 91

        Comment


        • #5
          I already have lots of code for mining Lucy And yes I jumped on EQEcon pretty much as soon as they went live. What eqtc has that no one else does though is the most complete and up to date list of *trivials*, which is what I am really after. I don't even need a PDF wrapper, the raw MySQL data files would suffice.

          But yes, I can see that spurious downloads by the curious and unwitting might well cause an unwanted bandwidth increase. I shall go to... another site that holds (less up to date) tradeskill trivials, and charges $$, so they can afford it .
          Nichola Smith
          Archon of Erollisi Marr
          Tunare

          Comment


          • #6
            Plus.

            We just don't want somoene (y****s... you know who he is) having an easy way to just grab our hand made data EASILLY and walk off and SELL it like he has done before.

            Lucy and magello have the advantage of an automated process. We do not.

            Also, I expect we will eventually do tribute value... BUT yes it is a manual process to enter it, a bit of back end coding to make it work, and more thought on how and when to display it. What we are waiting for is to be sure SOE is done changing the values (I am not convinced yet they are done) So that we don't get the whine "I made this hard to make item, you said it was X tribute and it is Y ( where y < x ) you suck how could you i am never coming back you suck!!!" becasue SOE changed it... not us.
            Ngreth Thergn

            Ngreth nice Ogre. Ngreth not eat you. Well.... Ngreth not eat you if you still wiggle!
            Grandmaster Smith 250
            Master Tailor 200
            Ogres not dumb - we not lose entire city to froggies

            Comment


            • #7
              Originally posted by Ngreth Thergn
              So that we don't get the whine "I made this hard to make item, you said it was X tribute and it is Y ( where y < x ) you suck how could you i am never coming back you suck!!!" becasue SOE changed it... not us.
              I have a theory that the site / message board is better when people like that are weeded out. People who can't understand that the owners of the site are human, and aren't just there to serve THEM.


              But I can agree with what you're saying.

              -Lilosh
              Venerable Noishpa Taltos , Planar Druid, Educated Halfling, and GM Baker.
              President and Founder of the Loudmouthed Sarcastic Halflings Society
              Also, Smalltim

              So take the fact of having a dirty mind as proof that you are world-savvy; it's not a flaw, it's an asset, if nothing else, it's a defense - Sanna

              Comment

              Working...
              X