Parsing bazaar log?

[Elenara]

Has anyone written or know of, a parser for the bazaar log file? I am curious to see how much money has gone through my character and I really don't fancy sitting down to add up 754 lines worth of numbers! Any other suggestions also gratefully received. :D

[xaanru]

Wouldn't it be easier just to write down your balance at the beginning of a bazaar session, and then at the end, and just calculate the difference?

[Gello]

This doesn't give a grand total but it's sort of neat if you do a lot of selling in bazaar. Originally posted by Kinuvan on the Druids' Grove:

Make this sold.bat file:
@echo off
find /i "purchased" "c:\program files\everquest\bzrlog_%2_%1.txt" | find /v "items listed above" | more
pause

to use:
> sold (trader_name) (server_number)

For my bazaar mule I use:
> sold spello 40

If you know your server's number (it's 40 for Karana), you can replace the %2 with the number. If you only have one seller in bazaar, you can replace %1 with that character's name.

You can also change the "| more" to "> sold.txt" (and remove the pause line), then move the .bat file to desktop. Then you can just run the .bat file and then open the created "sold.txt" for those who don't want to break out a command line.

If all of the above is confusing, just do this:
1. Right-click desktop, pick New, then Text File.
2. A new icon will appear on your desktop for you to name. Name it sold.bat. Say Yes to giving it a different extension.
3. Now right-click the sold.bat icon and choose Edit. In the notepad that opens, copy-paste this one line (and change it for your server and trader name):

find /i "purchased" "c:\program files\everquest\bzrlog_40_spello.txt" | find /v "items listed above" > sold.txt

4. Save and exit the file.

Now when you double-click sold.bat, a sold.txt file will be created on your desktop if one doesn't exist. If one does it will overwrite it. When you double-click sold.txt, it will look something like this:

---------- C:\PROGRAM FILES\EVERQUEST\BZRLOG_40_SPELLO.TXT
[Sat Aug 17 22:48:57 2002] Lyiana purchased 1 Withered Leather Skullcap for ( 75p).
[Sat Aug 17 23:16:13 2002] Laly purchased 1 Prelate's Lantern for ( 4250p).
[Sat Aug 17 23:21:37 2002] Entrepott purchased 1 Iron Oxide for ( 200p).
[Sat Aug 17 23:21:47 2002] Entrepott purchased 1 Crown of the Froglok Kings for ( 500p).
[Sat Aug 17 23:22:16 2002] Entrepott purchased 1 Iron Oxide for ( 200p).
[Sun Aug 25 21:13:19 2002] Mordus purchased 1 Crown of the Froglok Kings for ( 500p).
[Sun Aug 25 21:14:36 2002] Rainne purchased 1 Imbued Granite Spauldors for ( 5000p).
[Mon Aug 26 03:53:48 2002] Kxes purchased 1 Netted Kelp Tunic for ( 999p).
[Mon Aug 26 03:55:06 2002] Kxes purchased 1 Withered Leather Tunic for ( 1500p).
...etc

If the sold.txt file is empty, it means you have a typo somewhere. Manually look for your bzrlog_server_name.txt file and make sure you've got the right path and filename.

[Wyrdlan]

[code:1]
#! /usr/bin/perl

###########################
# Who are you?
# Change the file name to let me know
$file = "bzrlog_number_name.txt"
#
############################

#
open LOG, $file;

while( $line = <LOG> )
{
if(
($item,$ppval, $gpval, $spval, $cpval) =
($line =~
/purchased\s(.*)\sfor
\(
\s*
(\d*)p?\s*
(\d*)g?\s*
(\d*)s?\s*
(\d*)c?\s*
\)
/x
) # end match
) #end if
{
$items{ $item }++;
$ppval && ($pp += $ppval);
$gpval && ($gp += $gpval);
$spval && ($sp += $spval);
$cpval && ($cp += $cpval);
}

}
close LOG;

$sp += $cp % 10;
$cp = int( $cp / 10 );
$gp += $sp % 10;
$sp = int( $sp / 10 );
$pp += $gp % 10;
$gp = int( $gp / 10 );

open PARSE, ">>parsefile";
print $pp . "p " . $gp . "g " . $sp . "s " . $cp . "c\n";
print PARSE $pp . "p " . $gp . "g " . $sp . "s " . $cp . "c\n";

foreach $item (keys sort %items)
{
print PARSE $item . " -- : " . $items{ $item } . "\n";

print $item . " -- : " . $items{ $item } . "\n";
}
close PARSE;
[/code:1]

Get yourself a copy of http://www.activestate.com/perl for windows</a> and there you go.

Of course, it should be noted I did this at a glance at work, and it may or may not function as advertised. Anyone spot any glaring stupidity?


*Added comment per Lothay
*Added item counts
*Should work, I added it on to the end of my check-my-merchant-script here: http://www.eq-viatores.org/cgi-bin/vend.cgi

[Lothay]

Wyrd -

You need to comment the file to edit -
[code:1]$file = &quot;bzrlog_name_servernum.txt&quot; [/code:1] with the right file name, but other than that, looks good to me.

Edit -
Feh - Then when I actually try to use this, it fails on compile, line 5, near Open. I'm too scrambled to figure out what's to fix (dratted phone keeps RINGING).


- Lothay

[Caela]

I tried the sold.bat file, and it works great. I get a nice long list of everything I have sold in the bazaar (except for a few special orders I have to log onto my main and I sell the old fashioned way...)

But now I want to do more! I'm no programmer... And perl is over my head (I can install cgi scripts on my own server, but they had better have really good instructions and not be too complicated - I still have an EQ Guild Database program that I can't get to work... sigh)

But it would be nice to be able to find out how much of each thing - ie, what are my best sellers, and what time of day do I do most of the selling...

If anyone comes up with something to do with the data the sold.txt file spits out, I'm all ears...

[Wyrdlan]

I cleaned up a few other things and put a link to the script I run on my guild webserver that checks my own vender. Gah, I can't believe how much has passed through his hands, since that's only a PART of the log!

[Garulf]

[code:1]###########################
# Who are you?
# Change the file name to let me know
############################

open( LOG, &quot;D:\\EverQuest\\bzrlog_21_Sstumpy.txt&quot;);

$pp = 0;
$gp = 0;
$sp = 0;
$cp = 0;
$total = 0;

while( $line = &lt;LOG> )
{
if ( ($count, $item, $ppval, $gpval, $spval, $cpval) =
($line =~
/
purchased\s(\d+)\s
(.*)\sfor\s\(\s*
(\d+)*p*\s*
(\d+)*g*\s*
(\d+)*s*\s*
(\d+)*c*\s*\)
/x
)
)
{
$total += $count;
$items{ $item } += $count;
$ppval &amp;&amp; ($pp += $ppval);
$gpval &amp;&amp; ($gp += $gpval);
$spval &amp;&amp; ($sp += $spval);
$cpval &amp;&amp; ($cp += $cpval);
}
}
close( LOG);

$sp += $cp % 10;
$cp = int( $cp / 10 );
$gp += $sp % 10;
$sp = int( $sp / 10 );
$pp += $gp % 10;
$gp = int( $gp / 10 );

open (PARSE, &quot;>>C:\\parsefile&quot;);
print $total . &quot; items for &quot; . $pp . &quot;p &quot; . $gp . &quot;g &quot; . $sp . &quot;s &quot; . $cp . &quot;c\n&quot;;
print PARSE $total . &quot; items for &quot; . $pp . &quot;p &quot; . $gp . &quot;g &quot; . $sp . &quot;s &quot; . $cp . &quot;c\n&quot;;

foreach $item (keys %items)
{
printf PARSE &quot;%6d -: %s\n&quot;, $items{ $item }, $item;

printf &quot;%6d -: %s\n&quot;, $items{ $item }, $item;
}
close (PARSE); [/code:1]

This works with ActivePerl (perl for windows), I don't know enough perl to fix the 'foreach $item (keys sort %items)' line so it will actually sort the results:

Type of arg 1 to keys must be hash (not sort) at sold.pl line 51, near "%items)"
Execution of sold.pl aborted due to compilation errors.

so I just took out the sort keyword and it works (with no sorting of the output).

Make sure you change the filename for your bzrlog_ file.

Scariest part of it working is: I had no idea I'd sold 10,454 items for almost 250k pp!

[Garulf]

[code:1]###########################
# Who are you?
# Change the file name to let me know
############################

open( LOG, &quot;D:\\EverQuest\\bzrlog_21_Sstumpy.txt&quot;);

$pp = 0;
$gp = 0;
$sp = 0;
$cp = 0;
$total = 0;

while( $line = &lt;LOG> )
{
if ( ($count, $item, $ppval, $gpval, $spval, $cpval) =
($line =~
/
purchased\s(\d+)\s
(.*)\sfor\s\(\s*
(\d+)*p*\s*
(\d+)*g*\s*
(\d+)*s*\s*
(\d+)*c*\s*\)
/x
)
)
{
$total += $count;
$items{ $item } += $count;
$ppval &amp;&amp; ($pp += $ppval);
$gpval &amp;&amp; ($gp += $gpval);
$spval &amp;&amp; ($sp += $spval);
$cpval &amp;&amp; ($cp += $cpval);
}
}
close( LOG);

$sp += $cp % 10;
$cp = int( $cp / 10 );
$gp += $sp % 10;
$sp = int( $sp / 10 );
$pp += $gp % 10;
$gp = int( $gp / 10 );

open (PARSE, &quot;>>C:\\parsefile&quot;);
print $total . &quot; items for &quot; . $pp . &quot;p &quot; . $gp . &quot;g &quot; . $sp . &quot;s &quot; . $cp . &quot;c\n&quot;;
print PARSE $total . &quot; items for &quot; . $pp . &quot;p &quot; . $gp . &quot;g &quot; . $sp . &quot;s &quot; . $cp . &quot;c\n&quot;;

foreach $item (sort keys %items)
{
printf PARSE &quot;%6d -: %s\n&quot;, $items{ $item }, $item;

printf &quot;%6d -: %s\n&quot;, $items{ $item }, $item;
}
close (PARSE); [/code:1]

This works with ActivePerl (perl for windows).

Make sure you change the filename for your bzrlog_ file.

Scariest part of it working is: I had no idea I'd sold 10,454 items for almost 250k pp!

[Wyrdlan]

Problem is I accidentally put:


[code:1]keys sort[/code:1]

When it should be

[code:1]sort keys[/code:1]


Also:
[code:1]$pp = 0;
$gp = 0;
$sp = 0;
$cp = 0;
$total = 0; [/code:1]

is unnecessary. Perl sees them as undefined until used, and undefined plus 1 is 1.

Your print code, also, doesn't really need to be done that way, but TIMTOWTDOI. Thanks for the clean up on MSDOS file names.


NOTE In windows perl (Activeperl) use TWO \ where you would think you only need one! C:\\GAMES\\EverQuest\\bzrlog_NUM_NAME.txt

[Garulf]

Edited to fix the sorting (thanks Wrydlan), and the inits are needed, if you don't when it gets down to turning copper, silver and gold into plat, if you haven't sold anything for coppers or silver or gold, it complains about trying to do modulus and division on undefined values. If your bazaar log happens to be empty, it'll complain about everything, just suspenders and a belt.

As for the printing, I wanted a nice column format, so I put the number sold first.

[Wyrdlan]

Interesting. Consider fixing the double post, too.


Try this instead:
[code:1]
printf &quot;%s \t-:\t %6d\n&quot;, $item, $items{ $item };
[/code:1]

[Silverstorm Windseeker]

Hiyas all,

I hate it when trader goes LD and he made money but i have no idea what he has sold.

I want to try sold.bat but how can i find out my server number? (Brell Serilis)

Also, assuming i am smart enuff to make the bat file, how/when do i use it?
Just exe the bat file before going into eq, then set up trader mode?

Thanks for any help rendered..

[Teledriel]

On your EQ Dir, you will see the UI .INI , eqlog, .. and such, these also have your server number. I.E. eqlog_teledriel_21.txt (21 being Cazic-Thule) So if you have more than 1 server it may take time if you happen to be able to use the same name on differnet servers. Hope this helps.

[talshadra]

personally, i buy and sell. so this script parses the full eqlog (from the date the char went trader) and bzrlog. then prints out spent/got/profit on each item, and a grand total of cashflow.
tals
http//www.fysh.org/~tamsin/analyse

no apologies for the hackish or hardcoded nature.

[drael]

Okay, non computer person here. Can you walk me thru the basic steps?
I get the Activeperl program.
Copy the script into ?notepad and rename?
Save in eq directory?
Then how to start it running? Use perl -w yourscriptname.pl at the command prompt before I start eq?
Do I need to stop it? Is there a way to automatically have it start up when I reboot?

Thanks - off to read the activeperl faqs more in depth.

[Lothay]

Perl is a "run-time" scripting language.

That means the script is going to run only when you tell it to run.

It won't be runing in the background, parsing your bazaar log. You have to issue the command.

You could just make a shortcut with the command line in it, and set that in your "Startup" group. Then, each time the computer restarts, the script will execute.

~ Lothay

[Fror]

I found this webpage from castersrealm. Please note this is not my page, nor do I know the owner of the page, nor can I vouch for its accuracy.

http://karrde.viluppo.net/bzr_parse/

[Khama]

Can someone please educate me ....

OK I copy pasted the above pearl script into notepad. Called it bazzarparser.pl
Downloaded the active pearl
Got the two logs and my bazzarparer.pl file in the same folder
When I click on the bazaarparser.pl file A window opens, and I can see the parser work.
then BAM window gone .... no report ..... no new file in the folder.
The data is scrolling too fast for it to be usefull at that speed.

What do I need to do to view the data after its parsed?

[Khama]

Nevermind got it to run.

Just isnt what I am looking for.

I kinda need a Parser / Stripper

I need to know
What items sold in the last say 24 hours
What they sold for
What time they sold
and Who they sold to

And perhaps a total for that 24 hour period.

Anyone got anything like that, or links?

[Khama]

http://pub59.ezboard.com/feqcompanionfrm3.showMessage?topicID=104.topic

Show your support, if your interested.

[GwyrTunare]

Now, heres somthing that would be a huge disk hog, but REALLY worthwhile...

how about a way to track OTHER vendors stocks, prices, and sales???

almost like a "bazaar tickertape" ?

any ideas?