Ngreth weekend update? Your Views?

[Ngreth Thergn]

Heh. We already were So sitting with me for dinner was a nice chance to be in less of a spotlight than more! As was mentioned in the thread about if we were even invited... we already have good dev contact

[feressa]

To be honest, it looks like they missed the point. The biggest disaster of this year was not GoD and its poorly-tuned zones, it was the DX9/collision patch that introduced critical faults that are still not completely fixed. It's good that they're trying to rewrite the old engine, which was getting increasingly creaky, it's bad that they aren't prepared to spend the money needed for testers who will wring out the bugs before it goes live. SoE's attitude towards testing has been a constant issue for years, and I can't see any evidence they've really learnt their lesson here. The notion that guilds should receive some sort of in-game benefit from beta-testing new content is absurd, this needs to be done professionally. SoE has long made the claim that lots of staff members play the game regularly, are they unable to put together 54 people from the staff who know how to play and can test new content without leaking it to friends and guildmates?

I notice no comment on the unfinished GoD tradeskill content, are they planning to put in recipes for taelosian wheat for instance?

[Ngreth Thergn]

To be honest, it looks like they missed the point. The biggest disaster of this year was not GoD and its poorly-tuned zones, it was the DX9/collision patch that introduced critical faults that are still not completely fixed. It's good that they're trying to rewrite the old engine, which was getting increasingly creaky, it's bad that they aren't prepared to spend the money needed for testers who will wring out the bugs before it goes live. SoE's attitude towards testing has been a constant issue for years, and I can't see any evidence they've really learnt their lesson here. The notion that guilds should receive some sort of in-game benefit from beta-testing new content is absurd, this needs to be done professionally. SoE has long made the claim that lots of staff members play the game regularly, are they unable to put together 54 people from the staff who know how to play and can test new content without leaking it to friends and guildmates?

flinging money at a problem still will not cure all bugs. Only 400000 people will be able to find some bugs. But they did acknowlege that the DX9 thing was an issue. (they did not acknowlege it as a mistake like GoD though)

EQ staff is only about 60 people (not including guides, gm's and support staff (admin, human services...))... and I highly doubt 54 of them are skilled enough in the game itself to test the uber content. A good chunk of those 60 are artists... and while I am sure some of them are very good at the game, I do not expect all of them to be so. Some fo the Coders of the game spend so much time CODING the game, that they do not have as much epxperience playing the game as many players do. And the developers, who may be the most experienced, are nearly as busy as the coders, giving them the same disadvatage of just time to play the game.
MANY players are going to be more experienced at the game, then the people making EQ. Don't you want the most expierienced people testing?
The other problem... is you are asking people MAKING the content to test the content. The person(s) making the content have a certain blind spot to their own content (true in any industry) that makes testing their own content a problem.

I notice no comment on the unfinished GoD tradeskill content, are they planning to put in recipes for taelosian wheat for instance?

Becasue no-one brought it up. This allows for POSIBLE future content. GoD tradeskills content has 99% of what was intended for it in the game.

There are plans to tweek some of the smithing weapons. Not to use every single posible item dropped in GoD.

[feressa]

I want testing to be done properly and professionally, that's the area in which SoE is failing. People from my guild have joined up in these open betas for a couple of expansions now. Why do they do it? To get information and get a head start when it comes out, why else? Obviously the developers know this. Open beta testers have no other motivation because they aren't given any. The result is that the developers have no real ability to judge the quality of the information they're getting back from the players. The result is that genuine issues get overlooked. We've spent a long time beta-testing GoD in live play, currently we're 'testing' the caves in Txevu, and I can tell you it's getting pretty boring.

If they hire experienced players (there's a prety large pool of high-quality ex-players now :/) they know they're dealing with people who won't be looking to beat the encounter in live play. This is one area in which flinging money at the problem will solve many of their problems.

[Blat Splat]

flinging money at a problem still will not cure all bugs. Only 400000 people will be able to find some bugs. But they did acknowlege that the DX9 thing was an issue. (they did not acknowlege it as a mistake like GoD though)

EQ staff is only about 60 people (not including guides, gm's and support staff (admin, human services...))... and I highly doubt 54 of them are skilled enough in the game itself to test the uber content. A good chunk of those 60 are artists... and while I am sure some of them are very good at the game, I do not expect all of them to be so. Some fo the Coders of the game spend so much time CODING the game, that they do not have as much epxperience playing the game as many players do. And the developers, who may be the most experienced, are nearly as busy as the coders, giving them the same disadvatage of just time to play the game.
MANY players are going to be more experienced at the game, then the people making EQ. Don't you want the most expierienced people testing?
The other problem... is you are asking people MAKING the content to test the content. The person(s) making the content have a certain blind spot to their own content (true in any industry) that makes testing their own content a problem.

Throwing money at a problem will not fix it, but hiring people (or a few more if they already do) specifically for testing would help. It looks to me that they do not have a test plan that they run through before releases. Testers should have a checklist of what has changed they should test each of them before being released to the test server. I also agree that coders are the worst testers. 15 years of development has proven that to me over and over.

Also, it seems that the test server is not utilized as fully as it should be. It seems that it is really just another player server that gets the content first. I'll admit that some bugs are worked out that way, but maybe they should hire some of the people playing on it as testers. Or give the players there incentives to report bugs, like a month rebate. It would be CHEAP testing.

Overall, it sounds to me that if there are only 60 people working there, that they are very understaffed. And that understaffing is showing in the amount bugs/problems showing up in game. I'm sure that level of staffing worked fine for the first maybe 2nd expansions, but the games has grown so much that it becomes impossible to keep up with the bug fixes and undeveloped ideas that have plagued EQ for a while now.

Personally, I think if anything starts driving away people from EQ, it won't be other games. It will be due to the quality of EQ game play going down. People will tire of seeing new releases when the content of the last release or two has not been fully implemented and still has bugs.

It seems to me that there are many options for them to improve quality, such as better utilizing the test server, providing incentives for more people to use the test server, and hiring testers. It also seems unlikely to me that with the current subscriber base that they cannot afford to hire more. I just don't understand why they don't do it.

[Lothay]

Before this turns into a whine-fest, recall that Sony has theri own boards for you to discuss these issues.

Ngreth has been very polite about sharing his experience, and providing information from the event, but he is NOT an employee of SoE. Some people here seem to be pointing fingers at him as if he were. EQ Traders doesn't mind spirited debate, but doesn't support personal attacks or flaming. I'm not saying that's happened yet, but I am injecting a note of caution.

A few points:

At $13/month or less per subscriber, SoE isn't making enough money to have a QA staff in the hundreds. Before someone starts trumping up numbers, make very sure you find out what real hardware and bandwidth costs are. They have a large cost for support, hardware, bandwidth, artist, designers, and software engineers. Then you add in the cost for Quality Assurance, and the $13/month doesn't seem like a large number any longer. People in the QA arena aren't cheap. I know, I used to be field, and hope to be again soon.

Testing content and code is a massive undertaking. A single code change results in huge numbers of possible combinations making it literally impossible to test ever combination by a human being. Automation and massive suites of computers doing things by scripts catch a number of errors, and the huma testers catch more. But always the best place to catch a problem is in the user-base. If there are 40,000 subscribers to EQ, the chances are that the subscribers will discover a problem that would never have been found in QA simply on the percentages. It' i a mathematical near-certainty.

Hiring more people is probably not the answer. Making changes to their beta process will probably help. As will changing the the incentive factor of the test server. But even those won't catch everything.

I'm not arguing that SoE shouldn't do a lot of testing BEFORE something goes out to the players. I'm just arguing that you can't expect every bug, even some that seem pretty obvious, to be found in-house.

Now, in my opinion SoE doesn't have the best track record. The game is often frustrating, often broken, and caters to a style of play I don't find enjoyable. I voted with my feet, and left EQ. (I help here fpr personal reasons. The Denmom and the House Ogre are two of my best friends in the world, and the work I do here doesn't require that I actually play the game any longer.) But that doesn't mean that they don't accomplish some pretty amazing things even with all the handicaps. There are few games of any sort that are as rich in setting and detail as EQ. The sheer volume of EQ's content and the complexity of the various features is amazing. Yes, it's not perfect, but it's still pretty amazing. I'd say that the weekend summit shows that their hearts are in the right place, and they're making the required efforts. Give 'em some time and see what happens.

[Tudani]

If they hire experienced players (there's a prety large pool of high-quality ex-players now :/) they know they're dealing with people who won't be looking to beat the encounter in live play. This is one area in which flinging money at the problem will solve many of their problems.

They basically are going to be "hiring" experienced players to do their beta testing. At least for all of the raiding content, and probably much of the other content as well. However, instead of paying us with cash, they are offering us in game incentives/rewards for testing (such as after testing xxx content each player will get yyy augment reward or something). They are also going to be spreading the encounters out over many guilds, so that a) each encounter should be tested by several groups of players, and b) no one guild has any real advantage over another in progression, as well as giving the guilds time to keep up with progression on their home servers. I hope that they will extend this to testing single group and other content as well. It's very encouraging to me that SOE recognizes that their beta process is flawed, and is working on improving it.

[Chenier]

Becasue no-one brought it up. This allows for POSIBLE future content. GoD tradeskills content has 99% of what was intended for it in the game.

There are plans to tweek some of the smithing weapons. Not to use every single posible item dropped in GoD.

We talked about some tradeskill ideas in my round table group (how could I *not*?!)...

[feressa]

They basically are going to be "hiring" experienced players to do their beta testing. At least for all of the raiding content, and probably much of the other content as well. However, instead of paying us with cash, they are offering us in game incentives/rewards for testing (such as after testing xxx content each player will get yyy augment reward or something).

Yeah, this is what I'm talking about. It won't work. Professional testing requires professionals, people who are paid real money to do a proper job. They've tried exactly this sort of testing for LDoN and GoD. It didn't work, for the reasons I gave before.

Given the real decline in the EQ subscriber base, the management at Sony is going to have to face some hard facts about systemic problems in their company, of which the lack of testing is the most major.

Lothay, the only finger I'm pointing here is at Sony's management, not sure why you're trying to start an argument.

[Tudani]

It didn't work because a) guilds had no incentive to as a whole go test, as well as that many members from said guilds didnt' have beta accounts, b) many people who did end up testing the content were betabuffed people who both didn't really know how to work together, and didn't necessarily know how to play at the level for which the zones were designed. I think the plan from here on out is that guilds as a whole will be copied over and have plans in advance to test certain content, their testing will be rewarded by in game items and such. So say my guild is asked to test an encounter, we'll all log onto our beta chars and spend the evening testing the new content instead of raiding (and who better to test, we're the ones finding many of the bugs in GoD as is...we basically are beta testing GoD just on our server and with no incentive to do so) and then everybody who showed up will get a reward for testing that content, possibly something like a 60 hp aug or something, I really have no clue what they have in mind for rewards. If the guilds are doing it as a whole they'll have plans in advance to test that night, ensuring a good turnout for the testing also nobody will have to leave testing at raid time to go to their real servers. It may not be the best way, but I definately think that it is a much better way to do things than for what was done in GoD and many other expansions.

[Lothay]

Lothay, the only finger I'm pointing here is at Sony's management, not sure why you're trying to start an argument.

I'd like you to consider re-reading my post. My note was one of caution, not one of confrontation. I don't start arguments here. It would interfere with my job. Construing my post as an attempt to begin an argument would be in error.

As a moderator, I clearly label my opinions as mine, and keep them seperate from my moderation duties.

Feel free to point fingers at Sony. And to do so more effectively at Sony's own boards.

[Ngreth Thergn]

Yeah, this is what I'm talking about. It won't work. Professional testing requires professionals, people who are paid real money to do a proper job. They've tried exactly this sort of testing for LDoN and GoD. It didn't work, for the reasons I gave before.

Where do they find 54 of these that...
#1 are experienced in EQ
#2 are experienced in working TOGETHER

I am fairly certain this professional crew you are asking for plain does not exist.

If you are saying permanent staff where does the money from this come from?

No matter what you say EQ does not make so much money that they can add 54 more staff, especially at the payrate such a team above would expect... from the 13$ a month subscribers pay, while still paying the huge bandwidth fee, hardware mainteance, rent on 4 buildings, Customer service, the dev team (more than 54 people there), support staff, utilities for 4 large buildings (tons of electricity, phone, water, trash removal)

If it becomes a problem that they are loosing money running the game, there will no longer be a game.

[feressa]

I am fairly certain this professional crew you are asking for plain does not exist.

If you are saying permanent staff where does the money from this come from?

Well this is the problem, isn't it? SoE doesn't have professional testers.

I don't know the details of Sony accounts, and neither do you or anyone else posting on these boards. But Sony needs to know where to put the money (and sorry, $5mill a month has to go somewhere). They need to put a lot more of it into testing if they want to keep EQ going.

I want to see EQ keep going, that's the only reason I'm pointing this out. I really have no idea why people keep producing apologias when that's the last thing Sony needs right now.

/shrug Yes, this is a message for Sony, and I've certainly sent them feedbacks on it. I merely posted a comment pointing out something that's really obvious to any long-term player, and I'm lost as to why some people seem to want to argue the point. Anyway, I've said my bit and that's an end to it.

[muana]

testing

Another added benefit this might hold is that if a whole guild, such as the one Tudani is in chooses to do the testing it might open more areas up for other guilds on the server to "share" as such content. So say one guild on each server does test server stuff one nite and another guild (about equal in strength) tests it another nite. Since there is always a battle it seems between the top 2-4 guilds on a given server this may alleviate some of that tension (sorta like how time is done).

just my ramblings on of a lil enchanter

[Ngreth Thergn]

Well this is the problem, isn't it? SoE doesn't have professional testers.

I don't know the details of Sony accounts, and neither do you or anyone else posting on these boards. But Sony needs to know where to put the money (and sorry, $5mill a month has to go somewhere). They need to put a lot more of it into testing if they want to keep EQ going.

I want to see EQ keep going, that's the only reason I'm pointing this out. I really have no idea why people keep producing apologias when that's the last thing Sony needs right now.

/shrug Yes, this is a message for Sony, and I've certainly sent them feedbacks on it. I merely posted a comment pointing out something that's really obvious to any long-term player, and I'm lost as to why some people seem to want to argue the point. Anyway, I've said my bit and that's an end to it.

And I guess my last bit is you are asking them to spend themselves out of business.