Friday, August 6, 2010

L2Playtest

So the hot topic in NERO in Ohio these days is the extensive use of Playtests by WAR. In addition, they brought Nature Magic back from the grave. For those of you interested, you can find all of WAR's policies and playtests here.

While there has been some proper discourse on the subject of these playtests, I must say I am somewhat alarmed at the number of people, on both sides, that do not understand the concept of a playtest. Hell, I don't think that word even means what they think it means.

The most common, and most successful, model for game design is called Iterative Design.

Iterative design consists of a cycle of several steps. The first step is designing the product. Minds come up with what they can from a cloud of ideas to put a basic structure and goals together. The second step is prototyping. This is where you take the cloud of ideas that you prefer, and turn it into an actual product that can be used. This usually thins out vague concepts and forces designers to make hard choices. The third step in the design cycle is playtesting (or beta testing). This is where you give the user the product, see how it actual works compared to theoretical models, and get user feedback. The fourth step... well the fourth step is back to design, silly!

Most (good) designs uses this model. It just so happens that products with limited application (tools, parts, etc) often stall out at a point where the playtesting reveals no new information. However, it turns out that extremely complicated products (software) and games both exhibit an essentially infinite cycle. For software, the high complexity and internal referencing in the programming means that when you fix a bug, it will probably create another one. In some gaming, the application space is so large (whatever a player can think of within the rules) that there will inevitably be a lack of balance or holes that cause problems within the game. The tighter the ruleset (chess, go, checkers), the less playtesting is required. The more open the system (Tabletop and LARP systems), the more playtesting is required.

In summary, a playtest is intended to obtain feedback on a design prototype, and the results of that feedback will be fed into the next level of the design. So the WAR "Playtests" are actually prototypes. The process of testing them is the playtest.

Now you know what a playtest is and why it exists. Now I'll show you an example of doing it wrong.

Playtester - "So your prototype sucks. It sounds dumb. You're dumb."
Designer - "Your mom's dumb."

This is an example of poor feedback. If the playtester does not give reasons for their qualms and rely on subjective reasons ("This doesn't feel right."), they are not doing their job. This offers no real value to the designer.

Sometimes I'll hear this exchange as well. Once again, this is a sign of poor playtesting.

Playtester - "It appears that using X/y/z together creates a rather unbalanced situation. How do you plan to deal with that?"
Designer - "We expect our players to know not to do that."

In this example, the designer forgot the goal of the playtest is to make the game better. Playtesters are expected to try and break the system. Ignoring valuable feedback keeps the playtest from doing it's job. A prototype that ignores feedback is a (poorly) finished design.

So here's an example of a proper exchange regarding a playtest.

Playtester - "I know I haven't had a chance to test it yet, but it certainly seems like base 10 celestial damage is going to be a little bit excessive. I'm somewhat concerned about how powerful it will be when used by NPCs, especially against lower level players. Please test this."
Designer - "We will try and do some realistic testing in those situations to see if it's really as bad as it sounds. I appreciate the feedback."

And another...

Playtester - "It appears that using x/y/z together creates a rather unbalanced situation. How do you plan to deal with that."
Designer - "Well, from our feedback it appears that y is a bit too powerful, and z simply isn't going to make the cut. While the combo of x and nerfed y is still pretty powerful, we feel it is more in line with the other options a player would have."

So, as we can see, it's a playtester's job to find actual problems (or apparent problems) and voice their concerns. It's a designer's job to take actual player feedback and, you know, design around it. Playtesters voicing concerns to others and not to the designer is a bad idea. Designers ignoring honest feedback in regards to your prototypes during the playtest is a bad idea.

So let me ask this.

WAR (or OOC) Players using playtested skills - Have you given any feedback to the WAR Ownership regarding those playtest and how they function, whether written or verbal?
Out-Of-Chapter Players - Have you given any feedback to the WAR Ownership (or someone to take to the ownership) regarding the playtests and how you would expect them to function, whether written or verbal?
WAR Ownership - Have you honestly read, analyzed, and made decisions regarding how the new rules fit into the game, and whether any of them need to be dropped/modified?

If you answered 'No' to the above questions, then you have two options.

1. Either give feedback to the ownership (or someone to take to the ownership) or take a minute to read the honest feedback.
-or-
2. Shut the hell up.

12 comments:

  1. Every single skill my character has is a WAR-only playtest skill. Literally every one (what up, harmonics?).

    I haven't given the owners any feedback about it yet (other than my occasional "THIS IS AWESOME" exclamations mid-game), mainly because I rewrote the character only a month ago and have only been able to really PLAY the playtest for one event so far. There will definitely be feedback in the future though, because otherwise what's the point of playing a playtest?

    Also, I recognize that I've made some sacrifices in order to thoroughly dedicate myself to this playtest--namely, I can't play my character as she is/ought to be at any other chapter. I'm still on the fence as to how I feel about that. I think "resigned" is the best way to describe it.

    Still, I'd rather be forced to play a secondary or totally rewrite my primary for out-of-chapter play than not have a chance to be using my primary to playtest things that I think are incredibly cool.

    I feel it's important to remember, by the way, that the WAR-only playtests cut both ways. It does mean that out of chapter players may have to re-spend build in order to feel like they're getting a fair cop. But it also means that those of us who rely heavily on playtests to build our characters have to do the same if we leave the chapter, or not play our primaries at all.

    I'm willing to do that if it means a chance to help make the game better down the line.

    ReplyDelete
  2. Mechanics wise, it will always be difficult to playtest something as exhaustive as a completely new tree like Harmonics or Nature.

    Now, I will say that Harmonics is an outlier when looking at the playtest skills. It's the only one that has been run on a national level for years, and was not adopted. The version WAR is using is identical to the playtest that national ran. No changes.

    National decided that there wasn't enough content in harmonics to keep it in the game, but they did take the goal of harmonics (improved atmosphere and benefits to artistic casters) and integrated it into the new magic skills and new magic spells volume 2 playtests.

    As to the point about the playtest preventing travel, I see it as lose-lose for the game. This is a national game, and it's not supposed to be us vs. them. Anything that prevents players from traveling is unfortunate. WAR players not traveling OOC and OOC players not traveling to WAR hurts the game twofold.

    If you want a good turnout, you need to run a good game and home-grow your players. If you want a great turnout, you need to get traveling players. You can never be great by alienating outsiders.

    ReplyDelete
  3. I totally agree, and that was sort of my point--the lack of travel-ability with LCO playtests is a real shame. But, to me, having LCO playtests is worth the inconvenience of having to either rewrite or always play a secondary when going OOC.

    (Incidentally, I'm really happy that WAR kept the harmonics playtest and that they're letting me do personal plot with it, because my hope is to use that personal plot to eventually alter the playtest in a way that solves that very problem of limited content.)

    I do wish that chapters made a greater point of encouraging inter-chapter play by a.) always making it easy to re-spend build in those cases and b.) making sure that people KNOW their build can be re-spent.

    To me, the biggest demotivator in terms of inter-chapter play is not playtest differences (honestly, I find those interesting and I'm excited by the idea of trying out other chapters' playtests). I'm much more bothered by extreme differences in plot and NPC styles which mean that all the time my character invests in getting to know things about a given planar or elemental creature might be meaningless OOC because they're going to be played COMPLETELY differently.

    ReplyDelete
  4. While I agree with you about demotivators, it's important to know that we're biased. As WAR players we have to know less to be able to travel. OOC players need to know MORE to come play at WAR. Players who struggle enough with the rules as-is will definitely have a difficult time with the WAR playtest.

    ReplyDelete
  5. Out of curiosity--it's just not something I've ever needed to know--is WAR the only chapter currently running LCO playtests, or just the chapter that has the most? If any, what other chapters run LCO playtests right now?

    ReplyDelete
  6. Currently, I believe it is the only one with LCO playtests. Of course, NERO West is known for having some of their own at one point or another, so I'd have to do some digging to confirm that.

    ReplyDelete
  7. Come to think of it, I believe NCN is in the process of introducing Harmonics. I don't know if they have any other non-national playtests though.

    ReplyDelete
  8. Before I left the game for a few years (and have returned) I traveled out of chapter a lot. I went to my fair share of ARGO, PRO and KZOO games. It didn't bother me to make any changes to my character when I went to these events. And back in those days, we did have a lot of out of chapter people coming to our events.

    It seems like a lot of that has changed. I don't think that travel chapter is happening as much. Much of that is money, and travel time. However, with so many chapters close by, I would expect to see this change at some point.

    LCO effects are what make chapters different. WAR is WAR because of these playtests. War's players embrace the game that they play (even if it is different than others). If I have to go to a OOC chapter event and swing 10's with parries, I'm ok with that too.

    ReplyDelete
  9. LCO effects do not make WAR. Hard working plot and staff members make WAR. It just so happens that they use LCO effects in ways intended to make the game more fun for the players. If OOC players came and didn't want LCO effects, we'd make it fun for them without using those effects. We're cool like that.

    There is actually still a fair amount of travel within many of the chapters here in Ohio. What we don't see often is players coming from multiple states away. Alternatively, there's a fair amount of travel between games outside of Ohio, which creates a very real "Us vs. Them" feel that needs to be dispelled.

    I like swinging foam and throwing spells, and I'll swing foam with anyone, as long as they're not hitting me in the junk.

    ReplyDelete
  10. Yes sir. Sorry sir. I will shut the hell up now.

    ReplyDelete
  11. The jokes on you. You already gave insight on the playtests. You are contractually obligated to keep talking : )

    ReplyDelete
  12. Fair enough. But I generally think I made my points as far as the playtests.

    As to the matter of "Trusting your plot team" that was brought up, my grievances with WAR's administration have been well documented, and summarily dismissed. Trust is earned. But such is life.

    ReplyDelete