How To Give Tester Feedback

[dew]

There was a thread that started scratching the surface of the proper playtesting ethics, but it got locked due to poor morale and lack of focus. So let me start a new one that's solely focused on the titular focus. Don't expect an entire book that covers all the themes, for now it's just the wall of text I didn't get to send before the previous topic got locked. Feel free to bring up your particular advice though, I may expand on other focuses as well.

 

----

 

53 minutes ago, Razgriz said:

For maps, if you have to make saves in the middle of the map while trying to full complete it, you aren't playing the map properly for me to understand how difficult it really is. Since probably the dawn of SP mapping, maps were usually no doubt designed to be done without saves or checkpoints, you deciding to play it with saves and critique it outside of it's intended design does render your feedback mostly null.

Okay, while there's some truth in this, it's actually a double-edged sword. I'm gonna go wise man of the mountain here, so bear with me. Savescumming leads to trapping oneself in bad situations and missing the point of difficult map peaks, however constant restarting upon every death is a horrible, horrible way to playtest maps. Occasional checkpointing is pretty important, otherwise:

1. One might get bored by an endless boring loop of restarts and decide to not give any feedback at all;
2. Other might start crushing the map too much and arriving to the uncharted territory with a fat stack of health and saved up ammo.

 

The former is a matter of time and joy management, the latter is what ruined casual saveless gaming for me. It's just not fun when I get ambushed later on by something new, but I breeze through because I redid the first half without a scratch and am coasting with 200/200 + 600 cells. It also ruins my ability to assess the latter scenarios to a degree, which is why I consider proper "blindness" important for the first attempt.

 

What I do when I test is that I record my attempt into a demo - and when I die, I stitch a new demo from before I died and continue, but then at the end I send all of the partials to the mapper, but for the sake of genuine struggle in the rest of the map I "reloaded" with my current stack. Tryharding pistol starts is for the follow-up attempts when you're done with reconnaisance and it's you vs the map, not you vs the unknown. Telling the mapper that this and this trap killed me, an experienced player (pardon the flex), is a very important information that may in some cases need adjusting - or it may just stroke the mapper's ego. Reflecting upon the map after you've completely mastered it and beaten it into submission is less informative.

 

And before someone starts THAT argument: yes, it's different for slaughter or challenge maps where you "restart" between every setpiece or arena and get back your full stack. However then it's also worth it to assess every setpiece on its own merit and go through all of them once you've experienced them all. If the 9th arena is unbalanced, but I gotta grind through the previous 8 challenges to get to it, I may never really understand what's going on.

 

Finally, a major hint about savescumming: People are doing it wrong. Vast majority of people I've seen on streams use just one save, either a quicksave or a single slot to go through the entire wad. They think this mitigates the "cheating" aspect of saves, but they're actually trapping themselves, because you ruin your save with one unfortunate action. If you're using saves and want to avoid the pitfalls of the scummery for the sake of reasonable feedback, you actually need to go full hog and cycle through several saves. It lets you actually recover sensibly and you won't be seen as an idiot for whining about a chaingunner trap that you entered 47 times with 1% health on your stream.

[Phobus]

Saving well is indeed a skill.

 

 Another point worth raising is that, if you’re playing a map with a gameplay mod it wasn’t designed for, or even in a port with features it wasn’t designed for, you’re highly likely to experience something outside of the norm and your feedback is therefore equally likely to be of less value. This goes double for if your mod of choice significantly alters balance. This isn’t the mod makers fault, either - they can only test with so many maps before releasing. These mods are usually designed to make the base game more interesting on the 90th play through, and won’t necessarily work in a map finally tuned around specific weapons, pickups and enemies.

Edited March 12, 2021 by Phobus

[dew]

Here's another point why saveloading or demo stitching shouldn't be shunned in initial testing:

 

Sequence breaks. I often see something that looks like it could be a map breaker. A total one, or a shortuct, or maybe nothing at all. Without saves, you're left to write it down on a post-it note and move on, but I crave testing out that fancy rocket jump I just triangulated immediately - it is mine to claim! Then when I accidentally exit the map in 5 minutes and 13 seconds instead of the mapper-planned 40-minutes, now what?

 

14 minutes ago, Phobus said:

Saving well is indeed a skill.

This is such a great way to put it. I'm absolutely not advocating for savescumming the way it's being used on plenty of streams. On the other hand, a lot of streamers don't save at all and then suffer when their last save is... like, two maps ago. It is a potent tool for giving feedback, but not if used to bruteforce through scenarios outside of one's skill level, and you must always keep in mind how you altered the future events by coming better prepared for them.

[Nine Inch Heels]

Well now, this is a thread I can get behind...

 

A few things I have observed and/or learned while play-testing, even though I don't necessarily adhere to all of them myself at all times:

 

• With or without a demo - telling the mapper where you died - and most importantly in what way - is a key element to smoothen out fights that could just be too rough around the edges
   
• If you aim to provide a spliced demo that is more pleasurable to watch, you absolutely have to take notes of how many repeats a certain section took for you to figure it out reliably, because that's the only way for the mapper to address conveyance issues, if they so desire
   
• If you happen to be fortunate enough to work with a mapper who provides something akin to a changelog, focus on what has been changed primarily so you don't burn yourself out on seemingly "mundane" parts of the map that you already know are fine as is
   
• Don't hesitate to point out if something feels like a trivial time-sink, or if there's anything that can be cheesed somehow. You may not always spot all the possible cheese there is in a map, but whatever you find needs to be put on the table along with how you found the cheddar
   
• Sometimes, but not all the time, it might be a good idea to ask what manner of feedback the mapper is actually looking for. This might seem like it's going to narrow down how useful you can make yourself as a tester, but some mappers are very particular about certain ideas in their maps, and they might prefer for you to look for potential flaws elsewhere. It also helps to clarify what the mapper's design goals are, so that you are able to get into the map with the right mindset, when you're in the mood to play the thing you are supposed to be looking at
   
• In case you happen to get stuck somewhere in the map, ask the mapper what the solution to your problem might be, and see if you are able to pull that off, if that doesn't help you get past a section reliably, that's important for the mapper to know
   
• If you do have some gameplay preferences or biases that might be relevant for the mapper to know, so that they are able to put your feedback into perspective, it might be worth telling the mapper about that when you address parts of a map that go against your grain

[SiMpLeToNiUm]

Not sure if what I'm about to say is still on-topic, and feel free to correct me if it's not:

I think unfiltered, raw feedback is the most important as it gives the mapper the most realistic, "in the field" view for how someone can experience their map without any ground-rules and allows the player to play how they want to play. There's nothing wrong and everything to gain from seeing your map played from as many angles as possible (within the purview of the map's scope and as-is presentation; looking at you, gameplay mods) since the goal isn't necessarily perfection of a single map, but the gaining of broad knowledge, whether that be from one person using savescum tactics or another person doing an FDA; one person playing ITYTD or another on UV. If there are special parameters for how to play and enjoy the map, or things the mappers thinks players ought to keep in mind, that's fine but I think they ought to be outlined and communicated in the OP of the map's thread. 

I think it then becomes the further work of the mapper to either reject or accept the feedback posed, assess the value of it given their own personal criteria, and then take that information onboard for the next project or to improve the original if they so wish. This idea about (and to paraphrase as I understood it, and perhaps I misunderstood it so feel free to point out if I overstepped), "the mapper's idea of how the map ought to work" is not always the best metric for serving the best end-product. Sometimes you need to compromise slightly on your creative vision or your desire to have the map run a certain way because sometimes it just doesn't work out effectively to engage the player. At the end of the day, yeah, it's the mapper's baby, but if you decide to release to the public with the desire for feedback, be willing to accept said feedback and make changes. Player experience is still very important. The relationship between player/mapper is symbiotic and the feedback shared among them makes them both better at what they do.

Edited March 13, 2021 by SiMpLeToNiUm

[Decay]

3 minutes ago, SiMpLeToNiUm said:

I think unfiltered, raw feedback is the most important as it gives the mapper the most realistic, "in the field" view for how someone can experience their map without any ground-rules and allows the player to play how they want to play.

I'm still trying to formulate a good response to this topic but I very much like this approach. However it is worth warning that 1: it can be very contextual/dependent on who you are talking to in terms of familiarity or newness to content creation and 2: you may get people asking who shit in your breakfast if you post too bluntly too often

[Endless]

2 hours ago, Nine Inch Heels said:

 

• If you do have some gameplay preferences or biases that might be relevant for the mapper to know, so that they are able to put your feedback into perspective, it might be worth telling the mapper about that when you address parts of a map that go against your grain

I think this is especially important and something I can elaborate on a bit more: I haven't had much experience as a playtester, but as a reviewer of WADs I'm not too far from such a field (sometimes a review of projects that are in Beta or still WIP is valuable feedback for the author) and I've always been very careful about my biases or preferences. For example, I'm not a fan and I tend to have 0 patience for WADs that are ''slaughter'', however, I do my best to avoid that such a preference ends up hurting the feedback I can provide. We can notice this quite a bit in reviews from people who fail to diverge their tastes from the review they do, making it subjectively biased. People who hate Slaughterwads can often give, for example, 0 out of 10 to a WAD just for the simple fact that such WAD does not suit their tastes, completely ignoring any other kind of quality and blindly criticizing the author's work. This can damage the creator's perspective, especially if he or she is a newcomer.

 

I can quote @SiMpLeToNiUm on this. I did a little playtesting for his Modest Mapping Challenge map. A totally slaughter map that kicked my ass multiple times, however, despite not exactly being my cup of tea, I managed to see through my biases and preferences and enjoy the essence of your creation, so I managed to provide feedback that was valuable/encouraging.

 

TL;DR: Be transparent.

[GarrettChan]

6 hours ago, Nine Inch Heels said:

• Don't hesitate to point out if something feels like a trivial time-sink, or if there's anything that can be cheesed somehow. You may not always spot all the possible cheese there is in a map, but whatever you find needs to be put on the table along with how you found the cheddar

Yeah, I 100% support this idea. Like SG a Baron with lots of room or something. Also, agree with the cheese part. I always report cheeses, especially those easy to spot ones.

 

---------------------------------------------------------------------------------------------------------------------------------------------------------

 

This sounds rude, but it's exactly what's in my mind. Whether using save is totally not the point of playtesting. Using this as a standard is just a very dumb type of reasoning, which clearly is the evidence that person dew quoted has zero proper playtesting evidence. I guess that previous thread is sort of rooted from that "save or not save" type of thread.

 

I'm obvious not a very good player, and I'm not saying I'm the best tester in the world, but according to what I've tested, the most important thing to me is the communication between the tester and the mapper. Yeah, it sounds like nothing, but there's a lot of here. (I mention Flotsam because it's towards the extreme side, so it can make my point easier. I didn't get any sponsor from them :P)

 

• You have a general sense of difficulty. For example, when the mapper wants something like Plutonia type of difficulty, you can understand what's Plutonia and how hard it is. Of course, it has some wiggle room inside, but if someone wants a Plutonia type of map, and you use Flotsam's standard, then the feedback is definitely useless;

 

• You will try your best to break/cheese any encounter as much as possible. That's basically the way to make sure anything can go as intended as the mapper wants. Sometimes you can find difficulty cheese or interesting approach to something. This is a good thing to discuss with the mapper whether he/she likes to keep it or now. In this case, saving is very important because you can do a fight like a dozen times to see how it goes, and whether the intended strategy will work

 

• Please help the mapper to balance resources. Maps are not fun when there's too little ammo, but they are not fun when there's too much ammo either. In this part, I personally would say you can make the map a bit lenient to fit more players (yes, this point is somewhat contradict to my next point), but swimming in the sea of resource makes a map very boring;

 

• No need to consider all player base that the mappers is not going to face because this just hinders the design. A hard WAD, is supposed to be hard. It won't be something for a player who can't even beat Doom 2 on UV. Of course, I'm not saying difficulty implementation is not important. However, something like Flotsam, even HNTR won't be something for new players;

 

• Beating the map casually with save and doing the map legit saveless UV Max (or just beat it on UV saveless) style are equally important. They are for different type of experience. Doing it with save tend to gain more information about the blind play experience, while doing it saveless tend to gain more information about the strategy consistency and resource management.

[Bridgeburner56]

A conglomeration of thoughts in no particular order:

- Explain why you feel the way you do. Be it positive or negative, feelings are not helpful for the mapper if you don't say what has made you feel that way. Zero context means that the mapper won't be able to fix, or recreate what made a fight boring/fun.

 

- Live playtests are the best feedback, followed by recorded playthroughs. I'd say spliced demos are less good as they are non representative of the genuine experience. Much better to record the entire experience with deaths included. Showing the mapper what happened in subsequent attempts at tough encounters is invaluable (encounter learning curve is a thing that mappers pay attention to).

- Try to couch your feedback in level design theory if possible. This is not necessarily for everyone but if you can be clear in technical terms it will be easier for the mapper to adjust things and understand the feedback (people seem to appreciate this in my experience).

- Don't confuse honest feedback with being a cunt. You can be clear and direct without insulting the person who made the map.

 

- No one is beholden to anyone and everything you say is just your opinion. The mapper doesn't have to listen to the feedback if they disagree with it, and the player doesn't have to play the map if they don't like it. This is nobody's paid job and no one is under contract to do anything. Park your entitlement at the door.

 

- If you are finding things too hard and there are difficulty settings then for the love of god fucking use them.

 

- There is a difference between being an (early) playtester and playing a finished wad. If you've agreed to playtest something then be thorough. Being a play tester also  implies a more collaborative relationship. You are working with the mapper to try and make a better product.

 

- Pay attention to your "market", not everyone likes everything. People who play maps that are outside of their wheel house are more likely to give negative feedback that is unlikely to be useful (eg "OMG this is bullshit hard how can anyone enjoy this" when playing Sunder/Abandon/Dimensions etc).

 

- Management of expectations is important. Read the Doomworld thread/WAD text file/Moddb page in order to understand what you are in for. If you go into something like Stardate 20x7 expecting a gentle romp then you're gonna have a bad time (and this ties back to using difficulty settings). Just because you heard something was good does not mean that you will enjoy it.

 

- While saving to the point of it being a TAS is not representative of a true playthrough, don't feel obliged to do a saveless run of a map. 95% of players will use saves so using saves will be a more accurate representation of the average player's experience. If you (mappers) are expecting testers to do saveless runs then state it explicitly.

 

I have eaten all my pizza

Edited March 13, 2021 by Bridgeburner56

[Xaser]

The best feedback is the well-fleshed-out kind (e.g. "hey I think [x] because [blah]", rather than just "hey I think [x]") -- but, even the most basic, bullet-pointy feedback ("hey I think [x]") serves a very important role: it's data.

 

If a single person says "I don't like [x]", it may just be a difference of opinion. If over half the players say "I don't like [x]" then maybe [x] is something worth fixing. :P

Edited March 13, 2021 by Xaser

[Marisa the Magician]

I feel the need to contribute to this whole thing. I'll preface this with me stating that I have come to learn from past mistakes, and even if it's a painful learning process, I am still learning. I want to provide some pointers here, though:

• Regarding the use of gameplay mods: There are some things to take into account here, different approaches, basically.
  • You want to provide feedback for the map authors: Understand that mappers are not responsible for dealing with whatever changes a gameplay mod will bring about (excluding cases where the mod's usage is endorsed, though this is rare). The form of play that the authors intended takes priority.
  • You are explicitly testing a map WITH the mod: If there's some trouble, take this feedback to those responsible for the mod. In most cases we can't ensure that what we make meshes well with everything out there (it would be a most mentally straining ordeal to do so, trust me), but it's important to hear about player experiences to see if we can work something out.
  • (Special case) You are a mod maker testing your own mod: Explain this to the mappers beforehand, understand that they are not entitled to change things FOR you. If something's not working well, you can try to adjust for that, but the easiest choice is simply to let it be, you don't have to cover every ground and account for everything (again, as mentioned in the previous point).
• Getting emotional: This here is a big problem, and one myself and some other prominent figures are guilty of. Raging / getting salty will only hurt your ability to come up with reasonable feedback. This is VERY bad and nothing good will come out of it.
  • Everyone has different skillsets: If something's simply too hard for you to approach, acknowledge that. Play at a lower difficulty if possible, or simply... don't play it if it's not for you. There's nothing wrong with getting upset, but don't take it out on the mappers like it's their fault, they have their intended audience, and not everything is for everyone.
  • Calm down: THIS is something that needs to get through even the most stubborn among us (damn it, why did I have to write those two words together). Becoming increasingly frustrated by some hard encounter where you're dying over and over again, reloading the save, failing once more... don't "rinse and repeat" there, take a breather instead. You're only going to torture yourself and perform worse otherwise.

That's it from me, I guess. I don't expect everyone to agree with what I say, but I felt I needed to say it.