Project:Data Dictionary/Proposals

Mod Testing

• Form:SystemSpecs
  • Template:SystemSpecs
    • cpuspeed | CPUSpeed [Number]
    • cpucore | CPUCore [Number(list)]
    • gpumodel | GPUModel [String[list)]
    • gpuvram | GPUVram [String(list)]
    • multigpu | MultiGPU [Number(list)]
    • ssd | SSD [Boolean]
    • arch | OSArchitecture [Number(list)]
    • hasdlc | HasDLC [String (list-multi select)]

• Form:ModTest
  • Template:ModTest
    • newGame | NewGame [Boolean]
    • loadSave | CanLoadSave [Boolean]
    • loadSaveName | LoadSaveName [String(list)]
    • uninstall | CleanUninstall [Boolean]
    • errors | HasErrors [Boolean]
    • ctd | HasCTD [Boolean]
    • bloat | SaveGameBloat [Boolean]

• Form:Performance
  • Template:Performance
    • fps | FPSImpact [Number]
    • vram | VRAMImpact [Number]

ModAttributes: Some of the attributes for this template should be associated with the Mod page itself.

---

Propose that we add to Template:ModTest:

• runningdlc | RunningDLC [String (list-multi select)]

RE Template:ModAttributes:

• @s4n, should the internal objects of the Documentation property be converted to property "string" values of that property? Could you re-create that one as you see fit?
• Should we add the QualityImpact property to the end of this?

We should rename Template:PerformanceImpact to Template:PerformanceData and use the previous name to hold the performance result (i.e., "impact")

RE Template:SystemSpecs & Template PerformanceData:

• How can we set these to evaluate to the SystemType and PerformanceImpact Properties, respectively? These can either evaluate directly in a 1:1 or in aggregate as a query result of the "average" of these respective templates ...

~z929669Talk 21:00, November 27, 2012 (UTC)

* Yes to the Documentation properties. There should only be three (the ones currently designated as sub properties) all of type boolean.

* Per the talk last night, SystemSpecs will be queried for the user entering PerformanceData and subsequently stored on the report page for that PerformanceData. Beyond this, we'll need a query that can pull results from the PerformanceData entered and process accordingly. Still not sure on how we're going to do that.

Stoppingby4now (talk) 23:42, November 27, 2012 (UTC)

*Agree with adding the Running DLC properties, also, I didn't add QualityImpact to either list as I wasn't sure if we were going to move it at the time. As I said on the forums, where the QualityImpact goes depends on whether we want to empirically collect that info per-tester or if we just want to leave that as one of the attribute items to be fleshed out in discussion.

*What's the difference between PerformanceImpact and PerformanceData, or am I not understanding what you said? Doesn't all that info belong together?

*It seems Semantic Result Forms provides quite a few ways of presenting data usefully, especially the charts and data visualizations sections.

*In response to Stopping's forum post: don't we want PerformanceData and ModTest on the same report, thus the same form?

~FarloTalk 06:08, November 28, 2012 (UTC)

* Re: Performance data. No, that should be a separate form. The main ModTest form will be tied to a single page(ticket) that represents the mod being tested. There should then be a link to the form for adding performance data. If you included it in the same form, management could become far too much to manage effectively depending on how often performance data is submitted.

* Added list at the top as a starting point. Collapsed some things, expanded others. Discuss.

* In the list above, collapsed the DLC's to a single property. No need to keep adding these when we can make it multi-select and not have to worry about updating the form/template every time a new DLC comes out.

Stoppingby4now (talk) 06:17, November 28, 2012 (UTC)

* PerformanceImpact = Low, Med, High (or just low/high) and PerformanceData holds specific FPS values. This is just an idea to aggregate results based on pre-defined commonalities or thresholds. Think of it like a frequency histogram (you have to create buckets for small-mid overall n sizes, else you just get a bunch of 1s for each FPS/VRAM combo). Otherwise, this may be accomplished by the query though, so I defer to Stopping on that (thinking about it further, this seems more useful, but I am not sure of the trade offs).

* Agree with the idea of keeping Performance results separate from test results ... same goes for anything we are interested in collecting in aggregate.

* Also agree with the new org above. I suggest that we use consistent field/property names though (unless we use some consistent differentiator (e.g., lower vs upper case).

* I also like the idea of adding some if not all of the mod attributes to the mod pages (in the info box?)

~z929669Talk 07:17, November 28, 2012 (UTC)

* Alright, I'm a little confused. I thought each user was submitting their test results as far as whether the mod gives them CTDs, has errors, etc. individually but your explanation seems to imply that each mod/version only gets one ticket for this information. If that's the case, how are we going to determine the status of those properties between different machines with different results? If one tester does a test, doesn't get CTDs, and submits the report reflecting that, but then another tester does encounter CTDs, how are we going to handle that? I thought we were collecting this information per user/test and then compiling the results to see if we can find a correlation between certain system specs and certain errors. If we're leaving this kind of analysis/discussion/troubleshooting to a talk page/forum thread then that defeats a lot of the purpose in collecting the information semantically and it's not much better than bickering in the forums until we get enough yays/nays either way.

* I like the changes to SystemSpecs, that should give us a lot more detailed info. I'm just curious, but how would we query based on DLC with the consolidated property; is it as easy as "if this property contains the sub-string "Dawnguard" somewhere in it"?

* At least for the basic/attribute stuff, that'll definitely show up on the mod page, the question is whether we want to display some kind of generalization of the test/performance results as well. If we do, then that involves creating some rough guidelines so those aren't declared entirely subjectively.

~FarloTalk 07:38, November 28, 2012 (UTC)

* Re: template vs form names. I'm coming at this from the perspective of working with all of this in source, because the Semantic tools (creating forms, templates, etc.) are very limited in what they can do, and ultimately you are going to have to get your hands dirty (I create everything from scratch in source). When names are exactly the same, it's far more annoying to work with especially when you are editing after the fact. Refer to Template:Mod. I had changed some of them for this very reason because it was just too annoying having them exactly the same (ex. fps -> AffectsFPS, forumtid -> ForumTID, core -> IsCore, etc.). They still represent IMO, and in some cases are far easier to type (template fields also have the potential to be used many times, where-as properties are mainly used only to assign a value). But I'm also a heavy scripter (think programming where you pass in a value to a function/method, you don't want the parameter to be the same name as an existing internal variable). The absolute minimum I would be willing to accept is beginning lower-case for template fields, and upper-case for properties, but I will ask you to at least consider the above.

* Farlo makes a good point. We still want a single "ticket" per mod as a collection point, but perhaps that should just be a very simple form purely for creating the page, and maybe providing more basic information. ModTest could then be another entry point for users to submit information, and results can be automatically updated on the mod ticket page based on that information. For example, if just one user checks that they experienced a CTD, the mod would then be marked as having had a CTD so it can be examined further. We would also need a comment field for users to enter more detailed information. But we would still need a mechanism for overriding such input for cases such as false positives.

* Re: DLC. Yes, you can query on a property to see if it contains a specific value when it contains a list of items.

Example: {{#ask: [[DLC::Dawnguard]] ... }}. You can think of it as an array, but you don't have to specify an index. Just search for the data you are looking for, and if it's there the condition is true.

Stoppingby4now (talk) 09:56, November 28, 2012 (UTC)

* RE upcase vs lowcase fields/properties. Go ahead and make the call, but I think that all fields should be lower case and all properties should be proper or upper case for visual purposes. However, it is often difficult anbd nonsensical to think up different NAMES for essentially the same thing (which is more what I think we are dealing with in the SMW "scripting" case). I hate trying to come up with a unique identifier for the same concept for no more reason than to differentiate between two different types of metadata.

* RE the Performance Data and Mod-Test Data: Any Property that is variable could be treated like the Performance Data (i.e., meaningfully aggregated). HasErrors is definitely not variable and several other Properties might not be either (e.g., NewGame, CleanUninstal, LoadNewGame, SaveGameBloat); however, these latter could vary depending on the DLC background or even the Windows environment. Therefore, it does make sense to geoup these with Performance Data ... BUT, I predict that the variation will be low or nil for most. Nevertheless, it might be wise to group in Performance Data ... need to see some real examples of possible reports to be sure (Semantic Result Formats does sound intriguing here).

* Updated the breakout with respect to DLC Properties (changed name in SystemSpecs and added similar to ModTest)

~z929669Talk 17:01, November 28, 2012 (UTC)