How to Track Performance Analytics Events

Tracking Performance Analytics Events

Performance analytics help you track how users interact with different question types in a scene. The four types of questions supported are multiple choice, marker questions, scene hunts, and timed hunts.

Our sample scene, "Experience Performance Analytics", which demonstrates how to configure performance analytics in your scene, can be found in our examples scene folder under Packages/com.strivr.strivrsdk/Examples/Scenes/. Sideload an APK of this scene in your headset to interact with it. Or, to experiment with this scene in the editor with a simulated rig, you can add the device simulator rig that’s a sample of XR Interaction Toolkit 2.3.2 and drag the "XR Device Simulator" into this scene. Note: keeping this device simulator in your scene will prevent the VR rig from working in your APK.

Contact us if you want to tag questions and answers with concepts.

The Setup Process

To track these events, you’ll need to follow these steps:

  1. Set up an experience for your scene (see the Creating an experience section) and add a script with calls to start and finish the experience (StrivrAnalytics.StartExperience(StrivrExperienceData data) and StrivrAnalytics.FinishExperience()).

  2. Under the "Experience Questions" list, add to the list which will create a Performance Question ScriptableObject.

  3. Fill out the Performance Question ScriptableObject, and add to the Answers list to create associated Answer ScriptableObjects.

  4. You can then associate Answer ScriptableObjects to specific game objects in your scene, and from a script:

    1. Create a PerformanceQuestionEventData data object.

    2. Track the data object with this call, StrivrAnalytics.TrackPerformanceAnalyticsEvent(PerformanceQuestionEventData data)

For example:

PerformanceQuestionEventData data = new PerformanceQuestionEventData
{
    Id = questionId,
    AnswerId = answerId,
    TimeElapsed = timeElapsed,
    QuestionAttemptCount = attemptCount,
    DidUserInteract = didUserInteract,
};

StrivrAnalytics.TrackPerformanceAnalyticsEvent(data);

Scoring and Data Correctness

A PerformanceQuestionEventData describes question performance at the answer level, so a new TrackPerformanceAnalyticsEvent with the PerformanceQuestionEventData data should be called for every answer.

The correctness and score of a question are determined by three main factors:

  • 'Is Question Scored?' checkbox in the question scriptable object. This must be checked for the data to appear in the Portal.

  • 'Is Correct' checkbox in the answer scriptable object.

  • DidUserInteract property in the PerformanceQuestionEventData object.

By contrast to the question and answer performance data, the overall experience score is determined by StrivrAnalytics.TrackScore(int score, int maxScore). You must call this to see experience score data in Portal. You are responsible for keeping track of the user’s current and maximum possible score.

If a question should not be scored, then you should do a couple things: uncheck the 'Is Question Scored?' box in the question scriptable object and ensure your script doesn’t change the experience’s running score based on the answer interaction.

Also, please note that you must call StrivrAnalytics.TrackScore(int score, int maxScore) before calling StrivrAnalytics.FinishExperience(). In the "Experience Performance Analytics" example scene, all questions are scored, and the experience’s total score is managed by TrackExperienceAnalytics.cs once someone has finished the experience.

The Id and AnswerId fields are mandatory and cannot be empty. We recommend that you provide a value for all fields to ensure data completeness and accuracy, particularly for question performance analysis. If left unassigned, fields will automatically default to their base state (e.g., DidUserInteract will default to false).

Debugging and Project Maintenance

We have a tool to debug your experience’s question and answer hierarchy in case you’re not sure it’s set up correctly. In the toolbar, go to Strivr > Performance Questions > Log Experience Questions JSON for enabled scenes in build and Strivr Debug Log Experience Questions JSON for enabled scenes in build. The former is a more name friendly way to see if you’ve set up the hierarchy of your data correctly, as the naming will be the same or similar to the data you’ve filled out in the scriptable objects. The latter is a way to see a raw version of how Strivr digests the scriptable object information. This can be helpful to send us if you observe something unexpected from your setup. Note that for tagged concept IDs, if you enter an invalid GUID in the scriptable object, then you’ll see that the raw log will omit this invalid data. Also, note that you must have this scene enabled in your build settings in order for the log data to be printed in the console.

Deleting a parent scriptable object can leave orphaned question and answer objects in your project. To help keep your project organized, we provide a tool that scans for and deletes these orphaned assets. You can run it from the toolbar by navigating to Strivr > Clean Performance Questions > Delete orphaned questions and answers.

Test Mode must be disabled in your Project Settings and you must launch the APK from Player in order for this data to appear in Portal under Analytics > By Course and Analytics > By Learner.

Data Deletion

Deleting a question from the Unity project or removing it from your scene after its data is published will permanently delete the data you’ve previously generated. Note that it will be impossible to retrieve that once-generated data again. To put it another way, the data in Portal shows the currently existing scene’s experience and performance analytics data for the most recently published APK.