Head tracking on Google Cardboard

Head tracking with Google Cardboard on the LG G3 running Lollipop has a problem. Sometimes the view orientation suddenly jumps around. It quickly corrects on 2 axes, probably using the accelerometer. It doesn't correct on the remaining axis, probably because the compass is used for detecting the magnet instead of orientation. I've been using the Unity Cardboard plugin, and had the issue when using that.

Workaround

I've tested a workaround, which helps. Basically, detect when it's jumped by mistake, record the jump rotation, and rotate an equal amount in the opposite direction. In more detail:

• The modification is to CardboardHead.cs in version 0.4.10 of the plugin.
• Keep track of the Cardboard library's head tracking orientation.
• Compare with previous frame, noting when that orientation changes by too big an angle.
• In that case, store the rotation to get from the new (incorrect) view back to the old view, as a correction. Combine it with any previous corrections. Only use the rotation around the Y axis, the other axes will be corrected by the Cardboard library and doubling up the corrections would get the wrong answer.
• When putting the Cardboard orientation into a transform in Unity, apply the correction rotation first.

Second version of workaround

The first version still had vertical jumps as these were left up to the Google's Cardboard library to correct. Also, in the process of correcting these, the Y axis seemed to move sometimes. To fix this, I blocked all updates of the orientation from Google's Cardboard library, until the Y axis correction would (approximately) correct the current Cardboard library output to match the previous orientation. To account for actual head movement after the jump, or in case of erroneous orientation from before the jump, the required match to previous orientation gets looser every frame.

This is on GitHub here

Potential improvements

Jump detection

You could judge the angle change based on the time between frames, rather than against a fixed maximum angle change between frames. You could judge the rotation speed against a previous rotation speed - if the orientation was rotating in a particular direction, it may be common for it to rotate in that direction a little faster the next frame - don't count that as a jump. The threshold angle change could be tested at larger and smaller values to see what threshold works best.

Jump correction

While waiting for the post-jump output of the Cardboard library to stabilise, extrapolate orientation from previous rotation instead of freezing it in place

Fix the bug

Either modify or replace the head tracking code that makes the jumps in the first place. This would be the best, as the existing code always has a time when the head tracking isn't running due to needing to cancel a jump, and the detection of jumps will not be perfect. There are already newer versions of the Google Cardboard library than the one I encountered the bug on. They may have improved the issue.

Only apply to phones that need it

Maybe you can detect whether the phone is a G3, and only turn on the jump correction when it is (and for any others if you know they have the problem)

Workaround code change (first working version)

--- CardboardHead.cs.orig	2015-04-13 15:00:40.000000000 +0800
+++ CardboardHead.cs	2015-06-21 13:11:17.000000000 +0800
@@ -1,4 +1,5 @@
-// Copyright 2014 Google Inc. All rights reserved.
+// Copyright 2014 Google Inc. All rights reserved.
+// Copyright 2015 Caspian Maclean.
 //
 // Licensed under the Apache License, Version 2.0 (the "License");
 // you may not use this file except in compliance with the License.
@@ -21,6 +22,9 @@
 
   // If set, the head transform will be relative to it.
   public Transform target;
+  public Quaternion fixQuaternion;
+  public Quaternion lastRot;
+  public bool glitchFixStarted = false;
 
   // Determine whether head updates early or late in frame.
   // Defaults to false in order to reduce latency.
@@ -63,10 +67,21 @@
 
     if (trackRotation) {
       var rot = Cardboard.SDK.HeadRotation;
+      if (!glitchFixStarted) {
+        lastRot=rot;
+        fixQuaternion = Quaternion.Euler(0F,0F,0F);
+        glitchFixStarted = true;
+      }
+      if (Quaternion.Angle(rot,lastRot) > 15) {
+        var newFixQuaternion = fixQuaternion * lastRot * Quaternion.Inverse(rot);
+        //only correct y axis, cardboard will correct the other axes quickly, probably using accelerometer
+        fixQuaternion = Quaternion.Euler(0F,newFixQuaternion.eulerAngles.y, 0F);
+      }
+      lastRot=rot;
       if (target == null) {
-        transform.localRotation = rot;
+        transform.localRotation = fixQuaternion*rot;
       } else {
-        transform.rotation = rot * target.rotation;
+        transform.rotation = fixQuaternion * rot * target.rotation;
       }
     }