Initial SteamVR Tracking Experimentation

Initial SteamVR Tracking Experimentation

Motion Capturing a Mug in VR

I wanted my first through-and-through project/experiment with SteamVR to be something practical; I tossed around the idea of motion capturing my keyboard or mouse, some headphones, etc. – the usual things that folks want to be brought in to their virtual environment from the real world.  I settled on a mug because of two reasons: 1.) There’s enough unused exterior surface on the mug that I’d have plenty of room to place sensors and the Watchman board, and 2.) It sounded pretty convenient to grab a drink in VR without removing the HMD.  This timelapse shown above was recorded on the 17th of November, 2016, and covers approximately an 8 1/2 hour nonstop development process.  I can shave that time down to at least 2 hours now that I’ve got this run under my belt and know what quirks can be run into.  Once familiar enough, I could see having something hand-made together in 30 minutes being a practical timeline.

Modelling the .STL

So I started out with modelling a replica of the real-world mug in Blender (free and open-source 3D modelling software).  This was my first full-on experience with Blender, so a good chunk of my time was spent learning the interface and following a basic tutorial.  I probably gave this mug way too much fidelity, but hey, I’m proud of the little guy.  Maybe one of these days I’ll look into texturing my models with something that’s more than just UV grid.  More information regarding the render model can be found in my post from Day 3 of Synapse’s SteamVR Tracking training.

Verification Simulation

The HMD Designer simulation software is valuable tool for quantifying the optimization of custom shapes, as well as providing inspiration and validation for sensor placement.  For reference, the Vive controllers have 24 sensors present on them.  This mug uses 14 (a number chosen through a combination of physical limitations, simplicity’s sake, and quick simulation optimization).

Below is a model of the mug, along with some occlusion models:

The reason for these occlusion models was so that the simulation wouldn’t place sensors on or near the handle, inside the mug, or under the mug, as well as to be mindful of the Watchman board’s presence (occlusion models are a separate setting in HMD Designer, yet render the same as the tracked object).

All simulations were pretty similar, but this one felt most systematic and balanced from the batch that I ran:

Those graphs are pretty good!  Remember – the centre of each graph is the negative Z direction, unwrapping the other sides horizontally, and positive/negative Y spanning the top and bottom stretches of the graph.  Feel free to refer back to my coverage of Day 2 of the training for a more thorough explanation of the graphs.  But anyway, the centre of each graph would be the mug’s handle in this context.  These results show that there is more or less manageable optical tracking from all side views of the mug.  I used this placement as a launching point, primarily taking note of the sort of 5-point ‘X’ present on either side of the mug.  And so here’s a shot of the final sensor positions that I calculated into the JSON:

Note the 5-point ‘X’ being a lot more uniform.  An ‘X’ on either side meant using 10 sensors, but also a blind spot in the front and back of the mug.  To improve tracking beyond these 10 points, I set up the minimum number of sensors needed to catch a pose on the front of the mug – 4; and hence the total sensor count being 14.  Check out this post for an in-depth review into what dictates sensor count and positions.

Now obviously there’s a lot of red/orange/yellow in the graphs.  This is 100% not at all a marketable configuration, but it has enough blue/green zones to catch a pose from a front or either side view, and that’s satisfactory enough to move on and give this little experiment a go!  Plus SteamVR does a pretty fine job at being able to solve poses based on the IMU and just 1 or 2 sensors alone once it’s locked down to a known position.  So looking at the Initial Pose Possible graph, it’s clear that a front view or side view will catch a pose with deep blue confidence.  Then from that point, using the Number of Visible Sensors graph, tracking will be able to “survive” from most views that isn’t from above or below.

Again, I’d like to reiterate that a proper prototype should be held to far higher standards than what I’m going for in this run.  This was more so a test for me to ensure that I’m familiar with the SteamVR prototyping process.  This is 10 sensors less than a Vive controller – between that and the fact this was hand-assembled in one night, I think this reliability is pretty OK.


I am a tinkerer, and hardware is my jam.  This is always going to be the most fun part of the design process for me – hands-on development.  Here’s some shots of the final assembly:

Everything is affixed to the mug with pressure sensitive adhesive tape, and for the short-term that stuff holds pretty well.  Any long-term development and you’re going to want an epoxy or something else that has a lot more stick than just a bit of tape.  For reference and scale, the Watchman board is the small square at the bottom centre of the board suite, best seen in the third picture.  The application board is the larger board with the bright light, which the Watchman is plugged into; it also has the USB port and battery plugged in.  Then the third board is the FPGA / sensor breakout, which has all those white connectors that the ribbon wires are connected to.  That board attaches with the other end of the Watchman.  And that little black rectangle sticking out to the left of the mug is the basic antenna for wireless communication.

Calibration and 

Again bearing in mind that this was hand-measured and hand-assembled, the calibration and stability of tracking is pretty darn good!  But that’s not to say it’s without problems.  This clip shows how much error can come out of a poor calibration that only uses a small sampling range of tracking data:

And so that was a pretty unacceptable offset.  After quite a bit of troubleshooting (I think it was between the calibration and modelling the .STL that ate away most of my time during this project), thinking the error was an issue with defining the centre of the mug in the JSON file, I concluded that the calibration was the culprit.  Just think – if you want to achieve submillimeter accuracy, your sensors are going to need to be placed with submillimeter accuracy!  Not by eye and hand as was done in this project – they should be in line with where the JSON expects them to be with minimal offset if you want a properly positioned result.  So I moved on to recalibrate using a lot more tracking samples to work with, walking around the whole room so the system had plenty of different perspectives when solving for true sensor placement.  Below is the recalibrated result:

This new result certainly isn’t great, but I’d say it’s pretty acceptable considering how hastily this rig was put together.  Now let’s put it to use.

Having a Drink in VR!

As shown in the timelapse featured at the top of this post – it works!  I was able to see the mug and its render model whenever opening the SteamVR dashboard, and could easily access it for a drink whenever I needed.  But boy, was it terrifying taking that first sip – because of the HMD, I had to angle my head back along with the mug to the point where I was seriously concerned of flooding the headset.  Haha, thankfully such an event did not occur.  But I definitely foresee bottles and straws being a far more common way to drink from within VR in the future.

Oh, and it’s worth noting that the mug is wired in these videos because I made the mistake of plugging my SteamVR USB RF receiver directly into my PC (without a hub), and something they warned us about in the training is that proximity to USB 3.0 ports is a known issue that messes with RF communication.  During the week after this prototyping marathon I was able to sustain stable wireless tracking by plugging into the headset’s extra USB connector (which oddly enough is blue, implying it’s USB 3.0…)

Further Experimentation

The neat thing about making custom SteamVR tracked objects is that they communicate to your PC the same way your Vive controllers (or HMDs) do, so as far as SteamVR and its hosted experiences are concerned, my mug was just a third controller.  If I turned off one of the Vive controllers, then I was able to use my mug in place of it!  Only a few games actually bring in the render model itself (most will replace the render model with something more custom/stylized for that experience), but it was pretty sweet punching beats and petting a robo-dog with a mug, haha.

Another thing to take note of is that when the pin is floating / not connected, the analogue trigger believes it is pulled all the way.  You can see during the video I will grab things accidentally and not be able to let go.  Interesting!

I kept this mug around for a little while, using it during long sessions to keep hydrated without breaking immersion; it was pretty sweet that something I made in one got so much use.  But alas, exposed electronics attached to a vessel that’s host to water probably isn’t the smartest pairing…  Bottom line, I managed to water damage one of the application boards given to us in the hardware development kit.


Thankfully it wasn’t the extremely valuable Watchman board, but still – darn.  I was able to harvest the other application board out of the the reference object / mushroom from the training, but needless to say I didn’t use the mug anymore after that experience.

Next time…

I hope my ongoing documentation of this process is useful/interesting to you all!  Next SteamVR post, we’ll look into exploring 3D printed parts and some early ideas on a prototypical product :)

I love hearing from the community, so I encourage you to join in the discussions down in the comments, or feel free to write me an email via!  ‘Til next time.

Leave a Reply

143 Comments on "Initial SteamVR Tracking Experimentation"

newest oldest
Notify of
TonyVT Skarredghost

Wow! Really an amazing job!
And using the mug as a SteamVR controller is really awesome!


Very encouraging ;) ! Any thoughts on whether the hmd designer software will be made available to the public?

Steve Dune

This is really intense and hard work bro. Keep it up. Any suggestions where can I start learning VR development?


Cialis great britain

You said it nicely.!


Viagra 5 mg funziona

Really plenty of useful data.


Cialis vs viagra

Thank you! Lots of postings!


Viagra 20 mg

Good information, Thanks a lot.



Reliable write ups. Kudos!


Cialis 20mg

Terrific facts. Regards.


Viagra or viagra

Appreciate it, A good amount of content!


tadalafil 10 mg

Thanks! Helpful stuff.


cialis generico

Valuable info. Regards!



Many thanks, A lot of content!


buy generic cialis

You actually revealed this adequately.


write an essay for me

Good material. Regards!


viagra cialis levitra

You’ve made your point quite effectively.!


cialis 5 mg

Fantastic material. Thanks a lot!



Very handful of web-sites that happen to be comprehensive beneath, from our point of view are undoubtedly very well really worth checking out.


Interracial Porn

[…]that may be the end of this article. Right here you’ll come across some web pages that we feel you’ll enjoy, just click the hyperlinks over[…]



[…]Here are several of the web sites we recommend for our visitors[…]


dog memento

[…]below you will come across the link to some web pages that we feel you should visit[…]



[…]below you’ll find the link to some web-sites that we assume you must visit[…]


Uit eten

[…]just beneath, are a lot of completely not associated web pages to ours, nonetheless, they’re certainly worth going over[…]



[…]just beneath, are numerous totally not related websites to ours, however, they are surely worth going over[…]


pc games for windows 7

[…]we prefer to honor a lot of other online websites on the internet, even if they aren’t linked to us, by linking to them. Underneath are some webpages really worth checking out[…]


free apk download for pc

[…]we prefer to honor several other net web pages on the internet, even when they aren’t linked to us, by linking to them. Underneath are some webpages really worth checking out[…]


swan wand

[…]The details talked about inside the report are a number of the top out there […]


massage website theme

[…]usually posts some very interesting stuff like this. If you’re new to this site[…]


سکس ایرانی

[…]we prefer to honor several other web web pages around the net, even when they aren’t linked to us, by linking to them. Beneath are some webpages really worth checking out[…]


uv resin sladlp 3d printer

[…]the time to read or check out the content or web sites we have linked to beneath the[…]


cosmetic surgery marketing trends

[…]Here is an excellent Blog You might Obtain Interesting that we Encourage You[…]


kalendar 1986 goda

[…]Every once inside a while we pick blogs that we read. Listed beneath would be the newest web pages that we pick out […]


beauty tips according to zodiac sign

[…]that will be the end of this article. Right here you’ll discover some internet sites that we consider you will enjoy, just click the links over[…]


how to get a research job with no experience

[…]always a huge fan of linking to bloggers that I appreciate but really don’t get a good deal of link enjoy from[…]


pre med test

[…]always a huge fan of linking to bloggers that I really like but really don’t get a great deal of link adore from[…]


Adam and Eve Box

[…]Sites of interest we’ve a link to[…]


Unboxing Adam and Eve

[…]Here is a good Weblog You may Discover Fascinating that we Encourage You[…]


voyante marocaine lyon

[…]please go to the web sites we stick to, which includes this a single, as it represents our picks through the web[…]


bandar bola

[…]Every when inside a though we choose blogs that we read. Listed below are the most current web sites that we pick out […]


Alexander McQueen

[…]here are some hyperlinks to internet sites that we link to due to the fact we assume they’re worth visiting[…]


sw airlines

[…]just beneath, are many entirely not associated websites to ours, having said that, they’re surely really worth going over[…]



[…]Wonderful story, reckoned we could combine a couple of unrelated information, nevertheless definitely really worth taking a search, whoa did one find out about Mid East has got a lot more problerms as well […]


Bottega Veneta

[…]we prefer to honor quite a few other internet websites around the internet, even if they aren’t linked to us, by linking to them. Underneath are some webpages worth checking out[…]



[…]we came across a cool web page that you may well enjoy. Take a look should you want[…]


Salvatore Ferragamo

[…]check below, are some absolutely unrelated internet websites to ours, nonetheless, they’re most trustworthy sources that we use[…]


Tom Ford

[…]Here is a great Blog You might Discover Interesting that we Encourage You[…]



[…]The details mentioned in the report are a few of the very best obtainable […]


lunnyj calendar 2017

[…]Here are a few of the web pages we advocate for our visitors[…]


free download for pc windows

[…]here are some links to web-sites that we link to since we believe they are worth visiting[…]


xmobile pro

[…]please stop by the websites we stick to, like this one particular, because it represents our picks through the web[…]