ROSS RANTS: VIRTUAL REALITY LAUNCH

This is a blog post. To read the original post, please click here »

 

 

 

New video, finally! I've been wanting to make this episode for a month or two, although I was holding off until new videocards were announced last month as it's relevant to this topic (it's discussed in the video). If things had gone a little differently, I wouldn't have made this, because I wouldn't have thought it necessary. Unfortunately, things drifted into rant territory instead, so here it is!

 

This is the longest Ross Rants I've made so far and will likely be the longest for some time to come. I wasn't trying to make it long, I just had a lot to say, some of which you'll be hard-pressed to find elsewhere on the internet. I pretty much unloaded almost all my thoughts on VR that have been building up for a long time now. As you'll see in the video, I'm partially obsessed with the topic and have a love / hate attitude towards the medium that I'm trying to reconcile. Anyway, I hope my drive towards it ignites in other people, particularly 3D programmers.

 

The next video will be a Game Dungeon. I hope to have it out in under a week, but I'll just have to see how it goes.

 

COMMENTS

Regarding the "3D Theater" thing near the end in the "solutions" part, I'm happy to report that, on my Oculus Rift DK2, they have something like that.

 

I had to hack together a solution for the DK2 to work on SteamVR and it's buggy as all hell, but I was able to get it so that, in SteamVR, I could play Half-Life 2 in a virtual theater. In fact, that's the only way I could do it on my system. I could not get it to play Half-Life 2 entirely immersive. But that's probably due to it being a DK2 and not the CV1 or Vive. I have no experience with either of those.

 

However, the screen was "2D" in a 3D world. Still the same as viewing a 2D image on a monitor in our real 3D world, so it's not the solution you're looking for.

 

With this Ross Rants, I'm going to investigate this more.

awesome video Ross....its making me think about the rift vive and the PS4 Morpheus project

VorpX does offer the 3D-projection-in-cinema mode, for the games it can deal with.

 

Unfortunately, at some version it stopped supporting older revisions of the Oculus runtime, and it comes with a fantastically clever auto-update-online-at-startup system, which rules out rolling back to an older version that can be used with a DK1.

It's like these companies are deliberately doing a half-assed effort for some reason.

Ross, have you heard of OSVR? (http://www.osvr.org/)

 

Has anyone else actually tried it or dealt with it in any appreciable way?

Ross, I had jobs using 3D vision (on windows 7), I used 3D glasses, I found that Real 3D is possible with NVIDIA drivers and screens. The issue was not the driver but the product, with very certain result and method you could see really amazing things, depth perception so good you feel like the tip of objects follow you when you move around your head.

The differences I saw between the products that worked and those who didn't (I'm unsure if some of them really even matter) were 4 main ones:

 

1. detailed pictured, that weren't using flat textures, as many amount of details and shadows and bumps as possible in the picture. The more happens the better things are.

Flat "3d" textures that only look 3D with a 2D screen just hurts the illusion (if that makes sense)

 

2. High resolution, as high as it can be. This one is obvious, most games do this alright so I have no complaints there. I'll just add that I think that nothing should be a pure color, everything needs a little depth with texture.

 

3.There is a certain way 3D pictures on the screen are placed on each other, they need to have a common area at the right depth. Something that your eye gets its reference point from. Everything that diverts from that area in a certain direction, will seem either deeper or higher accordingly. The best result will happen when that point is at the center of your field of vision, it needs to make your character and the immediate area look close, while other things look far. putting that point too far will cause your head to hurt when looking at things that pop out too much, Putting the point too close, will make everything seem far, barely have any effect. I think the latter is the issue with most 3D games. The way developers implement that point of reference to prevent people from crossing their eyes too hard.

The best way to solve it (at least in my opinion), is to give the user a certain degree of control over it, I don't know how it can be done effectively, but there are many possibilities on how developers can implement this. It can't be too hard.

 

I hope this gives some perspective , NVIDIA have terrible idea on what 3d games should look, but they provide the hardware that can work with the right product.

Hey, quick and honest criticism - you tend to talk slow, and stretch out your words. I think you've done that a lot more in this video than you have in past ones. I still really enjoy your videos but this slightly hinders them. Now, I'm a pretty slow talker and in fact some of my friends and acquaintances have even told me this. But unlike you, I have an excuse: I talk slow because I think slow. In your case this doesn't apply - you're reading off of a script. Think you can work on that and try to stretch out your words less?

 

Here's one such occurence for reference:

https://www.youtube.com/watch?start=483&v=J8H5kYesvsM

Keep watching for a bit for more ("floating around")

 

Edit: I realize that you might be doing it intentionally for emphasis. I don't think it works - look for other ways to emphasize? The tone you use when you do it is good, but you don't have to also stretch the words when you use it.

 

With that out of the way, that was a fantastic video and surprisingly informative. I'm somewhat excited about VR (and more specifically, 3D) because it has the potential to enhance virtually any 3D game in existence. But I haven't had a chance to even try it, ever.

Thank you Ross for finally telling it like it is. I agree completely.

 

I also love that you referenced Star Trek: Generations in regards to wanting to be in good VR. I've only tried VR once on the DK1 a few years back, and I've made the exact comparison to the Nexus in regards to wanting to go back to it. I just hope they don't screw it up.

 

On the plus side, Bethesda announced that Fallout 4 and the new Doom are going to be made VR compatable, so that's good, right? Now for something like Bioshock: Infinite or Portal, assuming that I don't just constantly cry from my fear of heights why playing those in VR.

 

Also, what are your thoughts on the Omni? Does it work with a lot of games? Because if so, if I get one of those, too, I may just be a skinny man before the end of the year.

I use the HTC Vive, and I've been experimenting with a program I found on Steam, called MyDream Swift. I'd compare Swift with VorpX, although as competitors, one may work if the other doesn't.

 

I should say, though, "Operation: Get Ross Immersed" does sound worthy of a forum post. I'm certain that there's at least a few 3D programmers and fans who subscribe to Ross' channel, that could work together to set up the perfect immersion kit, whether it's 3D, virtual reality, or something else.

I also love that you referenced Star Trek: Generations in regards to wanting to be in good VR

The Nexus is really the closest analogy I can think of, since I really had some gaming euphoria moments back in 1999/2000 and would like to return to it.

 

Also, what are your thoughts on the Omni? Does it work with a lot of games?

I have no idea, but I think it has a lot of potential as long as the nausea is manageable. This is another reason to really want backwards support for some games that can handle it (maybe using my method), since there are many enormous existing game worlds. I do think sweat could be a concern, you might need a special washable pad to put on the VR device for long term use.

 

Hey, quick and honest criticism - you tend to talk slow, and stretch out your words. I think you've done that a lot more in this video than you have in past ones. I still really enjoy your videos but this slightly hinders them. Now, I'm a pretty slow talker and in fact some of my friends and acquaintances have even told me this. But unlike you, I have an excuse: I talk slow because I think slow. In your case this doesn't apply - you're reading off of a script. Think you can work on that and try to stretch out your words less?

 

Here's one such occurence for reference:

Keep watching for a bit for more ("floating around")

 

Edit: I realize that you might be doing it intentionally for emphasis. I don't think it works - look for other ways to emphasize? The tone you use when you do it is good, but you don't have to also stretch the words when you use it.

 

With that out of the way, that was a fantastic video and surprisingly informative. I'm somewhat excited about VR (and more specifically, 3D) because it has the potential to enhance virtually any 3D game in existence. But I haven't had a chance to even try it, ever.

This stuff isn't perfect. That video is the result of 3.5 hours of recording under strong heat. I was just glad to get it made. I can screw up or forget lines a lot and have to keep redoing them until I get them halfway decent. As for my speed, it could be slightly faster, but I don't think I'd want it dramatically faster, since I am going for emphasis.

 

Ross, I had jobs using 3D vision (on windows 7), I used 3D glasses, I found that Real 3D is possible with NVIDIA drivers and screens. The issue was not the driver but the product, with very certain result and method you could see really amazing crap, depth perception so good you feel like the tip of objects follow you when you move around your head.

The differences I saw between the products that worked and those who didn't (I'm unsure if some of them really even matter) were 4 main ones:

 

1. detailed pictured, that weren't using flat textures (obviously because I'm talking about real life photos), as many amount of details and shadows and bumps as possible in the picture. Im serious, the more happens the better things are.

Flat "3d" textures that only look 3D with a 2D screen just hurts the illusion (if that makes sense)

 

2. Limited color pallet, even black and white if possible, it sounds strange, but when you don't have colors distracting your vision, you're eyes concentrate on how everything different pops out, it works.

 

3. High resolution, as high as it can be. This one is obvious, most games do this alright so I have no complaints there. I'll just add that I think that nothing should be a pure color, everything needs a little depth with texture.

 

4.THIS IS IMPORTANT, possibly the reason many games fail with 3D.

There is a certain way 3D pictures on the screen are placed on each other, they need to have a common area at the right depth. Something that your eye gets its reference point from. Everything that diverts from that area in a certain direction, will seem either deeper or higher accordingly. The best result will happen (at least i think it will) when that point is at the center of your field of vision, it needs to make your character and the immediate area look close, while other things look far. putting that point too far will cause your head to hurt when looking at things that pop out too much, Putting the point too close, will make everything seem far, and barely have any effect. I think the latter is the issue with most 3D games. The way developers implement that point of reference to prevent people from crossing their eyes too hard.

The best way to solve it (at least in my opinion), is to give the user a certain degree of control on it, I don't know how it can be done effectively, but there are many possibilities on how developers can implement this. It can't be too hard.

 

I hope this gives some perspective (pun intended), NVIDIA have terrible idea on what 3d games should look, but they provide the hardware that can work with the right product.

I agree with #4 completely. Honestly, even though it's the same technology, the 3D effect from the Elsa Revelators was NOTICEABLY better than 3D vision, even though Nvidia bought them out and used the same technology. It took a lot of the "wow" factor away and I can't even explain why, but it sounds like you have a better grasp on it. As for flat textures, the illusion is obviously broken in 3D, although again, I'd be thrilled to play something with Quake or Unreal graphics in 3D, so I can scale my expectations accordingly. I'm not sure decent color would be an issue for me. As for low resolution, what I liked less were the lines between the pixels. The low resolution wasn't a huge problem for me, although it certainly does promote nearsighted tendencies since a lot of far off details can't be focused on.

 

 

I use the HTC Vive, and I've been experimenting with a program I found on Steam, called MyDream Swift. I'd compare Swift with VorpX, although as competitors, one may work if the other doesn't.

 

I should say, though, "Operation: Get Ross Immersed" does sound worthy of a forum post. I'm certain that there's at least a few 3D programmers and fans who subscribe to Ross' channel, that could work together to set up the perfect immersion kit, whether it's 3D, virtual reality, or something else.

Does that really work? On their Steam page, they have a picture of what looks like Borderlands, but it looks like the left and right eye are getting almost the exact same image (I did a quick photoshop test):

 

 

It seems like that would only create a flat image, not 3D.

Oculus has dropped its DRM suspiciously close to the release of this video, so it's one issue solved.

 

Source: http://arstechnica.com/gaming/2016/06/oculus-reverses-course-dumps-its-vr-headset-checking-drm/

Does that really work? On their Steam page, they have a picture of what looks like Borderlands, but it looks like the left and right eye are getting almost the exact same image (I did a quick photoshop test):

 

- image -

 

It seems like that would only create a flat image, not 3D.

Yup, I just went cross-eyed on it to gauge the 3D and it is absolutely flat.

 

Side-note, there's sadly a bunch of 3DS games that pull a similar "trick" - the game is completely flat, but it's pushed deeper into the screen to seem like it's 3D. In those cases I say, why bother? Just leave it flat at 2D and don't waste the battery. The most obnoxious one is Steel Diver: Sub Wars, for the simple reason that it's a Nintendo first-party game! They've pulled off some impressive 3D graphics before so for a graphically-mediocre game like that it should be easy.

Ross, I believe a lot of your issues with VR Sickness stem from the nature of the headset you're using and the movement system generally used for the DK2 and DK1.

 

I've had the DK1, DK2, and currently own a Vive. I'm not nauseated easily but the only time its happened was with the DK1.

It has three things going against it,

1. It can't track your actual head relative to any position, its simply a swiveling ball on a point in space, not very real to your head movement. The DK2 and newer don't have that issue.

2. The panel on the DK1 straight up blows. The refresh rate is at most 60hz and this is not good enough to keep up with what humans perceive as normal motion, even sitting pretty still. The DK2 is 75hz which is a little better and alleviates many more issues for a lot of people. CV1 and Vive are 90hz, further still improving everyone experiences. Many still get sick but that's more on the last issue.

3. Movement system is as big a part of VR experience as the 3D tracking is. For many people if the overall body movement and acceleration aren't anywhere close to what your senses tell you, nausea sets in quickly. That demo footage you showed of the DK1 were perfect examples. If you're being flung back and forth on a park ride but nothing else is syncing up you'll quickly get sick. This happens for many people even if your simply moving in a slow FPS with controller or keyboard. There simply isn't enough input on your body for it to feel right.

 

The Vive entertains a Room Scale experience from day one and mitigates number 3 by using a teleport system. You would think suddenly blinking into another part of the game world would be just as jarring as other systems but its surprisingly easy to handle. However this sort of system doesn't work for anything like a normal FPS as teleporting around breaks the typical flow. An omnidirectional treadmill would work to fix that, but pricey though. :/

Games like Windlands are good examples of requiring the iron stomach, featuring running, jumping, grappling around, parkour, and overall very fast movement speed. Even I will lean around in weird ways as my sense of balance is challenged with the overall movement. Playing seated helps but would still make many people hurl quickly.

 

Last I knew most Oculus movement systems are also teleport based, or the game is intended to be played seated/standing, no moving around the room. (Room scale will be supported but it has limited use without tracked controllers.)

 

Another movement system that both platforms have examples of is a world grappling mechanic where you pull yourself along the world by grabbing handles and ledges of it. It really is just moving the game world around your head, kinda like the Professors explanation of the ships movement in Futurama.

 

There are many other movement systems that are being worked on by different developers right now and not every system works best a particular game. For instance horror games probably can't make use of the teleport mechanic, least not that I've seen yet.

 

The game movement is most likely what stopped Nvidia from continuing on the path of direct 3D stereo scoping conversion to the HMDs. It made people hurl.

 

However, there is hope yet for converting games. Some games have been modded to hell and back to make use of new movement systems alongside proper 3D. There is a GTAV mod that adds teleport and independent hand movement for the Vive's tracked controllers. It breaks normal gameplay since you can move around in unintended ways but I've heard positive things about it as I don't own GTAV.

 

Other then mods there is your idea, the 3D Theater. Some companies do indeed make software for that to sit you down next to a huge ass 3D TV essentially. Valve includes it with Steam VR and launches almost any title into it, but will only display 3D if it uses a specific standard. Otherwise its just a 2D image on a huge TV.

 

 

BTW, I don't know where you're currently located but you can probably get a demo for the Oculus or Vive from either a local retail chain, Best Buy, Microsoft store, etc. Or there should still be community members open to let you try them at hosted events in public venues.

 

FYI If you're close enough I'm in the greater Baton Rouge area of Louisiana.

Really loved how long this video was, though to be fair I would pretty much just listen to you talking for hours on end about random topics if I could.

4.THIS IS IMPORTANT, possibly the reason many games fail with 3D.

There is a certain way 3D pictures on the screen are placed on each other, they need to have a common area at the right depth. Something that your eye gets its reference point from. Everything that diverts from that area in a certain direction, will seem either deeper or higher accordingly. The best result will happen (at least i think it will) when that point is at the center of your field of vision, it needs to make your character and the immediate area look close, while other things look far. putting that point too far will cause your head to hurt when looking at things that pop out too much, Putting the point too close, will make everything seem far, and barely have any effect. I think the latter is the issue with most 3D games. The way developers implement that point of reference to prevent people from crossing their eyes too hard.

The best way to solve it (at least in my opinion), is to give the user a certain degree of control on it, I don't know how it can be done effectively, but there are many possibilities on how developers can implement this. It can't be too hard.

 

"Focal plane" is the technical term. In real life this is determined by your eyeballs rotating in their sockets to converge at a particular point in 3D space. In order to properly replicate it, a VR headset (or 3D TV) would need eye tracking capability. There are rumors that the next gen of headsets will have this, but only rumors right now, so far as I know. The tech has been around for a couple of decades already, so it's mostly jusy a matter of getting it to fit inside the headset without screwing the price. Provided VR doesn't die while trying to get up to steam Like a lot of other 3D stuff did, eyeball tracking will inevitably be incorporated.

 

Fun fact: modern helicopter gunships use this to aim their nose turrets. The gunner has a helmet with an eye tracking eyepiece, and the gun automatically rotates to follow the gunner's eyeball, Predator shoulder cannon style.

 

In the interim, it seems to me like the same software used for depth of field could be easily adapted to focal planes. I mean both are basically about anticipating where in 3d space the player is looking, and setting a distance value accordingly, so a good DOF system should be able to double for a good FP system. Mind you that's a good DOF system; many DOF systems are either terrible (constantly focusing on the wrong things), or lazy (super-deep fixed DOF that's basically just an alternative to LOD fog).

 

Of course, the counterpoint would be that even good DOF effects aren't realistic, and can't be done realistically without eye tracking either.

 

On a more general note, the aesthetic camera design of the game can make a huge difference as well. I've been replaying the Tomb Raider reboot recently, and that game would be terrible for VR because of the way it handles camera movement. During gameplay the camera sways drunkenly with the most minor character movements, and the cutscenes have the most OTT shakey cam I've ever seen in any visual media. It's a great game, but there's something genuinely worrying in how the camera is designed. Like, it's not just someone applying "more is better" to compensate for a lack of know-how, it actually feels like someone trying to compensate for eyeball seizures or dyslexia-like visual field scrambling the way people who don't know they're losing their hearing compensate by shouting. I've never experienced motion sickness in my life, but I imagine that playing TR in VR would give me massive headaches, and would probably have Ross puking himself hollow within minutes.

 

I wouldn't want to play TR without a mod that reworks the camera system to get rid of all the swaying & shaking (and TBH, I would love for such a mod to exist anyway, as that is like the one big WTF flaw in the game regardless). Unfortunately, as far as I know TR is one of those games that can't be modded without cracking.

 

It also seems to me that a more "theater" type system like Ross describes would be almost mandatory for 3rd person titles in general, since head movements would not track proprioceptively to rotation around your character model the way they would in 1st person.

Interestingly, some of the cons listed for the Oculus Rift in the video are already becoming outdated, just a few days later.

 

Apparently, according to this Ars Technica article, Oculus is removing its DRM checking from its runtime (set to come out tomorrow?) in response to the uproar on the Internet over it.

 

I don't know which consumer version to buy, to be honest. I do like the room tracking of the Vive but I have to admit to myself that I'm lazy where it comes to games. I want to sit down. I know you can sit with the Vive as well, but I'm still puzzled over which is better. Ross says "none" at the moment because of the launch stupidity, but I really do want to see some kind of unfiltered pros and cons of each.

 

Now, I did try the Vive in a very controlled environment a while back at the Microsoft Store near Seattle. It was beautiful and I couldn't see the circular ridges Ross talks about in the video, but they gave me only like 5 minutes in the Vive because, well, there was a line and they only allowed you access to a few games (shooting flying robots, watching as a whale goes by a sunken ship, and TiltBrush). It wasn't enough time to make a decision on that versus the DK2, honestly. I would've liked some more variety such as Valve's "The Lab" or even some driving game where I could sit behind a steering wheel controller and drive a racecar in the Vive. Give me about a half hour with the sucker at least and let me pick the games. THAT would be ideal. But that wasn't to be.

 

I hope Ross does a more in-depth discussion of VR itself eventually rather than the rant on the VR launch. Now, I'm not complaining about the video, oh no. It was top notch and informative! I just want more. MORE!!!!

 

 

EDIT: And THIS idea, while looking kind of stupid, apparently relieves the 'instant nausea' affect of moving-forward games.

EDIT: And THIS idea, while looking kind of stupid, apparently relieves the 'instant nausea' affect of moving-forward games.

Why is this the first guy to think of this? It's basic human anatomy...

Excellent video Mr. Scott. I have to confess I've been feeling rather ambivalent about virtual reality, but your angry logic did make me care about the dodgy practices of both tech developers and the gaming media. I'm not one for conspiracy mongering but it's becoming increasingly harder to dismiss the notion that there is self-serving and systematic collusion going on between gaming journalism and gaming producers. I like how it escalates into full-blown Soran madness at the end of the video, and that closing dejection when you compare virtual reality to Romeo & Juliet...

...Give me about a half hour with the sucker at least and let me pick the games. THAT would be ideal. But that wasn't to be.

...

 

The best I can offer is to keep trying to get to demo both at some length. :7

 

Demoing things like Valve's "The Lab", and the "Oculus Home" environment, may not be entirely representative, because their respective art direction are both balanced around certain overall scene lighting levels, that happen to be optimal for minimising apparent artefacts caused by the fresnel lenses; A pleasant slightly-above-medium gray-ish - low on saturation and contrast; Think watercolour, but a tiny bit darker . :7

Go, instead, into a high contrast environment; A dark place dotted with bright accent lights, such as many cockpits in Elite Dangerous, whilst out in dark space, and the "god rays" may jump out at you and smear themselves intrusively all over your vision, millimetres from your eyes.

 

Overall, the Rift (Consumer Version 1) chooses more pixels per degree of your field of view (EDIT3: ...for a significantly sharper image), at the natural cost of a reduction in the width of the latter. This is mitigated a bit by a secondary sacrifice in binocular overlap; You'll still see almost as far to the sides as in a Vive, but the last 5-10 degrees out to either side, can only be seen by the eye on that side, as if you had the largest nose in the world. By all accounts, most people never seem to even notice this, but to me, personally, it is very annoying.

 

The Rift is also a more convenient experience; It is lighter, has adjustable headphones that are mounted on its rigid frame, which makes putting it on much the same as putting on a baseball cap, without any fiddling with straps and cables and separate headphones, whatsoever, and the Home environment starts right up, automatically, just from donning it.

 

Its lenses argueably offer a clearer and more consistent image, where text out to the sides is still pretty damn legible, whereas with the Vive, sharpness falls off, and you can get a bit of "double vision" between the outermost frenels lens rings. A few of us, however, experience a bit of distortion, in the Rift, when looking around, which can cause some "brain strain".

 

The Rift will get its own motion controllers, like the Vive's wands, sometime in the autumn. They are more designed to be "part of your hand", so to speak, taking shape from a hand in resting position, so that you can almost forget you are holding anything at all. They also have analogue buttons with capacitive touch sensing, for groups of fingers, which allows them to infer different hand gestures; E.g, squeeze the buttons under all four fingers, and take your thumb off all of the spots where it may rest, and that is interpreted as a thumbs up, or down, depending on where you are pointing the device.

 

The Vive is brighter, but possibly may not conform to sRGB standard - this appears to result in significantly higher overall contrast ratios in most stuff, compared to the Rift. On one hand, this kind of seems to add a bit of a "realistic" feel to the imagery - on the other, it causes a lot of loss in visual detail, with things in shadow crushing toward black, and bright things blowing out. If we had a good HDR and extensive colour gamut mastering standard, maybe an adaption of BT.2020, together with colour profiling for each HMD, things could be reined in, and those extra lumens exploited... (EDIT: Such a standard can not come soon enough, as far as I am concerned; I want today's content to look as right as possible in tomorrow's hardware, regardless of manifacturer, or API, or capability.) (EDIT2: ...and I also have some strong disagreeable opinions on the act of tying drivers and hardware/metaverse framework APIs to vendor frontends. )

 

Despite its numerous shortcomings (...and by most accounts untypically, I experience more fresnel glare than in the Rift), so far I find myself constantly turning to my Vive first - for standing and sitting experiences both, but that's me; People have wildly differing experiences with the devices; How well they fit the shape of each individual's face, alone, and how this affects not only comfort, but more importantly the optical path, tends to be rather important. :7

 

EDIT: And THIS idea, while looking kind of stupid, apparently relieves the 'instant nausea' affect of moving-forward games.

Why is this the first guy to think of this? It's basic human anatomy...

 

Not the first to think of it, I am pretty sure, but certainly one of the few to try it, in the face of rather condescending self-proclaimed know-it-all detractors. :7

 

The method has a few shortcomings; It can most noteably not overcome the lack of vestibular feedback from velocity changes, but it does give you a helpful amount of sensory input from the physical treading, and is the method I favour at this time (EDIT4: ...as a complement to room-scale free movement, mind you - not a replacement); just needs tracking of the user's feet, and perfect walk cycle phase matching. :7