Award Winning Unity3D Application

I’m very proud to announce that the mobile application that I have been working on for a lengthy amount of time has won an award!

The MMA Smarties Awards is the world’s top global mobile marketing awards celebrating innovation, creativity, and success. Working alongside Engine Creative and BIC, we won the award in the Mobile App category – competing against other bluechip marketing companies including Jaguar, Heineken and McDonalds.

smartie-award-team2

 

The Application uses Unity3D to it’s full potential, along with using the powerful Vuforia for Augmented Reality tracking technology which pushes the immersive and educational experience to new bounds.

smarties_devices_v2

The app has now been released in six countries within Europe (including France, UK, Spain and Germany) and further releases are planned for Latin American.

Microsoft Kinect & Unity3D

Recently, I’ve been messing around with Microsoft’s Kinect Sensor (v2) within Unity3D to see what interesting results I can achieve. I’ve only really done large scale projects with the previous version of their sensor. But I must say, this new sensor is a huge improvement! And its tracking is on point!

3d377852-0f21-4074-a3c2-35f418170848

For such a complex device, their developer API is crystal clear, and very supportive (in terms of multi-platforms). The only down side is that their SDK is Windows only (so I couldn’t develop on my Mac) – which isn’t the end of the world, but just an inconvenience.

Depth & Infrared Point Cloud

I wanted to experiment with the depth and infrared sensor and to get a point cloud representation of my room within Unity3D.

2016-05-11 (1) copy

I achieved this by  accessing both the depth and infrared sensor, and creating a reader for them both. A reader is what actually contains the data from the sensors, think of it like a ‘feed’ of data.

Once I had the data coming into Unity, I needed to actually do something with that. I dynamically built a 65,000 vertices Mesh with a MeshTopology of a Point – to give me the point cloud mesh. Each point represents a pixel from the depth sensor, whereby each pixel represents the distance in millimetres. This gave me a very accurate representation of my room in 3D via point cloud.

I took this one step further by colouring each point cloud with the pixel data given from the IR sensor.

The Mesh is then recalculated every frame (Update) to give me a real time scan of the room I was sat in.

The Result

Incase you can’t make out what’s going on, what you’re seeing is my room – 3D scanned into Unity in real time. You might be able to make out me, sitting on my chair in the middle. And as you can see, each point cloud has it’s own colour value given from the IR sensor.

This is a front isometric view – it almost looks like an image, but it’s not! This is just all the point clouds aligning up perfectly to give the impression that it’s a 2D flat image.

Vuforia & Digital Eyewear

Over the past few years I’ve been working on a wide range of projects that use Vuforia within Unity. For those of you who don’t know what Vuforia is, it’s a bunch of highly sophisticated libraries that allow you to track a physical image or object (called a target) in which you can augment digital content on-top of it.

6a00d83451706569e2017ee8115a91970d

It’s pretty cool stuff that allows you to do very exciting things with your mobile device! However, whats even more cool is digital eyewear! There are a bunch of digital eyewear technology that works with Vuforia, including the Microsoft HoloLens! This lets you to view your content as if it were in your environment in a much realistic way due to the fact that the computer is essentially projecting the digital content into your eyes, instead of viewing on a mobile screen!

A part from the HoloLens, and the huge price tag they’ve slapped on it (for development release), I’m currently looking for a more affordable piece of hardware in terms of optical projection.

Epson Moverio BT-300

Who knew Epson didn’t just make printers? Indeed, they also have a production line for digital eyewear, some affordable, others not so much. The Epson Moverio BT-300 is their latest product (release date TBC).

Epson-Moverio-BT-300

The eyewear comes equipped with dual OLED micro projectors that allow for the display of 3D content with a binocular resolution of 720p as well as a 5m pixel camera. The camera allows you to feed your scene into Vuforia, which in turn allows you to track and place digital content into your environment, augmented reality!

It also comes equipped with a head tracking sensor that allow you move 360 with the computer knowing of your rotation.

The eyewear is powered and built on an Android device. The device is basically like a powerful smartphone, but without a screen – instead it has a touchpad like surface with buttons and a scroll wheel.

Potential Projects

I’ve always wanted to get my hands on some tech like this! Just to play around really and see what interesting projects and ideas I can come up with. Hopefully, when these start rolling off the production-line, and into the commercial stream I just might be able to do just that!

This technology is very useful, especially when you’re in a hands on field that requires training and exercise, such as being on an oil rig, gas maintenance, commercial electrical maintenance, military, etc.

You can use the headset to augment instructions as well as 3D animation and content to help the user through the training exercise, not only that but you can send the live camera feed to a coach, manager or captain in whereby they can review and give feedback on your progress.

Microsoft HoloLens Breakdown

I get really excited when a big tech firm starts talking about Virtual reality (VR) and Augmented reality (AR).

By now, I’m sure a lot of you are aware that Microsoft are bring their own device to the VR/AR world of technology. They’ve already started to ship the development edition which means we will most likely start seeing first hand experience and reviews regarding the long awaited for device from Microsoft.

5704f8ff4ea9a

Virtual Reality Headset?

No, not quite. This device is not to be confused with the popular Oculus Rift which is indeed a true Virtual Reality headset. The HoloLens is basically a visor, so once you place it on your head, you will be able to see straight through it – just like some very expensive Safety Goggles you had to wear back in Chemistry class.

So this obviously allows you to see your own environment. But what’s interesting is that the headset has micro optical projectors built into it, that essentially projects digital content onto the ‘visor’ allowing you to see a mixture of your world, and the computers world, this is called Augmented Reality. It’s where the computer ‘augments’ content on-top of your own environment.

Virtual Reality headsets don’t do this, they put you into a complete virtual world in whereby all you see is computer generated content.

 

hololens

Augmented Reality

This by far, isn’t new technology at all, it has been around for decades mainly used as a heads up display for fighter pilots in the form of large bulky helmets. But as usual, technology over time starts to shrink in size, yet gain more power, which equals a better experience!

Sensor Overload

In order to view digital content into someones real environment (as an overlay – augmented) it requires the computer to understand about the environment you’re in, such as depth and 3D positioning.

This requires some serious sensors on board the head unit, which it does! It’s able to capture information about what you’re doing and the environment around you. This is achieved with an array of camera on the headset, such as 3D depth cameras, IR cameras and stereoscopic cameras. The CPU then takes all this information and is able to project the 3D content onto the visor of the headset, giving the illusion that it’s actually there, in your room!

Microsoft-HoloLens-Family-Room-RGB

The Illusion

Like I said, this is essentially giving the illusion that there is digital content within your environment, but it’s simply just a micro image you’re seeing within the visor of the headset, giving the impression that it is overlaid within, or on top of your actual environment. No one else will be able to see what you’re seeing, or interacting with, unless they are equipped with a headset too!

And yes – this may look odd if you’re running around a room whilst a fire breathing dragon is following you as you duck and roll to try and avoid being burnt to a crisp (so to speak). When in reality, none of this is happening, and you’ll look like a fool to someone who is just sat in the corner trying to read a news paper and drink their coffee.

With all that in mind, I honestly can’t wait until this device becomes commercial. I’m trying to get my hands on one, however they only ship to U.S and Canada currently, and it will set me back by $3,000!

 

Blogs Running!

I finally got round to setting up my blog and incorporating it into my website!

Stay tuned for up to date technology and app analysis and discussions, as well as many random posts to peak the interest in some of you!

First Hand Experience

A lot of the time, I manage to get my hands bits of technology that not many other people have the privilege to access, or just simply don’t have the resources to. Because of this, I will try and give my best impression, and review on certain technology, whether that by hardware, or software!

Developing & Modern World

If you have been living in a cave, you may not have heard about the world going crazy for Virtual and Augmented Reality technology. Not only does this have the people interested, but also developers and commercial CEOs (yes, they’re a different type of people!). Because of this, and me being a developer myself, I like to work on such projects that try and heavily involve all this type of technology.

More To Come! …

So with that in mind, there is indeed more to come with regards to technology, apps, and of course, the odd random post.