This week at Virtana has been spent attempting to build upon my work from last week. I have been working on textured boxes, adding a UVC camera to the simulation, and started working with WebRTC to send camera images.

I had to make textured boxes for the simulation from a set of images which involved joining images and making meshes. I had to join images together and determine the texture coordinates for them in the new image. This should have been easy as it was just to get the coordinates of each original image in the new image and divide by the size of the new image, but I was having trouble displaying the images correctly. The problem was that the image coordinates in numpy start from the the top as (0, 0) where as texture vertices start at the bottom with (0, 0). Once I realised this all I had to do was the reverse the order of the images when determining the texture coordinates.
The next thing that caused me problems with making the box mesh was the ordering of the vertices when making the faces for the box. I knew that the order the vertices was specified for the texture was important but I didn’t know how exactly it was supposed to be until I read up on the winding order of the vertices that make up the triangle. This determines which side of the triangle is visible.

Winding Order Example:
Winding Order Example
If I wanted the following triange to appear visible when looking at it fron the front, then I would have to specify the points in counter-clockwise order: 1-> 2 -> 3
If I wanted the following triange to appear visible when looking at it fron the back, then I would have to specify the points in clockwise order: 1 -> 3 -> 2

A UVC (or USB Video Class) Camera had to be added to the simulation. For this camera, I had didn’t have intrinsics for it so I had to make them up for now , and unlike the previous camera that I added doesn’t send depth data. Also in the sending of the images to a remote viewer, I had to use OpenCV methods to convert the image from RGB to BGR and to encode the image as this is what the client was expecting.

Colour Spaces Example:

RGB Color Space BGR Color Space
Colour Space RGB Colour Space BGR

Lastly, I have started working to output camera images and other data from the simualtion to be sent using WebRTC to a remote viewer.

OpenCV