krrnk | VIVR

// 2018.04.11
"VIVR” (Vibrating Virtual Reality) is a research project at the SOPI research group in Aalto University.
In this project, we are implementing a Virtual Reality Musical Instrument focusing on music interaction between the performer and the 3D environment.
Our attention focuses on the 3D environment and not on a particular 3D model of an instrument because we want to exploit the immersion feature of VR. Therefore, a 3D environment can be seen as a resonating body the musician is trapped within and that excites from the inside.
Within this vision, we are focusing on fours factors that are crucial in order to relate immersion and music interaction: 3D audio, spatial interaction, the performer and the sonic world.
I'm starting this blog in order to change my experiences in developing an instrument for VR and to keep track of how ideas develop over the process.

The technology we are working with is a HTC Vive setup as hardware, Unity 3d as game engine and SuperCollider as sound engine. The communication between HTC Vive and Unity 3d is done through the SteamVR SDK. The communication between Unity 3d and SuperCollider is done via OSC.
For this blog I will post code snippets of what might be relevant to the topic.

// 2018.04.11 The beginning of the idea
Since I started in Media Lab Helsinki I have been interested in sound and movement interaction. I'd like to graduate in this academic year so I was looking for a thesis idea. Back in 2014 or something, when I used VR for the first time I wasn't even triggered by it. Even though it is still pretty much focus in killing zombies games, I have to admit that nowadays there are some applications that bring new experiences out of it.
When using the VR set up at school, we made jokes about how disconnected we would get from reality. This immersion made me think that VR could be a perfect platform for an audiovisual installation. I thought of different ideas of how this installation could be. The one that sounded more interesting involved a sentient environment that would react audiovisually to user input. The objective would be to try to establish an audiovisual communication between the user and the environment until both would resonate on the same level.
At the same time, SOPI begun to work on a VR musical instrument. When exchanging ideas, the leader of the group, Koray Tahiroğlu, thought that it would be useful to combine forces and come up together with a VR musical instrument that would have certain degree of autonomy. In our discussions, we looked into the affordances of VR and how these could benefit music interaction. In combination with the ideas proposed by previous research on the field we came to the conclusion that 3D audio, spatial interaction, the performer and the sound world were essential factors in the relation of VR and music interaction.

// 2018.04.27 3D audio as a music feature
One of the first try outs I did with Unity and SuperCollider was simple test of sound spatialisation. I tried three different Ambisonic quarks/plugins in SuperCollider: ATK, SC-HOA, and AmbIEM. Although these three implementations have their own potential, I found that SC-HOA and ATK were the most respectful ones towards the sound source. AmbIEM seemed to filter the sound source more than the other two. Even though SC-HOA is specially good and well documented, I feel that it is taking a while for the SuperCollider developers to release it as a official plug-in. In the meantime and because I'm working on a university computer where having a non-supported software like SuperCollider is already a hassle, I have decided to use ATK.
It was during this simple test that I came across my first problem. My first attempt was to use Unity's cartesian coordinates directly. After trying to figure out why the translation of cartesian to spherical coordinates wouldn't work I realised that in spherical coordinates the axis are different. Therefore, the variables should be as follows:

OSCdef(\objectPosition, {
        |msg|

var axisX, axisY, axisZ, radial, azimuth, polar;

// From cartesian coordinates to spherical
        // front and rear
        axisX = msg[3]; // Z transform.position axis in Unity
        // left and right
        axisY = msg[1]; // X transform.position axis in unity
        // top to bottom
        axisZ = msg[2]; // Y transform.position axis in unity

// Sphereical coordinates formulas
        // radial sqrt(x^2 + y^2 + z^2)
        radial = sqrt((axisX ** 2) + (axisY ** 2) + (axisZ ** 2));
        // I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
        // Therefore my maximum value is 56.789083458003
        radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
        azimuth = atan2(axisY, axisX) * -1; // result will be in radians

// polar cos^(-1)(z/r)
        polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians

},
    '/objectPosition',
    nil,
    6161
);

Nevertheless, when working in VR we expect the user to move or at least rotate the head. Thus, we should get user's position and head rotation and apply them to the function above.

// Player's position
OSCdef(\hmdPos, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar;

// Cartesian coordinates
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	~hmdX = axisX;

// left and right
	axisY = msg[1]; // X transform.position axis in Unity
	~hmdY = axisY;

// top to bottom
	axisZ = msg[2]; // Y transform.position axis in Unity
	~hmdZ = axisZ;
},
'/HMDPosition',
nil,
6161
);

// Player's head rotation
OSCdef(\hmdRot, {
	|msg|

var roll, pitch, yaw;

// Receiving rotation euler angles from Unity
	roll = msg[3]; // Z transform.localRotation.eulerAngles axis in Unity
	~hmdRoll = roll;

pitch = msg[1]; // X transform.localRotation.eulerAngles axis in Unity
	// to simulate Ambisonics degrees
	// scale from 0-360 to -180-180
	~hmdPitch = pitch;
	~hmdPitch = ((~hmdPitch - 180) % 360) - 180;
	~hmdPitch = ~hmdPitch.linlin(-180, 180, -pi / 2, pi / 2) * -1;

yaw = msg[2]; // Y transform.localRotation.eulerAngles axis in Unity
	// to simulate Ambisonics degrees
	// scale from 0-360 to -180-180
	~hmdYaw = yaw;
	~hmdYaw = ((~hmdYaw - 180) % 360) - 180;
	~hmdYaw = ~hmdYaw.linlin(-180, 180, -pi, pi) * -1;
},
'/HMDRotation', // message address
nil,
6161 // message port
);

// Object's sound source location according to user's position and rotation
SCdef(\objectPosition, {
        |msg|

var axisX, axisY, axisZ, radial, azimuth, polar;

// From cartesian coordinates to spherical
        // front and rear
        axisX = msg[3]; // Z transform.position axis in Unity
        // left and right
        axisY = msg[1]; // X transform.position axis in unity
        // top to bottom
        axisZ = msg[2]; // Y transform.position axis in unity

// Sphereical coordinates formulas
        // radial sqrt(x^2 + y^2 + z^2)
        radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
        // I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
        // Therefore my maximum value is 56.789083458003
        radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
        azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
        azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
        polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
        polar = polar - ~hmdPitch;
    },
    '/objectPosition',
    nil,
    6161
);

The above idea works when trying to represent external sound sources. But for VIVR we wanted the user to be inside the instrument. Thus, my approach to make the 3D audio feature a musical feature was to give the user control on the sound spatialisation. In this manner, the user would be able to freely move the sound in the 3D the space. Unity's Physics.RayCast comes really handy in this sense. Taking into consideration that almost every 3D object that one would add to the environment would have a collider, we can spatialise the audio along the surfaces of those colliders. Unity's function is as follows:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class RayCastLocationL : MonoBehaviour {
    //---------------- Variables ----------------
    // Refer the OSC.cs file
    public OSC osc;

// Set Vive controllers
    // reference to the tracked obj aka controller
    private SteamVR_TrackedObject trackedObj;
    private SteamVR_Controller.Device Controller
    {
        get { return SteamVR_Controller.Input((int)trackedObj.index); }
    }

//---------------- Functions ----------------
    // Evaluates before start
    void Awake()
    {
        trackedObj = GetComponent<SteamVR_TrackedObject>();
    }

// Update is called once per frame
    void Update () {
        HandleInput();
    }

void HandleInput()
    {
        // OSC settings
        OscMessage oscMsg = new OscMessage();
        oscMsg.address = "/rayCastLocationL";

// Use physics engine to cast the ray and check if it has hit something
        RaycastHit hit;

// Target's coordinates
        Vector3 point;

// Raycasting
        if (Physics.Raycast(trackedObj.transform.position, trackedObj.transform.forward, out hit))
        {

// Hit will return a Vector3 point when hitting a collider
            point = hit.point;

// Add the coordinates to the OSC array
            oscMsg.values.Add(point.x);
            oscMsg.values.Add(point.y);
            oscMsg.values.Add(point.z);
            oscMsg.values.Add(hit.distance); // distance can be used for amplitude, filtering and reverberation

} else
        {
            // Initialise the OSC array in 0
            // so supercollider doesn't get confused
            // if the Raycast doesn't happen
            oscMsg.values.Add(0);
            oscMsg.values.Add(0);
            oscMsg.values.Add(0);
            oscMsg.values.Add(0);
        }

// Send the message to the client
        osc.Send(oscMsg);

}

Notice that the above function refers to the Left controller. This same script should be renamed as "RayCastLocationR" and added to the other HTC Vive controller. As it can be seen, I have added the OSC messages in this excerpt. I'm using this OSC script by Thomas Fredericks in order to get and send OSC in Unity.
One useful parameter that Raycasting provides us with is the distance between the collision point and the Raycast source. This parameter can be mapped to the ambisonic's proximity feature as well as to the amplitude. This allows us to have better virtual representation of how sound sources have a different volume and are filtered depending on their distance.
The following code excerpt displays the DSP implementation of the ambisonic module as well as the updated OSC function.

//  Declare encoder and decoder to use
~encoder = FoaEncoderKernel.newSpread;
~decoder = FoaDecoderKernel.newListen;

// Load Synth Definition
SynthDef(\ambisonicModule, {
	|inBus = 0, outBus = 0, angle = 0, azimuth = 0, elevation = 0, distance = 1, amp = 0.01|

var input, encode, postEncode, transform, decode, output;

input = In.ar(inBus);

// filter for proximity
	input = HPF.ar(input, 1);

encode = FoaEncode.ar(input, ~encoder);

transform = FoaTransform.ar(encode, 'proximity', distance);
	transform = FoaTransform.ar(transform, 'zoom', angle, azimuth, elevation);

decode = FoaDecode.ar(transform, ~decoder, amp);

output = Out.ar(outBus, decode);
}).add;

// Create 2 synths, one for each controller
s.sendMsg(\s_new, "ambisonicModule", 1011, 1, 1, \inBus, 0);
s.sendMsg(\s_new, "ambisonicModule", 1012, 1, 1, \inBus, 1);

// Player's position
OSCdef(\hmdPos, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar;

// Cartesian coordinates
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	~hmdX = axisX;

// left and right
	axisY = msg[1]; // X transform.position axis in Unity
	~hmdY = axisY;

// top to bottom
	axisZ = msg[2]; // Y transform.position axis in Unity
	~hmdZ = axisZ;
},
'/HMDPosition',
nil,
6161
);

// Player's head rotation
OSCdef(\hmdRot, {
	|msg|

var roll, pitch, yaw;

// Receiving rotation euler angles from Unity
	roll = msg[3]; // Z transform.localRotation.eulerAngles axis in Unity
	~hmdRoll = roll;

pitch = msg[1]; // X transform.localRotation.eulerAngles axis in Unity
	// to simulate Ambisonics degrees
	// scale from 0-360 to -180-180
	~hmdPitch = pitch;
	~hmdPitch = ((~hmdPitch - 180) % 360) - 180;
	~hmdPitch = ~hmdPitch.linlin(-180, 180, -pi / 2, pi / 2) * -1;

yaw = msg[2]; // Y transform.localRotation.eulerAngles axis in Unity
	// to simulate Ambisonics degrees
	// scale from 0-360 to -180-180
	~hmdYaw = yaw;
	~hmdYaw = ((~hmdYaw - 180) % 360) - 180;
	~hmdYaw = ~hmdYaw.linlin(-180, 180, -pi, pi) * -1;
},
'/HMDRotation', // message address
nil,
6161 // message port
);

// Controller's position
OSCdef(\rayCastLocationL, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// left and right
	axisY = msg[1]; // X transform.position axis in unity
	// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity

// Sphereical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Amplitude
	amp = rayDistance.linexp(0, 50, 1.0, 0.0001);

// Set SynthDefs
	s.sendMsg(\n_set, 1011, \angle, radial, \azimuth, azimuth, \elevation,  polar, \distance, distance, \amp, amp);

},
'/rayCastLocationL', // message address
nil,
6161 // message port
);

OSCdef(\rayCastLocationR, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// left and right
	axisY = msg[1]; // X transform.position axis in unity
	// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity

// Sphereical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Amplitude
	amp = rayDistance.linexp(0, 50, 1.0, 0.0001);

// Set SynthDefs
	s.sendMsg(\n_set, 1012, \angle, radial, \azimuth, azimuth, \elevation,  polar, \distance, distance, \amp, amp);

},
'/rayCastLocationR', // message address
nil,
6161 // message port
);

Finally, we can push the potential of the ATK by using it as a processing module at the same time. Following the chain: input (n channel signal) -> encoding (n channel signal) -> decoding (W, X, Y, Z channel signal) -> process (W, X, Y, Z channel signal) -> encoding (W, X, Y, Z channel signal) -> spatial transformation (W, X, Y, Z channel signal) -> decoding (n channel signal), one can distribute the signal process spatially, thus getting an output with a vibrating deepness and sonic texture that is not possible to achieve by processing the signal before the encoding or after the decoding. In the case of VIVR, this chain is implemented by adding a delay unit before the second encoding.
At the same time, because the action is happening within an environment it's useful to add a reverb module after the ambisonic process.

//  Declare encoder and decoder to use
~encoder = FoaEncoderKernel.newSpread;
~decoder = FoaDecoderKernel.newListen;

// Load Synth Definition
SynthDef(\ambisonicModule, {
	|inBus = 0, outBus = 0, angle = 0, azimuth = 0, elevation = 0, distance = 1, amp = 0.01|

var input, encode, postEncode, transform, decode, output;

input = In.ar(inBus);

// filter for proximity
	input = HPF.ar(input, 1);

encode = FoaEncode.ar(input, ~encoder);

// apply effects
	postEncode = FoaDecode.ar(encode, FoaDecoderMatrix.newBtoA);
	postEncode = postEncode + DelayC.ar(postEncode, 0.2, delTime);
	encode =  FoaEncode.ar(postEncode, FoaEncoderMatrix.newAtoB);

transform = FoaTransform.ar(encode, 'proximity', distance);
	transform = FoaTransform.ar(transform, 'zoom', angle, azimuth, elevation);

decode = FoaDecode.ar(transform, ~decoder, amp);
	decode = GVerb.ar(decode, 50, 2, damping, dryLevel);

output = Out.ar(outBus, decode);
}).add;

// Create 2 synths, one for each controller
s.sendMsg(\s_new, "ambisonicModule", 1011, 1, 1, \inBus, 0);
s.sendMsg(\s_new, "ambisonicModule", 1012, 1, 1, \inBus, 1);

// Player's position
OSCdef(\hmdPos, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar;

// Cartesian coordinates
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	~hmdX = axisX;

// left and right
	axisY = msg[1]; // X transform.position axis in Unity
	~hmdY = axisY;

// top to bottom
	axisZ = msg[2]; // Y transform.position axis in Unity
	~hmdZ = axisZ;
},
'/HMDPosition',
nil,
6161
);

// Player's head rotation
OSCdef(\hmdRot, {
	|msg|

var roll, pitch, yaw;

// Receiving rotation euler angles from Unity
	roll = msg[3]; // Z transform.localRotation.eulerAngles axis in Unity
	~hmdRoll = roll;

pitch = msg[1]; // X transform.localRotation.eulerAngles axis in Unity
	// to simulate Ambisonics degrees
	// scale from 0-360 to -180-180
	~hmdPitch = pitch;
	~hmdPitch = ((~hmdPitch - 180) % 360) - 180;
	~hmdPitch = ~hmdPitch.linlin(-180, 180, -pi / 2, pi / 2) * -1;

yaw = msg[2]; // Y transform.localRotation.eulerAngles axis in Unity
	// to simulate Ambisonics degrees
	// scale from 0-360 to -180-180
	~hmdYaw = yaw;
	~hmdYaw = ((~hmdYaw - 180) % 360) - 180;
	~hmdYaw = ~hmdYaw.linlin(-180, 180, -pi, pi) * -1;
},
'/HMDRotation', // message address
nil,
6161 // message port
);

// Controller's position
OSCdef(\rayCastLocationL, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp, delTime, dryLevel, damping;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// left and right
	axisY = msg[1]; // X transform.position axis in unity
	// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity

// Sphereical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Amplitude
	amp = rayDistance.linexp(0, 50, 1.0, 0.0001);

// Delay Time
	delTime = rayDistance.linexp(0, 50, 0.01, 0.1);

// Reverb parameters
	dryLevel = rayDistance.linexp(5, 50, 1.00, 0.001);
	damping = rayDistance.linexp(3, 50, 1, 0.1);

// Set SynthDefs
	s.sendMsg(\n_set, 1011, \angle, radial, \azimuth, azimuth, \elevation,  polar, \distance, distance, \amp, amp, \delTime, delTime, \dryLevel, dryLevel, \damping, damping);

},
'/rayCastLocationL', // message address
nil,
6161 // message port
);

OSCdef(\rayCastLocationR, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp, delTime, dryLevel, damping;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// left and right
	axisY = msg[1]; // X transform.position axis in unity
	// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity

// Sphereical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Amplitude
	amp = rayDistance.linexp(0, 50, 1.0, 0.0001);

// Delay Time
	delTime = rayDistance.linexp(0, 50, 0.01, 0.1);

// Reverb parameters
	dryLevel = rayDistance.linexp(5, 50, 1.00, 0.001);
	damping = rayDistance.linexp(3, 50, 1, 0.1);

// Set SynthDefs
	s.sendMsg(\n_set, 1012, \angle, radial, \azimuth, azimuth, \elevation,  polar, \distance, distance, \amp, amp, \delTime, delTime, \dryLevel, dryLevel, \damping, damping);

},
'/rayCastLocationR', // message address
nil,
6161 // message port
);

// 2018.05.07 Setting up to deform the environment
The concept of VIVR is to make the users feel that they are inside the instrument. That's why the environment feels like being inside a room. Nevertheless, as soon as users get hands on VIVR they can realise that the notion of a cubed room can be completely destroyed. I set up a space of 6 surfaces (floor, ceiling and 4 side walls) using custom 3D shapes based on the cobination of two tutorials I found on the site http://catlikecoding.com/unity/tutorials/ .
In this website Jasper Flick wrote this Rounded Cube tutorial on how to make rounded cubes that can change shape according to the parameters of X, Y, Z sizes and roundness. By setting this parameters in the Unity Editor to X Size=55; Y Size=2; Z Size 55; and Roundness=2 one can get a procedural 3D shaped wall.
Using the script from the rounded cube tutorial inside the Mesh Deformation tutorial instead of the Cube Sphere proposed by the autor one can deform the walls according to user input. The core scripts of this turorial are the MeshDeformer.cs script attached to the Game Object that one wants to deform and the MeshDeformerInput.cs attached to Game Object that is intended to be the deformation source. The MeshDeformerInput.cs script has two parameters to control: the force applied to the deformed object, and a force offset to make the the deformation follow the direction of the force input. On the deformed object, the MeshDeformer.cs script has public paramters of Spring Force and Damping. Spring Force sets how big is leap of the vertices jumping back and forth. The Damping paramter sets how smoothly this bouncing happens.
In VIVR the MeshDeformerInput.cs script is attached to each HTC Vive Controller but the force comes from the FFT values of the sound sources. I'm passing the audio output of my sound sources through a FFT chain to read raw FFT values, a Loudness chain and an Amplitude tracker. The values of each module are multiplied and scaled by a forceIndex variable in order to get the total magnitude of FFT values that would be output as force. A few code examples would help to explain this better.

// Raw FFT tracker implemented by SuperCollider mailing list user Block
// http://new-supercollider-mailing-lists-forums-use-these.2681727.n2.nabble.com/getting-fft-values-tp7594791p7595072.html
(
//---------- System options ----------
s.latency = 0.05;
s.options.sampleRate = 48000;
s.options.numOutputBusChannels = 2;
s.options.numAudioBusChannels = 1024 * 2; // default is 1024
s.options.numWireBufs = 64 * 16;
s.options.memSize = 2.pow(20);

//---------- DSP ----------
s.waitForBoot{
    // FFT
    ~bufSize = 512; // Buffer size, try with 512 if you have a fast machine
    ~fftValuesBuf1 = Bus.control(s, ~bufSize.div(2));
    ~fftValuesBuf2 = Bus.control(s, ~bufSize.div(2));

s.sync;

// ATK
    ~encoder = FoaEncoderKernel.newSpread;
    ~decoder = FoaDecoderKernel.newListen;

s.sync;

// Samples for granular synthesis
    ~path1 = Platform.resourceDir +/+ "/sounds/a11wlk01.wav";
    ~sample1 = Buffer.read(s, ~path1, bufnum: 0);
    ~path2 = Platform.resourceDir +/+ "/sounds/a11wlk01-44_1.aiff";
    ~sample2 = Buffer.read(s, ~path2, bufnum: 1);

s.sync;

// FFT tracker
    SynthDef(\fftTracker, {
        |in = 21, amp = 1, outBus = 0|

// Audio in (mono)
        var z = In.ar(in);

// FFT chain
        var chain = FFT(LocalBuf(~bufSize), z);

// Unpack each buffer of the chain/2 (mono)
        Array.fill(~bufSize.div(2), {
            |i|
            var a = Unpack1FFT(chain, ~bufSize, i); // extracts a single scalar value as a demand-rate stream.
            var d = Demand.kr(trig: chain >= 0, reset: 0, demandUGens: a); // demand the values extracted by the Unpack UGen
            Out.kr(outBus + i, d.min(1));
        });

}).load;

// Audio signal
    SynthDef(\grain, {
        |outBus = 21, bufnum = 0, triggerRate = 3, rate = 1, bufferPos = 0, grainSize = 0.3, grainAmp = 0.1, amp = 1, lagTime = 0.1,  id =  1|

var trigger, centerPos, signal, output;

trigger = Impulse.ar(triggerRate);

centerPos = bufferPos * BufDur.kr(bufnum);

signal = TGrains.ar(1, trigger, bufnum, rate * BufRateScale.kr(bufnum), bufferPos  * BufFrames.kr(bufnum), grainSize, amp: Lag2.kr(grainAmp, lagTime));

output = signal * amp;
        output = Mix(output);

Out.ar(outBus, output);

// OSC
        // tracking amplitude
        SendTrig.ar(Impulse.ar(40), id, output);

}).send(s);

// FFT Loudness
    SynthDef(\loudness, {
        |inBus = 30, outBus = 601, id = 11|

var input, fft, loudness;

input = In.ar(inBus);

fft = FFT(LocalBuf(1024), input, wintype: 1); //using LocalBuf per HelpFile recommendation, 1024 frame size recommended for samp rates 44100 and 48000 and 2048 for rates 88200 and 96000

loudness = Loudness.kr(fft);

Out.kr(outBus, loudness);

SendTrig.ar(Impulse.ar(40), id, loudness);

}).add;

// ATK
    SynthDef(\atk, {
        |inBus = 0, outBus = 0, angle = 0, azimuth = 0, elevation = 0, distance = 1, amp = 0.01|

var input, encode, postEncode, transform, decode, output;

input = In.ar(inBus);

// filter for proximity
        input = HPF.ar(input, 1);

encode = FoaEncode.ar(input, ~encoder);

transform = FoaTransform.ar(encode, 'proximity', distance);
        transform = FoaTransform.ar(transform, 'zoom', angle, azimuth, elevation);

decode = FoaDecode.ar(transform, ~decoder, amp);

output = Out.ar(outBus, decode);

}).add;

};
);

(
//---------- Variables ----------
// This array has to be created for each audio source one wants to track
// this particular examples demonstrates controller 1 and 2
var fftArray1 = Array.fill(~bufSize, 0);
var fftValues1 = 0;
var fftArray2 = Array.fill(~bufSize, 0);
var fftValues2 = 0;

// The values got out of the fft multiplication by the loudness and amplitude
// are so small that the forceIndex has to be this big
var forceIndex = 200000;

// Once more, these would have to be created for each audio soure
var amp1 = 0;
var loudness1 = 0;
var oldLoudness1 = 0;
var loudnessVel1 = 0;

var amp2 = 0;
var loudness2 = 0;
var oldLoudness2 = 0;
var loudnessVel2 = 0;

// Set up OSC client
var client = NetAddr.new("127.0.0.1", 12345);

//---------- Create synths ----------
s.sendMsg(\s_new, "grain", 1011, 1, 1, \outBus, 21); // controller 1
s.sendMsg(\s_new, "grain", 1012, 1, 1, \outBus, 22, \id, 2, \bufnum, 1); // controller 2

s.sendMsg(\s_new, "fftTracker", 1021, 1, 1, \inBus, 21, \outBus, 0); // controller 1
s.sendMsg(\s_new, "fftTracker", 1022, 1, 1, \inBus, 22, \outBus, 257); // controller 2

s.sendMsg(\s_new, "loudness", 1031, 1, 1, \inBus, 21); // controller 1
s.sendMsg(\s_new, "loudness", 1032, 1, 1, \inBus, 22, \outBus, 602, \id, 12); // controller 2

s.sendMsg(\s_new, "atk", 1041, 1, 1, \inBus, 21); // controller 1
s.sendMsg(\s_new, "atk", 1042, 1, 1, \inBus, 22); // controller 2

//---------- Set SynthDefs ----------
// FFT trackers bus assigment
s.sendMsg(\n_set, 1021, \bus, ~fftValuesBuf1);
s.sendMsg(\n_set, 1022, \bus, ~fftValuesBuf2);

//---------- OSC Functions ----------
OSCdef(\amp, {
    |msg|

// OSC ID
    var id = msg[2];

// FFT buffer size
    var size = ~bufSize.div(2);

if(id == 1){
        amp1 = msg[3];
    };

if(id == 2){
        amp2 = msg[3];
    };

if(id == 11){
        loudness1 = msg[3];

// Calculate velocity/range of change
        fork{
            oldLoudness1 = loudness1;
            0.001.wait;
        };
        loudnessVel1 = (loudness1 - oldLoudness1).abs;

// Apply it to the SynthDef
        s.sendMsg(\n_set, 1011, \grainSize, loudnessVel1.linlin(0, 1, 1, 0.1), \triggerRate, loudnessVel1.linlin(0, 1, 3, 6));
    };

if(id == 12){
        loudness2 = msg[3];

// Calculate velocity/range of change
        fork{
            oldLoudness2 = loudness2;
            0.001.wait;
        };
        loudnessVel2 = (loudness2 - oldLoudness2).abs;

// Apply it to the SynthDef
        s.sendMsg(\n_set, 1012, \grainSize, loudnessVel2.linlin(0, 1, 1, 0.1), \triggerRate, loudnessVel2.linlin(0, 1, 3, 6));
    };
    // Get the values of each channels and put it into index 1 of the array
    ~fftValuesBuf1.getnSynchronous(~bufSize.div(2)).do{
        |val, i|

fftArray1.put([size-i, size+i], val);
        // fftArray1[1].postln; // Here are the values
        fftValues1 = fftArray1[1];
    };
    ~fftValuesBuf2.getnSynchronous(~bufSize.div(2)).do{
        |val, i|

fftArray2.put([size-i, size+i], val);
        // fftArray2[1].postln; // Here are the values
        fftValues2 = fftArray2[1];
    };
    // Multiply all the values in order to get the magnitude of changes
    ~totalFFT1 = fftValues1 * amp1 * loudness1 * forceIndex;
    ~totalFFT2 = fftValues2 * amp2 * loudness2 * forceIndex;

// Send message to client
    client.sendMsg("/fft1", ~totalFFT1.abs, amp1.range(-0.1, 0.1), loudness1);
    client.sendMsg("/fft2", ~totalFFT2.abs, amp2.range(-0.1, 0.1), loudness2);

},
'/tr',
s.addr
);

);

On the Unity side, the MeshDeformerControllerInputL.cs script receives the FFT values from OSC, sets them as force and sends them to each Game Object with a MeshDeformer.cs script attached to it. We would have to do the same for the MeshDeformerControllerInputR.cs script but chaninging the OSC receiving address to "/fft2" and the sending address to "/rayCastLocationR".

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class MeshDeformerControllerInputL : MonoBehaviour {
    //---------------- Variables ----------------
    // Refer the OSC.cs file
    public OSC osc;

// Set Vive controllers
    // reference to the tracked obj aka controller
    private SteamVR_TrackedObject trackedObj;
    private SteamVR_Controller.Device Controller
    {
        get { return SteamVR_Controller.Input((int)trackedObj.index); }
    }

// Physics
    public float force;
    public float forceOffset;

// FFT
    float fft;

//---------------- Functions ----------------
    // Evaluates before start
    void Awake()
    {
        trackedObj = GetComponent<SteamVR_TrackedObject>();
    }

// Use this for initialization
    void Start () {
        // Declare OSC Address
        osc.SetAddressHandler("/fft1", OnReceiveFFT);
    }

void OnReceiveFFT(OscMessage message)
    {
        // Acessing the ~totalFFT1.abs variable from SuperCollider
        fft = message.GetFloat(0);
        force = fft;
    }

// Update is called once per frame
    void Update () {
        HandleInput();
    }

void HandleInput()
    {
         // OSC settings
        OscMessage oscMsg = new OscMessage();
        oscMsg.address = "/rayCastLocationL";

// Use physics engine to cast the ray and check if it has hit something
        RaycastHit hit;

// Target's coordinates
        Vector3 point;

// Raycasting
        if (Physics.Raycast(trackedObj.transform.position, trackedObj.transform.forward, out hit))
        {
            MeshDeformer deformer = hit.collider.GetComponent<MeshDeformer>();
            Renderer renderer = hit.collider.GetComponent<Renderer>();

// Hit will return a Vector3 point when hitting a collider
            point = hit.point;

if (deformer)
            {
                // Add offsett so the vertices are always pushed into the surface
                point += hit.normal * forceOffset;

// Deform the mesh
                 deformer.AddDeformingForce(point, force);

}

// add the elements to the OSC array
            oscMsg.values.Add(point.x);
            oscMsg.values.Add(point.y);
            oscMsg.values.Add(point.z);
            oscMsg.values.Add(hit.distance);

}

// add 0 to the OSC array
        // so supercollider doesn't get confused
        oscMsg.values.Add(0);
        oscMsg.values.Add(0);
        oscMsg.values.Add(0);
        oscMsg.values.Add(0);

// Send the message to the client
        osc.Send(oscMsg);

}

By using the FFT magnitude as the force of deformation it is possible to get a visual feedback that matches the sound output, thus making the experience more coherant and immersive. Still, there are many features of music and visual interaction that have to be taken into account and that I'll save for future entries.

// 2018.05.15 Enhancing interaction I
So far I covered basic 3D audio and mesh deformation based on movement interaction. Nevertheless, in order to make this interaction more meaningful to the users, I added several different parameter controls based on the same kind of interaction. Thus, at the same time that users distort the environment and move the sound around, they can play with reverberation and buffer position and grain size of a granular synth or freeze the whole audio through an FFT module.
If you have been reading this blog, you might have noticed that I have barely mentioned the DSP modules I am using as sound sources. This is because at this moment they are being used as a placeholder. Once we have made a whole sense of our interaction system, we would like to use the capabilities of a VR workstation to develop heavy sound sources modules.
The following code example displays the DSP modules I am currently using.

// Samples for granular synthesis
~path1 = Platform.resourceDir +/+ "/sounds/a11wlk01.wav";
~sample1 = Buffer.read(s, ~path1, bufnum: 0);
~path2 = Platform.resourceDir +/+ "/sounds/a11wlk01-44_1.aiff";
~sample2 = Buffer.read(s, ~path2, bufnum: 1);

// Audio signal
SynthDef(\grain, {
	|outBus = 21, bufnum = 0, triggerRate = 3, rate = 1, bufferPos = 0, grainSize = 0.3, grainAmp = 0.1, amp = 1, lagTime = 0.1,  id =  1|

var trigger, centerPos, signal, output;

trigger = Impulse.ar(triggerRate);

centerPos = bufferPos * BufDur.kr(bufnum);

signal = TGrains.ar(1, trigger, bufnum, rate * BufRateScale.kr(bufnum), bufferPos  * BufFrames.kr(bufnum), grainSize, amp: Lag2.kr(grainAmp, lagTime));

output = signal * amp;
	output = Mix(output); // just to note that the FFT modules should get a mono signal

Out.ar(outBus, output);

// OSC
	SendTrig.ar(Impulse.ar(40), id, output);

}).send(s);

// FFT module
SynthDef(\freeze, {
	|inBus = 20, outBus = 30, freeze = 0, amp = 0.9|

var input, fft, freezer, output;

input = In.ar(inBus);

fft = FFT(LocalBuf(1024), input);

freezer = PV_Freeze(fft, freeze);

output = IFFT(freezer) * amp;

Out.ar(outBus, output);

}).add;

// Ambisonics module
SynthDef(\atk, {
	|inBus = 30, outBus = 0, angle = 0, azimuth = 0, elevation = 0, distance = 1, preAmp = 0.9, amp = 0.9, revTime = 2, dryLevel = 1, damping = 1, delTime = 0.01, lagTime = 0.1|

var input, encode, postEncode, transform, decode, output;

input = In.ar(inBus);

// filter for proximity
	input = HPF.ar(input, 1);

input = input * Lag2.kr(preAmp, lagTime);
	input = CompanderD.ar(input, 0.333);
	input = Limiter.ar(input, 0.93);

encode = FoaEncode.ar(input, ~encoder);

// apply effects
	postEncode = FoaDecode.ar(encode, FoaDecoderMatrix.newBtoA);
	postEncode = postEncode + DelayC.ar(postEncode, 0.2, delTime);
	encode =  FoaEncode.ar(postEncode, FoaEncoderMatrix.newAtoB);

transform = FoaTransform.ar(encode, 'proximity', distance);
	transform = FoaTransform.ar(transform, 'zoom', angle, azimuth, elevation);

// decode = FoaDecode.ar(transform, ~decoder, amp);
	decode = FoaDecode.ar(transform, ~decoder, Lag2.kr(amp, lagTime));

output = GVerb.ar(decode, 50, Lag2.kr(revTime, lagTime), Lag2.kr(damping, lagTime), Lag2.kr(dryLevel, lagTime));

output = Limiter.ar(output, 0.93);

output = Out.ar(outBus, decode);
}).add;

For each controller I created 3 synths, a grain synth, an FFT Freeze synth and an Ambisonics one. This way users could have indepent music control on each extremity. The granular synth is controlled in two different ways. One follows the RayCastLocation, so depending where users are pointing they can change the buffer position. In this OSC Definition I am also measuring how far is the controller from the user and how big is the velocity of this movements. This is used to control the amplitude and lag time of both the grain module and the Ambisonics module. The other mapping of the grain synth is based on the velocity of the FFT loudness measurement. The bigger the difference between the new loudness value and the old value the bigger the grain size and the trigger rate are.
Notice that in the following excerpt I have edit the OSCdef(\amp) just to focus on the loudness interaction and that I did not add the OSCdef(\rayCastLocationR) as it's the same as the left one, but changing the synths' ID's.

(
//---------- Variables ----------
var loudness1 = 0;
var oldLoudness1 = 0;
var loudnessVel1 = 0;

var loudness2 = 0;
var oldLoudness2 = 0;
var loudnessVel2 = 0;

var oldRadialL = 0;
var radialVelL = 0;
var radialVelAmpL = 0;

var oldRadialR = 0;
var radialVelR = 0;
var radialVelAmpR = 0;

//---------- Create synths ----------
s.sendMsg(\s_new, "grain", 1011, 1, 1, \outBus, 21); // controller 1
s.sendMsg(\s_new, "grain", 1012, 1, 1, \outBus, 22, \id, 2, \bufnum, 1); // controller 2

s.sendMsg(\s_new, "loudness", 1031, 1, 1, \inBus, 21); // controller 1
s.sendMsg(\s_new, "loudness", 1032, 1, 1, \inBus, 22, \outBus, 602, \id, 12); // controller 2

//---------- OSC Functions ----------
OSCdef(\amp, {
    |msg|

// OSC ID
    var id = msg[2];

if(id == 11){
        loudness1 = msg[3];

// Calculate velocity/range of change
        fork{
            oldLoudness1 = loudness1;
            0.001.wait;
        };
        loudnessVel1 = (loudness1 - oldLoudness1).abs;

// Apply it to the SynthDef
        s.sendMsg(\n_set, 1011, \grainSize, loudnessVel1.linlin(0, 1, 1, 0.1), \triggerRate, loudnessVel1.linlin(0, 1, 3, 6));
    };

if(id == 12){
        loudness2 = msg[3];

// Calculate velocity/range of change
        fork{
            oldLoudness2 = loudness2;
            0.001.wait;
        };
        loudnessVel2 = (loudness2 - oldLoudness2).abs;

// Apply it to the SynthDef
        s.sendMsg(\n_set, 1012, \grainSize, loudnessVel2.linlin(0, 1, 1, 0.1), \triggerRate, loudnessVel2.linlin(0, 1, 3, 6));
    };

},
'/tr',
s.addr
);

// Controller's position
OSCdef(\rayCastLocationL, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp, dryLevel, revTime, lagTime, bufferPos, grainAmp;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// left and right
	axisY = msg[1]; // X transform.position axis in unity
	// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity

// Sphereical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);
	amp = rayDistance.linexp(0, 50, 0.5, 0.25);

// Reverb
	dryLevel = rayDistance.linexp(5, 50, 1.00, 0.001);
	revTime = radialVelAmpL.expexp(0.0, 1.0, 2.0, 10);

// Grain
	bufferPos = radial.linlin(0, pi / 2, 0, 1);
	grainAmp = rayDistance.linexp(0, 50, 0.8, 0.1);

// Lag
	lagTime = radialVelAmpL.expexp(0.0001, 1.0, 0.1, 10.0);

// Radial Velocity
	fork{
		oldRadialL = radial;
		0.001.wait;
	};

radialVelL = (radial - oldRadialL).abs;
	radialVelAmpL = radialVelL.linexp(0.0, 0.0005, 0.0001, 1.0);
	radialVelL = radialVelL.linlin(0.0, 0.1, 0.0, 2.0);

// Set SynthDefs
	s.sendMsg(\n_set, 1041, \angle, radial, \azimuth, azimuth, \elevation,  polar, \distance, distance, \revTime, revTime,  \dryLevel, dryLevel, \lagTime, lagTime, \preAmp, amp * radialVelAmpL);
	s.sendMsg(\n_set, 1011, \bufferPos, bufferPos, \lagTime, lagTime, \grainAmp, grainAmp);

},
'/rayCastLocationL', // message address
nil,
6161 // message port
);

);

The FFT Freeze module required some mixing so even though it is activated just by pressing the grip of the Vive Controllers and sending a value of 1 or 0 to SuperCollider, it does affect the other two modules amplitude. When it is active, the amplitude of the FFT module rises as well as the one from the Ambisonics module, and it decreases when it isn't active. Furthermore, because the user is not only able to freeze the sound, but also the 3D meshes, it's useful to keep track of what mesh is being deformed so we can keep an audio loop in it. The freezing of the 3D meshes is simply done by turning the spring force to 0.
The following excerpt displays how this freezing happens inside unity. Once more, the same script should be adapted to the second controller.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class MeshDeformerControllerInputL : MonoBehaviour {
    //---------------- Variables ----------------
    // Refer the OSC.cs file
    public OSC osc;

// Set Vive controllers
    // reference to the tracked obj aka controller
    private SteamVR_TrackedObject trackedObj;
    private SteamVR_Controller.Device Controller
    {
        get { return SteamVR_Controller.Input((int)trackedObj.index); }
    }

// Physics
    public float force;
    public float forceOffset;

// Access other game objects
    public GameObject floor;
    public GameObject ceiling;
    public GameObject wall1;
    public GameObject wall2;
    public GameObject wall3;
    public GameObject wall4;

public GameObject theOtherController;

public bool pressingGrip = false;
    public bool freezingFloor = false;
    public bool freezingCeiling = false;
    public bool freezingWall1 = false;
    public bool freezingWall2 = false;
    public bool freezingWall3 = false;
    public bool freezingWall4 = false;

// Freeze OSC var
    float freeze = 0;

//---------------- Functions ----------------
    // Evaluates before start
    void Awake()
    {
        trackedObj = GetComponent<SteamVR_TrackedObject>();
    }

// Use this for initialization
    void Start () {
        // Declare OSC Address
        osc.SetAddressHandler("/fft1", OnReceiveFFT);
    }

void OnReceiveFFT(OscMessage message)
    {
        force = message.GetFloat(0);
    }

// Update is called once per frame
    void Update () {
        HandleInput();
    }

void HandleInput()
    {
        // OSC
        OscMessage oscMsg = new OscMessage();
        OscMessage oscMsgCollider = new OscMessage();
        OscMessage oscMsgFreezer = new OscMessage();

// set osc address
        oscMsg.address = "/rayCastLocationL";
        oscMsgCollider.address = "/leftCollider";
        oscMsgFreezer.address = "/leftFreezer";

// use physics engine to cast the ray and check if it has hit something
        RaycastHit hit;
        Vector3 point;
        if (Physics.Raycast(trackedObj.transform.position, trackedObj.transform.forward, out hit))
        {
            MeshDeformer deformer = hit.collider.GetComponent<MeshDeformer>();
            Renderer renderer = hit.collider.GetComponent<Renderer>();

if (Controller.GetHairTrigger())
            {
                point = new Vector3(Random.Range(0f, 50f), Random.Range(0f, 25f), Random.Range(-50f, 0f));
            }
            else
            {
                point = hit.point;
            }

if (deformer)
            {
                //Vector3 point = hit.point;
                // add offsett so the vertices are always pushed into the surface
                point += hit.normal * forceOffset;

// Deform the mesh
                if (hit.collider.gameObject == polygon1)
                {
                    deformer.AddDeformingForce(point, forcePolygon1);
                }

if (hit.collider.gameObject == polygon2)
                {
                    deformer.AddDeformingForce(point, forcePolygon2);
                }

if (hit.collider.gameObject == polygon3)
                {
                    deformer.AddDeformingForce(point, forcePolygon3);
                }

if (hit.collider.gameObject == polygon4)
                {
                    deformer.AddDeformingForce(point, forcePolygon4);
                }

if (hit.collider.gameObject == floor || hit.collider.gameObject == ceiling || hit.collider.gameObject == wall1 || hit.collider.gameObject == wall2 || hit.collider.gameObject == wall3 || hit.collider.gameObject == wall4)
                {
                    deformer.AddDeformingForce(point, force);
                }

}

// Freeze meshes with grip press
            if (Controller.GetPress(SteamVR_Controller.ButtonMask.Grip))
            {
                pressingGrip = true;
            }
            else
            {
                pressingGrip = false;
            }

if (Controller.GetPress(SteamVR_Controller.ButtonMask.Grip))
            {
                if (hit.collider.gameObject == floor)
                {
                    floor.GetComponent<MeshDeformer>().springForce = 0;

freezingFloor = true;
                }
                else
                {
                    freezingFloor = false;
                }

if (hit.collider.gameObject == ceiling)
                {
                    ceiling.GetComponent<MeshDeformer>().springForce = 0;

freezingCeiling = true;
                }
                else
                {
                    freezingCeiling = false;
                }

if (hit.collider.gameObject == wall1)
                {
                    wall1.GetComponent<MeshDeformer>().springForce = 0;

freezingWall1 = true;
                }
                else
                {
                    freezingWall1 = false;
                }

if (hit.collider.gameObject == wall2)
                {
                    wall2.GetComponent<MeshDeformer>().springForce = 0;

freezingWall2 = true;
                }
                else
                {
                    freezingWall2 = false;
                }

if (hit.collider.gameObject == wall3)
                {
                    wall3.GetComponent<MeshDeformer>().springForce = 0;

freezingWall3 = true;
                }
                else
                {
                    freezingWall3 = false;
                }

if (hit.collider.gameObject == wall4)
                {
                    wall4.GetComponent<MeshDeformer>().springForce = 0;

freezingWall4 = true;
                }
                else
                {
                    freezingWall4 = false;
                }

freeze = 1;
            }
            else
            {
                if (theOtherController.GetComponent<MeshDeformerControllerInputR>().freezingFloor == false)
                {

floor.GetComponent<MeshDeformer>().springForce = 20;
                }

if (theOtherController.GetComponent<MeshDeformerControllerInputR>().freezingCeiling == false)
                {
                    ceiling.GetComponent<MeshDeformer>().springForce = 20;
                }

if (theOtherController.GetComponent<MeshDeformerControllerInputR>().freezingWall1 == false)
                {
                    wall1.GetComponent<MeshDeformer>().springForce = 20;
                }

if (theOtherController.GetComponent<MeshDeformerControllerInputR>().freezingWall2 == false)
                {
                    wall2.GetComponent<MeshDeformer>().springForce = 20;
                }

if (theOtherController.GetComponent<MeshDeformerControllerInputR>().freezingWall3 == false)
                {
                    wall3.GetComponent<MeshDeformer>().springForce = 20;
                }

if (theOtherController.GetComponent<MeshDeformerControllerInputR>().freezingWall4 == false)
                {
                    wall4.GetComponent<MeshDeformer>().springForce = 20;
                }

freeze = 0;
            }

if (hit.collider.tag == "wall")
            {
                oscMsgCollider.values.Add(0);
            }

}

// add the elements to the OSC array
            oscMsg.values.Add(point.x);
            oscMsg.values.Add(point.y);
            oscMsg.values.Add(point.z);
            oscMsg.values.Add(hit.distance);
            oscMsgFreezer.values.Add(freeze);

}

// add 0 to the OSC array
        // so supercollider doesn't get confused
        oscMsg.values.Add(0);
        oscMsg.values.Add(0);
        oscMsg.values.Add(0);
        oscMsg.values.Add(0);

// Send the message to the client
        osc.Send(oscMsg);
        osc.Send(oscMsgCollider);
        osc.Send(oscMsgFreezer);

}
}

With the values send by Unity we can keep track in SuperCollider of the meshes being frozen and by which controller. As mentioned, this is useful so we can resolve which and when a frozen should be frozen. In order to keep it clear to myself, I wrote some boolean conditions. Probably it will same not really necessary when working with just two controllers, but in future entries this logic will make more sense.

(
//---------- Variables ----------
var loudness1 = 0;
var oldLoudness1 = 0;
var loudnessVel1 = 0;

var loudness2 = 0;
var oldLoudness2 = 0;
var loudnessVel2 = 0;

var oldRadialL = 0;
var radialVelL = 0;
var radialVelAmpL = 0;

var oldRadialR = 0;
var radialVelR = 0;
var radialVelAmpR = 0;

var leftColliderWall = true;
var rightColliderWall = true;

var freezerLeft = 0;
var freezerRight = 0;

var masterAmpFrozen = 0.333;
var masterAmpUnFrozen = 0.7;

//---------- Create synths ----------
s.sendMsg(\s_new, "grain", 1011, 1, 1, \outBus, 21); // controller 1
s.sendMsg(\s_new, "grain", 1012, 1, 1, \outBus, 22, \id, 2, \bufnum, 1); // controller 2

s.sendMsg(\s_new, "loudness", 1031, 1, 1, \inBus, 21); // controller 1
s.sendMsg(\s_new, "loudness", 1032, 1, 1, \inBus, 22, \outBus, 602, \id, 12); // controller 2

s.sendMsg(\s_new, "freeze", 1061, 1, 1, \inBus, 21, \outBus, 31); // controller 1
s.sendMsg(\s_new, "freeze", 1062, 1, 1, \inBus, 22, \outBus, 32); // controller 2

s.sendMsg(\s_new, "atk", 1041, 1, 1, \inBus, 31); // controller 1
s.sendMsg(\s_new, "atk", 1042, 1, 1, \inBus, 32); // controller 2

//---------- OSC Functions ----------
// RayCast Collider
OSCdef(\leftCollider, {
	|msg|

var colliderType = msg[1];

if(colliderType == 0){
		leftColliderWall = true;
	} // else
	{
		leftColliderWall = false;
	};

},
'/leftCollider', // message address
nil,
6161 // message port
);

OSCdef(\rightCollider, {
	|msg|

var colliderType = msg[1];

if(colliderType == 0){
		rightColliderWall = true;
	} // else
	{
		rightColliderWall = false;
	};

},
'/rightCollider', // message address
nil,
6161 // message port
);

// rayCast Collider Freezer
OSCdef(\leftFreezer, {
	|msg|

var freeze = msg[1];

freezerLeft = freeze;

if(leftColliderWall == true){
		s.sendMsg(\n_set, 1061, \freeze, freeze);

if(freeze == 1){
			// Because it clips when both controllers are freezing
			if((freezerLeft == 1) && (freezerRight == 1)){
				s.sendMsg(\n_set, 1061, \amp, 0.7);
			} // else
			{
				s.sendMsg(\n_set, 1061, \amp, 1);
			};
		} // else
		{
			s.sendMsg(\n_set, 1061, \amp, 0.7);
		};
	};
},
'/leftFreezer', // message address
nil,
6161 // message portt5
);

OSCdef(\rightFreezer, {
	|msg|

var freeze = msg[1];

freezerRight = freeze;

if(rightColliderWall == true){
		s.sendMsg(\n_set, 1062, \freeze, freeze);

if(freeze == 1){
			// Because it clips when both controllers are freezing
			if((freezerLeft == 1) && (freezerRight == 1)){
				s.sendMsg(\n_set, 1062, \amp, 0.7);
			} // else
			{
				s.sendMsg(\n_set, 1062, \amp, 1);
			};
		} // else
		{
			s.sendMsg(\n_set, 1062, \amp, 0.7);
		};
	};
},
'/rightFreezer', // message address
nil,
6161 // message port
);

// Controller's position
OSCdef(\rayCastLocationL, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp, dryLevel, revTime, lagTime, bufferPos, grainAmp;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// left and right
	axisY = msg[1]; // X transform.position axis in unity
	// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity

// Sphereical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Reverb
	dryLevel = rayDistance.linexp(5, 50, 1.00, 0.001);
	revTime = radialVelAmpL.expexp(0.0, 1.0, 2.0, 10);

// Grain
	bufferPos = radial.linlin(0, pi / 2, 0, 1);
	grainAmp = rayDistance.linexp(0, 50, 0.8, 0.1);

// Lag
	lagTime = radialVelAmpL.expexp(0.0001, 1.0, 0.1, 10.0);

// Radial Velocity
	fork{
		oldRadialL = radial;
		0.001.wait;
	};

radialVelL = (radial - oldRadialL).abs;
	radialVelAmpL = radialVelL.linexp(0.0, 0.0005, 0.0001, 1.0);
	radialVelL = radialVelL.linlin(0.0, 0.1, 0.0, 2.0);

// Set SynthDefs
	s.sendMsg(\n_set, 1041, \angle, radial, \azimuth, azimuth, \elevation,  polar, \distance, distance, \revTime, revTime,  \dryLevel, dryLevel, \lagTime, lagTime);
	s.sendMsg(\n_set, 1011, \bufferPos, bufferPos, \lagTime, lagTime, \grainAmp, grainAmp);

if((freezerLeft == 0) || (freezerRight == 0)){
		amp = rayDistance.linexp(0, 50, 0.5, 0.25);
		s.sendMsg(\n_set, 1041, \preAmp, amp * radialVelAmpL);
	} // else
	{
		amp = rayDistance.linexp(0, 50, 0.9, 0.45);
		s.sendMsg(\n_set, 1041, \preAmp, amp);
	};
},
'/rayCastLocationL', // message address
nil,
6161 // message port
);

OSCdef(\rayCastLocationR, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp, dryLevel, revTime, lagTime, bufferPos, grainAmp;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// left and right
	axisY = msg[1]; // X transform.position axis in unity
	// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity

// Sphereical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-25.0, 25.0, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Reverb
	dryLevel = rayDistance.linexp(5, 50, 1.00, 0.001);
	revTime = radialVelAmpR.expexp(0.0, 1.0, 2.0, 10);

// Grain
	bufferPos = radial.linlin(0, pi / 2, 0, 1);
	grainAmp = rayDistance.linexp(0, 50, 0.8, 0.1);

// Lag
	lagTime = radialVelAmpR.expexp(0.0001, 1.0, 0.1, 10.0);

// Radial Velocity
	fork{
		oldRadialR = radial;
		0.001.wait;
	};

radialVelR = (radial - oldRadialR).abs;
	radialVelAmpR = radialVelR.linexp(0.0, 0.0005, 0.0001, 1.0);
	radialVelR = radialVelR.linlin(0.0, 0.1, 0.0, 2.0);

// Set SynthDefs
	s.sendMsg(\n_set, 1042, \angle, radial, \azimuth, azimuth, \elevation,  polar, \distance, distance, \revTime, revTime,  \dryLevel, dryLevel, \lagTime, lagTime);
	s.sendMsg(\n_set, 1012, \bufferPos, bufferPos, \lagTime, lagTime, \grainAmp, grainAmp);

if((freezerLeft == 0) || (freezerRight == 0)){
		amp = rayDistance.linexp(0, 50, 0.5, 0.25);
		s.sendMsg(\n_set, 1042, \preAmp, amp * radialVelAmpL);
	} // else
	{
		amp = rayDistance.linexp(0, 50, 0.9, 0.45);
		s.sendMsg(\n_set, 1042, \preAmp, amp);
	};
},
'/rayCastLocationR', // message address
nil,
6161 // message port
);

);

All these small additions to direct interaction between the user and the environment make the whole experience more immersive. Giving users control over parameters that they would expect to change, but that does not require an extra effort helps bringing a more intuitive and effective interaction.
In the following entry I will cover how interaction can be enhanced further by giving the environment some sort of autonomy.

// 2018.06.02 Enhancing interaction II
Giving the system some autonomy so it acts on its own can bring an upper level of connection with the user. One can set the system to act or react in different ways in order to estimulate user's activity by messuring the activity of the user. For example, when user activity diminishes the environment can propose new musical gestures. On the contrary, when the user seems engaged enough the environment can potentiate this engagement by adding new layers of sound or visual interaction.
So far in VIVR I implemented this two types of system autonomy. In this entry, I'll explain how the system reacts when it feels that the user is not engaged anymore and in the next one, I'll describe my approach to keep a higher level of engagement. In both cases it's necessary to give the environment its own synths and 3d objects.
I'll start with the SuperCollider synth. In this case, the enviornment holds a similar version of the granular synth, but with amplitude modulation because other LFO's are what are gonna give it movement/life. Nevertheless, this synth would not go through the FFT freezing module.

// Samples for granular synthesis
~path3 = PathName(thisProcess.nowExecutingPath).parentPath ++ "wavxtsy.wav";
~sample3 = Buffer.readChannel(s, ~path3, channels: 0, bufnum: 2);

// Audio signal
SynthDef(\grainEnvironment, {
    |outBus = 30, inBusKr = 650, bufnum = 0, triggerRate = 3, rate = 1, bufferPos = 0, grainSize = 0.3, grainAmp = 0.1, amp = 1, id =  1|

var trigger, centerPos, signal, output, ampMod;

trigger = Impulse.ar(triggerRate);

centerPos = bufferPos * BufDur.kr(bufnum);

signal = TGrains.ar(1, trigger, bufnum, rate * BufRateScale.kr(bufnum), Lag2.kr(centerPos, 0.1), grainSize, amp: grainAmp);

ampMod = In.kr(inBusKr);

output = signal * ampMod * amp;
    output = Mix(output);

Out.ar(outBus, output);
	// OSC
    SendTrig.ar(Impulse.ar(~oSCRate), id, output);
}).send(s);

SynthDef(\environmentAmpMod1, {
    |outBus = 650, rate1 = 1, rate2 = 1, rate3 = 1, amp = 1, id1 = 1, id2 = 1, id3 = 1|

var signal1, signal2, signal3;

signal1 = SinOsc.kr(rate1, 0, amp);
    signal2 = LFNoise2.kr(rate2, amp);
    signal3 = LFNoise2.kr(rate3, amp);

Out.kr(outBus, signal1);

// OSC
    SendTrig.ar(Impulse.ar(~oSCRate), id1, signal1);
    SendTrig.ar(Impulse.ar(~oSCRate), id2, signal2);
    SendTrig.ar(Impulse.ar(~oSCRate), id3, signal3);

}).add;

One can determine how active the user is at the moment by measuring the average rate of activity of both controllers. If the user is not active for a period of time that seems too long to be considered a musical silence then the environment will start acting and thus inducing the user to restart the musical activity.
In the following excerpt I calculate this controllers' magnitude average. If it is too low, then it starts a counter, if this counter gets to an X amount of time, the enviornment starts acting on its own.
Finally, in the same way that I did with the controllers, I send the total FFT magnitude values to Unity.

(
//---------- Variables ----------
var fftArray1 = Array.fill(~bufSize, 0);
var fftArray2 = Array.fill(~bufSize, 0);
var fftArray3 = Array.fill(~bufSize, 0);
var fftValues1 = 0;
var fftValues2 = 0;
var fftValues3 = 0;

var forceIndex = 200000;

var client = NetAddr.new("127.0.0.1", 12345); // for OSC

var environmentCameraCounter = 0;
var environmentCameraCounterRoutine;
var environmentCameraCounterPlays = false;
var environmentCameraCounterResets = false;
var environmentCameraPlays = false;
var environmentCameraResets = false;

var amp1 = 0;
var loudness1 = 0;
var oldLoudness1 = 0;
var loudnessVel1 = 0;

var amp2 = 0;
var loudness2 = 0;
var oldLoudness2 = 0;
var loudnessVel2 = 0;

var amp3 = 0;
var loudness3 = 0;
var oldLoudness3 = 0;
var loudnessVel3 = 0;

var ambAmp1 = 0;
var ambAmp2 = 0;
var ambAmp3 = 0;

var posX;
var posY;
var posZ;
var rotX;
var rotY;
var rotZ;

var polygonRotX;
var polygonRotY;
var polygonRotZ;

var oldRadialL = 0;
var radialVelL = 0;
var radialVelAmpL = 0;

var oldRadialR = 0;
var radialVelR = 0;
var radialVelAmpR = 0;

var radialVelAvg = 0;

var freezerLeft = 0;
var freezerRight = 0;

var freezerCompThresh = 0.9;

var masterAmpFrozen = 0.333;
var masterAmpUnFrozen = 0.7;

var tempoRoutines = TempoClock(1);

//---------- Create synths ----------
s.sendMsg(\s_new, "grain", 1011, 1, 1, \outBus, 21); // controller 1
s.sendMsg(\s_new, "grain", 1012, 1, 1, \outBus, 22, \id, 2, \bufnum, 1); // controller 2

s.sendMsg(\s_new, "grainEnvironment", 1013, 1, 1, \outBus, 23, \id, 3, \bufnum, 2, \inBusKr, 650); // environment camera

s.sendMsg(\s_new, "fftTracker", 1021, 1, 1, \inBus, 21, \outBus, 0); // controller 1
s.sendMsg(\s_new, "fftTracker", 1022, 1, 1, \inBus, 22, \outBus, 257); // controller 2
s.sendMsg(\s_new, "fftTracker", 1023, 1, 1, \inBus, 33, \outBus, 514);  // environment camera

s.sendMsg(\s_new, "loudness", 1031, 1, 1, \inBus, 21); // controller 1
s.sendMsg(\s_new, "loudness", 1032, 1, 1, \inBus, 22, \outBus, 602, \id, 12); // controller 2
s.sendMsg(\s_new, "loudness", 1033, 1, 1, \inBus, 23, \outBus, 603, \id, 13); // environment camera

s.sendMsg(\s_new, "freeze", 1061, 1, 1, \inBus, 21, \outBus, 31); // controller 1
s.sendMsg(\s_new, "freeze", 1062, 1, 1, \inBus, 22, \outBus, 32); // controller 2

s.sendMsg(\s_new, "atk", 1041, 1, 1, \inBus, 41); // controller 1
s.sendMsg(\s_new, "atk", 1042, 1, 1, \inBus, 42); // controller 2
s.sendMsg(\s_new, "atk", 1043, 1, 1, \inBus, 23); // environment camera

s.sendMsg(\s_new, "environmentAmpMod1", 1053, 1, 1, \outBus, 650, \rate1, 0.1, \rate2, 0.1, \rate3, 0.1, \id1, 23, \id2, 24, \id3, 25);

//---------- OSC Functions ----------
//  Environment's actions
OSCdef(\environmentReacts, {
	var ampAvg;

ampAvg = (ambAmp1 * ambAmp2) / 2;

// Environment Camera Counter
	if(ampAvg > 0.00009){
		if(environmentCameraCounterResets == false){
			environmentCameraCounter = 0;  // just to make sure

environmentCameraCounterPlays = false;
			environmentCameraCounterResets = true;

environmentCameraCounterRoutine.stop;
		};

} // else
	{
		if(environmentCameraCounterPlays == false){
			environmentCameraCounter = 0; // just to make sure
			polygonsCounter = 0;

environmentCameraCounterPlays = true;
			environmentCameraCounterResets = false;

environmentCameraCounterRoutine = Routine({
				loop{
					environmentCameraCounter = environmentCameraCounter + 1;
					"Environment camera counter: ".post;
					environmentCameraCounter.postln;
					1.wait;
				};
			}).play(tempoRoutines);

};

// Events on the environment's camera synth
	if(environmentCameraCounter >= 10){
		if(environmentCameraPlays == false){

s.sendMsg(\n_set, 1013, \amp, 1);
			s.sendMsg(\n_set, 1053, \amp, 1);
			"activating enviornment camera".postln;

environmentCameraPlays = true;
			environmentCameraResets = false;

};
	} // else
	{
		if(environmentCameraResets == false){

s.sendMsg(\n_set, 1053, \amp, 0);
			"deactivating environment camera".postln;

environmentCameraPlays = false;
			environmentCameraResets = true;
		};

};

},
'/tr',
s.addr
);

// FFT calculation and Unity communication
OSCdef(\amp, {
	|msg|

var id = msg[2];

var size = ~bufSize.div(2);

if(id == 1){
		amp1 = msg[3];

};

if(id == 2){
		amp2 = msg[3];

};

if(id == 3){
		amp3 = msg[3];

};

if(id == 4){
		amp4 = msg[3];

};

if(id == 5){
		amp5 = msg[3];

};

if(id == 6){
		amp6 = msg[3];

};

if(id == 7){
		amp7 = msg[3];

};

if(id == 11){
		loudness1 = msg[3];

fork{
			oldLoudness1 = loudness1;
			0.001.wait;
		};
		loudnessVel1 = (loudness1 - oldLoudness1).abs;

};

if(id == 12){
		loudness2 = msg[3];

fork{
			oldLoudness2 = loudness2;
			0.001.wait;
		};
		loudnessVel2 = (loudness2 - oldLoudness2).abs;

};

if(id == 13){
		loudness3 = msg[3];

fork{
			oldLoudness3 = loudness3;
			0.001.wait;
		};
		loudnessVel3 = (loudness3 - oldLoudness3).abs;

};

if(id == 23){
		var minIn = -1.0;
		var maxIn = 1.0;

var modAmp = msg[3];

posX = modAmp.linlin(minIn, maxIn, 0.0, 50.0);
		rotX = modAmp.linlin(minIn, maxIn, 0.0, 360.0);
	};

if(id == 24){
		var minIn = -1.0;
		var maxIn = 1.0;

var modAmp = msg[3];

posY = modAmp.linlin(minIn, maxIn, 0.0, 50.0);
		rotY = modAmp.linlin(minIn, maxIn, 0.0, 360.0);
	};

if(id == 25){
		var minIn = -1.0;
		var maxIn = 1.0;

var modAmp = msg[3];

posZ = modAmp.linlin(minIn, maxIn, -50.0, 0.0);
		rotZ = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

radialVelAvg = (radialVelL * radialVelR) / 2;

// Get the values of each channels and put it into index 1 of the array
	~fftValuesBuf1.getnSynchronous(~bufSize.div(2)).do{
		|val, i|

fftArray1.put([size-i, size+i], val);
		fftValues1 = fftArray1[1];
	};

~fftValuesBuf2.getnSynchronous(~bufSize.div(2)).do{
		|val, i|

fftArray2.put([size-i, size+i], val);
		fftValues2 = fftArray2[1];
	};

~fftValuesBuf3.getnSynchronous(~bufSize.div(2)).do{
		|val, i|

fftArray3.put([size-i, size+i], val);
		fftValues3 = fftArray3[1];
	};

~totalFFT1 = fftValues1 * amp1 * loudness1 * forceIndex * radialVelAvg;

~totalFFT2 = fftValues2 * amp2 * loudness2 * forceIndex * 50 * radialVelAvg;

~totalFFT3 = fftValues3 * amp3 * loudness3 * forceIndex * 100 * radialVelAvg;

// send message to client
	client.sendMsg("/fft1", ~totalFFT1.abs, amp1.range(-0.1, 0.1), loudness1, ~totalFFT4.abs, ~totalFFT5.abs, ~totalFFT6.abs, ~totalFFT7.abs);
	client.sendMsg("/fft2", ~totalFFT2.abs, amp2.range(-0.1, 0.1), loudness2,~totalFFT4.abs, ~totalFFT5.abs, ~totalFFT6.abs, ~totalFFT7.abs);
	client.sendMsg("/fft3", ~totalFFT3.abs, amp3.range(-0.1, 0.1), loudness3);
	client.sendMsg("/environmentPosRot", posX, posY, posZ, rotX, rotY, rotZ);

},
'/tr',
s.addr
);

// Controller's position
OSCdef(\rayCastLocationL, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp, dryAmp, revTime, dryLevel, lagTime, triggerRate, grainSize, grainAmp, bufferPos, distIndex;

// post incoming msg
	// msg.postln;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// axisX.postln;

// left and right
	axisY = msg[1]; // X transform.position axis in unity

// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity
	// rayDistance.postln;

// Spherical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians
	// "radial: ".post;
	// radial.postln;

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;
	// "azimuth: ".post;
	// azimuth.postln;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-28.92, 28.92, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;
	// "polar: ".post;
	// polar.postln;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Radial Velocity
	fork{
		oldRadialL = radial;
		0.001.wait;
	};

radialVelL = (radial - oldRadialL).abs;
	radialVelAmpL = radialVelL.linexp(0.0, 0.0005, 0.0001, 1.0);
	radialVelL = radialVelL.linlin(0.0, 0.1, 0.0, 2.0);

// Ambisonic Amplitude for Avg
	ambAmp1 = rayDistance.linexp(0, 50, 1.0, 0.0001);
	ambAmp1  = ambAmp1 * radialVelAmpL;

// Synths' parameters
	dryAmp = radialVelAmpL;
	revTime = radialVelAmpL.expexp(0.0001, 1.0, 2.0, 20);
	dryLevel = rayDistance.linexp(5, 50, 1.00, 0.001);
	lagTime = radialVelAmpL.expexp(0.0001, 1.0, 0.1, 1.0);
	triggerRate = radialVelAmpL.expexp(0.0001, 1.0, 6.0, 1.0);
	grainSize = radialVelAmpL.expexp(0.0001, 1.0, 0.1, 1.0);
	grainAmp = rayDistance.linexp(0, 50, 0.8, 0.1);
	bufferPos = radial.linlin(0, pi / 2, 0, 1);
	distIndex = radialVelAmpL.expexp(0.0001, 1.0, 1.0, 10.0);

// Set SynthDefs
	s.sendMsg(\n_set, 1041, \angle, radial, \azimuth, azimuth, \elevation,  polar, \dryAmp, dryAmp, \revTime, revTime,  \dryLevel, dryLevel, \lagTime, lagTime);
	s.sendMsg(\n_set, 1011, \triggerRate, triggerRate, \grainSize, grainSize, \grainAmp, grainAmp, \bufferPos, bufferPos, \lagTime, lagTime);
	s.sendMsg(\n_set, 1071, \distIndex, distIndex);

if((freezerLeft == 0) || (freezerRight == 0)){
		amp = rayDistance.linexp(0, 50, 0.5, 0.25);

s.sendMsg(\n_set, 1041, \preAmp, amp * radialVelAmpL);
	} // else
	{
		amp = rayDistance.linexp(0, 50, 0.9, 0.45);

s.sendMsg(\n_set, 1041, \preAmp, amp);
	};
},
'/rayCastLocationL', // message address
nil,
6161 // message port
);

OSCdef(\rayCastLocationR, {
	|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, amp, dryAmp, revTime, dryLevel, lagTime, triggerRate, grainSize, grainAmp, bufferPos, distIndex;

// post incoming msg
	// msg.postln;

// From cartesian coordinates to spherical
	// front and rear
	axisX = msg[3]; // Z transform.position axis in Unity
	// axisX.postln;

// left and right
	axisY = msg[1]; // X transform.position axis in unity

// top to bottom
	axisZ = msg[2]; // Y transform.position axis in unity

// distance
	rayDistance = msg[4]; // RayCastHit hit.distance in Unity
	// rayDistance.postln;

// Spherical coordinates formulas
	// radial sqrt(x^2 + y^2 + z^2)
	radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
	// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
	// Therefore my maximum value is 56.789083458003
	radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians
	// "radial: ".post;
	// radial.postln;

// azimuth atan2(y, x)
	azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
	azimuth = azimuth - ~hmdYaw;
	// "azimuth: ".post;
	// azimuth.postln;

// polar cos^(-1)(z/r)
	polar = axisZ.linlin(-28.92, 28.92, -pi / 2, pi / 2); // scale to radians
	polar = polar - ~hmdPitch;
	// "polar: ".post;
	// polar.postln;

// Distance
	distance = rayDistance.linlin(0, 50, 0.1, 5);

// Radial Velocity
	fork{
		oldRadialR = radial;
		0.001.wait;
	};

radialVelR = (radial - oldRadialR).abs;
	radialVelAmpR = radialVelR.linexp(0.0, 0.0005, 0.0001, 1.0);
	radialVelR = radialVelR.linlin(0.0, 0.1, 0.0, 2.0);

// Ambisonic Amplitude for Avg
	ambAmp2 = rayDistance.linexp(0, 50, 1.0, 0.0001);
	ambAmp2  = ambAmp2 * radialVelAmpR;

// Synths' parameters
	dryAmp = radialVelAmpR;
	revTime = radialVelAmpR.expexp(0.0001, 1.0, 2.0, 20);
	dryLevel = rayDistance.linexp(5, 50, 1.00, 0.001);
	lagTime = radialVelAmpR.expexp(0.0001, 1.0, 0.1, 1.0);
	triggerRate = radialVelAmpR.expexp(0.0001, 1.0, 6.0, 1.0);
	grainSize = radialVelAmpR.expexp(0.0001, 1.0, 0.1, 1.0);
	grainAmp = rayDistance.linexp(0, 50, 0.8, 0.1);
	bufferPos = radial.linlin(0, pi / 2, 0, 1);
	distIndex = radialVelAmpR.expexp(0.0001, 1.0, 1.0, 10.0);

// Set SynthDefs
	s.sendMsg(\n_set, 1042, \angle, radial, \azimuth, azimuth, \elevation,  polar, \dryAmp, dryAmp, \revTime, revTime,  \dryLevel, dryLevel, \lagTime, lagTime);
	s.sendMsg(\n_set, 1012, \triggerRate, triggerRate, \grainSize, grainSize, \grainAmp, grainAmp, \bufferPos, bufferPos, \lagTime, lagTime);
	s.sendMsg(\n_set, 1072,  \distIndex, distIndex);

if((freezerLeft == 0) || (freezerRight == 0)){
		amp = rayDistance.linexp(0, 50, 0.5, 0.25);

s.sendMsg(\n_set, 1042, \preAmp, amp * radialVelAmpR);
	} // else
	{
		amp = rayDistance.linexp(0, 50, 0.9, 0.45);

s.sendMsg(\n_set, 1042, \preAmp, amp);
	};
},
'/rayCastLocationR', // message address
nil,
6161 // message port
);
);

In Unity I created a new 3d Capsule, but without a Mesh Renderer so it becomes invisible. This object holds two scripts, a movement script and a Mesh Deformer script. Thus, the object can be active all the time and it won't be until it starts moving that it will become noticeable. The Mesh Deformer script is the same as the one used for the controllers, but chaninging the osc address. The EnviornmentCamMovement.cs is rather simple. I get the scaled amplitude values from the environmentAmpMod1 synth and set them as axis of position and rotation.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class EnvironmentCamMovement : MonoBehaviour
{
    //---------- Variables ----------
    public OSC osc;

float posX;
    float posY;
    float posZ;

float rotX;
    float rotY;
    float rotZ;

//---------- Functions ----------
    // Use this for initialization
    void Start()
    {
        // Set OSC addresses
        osc.SetAddressHandler("/environmentPosRot", OnReceiveRotPos);
    }

// OSC Function
    void OnReceiveRotPos(OscMessage message){
        posX = message.GetFloat(0);
        posY = message.GetFloat(1);
        posZ = message.GetFloat(2);

rotX = message.GetFloat(3);
        rotY = message.GetFloat(4);
        rotZ = message.GetFloat(5);
    }

// This function is called every fixed framerate frame, if the MonoBehaviour is enabled.
    void FixedUpdate()
    {
        transform.position = new Vector3(posX, posY, posZ);
        transform.localEulerAngles = new Vector3(rotX, rotY, rotZ);
    }

// Update is called once per frame
    void Update()
    {

}
}

In this case, when the environment feels that the user has lost engagement it starts travelling deforming itself in a slow manner, thus the LFO's. The amplitude modulator that goes to the granular module of the environment is a Sine Oscillator, so it fades in and out in a way that makes it easy to notice. I consider this sonic gesture to induce new slow gestures from the user that will give place to a new musical part.
In the following entry I'll describe how I set new levels of interaction when the user's level of activity is high.

// 2018.07.09 Enhancing interaction III
Lately, I've been working and revising this project and I realized of some events and functions that would have worked better in a different way. This entry explains how VIVR's third level of interaction works and updates some functions that have been presented before.
Up until now, the sonic landscape of VIVR holds just two synths the user can interact with + the environment's one. This can be enough if one is just looking for an express experience users can interact with. Nevertheless, when one is looking for user's engagement, providing an accomplishment for user's effort that would extend the experience that has happened so far turns out to be effective.
In VIVR's case, I measured the level of activity of the user based on how long the users keeps a high level of acceleration in their movements. When a high level of acceleration is kept during an X amount of time, a series of 4 new polygons appear inside the environment. These polygons hold their own instance of the granular synth, with a different sample each. However, the mapping of the sound parameters of these synths is different from the controllers. Instead of reacting to Raycast hit location of the controllers they react to users position in the space. An arppegiator is set to their trigger rate and pitch scale so they can provide the user with a different form of musical interaction.
In addition, two new sub levels of interaction are attached to the user's acceleration constant. When under this parent level the user performs another constant time of high acceleration, the polygons will start rotating around the user. Moreover, when this constant rate of acceleration is bigger than X value, the polygons would not only rotate in one dimension but in 3 dimensions. Taking into consideration that the instrument uses 3D audio, these rotations not only provide a new sonic landscape because of the interaction with user's position. They also bring new transformations to the 3D sonic environment.
The following excerpt displays an updated version of the granular synth and the one that holds the LFO's that direct the polygons' movement.

s.waitForBoot{
	~path1 = PathName(thisProcess.nowExecutingPath).parentPath ++ "taapet.wav";
	~path2 = PathName(thisProcess.nowExecutingPath).parentPath ++ "carl.wav";
	~path3 = PathName(thisProcess.nowExecutingPath).parentPath ++ "wavxtsy.wav";
	~path4 = PathName(thisProcess.nowExecutingPath).parentPath ++ "bartok.wav";
	~path5 = PathName(thisProcess.nowExecutingPath).parentPath ++ "rap.wav";
	~path6 = PathName(thisProcess.nowExecutingPath).parentPath ++ "devil.wav";
	~path7 = PathName(thisProcess.nowExecutingPath).parentPath ++ "destructo.wav";
	~path8 = PathName(thisProcess.nowExecutingPath).parentPath ++ "yellow.wav";

~sample1 = Buffer.readChannel(s, ~path1, channels: 0, bufnum: 0);
	~sample2 = Buffer.readChannel(s, ~path2, channels: 0, bufnum: 1);
	~sample3 = Buffer.readChannel(s, ~path3, channels: 0, bufnum: 2);
	~sample4 = Buffer.readChannel(s, ~path4, channels: 0, bufnum: 3);
	~sample5 = Buffer.readChannel(s, ~path5, channels: 0, bufnum: 4);
	~sample6 = Buffer.readChannel(s, ~path6, channels: 0, bufnum: 5);
	~sample7 = Buffer.readChannel(s, ~path7, channels: 0, bufnum: 6);
	~sample8 = Buffer.readChannel(s, ~path8, channels: 0, bufnum: 7);

s.sync;

~oSCRate = 40;

s.sync;

// Audio signal
	SynthDef(\grain, {
		|outBus = 21, bufnum = 0, triggerRate = 3, rate = 1, bufferPos = 0, grainSize = 0.3, grainAmp = 0.1, amp = 1, lagTime = 0.1|

var trigger, centerPos, signal, output;

trigger = Impulse.ar(triggerRate);

centerPos = bufferPos * BufDur.kr(bufnum);

signal = TGrains.ar(1, trigger, bufnum, rate * BufRateScale.kr(bufnum), Lag2.kr(centerPos, 0.1), grainSize, amp: Lag2.kr(grainAmp, lagTime));

output = signal * amp;
		output = Mix(output);

Out.ar(outBus, output);

}).send(s);

SynthDef(\polygonsRotMod, {
		|outBus = 650, rate1 = 1, rate2 = 1, rate3 = 1, amp = 1, id1 = 1, id2 = 1, id3 = 1|

var signal1, signal2, signal3;

signal1 = SinOsc.kr(rate1, 0, amp);
		signal2 = SinOsc.kr(rate2, 0, amp);
		signal3 = SinOsc.kr(rate3, 0, amp);

Out.kr(outBus, signal1);

// OSC
		SendTrig.ar(Impulse.ar(~oSCRate), id1, signal1);
		SendTrig.ar(Impulse.ar(~oSCRate), id2, signal2);
		SendTrig.ar(Impulse.ar(~oSCRate), id3, signal3);

}).add;
};

The following code excerpt shows the creation of the polygons' synths, the arpeggiators patterns and how the different levels of interaction are triggered by the controller's acceleration.

{
	//---------- Variables ----------
	var fftArrayUser = Array.fill(~bufSize, 0);
	var fftArrayEnvironment = Array.fill(~bufSize, 0);
	var fftValuesUser = 0;
	var fftValuesEnvironment = 0;

var forceIndex = 2500000;

var client = NetAddr.new("127.0.0.1", 12345); // for OSC

var polygonsCounter = 0;
	var polygonsCounterRoutine;
	var polygonsCounterPlays = false;
	var polygonsCounterResets = false;
	var polygonsPlay = false;
	var polygonsReset = false;

var polygonsRotating = false;

var leftTrackpadOn = false;
	var rightTrackpadOn = false;

var amp1 = 0;
	var loudness1 = 0;

var amp2 = 0;
	var loudness2 = 0;

var polygonRotX;
	var polygonRotY;
	var polygonRotZ;

var velocityL = 0;
	var accelerationL = 0;

var velocityR = 0;
	var accelerationR = 0;

var radialVelAvg = 0;

var freezerLeft = 0;
	var freezerRight = 0;

var tempoRoutines = TempoClock(1);
	var tempoPolygon1 = TempoClock.default;
	var tempoPolygon2 = TempoClock.default;
	var tempoPolygon3 = TempoClock.default;
	var tempoPolygon4 = TempoClock.default;

var grainRatePolygon1 = Pdef(\grainRatePolygon1, Pseq([0.5, 1, 2, 0.5, 3, 0.5], inf));
	var grainRatePolygon2 = Pdef(\grainRatePolygon2, Pseq([1.5, 1, 0.5, 1, 0.5, 2], inf));
	var grainRatePolygon3 = Pdef(\grainRatePolygon3, Pseq([2.5, 2, 1.5, 1, 0.5, 0.5], inf));
	var grainRatePolygon4 = Pdef(\grainRatePolygon4, Pseq([2, 1.5, 2, 0.5, 2, 0.5], inf));

var rateSequencerPolygon1 = Pbind(
		\type, \set,
		\id, 1014,
		\args, #[\rate],
		\rate, grainRatePolygon1
	);
	var rateSequencerPolygon2 = Pbind(
		\type, \set,
		\id, 1015,
		\args, #[\rate],
		\rate, grainRatePolygon2
	);
	var rateSequencerPolygon3 = Pbind(
		\type, \set,
		\id, 1016,
		\args, #[\rate],
		\rate, grainRatePolygon3
	);
	var rateSequencerPolygon4 = Pbind(
		\type, \set,
		\id, 1017,
		\args, #[\rate],
		\rate, grainRatePolygon4
	);

//---------- Create synths ----------
	s.sendMsg(\s_new, "grain", 1011, 1, 1, \outBus, 21); // controller 1
	s.sendMsg(\s_new, "grain", 1012, 1, 1, \outBus, 22, \bufnum, 1); // controller 2
	s.sendMsg(\s_new, "grain", 1014, 1, 1, \outBus, 24, \id, 4, \bufnum, 4); // polygon1
	s.sendMsg(\s_new, "grain", 1015, 1, 1, \outBus, 25, \id, 5, \bufnum, 5); // polygon2
	s.sendMsg(\s_new, "grain", 1016, 1, 1, \outBus, 26, \id, 6, \bufnum, 6); // polygon3
	s.sendMsg(\s_new, "grain", 1017, 1, 1, \outBus, 27, \id, 7, \bufnum, 7); // polygon4

s.sendMsg(\s_new, "polygonsRotMod", 1054, 1, 1, \outBus, 651, \rate1, 0.1, \rate2, 0.1, \rate3, 0.1, \id1, 26, \id2, 27, \id3, 28);

s.sendMsg(\s_new, "fftTracker", 1021, 1, 1, \inBus, 0, \outBus, 0); // user
	s.sendMsg(\s_new, "fftTracker", 1023, 1, 1, \inBus, 63, \outBus, 257);  // environment camera

s.sendMsg(\s_new, "loudness", 1031, 1, 1, \inBus, 61, \outBus, 521); // controller 1
	s.sendMsg(\s_new, "loudness", 1032, 1, 1, \inBus, 62, \outBus, 522, \id, 12); // controller 2

//---------- Set SynthDefs ----------
	// FFT trackers
	s.sendMsg(\n_set, 1021, \bus, ~fftValuesBufUser);
	s.sendMsg(\n_set, 1023, \bus, ~fftValuesBufEnvironment);

//---------- OSC Functions ----------
	OSCdef(\amp, {
		|msg|

var id = msg[2];

var size = ~bufSize.div(2);

// msg.postln;
		if(id == 0){
			amp1 = msg[3];
			// amp1.postln;L
		};

if(id == 2){
			amp2 = msg[3];
			// amp2.postln;
		};

if(id == 11){
			loudness1 = msg[3];
			// loudness1.postln;
		};

if(id == 12){
			loudness2 = msg[3];
			// loudness2.postln;
		};

if(id == 26){
			var minIn = -1.0;
			var maxIn = 1.0;

var modAmp = msg[3];

polygonRotX = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

if(id == 27){
			var minIn = -1.0;
			var maxIn = 1.0;

var modAmp = msg[3];

polygonRotY = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

if(id == 28){
			var minIn = -1.0;
			var maxIn = 1.0;

var modAmp = msg[3];

polygonRotZ = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

radialVelAvg = (radialVelL * radialVelR) * 0.5;
		radialVelAvg = radialVelAvg.linlin(0.0, 2.0, 1.0, 2.0);
		// radialVelAvg.postln;

// Get the values of each channels and put it into index 1 of the array
		~fftValuesBufUser.getnSynchronous(~bufSize.div(2)).do{
			|val, i|

fftArrayUser.put([size-i, size+i], val);
			// fftArrayUser[1].postln; // Here are the values
			fftValuesUser = fftArrayUser[1];
		};

~totalFFT1 = fftValuesUser * amp1 * loudness1 * forceIndex * radialVelAvg;

~totalFFT2 = fftValuesUser * amp2 * loudness2 * forceIndex * radialVelAvg;

~totalFFT4 = fftValuesUser *  forceIndex;
		~totalFFT5 = fftValuesUser * forceIndex;
		~totalFFT6 = fftValuesUser * forceIndex;
		~totalFFT7 = fftValuesUser * forceIndex;

// send message to client
		client.sendMsg("/fft1", ~totalFFT1.abs, ~totalFFT4.abs, ~totalFFT5.abs, ~totalFFT6.abs, ~totalFFT7.abs);
		client.sendMsg("/fft2", ~totalFFT2.abs, ~totalFFT4.abs, ~totalFFT5.abs, ~totalFFT6.abs, ~totalFFT7.abs);
	},
	'/tr',
	s.addr
	);

OSCdef(\environmentReacts, {
		var accAvg;

accAvg = (accelerationL + accelerationR) / 2;
		// accAvg.postln;

// Environment Camera Counter
		if(accAvg > 250){
			if(environmentCameraCounterResets == false){
				environmentCameraCounter = 0;  // just to make sure

environmentCameraCounterPlays = false;
				environmentCameraCounterResets = true;

environmentCameraCounterRoutine.stop;
			};

} // else
		{
			if(environmentCameraCounterPlays == false){
				environmentCameraCounter = 0; // just to make sure
				polygonsCounter = 0;

environmentCameraCounterPlays = true;
				environmentCameraCounterResets = false;

environmentCameraCounterRoutine = Routine({
					loop{
						environmentCameraCounter = environmentCameraCounter + 1;
						"Environment camera counter: ".post;
						environmentCameraCounter.postln;
						1.wait;
					};
				}).play(tempoRoutines);

};

// Events on the environment's camera synth
		if(environmentCameraCounter >= 10){
			if(environmentCameraPlays == false){

s.sendMsg(\n_set, 1013, \amp, 1);
				s.sendMsg(\n_set, 1053, \amp, 1);
				"activating enviornment camera".postln;

environmentCameraPlays = true;
				environmentCameraResets = false;

};
		} // else
		{
			if(environmentCameraResets == false){

s.sendMsg(\n_set, 1053, \amp, 0);
				"deactivating environment camera".postln;

environmentCameraPlays = false;
				environmentCameraResets = true;
			};

};

// polygons Counter
		if(accAvg < 250){

if(polygonsCounterResets == false){
				polygonsCounter = 0;  // just to make sure

polygonsCounterPlays = false;
				polygonsCounterResets = true;

polygonsCounterRoutine.stop;
			};

} // else
		{
			if(polygonsCounterPlays == false){
				polygonsCounter = 0; // just to make sure

polygonsCounterPlays = true;
				polygonsCounterResets = false;

polygonsCounterRoutine = Routine({
					loop{
						polygonsCounter = polygonsCounter + 1;
						"Polygons counter: ".post;
						polygonsCounter.postln;
						1.wait;
					};
				}).play(tempoRoutines);

};

// Events on the polygons
		if((polygonsCounter >= 4) && ((leftTrackpadOn == true) || (rightTrackpadOn == true))){
			if(polygonsPlay == false){

client.sendMsg("/polygonsSwitch", 1);

forceIndex = 10000000;

"activating polygons".postln;

polygonsPlay = true;
				polygonsReset = false;

rateSequencerPolygon1.play(tempoPolygon1);
				rateSequencerPolygon2.play(tempoPolygon2);
				rateSequencerPolygon3.play(tempoPolygon3);
				rateSequencerPolygon4.play(tempoPolygon4);

};

if(polygonsCounter >= 4){
				client.sendMsg("/polygonsRotation", 0, polygonRotY, 0);

// Adjust rotation speed depending on user input
				if(accAvg > 250){

s.sendMsg(\n_set, 1054, \rate1, [-0.2, 0.2].choose, \rate2, rrand(0.2, 0.25), \rate3, [-0.2, 0.2].choose);

client.sendMsg("/polygonsRotation", polygonRotX, polygonRotY, polygonRotZ);

polygonsRotating = true;

} // else
				{
					if((accAvg > 100) && (accAvg < 250)){
						s.sendMsg(\n_set, 1054, \rate2, rrand(0.4, 0.6));
					} // else
					{
						s.sendMsg(\n_set, 1054, \rate2, rrand(0.08, 0.1));
					};

client.sendMsg("/polygonsRotation", 0, polygonRotY, 0);
					polygonsRotating = true;

};

} // else
			{
				client.sendMsg("/polygonsRotation", 0, 0, 0);

polygonsRotating = false;

};

} // else
		{
			polygonsRotating = false;

if(polygonsReset == false){

if(environmentCameraCounter > 10){

client.sendMsg("/polygonsSwitch", 0);
					forceIndex = 2500000;

"deactivating polygons".postln;

polygonsPlay = false;
					polygonsReset = true;
					polygonsRotating = false;

rateSequencerPolygon1.stop;
					rateSequencerPolygon2.stop;
					rateSequencerPolygon3.stop;
					rateSequencerPolygon4.stop;

};

};
	},
	'/tr',
	s.addr
	);

// Controller's Velocity and Acceleration
	OSCdef(\controllerLVelAcc, {
		|msg|

var velX, velY, velZ, accX, accY, accZ;

// post incoming msg
		// msg.postln;

// Cartesian coordinates
		velX = msg[1]; // X velocity axis in Unity
		velY = msg[2]; // Y velocity axis in unity
		velZ = msg[3]; // Z velocity axis in unity

accX = msg[4]; // X acceleration axis in Unity
		accY = msg[5]; // Y acceleration axis in unity
		accZ = msg[6]; // Z acceleration axis in unity

// Average velocity
		velocityL = (velX + velY + velZ) / 3;
		// velocityL.abs.postln.;

// Average acceleration
		accelerationL = ((accX + accY + accZ) / 3).abs;

},
	'/ControllerLVelAcc', // message address
	nil,
	6161 // message port
	);

OSCdef(\controllerRVelAcc, {
		|msg|

var velX, velY, velZ, accX, accY, accZ;

// post incoming msg
		// msg.postln;

// Cartesian coordinates
		velX = msg[1]; // X velocity axis in Unity
		velY = msg[2]; // Y velocity axis in unity
		velZ = msg[3]; // Z velocity axis in unity

accX = msg[4]; // X acceleration axis in Unity
		accY = msg[5]; // Y acceleration axis in unity
		accZ = msg[6]; // Z acceleration axis in unity

// Average velocity
		velocityR = (velX + velY + velZ) / 3;
		// velocityL.abs.postln.;

// Average acceleration
		accelerationR = ((accX + accY + accZ) / 3).abs;

},
	'/ControllerRVelAcc', // message address
	nil,
	6161 // message port
	);
};

On the Unity side, each polygon is an empty object that holds a RoundedCube.cs, a MeshDeformer.cs, a MeshRenderedSwitch.cs, and Polygon++"polygon index"++PosSendOSC.cs script. All of them are under a parent that holds the PolyginRotate.cs script. I have already posted the RoundedCube.cs and MeshDeformer.cs scripts, the only difference in the Polygons game objects is that they hold values of "2" for the four public variables of the RoundedCube.cs script: xSize, ySize, zSize and Roundness.
The following script shows how the mesh renderer of the polygons is activated.

The next script displays how the location of the polygons is sent via OSC. This case is a simpler script than the controllers' one because it's a constant communication.

And here is how the polygons rotate in a group. This script belongs to the parent that holds the 4 polygons.

Coming back to the SuperCollider side, the following excerpt shows how the parameters of the polygons' synths are handled. It also displays a couple of functions that set the boolean variables so the system knows if the user is pointing to the polygons or the walls. Moreover, I had to integrate some mixing functions in order to keep a balanced output level when activating the freezing module.

{
		//---------- Variables ----------
		var fftArrayUser = Array.fill(~bufSize, 0);
		var fftArrayEnvironment = Array.fill(~bufSize, 0);
		var fftValuesUser = 0;
		var fftValuesEnvironment = 0;

var forceIndex = 2500000;

var client = NetAddr.new("127.0.0.1", 12345); // for OSC

var environmentCameraCounter = 0;
		var environmentCameraCounterRoutine;
		var environmentCameraCounterPlays = false;
		var environmentCameraCounterResets = false;
		var environmentCameraPlays = false;
		var environmentCameraResets = false;

var leftColliderWall = true;
		var rightColliderWall = true;
		var leftColliderPolygon1 = false;
		var leftColliderPolygon2 = false;
		var leftColliderPolygon3 = false;
		var leftColliderPolygon4 = false;
		var rightColliderPolygon1 = false;
		var rightColliderPolygon2 = false;
		var rightColliderPolygon3 = false;
		var rightColliderPolygon4 = false;

var polygonsCounter = 0;
		var polygonsCounterRoutine;
		var polygonsCounterPlays = false;
		var polygonsCounterResets = false;
		var polygonsPlay = false;
		var polygonsReset = false;

var polygonsRotating = false;

var leftTrackpadOn = false;
		var rightTrackpadOn = false;

var amp1 = 0;
		var loudness1 = 0;

var amp2 = 0;
		var loudness2 = 0;

var posX;
		var posY;
		var posZ;
		var rotX;
		var rotY;
		var rotZ;

var polygonRotX;
		var polygonRotY;
		var polygonRotZ;

var oldRadialL = 0;
		var radialVelL = 0;
		var radialVelAmpL = 0;
		var velocityL = 0;
		var accelerationL = 0;

var oldRadialR = 0;
		var radialVelR = 0;
		var radialVelAmpR = 0;
		var velocityR = 0;
		var accelerationR = 0;

var oldRadialPolygon1 = 0;
		var radialVelPolygon1 = 0;
		var radialVelAmpPolygon1 = 0;

var oldRadialPolygon2 = 0;
		var radialVelPolygon2 = 0;
		var radialVelAmpPolygon2 = 0;

var oldRadialPolygon3 = 0;
		var radialVelPolygon3 = 0;
		var radialVelAmpPolygon3 = 0;

var oldRadialPolygon4 = 0;
		var radialVelPolygon4 = 0;
		var radialVelAmpPolygon4 = 0;

var radialVelAvg = 0;

var freezerLeft = 0;
		var freezerRight = 0;

var freezerCompThresh = 0.9;

var ampFrozen = 0.333;
		var ampUnFrozenControllers = 0.7;
		var ampUnFrozenPolygons = 0.666;

var tempoRoutines = TempoClock(1);
		var tempoPolygon1 = TempoClock.default;
		var tempoPolygon2 = TempoClock.default;
		var tempoPolygon3 = TempoClock.default;
		var tempoPolygon4 = TempoClock.default;

var grainRatePolygon1 = Pdef(\grainRatePolygon1, Pseq([0.5, 1, 2, 0.5, 3, 0.5], inf));
		var grainRatePolygon2 = Pdef(\grainRatePolygon2, Pseq([1.5, 1, 0.5, 1, 0.5, 2], inf));
		var grainRatePolygon3 = Pdef(\grainRatePolygon3, Pseq([2.5, 2, 1.5, 1, 0.5, 0.5], inf));
		var grainRatePolygon4 = Pdef(\grainRatePolygon4, Pseq([2, 1.5, 2, 0.5, 2, 0.5], inf));

var rateSequencerPolygon1 = Pbind(
			\type, \set,
			\id, 1014,
			\args, #[\rate],
			\rate, grainRatePolygon1
		);
		var rateSequencerPolygon2 = Pbind(
			\type, \set,
			\id, 1015,
			\args, #[\rate],
			\rate, grainRatePolygon2
		);
		var rateSequencerPolygon3 = Pbind(
			\type, \set,
			\id, 1016,
			\args, #[\rate],
			\rate, grainRatePolygon3
		);
		var rateSequencerPolygon4 = Pbind(
			\type, \set,
			\id, 1017,
			\args, #[\rate],
			\rate, grainRatePolygon4
		);

//---------- Create synths ----------
		s.sendMsg(\s_new, "grain", 1011, 1, 1, \outBus, 21); // controller 1
		s.sendMsg(\s_new, "grain", 1012, 1, 1, \outBus, 22, \bufnum, 1); // controller 2
		s.sendMsg(\s_new, "grain", 1014, 1, 1, \outBus, 24, \id, 4, \bufnum, 4); // polygon1
		s.sendMsg(\s_new, "grain", 1015, 1, 1, \outBus, 25, \id, 5, \bufnum, 5); // polygon2
		s.sendMsg(\s_new, "grain", 1016, 1, 1, \outBus, 26, \id, 6, \bufnum, 6); // polygon3
		s.sendMsg(\s_new, "grain", 1017, 1, 1, \outBus, 27, \id, 7, \bufnum, 7); // polygon4

s.sendMsg(\s_new, "grain", 1018, 1, 1, \outBus, 28, \bufnum, 5); // controller 1
		s.sendMsg(\s_new, "grain", 1019, 1, 1, \outBus, 29, \bufnum, 3); // controller 2

s.sendMsg(\s_new, "xFader", 1091, 1, 1, \inBus1, 21, \inBus2, 28, \outBus, 71); // controller 1
		s.sendMsg(\s_new, "xFader", 1092, 1, 1, \inBus1, 22, \inBus2, 29, \outBus, 72); // controller 2

s.sendMsg(\s_new, "grainEnvironment", 1013, 1, 1, \outBus, 23, \id, 3, \bufnum, 2, \inBusKr, 650); // environment camera

s.sendMsg(\s_new, "freeze", 1061, 1, 1, \inBus, 71, \outBus, 31); // controller 1
		s.sendMsg(\s_new, "freeze", 1062, 1, 1, \inBus, 72, \outBus, 32); // controller 2
		s.sendMsg(\s_new, "freeze", 1063, 1, 1, \inBus, 24, \outBus, 34); // polygon1
		s.sendMsg(\s_new, "freeze", 1064, 1, 1, \inBus, 25, \outBus, 35); // polygon2
		s.sendMsg(\s_new, "freeze", 1065, 1, 1, \inBus, 26, \outBus, 36); // polygon3
		s.sendMsg(\s_new, "freeze", 1066, 1, 1, \inBus, 27, \outBus, 37); // polygon4

s.sendMsg(\s_new, "shifter", 1071, 1, 1, \inBus, 31, \outBus, 41); // controller 1
		s.sendMsg(\s_new, "shifter", 1072, 1, 1, \inBus, 32, \outBus, 42); // controller 2

s.sendMsg(\s_new, "looper", 1081, 1, 1, \inBus, 41, \outBus, 51, \bufnum, 521); // controller 1
		s.sendMsg(\s_new, "looper", 1082, 1, 1, \inBus, 42, \outBus, 52, \bufnum, 522); // controller 2

s.sendMsg(\s_new, "atk", 1041, 1, 1, \inBus1, 41, \inBus2, 51, \outBus2, 61, \id, 0); // controller 1
		s.sendMsg(\s_new, "atk", 1042, 1, 1, \inBus1, 42, \inBus2, 52,  \outBus2, 62, \id, 2); // controller 2
		s.sendMsg(\s_new, "atk", 1043, 1, 1, \inBus1, 23, \inBus2, 23, \outBus2, 63, \id, 3); // environment camera
		s.sendMsg(\s_new, "atk", 1044, 1, 1, \inBus1, 34, \inBus2, 34, \outBus2, 64, \id, 4); // polygon 1
		s.sendMsg(\s_new, "atk", 1045, 1, 1, \inBus1, 35, \inBus2, 35, \outBus2, 65, \id, 5); // polygon 2
		s.sendMsg(\s_new, "atk", 1046, 1, 1, \inBus1, 36, \inBus2, 36, \outBus2, 66, \id, 6); // polygon 3
		s.sendMsg(\s_new, "atk", 1047, 1, 1, \inBus1, 37, \inBus2, 37, \outBus2, 67, \id, 7); // polygon 4

s.sendMsg(\s_new, "environmentAmpMod1", 1053, 1, 1, \outBus, 650, \rate1, 0.1, \rate2, 0.1, \rate3, 0.1, \id1, 23, \id2, 24, \id3, 25);
		s.sendMsg(\s_new, "polygonsRotMod", 1054, 1, 1, \outBus, 651, \rate1, 0.1, \rate2, 0.1, \rate3, 0.1, \id1, 26, \id2, 27, \id3, 28);

s.sendMsg(\s_new, "fftTracker", 1021, 1, 1, \inBus, 0, \outBus, 0); // user
		s.sendMsg(\s_new, "fftTracker", 1023, 1, 1, \inBus, 63, \outBus, 257);  // environment camera

s.sendMsg(\s_new, "loudness", 1031, 1, 1, \inBus, 61, \outBus, 521); // controller 1
		s.sendMsg(\s_new, "loudness", 1032, 1, 1, \inBus, 62, \outBus, 522, \id, 12); // controller 2
		s.sendMsg(\s_new, "loudness", 1033, 1, 1, \inBus, 63, \outBus, 523, \id, 13); // environment camera
		s.sendMsg(\s_new, "loudness", 1034, 1, 1, \inBus, 64, \outBus, 524, \id, 14); // polygon1
		s.sendMsg(\s_new, "loudness", 1035, 1, 1, \inBus, 65, \outBus, 525, \id, 15); // polygon2
		s.sendMsg(\s_new, "loudness", 1036, 1, 1, \inBus, 66, \outBus, 526, \id, 16); // polygon3
		s.sendMsg(\s_new, "loudness", 1037, 1, 1, \inBus, 67, \outBus, 527, \id, 17); // polygon4

//---------- Set SynthDefs ----------
		// FFT trackers
		s.sendMsg(\n_set, 1021, \bus, ~fftValuesBufUser);
		s.sendMsg(\n_set, 1023, \bus, ~fftValuesBufEnvironment);

//---------- OSC Functions ----------
		OSCdef(\amp, {
			|msg|

var id = msg[2];

var size = ~bufSize.div(2);

// msg.postln;
			if(id == 0){
				amp1 = msg[3];
				// amp1.postln;L
			};

if(id == 2){
				amp2 = msg[3];
				// amp2.postln;
			};

if(id == 3){
				amp3 = msg[3];
				// amp3.postln;
			};

if(id == 4){
				amp4 = msg[3];
				// amp4.postln;
			};

if(id == 5){
				amp5 = msg[3];
				// amp5.postln;
			};

if(id == 6){
				amp6 = msg[3];
				// amp6.postln;
			};

if(id == 7){
				amp7 = msg[3];
				// amp7.postln;
			};

if(id == 11){
				loudness1 = msg[3];
				// loudness1.postln;
			};

if(id == 12){
				loudness2 = msg[3];
				// loudness2.postln;
			};

if(id == 13){
				loudness3 = msg[3];
				// loudness3.postln;
			};

if(id == 14){
				loudness4 = msg[3];
				// loudness4.postln;
			};

if(id == 15){
				loudness5 = msg[3];
				// loudness5.postln;
			};

if(id == 16){
				loudness6 = msg[3];
				// loudness6.postln;
			};

if(id == 17){
				loudness7 = msg[3];
				// loudness7.postln;
			};

if(id == 23){
				var minIn = -1.0;
				var maxIn = 1.0;

var modAmp = msg[3];

posX = modAmp.linlin(minIn, maxIn, 0.0, 50.0);
				rotX = modAmp.linlin(minIn, maxIn, 0.0, 360.0);
			};

if(id == 24){
				var minIn = -1.0;
				var maxIn = 1.0;

var modAmp = msg[3];

posY = modAmp.linlin(minIn, maxIn, 0.0, 50.0);
				rotY = modAmp.linlin(minIn, maxIn, 0.0, 360.0);
			};

if(id == 25){
				var minIn = -1.0;
				var maxIn = 1.0;

var modAmp = msg[3];

posZ = modAmp.linlin(minIn, maxIn, -50.0, 0.0);
				rotZ = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

if(id == 26){
				var minIn = -1.0;
				var maxIn = 1.0;

var modAmp = msg[3];

polygonRotX = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

if(id == 27){
				var minIn = -1.0;
				var maxIn = 1.0;

var modAmp = msg[3];

polygonRotY = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

if(id == 28){
				var minIn = -1.0;
				var maxIn = 1.0;

var modAmp = msg[3];

polygonRotZ = modAmp.linlin(minIn, maxIn, 0.0, 360.0);

};

radialVelAvg = (radialVelL * radialVelR) * 0.5;
			radialVelAvg = radialVelAvg.linlin(0.0, 2.0, 1.0, 2.0);
			// radialVelAvg.postln;

// Get the values of each channels and put it into index 1 of the array
			~fftValuesBufUser.getnSynchronous(~bufSize.div(2)).do{
				|val, i|

fftArrayUser.put([size-i, size+i], val);
				// fftArrayUser[1].postln; // Here are the values
				fftValuesUser = fftArrayUser[1];
			};

~fftValuesBufEnvironment.getnSynchronous(~bufSize.div(2)).do{
				|val, i|

fftArrayEnvironment.put([size-i, size+i], val);
				// fftArrayEnvironment[1].postln; // Here are the values
				fftValuesEnvironment = fftArrayEnvironment[1];
			};

~totalFFT1 = fftValuesUser * amp1 * loudness1 * forceIndex * radialVelAvg;
			// ~totalFFT1.postln;
			~totalFFT2 = fftValuesUser * amp2 * loudness2 * forceIndex * radialVelAvg;
			// ~totalFFT2.postln;
			~totalFFT3 = fftValuesEnvironment * forceIndex;
			// ~totalFFT2.postln;
			~totalFFT4 = fftValuesUser *  forceIndex;
			~totalFFT5 = fftValuesUser * forceIndex;
			~totalFFT6 = fftValuesUser * forceIndex;
			~totalFFT7 = fftValuesUser * forceIndex;

// send message to client
			client.sendMsg("/fft1", ~totalFFT1.abs, ~totalFFT4.abs, ~totalFFT5.abs, ~totalFFT6.abs, ~totalFFT7.abs);
			client.sendMsg("/fft2", ~totalFFT2.abs, ~totalFFT4.abs, ~totalFFT5.abs, ~totalFFT6.abs, ~totalFFT7.abs);
			client.sendMsg("/fft3", ~totalFFT3.abs);
			client.sendMsg("/environmentPosRot", posX, posY, posZ, rotX, rotY, rotZ);
			// "fftValues1: ".post;
			// fftValues1.postln;
			// "fftValues2: ".post;
			// fftValues2.postln;
			// "fftValues3: ".post;
			// fftValues3.postln;
		},
		'/tr',
		s.addr
		);

OSCdef(\environmentReacts, {
			var accAvg;

accAvg = (accelerationL + accelerationR) / 2;
			// accAvg.postln;

// Environment Camera Counter
			if(accAvg > 250){
				if(environmentCameraCounterResets == false){
					environmentCameraCounter = 0;  // just to make sure

environmentCameraCounterPlays = false;
					environmentCameraCounterResets = true;

environmentCameraCounterRoutine.stop;
				};

} // else
			{
				if(environmentCameraCounterPlays == false){
					environmentCameraCounter = 0; // just to make sure
					polygonsCounter = 0;

environmentCameraCounterPlays = true;
					environmentCameraCounterResets = false;

environmentCameraCounterRoutine = Routine({
						loop{
							environmentCameraCounter = environmentCameraCounter + 1;
							"Environment camera counter: ".post;
							environmentCameraCounter.postln;
							1.wait;
						};
					}).play(tempoRoutines);

};

// Events on the environment's camera synth
			if(environmentCameraCounter >= 10){
				if(environmentCameraPlays == false){

s.sendMsg(\n_set, 1013, \amp, 1);
					s.sendMsg(\n_set, 1053, \amp, 1);
					"activating enviornment camera".postln;

environmentCameraPlays = true;
					environmentCameraResets = false;

};
			} // else
			{
				if(environmentCameraResets == false){

s.sendMsg(\n_set, 1053, \amp, 0);
					"deactivating environment camera".postln;

environmentCameraPlays = false;
					environmentCameraResets = true;
				};

};

// polygons Counter
			if(accAvg < 250){

if(polygonsCounterResets == false){
					polygonsCounter = 0;  // just to make sure

polygonsCounterPlays = false;
					polygonsCounterResets = true;

polygonsCounterRoutine.stop;
				};

} // else
			{
				if(polygonsCounterPlays == false){
					polygonsCounter = 0; // just to make sure

polygonsCounterPlays = true;
					polygonsCounterResets = false;

polygonsCounterRoutine = Routine({
						loop{
							polygonsCounter = polygonsCounter + 1;
							"Polygons counter: ".post;
							polygonsCounter.postln;
							1.wait;
						};
					}).play(tempoRoutines);

};

// Events on the polygons
			if((polygonsCounter >= 4) && ((leftTrackpadOn == true) || (rightTrackpadOn == true))){
				if(polygonsPlay == false){

client.sendMsg("/polygonsSwitch", 1);

forceIndex = 10000000;

"activating polygons".postln;

polygonsPlay = true;
					polygonsReset = false;

rateSequencerPolygon1.play(tempoPolygon1);
					rateSequencerPolygon2.play(tempoPolygon2);
					rateSequencerPolygon3.play(tempoPolygon3);
					rateSequencerPolygon4.play(tempoPolygon4);

};

if(polygonsCounter >= 4){
					client.sendMsg("/polygonsRotation", 0, polygonRotY, 0);

// Adjust rotation speed depending on user input
					if(accAvg > 250){

s.sendMsg(\n_set, 1054, \rate1, [-0.2, 0.2].choose, \rate2, rrand(0.2, 0.25), \rate3, [-0.2, 0.2].choose);

client.sendMsg("/polygonsRotation", polygonRotX, polygonRotY, polygonRotZ);

polygonsRotating = true;

} // else
					{
						if((accAvg > 100) && (accAvg < 250)){
							s.sendMsg(\n_set, 1054, \rate2, rrand(0.4, 0.6));
						} // else
						{
							s.sendMsg(\n_set, 1054, \rate2, rrand(0.08, 0.1));
						};

client.sendMsg("/polygonsRotation", 0, polygonRotY, 0);
						polygonsRotating = true;

};

} // else
				{
					client.sendMsg("/polygonsRotation", 0, 0, 0);

polygonsRotating = false;

};

} // else
			{
				polygonsRotating = false;

if(polygonsReset == false){

if(environmentCameraCounter > 10){

client.sendMsg("/polygonsSwitch", 0);
						forceIndex = 2500000;

"deactivating polygons".postln;

polygonsPlay = false;
						polygonsReset = true;
						polygonsRotating = false;

rateSequencerPolygon1.stop;
						rateSequencerPolygon2.stop;
						rateSequencerPolygon3.stop;
						rateSequencerPolygon4.stop;

};

};
		},
		'/tr',
		s.addr
		);

// Player's position
		OSCdef(\hmdPos, {
			|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, crossfade;

// post incoming msg
			// msg.postln;

// Cartesian coordinates
			// front and rear
			axisX = msg[3]; // Z transform.position axis in Unity
			~hmdX = axisX;

// left and right
			axisY = msg[1]; // X transform.position axis in Unity
			~hmdY = axisY;

// top to bottom
			axisZ = msg[2]; // Y transform.position axis in Unity
			~hmdZ = axisZ;

// Synths' parameters
			crossfade = ~hmdY.linlin(0.0, 50.0, -1.0, 1.0);

// Set SynthDefs
			s.sendMsg(\n_set, 1091, \crossfade, crossfade);
			s.sendMsg(\n_set, 1092, \crossfade, crossfade);
		},
		'/HMDPosition', // message address
		nil,
		6161 // message port
		);

// Player's rotation
		OSCdef(\hmdRot, {
			|msg|

var roll, pitch, yaw;

// post incoming msg
			// msg.postln;

// Receiving rotation euler angles from Unity
			roll = msg[3]; // Z transform.localRotation.eulerAngles axis in Unity
			~hmdRoll = roll; // This variable is not in use

pitch = msg[1]; // X transform.localRotation.eulerAngles axis in Unity
			// to simulate Ambisonics degrees
			// scale from 0-360 to -180-180
			~hmdPitch = pitch;
			~hmdPitch = ((~hmdPitch - 180) % 360) - 180;
			~hmdPitch = ~hmdPitch.linlin(-180, 180, -pi / 2, pi / 2) * -1;

yaw = msg[2]; // Y transform.localRotation.eulerAngles axis in Unity
			// to simulate Ambisonics degrees
			// scale from 0-360 to -180-180
			~hmdYaw = yaw;
			~hmdYaw = ((~hmdYaw - 180) % 360) - 180;
			~hmdYaw = ~hmdYaw.linlin(-180, 180, -pi, pi) * -1;
		},
		'/HMDRotation', // message address
		nil,
		6161 // message port
		);

// Controller's Velocity and Acceleration
		OSCdef(\controllerLVelAcc, {
			|msg|

var velX, velY, velZ, accX, accY, accZ;

// post incoming msg
			// msg.postln;

// Cartesian coordinates
			velX = msg[1]; // X velocity axis in Unity
			velY = msg[2]; // Y velocity axis in unity
			velZ = msg[3]; // Z velocity axis in unity

accX = msg[4]; // X acceleration axis in Unity
			accY = msg[5]; // Y acceleration axis in unity
			accZ = msg[6]; // Z acceleration axis in unity

// Average velocity
			velocityL = (velX + velY + velZ) / 3;
			// velocityL.abs.postln.;

// Average acceleration
			accelerationL = ((accX + accY + accZ) / 3).abs;

},
		'/ControllerLVelAcc', // message address
		nil,
		6161 // message port
		);

OSCdef(\controllerRVelAcc, {
			|msg|

var velX, velY, velZ, accX, accY, accZ;

// post incoming msg
			// msg.postln;

// Cartesian coordinates
			velX = msg[1]; // X velocity axis in Unity
			velY = msg[2]; // Y velocity axis in unity
			velZ = msg[3]; // Z velocity axis in unity

accX = msg[4]; // X acceleration axis in Unity
			accY = msg[5]; // Y acceleration axis in unity
			accZ = msg[6]; // Z acceleration axis in unity

// Average velocity
			velocityR = (velX + velY + velZ) / 3;
			// velocityL.abs.postln.;

// Average acceleration
			accelerationR = ((accX + accY + accZ) / 3).abs;

},
		'/ControllerRVelAcc', // message address
		nil,
		6161 // message port
		);

1.wait;

// Polygon's position
		OSCdef(\polygon1Location, {
			|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, preAmp, dryAmp, revTime, dryLevel, lagTime, triggerRate, grainSize, grainAmp, bufferPos;

// post incoming msg
			// msg.postln;

// From cartesian coordinates to spherical
			// front and rear
			axisX = msg[3]; // Z transform.position axis in Unity
			// axisX.postln;

// left and right
			axisY = msg[1]; // X transform.position axis in unity

// top to bottom
			axisZ = msg[2]; // Y transform.position axis in unity

// distance
			rayDistance = msg[4]; // RayCastHit hit.distance in Unity
			// rayDistance.postln;

// Spherical coordinates formulas
			// radial sqrt(x^2 + y^2 + z^2)
			radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
			// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
			// Therefore my maximum value is 56.789083458003
			radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians
			// "radial: ".post;
			// radial.postln;

// azimuth atan2(y, x)
			azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
			azimuth = azimuth - ~hmdYaw;
			// "azimuth: ".post;
			// azimuth.postln;

// polar cos^(-1)(z/r)
			polar = axisZ.linlin(-28.92, 28.92, -pi / 2, pi / 2); // scale to radians
			polar = polar - ~hmdPitch;
			// "polar: ".post;
			// polar.postln;

// Radial Velocity
			fork{
				oldRadialPolygon1 = radial;
				0.001.wait;
			};

radialVelPolygon1 = (radial - oldRadialPolygon1).abs;
			radialVelAmpPolygon1 = radialVelPolygon1.linexp(0.0, 0.0005, 0.0001, 1.0);

// Synths' parameters
			dryAmp = radialVelAmpPolygon1;
			revTime = radialVelAmpPolygon1.expexp(0.0001, 1.0, 2.0, 20);
			dryLevel = radial.linexp(0, pi / 2, 1.00, 0.001);
			triggerRate = radial.linexp(0, pi / 2, 60.0, 16.0);
			tempoPolygon1.tempo = triggerRate;
			grainSize = radial.linexp(0, pi / 2, 0.1, 1.0);
			grainAmp = radial.linexp(0, pi / 2, 1.0, 0.5);
			bufferPos = radial.linlin(0, pi / 2, 0, 1);

if(polygonsPlay == true){
				if((polygonsRotating == true) || (leftColliderPolygon1 == true) || (rightColliderPolygon1 == true) || (radial < 0.2)){
					// if the Polygon is active then calculate spatialization
					s.sendMsg(\n_set, 1044, \angle, radial, \azimuth, azimuth, \elevation, polar, \dryAmp, dryAmp, \revTime, revTime, \dryLevel, dryLevel, \preAmp1, 1, \preAmp2, 1);
					s.sendMsg(\n_set, 1014, \triggerRate, triggerRate, \grainAmp, grainAmp, \grainSize, grainSize, \bufferPos, bufferPos, \amp, 0.666);

// Turn amp up or down depending if it is frozen or not
					if((freezerLeft == 0) || (freezerRight == 0)){
						preAmp = radial.linexp(0, pi / 2, 0.25, 0.1);

s.sendMsg(\n_set, 1044, \preAmp, preAmp);
					} // else
					{
						preAmp = radial.linexp(0, pi / 2, 1.0, 0.666);

s.sendMsg(\n_set, 1044, \preAmp, preAmp);
					};

} // else
				{
					s.sendMsg(\n_set, 1044, \preAmp, 0);
				};
			} // else
			{
				s.sendMsg(\n_set, 1044, \preAmp, 0);
			};

},
		'/polygon1Location', // message address
		nil,
		6161 // message port
		);

OSCdef(\polygon2Location, {
			|msg|

var axisX, axisY, axisZ, radial, azimuth, polar, rayDistance, distance, preAmp, dryAmp, revTime, dryLevel, lagTime, triggerRate, grainSize, grainAmp, bufferPos;

// post incoming msg
			// msg.postln;

// From cartesian coordinates to spherical
			// front and rear
			axisX = msg[3]; // Z transform.position axis in Unity
			// axisX.postln;

// left and right
			axisY = msg[1]; // X transform.position axis in unity

// top to bottom
			axisZ = msg[2]; // Y transform.position axis in unity

// distance
			rayDistance = msg[4]; // RayCastHit hit.distance in Unity
			// rayDistance.postln;

// Spherical coordinates formulas
			// radial sqrt(x^2 + y^2 + z^2)
			radial = sqrt(((axisX - ~hmdX) ** 2) + ((axisY - ~hmdY) ** 2) + ((axisZ - ~hmdZ) ** 2));
			// I'm working with Unity dimension of 50 (front-rear), 50 (left-right) and 25 (top-bottom)
			// Therefore my maximum value is 56.789083458003
			radial = radial.linlin(0, 56.789083458003, 0, pi / 2); // scale to radians
			// "radial: ".post;
			// radial.postln;

// azimuth atan2(y, x)
			azimuth = atan2(axisY - ~hmdY, axisX - ~hmdX) * -1; // result will be in radians
			azimuth = azimuth - ~hmdYaw;
			// "azimuth: ".post;
			// azimuth.postln;

// polar cos^(-1)(z/r)
			polar = axisZ.linlin(-28.92, 28.92, -pi / 2, pi / 2); // scale to radians
			polar = polar - ~hmdPitch;
			// "polar: ".post;
			// polar.postln;

// Radial Velocity
			fork{
				oldRadialPolygon2 = radial;
				0.001.wait;
			};

radialVelPolygon2 = (radial - oldRadialPolygon2).abs;
			radialVelAmpPolygon2 = radialVelPolygon2.linexp(0.0, 0.0005, 0.0001, 1.0);

// Synths' parameters
			dryAmp = radialVelAmpPolygon2;
			revTime = radialVelAmpPolygon2.expexp(0.0001, 1.0, 2.0, 20);
			dryLevel = radial.linexp(0, pi / 2, 1.00, 0.001);
			triggerRate = radial.linexp(0, pi / 2, 60.0, 16.0);
			tempoPolygon2.tempo = triggerRate;
			grainSize = radial.linexp(0, pi / 2, 0.1, 1.0);
			grainAmp = radial.linexp(0, pi / 2, 1.0, 0.5);
			bufferPos = radial.linlin(0, pi / 2, 0, 1);

if(polygonsPlay == true){
				if((polygonsRotating == true) || (leftColliderPolygon2 == true) || (rightColliderPolygon2 == true) || (radial < 0.2)){
					// if Polygon active then calculate spatialization
					s.sendMsg(\n_set, 1045, \angle, radial, \azimuth, azimuth, \elevation, polar, \dryAmp, dryAmp, \revTime, revTime, \dryLevel, dryLevel, \preAmp1, 1, \preAmp2, 1);
					s.sendMsg(\n_set, 1015, \triggerRate, triggerRate, \grainAmp, grainAmp, \grainSize, grainSize, \bufferPos, bufferPos, \amp, 0.666);

// Turn amp up or down depending if it is frozen or not
					if((freezerLeft == 0) || (freezerRight == 0)){
						preAmp = radial.linexp(0, pi / 2, 0.75, 0.333);

s.sendMsg(\n_set, 1045, \preAmp, preAmp);
					} // else
					{
						preAmp = radial.linexp(0, pi / 2, 1.0, 0.666);

s.sendMsg(\n_set, 1045, \preAmp, preAmp);
					};

} // else
				{
					s.sendMsg(\n_set, 1045, \preAmp, 0);
				};
			} // else
			{
				s.sendMsg(\n_set, 1045, \preAmp, 0);
			};
		},
		'/polygon2Location', // message address
		nil,
		6161 // message port
		);

OSCdef(\polygon3Location, {
			|msg|