Face Tracking

The above is using the face tracking example created by Kyle Mcdonald here. I will explain this example below and the minor modifications I made below. First here are a couple of general notes:

3. Here is a great survey by Kyle of the various logics and underpinnings of face tracking methods. I would encourage all of you to read this as it might inspire ways of framing your projects. There are many strategies and some of them touch on the politics of face tracking.

Below is a break down of the code that makes up the example in the video above. You can find the code here.

The general structure is a live video feed. The tracking library is then used to analyze the video feed and the movement of pixels to create a track of detected face. The tracking library then returns a list of 63 points in an array of x,y coordinates.  In this example, the sketch draws a line through these points and labels them by number. I have added a slider that allows you to cycle through these points. This can help you find the points that represent your eyes or corners of your mouth for instance.

var capture;
var tracker
var w = 640,
    h = 480;

var slider;
var showtrack = true;

These are the global variable declarations. Capture is an object that will hold the video. Tracker is an object that will hold the tracked points. I have also added the ability to turn off the tracker by pressing the shift key. This will be helpful when you build onto this sketch and you want to use the data to show other graphics and you want to be able to turn the track on and off for diagnostics.

function setup() {
    createCanvas(w, h);
    capture = createCapture({
        audio: false,
        video: {
            width: w,
            height: h
        }
    }, function() {
        console.log('capture ready.')
    });
    capture.elt.setAttribute('playsinline', '');
    capture.size(w, h);
    capture.hide();

    colorMode(HSB);

    tracker = new clm.tracker();
    tracker.init();
    tracker.start(capture.elt);
  
    slider = createSlider(0,62,0);
}

In the setup function there are really three blocks of code:
1. Setting up the video capture with the capture object and making it the same size as the canvas.
2. Setting up the tracker and start the tracking using the video capture.
3. Initializing the slider.

Below is the run function code:

function draw() {
    image(capture, 0, 0, w, h);
    var positions = tracker.getCurrentPosition();

This shows the video capture on the canvas at position (0,0) and the width and height based on the variables that define the canvas. The tracked positions are stored in an array called positions

   if(showtrack == true){
      noFill();
      stroke(255);
      beginShape();
      for (var i = 0; i < positions.length; i++) {
          vertex(positions[i][0], positions[i][1]);
      }
      endShape();

The conditional that wraps this will draw the track is the boolean variable showtrack is true(the shift key toggles this). The first block of code uses a loop to cycle through the track positions and draw a multisegmented shape as a white line.

      noStroke();
      for (var i = 0; i < positions.length; i++) {
          fill(map(i, 0, positions.length, 0, 360), 50, 100);
          ellipse(positions[i][0], positions[i][1], 4, 4);
          textSize(10);
          text(i, positions[i][0], positions[i][1]);
      }

The next block of code is a loop that labels the index of each track point and adds an ellipse. The fill is mapped using the HSB colormode defined in setup. This gives the labels a gradient.

     if (positions.length > 0) {
          noStroke();
          fill(0, 255, 255);
          ellipse(positions[slider.value()][0], positions[slider.value()][1], 20, 20);
        textSize(32);
        fill(255);
        text(slider.value(), 10, height-10);
      }
    }

This block of code draws an ellipse at the track point based on the slider value. The slider value (0-63) is used as the index to get the track position. This needs to be in a conditional that makes sure the track array has been generated. Otherwise, it will produce an error. If the length of positions array is greater than 0 then that means a track has been generated. The loops above don’t need this because if the array has a length of 0 then the loops won’t run.

    if (positions.length > 0) {
      line(positions[27][0],positions[27][1],positions[29][0],positions[29][1]);
    }
}

The last bit of code in the run function draws a line between the center of each eye. This is outside of the showtrack conditional so it stays on when you press the shift key.

function keyPressed() {
  if (keyCode == SHIFT) {
    if(showtrack == true){
      showtrack = false;
    }else{
      showtrack = true;
    }
  }
}

The last block of code is simply a function that toggles the show track boolean variable between true and falls when the shift key is pressed.