Install for your platform

Choose the complete setup guide for macOS, Linux, or Windows. All platforms need a Kinect v2 power adapter and a USB 3.0 host connection. Linux users must also install the repository's udev rule before opening the device without root.

Build and run Protonect

Configure and build from the repository root:

cmake -S . -B build

cmake --build build --target Protonect

Then run the example from the build output. Single-config generators normally place it here:

./build/bin/Protonect

Multi-config generators such as Visual Studio add a configuration directory, for example build/bin/Release/Protonect.exe. A viewer showing color, IR, and depth confirms that device access, USB transfers, decoding, and the selected depth pipeline all work. If it fails, go directly to Troubleshooting.

API walkthrough

The snippets below come from examples/Protonect.cpp, which remains the full working reference.

Headers

Include the device and frame-listener APIs. Registration and logging are only needed when your application uses them.

#include <libfreenect2/libfreenect2.hpp>
#include <libfreenect2/frame_listener_impl.h>
#include <libfreenect2/registration.h>
#include <libfreenect2/packet_pipeline.h>
#include <libfreenect2/logger.h>

Logging

Install a logger before opening a device when you need more or less detail.

libfreenect2::setGlobalLogger(libfreenect2::createConsoleLogger(libfreenect2::Logger::Debug));

libfreenect2 will have an initial global logger created with createConsoleLoggerWithDefaultLevel(). You do not have to explicitly call this if the default is already what you want.

Applications can also implement libfreenect2::Logger. The callback can run from multiple library threads, so custom loggers must synchronize their own state. This example writes messages to a file:

#include <fstream>
#include <cstdlib>
class MyFileLogger: public libfreenect2::Logger
{
private:
  std::ofstream logfile_;
public:
  MyFileLogger(const char *filename)
  {
    if (filename)
      logfile_.open(filename);
    level_ = Debug;
  }
  bool good()
  {
    return logfile_.is_open() && logfile_.good();
  }
  virtual void log(Level level, const std::string &message)
  {
    logfile_ << "[" << libfreenect2::Logger::level2str(level) << "] " << message << std::endl;
  }
};

  MyFileLogger *filelogger = new MyFileLogger(getenv("LOGFILE"));
  if (filelogger->good())
    libfreenect2::setGlobalLogger(filelogger);
  else
    delete filelogger;

Discover a device

Create a context and enumerate devices before requesting a serial number.

  libfreenect2::Freenect2 freenect2;
  libfreenect2::Freenect2Device *dev = 0;
  std::unique_ptr<libfreenect2::PacketPipeline> pipeline;

  if(freenect2.enumerateDevices() == 0)
  {
    std::cout << "no device connected!" << std::endl;
    return -1;
  }
 
  if (serial == "")
  {
    serial = freenect2.getDefaultDeviceSerialNumber();
  }

The default pipeline selects the best available implementation for the current build. To require a particular backend, construct a libfreenect2::PacketPipeline explicitly. The example includes the supported selection pattern:

pipeline.reset(new libfreenect2::CpuPacketPipeline());

Open and configure

Open by serial number, optionally passing the chosen pipeline.

dev = freenect2.openDevice(serial, pipeline.release());

Attach listeners before starting the device. A libfreenect2::SyncMultiFrameListener groups the requested frame types by arrival; it does not guarantee a timestamp delta or hardware synchronization. Use libfreenect2::TimestampAlignedFrameListener when a bounded device-timestamp threshold is required, and read Frame timing and software pairing for the clock constraints.

  int types = 0;
  if (enable_rgb)
    types |= libfreenect2::Frame::Color;
  if (enable_depth)
    types |= libfreenect2::Frame::Ir | libfreenect2::Frame::Depth;
  libfreenect2::SyncMultiFrameListener listener(types);
  libfreenect2::FrameMap frames;
 
  dev->setColorFrameListener(&listener);
  dev->setIrAndDepthFrameListener(&listener);

Configure depth clipping and filters before start(); see Runtime configuration reference.

Start processing

  if (enable_rgb && enable_depth)
  {
    if (!dev->start())
      return -1;
  }
  else
  {
    if (!dev->startStreams(enable_rgb, enable_depth))
      return -1;
  }
 
  std::cout << "device serial: " << dev->getSerialNumber() << std::endl;
  std::cout << "device firmware: " << dev->getFirmwareVersion() << std::endl;

Registration uses the factory IR and color camera parameters unless your application explicitly supplies replacements.

libfreenect2::Registration registration(dev->getIrCameraParams(), dev->getColorCameraParams());

libfreenect2::Frame undistorted(512, 424, 4), registered(512, 424, 4);

Receive and register frames

Wait for the requested frame set, then retrieve color and depth by frame type.

  while(!protonect_shutdown && (framemax == (size_t)-1 || framecount < framemax))
  {
    if (!listener.waitForNewFrame(frames, 10*1000)) // 10 sconds
    {
      std::cout << "timeout!" << std::endl;
      return -1;
    }
    libfreenect2::Frame *rgb = frames[libfreenect2::Frame::Color];
    libfreenect2::Frame *ir = frames[libfreenect2::Frame::Ir];
    libfreenect2::Frame *depth = frames[libfreenect2::Frame::Depth];
    (void)ir;

libfreenect2::Frame documents dimensions, pixel formats, timestamps, and ownership. The registration call below creates undistorted depth and color aligned to the 512×424 depth image:

registration.apply(rgb, depth, &undistorted, &registered);

Release each frame map after the application has finished reading it.

listener.release(frames);

}

Stop and close

Stop streaming before closing the device and destroying the pipeline.

dev->stop();

dev->close();

A device can be paused with libfreenect2::Freenect2Device::stop and restarted with libfreenect2::Freenect2Device::start:

  if (protonect_paused)
    devtopause->start();
  else
    devtopause->stop();
  protonect_paused = !protonect_paused;

Where to go next

Registration and coordinate mapping recipes explains coordinate mapping and point-cloud construction.
Recording, replay, and multiple Kinects covers offline processing and multiple sensors.
Conventional camera calibration profiles and Per-device depth correction cover explicit camera and depth correction profiles.
Guides lists every task-oriented guide.
API topics and the class list provide the complete public reference.

Table of Contents