High Speed Capture

This example shows how to use the FrameQueueSink to capture all incoming images, even if the processing task takes longer than the frame interval.

The program can be found in the %TOPLEVEL%\Samples\VC\HighSpeedCapture directory. In order to run the program, open the solution file HighSpeedCapture.sln in this directory and select Build -> Build HighSpeedCapture in the menu. You can execute the program by selecting Debug -> Start.

Overview

First, the program asks the user to select and configure a video capture device. After a device was selected, live images from the device are displayed.

In the bottom half of the application window, the destination directory and the total size of memory used to buffer incoming image data can be configured. Press Maximize to let the program allocate as much memory as possible without using the swap file.

Once the Start button is pressed, all images that are received from the camera are saved into the destination directory. If the hard disk is slower than the camera, the images that have not been saved yet are stored in memory until there is time to save them. Only if all buffer memory is filled, the application will drop frames and not save them.

When the Stop button is pressed, all images that are still buffered in memory are processed and saved to the hard disk.

Setting up the Sink

The sink is created in OnBnClickedDevice, after the video format has been selected by the user:

GUID sinkFormat = SelectMatchingSinkFormat( m_Grabber.getVideoFormat().getSubtype() );
m_pSink = DShowLib::FrameQueueSink::create( *this, sinkFormat );
m_pSink->setSinkMode( DShowLib::GrabberSinkType::ePAUSE );
m_Grabber.setSinkType( m_pSink );
m_Grabber.startLive();

The program uses a helper function, SelectMatchingSinkFormat, to select a sink buffer format depending on the video format.

When the sink is created by calling FrameQueueSink::create, the a reference to this is passed as listener. That way, we can get receive notifications directly in our dialog class.

The sink is initially put into the GrabberSinkType::ePAUSE state, so that it does not process images until it is started by the user.

Buffer Management

The TryAllocateBuffers function uses FrameQueueSink::allocAndQueueBuffers to enqueue the number of buffers requested based on the total buffer memory size specified by the user.

Callback Function

The FrameQueueSinkListener::framesQueued is implemented in CHighSpeedCaptureDlg:

void CHighSpeedCaptureDlg::framesQueued( DShowLib::FrameQueueSink& sink )
{
    std::vector<smart_ptr<DShowLib::FrameQueueBuffer>> buffers = sink.popAllOutputQueueBuffers();
    for( size_t i = 0; i < buffers.size(); ++i )
    {
        CString fileName;
        fileName.Format( TEXT("%s\\image_%d.bmp"), m_destinationFolder.GetString(), m_frameNumber++ );
        DShowLib::saveToFileBMP( *buffers[i], fileName );
        UpdateStats( sink, m_sinkStats, buffers.size() - i, 1 );
        sink.queueBuffer( buffers[i] );
    }
    UpdateStats( sink, m_sinkStats, 0, 0 );
}

In the callback function, all images currently in the sink's output queue are popped and then saved to disk.

Image Statistics

The program displays both driver and sink frame count information:

DShowLib::DriverFrameDropInformation streamStats = {};
m_Grabber.getDriverFrameDropInformation( streamStats );
uint64_t driver_delivered = streamStats.FramesDelivered;
uint64_t driver_dropped = streamStats.FramesDroppedDueToPacketLoss + streamStats.FramesDroppedDueToTransforms
                        + streamStats.FramesDroppedDueToApplicationQueue + streamStats.FramesDroppedDueUnspecified;
DShowLib::FrameQueueSink::FrameCountInformation sinkStats = m_pSink->getFrameCountInfo();
int64_t sink_copied = sinkStats.framesCopied;
int64_t sink_dropped = sinkStats.framesDropped;
txt.Format( TEXT( "Driver: delivered %d, dropped %d - Sink: copied %d, dropped %d - Processed: %d" ), driver_delivered, driver_dropped, sink_copied, sink_dropped, stats.numProcessed );
m_lblCaptureInfo.SetWindowText( txt );

The driver's frame counters are accessed using Grabber::getDriverFrameDropInformation. Driver frame drops happen when there is a problem during transmission, or the driver did not have enough CPU resources available to perform all requested image processing operations.

On the other hand, the sink frame counters are queried by calling FrameQueueSink::getFrameCountInfo. The sink drops a frame when there is no free buffer available for the received image to be copied to.

<< Programmer's Guide