CGImageRef访问像素数据的更快方式？

Question

我目前的方法是：

CGDataProviderRef provider = CGImageGetDataProvider(imageRef); 
imageData.rawData = CGDataProviderCopyData(provider); 
imageData.imageData = (UInt8 *) CFDataGetBytePtr(imageData.rawData); 

我只得到约每秒30帧。我知道性能命中的一部分是复制数据，如果我可以访问字节流并且不会自动为我创建副本，那将会很好。

我试图让它尽快处理CGImageRefs，有没有更快的方法？

这是我的工作方案片段：

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification { 
    // Insert code here to initialize your application 
    //timer = [NSTimer scheduledTimerWithTimeInterval:1.0/60.0 //2000.0 
    //           target:self 
    //          selector:@selector(timerLogic) 
    //          userInfo:nil 
    //          repeats:YES]; 
    leagueGameState = [LeagueGameState new]; 

    [self updateWindowList]; 
    lastTime = CACurrentMediaTime(); 






    // Create a capture session 
    mSession = [[AVCaptureSession alloc] init]; 

    // Set the session preset as you wish 
    mSession.sessionPreset = AVCaptureSessionPresetMedium; 

    // If you're on a multi-display system and you want to capture a secondary display, 
    // you can call CGGetActiveDisplayList() to get the list of all active displays. 
    // For this example, we just specify the main display. 
    // To capture both a main and secondary display at the same time, use two active 
    // capture sessions, one for each display. On Mac OS X, AVCaptureMovieFileOutput 
    // only supports writing to a single video track. 
    CGDirectDisplayID displayId = kCGDirectMainDisplay; 

    // Create a ScreenInput with the display and add it to the session 
    AVCaptureScreenInput *input = [[AVCaptureScreenInput alloc] initWithDisplayID:displayId]; 
    input.minFrameDuration = CMTimeMake(1, 60); 

    //if (!input) { 
    // [mSession release]; 
    // mSession = nil; 
    // return; 
    //} 
    if ([mSession canAddInput:input]) { 
     NSLog(@"Added screen capture input"); 
     [mSession addInput:input]; 
    } else { 
     NSLog(@"Couldn't add screen capture input"); 
    } 

    //**********************Add output here 
    //dispatch_queue_t _videoDataOutputQueue; 
    //_videoDataOutputQueue = dispatch_queue_create("com.apple.sample.capturepipeline.video", DISPATCH_QUEUE_SERIAL); 
    //dispatch_set_target_queue(_videoDataOutputQueue, dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0)); 

    AVCaptureVideoDataOutput *videoOut = [[AVCaptureVideoDataOutput alloc] init]; 
    videoOut.videoSettings = @{ (id)kCVPixelBufferPixelFormatTypeKey : @(kCVPixelFormatType_32BGRA) }; 
    [videoOut setSampleBufferDelegate:self queue:dispatch_get_main_queue()]; 

    // RosyWriter records videos and we prefer not to have any dropped frames in the video recording. 
    // By setting alwaysDiscardsLateVideoFrames to NO we ensure that minor fluctuations in system load or in our processing time for a given frame won't cause framedrops. 
    // We do however need to ensure that on average we can process frames in realtime. 
    // If we were doing preview only we would probably want to set alwaysDiscardsLateVideoFrames to YES. 
    videoOut.alwaysDiscardsLateVideoFrames = YES; 

    if ([mSession canAddOutput:videoOut]) { 
     NSLog(@"Added output video"); 
     [mSession addOutput:videoOut]; 
    } else {NSLog(@"Couldn't add output video");} 


    // Start running the session 
    [mSession startRunning]; 

    NSLog(@"Set up session"); 
} 




- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection 
{ 
    //NSLog(@"Captures output from sample buffer"); 
    //CMFormatDescriptionRef formatDescription = CMSampleBufferGetFormatDescription(sampleBuffer); 
/* 
     if (self.outputVideoFormatDescription == nil) { 
      // Don't render the first sample buffer. 
      // This gives us one frame interval (33ms at 30fps) for setupVideoPipelineWithInputFormatDescription: to complete. 
      // Ideally this would be done asynchronously to ensure frames don't back up on slower devices. 
      [self setupVideoPipelineWithInputFormatDescription:formatDescription]; 
     } 
     else {*/ 
      [self renderVideoSampleBuffer:sampleBuffer]; 
     //} 
} 

- (void)renderVideoSampleBuffer:(CMSampleBufferRef)sampleBuffer 
{ 
    //CVPixelBufferRef renderedPixelBuffer = NULL; 
    //CMTime timestamp = CMSampleBufferGetPresentationTimeStamp(sampleBuffer); 

    //[self calculateFramerateAtTimestamp:timestamp]; 

    // We must not use the GPU while running in the background. 
    // setRenderingEnabled: takes the same lock so the caller can guarantee no GPU usage once the setter returns. 
    //@synchronized(_renderer) 
    //{ 
    // if (_renderingEnabled) { 
    CVPixelBufferRef sourcePixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); 

    const int kBytesPerPixel = 4; 

    CVPixelBufferLockBaseAddress(sourcePixelBuffer, 0); 

    int bufferWidth = (int)CVPixelBufferGetWidth(sourcePixelBuffer); 
    int bufferHeight = (int)CVPixelBufferGetHeight(sourcePixelBuffer); 
    size_t bytesPerRow = CVPixelBufferGetBytesPerRow(sourcePixelBuffer); 
    uint8_t *baseAddress = CVPixelBufferGetBaseAddress(sourcePixelBuffer); 

    int count = 0; 
    for (int row = 0; row < bufferHeight; row++) 
    { 
     uint8_t *pixel = baseAddress + row * bytesPerRow; 
     for (int column = 0; column < bufferWidth; column++) 
     { 
      count ++; 
      pixel[1] = 0; // De-green (second pixel in BGRA is green) 
      pixel += kBytesPerPixel; 
     } 
    } 

    CVPixelBufferUnlockBaseAddress(sourcePixelBuffer, 0); 


    //NSLog(@"Test Looped %d times", count); 

    CIImage *ciImage = [CIImage imageWithCVImageBuffer:sourcePixelBuffer]; 


    /* 
    CIContext *temporaryContext = [CIContext contextWithCGContext: 
              [[NSGraphicsContext currentContext] graphicsPort] 
                  options: nil]; 

    CGImageRef videoImage = [temporaryContext 
          createCGImage:ciImage 
          fromRect:CGRectMake(0, 0, 
               CVPixelBufferGetWidth(sourcePixelBuffer), 
               CVPixelBufferGetHeight(sourcePixelBuffer))]; 

    */ 

    //UIImage *uiImage = [UIImage imageWithCGImage:videoImage]; 

    // Create a bitmap rep from the image... 
    NSBitmapImageRep *bitmapRep = [[NSBitmapImageRep alloc] initWithCIImage:ciImage]; 
    // Create an NSImage and add the bitmap rep to it... 
    NSImage *image = [[NSImage alloc] init]; 
    [image addRepresentation:bitmapRep]; 
    // Set the output view to the new NSImage. 
    [imageView setImage:image]; 

    //CGImageRelease(videoImage); 



    //renderedPixelBuffer = [_renderer copyRenderedPixelBuffer:sourcePixelBuffer]; 
    // } 
    // else { 
    //  return; 
    // } 
    //} 

    //Profile code? See how fast it's running? 
    if (CACurrentMediaTime() - lastTime > 3) //10 seconds 
    { 
     float time = CACurrentMediaTime() - lastTime; 
     [fpsText setStringValue:[NSString stringWithFormat:@"Elapsed Time: %f ms, %f fps", time * 1000/loopsTaken, (1000.0)/(time * 1000.0/loopsTaken)]]; 
     lastTime = CACurrentMediaTime(); 
     loopsTaken = 0; 
     [self updateWindowList]; 
     if (leagueGameState.leaguePID == -1) { 
      [statusText setStringValue:@"No League Instance Found"]; 
     } 
    } 
    else 
    { 
     loopsTaken++; 
    } 

} 

我甚至通过数据循环后得到每秒的一个非常漂亮的60帧。

它捕获屏幕，我得到的数据，我修改数据，我重新显示数据。

Answer 1

你的意思是哪个“字节流”？ CGImage代表最终的位图数据，但仍然可能被压缩。位图可能当前存储在GPU上，因此获取它可能需要GPU-> CPU提取（这是昂贵的，并且当你不需要时应该避免）。

如果你想比30fps的更大做到这一点，你可能要重新考虑你如何攻击问题，并使用专为像核心图片，核心显卡，或金属工具。核心图形针对显示进行了优化，而不是处理（并且绝对不是实时处理）。 Core Image等工具的一个主要区别是，您可以在GPU上执行更多工作，而无需将数据重新混合回CPU。这对于维护快速管道来说是绝对关键的。只要有可能，你想避免得到实际的字节。

如果您已经有CGImage，可以使用imageWithCGImage:将其转换为CIImage，然后使用CIImage进一步处理它。如果您确实需要访问字节，您的选项就是您正在使用的选项，或者使用CGContextDrawImage将其呈现为位图上下文（这也需要复制）。没有任何承诺CGImage在任何给定的时间都有一堆位图字节可供您查看，并且它不提供“锁定缓冲区”方法，就像在Core Video等实时框架中发现的那样。

一些非常好的介绍给高速图像处理的WWDC视频：

• WWDC 2013届509核心图像效果和技术
• WWDC 2014届在核心的图像514个进展
• WWDC 2014会话603-605使用金属