SlideShare una empresa de Scribd logo
1 de 24
Descargar para leer sin conexión
Core Image



Mark Pavlidis
Co-founder
Flixel Photos Inc.

Toronto Cocoa and WebObjects Developers Group
2013/01/08
Motivation
Why use Core Image?
 •   Image processing and analysis
 •   Fast, efficient filters
 •   Auto enhancement
 •   Feature detection
 •   Still and “real-time” video
 •   Simple Objective-C API
Definitions
• Image Filter
 • a single transform or effect
 • built-in or loaded from an Image Unit plugin (OS X only)
• Framework
    <CoreImage/CoreImage.h>


• Key Classes
      CIImage, CIFilter, CIContext
Overview
Apply an image filter to the source pixel data



                       Monochrome
                          Filter




 • Pixel Accurate
 • Non-destructive
 • Filters can be chained together
Core Image Runtime
How it works
Core Image Runtime
How it works




 • Operates on image data types from:
   • Core Graphics
   • Core Video
   • Image I/O
Core Image Runtime
How it works




 • Built-in or plugin filters apply effects
 • Written in the Core Image Kernel Language
 • JIT complier assembles instruction pipeline
Core Image Runtime
How it works




 • Executed on the GPU or CPU
 • Low-level graphics processing encapsulated by CI
 • Performance: GLSL capabilities of the GPU or
   processing power of the CPU
Core Image Inputs
• Photo Library and Files
   imageWithContentsOfURL:

• Live Video Capture
   imageWithCVPixelBuffer:
   imageWithCVImageBuffer:

• In Memory
   imageWithCGImage:

• GL Texture
   imageWithTexture:size:flipped:colorSpace:
Core Image Outputs
• CGImageRef
 • Output to UIImage, ImageIO, ALAssetLibrary
   createCGImage:fromRect:

• CAEAGLLayer
   drawImage:inRect:fromRect:
• CVPixelBufferRef
   render:toCVPixelBuffer:

• Bitmap
   render:toBitmap:rowBytes:bounds:format:colorSpace:
Simple Filter
// Create a CIImage
CIImage *ciImage = [CIImage imageWithContentsOfURL:myURL];



// Create a CIFilter
CIFilter *filter = [CIFilter
                  filterWithName:@”CIColorMonochrome”];
[filter setValue:ciImage forKey:kCIInputImageKey];
[filter setValue:@(0.5) forKey:@”inputIntensity];



// Render the filter output image into a UIImage
UIImage *uiImage = [UIImage
                   imageWithCIImage:filter.outputImage];
Filter Chains
// Filter 1
CIImage *output = [CIFilter
                   filterWithName:@”CIColorMonochrome”]
                   keysAndValues:
                   kCIInputImageKey, ciImage,
                   @”inputIntensity, @(0.5),
                   nil].outputImage;

// Filter 2
output = [CIFilter
          filterWithName:@”CIVignette”] keysAndValues:
          kCIInputImageKey, output,
          @”inputIntensity, @(0.5),
          @”inputRadius, @(1.5),
          nil].outputImage;

•CI defers pixel processing until render is requested
•CI optimizes the render graph (e.g., sepia then scale)
Math!
• An image filter is kernel
• Output is the convolution of the source & kernel
      Source                                 Output
                           Kernel

  2     2      2       1     1      1

  2     1      3      -1     2      1          6

  2     2      1      -1     -1     1




• Commutative, associative, distributive, associative
 with scalar multiplication, multiplicative identity
Demo - QuartzComposer
Demo - Simple filter
Tips and Best Practices
 • CIImage and CIFilter are autoreleased
    • Use autorelease pools
 • CIImage, CIContext are immutable and thread-safe,
  CIFilter is not
 • CIContext is expensive
    • Stores a lot of state information so reuse them
 • Avoid CA animations when rendering on GPU
 • Use smaller images when possible
    • Use Core Animation to upscale view, texture, framebuffer
 • Disable colour management
    • If real-time performance and/or reduced quality is
      unnoticeable
Face Detection
CIDetector
• Identifies rectangles that contain human faces
• Feature detection
    • eyes & mouth position, tracking ID and frame count (video)

 CIContext *context = [CIContext
                       contextWithOptions:nil];
 NSDictionary *opts = @{CIDetectorAccuracy,
 !    ! ! ! ! ! ! CIDetectorAccuracyHigh};
 CIDetector   *detector = [CIDetector
 !    ! !     ! ! ! detectorOfType:CIDetectorTypeFace
 !    ! !     ! ! ! ! !       context:context
 !    ! !     ! ! ! ! !       options:opts];
 NSArray *features = [detector featuresInImage:myImage
 !    ! ! ! ! !                        options:opts];
Demo - Redacted Faces
OS X and iOS Differences
Portability
 • Some CIImage create methods differ
 • Has more built-in filters
 • Permits custom filter plugins
 • filter.outputImage is iOS only, use
   [filter valueForKey:kCIOutputImage]
 • Remember to [filter setDefaults]
 • All input parameter keys have strings defined
Alternatives
• GPUImage (iOS)
   ✓Hand-tuned OpenGL ES shaders that are much faster than CI
   ✓Custom filters
   - Lacks advanced features of Core Image
• Aviary
   ✓ Everyone is using it (Twitter, Flickr)
   - Everyone is using it
   • Only still images
• Hardcore - Write your own GL shaders
   • Don’t be silly
Summary
Core Image


 • Fast and efficient image processing framework
 • Simple Objective-C API
 • Encapsulates the low-level instruction details
 • Many built-in filters
 • Advanced processing features:
    • Face Detection
    • Auto Enhancement
    • Real-time video filters
More Information
• Core Image Programming Guide
• WWDC 2012 Videos
    •   Session 510 - Getting Started with Core Image

    •   Session 511 - Core Image Techniques

• Sample Code
 • https://github.com/mhpavl/FilterDemo
 • https://github.com/mhpavl/RedactedFaces
 • Core Image Funhouse (TBD)
[self release];
 Mark Pavlidis
 mark@pavlidis.ca
 Twitter: @mhp
 ADN: @mhp
Core Image

Más contenido relacionado

La actualidad más candente

Xamarin 9/10 San Diego Meetup
Xamarin 9/10 San Diego MeetupXamarin 9/10 San Diego Meetup
Xamarin 9/10 San Diego MeetupSeamgen
 
Design your 3d game engine
Design your 3d game engineDesign your 3d game engine
Design your 3d game engineDaosheng Mu
 
【Unite 2017 Tokyo】Unity5.6での2D新機能解説
【Unite 2017 Tokyo】Unity5.6での2D新機能解説【Unite 2017 Tokyo】Unity5.6での2D新機能解説
【Unite 2017 Tokyo】Unity5.6での2D新機能解説Unity Technologies Japan K.K.
 
OGDC2013_ Spine Animation_ Mr Alviss Ha
OGDC2013_ Spine Animation_ Mr Alviss HaOGDC2013_ Spine Animation_ Mr Alviss Ha
OGDC2013_ Spine Animation_ Mr Alviss Haogdc
 
Gdc 14 bringing unreal engine 4 to open_gl
Gdc 14 bringing unreal engine 4 to open_glGdc 14 bringing unreal engine 4 to open_gl
Gdc 14 bringing unreal engine 4 to open_glchangehee lee
 

La actualidad más candente (8)

Xamarin 9/10 San Diego Meetup
Xamarin 9/10 San Diego MeetupXamarin 9/10 San Diego Meetup
Xamarin 9/10 San Diego Meetup
 
Design your 3d game engine
Design your 3d game engineDesign your 3d game engine
Design your 3d game engine
 
【Unite 2017 Tokyo】Unity5.6での2D新機能解説
【Unite 2017 Tokyo】Unity5.6での2D新機能解説【Unite 2017 Tokyo】Unity5.6での2D新機能解説
【Unite 2017 Tokyo】Unity5.6での2D新機能解説
 
OGDC2013_ Spine Animation_ Mr Alviss Ha
OGDC2013_ Spine Animation_ Mr Alviss HaOGDC2013_ Spine Animation_ Mr Alviss Ha
OGDC2013_ Spine Animation_ Mr Alviss Ha
 
Android - Graphics Animation in Android
Android - Graphics Animation in AndroidAndroid - Graphics Animation in Android
Android - Graphics Animation in Android
 
ProjectsSummary
ProjectsSummaryProjectsSummary
ProjectsSummary
 
Gl efficiency
Gl efficiencyGl efficiency
Gl efficiency
 
Gdc 14 bringing unreal engine 4 to open_gl
Gdc 14 bringing unreal engine 4 to open_glGdc 14 bringing unreal engine 4 to open_gl
Gdc 14 bringing unreal engine 4 to open_gl
 

Similar a Core Image

Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)Chris Adamson
 
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual Machine
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual MachineDino2 - the Amazing Evolution of the VA Smalltalk Virtual Machine
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual MachineESUG
 
Core Animation
Core AnimationCore Animation
Core AnimationBob McCune
 
High resolution animated scenes from stills
High resolution animated scenes from stillsHigh resolution animated scenes from stills
High resolution animated scenes from stillsCarolyn Rose
 
Hi performance table views with QuartzCore and CoreText
Hi performance table views with QuartzCore and CoreTextHi performance table views with QuartzCore and CoreText
Hi performance table views with QuartzCore and CoreTextMugunth Kumar
 
AISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the EdgeAISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the EdgeBill Liu
 
iOS Game Development With UIKit
iOS Game Development With UIKitiOS Game Development With UIKit
iOS Game Development With UIKitMartin Grider
 
Image Conversion Library
Image Conversion LibraryImage Conversion Library
Image Conversion LibraryIshita Gupta
 
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...Gerke Max Preussner
 
Ci for i-os-codemash-01.2013
Ci for i-os-codemash-01.2013Ci for i-os-codemash-01.2013
Ci for i-os-codemash-01.2013Kevin Munc
 
TMD2063 | Digital Animation - Chapter 3
TMD2063 | Digital Animation - Chapter 3TMD2063 | Digital Animation - Chapter 3
TMD2063 | Digital Animation - Chapter 3Diyana Harithuddin
 
Dimond recognition system
Dimond recognition systemDimond recognition system
Dimond recognition systemAmit Gandhi
 
Core image presentation
Core image presentationCore image presentation
Core image presentationKyle Stewart
 
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architectureIEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architectureSebastien Kuntz
 
Masterin Large Scale Java Script Applications
Masterin Large Scale Java Script ApplicationsMasterin Large Scale Java Script Applications
Masterin Large Scale Java Script ApplicationsFabian Jakobs
 
Motion design in FIori
Motion design in FIoriMotion design in FIori
Motion design in FIoriRoman Rommel
 

Similar a Core Image (20)

Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
Core Image: The Most Fun API You're Not Using (CocoaConf Columbus 2014)
 
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual Machine
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual MachineDino2 - the Amazing Evolution of the VA Smalltalk Virtual Machine
Dino2 - the Amazing Evolution of the VA Smalltalk Virtual Machine
 
Core Animation
Core AnimationCore Animation
Core Animation
 
High resolution animated scenes from stills
High resolution animated scenes from stillsHigh resolution animated scenes from stills
High resolution animated scenes from stills
 
Hi performance table views with QuartzCore and CoreText
Hi performance table views with QuartzCore and CoreTextHi performance table views with QuartzCore and CoreText
Hi performance table views with QuartzCore and CoreText
 
Cocos2d programming
Cocos2d programmingCocos2d programming
Cocos2d programming
 
AISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the EdgeAISF19 - Unleash Computer Vision at the Edge
AISF19 - Unleash Computer Vision at the Edge
 
iOS Game Development With UIKit
iOS Game Development With UIKitiOS Game Development With UIKit
iOS Game Development With UIKit
 
Image Conversion Library
Image Conversion LibraryImage Conversion Library
Image Conversion Library
 
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...
GDC Europe 2014: Unreal Engine 4 for Programmers - Lessons Learned & Things t...
 
Ci for i-os-codemash-01.2013
Ci for i-os-codemash-01.2013Ci for i-os-codemash-01.2013
Ci for i-os-codemash-01.2013
 
Animation in iOS
Animation in iOSAnimation in iOS
Animation in iOS
 
TMD2063 | Digital Animation - Chapter 3
TMD2063 | Digital Animation - Chapter 3TMD2063 | Digital Animation - Chapter 3
TMD2063 | Digital Animation - Chapter 3
 
Dimond recognition system
Dimond recognition systemDimond recognition system
Dimond recognition system
 
Core image presentation
Core image presentationCore image presentation
Core image presentation
 
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architectureIEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
IEEE VR-SEARIS 2014 Keynote - MiddleVR - Philosophy and architecture
 
Masterin Large Scale Java Script Applications
Masterin Large Scale Java Script ApplicationsMasterin Large Scale Java Script Applications
Masterin Large Scale Java Script Applications
 
Introduction to OCI Image Technologies Serving Container
Introduction to OCI Image Technologies Serving ContainerIntroduction to OCI Image Technologies Serving Container
Introduction to OCI Image Technologies Serving Container
 
Diagnosing issues in your ASP.NET applications in production with Visual Stud...
Diagnosing issues in your ASP.NET applications in production with Visual Stud...Diagnosing issues in your ASP.NET applications in production with Visual Stud...
Diagnosing issues in your ASP.NET applications in production with Visual Stud...
 
Motion design in FIori
Motion design in FIoriMotion design in FIori
Motion design in FIori
 

Último

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 

Último (20)

Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 

Core Image

  • 1. Core Image Mark Pavlidis Co-founder Flixel Photos Inc. Toronto Cocoa and WebObjects Developers Group 2013/01/08
  • 2. Motivation Why use Core Image? • Image processing and analysis • Fast, efficient filters • Auto enhancement • Feature detection • Still and “real-time” video • Simple Objective-C API
  • 3. Definitions • Image Filter • a single transform or effect • built-in or loaded from an Image Unit plugin (OS X only) • Framework <CoreImage/CoreImage.h> • Key Classes CIImage, CIFilter, CIContext
  • 4. Overview Apply an image filter to the source pixel data Monochrome Filter • Pixel Accurate • Non-destructive • Filters can be chained together
  • 6. Core Image Runtime How it works • Operates on image data types from: • Core Graphics • Core Video • Image I/O
  • 7. Core Image Runtime How it works • Built-in or plugin filters apply effects • Written in the Core Image Kernel Language • JIT complier assembles instruction pipeline
  • 8. Core Image Runtime How it works • Executed on the GPU or CPU • Low-level graphics processing encapsulated by CI • Performance: GLSL capabilities of the GPU or processing power of the CPU
  • 9. Core Image Inputs • Photo Library and Files imageWithContentsOfURL: • Live Video Capture imageWithCVPixelBuffer: imageWithCVImageBuffer: • In Memory imageWithCGImage: • GL Texture imageWithTexture:size:flipped:colorSpace:
  • 10. Core Image Outputs • CGImageRef • Output to UIImage, ImageIO, ALAssetLibrary createCGImage:fromRect: • CAEAGLLayer drawImage:inRect:fromRect: • CVPixelBufferRef render:toCVPixelBuffer: • Bitmap render:toBitmap:rowBytes:bounds:format:colorSpace:
  • 11. Simple Filter // Create a CIImage CIImage *ciImage = [CIImage imageWithContentsOfURL:myURL]; // Create a CIFilter CIFilter *filter = [CIFilter filterWithName:@”CIColorMonochrome”]; [filter setValue:ciImage forKey:kCIInputImageKey]; [filter setValue:@(0.5) forKey:@”inputIntensity]; // Render the filter output image into a UIImage UIImage *uiImage = [UIImage imageWithCIImage:filter.outputImage];
  • 12. Filter Chains // Filter 1 CIImage *output = [CIFilter filterWithName:@”CIColorMonochrome”] keysAndValues: kCIInputImageKey, ciImage, @”inputIntensity, @(0.5), nil].outputImage; // Filter 2 output = [CIFilter filterWithName:@”CIVignette”] keysAndValues: kCIInputImageKey, output, @”inputIntensity, @(0.5), @”inputRadius, @(1.5), nil].outputImage; •CI defers pixel processing until render is requested •CI optimizes the render graph (e.g., sepia then scale)
  • 13. Math! • An image filter is kernel • Output is the convolution of the source & kernel Source Output Kernel 2 2 2 1 1 1 2 1 3 -1 2 1 6 2 2 1 -1 -1 1 • Commutative, associative, distributive, associative with scalar multiplication, multiplicative identity
  • 15. Demo - Simple filter
  • 16. Tips and Best Practices • CIImage and CIFilter are autoreleased • Use autorelease pools • CIImage, CIContext are immutable and thread-safe, CIFilter is not • CIContext is expensive • Stores a lot of state information so reuse them • Avoid CA animations when rendering on GPU • Use smaller images when possible • Use Core Animation to upscale view, texture, framebuffer • Disable colour management • If real-time performance and/or reduced quality is unnoticeable
  • 17. Face Detection CIDetector • Identifies rectangles that contain human faces • Feature detection • eyes & mouth position, tracking ID and frame count (video) CIContext *context = [CIContext contextWithOptions:nil]; NSDictionary *opts = @{CIDetectorAccuracy, ! ! ! ! ! ! ! CIDetectorAccuracyHigh}; CIDetector *detector = [CIDetector ! ! ! ! ! ! detectorOfType:CIDetectorTypeFace ! ! ! ! ! ! ! ! context:context ! ! ! ! ! ! ! ! options:opts]; NSArray *features = [detector featuresInImage:myImage ! ! ! ! ! ! options:opts];
  • 19. OS X and iOS Differences Portability • Some CIImage create methods differ • Has more built-in filters • Permits custom filter plugins • filter.outputImage is iOS only, use [filter valueForKey:kCIOutputImage] • Remember to [filter setDefaults] • All input parameter keys have strings defined
  • 20. Alternatives • GPUImage (iOS) ✓Hand-tuned OpenGL ES shaders that are much faster than CI ✓Custom filters - Lacks advanced features of Core Image • Aviary ✓ Everyone is using it (Twitter, Flickr) - Everyone is using it • Only still images • Hardcore - Write your own GL shaders • Don’t be silly
  • 21. Summary Core Image • Fast and efficient image processing framework • Simple Objective-C API • Encapsulates the low-level instruction details • Many built-in filters • Advanced processing features: • Face Detection • Auto Enhancement • Real-time video filters
  • 22. More Information • Core Image Programming Guide • WWDC 2012 Videos • Session 510 - Getting Started with Core Image • Session 511 - Core Image Techniques • Sample Code • https://github.com/mhpavl/FilterDemo • https://github.com/mhpavl/RedactedFaces • Core Image Funhouse (TBD)
  • 23. [self release]; Mark Pavlidis mark@pavlidis.ca Twitter: @mhp ADN: @mhp