Notes on Rendering 2D Graphics on a Mac

By Evan Miller

February 5, 2011

I’ve spent a lot of time writing a simple maps program for Mac. Apple has several technologies available for rendering graphics, and I thought I’d offer up some notes on my experiences with them. I am performing this service because Apple is utterly without shame in promoting their technologies in their technical documentation, and it can be difficult to determine in advance whether it is best to use Cocoa Drawing, Quartz 2D, Core Animation, Core Image, or some combination for a particular task.

My task was to render 3,000 vector images (U.S. counties) with fill colors that might change when the user clicks on a new data set. Pretty simple.

The goal: draw this quickly

Phase 1: Cocoa Drawing

I first wrote the drawing routines with Cocoa (NSBezierPath and the like). However, I ran into several issues:

  1. The shapes sometimes differed based on the fill color. I.e., county borders sometimes appeared to move for no reason. For applications that are remotely scientific, this behavior is completely unacceptable.

  2. Even after I reduced the complexity of the shapes to increase performance, drawing 3,000 counties still took about 750 milliseconds. From the user’s perspective, the program seemed slow and bloated.

  3. Cocoa Drawing seemed to allocate a lot of memory buffers for its own mysterious uses.

With these issues in mind, I heard the siren song of the GPU. The graphics card should be able to help with graphics programming… right?

Phase 2: Core Image

OpenGL seemed like a pain to learn, so I decided to check out Apple’s Core Image, which supposedly lets you take advantage of the graphics card without taking time to learn OpenGL.

Apple is absolutely relentless in promoting the virtues of Core Image in their technical documentation. Here are some examples, taken from their Core Image Programming Guide, along with my commentary:

“This chapter… discusses how Core Image works behind the scenes to achieve fast, stunning, near real-time image processing.”

(I’m sorry, the word “stunning” does not belong in technical documentation… unless it is for a Taser.)

“Core Image operations are opaque to you; your software just works.”

(You haven’t seen my software!)

“Lazy evaluation is one of the practices that makes Core Image fast and efficient.”
“… it actually minimizes the number of pixels used in any calculation.”
“The compiler performs CSE and peephole optimization.”

I have no clue what that last one means, but it sounds amazing and I want it.

“Your application benefits from Core Image caching without needing to know the details of how caching is implemented.”
“Again, you don’t need to concern yourself with the details of the compilation techniques.”

Thank you, team of patronizing technical writers.

So: I bought it. I rewrote my drawing routines to cache each county image and repaint it on demand using Core Image.

This was a mistake. Drawing 3,000 counties slowed down from 750 milliseconds (with Cocoa Drawing) to about 3 full seconds using Core Image. What?? What about the peephole optimization and real-time image processing?? What about me not knowing needing to know the details of GPU programming and my software Just Working?? Apple, help!!

Well, here’s what Apple doesn’t tell you. You actually need to understand how graphics cards work before you sit down and try to use their GPU libraries. To understand when GPU programming is and is not appropriate, there are really only two numbers you need to know:

  1. A round-trip to the GPU takes about a millisecond.

  2. The throughput to the GPU is about a gigabyte per second.

The problem I hit was this: Core Image works great if you have one image to manipulate, but if you have 3,000 images, it’s a disaster. Core Image was waiting for each image to return to ask for the next one. There was no way to pipeline 3,000 county images. For that you’d need good old OpenGL. And at a millisecond a pop, rendering all 3,000 pushed the rendering time up to 3 seconds. So much for Core Image.

Phase 3: Core Animation

Next up was Core Animation, which I figured might take care of the graphics using its own GPU magic and give me some nice transition effects to boot. Their technical documentation promises:

“Using Core Animation, developers can create dynamic user interfaces for their applications without having to use low-level graphics APIs such as OpenGL to get respectable animation performance.”
“[It has a] lightweight data structure. You can display and animate hundreds of layers simultaneously.”
“Improved application performance.”
“And, not only does it take full advantage of today’s hardware, it will be able to take full advantage of tomorrow’s hardware as well.”

So I rewrote the drawing code to use Core Animation, with one layer per county. I clipped around each county border, set the interior to be transparent, and colored the county by changing the layer’s background color. I thought it was a pretty clever approach, and the result looked nice, but:

  1. Application start-up took several seconds.

  2. With 3,000 layers, the animation was too jerky to be usable

  3. The program used up an additional 300 MB of RAM.

So much for “a lightweight data structure” and “improved application performance”. Perhaps Apple should update the documentation to say:

“Not only will it eat up all the RAM in today’s hardware, but it will eat up all the RAM in tomorrow’s hardware as well.”

I suppose I could afford the RAM, but the long start-up time was killing me. Core Animation was out the window.

Phase 4: Quartz 2D (Core Graphics)

At this point I was ready to come crawling back to Cocoa Drawing. But since Core Image and Core Animation had required me to rewrite the routines in Quartz 2D (a C API), I figured I would just stick with that instead of moving back to the Objective-C Cocoa API.

And guess what? Quartz 2D by itself worked pretty well. The drawing time was down to about 150 milliseconds, which was acceptable from a usability perspective, and the RAM usage wasn’t bad. Furthermore, the lines seemed cleaner than Cocoa Drawing had rendered them, and the county borders didn’t move when I changed the fill color.

I still had to employ some tricks to make the application feel responsive when you resize the window or draw a selection box (for this, Apple’s Drawing Performance Guidelines are actually quite helpful). But overall, I am impressed. I was tempted by the promise of GPU-accelerated graphics as delivered by Core Image or Core Animation, but these technologies are oversold.

The moral of the story is: Core Image is designed to process a small number of large images, but is ill-suited to process a large number of small images. Core Animation is able to process a large number of small images quickly, but Core Animation is costly in terms of memory consumption and initialization time.

For now, I’m sticking with plain C and Quartz 2D.

You’re reading evanmiller.org, a random collection of math, tech, and musings. If you liked this you might also enjoy:

Get new articles as they’re published, via LinkedIn, Twitter, or RSS.

Want to look for statistical patterns in your MySQL, PostgreSQL, or SQLite database? My desktop statistics software Wizard can help you analyze more data in less time and communicate discoveries visually without spending days struggling with pointless command syntax. Check it out!


Wizard
Statistics the Mac way

Back to Evan Miller’s home page – Subscribe to RSS – LinkedIn – Twitter