Learning iPhone's OpenGL ES | Forays Into iPhone OpenGL ES | Serg Koren

Pointers

Tue, 04 Aug 2009 09:32:29 -0400

I guess I'm at a crossroads here. The stuff I've presented should be enough to kick-start you into OpenGL ES on the iPhone. There is lots more to learn, however. The problem is that I don't want to take on examples that have already been done elsewhere. There are a ton of OpenGL and nowadays OpenGL ES on iPhone sites. Even the Wikipedia has mini-tutorial like pages on OpenGL coding. Also, I'm deep into real work and my own iPhone app development. So the crossroads is, where do I go from here with this blog. I never intended it to be a full-time blog, just one that would share my own learning experiences. However, I don't want to drop it completely.

I've had some interest and questions from a few people as part of this series. I thank you for the support !

So, I'll leave it up to you. What would you like to see me tackle in terms of OpenGL ES. Something manageable, please. Email me and let me know what you think. If I get any responses I'll tackle them as my time permits.

In any event, here are some other good online sources for OpenGL information:

Wikipedia - just search for OpenGL ES topics

Open GL ES Reference Manual - good reference source for when you want the details.

NeHe Productions - news, great tutorials, game coding

Apple - sample code, tools, etc.

iCodeBlog - good getting-started iPhone tutorials, some OpenGL ES stuff

ManiacDev - other pointers. Good source.

So that's it for this time around. Let me know what you would like to see. Otherwise, thanks again, and see you next time.

Part VI - GL Paint Dissected - EAGLView.m

Tue, 07 Jul 2009 10:10:08 -0400

Last installment I went through the header file for the EAGLView. We saw that it was actually fairly straightforward with the addition of a delegate declarations, a depth buffer and some utility functions to convert coordinates. Let's dive into the implementation and see how these are used.

The first thing we see are the property synthesizers followed by a class method:

@implementation EAGLView

@synthesize delegate=_delegate, autoresizesSurface=_autoresize, surfaceSize=_size, framebuffer = _framebuffer, pixelFormat = _format, depthFormat = _depthFormat, context = _context;

+ (Class) layerClass

{ return [CAEAGLLayer class]; } This just returns the class of the CAEAGLLayer. Next: - (BOOL) _createSurface { CAEAGLLayer* eaglLayer = (CAEAGLLayer*)[self layer]; CGSize newSize; GLuint oldRenderbuffer; GLuint oldFramebuffer; if(![EAGLContext setCurrentContext:_context]) { return NO; } This method creates a surface to draw on, and returns a flag indictating whether it was created or not. The first attribute eagllayer points to our layer. We then have a size and buffer pointers for rendering and our display buffer. If we can't set a current context we return false (NO). newSize = [eaglLayer bounds].size; newSize.width = roundf(newSize.width); newSize.height = roundf(newSize.height); Here we set the size of the layer we are creating to our bounds rectangle. glGetIntegerv(GL_RENDERBUFFER_BINDING_OES, (GLint *) &oldRenderbuffer); glGetIntegerv(GL_FRAMEBUFFER_BINDING_OES, (GLint *) &oldFramebuffer); We cast our pointers to integers using an OpenGL utility function glGetIntegerv (v stands for value). glGenRenderbuffersOES(1, &_renderbuffer); glBindRenderbufferOES(GL_RENDERBUFFER_OES, _renderbuffer); This creates a new rendering buffer and binds to it. if(![_context renderbufferStorage:GL_RENDERBUFFER_OES fromDrawable:(id<EAGLDrawable>)eaglLayer]) { glDeleteRenderbuffersOES(1, &_renderbuffer); glBindRenderbufferOES(GL_RENDERBUFFER_BINDING_OES, oldRenderbuffer); return NO; }

We then see if we can get the storage for the render buffer in our layer and if we can't we delete the renderbuffer we just created, unbind it, and return false from the method. If we can't get the storage for the renderbuffer, we can't create our layer. As a reminder, layers are related to the depthbuffer we saw in the declarations.

glGenFramebuffersOES(1, &_framebuffer);

glBindFramebufferOES(GL_FRAMEBUFFER_OES, _framebuffer);

If we can get the renderbuffer set up we then generate a framebuffer and bind the pointer.

if (_depthFormat) {

glGenRenderbuffersOES(1, &_depthBuffer); glBindRenderbufferOES(GL_RENDERBUFFER_OES, _depthBuffer); glRenderbufferStorageOES(GL_RENDERBUFFER_OES, _depthFormat, newSize.width, newSize.height); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES, GL_RENDERBUFFER_OES, _depthBuffer); } If we're using a depthbuffer (see the the last installment), we generate and bind a depth buffer in the first two lines in the 'if' statement. Next we get the framebuffer's storage, and attach the framebuffer to the renderbuffer. This essentially associates the two buffers so we can handle them together. _size = newSize; if(!_hasBeenCurrent) { glViewport(0, 0, newSize.width, newSize.height); glScissor(0, 0, newSize.width, newSize.height); _hasBeenCurrent = YES; } else { glBindFramebufferOES(GL_FRAMEBUFFER_OES, oldFramebuffer); } glBindRenderbufferOES(GL_RENDERBUFFER_OES, oldRenderbuffer); Next we save off the size of the layer, and check our one-time flag (see the last installment). If the flag is not set (first time in) we set our viewport. The viewport is the virtual window we are looking through onto our scene. In this case, the viewport is the size of our layer...we see our entire layer head on. The glScissor call creates something known as a "scissor box". The scissor box is more commonly known as a "drawing clipping rectangle". Things will only be drawn in the bounds of the scissor box even if what we draw goes outside the box. If this isn't the first time, we bind the framebuffer to our "old" buffer. In either case we bind the renderbuffer to the "old" buffer.

// Error handling here

[_delegate didResizeEAGLSurfaceForView:self];

return YES;

Strangely, there is a comment for error handling but there is none. We then invoke our delegate method to inform it we've resized our surface. At this point the surface is created and we return true.

Next, we have a complimentary method that destroys a surface (one we created above).

- (void) _destroySurface

{ EAGLContext *oldContext = [EAGLContext currentContext]; if (oldContext != _context) [EAGLContext setCurrentContext:_context]; if(_depthFormat) { glDeleteRenderbuffersOES(1, &_depthBuffer); _depthBuffer = 0; } glDeleteRenderbuffersOES(1, &_renderbuffer); _renderbuffer = 0; glDeleteFramebuffersOES(1, &_framebuffer); _framebuffer = 0; if (oldContext != _context) [EAGLContext setCurrentContext:oldContext]; } The first thing that happens here is we save our current context. IF the current context isn't our old previous context, we make the previous context current. If we have a depthbuffer, we delete it. We delete our render and framebuffers. We then restore the context we saved off in the first line as the current context. - (id) initWithFrame:(CGRect)frame { return [self initWithFrame:frame pixelFormat:GL_RGB565_OES depthFormat:0 preserveBackbuffer:NO]; } - (id) initWithFrame:(CGRect)frame pixelFormat:(GLuint)format { return [self initWithFrame:frame pixelFormat:format depthFormat:0 preserveBackbuffer:NO]; } We have some initializers here. These allow us to create our EAGLEView using different initializers. The next initWithFrame is a bit more complex so lets go through it. - (id) initWithFrame:(CGRect)frame pixelFormat:(GLuint)format depthFormat:(GLuint)depth preserveBackbuffer:(BOOL)retained { if((self = [super initWithFrame:frame])) { CAEAGLLayer* eaglLayer = (CAEAGLLayer*)[self layer]; The first line here just calls one of the previous init methods passing in our layer. eaglLayer.drawableProperties = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithBool:YES], kEAGLDrawablePropertyRetainedBacking, kEAGLColorFormatRGBA8, kEAGLDrawablePropertyColorFormat, nil]; This sets up the layer's drawable properties. The two main ones sets up the ability to save off our buffer for later use (we do this in RGBA8 (bit) form). The kEAGLDrawablePropertyColorFormat specifies a color object to use (nil in this case). So we aren't using an internal color buffer. _format = format; _depthFormat = depth; Save off the format and depth. _context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES1]; if(_context == nil) { [self release]; return nil; } Create an EAGL context using initWithAPI: This call initializes an EAGL context and specifies that the context conforms to OpenGL ES 1.0 calls. If we can't get a context clean up and return. Finally, try creating a surface (using the method above) and return an initialized view. if(![self _createSurface]) { [self release]; return nil; } } return self; } The dealloc call cleans up our view by destroying the surface and releasing the context. - (void) dealloc { [self _destroySurface]; [_context release]; _context = nil; [super dealloc]; } The next method: - (void) layoutSubviews { CGRect bounds = [self bounds]; if(_autoresize && ((roundf(bounds.size.width) != _size.width) || (roundf(bounds.size.height) != _size.height))) { [self _destroySurface]; [self _createSurface]; } } checks to see if autoresize is set and if it is the size has been changed. If it has we recreate the drawing surface in our view. - (void) setAutoresizesEAGLSurface:(BOOL)autoresizesEAGLSurface; { _autoresize = autoresizesEAGLSurface; if(_autoresize) [self layoutSubviews]; } The setAutoresizesEAGLSurface controls whether the surface autoresize or not based on the flag passed in. The next three methods are utility methods for dealing with the context: - (void) setCurrentContext { if(![EAGLContext setCurrentContext:_context]) { printf("Failed to set current context %p in %s\n", _context, __FUNCTION__); } } - (BOOL) isCurrentContext { return ([EAGLContext currentContext] == _context ? YES : NO); } - (void) clearCurrentContext { if(![EAGLContext setCurrentContext:nil]) printf("Failed to clear current context in %s\n", __FUNCTION__); } These are straightforward so I won't explain them. Next we have a method called swapBuffers. - (void) swapBuffers { EAGLContext *oldContext = [EAGLContext currentContext]; GLuint oldRenderbuffer; if(oldContext != _context) [EAGLContext setCurrentContext:_context]; The first thing this does is save off the current context and creates a local renderBuffer pointer. It then sees if the previous context (_context) the one we just saved. if not we make the previous one current. // CHECK_GL_ERROR(); glGetIntegerv(GL_RENDERBUFFER_BINDING_OES, (GLint *) &oldRenderbuffer); glBindRenderbufferOES(GL_RENDERBUFFER_OES, _renderbuffer); Again a comment with no code (a reminder) and we get the integer value of the old renderbuffer and save it in our local variable oldRenderbuffer. We then bind to the previous renderbuffer (_renderbuffer). This essentially makes the previous buffer the active one and we saved the current one--we swapped the render buffers. This might be a little confusing with "previous" and "current" all over the place. The key to understanding this is that in this example variables with an underscore as part of the name are "previous"ly active, whereas the names without the underscore are currently active. if(![_context presentRenderbuffer:GL_RENDERBUFFER_OES]) printf("Failed to swap renderbuffer in %s\n", __FUNCTION__); if(oldContext != _context) [EAGLContext setCurrentContext:oldContext]; } We check to see if the swap worked, and then we check our context and adjust our context accordingly. The final two methods are the utility methods which convert coordinates between the view and the surface systems: - (CGPoint) convertPointFromViewToSurface:(CGPoint)point { CGRect bounds = [self bounds]; return CGPointMake((point.x - bounds.origin.x) / bounds.size.width * _size.width, (point.y - bounds.origin.y) / bounds.size.height * _size.height); } - (CGRect) convertRectFromViewToSurface:(CGRect)rect { CGRect bounds = [self bounds]; return CGRectMake((rect.origin.x - bounds.origin.x) / bounds.size.width * _size.width, (rect.origin.y - bounds.origin.y) / bounds.size.height * _size.height, rect.size.width / bounds.size.width * _size.width, rect.size.height / bounds.size.height * _size.height); } @end These utility methods are not OpenGL so I'll let you figure them out. Ok, that's the implementation of the view. It just creates a view and manages the buffers and surface content. It's all the infrastructure to the drawing logic we saw in PaintingView.m That's it for the GLPaint example. We saw how to draw based on touch, how to replay stored drawing data, and how to set up the drawing environment. Again this example is pretty basic; it's not really game-quality. But, the example does provide some basic tools you might use in a game. So what's the next installment going to cover? I think I want to take a basic OpenGL game and take it apart to learn how a simple OpenGL game works. I'll try to find an example to go through for next time. As always if you have comments or suggestions, feel free to email or Twitter me to let me know what you think. Until next time...

Part V - GLPaint Dissected - EAGLView.h

Tue, 23 Jun 2009 10:06:37 -0400

Before I dive into this installment, let me mention some interesting news. Apple released iPhone 3.0 which has some news dealing with OpenGL ES. Namely, 3.0 supports OpenGL ES 2.0 which has a lot of cool capabilities over the older OpenGL ES 1.2. Additionally, 3.0 runs faster, and if you upgrade to an iPhone 3gs OpenGL ES runs up to about 4x faster! This means smoother animations and more power to animate. That's a very significant improvement in performance and great news for game developers who use OpenGL. You may also be wondering what this means as it relates to what we've been learning/going over. Actually, it doesn't mean anything since OpenGL ES 2.0 uses everything 1.2 does. So everything still works. So lets keep going!

In the previous installment we went through the PaintingView and saw how the app draws and animates the opening recorded writing. In this part I'll go into the EAGLView. We saw the version of EAGLView that comes with the iPhone SDK template. This version is a bit different and has been customized. Let's start with the header file.

@class EAGLView;

//PROTOCOLS:

@protocol EAGLViewDelegate <NSObject>

- (void) didResizeEAGLSurfaceForView:(EAGLView*)view; //Called whenever the EAGL surface has been resized

@end

The first thing we run into is a protocol definition which will handle resizing our EAGLView. We'll see under what conditions this gets called later when we get into the implementation code.

//CLASS INTERFACE:

/*

This class wraps the CAEAGLLayer from CoreAnimation into a convenient UIView subclass.

The view content is basically an EAGL surface you render your OpenGL scene into.

Note that setting the view non-opaque will only work if the EAGL surface has an alpha channel.

*/

@interface EAGLView : UIView

The comment pretty much explains the point of this view.  That's one nice thing about this sample is the abundance of good comments.

@private GLuint _format; GLuint _depthFormat; BOOL _autoresize; EAGLContext *_context; GLuint _framebuffer; GLuint _renderbuffer; GLuint _depthBuffer; CGSize _size; BOOL _hasBeenCurrent; id<EAGLViewDelegate> _delegate; } Here are the properties. The main thing to note here is that we declare three separate buffers. A framebuffer that will be the displayed image, a renderbuffer (our backbuffer) where we draw before we display, and a depthBuffer which as its name implies stores depth information. Depth is used to determine which pixels are "in front of" other pixels. Pixels that are "behind" other pixels don't get drawn on the screen since they are obscured by the pixels in "front". It's a way of optimizing the drawing of complex scenes using layers. the GLuint _depthFormat is essentially will be used as a flag to determine whether we store/use the depthbuffer. The boolean _hasBeenCurrent will be used as a toggle for some setup code that needs to be done once. We then have a variable to handle our delegate. Next we get to the methods: - (id) initWithFrame:(CGRect)frame; //These also set the current context - (id) initWithFrame:(CGRect)frame pixelFormat:(GLuint)format; - (id) initWithFrame:(CGRect)frame pixelFormat:(GLuint)format depthFormat:(GLuint)depth preserveBackbuffer:(BOOL)retained; These are just custom initializers that allow us to initialize this view different ways. We then have some property declarations: @property(readonly) GLuint framebuffer; @property(readonly) GLuint pixelFormat; @property(readonly) GLuint depthFormat; @property(readonly) EAGLContext *context; The next few methods appear to deal with resizing the view (and related somehow to the delegate). We'll see how when we dive into the implementation. @property BOOL autoresizesSurface; //NO by default - Set to YES to have the EAGL surface automatically resized when the view bounds change, otherwise the EAGL surface contents is rendered scaled @property(readonly, nonatomic) CGSize surfaceSize; @property(assign) id<EAGLViewDelegate> delegate; We then have some custom getter-setters for the context which won't be handled via a property. - (void) setCurrentContext; - (BOOL) isCurrentContext; - (void) clearCurrentContext; Our main animation method comes next: - (void) swapBuffers; //This also checks the current OpenGL error and logs an error if needed Finally, we have some utility methods that allow us to convert from iPhone to OpenGL ES coordinates. - (CGPoint) convertPointFromViewToSurface:(CGPoint)point; - (CGRect) convertRectFromViewToSurface:(CGRect)rect; @end

That wasn't too bad.  We covered a lot of common details in earlier installments, so you may want to review those to refresh your recollection.  Next time we'll start on the actual implementation of this EAGLView.  It has some new things but its a fairly short file overall.

Part IV - GLPaint Dissected - PaintingView

Sun, 17 May 2009 11:13:09 -0400

In the last installment I started taking PaintingView.m apart. We saw a lot of setup and how the brush shape is defined from an image. We also learned a bit about blending colors as they are drawn. This time around, I'll finish up PaingtingView.m.

Picking up from where we left off.

// Erases the screen
- (void) erase
{
	//Clear the buffer
	glClear(GL_COLOR_BUFFER_BIT);



	//Display the buffer
	[self swapBuffers];
}

As the comment indicates this is used to clear the screen. The first thing that this does is call glClear passing in GL_COLOR_BUFFER_BIT which is an OpenGL flag that indicates we want to clear all the buffers that are currently enabled for color drawing.  As we've gleaned, OpenGL ES uses a lot of different buffer types, there are other flags that you can pass into glClear to clear those other buffers (e.g., depth, stencil, etc.)  The next call calls our swapBuffers method which is defined in the EAGLEView.m file.  As we saw from the XCode OpenGL ES template teardown, animation involves swapping the currently displayed buffer with the one for the next frame in a loop.  So what is actually happening here in the "erase" method is that we clear the backing buffer, and swap our buffers to make the one we just cleared visible--the screen clears.

Next we get renderLineFromPoint:toPoint:

// Drawings a line onscreen based on where the user touches - (void) renderLineFromPoint:(CGPoint)start toPoint:(CGPoint)end { static GLfloat* vertexBuffer = NULL; static NSUInteger vertexMax = 64; NSUInteger vertexCount = 0, count, i; This sets up a pointer to a vertex buffer, and variables to track the maximum number and current vertex count. Remember vertexes are the "corners" of a drawing such as a square. But why do we have corners when drawing a curve with our finger? "Corner" is a loose definition. "Vertex" is really just a point that connects two lines. So a smooth curve is really just a bunch of points connected by lines (the closer the points are to each other the smoother the curve looks). The variables "count" and "i" are probably going to be used in loops. Ok, so we have a pointer to a vertex buffer, but we don't have a buffer yet. Let's get one: //Allocate vertex array buffer if(vertexBuffer == NULL) vertexBuffer = malloc(vertexMax * 2 * sizeof(GLfloat)); This uses a basic "c" function malloc to grab a chunnk of memory based on the number of vertices (64) we want to track and the size of the storage required per vertex. // Add points to the buffer so there are drawing points every X pixels count = MAX(ceilf(sqrtf((end.x - start.x) * (end.x - start.x) + (end.y - start.y) * (end.y - start.y)) / kBrushPixelStep), 1); Ok, this is pretty much just geometry. I'm not going to go through it bit by bit, but basically this just gets the number of points on the line we will draw based on its length divided by the number of pixels between points which we defined as 3 earlier. This uses the passed in parameters to determine the start and end points of the line we want. for(i = 0; i < count; ++i) { if(vertexCount == vertexMax) { vertexMax = 2 * vertexMax; vertexBuffer = realloc(vertexBuffer, vertexMax * 2 * sizeof(GLfloat)); } vertexBuffer[2 * vertexCount + 0] = start.x + (end.x - start.x) * ((GLfloat)i / (GLfloat)count); vertexBuffer[2 * vertexCount + 1] = start.y + (end.y - start.y) * ((GLfloat)i / (GLfloat)count); vertexCount += 1; } This loop goes through and actually builds the points into the vertex array based on the count we just calculated. We get an interval by dividing the line based on the count and offset to the point based on the interval. Again this isn't OpenGL, just geometry at work. OpenGL depends a lot on geometry so if you slept through those classes, you will need to brush up. We've just taken a line and created a bunch of points (vertices) which we will connect with lines. Let's draw the points. //Render the vertex array glVertexPointer(2, GL_FLOAT, 0, vertexBuffer); glDrawArrays(GL_POINTS, 0, vertexCount); We feed our vertexBuffer into glVertexPointer which takes an array and converts it into an OpenGL style vertex array--we just had a "c" style array. The first parameter "2" indicates the number of coordinates per vertex, so we just use x,y. You can actually go to 4 which is the default--I'll leave it up to you to figure out what 4 coordinates mean. Each coordinate is a GL_FLOAT. The 0 is again the "stride" which we explained earlier--the offsets in the area between points, so a 0 indicates there is no dead-space between points in the array. glDrawArrays is an OpenGL call that draws the array specified. We want to draw an array of points (GL_POINTS). There are other primitives we could draw, see the OpenGL ES documentation. The last parameter is the number of points to draw. Once this method runs we've drawn the line, but we've drawn it offscreen on our backing buffer so the user doesn't see the drawing being built. We now need to swap our buffers to bring the drawing to the user. // Display the buffer [self swapBuffers]; } So this method takes two points and draws a line between the two points and displays it to the user. This is the core of the GLPaint example from the standpoint of drawing. All we need to do is pass in two consecutive points that the user touches on the screen, in order to draw a line between those two points. If we monitor the touches continually (in a loop say) we can continue tracing the route of the finger on the screen. See the previous discussion on how this might work. Next up is: // Reads previously recorded points and draws them onscreen. This is the Shake Me message that appears when the application launches. - (void) playback:(NSMutableArray*)recordedPaths { As the comments shows this is used to playback the recorded data (which we discovered is stored in the Recording.data resource. So we have enough information now to guess at how this might work without looking at the code (we'll still do that). We need pairs of points. We pass those pairs of pairs of points to renderLinefromPoint:toPoint: in order to draw lines. So playback: will probably pull pairs of points from Recording.data and pass them to renderLineFromPoint:toPoint: in a loop. Let's see. NSData* data = [recordedPaths objectAtIndex:0]; CGPoint* point = (CGPoint*)[data bytes]; NSUInteger count = [data length] / sizeof(CGPoint), i; We set up an NSData object pointer that points to the first item in recordedPaths. If you remember this was initialized in our initWithFrame: method to be the contents of the resource. We get a CGPoint pointer to the raw data in our NSData and the number of points (count) in the data block. //Render the current path for(i = 0; i < count - 1; ++i, ++point) [self renderLineFromPoint:*point toPoint:*(point + 1)]; This loop does what I just explained would happen. Go through our recorded data and draw lines between points. The end point becomes the new start point every time through the loop. //Render the next path after a short delay [recordedPaths removeObjectAtIndex:0]; if([recordedPaths count]) [self performSelector:@selector(playback:) withObject:recordedPaths afterDelay:0.01]; } Ok, so what does this do? Without actually diving into how the Recording.data is organized it looks like it is organized into an array. Each element of the array corresponds to something in the "SHAKE ME" animation and we draw each "something" after a delay. What does "something" correspond to? A line, a letter, or a word? How can we tell? Well, we could dive into the debugger. It's easier, however to change the delay of 0.01 to 1.00 and run the app to see where the delay is. If you do that, it's pretty obvious the "something" corresponds in a line or stroke in "SHAKE ME". So the Recording.data is an array of letter strokes! This should show you that you can learn a lot just by playing with the code without having to get into the nitty-gritty of the syntax or debugging. Once you figure out what something does, then you can focus on the syntax and all the options--if you ever need them. The next three methods deal with handling the iPhone touches and drawing them properly. First up: // Handles the start of a touch - (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event { CGRect bounds = [self bounds]; UITouch* touch = [[event touchesForView:self] anyObject]; firstTouch = YES; //Convert touch point from UIView referential to OpenGL one (upside-down flip) location = [touch locationInView:self]; location.y = bounds.size.height - location.y; } The interesting thing here is that the OpenGL ES coordinate system is upside-down compared to the iPhone one--so we need to flip the Y axis. So we store the first point in the touch in the proper coordinate system (OpenGL ES). // Handles the continuation of a touch. - (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event { CGRect bounds = [self bounds]; UITouch* touch = [[event touchesForView:self] anyObject]; //Convert touch point from UIView referential to OpenGL one (upside-down flip) if (firstTouch) { firstTouch = NO; previousLocation = [touch previousLocationInView:self]; previousLocation.y = bounds.size.height - previousLocation.y; } else { location = [touch locationInView:self]; location.y = bounds.size.height - location.y; previousLocation = [touch previousLocationInView:self]; previousLocation.y = bounds.size.height - previousLocation.y; } // Render the stroke [self renderLineFromPoint:previousLocation toPoint:location]; } This does the same in flipping the Y axis. It also connects the new point to the first point. It then makes the current point the first point (again refer to the previous installment if you need a visual diagram of what is happening). As long as the touch moves we get notified and this method gets called, which continues to draw the line connecting the start to the end point and making the end point the new start. Finally, we remove our finger from the screen: // Handles the end of a touch event when the touch is a tap. - (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event { CGRect bounds = [self bounds]; UITouch* touch = [[event touchesForView:self] anyObject]; if (firstTouch) { firstTouch = NO; previousLocation = [touch previousLocationInView:self]; previousLocation.y = bounds.size.height - previousLocation.y; [self renderLineFromPoint:previousLocation toPoint:location]; } } This pretty much just connects the first point to this point and we're done. We have no need to make the last point the new start point. So, that is it. This installment is definitely more interesting than the previous one. This is where all the drawing happens after all the setup in the last part. Next time we'll get into the actual EAGLView that this sample uses. It's a bit different from the one we saw in the XCode template example, so it's worth going over. Until next time. Again feel free to comment and email.

Part III - GLPaint Dissected - PaintingView

Fri, 24 Apr 2009 13:37:01 -0400

In the last installment, we went through the AppController of GLPaint and saw that there was minimal OpenGL ES code in it. This time around, I'm going to tackle the files PaintingView.h and initWithFrame in PaintingView.m so let's dive right in.

The first thing we come across is a set of defines:

#define kBrushOpacity		(1.0 / 3.0)

#define kBrushPixelStep		3

#define kBrushScale		2

#define kLuminosity		0.75

#define kSaturation		1.0

which itemize the characteristics of the "paint brush" we are using.  If you want to see what they do, try changing some of these values and rerunning the sample.

The actual interface comes next:

@interface PaintingView : EAGLView

{ GLuint brushTexture; GLuint drawingTexture; GLuint drawingFramebuffer; CGPoint location; CGPoint previousLocation; Boolean firstTouch; } @property(nonatomic, readwrite) CGPoint location; @property(nonatomic, readwrite) CGPoint previousLocation; - (void) erase; @end As you can see, this is a subclass of EAGLView which we went over in the first set of walkthroughs. The PaintingView here, intruduces a set of properties, the first three are OpenGL ES related. The first two define brushTexture, and drawingTexture as GLuints (again, this is an OpenGL ES-specific data type, namely an unsigned integer.) This probably has to do with the "fuzziness" you see when drawing in GLPaint. The next two are iPhone CoreGraphics (CG) point definitions. These aren't OpenGL ES. When you draw on the screen you have to draw by keeping track of the points your "touch" passes over in the view. Specifically, we need to keep track of the last point, and the current point. We draw a line between the two points. We then set the last point to be the current point and as the finger moves, it writes to a new point. We then connect these two points, and continue as long as the finger is touching the screen. I'm betting that's what these two properties are used for. We have the property declarations for these properties, and finally a single method to erase the display. That's it for PaintingView.h The .m file starts normally with the synthesis of the declared properties: @implementation PaintingView @synthesize location; @synthesize previousLocation; The first method is initWithFrame - (id) initWithFrame:(CGRect)frame { NSMutableArray* recordedPaths; CGImageRef brushImage; CGContextRef brushContext; GLubyte *brushData; size_t width, height; This declares some local properties. The first property is interesting. It's a mutable array called "recordedPaths". I bet this will keep track of the points we move through (see image above) or the opening animation--we'll see which in a moment.. The brush image which is declared as a CGImageRef, and then a brush context. These are again CoreGraphics objects. This is followed by brushData which is an OpenGL ES unishined byte. At this point, I'm not sure what this is used for yet. We then get some dimensions for managing the width and height--of what? Again, we don't have enough to guess at. So let's look at the code and find out. The first line is an "if" statement: if((self = [super initWithFrame:frame pixelFormat:GL_RGB565_OES depthFormat:0 preserveBackbuffer:YES])) { [self setCurrentContext]; What this does is call our parent's initWithFrame and passes in a specific pixel format: GL_RGB565_OES. Doing a little digging this turns out to be a constant that allows your Open GL ES program to deal with textures (rather than solid colors). Textures are managed and stored in their own separate memory areas known as "texture memory" by Open GL. We then make our PaintingView the current drawing context. // Create a texture from an image // First create a UIImage object from the data in a image file, and then extract the Core Graphics image brushImage = [UIImage imageNamed:@"Particle.png"].CGImage; // Get the width and height of the image width = CGImageGetWidth(brushImage); height = CGImageGetHeight(brushImage); This block of code pulls the "Particle.png" image from our resources and makes it our brush image. But when you run the program your brush doesn't look like this, what's happening? Well it appears the program doesn't use the Particle.png as a brush but rather as a texture to draw with. Note the next comment: // Texture dimensions must be a power of 2. If you write an application that allows users to supply an image, // you'll want to add code that checks the dimensions and takes appropriate action if they are not a power of 2. So what are the width and height? Run the app in the debugger and you see that width = height = 64, which is a power of 2. What happens if it's not? Change width or height and run it again and see! If width is set to 65 we get: Is this wrong? Does it crash? No. The comment is just a warning that the results you get may not be perfect unless the dimensions are a power of 2. Maybe you want this effect. The next block does a lot.: // Make sure the image exists if(brushImage) { // Allocate memory needed for the bitmap context brushData = (GLubyte *) malloc(width * height * 4); // Use the bitmatp creation function provided by the Core Graphics framework. brushContext = CGBitmapContextCreate(brushData, width, width, 8, width * 4, CGImageGetColorSpace(brushImage), kCGImageAlphaPremultipliedLast); // After you create the context, you can draw the image to the context. CGContextDrawImage(brushContext, CGRectMake(0.0, 0.0, (CGFloat)width, (CGFloat)height), brushImage); // You don't need the context at this point, so you need to release it to avoid memory leaks. CGContextRelease(brushContext); If the image exists we draw the image to a temporary context that we set up in these lines. Now we get to some OpenGL ES. // Use OpenGL ES to generate a name for the texture. glGenTextures(1, &brushTexture); glGenTextures is an OpenGL ES call that creates a texture name and returns it in the brushTexture variable. The 1 parameter says generate one name. Again, the way we do things in OpenGL is we create a name and then we bind/attach the actual thing to the generated name. Just so you know, the name that's returned is not really a name, but a unique integer. // Bind the texture name. glBindTexture(GL_TEXTURE_2D, brushTexture); This associates our name with a 2D texture. The other option is GL_TEXTURE_1D. Once you execute the bind command, you can work with the texture. // Specify a 2D texture image, providing the a pointer to the image data in memory glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, brushData); The next call to glTexImage2D actually takes a portion our brush data (Particle.png drawn in our context) and makes it a drawable texture. That is, we don't have to use our whole image as a texture. The first parameter specifies the type of texture (2D), 0 indicates base image data (this deals with something known as "mipmaps" which we may discuss in a future blog entry), GL_RGBA indicates the number of color components (R,G,B, and alpha). The width and height indicate the width and height of the texture image. Both of these are supported up to 64 textels (texture pixels) each. The second 0 indicates we don't want a border, and the second GL_RGBA states the format of the image data (which can be different than the number of color components.) GL_UNSIGNED_BYTE indicates the size of the pixel data--each pixel in the image is an unsigned byte. And finally, brushData is a pointer to our texture image to use. // Release the image data; it's no longer needed free(brushData); We release our source texture data. Our texture is set up. // Set the texture parameters to use a minifying filter and a linear filer (weighted average) glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); You can think of glTexParameteri as a call that determines how your texture is applied to various surfaces. Here we set it up to work in our view. The first parameter is just to specify this is a 2D texture, the second GL_TEXTURE_MIN_FILTER specifies that we want to specify how to map a texture pixel (textel) from our texture map onto an area larger than a single pixel. This is known as minifying filter. The actual rule for minifying is GL_LINEAR. GL_LINEAR takes the weighted average of the 4 closest textels from the texture and uses that to map the pixel. So, this allows OpenGL ES to interpolate a pixel that needs to be mapped. // Enable use of the texture glEnable(GL_TEXTURE_2D); We then enable (make active) the texture so we can use it. // Set a blending function to use glBlendFunc(GL_SRC_ALPHA, GL_ONE); glBlendFunc sets up how we the texture map is blended onto the target view. GL_SRC_ALPHA says we want to use the alpha component of the source (the texture) to override the alpha of the destination (the view). The second parameter GL_ONE says use the color components of the destination. The first parameter therefore deals with how the source is blended, the second how the destination is blended. // Enable blending glEnable(GL_BLEND); } We then enable blending when we draw. To see how this works, draw a red line, then a green line that crosses it, and then a blue one that crosses the two. You end up with white due to the blending. If you draw slowly, you'll also get white, if you draw fast you get a purer color due to less blending taking place. The next block takes care of actually setting up most of the OpenGL state to use. //Set up OpenGL states glDisable(GL_DITHER); glMatrixMode(GL_PROJECTION); glOrthof(0, frame.size.width, 0, frame.size.height, -1, 1); glMatrixMode(GL_MODELVIEW); glEnable(GL_TEXTURE_2D); glEnableClientState(GL_VERTEX_ARRAY); glEnable(GL_BLEND); glBlendFunc(GL_SRC_ALPHA, GL_ONE); glEnable(GL_POINT_SPRITE_OES); glTexEnvf(GL_POINT_SPRITE_OES, GL_COORD_REPLACE_OES, GL_TRUE); glPointSize(width / kBrushScale); Some of these I've already gone over so I won't explain them again. Check the previous installments for more information. The new calls here are to disable dithering. What happens if you comment this out? glMatrixMode specifies that the glOrthof call will work on the GL_PROJECTION matrix (remember the matrix discussion?) rather than on the texture matrix, etc. The second glMatrixMode says that the remainder of the calls will work on the modelview. We enable texturing, and the vertex array (remember our square?) via the call to glEnableClientState. We enable blending set up a blending function to use, The next two lines: glEnable(GL_POINT_SPRITE_OES); glTexEnvf(GL_POINT_SPRITE_OES, GL_COORD_REPLACE_OES, GL_TRUE); use OpenGL's particle system to create a sprite (think of a graphical sprite as a rubber stamp, except that once you use the stamp, you can move the image around.) The particle system is used to deal with things like explosions, stars, etc. That is, things that look like points. The second line is similar to the previous glTexParameteri call except that this one deals with setting up the texturing environment as a whole. The first parameter says we want to deal with the sprite's environment, the second parameter indicates the actual parameter in the environment we want to modify, and GL_TRUE is the value of GL_COORD_REPLACE_OES. So what does GL_COORD_REPLACE_OES do? It replaces the part of the view being drawn (mapped) with our particle. If you set this to GL_FALSE nothing will be drawn. The final line in this block sets the particle size to draw to 32: glPointSize(width / kBrushScale) Essentially, this sets the size of the brush that will be drawn with. Next is some interesting stuff: //Make sure to start with a cleared buffer [self erase]; //Playback recorded path, which is "Shake Me" recordedPaths = [NSMutableArray arrayWithContentsOfFile:[[NSBundle mainBundle] pathForResource:@"Recording" ofType:@"data"]]; if([recordedPaths count]) [self performSelector:@selector(playback:) withObject:recordedPaths afterDelay:0.2]; } return self; } We clear our view, and then get the data block we looked at in the previous installment "Recording". This is the data that draws "Shake Me" on the screen when it first launches. If there is data to be played back, we invoke the method "playback:" passing it the recorded data to play back. We then come to the end of initWithFrame. So what have we learned? We've learned about how to load an image as a texture, how to set up the texturing environment, and also how to use the particle system to define a particle as a point "sprite" as a brush. One thing that is pretty evident to me is that there is a lot of setup that you have to do with OpenGL to get anything to happen. You may be wondering, is it worth it? Yes if you want your graphics to look good, work efficiently, and across multiple platforms. Could you do this in a simpler manner? Probably. Something as relatively simple as GLPaint could be done easier using CoreGraphics, with less code. But once you get to full blown applications and games, OpenGL gives you the quality you need. What's next? We'll finish up our dissection of PaintingView.m and GLPaint and get into how the actual drawing code works. Again, feel free to comment.

Part I - GLPaint Dissected - Overview

Sun, 29 Mar 2009 12:36:01 -0400

In the previous installment I finished going through the XCode OpenGL ES template that is used with the iPhone. I recently received a couple of emails asking me to go into drawing background images and to dissect the GLPaint sample that you can download off the developers site. I've decided to go into the GLPaint example. This might help with understanding how to set a background image as well.

You can download the GLPaint sample from Apple's iPhone developer site if you're registered: http://developer.apple.com/iphone/library/navigation/SampleCode.html

So what does GLPaint do? It allows you to use your finger to draw/paint with various colors. Shaking the iPhone will erase the image. Note that you can't shake the iPhone using the emulator in XCode. When you first run the application it presents a screen that looks like:

Notice that when you run the application it actually animates drawing the "Shake Me!" screen. Pretty cool. We'll try to explain how this happens.

You can then select a color from the palette on the bottom and draw using your finger.

The other thing about this application is that it produces sounds (useful, but not really OpenGL ES related.)

So lets take a look at the actual XCode project:

Starting at the top we have a ReadMe file which you should definitely read since it helps to explain some of the other pieces. The "Classes" group

has the main application controller class, and the classes responsible for doing the actual drawing and for generating the sound effects. We won't go into the code that's in SoundEffect.h and SoundEffect.m since they have nothing to do with OpenGL ES and are pure Cocoa. I'll leave it up to you to figure out how they work. We'll talk about how they are used however.

The OpenGL ES Support group contains

Now this is something that sounds familiar. I find that it always helps to attack an unknown program starting with something I already feel comfortable with. This lets us not only reenforce what we've already learned but helps us not get too overwhelmed too fast. But in order to understand the context of these files we need to get a feel for how they are used. So to do that we should go through the AppController first.

Finally we have the Resources group

This has all of the graphics and sounds the program uses. If you select each of these in turn you can see what they look like and this will help you orient yourself with how they are actually used in the application. For example Green.png is a green square that is used as the green control on the color palette.

You may be wondering what the Recording.data resource is. It looks like a property list that has gobbeldy-gook.

I'm willing to bet that this is the data that is used to drive the opening animation. We'll try to confirm this when we get into the actual code.

Also note that there is no nib or xib file in this project. That tells me that this is a good project to use as an example for creating OpenGL ES views programmatically. The XCode template we went through last time builds the view using IB and some code.

One thing you should notice is what is in the Frameworks group:

This project uses sound so it uses the AudioToolkit as well as the OpenGLES framework. So if you start an OpenGL ES application from scratch rather than by using the OpenGL ES template, you'll have to include the OpenGLES framework manually. The OpenGL ES template takes care of this step for you.

So what next? Well there are only a couple of files we have to deal with from an OpenGL ES perspective. The AppController, EAGLView, and PaintingView. Next installment we'll go through AppController which drives everything.

Part II - GLPaint Dissected - AppController

Sat, 11 Apr 2009 09:27:38 -0400

In the last installment, I presented an overview of the GLPaint sample program that you can download from the Apple iPhone developer site. This time around, I'll try to go through the AppController files. I should probably explain how I write these. I don't pre-plan anything apart from the main file or files I'll tackle in a given installment. I then dive into the code and walk through it for the first time. I write my thoughts and observations as they occur to me. That way you get pretty much my raw thought processes as I learn along with you. I focus on the OpenGL code since I'm assuming you have at least some basic Cocoa familiarity. So let's dive in.

Remember GLPaint allows you to "paint" using a palette and your finger. As you move your finger around on the screen it has to be tracked. Here is the AppController.h file.

#import "PaintingView.h"

#import "SoundEffect.h"

//CLASS INTERFACES:

@interface AppController : NSObject <UIAccelerometerDelegate>

	UIWindow			*window;

	PaintingView		*drawingView;

	UIAccelerationValue	myAccelerometer[3];

	SoundEffect			*erasingSound;

	SoundEffect			*selectSound;

	CFTimeInterval		lastTime;

@end

That's the entire header of the AppController. The only things that seem to be OpenGL ES related are the PaintingView (which I assume will be used to do the actual painting) and the CFTimeInterval lastTime, which I'm betting will be used in sampling the drawing being done. We'll see once we dive into the implementation. I find it useful to come up with what I think might be going on as a "guess". I don't spend a lot of time on it, but it tends to get me mentally oriented even if the actual code proves me wrong later on. It's better to have a plan and change it along the way, than not to have a plan at all and get lost.

So on to the implementation. The first thing we encounter is a bunch of constants:

#import "AppController.h"

//CONSTANTS:

#define kPaletteHeight			30

#define kPaletteSize			5

#define kAccelerometerFrequency		25 //Hz

#define kFilteringFactor		0.1

#define kMinEraseInterval		0.5

#define kEraseAccelerationThreshold	2.0

// Padding for margins

#define kLeftMargin			10.0

#define kTopMargin			10.0

#define kRightMargin			10.0

The last three are pretty straightforward. They define a margin. A margin for what? Well we only have two on-screen things. The drawing view and the palette, so the margins will be used for one or both of those.

The top section is a bit more interesting. It has the palette size which sets our color choices.The next four are accelerometer related. What is the accelerometer used for? If you run the program you see that shaking the iPhone will erase your drawing. So these are settings used to detect a shake Again these aren't OpenGL ES related per se but they are interesting if you haven't done any accelerometer coding.

The next thing about GLPaint is that it has much better internal documentation than the OpenGL ES template that we went through earlier.

/*

   HSL2RGB Converts hue, saturation, luminance values to the equivalent red, green and blue values.

   For details on this conversion, see Fundamentals of Interactive Computer Graphics by Foley and van Dam (1982, Addison and Wesley)

   You can also find HSL to RGB conversion algorithms by searching the Internet.

   See also http://en.wikipedia.org/wiki/HSV_color_space for a theoretical explanation

*/

static void HSL2RGB(float h, float s, float l, float* outR, float* outG, float* outB)

HSL2RGB seems to be a "standard" conversion that Apple pulled from a book. No reason not to use other people's code if you don't pirate it. This isn't OpenGL ES, just a helper/utility function. The iPhone frameworks tend to deal with RGBA (Red, Green, Blue, Alpha) rather than hue, saturation, luminance, otherwise this function would be built into the frameworks. So why does this application use HSL rather than RGBA? I don't know, probably because the person who wrote it came from another development environment to the iPhone or this sample is ported from some other program.

Now on to the iPhone code.

- (void) applicationDidFinishLaunching:(UIApplication*)application

	CGRect					rect = [[UIScreen mainScreen] applicationFrame];

	CGFloat					components[3];

	//Create a full-screen window

	window = [[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]];

	[window setBackgroundColor:[UIColor blackColor]];

"applicationDidFinishLaunching" first sets up a rectangle that encompasses the screen minus the status bar on the top, and then three floats "components" which may be used to hold HSL values?

We then create a new window and set its background to black.

	//Create the OpenGL drawing view and add it to the window

	drawingView = [[PaintingView alloc] initWithFrame:CGRectMake(rect.origin.x, rect.origin.y, rect.size.width, rect.size.height)]; // - kPaletteHeight

	[window addSubview:drawingView];

This next bit of code creates our actual OpenGL ES drawing view. So this is a drawing layer on top of our window. Our drawing view is a subclass of "PaintingView" which we'll go over later. The comment may be confusing. This looks like "test code". This is something I occasionally do. I try some code for example (the line without the comment):

drawingView = [[PaintingView alloc] initWithFrame:CGRectMake(rect.origin.x, rect.origin.y, rect.size.width, rect.size.height - kPaletteHeight )];

And for some reason it doesn't work, either because it doesn't do what I expect it to do, or the code isn't "elegant". So I comment out the bit in question and rework the code elsewhere. I leave the comment to remind me that this is what it used to be and that I changed it someplace else. This is just a developer technique, but if you just see the comment you may be scratching your head trying to figure out what it means. It's not a comment that explains, it's a comment that reminds the developer. So just ignore it.

	// Create a segmented control so that the user can choose the brush color.

	UISegmentedControl *segmentedControl = [[UISegmentedControl alloc] initWithItems:

											[NSArray arrayWithObjects:

												[UIImage imageNamed:@"Red.png"],

												[UIImage imageNamed:@"Yellow.png"],

												[UIImage imageNamed:@"Green.png"],

												[UIImage imageNamed:@"Blue.png"],

												[UIImage imageNamed:@"Purple.png"],

												nil]];

This creates an segmented control that will be used to manage our palette color choices (our color palette).

	// Compute a rectangle that is positioned correctly for the segmented control you'll use as a brush color palette

	CGRect frame = CGRectMake(rect.origin.x + kLeftMargin, rect.size.height - kPaletteHeight - kTopMargin, rect.size.width - (kLeftMargin + kRightMargin), kPaletteHeight);

	segmentedControl.frame = frame;

	// When the user chooses a color, the method changeBrushColor: is called.

	[segmentedControl addTarget:self action:@selector(changeBrushColor:) forControlEvents:UIControlEventValueChanged];

	segmentedControl.segmentedControlStyle = UISegmentedControlStyleBar;

	// Make sure the color of the color complements the black background

	segmentedControl.tintColor = [UIColor darkGrayColor];

	// Set the third color (index values start at 0)

	segmentedControl.selectedSegmentIndex = 2;

	// Add the control to the window

	[window addSubview:segmentedControl];

	// Now that the control is added, you can release it

	[segmentedControl release];

This sets up the attributes of the palette. No OpenGL ES.

    // Define a starting color

	HSL2RGB((CGFloat) 2.0 / (CGFloat)kPaletteSize, kSaturation, kLuminosity, &components[0], &components[1], &components[2]);

	// Set the color using OpenGL

	glColor4f(components[0], components[1], components[2], kBrushOpacity);

Here we get OpenGL stuff. First we call our helper and pass in some constants that are defined in some other file and we get RGB returned in our components array. We are getting a default color. The glColor4f call is an OpenGL ES call. It takes a set of r,g,b,a parameters and sets the current color.

The remainder of applicationDidFinishLaunching is straightforward:

//Show the window [window makeKeyAndVisible]; // Look in the Info.plist file and you'll see the status bar is hidden // Set the style to black so it matches the background of the application [application setStatusBarStyle:UIStatusBarStyleBlackTranslucent animated:NO]; // Now show the status bar, but animate to the style. [application setStatusBarHidden:NO animated:YES]; //Configure and enable the accelerometer [[UIAccelerometer sharedAccelerometer] setUpdateInterval:(1.0 / kAccelerometerFrequency)]; [[UIAccelerometer sharedAccelerometer] setDelegate:self]; //Load the sounds NSBundle *mainBundle = [NSBundle mainBundle]; erasingSound = [[SoundEffect alloc] initWithContentsOfFile:[mainBundle pathForResource:@"Erase" ofType:@"caf"]]; selectSound = [[SoundEffect alloc] initWithContentsOfFile:[mainBundle pathForResource:@"Select" ofType:@"caf"]]; }

The next method dealloc is likewise easy:

// Release resources when they are no longer needed, - (void) dealloc { [selectSound release]; [erasingSound release]; [drawingView release]; [window release]; [super dealloc]; } Next is another graphics method: // Change the brush color - (void)changeBrushColor:(id)sender { CGFloat components[3]; //Play sound [selectSound play]; //Set the new brush color HSL2RGB((CGFloat)[sender selectedSegmentIndex] / (CGFloat)kPaletteSize, kSaturation, kLuminosity, &components[0], &components[1], &components[2]); glColor4f(components[0], components[1], components[2], kBrushOpacity); } Whenever the user changes colors (taps the palette) we hear a sound, and we call our helper again passing in the tapped palette color ([sender selectedSegmentIndex]). We divide this by 5 (kPaletteSize -- the number of colors on the palette). This passes the selected color into the helper and we get RGB out again. This becomes our current color using the glColor4f invocation again. Finally, the last method "accelerometer" just detects an iPhone shake and erases the drawingView. No OpenGL ES here either. // Called when the accelerometer detects motion; plays the erase sound and redraws the view if the motion is over a threshold. - (void) accelerometer:(UIAccelerometer*)accelerometer didAccelerate:(UIAcceleration*)acceleration { UIAccelerationValue length, x, y, z; //Use a basic high-pass filter to remove the influence of the gravity myAccelerometer[0] = acceleration.x * kFilteringFactor + myAccelerometer[0] * (1.0 - kFilteringFactor); myAccelerometer[1] = acceleration.y * kFilteringFactor + myAccelerometer[1] * (1.0 - kFilteringFactor); myAccelerometer[2] = acceleration.z * kFilteringFactor + myAccelerometer[2] * (1.0 - kFilteringFactor); // Compute values for the three axes of the acceleromater x = acceleration.x - myAccelerometer[0]; y = acceleration.y - myAccelerometer[0]; z = acceleration.z - myAccelerometer[0]; //Compute the intensity of the current acceleration length = sqrt(x * x + y * y + z * z); // If above a given threshold, play the erase sounds and erase the drawing view if((length >= kEraseAccelerationThreshold) && (CFAbsoluteTimeGetCurrent() > lastTime + kMinEraseInterval)) { [erasingSound play]; [drawingView erase]; lastTime = CFAbsoluteTimeGetCurrent(); } } That's it for AppController. As you see, there is very little OpenGL ES in this code. It amounts to using a single call "glColor4f" to select a current color. Everything else is normal iPhone Cocoa and the one helper routine. Next time, we'll go into the PaintingView, which as we'll see has a lot of OpenGL ES in it. It was still worth going into the AppController code since this is what really drives the rest of the code.

Part V - The iPhone OpenGL ES XCode Template Dissected - The Delegate Code

Sun, 15 Mar 2009 10:24:02 -0400

In the previous installment I went through the remainder of the code in the EAGLView.m file in which we dealt with the buffer management, drawing, and FBO (frame buffer object) code needed to animate our square.

In this module, I'll go through the rest of the template code, namely the delegate files gAppDelegate.h and gAppDelegate.m.

The header, gAppDelegate.h looks like:

//

//  glAppDelegate.h

//  gl

//

//

#import <UIKit/UIKit.h>

@class EAGLView;

@interface glAppDelegate : NSObject <UIApplicationDelegate> {

    UIWindow *window;

    EAGLView *glView;

@property (nonatomic, retain) IBOutlet UIWindow *window;

@property (nonatomic, retain) IBOutlet EAGLView *glView;

@end

Here we see the declaration of a single class EAGLView which has an iPhone window declared as UIWindow, and the view the template uses declared as EAGLView within it. There are also the corresponding property declarations. Nothing OpenGL ES specific in all of this. The first section of glAppDelegate.m imports the EAGLView.h header for the class we covered in the previous installments. It also has the synthesized properties for the two properties we saw in the header. // // glAppDelegate.m // gl // // #import "glAppDelegate.h" #import "EAGLView.h" @implementation glAppDelegate @synthesize window; @synthesize glView; The first method is a UIView delegate method (as are all of the methods in this class). - (void)applicationDidFinishLaunching:(UIApplication *)application { glView.animationInterval = 1.0 / 60.0; [glView startAnimation]; } Here we see where we set the frame rate for our animation, which is set to 1/60 of a second, which translates to 60 frames-per-second (fps). We then call the method which starts the animation. This is all done once the application loads. The next method is interesting. - (void)applicationWillResignActive:(UIApplication *)application { glView.animationInterval = 1.0 / 5.0; } What it does is lowers the frame rate to 20fps when the application is not active. "applicationWillResignActive:" gets invoked when a popup alert is displayed or when the iPhone is locked. This is interesting since this application has no built-in popup overlays. So the only type we will probably see is something like a low battery warning, or some other system alert. So why do we change the animation? The main reason would be to save battery power. The faster the animation rate, the more battery power things tend to consume. I'm assuming animation on the iPhone works the same way--otherwise there wouldn't be a reason to do anything in this method. I'd go one step further--because this method is called when you lock your iPhone, I'd set the animation interval to 0 fps to shut down animation completely in order to minimize the battery usage while locked. Once the application becomes active again (the iPhone is unlocked or the popup alert is dismissed) we call: - (void)applicationDidBecomeActive:(UIApplication *)application { glView.animationInterval = 1.0 / 60.0; } which resets our frame rate back to 60fps. And finally, once our application shuts down we invoke dealloc: - (void)dealloc { [window release]; [glView release]; [super dealloc]; } @end which frees our resources. But what about the MainWindow.xib file? Well there's really nothing in it that is OpenGL ES-specific. Take a look and you'll see it merely creates a delegate and wires it up to the window. That's it! At it's core the XCode iPhone OpenGL ES template isn't very complicated, but the difficulty comes in understanding the API and where and when to use it properly. I hope this has helped a bit in the high-level understanding of OpenGL ES. I know this has helped me a bit to clear away some of the fog, as I mentioned this is my first look into OpenGL or OpenGL ES. I always find, that if you can follow the high-level logic of an application, figuring out the details of the method call interactions amounts to looking things up in the documentation and taking your time to explore a bit. Now that I've finished this series what is next? Well I've received a request from someone to go through the iPhone sample Apple provides called GLPaint which uses touch and OpenGL ES to paint. So next time, look forward to a new series based on that.

Part IV - The iPhone OpenGL ES XCode Template Dissected - The Remaineder

Sat, 28 Feb 2009 14:03:53 -0500

In the last installment we spent a lot of time going into the drawView method. We learned how OpenGL ES uses various matrices to control how things are drawn as well as the things that are drawn. We saw that the standard template in XCode has some interesting things about it, such as rotating the view around the square rather than rotating the square itself.

In this installment we'll go through the remainder of the EAGLView.m file. Let's get started.

The next method after drawView is one called "layoutSubviews".

- (void)layoutSubviews {

    [EAGLContext setCurrentContext:context];

    [self destroyFramebuffer];

    [self createFramebuffer];

    [self drawView];

This method overrides the standard layoutSubviews method.  It sets the current OpenGL ES context to our context, and then it calls the destroy and createFramebuffer methods that we saw are declared as private to this class.  We then invoke the drawView method we dissected last time.    Basically, we are making sure our view is initialized properly since we don't really layout any subviews here.

The next method creates the frameBuffer.

- (BOOL)createFramebuffer { glGenFramebuffersOES(1, &viewFramebuffer); glGenRenderbuffersOES(1, &viewRenderbuffer); These two lines return a number (in this case 1) of buffer names of the appropriate type (frame and render). This is something I'm still trying to wrap my mind around. OpenGL ES doesn't give you a buffer object, but rather a name associated with a buffer object. That's what these two lines do. The next two lines: glBindFramebufferOES(GL_FRAMEBUFFER_OES, viewFramebuffer); glBindRenderbufferOES(GL_RENDERBUFFER_OES, viewRenderbuffer); actually bind (associate) an actual buffer to the name returned. I'm not sure why OpenGL requires two steps when one call could give you a name and bind it to a buffer of the appropriate time. It seems a bit obtuse to me. [context renderbufferStorage:GL_RENDERBUFFER_OES fromDrawable:(CAEAGLLayer*)self.layer]; This method actually allocates the buffer storage for our view (context). In all of these methods we see "_OES" which is the OpenGL ES specific versions. So in this line GL_RENDERBUFFER_OES allocates an OpenGL ES version of a render buffer. glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, viewRenderbuffer); This takes our renderbuffer and attaches it to our framebuffer. So we allocated the two buffers, and we associate them together. Again, the framebuffer is memory, while the renderbuffer is the memory that actually gets drawn and related to the hardware graphics memory. Think of this as mapping a chunk of memory that the rendering (drawing) hardware will use to hold our drawing. More interesting, the parameter GL_COLOR_ATTACHMENT0_OES is used to attach a color texture to our rendering buffer. In this code think of this as mapping the two buffers together and assigning a color palette to those buffers--the OES one. glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_WIDTH_OES, &backingWidth); glGetRenderbufferParameterivOES(GL_RENDERBUFFER_OES, GL_RENDERBUFFER_HEIGHT_OES, &backingHeight); The backingWidth and backingHeight are actually retrieved by glGetRenderbufferParameterivOES for our renderbuffer. Remember we used these in our viewPort method as part of the glViewport call. Next we get the same set of logic with a check: if (USE_DEPTH_BUFFER) { glGenRenderbuffersOES(1, &depthRenderbuffer); glBindRenderbufferOES(GL_RENDERBUFFER_OES, depthRenderbuffer); glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_DEPTH_COMPONENT16_OES, backingWidth, backingHeight); glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_DEPTH_ATTACHMENT_OES, GL_RENDERBUFFER_OES, depthRenderbuffer); } USE_DEPTH_BUFFER is set to 0 by a define up top: #define USE_DEPTH_BUFFER 0 If we rerun this with a value of 1 nothing seems to change. This snippet of code allocates a third buffer (a depthRenderbuffer) and pretty much does the same thing as the code we just went through. So what is a depthRenderbuffer? A depth render buffer is used to manage such things as shadows, alpha blending, etc.--basically things associated with drawing our object but not the object itself. No render buffer: With a depth render buffer. No obvious changes in the standard XCode template. Finally, this method checks for a successful creation of our buffer: if(glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES) != GL_FRAMEBUFFER_COMPLETE_OES) { NSLog(@"failed to make complete framebuffer object %x", glCheckFramebufferStatusOES(GL_FRAMEBUFFER_OES)); return NO; } return YES; glCheckFramebufferStatusOES checks the completness of the buffer's creation. The completeness is measured based on whether all of the required attributes for a framebuffer have been defined. This call returns information on the attributes that are wrong or missing if it doesn't return GL_FRAMEBUFFER_COMPLETE_OES. Throughout this installment we've been talking about something I've been calling a framebuffer. More specifically, OpenGL calls this a "frame buffer object" or FBO. It is more than just a buffer full of memory, but that's a major part. It represents a real object that includes the buffer as well as images, textures, colors, etc. The glCheckFramebufferStatusOES call checks the completeness or "sanity" of the object in terms of how drawable it is. It's a good idea to check this prior to trying to render this buffer. If the framebuffer is created correctly we return true, otherwise false. So now the idea of getting a name and attaching things to it makes a bit more sense. We aren't dealing with just a buffer, but a buffer object. We get a name (basically the type of FBO we want) then we bind various properties to it one at a time. So it's more complicated than just combining two calls into one. The question may still be, why can't we just use setters to do all this to our FBO? The answer comes down to the fact that OpenGL is procedural "c" and although it has this thing known as an FBO, it's not an object in the OOP sense, so we have to set its properties via multiple procedural calls. The next method: - (void)destroyFramebuffer { glDeleteFramebuffersOES(1, &viewFramebuffer); viewFramebuffer = 0; glDeleteRenderbuffersOES(1, &viewRenderbuffer); viewRenderbuffer = 0; if(depthRenderbuffer) { glDeleteRenderbuffersOES(1, &depthRenderbuffer); depthRenderbuffer = 0; } } is pretty straightforward. It deletes our frame and render buffers. It optionally destroys the depthRenderbuffer as well if our define is set. These are all 'c' based routines so we don't need to [x release], although setting our buffers to 0 makes sure we are in a known state in case we need to check. Next, - (void)startAnimation { self.animationTimer = [NSTimer scheduledTimerWithTimeInterval:animationInterval target:self selector:@selector(drawView) userInfo:nil repeats:YES]; } is pure Cocoa. We create an NSTimer object that repeats every "animationInterval" periods (that was set to 1/60th of a second in our initWithCoder method). Every 1/60th of a second we invoke drawView to render our frame. This method is followed by: - (void)stopAnimation { self.animationTimer = nil; } This frees our timer stopping the periodic call to drawView and is invoked by the next method. - (void)setAnimationInterval:(NSTimeInterval)interval { animationInterval = interval; if (animationTimer) { [self stopAnimation]; [self startAnimation]; } } This is a simple setter method for animationInterval. As part of setting the interval we stop and free any old timer, and create a new timer with the new interval. So actually setting the animationInterval value in initWithCoder causes the animation to begin. Finally, we haver our dealloc method which stops the animation and releases our context after detaching it from our OpenGL ES environment by setting it to nil. Again because OpenGL ES is "c" based, we need to free any memory allocated using "c' functions--we didn't allocate any so there is nothing to do in the dealloc of this sample application. - (void)dealloc { [self stopAnimation]; if ([EAGLContext currentContext] == context) { [EAGLContext setCurrentContext:nil]; } [context release]; [super dealloc]; } That takes care of the EAGLView.m file. We've learned how to set up our buffers and verify they are correct and complete, and got our mind around FBOs. We saw how to draw the image, and how to control the animation. We have a couple more minor helper methods to go through that are part of the delegate. We'll tackle those next time.

Part III - The iPhone OpenGL ES XCode Template Dissected - drawView Logic

Mon, 23 Feb 2009 16:44:32 -0500

Back again for more. Last time I went over the initWithCoder method that set up our canvas and drawing layer. We learned a bit about the EAGLDrawable protocol, and acquired a context and made sure it was set to our current drawing context. This time around we'll dig into the drawView method which does the work of actually drawing something using OpenGL ES. Let's start.

We get a nice comment:

// Replace the implementation of this method to do your own custom drawing

So this is the method to focus on modifying when we want to spread our OpenGL wings and draw our own stuff. What follows the commment looks like:

    const GLfloat squareVertices[] = {

        -0.5f, -0.5f,

        0.5f,  -0.5f,

        -0.5f,  0.5f,

        0.5f,   0.5f,

};

This line is a simple example of defining the coordinates of a square using the GLfloat data type. Again we need to make sure we use the OpenGL intrinsic datatypes when doing this. The definition is a matrix called squareVertices. Graphics systems tend to talk about "matrices" instead of "arrays". Each line represents the x,y coordinates of a corner of the square. Graph it out. But first you have to understand the coordinate system of OpenGL. A little research shows its a standard one--no inverted axes. So plotting out our square, we see it's centered around (0,0) and it doesn't have a z coordinate so its a flat square. Nice and simple. The next line looks similar to this: const GLubyte squareColors[] = { 255, 0, 0, 255, 0, 255, 0, 255, 0, 0, 0, 0, 255, 0, 255, 255, }; This is like our vertex matrix except that its a matrix with 4 elements called squareColors, and is an OpenGL ES ubyte -- unsigned byte. Why unsigned, because you can't have a negative RGBA value. These colors represent the kEAGLColorFormatRGBA8 color format we discussed last time. Each line therefore is a red, green, blue, alpha value. Now why are there four lines? This isn't very obvious unless you run the program and observe the colors. The give away is line three (0,0,0,0)--black. If you look at the spinning cube, one of its corners is black, and the other corners map to the other lines in the matrix. The colors blend toward the center of the square. So the template provides a fairly interesting example in that it doesn't just spin a square with a solid color. Try changing some of the values in the matrix, and rerun the application. This is what changing the matrix to: const GLubyte squareColors[] = { 255, 0, 0, 255, 255, 0, 0, 255, 255, 0, 0, 0, 255, 0, 255, 255, }; looks like: The next line should be understandable from the last session: [EAGLContext setCurrentContext:context]; We set the current OpenGL context to our context. This line seems superfluous in the template (commenting it out doesn't change anything). However, it's probably a good idea to make sure your context is set within the drawView logic in case some other part of the application resets it to something else. glBindFramebufferOES(GL_FRAMEBUFFER_OES, viewFramebuffer); According to the spec the glBindFramebufferOES method allows us to bind a framebuffer to a name. So we are binding our viewFramebuffer to something called GL_FRAMEBUFFER_OES. It took a while for me to figure out what this is since the specification and the iPhone docs don't explain this. GL_FRAMEBUFFER_OES is the definition of an Open GL ES (OES) framebuffer. So on the iPhone we bind our buffer to a definition of what it means to be an Open GL ES framebuffer. In plain old OpenGL we can bind it to other implementations of a framebuffer. The next line: glViewport(0, 0, backingWidth, backingHeight); This is a call to OpenGL to set up the internal coordinate system to the size and coordinate system of the physical display. This call returns the values of backingWidth and backingHeight which depends on the implementation or the iPhone. The next few lines are grouped together and that tells me they are related in function: glMatrixMode(GL_PROJECTION); glLoadIdentity(); glOrthof(-1.0f, 1.0f, -1.5f, 1.5f, -1.0f, 1.0f); glMatrixMode(GL_MODELVIEW); glRotatef(0.0f, 0.0f, 0.0f, 1.0f); The line glMatrixMode(GL_PROJECTION); sets the current matrix to a projection matrix. The confusing thing here may be that it doesn't refer to our squareVertices and squareColors matrices, but rather to an internal matrix. This internal matrix determines how coordinates are transformed. glMatrixMode can be set to these values: • GL_MODELVIEW which applies tranformations to something known as a modelview matrix stack. This allows us to position the view in the scene. That is it allows us to move the virtual viewpoint or camera around. • GL_PROJECTION applies transformations to a projection matrix stack. This allows us to specify how the 3D scene is mapped to our 2D display. Think of this as looking through a window. • GL_TEXTURE applies transformations to a texture matrix stack. This allows us to specify how textures (skins) are applied to our objects. In the template we are specifying that everything we do will affect the GL_PROJECTION matrix stack which is our window onto the square. glLoadIdentity(); replaces whatever matrix is in effect with an "identity matrix." Think of the identity matrix as the "default matrix". Because we set the glMatrixMode, this identity matrix applies to the projection matrix. This is a complicated way of saying we have reset our 2D window on our 3D image to a default window--its at a known state. glOrthof(-1.0f, 1.0f, -1.5f, 1.5f, -1.0f, 1.0f); takes our window (the identity matrix) and transforms it. Before I try to explain this, let's look at the parameters. They map as follows: -1.0f, 1.0f, -1.5f, 1.5f, -1.0f, 1.0f left, right, bottom,top, near, far These specify the positions of the planes that determine our window. So our window is 2 units wide, 3 units high and 2 deep (think of our window as an acrylic cube that captures our scene.) The width and height 2x3 is coincidentally the same aspect ratio as the iPhone's aspect ratio--not the screen, but the entire iPhone. So what is this "window" that we're dealing with? It's really the definition of how the 3d view is seen in two dimension and how it is clipped. The next line: glMatrixMode(GL_MODELVIEW); resets the current matrix to be the GL_MODELVIEW which as I said controls how the view (our square) is positioned. So we are done positioning the window, we now have to position the square so it shows up in the window. This is done by: glRotatef(0.0f, 0.0f, 0.0f, 1.0f); glRotate rotates the view a given number of degrees (not radians) around a point. The parameters are (angle,x,y,z). What this does isn't very obvious at first glance. Why would we want to rotate around z of 1.0? Z is the axis that is perpendicular to the screen straight at you. So we are rotating our viewport 3 degrees to the right. That would produce a square that is tilted. That doesn't make sense until you realize this method drawView is used to do just that--and we are animating. So everytime this method is called, we rotate three degrees. At 60fps (see the first part of this series) multiplied by 3 degrees, we get a full rotation of the square (360 degrees) in one second. To show you what this line controls, go ahead and comment it out, or change the 3 degrees to something else and run the app again. An important thing to note about this application. We don't rotate the actual square. We rotate the view *AROUND THE SQUARE*. At this point we've rotated the view, but we haven't even drawn the square yet. When we get around to drawing the square into our view, it will appear rotated. There is more to this app than first meets the eye! So what is left to do? We have our square (actually the view) spinning. The next two lines are again related. glClearColor(0.5f, 0.5f, 0.5f, 1.0f); glClear(GL_COLOR_BUFFER_BIT); The first call to glClearColor specifies the clear color for the color buffer (RGBA again). This is the background color you see behind the spinning square. So after every frame we clear the frame to this color. That's what glClear does. The parameter GL_COLOR_BUFFER_BIT indicates we want to clear our drawing buffer (you can clear other buffers as well.) Try commenting out each line in turn, to see the effect. Commenting out the glClear command lets the previous frame to stay on the screen and each succeeding frame will draw on top of the old image. This produces a circle effect. The next lines look like they actually draw the square. glVertexPointer(2, GL_FLOAT, 0, squareVertices); glEnableClientState(GL_VERTEX_ARRAY); glColorPointer(4, GL_UNSIGNED_BYTE, 0, squareColors); glEnableClientState(GL_COLOR_ARRAY); glVertexPointer is a call that sets up a pointer to a matrix of vertices (corners). In this case we point to our square matrix. The first parameter specifies how many coordinates there are per vertex; in our square there are only two, an x and a y--no z.) The second parameter specifies the datatype that defines a vertex. Our matrix is declared as GLfloat squareVertices, so we specify a constant GL_FLOAT that describes GLfloats. The third parameter (set to 0), is something known as "stride". It's the byte offset between vertices, and the default is 0. I have no idea why this might be useful, but I think I can be safe always leaving it as 0 if I define vertex arrays similar to the way they are for the squareVertices matrix. So we have just pointed OpenGL at our array of corners that will be used to define our square. glEnableClientState(GL_VERTEX_ARRAY); Seemingly, as in other OpenGL ES calls, this one depends on different types of arrays. glEnableClientState enables the array type passed in as a parameter as being the "active" array. In this line, we activate the GL_VERTEX_ARRAY as being the active one. Essentially, this enables our squareVertices are as being active, and being displayable. So not only do we have to tell OpenGL about our vertices, we also have to tell it, that the matrix can be displayed. glColorPointer(4, GL_UNSIGNED_BYTE, 0, squareColors); This is similar to the glVertexPointer call. It points OpenGL ES to our matrix of colors squareColors. The parameters are size, which is set to 4 (R,G,B, and A), the GLtype again which is GL_UNSIGNED_BYTE, the stride again (0), and the actual color matrix to use. We then enable the color array as being active using: glEnableClientState(GL_COLOR_ARRAY); Try commenting these last two lines out to see what happens. The next line glDrawArrays(GL_TRIANGLE_STRIP, 0, 4); obviously draws using the the "enabled" arrays. But what is this GL_TRIANGLE_STRIP? It's a drawing primitive that draws a triangle between vertices. There are various choices, but to draw our square the template uses this triangular method. So what does that mean or look like? The second parameter is 0 which is the index number of the first element to draw, while the 4 is the count of vertices to render. If you're still confused by the notion of GL_TRIANGLE_STRIP try adding an extra vertex to our square, and change the count parameter in glDrawArrays to 5. You should be able to see a triangle. (Just don't choose an existing vertex location.) The next line glBindRenderbufferOES(GL_RENDERBUFFER_OES, viewRenderbuffer); we've seen at the start of this method. The only difference is we bind to the viewRenderbuffer instead of the viewFramebuffer. So we swap buffers. [context presentRenderbuffer:GL_RENDERBUFFER_OES]; The presentRenderbuffer method actually displays the buffer to the iPhone display. We see the image. That's the drawView method. There's a lot going on, especially in micromanaging the various internal OpenGL ES array states. Now that we've gone through the various calls, let's step back and review from a high level perspective. • drawView is the method that gets called by the timer to display a single frame of our OpenGL ES scene. This is the method you want to modify to customize your animation. • We have to set up two matrixes; one for the object we are drawing, and another for the color set that will be used to paint the object. • We bind OpenGL to one of our buffers (the viewFramebuffer). The viewFramebuffer becomes the one drawing will occur in. Drawing occurs offscreen. • We use glViewport to set our internal coordinate system and get back our display dimensions (in OpenGL ES terms.) • We set our window on the scene by specifying glMatrixMode(GL_PROJECTION), loading an identity matrix, and then applying a boundary cube using glOrthof(). Our window is a cube that encloses our scene. • In the template, we actually move the viewport by spinning it around the square, rather than spinning the square itself. This is done via a rotation transformation to the viewport. • We then set up a background color that will be used to erase the display prior to painting our scene. This is done by specifying a clear color and clearing the scene. • We finally get around to our square by setting a GL pointer to point to our square matrix. We enable it for drawing. We do the same thing for our color matrix. • The object gets rendered by calling glDrawArrays. This completes imaging the scene in our offscreen viewFramebuffer. • We render the contents of our offscreen viewFramebuffer and the scene is blitted (flipped) onto the iPhone display. • We switch our binding to our viewRenderbuffer. The next time drawView gets called, we flip back to our viewFramebuffer. I'm obviously simplifying a lot of stuff here, but this is to keep things understandable. At this stage, I'm more interested in how the Open GL ES calls fit together from a high level and what they do. Hopefully, I've been able to provide a view of the forest as well as some of the leaves on the trees. In the next installment, I'll finish up with the implementation of drawView, since its mostly smaller methods.

Part II - The iPhone OpenGL ES XCode Template Dissected - initWithCoder Logic

Sat, 21 Feb 2009 22:20:41 -0500

In the first part of this I went through the default XCode Template Header that is created when you start an iPhone OpenGL ES project. In this session, I'll work through the code in the actual implementation of that OpenGL ES-specific UIView. Because I don't want to be overwhelmed--or overwhelm you I'll take it one method at a time. So in this installment I'll work on the first real method initWithCoder, but first the preliminaries.

The main things I take away from the header is that OpenGL ES uses various buffers to do the drawing and animation, we always draw into a "context", and we need to make sure we deal with OpenGL ES on its own terms, by using OpenGL ES datatypes rather than those of c or Objective-C, since they seem to be platform independent.

Ok, lets dive into the implementation EAGLView.m.

The first thing to note are the imports:

#import <QuartzCore/QuartzCore.h>

#import <OpenGLES/EAGLDrawable.h>

#import "EAGLView.h"

OpenGL ES not only requires its own header, EAGLDrawable.h, but the Quartz header as well. If you're new to Mac or iPhone development, Quartz is the imaging (drawing) framework used by these platforms. So OpenGL ES doesn't replace the fundamental drawing capabilities of the iPhone (or Mac), but rather adds a layer of capability on top of it. That is to say, it seems you can't create a pure OpenGL ES application. This makes sense, since Quartz is used to handle the platform specific UI widgets and layer drawing. And you can't really have an application without these.

We of course also import the header we went through in the first part.

What follows next is:

// A class extension to declare private methods

@interface EAGLView ()

@property (nonatomic, retain) EAGLContext *context;

@property (nonatomic, assign) NSTimer *animationTimer;

- (BOOL) createFramebuffer;

- (void) destroyFramebuffer;

@end

As the comment points out this is the typical way of extending the interface (header) to declare private methods. The main thing that happens here is that we declare the attributes for the OpenGL ES context, and the timer we discussed last time. This is done using the @property command. Two additional private methods are declare in addition to the three public methods in the header. The two appear complimentary, in that they manage the lifecycle of a framebuffer. This again makes sense, since it would be bad design to have one UIView try to create buffers used by another UIView. In this template project there is only one view, but still it's a good idea to have the view manage its own buffers. Isolation is a good thing, as is cohesion. Finally we get to the start of the implementation: @implementation EAGLView @synthesize context; @synthesize animationTimer; @synthesize animationInterval; Nothing new here really, we just synthesize the getters and setters for the properties we defined earlier. We next see our first true method. // You must implement this method + (Class)layerClass { NSLog([[CAEAGLLayer class]description]); return [CAEAGLLayer class]; } This is a class method that returns the class of CAEAGLLayer. If you remember, this is a wrapper around our drawing environment. So what does this return? It returns a class CAEAGLLayer. The comment says that you "must implment this". What happens if you don't? If you take out this method and try to run the application, it throws an exception: *** -[CALayer setDrawableProperties:]: unrecognized selector sent to instance 0x5242b0 Digging through the docs tells me that CAEAGLLayer conforms to the EAGLDrawable protocol which says that our CAEAGLLayer can be used to draw on and be displayed by an EAGLContext. We have one of those defined in our interface. So when we draw to our context, it actually does the display on our layer. The setDrawableProperties is part of the EAGLDrawable protocol. By not implementing this peculiar method, we break the protocol. I'll work on figuring out when and how it gets called later. Next is the init method the real meat of this article. //The GL view is stored in the nib file. When it's unarchived it's sent -initWithCoder: - (id)initWithCoder:(NSCoder*)coder { if ((self = [super initWithCoder:coder])) { A quick peek at the xib file doesn't really show anything OpenGL specific other than the fact that the main window is a subclass of us--which makes sense. We'll get into the xib later. We finally see something new: // Get the layer CAEAGLLayer *eaglLayer = (CAEAGLLayer *)self.layer; The self.layer returns the generic UIView's CoreAnimation layer. This is the layer that is used for drawing things on the view. This is a drawing surface used by the iPhone. We cast this to a CAEAGLLayer, so we can deal with the drawing layer as if it were an OpenGL ES drawing layer instead of a CoreAnimation one. eaglLayer.opaque = YES; Then makes the layer opaque. Interesting in that this is done to the underlying UIView, not to the coerced OpenGL ES layer. This doesn't really do anything obvious in the template. If you comment this line out and rerun the app it looks identical. You probably want to set the layer to be opaque in the event you want your view to be a subview on top of something else. The next line: eaglLayer.drawableProperties = [NSDictionary dictionaryWithObjectsAndKeys: [NSNumber numberWithBool:NO], kEAGLDrawablePropertyRetainedBacking, kEAGLColorFormatRGBA8, kEAGLDrawablePropertyColorFormat, nil]; handles the EAGLDrawable protocol by setting the drawableProperties which is a dictionary that describes the properties of the drawing surface (our layer.) This is defined as a dictionary. So it seems OpenGL ES allows us to specify how the background looks/acts in addition to the actual thing we are drawing. That is to say, we not only have to worry about drawing a square (or something), but have to also worry about drawing the background properly by setting its drawableProperties. There are two pairs in this dictionary. The first key: kEAGLDrawablePropertyRetainedBacking is set to 'false'. What kEAGLDrawablePropertyRetainedBacking does is determine whether the surface retains whatever is drawn on it. The way I understand this is when you draw on the surface (layer) the image drawn is not treated as an object that can be refreshed by the surface, but rather as a phantom image that either the code has to refresh, or is subject to erasure. A simpler way to put it is the layer won't autorefresh its contents. The documentation states this is the preferred and default setting. That's my interpretation--let me know if I'm mistaken. The second key: kEAGLDrawablePropertyColorFormat is a little easier to understand. This sets the internal color buffer for the layer. The template sets this to: kEAGLColorFormatRGBA8 which is a 16 bit RGB with Alpha. (This amounts to 2 to the 16th power if you didn't know--a ton of color choices.) This is the default value In fact this is the default dictionary for drawableProperties. What is interesting, is if you comment out this line and rerun the application, the color scheme of the cube seems to change--it has more white. I'm not sure why. So that takes care of setting up our drawing layer. We now need to deal with the context. context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES1]; So this creates our OpenGL ES drawing context, and specifies that it uses the OpenGL ES 1.0 API. kEAGLRenderingAPIOpenGLES1 is the only option available for the iPhone as of 2.2. In a more general sense, I guess you can create an OpenGL context, and it conforms to a given OpenGL API. The next snippet is important. if (!context || ![EAGLContext setCurrentContext:context]) { [self release]; return nil; } This checks to make sure we got a valid context object, and sets the current OpenGL ES context to be the context we just created. If we can't set the context we exit. Why is that? Well from our dissection of the header, we found out that we can only draw into a context, so if we can't get one or set it to be the current context, we can't draw anything, so we may as well quit. animationInterval = 1.0 / 60.0; Sets the animation interval (frame rate) to 1/60 of a second, so fps is 60. That's very good for animation. That's the end of the initWithCoder method. Next time we'll tackle the drawView method.

Part I - The iPhone OpenGL ES XCode Template Header Dissected

Sun, 15 Feb 2009 11:20:38 -0500

If you create, build and run the sample template you end up with an iPhone application that spins a multicolored square. Let's dive in and see if I can make sense of this.

The critical file is EAGLView.m, so let's take a look at how it's defined.

/*

This class wraps the CAEAGLLayer from CoreAnimation into a convenient UIView subclass.

The view content is basically an EAGL surface you render your OpenGL scene into.

Note that setting the view non-opaque will only work if the EAGL surface has an alpha channel.

*/

The comment refers to CAEAGLLayer which the iPhone docs state is the canvas in which OpenGL ES drawing occurs.  It also says that this is displayed by  CoreAnimation--my guess then is that CoreAnimation is responsible for doing at least some of the graphics "heavy lifting" for OpenGL ES.  I could be wrong, but I'm going with that until something else proves me wrong.  Does it matter?  Probably, if  I"m interested in performance issues, but at this stage I'm just trying to wrap my mind around what is going on.

AGLView.h says that this is a plain UIView. No protocols or delegates; nice and simple. All of the properties are private. The first thing we really see that is new are the two dimensions

/* The pixel dimensions of the backbuffer */ GLint backingWidth; GLint backingHeight; Ok, the first question is, what is a "backbuffer"? I have a little basoc animation-programming background, so I have a guess. I'm willing to guess that the backbuffer is the offscreen buffer we draw to while the actual screen view displays a completed rendering. We do this in order to not show the screen actually drawing. So we "blit" the backbuffer to the main view in order to animate a frame.

The actual dimensions are delared as GLint. Obviously, these are integers, but why declare them as GLint? According to the spec, GLints datatypes are not "c" datatypes, and a GLint is a 32bit integer. Does it matter from the perspective of our work? Probably not. It's sufficient that we know we just use the GL-specific datatypes rather than the iPhone or "c" datatypes. So we've just defined variables to hold the height and width of the backbuffer. What if we don't want to do animation? We can probably get rid of the backbuffer if all we want to do is create a static screen using OpenGL ES.

The next property is defined as:

EAGLContext *context;

The "context" is the environment to which we draw, and everything we draw uses a context. The iPhone docs say we have to bind/attach a framebuffer before we can actually use the context. So how is a backbuffer different from a framebuffer? A buffer is just a glob of memory. The backbuffer is basically a framebuffer that will be displayed--think of it as the next framebuffer. The spec says specifically that a framebuffer is related to the graphics hardware. Do I have to worry about this? Probably--but not from the point of learning. I'll probably have to figure out whether/how the iPhone graphics chips deal with framebuffers if I want optimal performance, but for right now, that's just a point of interest.

The next bit:

    /* OpenGL names for the renderbuffer and framebuffers used to render to this view */

GLuint viewRenderbuffer, viewFramebuffer;

uses the term "renderbuffer." A "renderbuffer" is just the buffer that is being drawn--it is a 2D pixel image. Why is this important? A quick browse of OpenGL on the web and even the EAGLView.m file shows that OpenGL uses matrices (arrays) to describe an object's vertices (points). So a lot of OpenGL ES is based on vector graphics, not solid images. The solid images come from rendering (drawing) a vector image and then "filling it in". A vector image is a based on lines and points. But the spinning square is made up of colored pixels. It seems obvious we need to somehow convert the vector representation to a bitmap one. The renderbuffer is the result of that transformation, is my guess.

A vector graphics cube

A bitmap cube.

Now that we have a "gut feel" for these buffers, the next thing to note is that they are declared as GLuints--integers again. So these must be pointers--but they don't use the 'c' * pointer syntax? Interesting if true--I guess we'll see how these variables are used.

/* OpenGL name for the depth buffer that is attached to viewFramebuffer, if it exists (0 if it does not exist) */

GLuint depthRenderbuffer;

This appears to describe the bit-depth of the renderBuffer. But the comment refers to viewFrameBuffer--confusing. Is it one, or the other or both? I'm willing to bet its both. After all, following my logic (which I admit can be totally wrong since I'm learning this for the first time) both buffers get displayed on the iPhone, so its a good idea to have a consistent bit-depth for every buffer that gets displayed. depthRenderBuffer will probably be used to hold a number that describes the max number of bits used to hold color information (maximum number of colors).

Finallly we have two Cocoa properties:

NSTimer *animationTimer;

NSTimeInterval animationInterval;

Nothing fancy here, a timer and a timer interval that I bet will be used to drive the animation by triggering how fast (frequently) the frames flip.

We then have the property and method declarations:

@property NSTimeInterval animationInterval;

   - (void)startAnimation;

   - (void)stopAnimation;

   - (void)drawView;

Nice understandable names.

That's it for part one.  Let me know what you think, or if I'm totally off track; I'm sure I've made some incorrect assumptions along the way, but as I said this is my attempt at learning.  You learn by making mistakes.