NSArray @property backed by a NSMutableArray

Saw a bunch of posts today on Stack Overflow asking how to implement this:

NSArray @property backed by a NSMutableArray

Hiding privately mutable properties behind immutable interfaces in Objective-C (my answer)

Protect from adding object to NSMutableArray in public interface

Having readonly nsarray property and nsmutable array not readonly with the same name and _ in xcode 4.2 vs 4.5

How to expose an NSMutableArray as an NSArray as a return type from a method

The main reason is to expose an array as immutable in the public interface, to convey that it shouldn’t be modified externally, but can be mutated internally. This is a typical thing you would want to do in a Model class, e.g. having an insertObject method and then notifying controllers via a delegate and notifications. One of the stumbling blocks I noticed is people were trying to implement is solely with their knowledge of properties, where you can redefine a readonly property as readwrite internally using a category, unfortunately you cannot change its type. With some understanding of ObjC before automatic synthesis the solution is to manually implement the ivar that backs the property, like this:

.h file:

@interface Model : NSObject

@property (nonatomic, strong) NSArray *objects;

-(void)insertObject:(id)object

// todo: delegate
@end

.m file:

@implementation Model{
    NSMutableArray* _objects;
}

-(void)insertObject:(id)object{
    if(!_objects){
        _objects = [[NSMutableArray alloc] init];
    }
    [_objects addObject:object];
}

@end

Just one of the many reasons developers have such a difficulty extracting their model from the view controller in Apple’s default project templates.

NSOperationQueue maxConcurrentOperationCount 1 Serial Queue Problem

Around the web and even in Apple’s WWDC 2015 Advanced NSOperations talk they say:

“If we set the max concurrent operation count of an NSOperationQueue to be 1, then we essentially make our NSOperationQueue into a serial operation queue.”

Sometimes developers also refer to this configuration as a first in first out (FIFO) queue. It works ok in very simple cases but by ignoring operation dependencies things can go bad fast. To illustrate what I mean, see this sample code:

    NSOperationQueue *queue = [[NSOperationQueue alloc] init];
    queue.maxConcurrentOperationCount = 1;
    
    NSBlockOperation *op1 = [NSBlockOperation blockOperationWithBlock:^{
        NSLog(@"op1");
    }];
    
    NSBlockOperation *op2 = [NSBlockOperation blockOperationWithBlock:^{
        NSLog(@"op2");
    }];
    
    NSBlockOperation *op3 = [NSBlockOperation blockOperationWithBlock:^{
        NSLog(@"op3");
    }];
    
    NSBlockOperation *waitOp = [NSBlockOperation blockOperationWithBlock:^{
        NSLog(@"op4");
    }];
    
    // uncomment to fix problem
    // [op2 addDependency:op1];
    // [op3 addDependency:op2];

    [op2 addDependency: waitOp];
    
    [queue addOperations:@[op1, op2, op3] waitUntilFinished:NO];

The output from this code is:

2017-01-07 22:11:43.322770 OpTest[24703:35783606] op1
2017-01-07 22:11:43.322846 OpTest[24703:35783564] op3

What has happened is since op2 has a dependency on waitOp (which has been made unable to be started), op3 has skipped the queue and jumped ahead of op2. If it was a serial queue then you might expect only “op1” to be output and the queue waiting on waitOp to finish to then start op2.

By uncommenting the 2 addDependency lines we then get the expected behaviour:

2017-01-07 22:27:23.876388 OpTest[24909:35842006] op1

The funny thing is that with the dependencies set up correctly we can now remove the line that sets the maxConcurrentOperationCount and the behaviour stays correct. The problem is that usually developers want to retry their operations which does require understanding dependencies in which case I believe it would have been better to learn them from the outset rather than use this trick. Also if they do then they will be one step closer to learning the real power which is cross-queue dependencies.

YapDatabase Disappointing CloudKit Example

It’s not often you see code as disappointing as this, particularly because CloudKit is so well designed for the situation being described, but it’s actually very common. I think it happens when you have a developer with speciality in one area, trying to apply it to their understanding of how to implement something in a new area, and just getting the job done as quickly as possible.

I was searching Github for examples of how people are handling CKErrorChangeTokenExpired, and came across this CloudKitManager.m by YapDatabase. In scrolling through I noticed this function that looks to be chaining some CloudKit setup operations together:

- (void)continueCloudKitFlow
{
    DDLogInfo(@"%@ - %@", THIS_FILE, THIS_METHOD);
    
    if (self.needsCreateZone)
    {
        [self createZone];
    }
    else if (self.needsCreateZoneSubscription)
    {
        [self createZoneSubscription];
    }
    else if (self.needsFetchRecordChangesAfterAppLaunch)
    {
        [self fetchRecordChangesAfterAppLaunch];
    }

First thought is, since these methods are usually all async and dependent on one another how can they all be called in line like this? Taking a look at one of these methods gives an ugly surprise! (And I don’t mean them using tabs instead of spaces but yeh that’s ugly too! Had to even fix them just to paste into WordPress.)

- (void)createZone
{
    dispatch_async(setupQueue, ^{ @autoreleasepool {
        
        // Suspend the queue.
        // We will resume it upon completion of the operation.
        // This ensures that there is only one outstanding operation at a time.
        dispatch_suspend(setupQueue);
        
        [self _createZone];
    }});
}

So all of the methods are called one after another instantly, but to make the actual methods (prefixed with underscore) run serially, the developer runs them as blocks on a GCD dispatch queue, and when each block starts it pauses the  queue to prevent the other ones being called. Unbelievable!

The question you are all asking is why on earth didn’t the developer simply make use of CloudKit’s beautiful use of NSOperations and just set up dependencies via an NSOperationQueue to run them serially automatically? I would guess the answer is this particular developer is an expert in GCD (based on the information in the Readme of the project CocoaAsyncSockets), so put simply, they used what they knew to solve the problem.

It’s one of those situations where they got the job done, so that’s great, but I think if everyone takes a moment to learn the NSOperation API properly and see how amazingly well CloudKit builds upon it, it would bring much more enjoyment to developers.

In case you were wondering how it should be done, is the CloudKit operations don’t have to be added to a database or container via addOperation to be processed, a database property can be set on the operations and then it can be added to a custom operation queue, thus allows you to group your operations together, and with dependencies order them serially. It is easy to miss but in the CKDatabase header it says “or schedule operations on your own queue”, that’s the secret.

Xcode 8 iOS 10 Non-Stop Device Logs

This post is about my Stack Overflow answer.

In Xcode, Window->Devices, choosing a device and clicking the up arrow in the bottom bar brings up the console and this traditionally allowed you to view output from your app when not running in the debugger, e.g. for testing background launching behaviour. Unfortunately as of Xcode 8 and iOS 10 there is now a lot more log output – very low level debug logging is on so hundreds of lines pass by every second, making it impossible to see the logs you are interested in.

The solution is to upgrade to macOS Sierra, the new Console app now displays connected devices, and it allows filtering the log by many different categories, process is probably the most handy where you can simply enter the name of your app’s executable. This now makes the Xcode Devices log redundant, it never was very good at search anyway.

CKRecordZoneSubscription Notes

If you attempt to set desiredKeys on a notificationInfo for the new iOS 10 CKRecordZoneSubscription class, saving results in the runtime error:

<CKError 0x17424b610: "Invalid Arguments" (12/2006); server message = "cannot add additionalFields to this subscription type"; uuid = 4E08C616-97AA-4E9A-B584-B7972B1CD99B; container ID = "iCloud.com.x">

Which makes sense given the notification is about a zone changing and not a record, however one might assume it can send the record that was responsible for the change but looking to CKRecordZoneNotification shows no record properties are available. It’s always kind of disappointing to see subclassing not working out, i.e. where the subclass is denied features of the parent, it can sometimes point to bad class design however in this case Apple could have handled it better either by client side validation or at least a note in the header. It’s extra disappointing given in iOS 10 they refactored from flat CKSubscription inits for the different subscription types (which did have their own limitations, e.g. initWithZoneID only allowed zero for options) to subclasses. I hope they’ve thought this new design through because if it is refactored again it could be a real headache for framework backwards compatibility.

 

CloudKit Syncing References

Here are some references from around the web on how to implement CloudKit syncing. It will be added to…

9th August 2017

In the Build Better Apps with CloudKit Dashboard WWDC 2017 video at 4:30 Dave Browning presents a ToDo list app that syncs from Core Data with CloudKit 6:15. I wrote to Dave in June and he told me that they do plan to release the source for that app, they just need to clean it up a bit and review it internally first. He said he’d let me know once it has been published but it has been a few months now. I’ll update this post if it gets released.

5th January 2017

As I Learn CloudKit Syncing by Eric Allam

Probably my favourite tutorial so far, Part 1 follows the example given in the Advanced CloudKit talk from 2014, unfortunately the talk and this tutorial doesn’t get as far as using push notifications via a CKRecordZoneSubscription and resorts to fetch only. It correctly uses CKFetchRecordChangesOperation for this, however it contains a mistake in part 4. It says that only changed properties are included in the CKRecord given by the recordChangedBlock, in fact the full record is included by default, and it now also supports a desiredKeys in the newer version of this class named CKFetchRecordZoneChangesOperation. It correctly identifies using NSOperation dependencies as the right approach for queuing related operations, however in seek of queue error handling Eric looks to the Advanced NSOperations talk and corresponding open source project, which unfortunately is of a different design to CloudKit NSOperations and he hits some problems.

5th January 2017

Seam – Seamless CloudKit Sync with CoreData

This project provides tight coupling with CoreData that seems to go against Apple’s designs of keeping frameworks more loose. For example, it wraps conflict handling which might be something the developer requires full control over. It’s always the same with wrapping, you end up hiding away something needed and then having to write more code to expose it again. Opting certain entities out of syncing requires extra effort, and it appears to support only an all-or-nothing sync, rather than say just up or down. The project was coded in Swift so unfortunately fell victim to the constant syntax changes of the language, and has been left broken on Swift 3.0. Furthermore, NSOperation is designed around returning errors rather than exceptions that Swift uses, I’m not sure how well the developer handles this but it does seem a potential area of weakness.

5th January 2017

CloudKit + Core Data + NSOperations – Syncing by Nick Harris

This is the top tutorial on Google for CloudKit syncing, but sadly has mistakes and leaves many unanswered questions. Uniquely, Nick uses a record zone per record type which obviously isn’t the right approach for these kind of records, he realises this later, and the reason is that CKRecordZoneSubscription can be scoped to a recordType. Another mistake is by subclassing CKFetchRecordsOperation to perform processing, however he strangely does then use correctly use NSOperation dependencies for the sync method. One interesting approach is to use a separate entity to store deletes, so that the real entity can be deleted as normal rather than just soft deleted; this has the advantage that predicates don’t need the additional where status != deleted.  There doesn’t appear to be any error handling if any one of the operations fails, furthermore the synchronous processing NSOperation just fatal errors if anything goes wrong with the Core Data method calls. I believe if he had access to an asynchronous NSOperationQueue class that supports errors and cancellation it would have helped a lot. Written in Swift so again falls victim to syntax changes.

5th January 2017

Apple CloudKit Engineer on Stack Overflow

A developer named farktronix gives great insight into the inner workings of CKFetchRecordZoneChangesOperation on the server. From the name of the class you might think that this returns all changes or deletes, which at first glance would appear far too much data to be useful for an initial sync, so for example, you might think instead to query to get the first set. However the developer states that the server coalesces changes before sending them (which gives the bug being discussed), this means that if a record was created and then deleted, the server is smart enough not to send either of those events in the fetch changes operation. This is a really great piece of insider info, and hopefully this user shares more in the future.

Enabling iOS 9.3 SDK in Xcode 8 for iOS 10

Some of the new iOS 10 features contain link-time checks, that is they are only enabled if the app was linked to the iOS 10 SDK. When working on an open source library it is necessary to debug when linked to an older SDK, e.g. iOS 9.3, on a newer device iOS, e.g. iOS 10. It currently isn’t possible to deploy to a newer iOS device with the old Xcode designed for the older SDK. For this reason it can be necessary to enable the older SDK in the newer Xcode, which can be done as follows:

Note: these commands assume Xcode 7 is at \Applications\Xcode.app and Xcode 8 is at \Applications\Xcode-beta.app

sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS9.3.sdk

sudo ln -s /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator9.3.sdk

sudo /usr/libexec/PlistBuddy -c "Set :MinimumSDKVersion 9.3" /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneOS.platform/Info.plist 

sudo /usr/libexec/PlistBuddy -c "Set :MinimumSDKVersion 9.3" /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Info.plist

After running these commands, restart Xcode 8. Now you can set the Base SDK to iOS 9.3 and deploy to an iOS 10 device.

Any newer code generation features in Xcode 8 will need to be turned off, e.g. a Core Data model will need to have code generation set to none.

Welcome to my Developer Blog

I thought it would be good to start this developer blog with some photos of how my iOS development began. At university during my Ph.D. I had been building .NET Compact Framework apps in C# for Pocket PCs, usually on iPaqs with GPS sleeves and using Wi-Fi.

A bunch of iPaqs and I – 5th July 2004

Then in 2007 the iPhone came out. That summer, during my job at Microsoft as a lowly intern we bought iPhones for “research”.

Me at Microsoft – 25th July 2007

Given it was initially a US only product and SIM locked to AT&T I followed Geohot’s live blog on jailbreaking for purposes of SIM unlocking. We used Microsoft’s micro-soldering equipment to attach the wires for the hardware jailbreak, then performed the SIM unlock and then I could begin using the iPhone on a network the UK, must have been one of the first!

My first iPhone open with the jailbreaking wires attached – 25th August 2007

Microsoft was a fantastic experience but I also learned that I never wanted to work at a corporation again. So from there I began reverse engineering the frameworks and building custom apps and even distributing them via Installer that was released that same summer, and then the following year, Cydia. My first jailbreak apps were primarily around Wi-Fi scanning – my own app WiFiFoFum and a Wi-Fi positioning system for Navizon. Apple picked up on this jailbreak app phenomenon and subsequently released their public SDK and App Store, at which point WiFiFoFum was almost already to go and was one of the first apps in the store. Everything was going great with thousands of sales per day until unfortunately Apple decided to ban it for using private APIs. They still to this day do not allow Wi-Fi scanners in the store.

Since then I have been perfecting my skills, researching everything about iOS, Macs, Cocoa and Objective-C, learning all the new technologies as they become available. I develop my own apps and ones for academic research projects and the odd bit of consultancy. I’m still very interested in Wi-Fi but more recently I have developed an interest in realtime syncing of information between devices and apps for health. Hopefully you will find any information on this blog useful.