Notes from a human

I like building things. Probably more than I like running or playing with things. I was the guy that was building a perfect society in StarCraft while Zerglings overan my city. SimCity was my kryptonite. I was always more interesting in shell scripts and perl scripts than giant apps. I am always trying to build a re-usable workbook when all I really have to do is build a quick spreadsheet. I guess I like systems and tools. This is the story of how I built a MacOS utility application while starting to build an iOS app.

My professional background is in scripting languages of every kind, Java, and enterprise web applications. I've also been a Mac geek for years and have thought a lot about building my own MacOS or iOS apps. But my half-hearted attempts at building an iOS app always stalled at the tutorial level. Like everything else I've ever learned, I needed a project or a client to drive me, and that motivation was never quite strong enough to get me over the mountain of domain knowledge I'd have to climb first. But lately, like everyone else, I've spent the last year obsessing over LLMs and AI-assisted coding. Enter Claude. Claude helped me build something cool¹, taught me the basics of macOS and iOS app development, and sent me down a string of super fun side quests.

My first idea came out of my side job as a kids' sports taxi driver. I spend a lot of hours driving my kids to sports and then watching the practices and games. Somewhere in there I started looking at sports apps. There are a lot of fun apps that record and analyze video. Cameras that other teams and parents hoist on super tall tripods that record and analyze games. Some more focused tools that claim to analyze game or player mechanics and suggest improvements. That took me down a side quest to understand how all of these tools work. Which led me to the current state of computer vision and ML models. That all seems pretty cool but it all seemed to rely on models and code in the cloud. Apple has been making noise for a while about ML models. so what works on iOS? So with the help of Claude I got a working app going on iOS but kept hitting the limits of the vision models available to me. That part alone was a thrill. I had a real, working iOS app doing object recognition. Something that I could keep iterating on. But more importantly, I finally understood more about how a modern Swift app gets built. So I reactivated my Apple developer account, started reading documentation, and watched WWDC videos. I'm no expert. I depend heavily on the skills that other experts are releasing into the world to tell me what good looks like in modern Swift development. And I'm a long, long way from building on my own. But the same skills I use every day directing Claude and similar tools for co-development carried me much further than I ever expected. And that is the promise of burning all of the tokens on LLMs, isn't it? Turbo charge our development cycles, while hopefully bringing our skills as developers to help recognize what good looks like².

And that ML model side quest kept going. I dug into which models run well on iOS, what's actually out there, and how models get trained in the first place. There are a lot of really great models out there that have general object recognition capabilities. Some of them even run decently on iOS. But if you want to recognize something specific, and do it reliably, you start to hunt for ML Models that are trained on specific datasets. Essentially thousands of images of the objects you want to recognize, all labeled with bounding boxes, or polygons, or masks. In sports vision, the big sports get a lot of attention. Football (not American) has piles of models and public datasets fully annotated with labels. Other sports, not so much. The models that work beautifully on football (still talking about soccer) don't fully generalize to other sports, except at the very highest level.

So I read up on training models with custom datasets, and briefly got lost in Hugging Face and Roboflow. Turns out I did need my own model, especially if I didn't want to depend on online services and calling APIs. And models that run locally on iOS come with their own specific requirements. After one more side quest into the many ways to train a model, Roboflow model training tools, Apple's CreateML application, and converting models like RF-DETR to CoreML, I came up for air and started labeling images.

I started using Roboflow and then Label Studio. I didn't love either one. There were subscription costs. There was the jankiness of working in a browser. There was the mountain of python installs. Turns out there are a few apps on the App Store, but they seemed too limited to me, or had not been updated in a while. But let's be honest, I just wanted to build my own thing. So I decided take my workflow local. Uh oh. Another side quest. So I built a macOS app to annotate images and export datasets for object recognition model training. It works for me. Pretty darn well, in fact. And that made me wonder whether other people might want it too. People like me, before all these side quests, who don't want to pay a subscription, don't need a big web-based collaboration tool, and don't want to install a mountain of Python tooling. Surely there is at least one other person out there in the world who would find this useful. Welp. Side quest number four was about how the heck you release a MacOS app into the world these days. And side quest number five was this web site.

But here's what came out of side quest number three: yaAnnotator. Yet another Annotator. A macOS app for building object-detection datasets entirely on your Mac. Import and export the most common dataset formats. I bundle two models to enable "smart select" and object detection on a whole image to get you started. Bring your own models to speed up annotation even more. And this is all pre-MacOS 27. I'm actively using it. I hope you will too. Go get it from the App store and build the dataset of your dreams.

There are probably a lot more features to add. But somebody famous once said: "Real Artists Ship." So Version 1 is out, version 1.2 is on its way, and I am excited to incorporate MacOS 27. And I might even get back to that first idea one of these days ...