I let a local AI sort and tag forty thousand photos
# I let a local AI sort and tag forty thousand photos
My photo library was about forty thousand images deep. A decade of them in one giant pile: screenshots next to birthdays next to blurry receipts next to actual memories I would be sad to lose. The mess itself was tolerable. The unfindability was the real problem. If I wanted the photo of my dad at the beach from a few summers back, the only tool I had was scrolling, and scrolling always ended in giving up.
So I fixed it properly. A vision model running on my own graphics card walked the entire library, described every photo, tagged it, and proposed where it should live. Nothing was uploaded anywhere. Here is how the pieces fit together, what the model got wrong, and whether I would tell you to build the same thing.
Why nothing left my machine
Most easy photo organizing tools want your library in their cloud first, and for plenty of people that trade is fine. These are my family photos. My kid, my parents, the inside of my house, where we go, and by implication when we are away. I did not want that data sitting on a server I do not control, feeding whatever training pipeline it feeds. So the project had one hard rule from the start: every photo stays on my drive, the model that looks at them runs on my card, and no image is transmitted anywhere.
That rule shaped everything. It ruled out paying for a slick app and forced me to understand the pieces and wire them together myself. In exchange, I never have to wonder who else can see my photos. The answer is nobody.
The second reason ran nearly as deep: I did not want to rent this forever. Cloud photo tools are mostly subscriptions, and when you stop paying, the convenience walks out with your money, sometimes along with access itself. I wanted something that will still work in ten years regardless of any company's pricing page or continued existence. On my own hardware, the only dependency keeping it alive is me. No account to be locked out of, no service to sunset, no terms that change overnight. For a pile of memories I want to find when I am old, that durability mattered as much as the privacy.
A vision model turns pictures into words
The core of the build is simpler than it sounds. A vision model takes an image and returns words: a plain description of what it sees. A dog on a wooden floor in sunlight. A plate of food on a restaurant table. A screenshot of a calendar. Same family of model that drafts my scripts, except this one can look.
That translation is the entire unlock. Computers are hopeless at searching raw pixels and brilliant at searching text. So the pipeline walks every photo once, asks the model to describe it, and saves the description alongside the image. One boring pass, and forty thousand silent files can suddenly say what they are. Nothing magic is happening. The model is just extremely patient, and it feels exactly the same about photo nine thousand as it did about photo one.
Descriptions become tags, tags become search
A paragraph per photo is useful, and it is still the wrong shape for finding things. The next step boils each description down to a handful of short labels: beach, food, screenshot, dog, document, people, sunset. Those tags attach to the image, and the library becomes genuinely searchable. Every photo tagged beach, every screenshot, every picture with food in it, each answered in under a second instead of ten minutes of scrolling.
The tags also surfaced things I would never have labelled by hand. A few hundred of my photos turned out to be screenshots and receipts, the junk that clogs a library and buries the real memories. Tagged as a group, they finally became dealable as a group, instead of something I stumbled past one at a time forever.
The surprise was how much I use the full descriptions, beyond the tags. Because the model wrote an actual sentence about each photo, I can search for things no planned tag would ever cover. A child holding a yellow balloon. A sunset over water with a boat in it. The description sits there in plain words, text search is instant, and my fuzzy memory of a photo suddenly has something exact to match against. I assumed I would browse by category. In practice I type roughly what I remember and the photo surfaces. That has become the feature I touch most.
Sorting is rules on top, and it proposes before it moves
With tags in place, sorting becomes a layer of plain rules. Screenshots go to a screenshots folder. Receipts and documents go to paperwork. Photos with people plus a tag like party or birthday group together. The model does the genuinely hard part, understanding what is in each frame, and simple rules I wrote decide where each kind of thing belongs.
One design decision I would defend to anyone: the system never rearranges the library in one shot. It first writes out a plan, a long list of this photo would move here, that one there, and I read a decent chunk of it before anything moves. The photos stay put until I say go. For a tool holding ten years of memories, suggest first is the only sane mode. A tool that proposes is safe to iterate on. A tool that just acts on forty thousand irreplaceable files is terrifying.
What it gets wrong, confidently
The demos never show this part. The model makes mistakes, and it makes them with total confidence. It called my black cat a dog more than once. It looked at snow and said beach, since bright and white was apparently close enough. It fixated on a poster in the background and tagged the entire photo as if the poster were the subject.
None of these are disasters, and they add up. Trust every tag blindly and you get a library sorted confidently and wrongly, which is arguably worse than no sorting at all. So the tags are a fast first pass, never a verdict. For the folders I actually care about, the family ones, I still skim before trusting a grouping. The model does ninety percent of a job I was never going to do by hand, in an evening. The last ten percent stays mine, and I have made my peace with that arithmetic.
There is a quieter failure that took longer to notice. The model cannot tell apart things that look similar but mean different things to me. It does not know which child in a frame is mine, and frankly I would not want it to. It lumps two entirely different trips together because both happened on a beach. It has no idea that one blurry photo is precious because of who is in it while a sharp one is junk. That meaning lives in my head and no amount of describing pixels extracts it. Once I stopped asking the model what pictures mean and kept it on what pictures contain, the disappointment stopped and the value went up.
What it costs to run
People assume a build like this needs serious money. The vision model runs on a card I already owned, an old 1080 Ti several years past its prime. Slow by modern standards, and slow is fine here, because nobody sits waiting for it. I kick the job off in the evening and it grinds overnight. The full forty thousand photo pass took a couple of nights. After that, only new photos need processing, a handful a day, done in seconds.
Ongoing cost is effectively electricity on hardware I already had. No monthly fee, no per photo charge, no subscription ticking away. The real spend was one weekend of setup and rule writing, and that weekend now pays me back every time I find a photo in seconds instead of surrendering after ten minutes.
What stayed my job
The model does not decide what to delete. It can report four hundred near identical screenshots; whether they go is my call every time, because I am the only one who knows which still matter. It did not design the folder structure either. By year, by person, by event, that shape came from me, since it is a taste decision rather than a technical one. And it did not replace actually looking at my own memories. If anything I do more of that now, because I can finally find them.
The division of labor lands where it should. The machine handled the part no human was ever going to do, describing forty thousand photos one by one. What to keep, and what any of it means, stayed with me.
Should you build one
For some people, honestly no. If your photos already feel manageable, or a cloud app holding them does not bother you, this is more effort than it returns. If you have a giant unsorted pile and the privacy angle nags at you even slightly, a weekend of setup is genuinely worth it. No new computer required. An old graphics card and some overnight patience cover it.
There is also a quiet satisfaction that the thing which understands my photo library lives under my desk rather than in someone else's data center. I plan to keep refining mine, and a deeper walkthrough of the exact model and the sorting rules may follow. More builds like this, without handing your life to the cloud, at [xavierfok.com](/).
Get new guides and videos first — join the Telegram channel.