Phone vs Server AI Agents: Where the Work Happens
By Clifford · Product research
Written by the Memex team. Platform documentation and Memex source checked on October 17, 2026. This is an architecture discussion, not a measured battery, speed or background-reliability benchmark. The proposed hybrid design is not a shipped Memex feature.
You record a voice note after a walk: “We could visit Kyoto in spring, but I don’t want another trip packed with appointments.” Today, you want to keep the thought and perhaps connect it to an earlier note. Months later, you may want an agent to investigate dates and travel options. The record is useful long before the research task exists.
Both interactions can appear in the same chat window. One belongs close to a moment you are living; the other may need to continue after you put the phone away. Where the agent runs changes what can happen in that interval—and what the app owes you when the work stops.
Three locations, three separate decisions
An agent’s coordinator reads input, asks a model what to do, calls tools and records the outcome. That coordinator can run in a phone app while the model runs at an API provider. The archive is a third location: keeping your journal on the phone does not determine where inference happens. Tool execution has its own boundary too: a phone coordinator may edit a local file or call a remote calendar API.
“Local model” also needs a location. If your laptop’s Ollama server actually runs the chosen model locally, inference happens on the laptop; merely connecting a phone to that endpoint establishes neither phone-only inference nor the absence of cloud forwarding. The comparison below concerns the coordinator and its tools, rather than assuming all three layers move together.
| Decision | Phone coordinator | Server coordinator |
|---|---|---|
| Agent loop | Runs within the app’s execution environment. | Runs in a remote worker or service. |
| Records | Can work against the app’s local archive. | Needs authorized access to relevant records or a supplied copy. |
| Model inference | May use an on-device model or a remote endpoint. | May use a model on the host or a separate endpoint. |
| After the phone app closes | Depends on OS runtime and recovery mechanisms. | Can continue independently if the host and job system remain available. |
| Tools and permissions | Bounded by app permissions and the operating system. | Bounded by the host’s tools, credentials and explicit device connections. |
Background execution is a platform contract
On iOS, an app cannot turn an arbitrary loop into a permanent service. The system manages background runtime. iOS and iPadOS 26 add BGContinuedProcessingTask for work explicitly started by a person, with visible progress, cancellation and expiration handling. It broadens what an app can finish after leaving the foreground; it is not blanket permission for unattended work to start whenever it likes.
Android distinguishes ordinary asynchronous work, persistent jobs and foreground services. WorkManager is intended for persistent work, while foreground services have visibility and start restrictions. Some service types also have version- and target-dependent time budgets. An async function by itself establishes none of these execution rights.
A server removes the phone lifecycle from the coordinator’s immediate execution environment. It still needs a running host, a scheduler or queue, valid credentials, failure handling and monitoring. A managed cloud service and a computer you keep running at home can both play this role, with different operating responsibilities.
Apple: background execution and continued processing · Android: background work · Android: service timeouts
A durable task is not the same as a resumable action
Consider a tool that creates a calendar event. The event is created, but the response is lost before the agent records success. Retrying the tool might create a second event. Moving the coordinator to a server does not remove this ambiguity.
Reliable recovery needs durable task state and a strategy for effects outside that state. A stable action identifier can let a supporting API recognize a retry; otherwise the agent may need to check what actually happened before acting again. AWS’s discussion of idempotent APIs explains why a caller-provided request identifier matters.
A queue can remember that a task needs work while its handler still restarts from the beginning. Resuming every model call or tool step requires additional state and tool-specific guarantees. “Recovers after interruption” should never quietly become “every action happens exactly once.”
The difference reaches the screen
For a short reflection, a phone coordinator can read the selected local note and ask a follow-up while you are still thinking about it. It cannot automatically inspect every other app: access remains subject to permissions and available interfaces. A server agent needs a deliberate channel to receive the same context, especially if the latest note has never been synced. A server can be equipped with a browser and command-line tools; those do not grant it access to phone-only content.
For the later travel investigation, remote execution can support “come back when it is done.” The result can wait for the phone to reconnect. The app still owes you a meaningful status: researching, waiting for connectivity, paused, failed or needing your answer. A spinner that stays on the screen after execution has stopped hides the most important fact about the job.
Local coordination does not guarantee a faster answer. If both designs call the same remote model, inference and network delays may dominate. On-device inference shifts computation to the phone, with device-specific memory, energy and thermal constraints. Likewise, saving a note offline and completing a workflow that needs a remote model are different capabilities.
Privacy and cost follow the route the work takes
A phone coordinator can select a few relevant passages without first replicating the whole archive to an execution server. Those passages can still leave the device in a model request. A server can also use narrow permissions and limited retention. To evaluate either design, trace the records, prompts, tool outputs, logs and backups—not just the machine running the loop.
Model choice matters separately. Choosing an endpoint gives you a choice of processing boundary; it does not make every provider private or offline. Hosting another worker can add compute and maintenance costs, while phone execution uses device resources and can still incur API charges. Without a defined workload and measurements, there is no useful universal “cheaper” or “more efficient” winner.
Choosing the model connection · Where Memex records are stored
What the Memex source actually supports
At the source revision inspected for this article, Memex has a persistent local task queue with task states, execution ownership, stale-task recovery and configurable retries. A background coordinator connects task activity to the app lifecycle. Android has a WorkManager queue-drain path; the iOS bridge includes processing tasks and an iOS 26 continued-processing path with progress and expiration handling.
The background worker also records paused work and can arrange another attempt. These are mechanisms for managing interruptions. They do not prove uninterrupted runtime, exact recovery at every internal step or prevention of all duplicate external effects. This source inspection is not an installed-device test, and support in a particular released build must be checked separately.
Open source makes those boundaries inspectable and gives developers room to modify them. Memex’s useful starting point is a personal archive and configurable workflows close to it. It does not require every occasional personal task to become a permanently running server job.
Memex task queue and recovery · Android queue scheduling · iOS background bridge · Background worker and pause handling
A possible hybrid: hand off a job, keep its history
A future design could keep original records on the phone and hand a bounded investigation to another host. For the travel request, the phone might send the selected dates, budget and relevant preferences. The server would return a shortlist with sources rather than needing a permanent copy of every journal entry.
The handoff would need a job identity, an input snapshot, allowed tools, a spending limit and a clear owner of the next step. Suppose you change “a quiet trip” to “a trip with friends” while it is running. The returning shortlist should identify the earlier preferences it used and ask whether to revise, rather than silently replacing your newer plan.
Cancellation would have to distinguish stopping further research from undoing an action already taken. A returned draft can be reviewed; a booking needs a separate approval boundary and a way to recognize repeated requests. These are proposed design rules, not shipped Memex phone-to-server delegation features. Better on-device models would not abolish OS lifecycle rules, and more server compute would not supply missing personal context.
The record is ready when a task appears
A voice note should be worth saving even when no long task follows it. For many everyday moments, the useful interaction is capture, a brief conversation and a record you can revisit. Keeping that path close to the phone fits the way the material arrives.
Months after the walk, the useful starting point for travel research is already there: what you hoped for, in your own words, with its date and later revisions. A longer-lived worker can take on the occasional investigation. The everyday record gives it something worth working from.
Personal AI starts with everyday life · Try a bounded weekly-review workflow