Why my iPhone app feels slow even with fast core ML models

Core ML models run in 8ms. Although the benchmarks look good, the app still feels slow. Camera previews are choppy, scrolling is unresponsive, buttons feel slow, and animations drop frames. After profiling my application, I noticed something unexpected. The model wasn’t the bottleneck. The real problem is everything happening around it: thread scheduling, preprocessing, memory allocation, model initialization, UI updates, etc. Accelerated models are just one part of building responsive on-device AI experiences.

The latest iPhones can run incredibly sophisticated machine learning models directly on the device, but providing a smooth user experience requires more than just optimizing inference speed. The entire application pipeline, from capturing input data to rendering the final result, determines the responsiveness of your app.

This article describes five practical techniques that can significantly improve the performance of on-device machine learning applications.

Why even high-speed models still feel slow

iOS has a user interface main thread.

At 60 FPS, every frame has approximately 16.7ms Complete. With a ProMotion display running at 120Hz, that budget drops to approx. 8.3ms.

Meanwhile, the system must:

• Handling touch events
• Update layout
• rendering animation
• draw next frame

When machine learning inference runs synchronously on the main thread, it quickly consumes some of your limited frame budget.

// ❌ Blocks UI rendering
let prediction = try model.prediction(input: input)
label.text = prediction.label

Even though the prediction itself takes only a few milliseconds, the surrounding work (image preprocessing, memory allocation, model manipulation, UI updates) can cause the execution to exceed the available frame time.

The result is not a slow model.

It slows down due to frame drops.

1. Take inference off the main thread

Swift Concurrency makes it much easier to run machine learning workloads asynchronously.

Rather than blocking UI rendering, it performs inference in the background and returns to the main actor only when updating interface elements.

func classify(_ input: MyModelInput) async throws -> String {
    let output = try await Task.detached(priority: .userInitiated) {
        try self.model.prediction(input: input)
    }.value

    return output.label
}

Next, update the UI.

Task {
    let result = try await classify(input)

    await MainActor.run {
        self.resultLabel.text = result
    }
}

This keeps the rendering pipeline available while the model runs independently.

This separation is especially important for camera-based applications, where continuous frame processing can easily conflict with UI rendering.

2. Load the model once

A common performance mistake is recreating the Core ML model for each prediction.

// ❌ Expensive
func predict(_ input: Input) throws -> Output {
    let model = try MyModel()
    return try model.prediction(input: input)
}

Model initialization is not free. This includes loading weights, allocating resources, and preparing the model for execution.

Instead, initialize the model once at application startup or through dependency injection.

final class Classifier {

    private let model: MyModel

    init() throws {
        let configuration = MLModelConfiguration()
        configuration.computeUnits = .all

        self.model = try MyModel(configuration: configuration)
    }

    func predict(_ input: Input) throws -> Output {
        try model.prediction(input: input)
    }
}

A single model instance can typically process thousands of predictions without repeated initialization overhead.

3. Choose compute units intentionally

Core ML controls where inference is performed.

There is no universally optimal setting.

For example, applications that use Metal, ARKit, or continuous camera rendering may improve overall performance by avoiding unnecessary GPU contention.

The best approach is to benchmark different configurations using realistic workloads and target devices.

4. Optimize your entire ML pipeline

Inference is just one step in the workflow.

A typical image-based prediction pipeline looks like this:

guard !isProcessing else { return }

isProcessing = true

Task.detached(priority: .userInitiated) {

    defer {
        Task { @MainActor in
            self.isProcessing = false
        }
    }

    // Preprocess image
    // Run Core ML prediction
    // Process results
}

let start = CFAbsoluteTimeGetCurrent()

let prediction = try model.prediction(input: input)

let elapsed = (CFAbsoluteTimeGetCurrent() - start) * 1000

print("Inference: \(elapsed) ms")