Actualité
Google refreshes Android Bench with Harbor framework and eight new AI models
Google has updated Android Bench with a new evaluation framework, eight additional AI models and an open invitation for developers to contribute tasks and results.
Google has refreshed Android Bench, its leaderboard for measuring how well AI models handle real Android development work. In an official Android Developers announcement published on July 8, 2026, the company said it had adopted the Harbor framework, added eight models and opened the benchmark to community contributions.
Android Bench is designed to answer a narrower question than a general AI leaderboard: can a model help solve the kinds of problems Android developers face in real projects? Google’s task set covers areas such as Jetpack Compose migrations, wearable networking and platform API changes. The benchmark is intended to give developers and model makers a more transparent way to compare coding assistance for Android applications.
A new framework for a moving target
Google said the July release standardizes Android Bench on Harbor, a framework intended to make evaluations easier to run, reproduce and compare. The update also includes a newer version of the benchmarking agent and changes to the way models interact with tools.
That methodological change matters because the benchmark does not simply ask a model to produce an answer in a chat window. Models must work through development tasks, interact with a software environment and attempt changes that can be evaluated. Google reran the models after moving to Harbor to establish a new baseline. As a result, the company warns that scores may shift slightly, while historical results remain available in the archive.
The published Android Bench methodology explains that the benchmark evaluates real-world Android issues and pull requests drawn from open-source projects. The current composition contains 100 tasks selected from a much larger pool, with coverage across application architecture, UI, persistence, asynchronous programming, build configuration, camera, media, foldable adaptations and runtime permissions.
Eight new models enter the leaderboard
The July update adds Claude Fable 5, Claude Sonnet 5, Claude Opus 4.8, GLM 5.2, Kimi K2.7 Code, MiniMax M3, Qwen 3.7 Plus and Qwen 3.7 Max. Google’s published results place Claude Fable 5 at the top with a score of 84.5, followed by GPT 5.5 at 80.2 and Claude Sonnet 5 at 76.2.
Among the open-weight models listed by Google, GLM 5.2 leads with 72.2, followed by Kimi K2.7 Code at 70.4. Those figures represent performance within Android Bench’s own task set and evaluation process. They should not be read as a universal ranking of coding ability, general intelligence or the quality of apps produced without human review.
Google has also expanded the information shown alongside model performance. The leaderboard includes dimensions such as cost, token usage and latency, allowing developers to consider efficiency as well as the percentage of tasks completed. That makes the results more practical for teams choosing an assistant for daily development, where a slightly lower score may still be attractive if it comes with lower cost or faster feedback.
Why Android-specific testing matters
Generic software benchmarks can miss the details that make mobile development difficult. Android projects must account for changing screen sizes, lifecycle behavior, permission rules, device fragmentation, background limits and a large ecosystem of libraries. A model that writes plausible Kotlin code may still fail when it has to preserve state across configuration changes, migrate a Compose interface or adapt an experience to a foldable display.
Android Bench tries to make those constraints visible. Google says its tasks are drawn from merged pull requests that include validation such as unit or instrumentation tests. The methodology also describes safeguards intended to reduce contamination risk, including canary strings, manual reviews of agent trajectories and reproducible evaluation configurations.
That approach gives the leaderboard more context than a score alone. It also exposes its limits. The benchmark represents a curated sample of Android engineering problems, not every app architecture, device configuration or production incident. Results can change as the task set, models and evaluation tools evolve.
Developers can now contribute
Google is inviting the Android developer community to help shape future releases. Developers can submit Android development tasks that reflect problems they consider important, or run evaluations against the existing dataset and share their results. The company says submitted tasks will be reviewed before they are considered for inclusion.
The Android Bench portal now lists the updated leaderboard, release notes and links to the methodology. Google also points developers toward the benchmark’s dataset and evaluation environment so that researchers and engineering teams can inspect the setup rather than relying only on published rankings.
For Android teams, the immediate value is less about finding a single “best” model than about establishing a repeatable test for their own workflows. A team could compare assistants on tasks involving its preferred libraries, review how reliably they handle Android API changes and track cost or latency before adopting an agent in production. Human review remains essential, especially for changes affecting permissions, privacy, authentication or user data.
Android Bench’s latest update therefore signals a broader shift in mobile software development: AI coding tools are becoming easier to compare, but the useful comparisons are the ones grounded in realistic tasks, transparent procedures and clear limits. Google’s move to Harbor and its decision to accept community contributions should give Android developers more ways to inspect those claims and help make future evaluations more representative.