AutoTailor: Automatic and Efficient Adaptive Model Deployment for Diverse Edge Devices

arXiv:2511.22355 · cs.LG · Submitted 2025-11-27 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "AutoTailor: Automatic and Efficient Adaptive Model Deployment for Diverse Edge Devices".

Jane: The gist On-device machine learning (ML) has become a fundamental component of emerging mobile applications, and AutoTailor presents the first framework to enable automated,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we’re looking at the title of AutoTailor again. It’s about making adaptive model deployment automatic and efficient specifically for diverse edge devices. Jane And that really points to the problem they're trying to solve, which is that traditional static DNN deployment just doesn't work anymore because every edge device is different in hardware and what performance it needs thirty-two thirty-three thirty-nine <ref:2511.22355#pg3>. Tom Right. So AutoTailor steps in by trying to fix this whole process from start to finish.

Lu: What’s interesting about their title is that they are focusing on automation for the SuperNet approach, which is a way to make many model variants from one pre-trained model <ref:2511.22355#pg1>. They are tackling the manual work that usually comes with those methods.

Meng: Manual work means engineers have to manually transform a static DNN into something dynamic that supports multiple architectural modifications, which bloats the codebase and demands deep expertise in DNN architecture design <ref:2511.22355#pg2>. That sounds like a huge headache when you're shipping models to production Meng.

Jane: So they are claiming this framework is the first one to enable automated, end-to-end SuperNet-based adaptive model deployment for edge devices <ref:2511.22355#pg1>. That means they claim it handles everything automatically from taking a static model to deploying all those different variants onto the edge hardware.

Tom: That's the big claim there. So instead of users having to do all that manual transformation, AutoTailor aims to handle it by using a computation graph-guided compilation approach to automatically transform user-provided ML models into SuperNets <ref:2511.22355#pg1>.

Jane: And that compilation method is key because it uses the structure of the original model to figure out what parts can be modified without anyone having to manually code those changes <ref:2511.22355#pg1>.

Lu: I see their focus on graph structural consistency, which means they are using the computation graph of the original DNN to automatically identify transformable modules and replace them directly <ref:2511.22355#pg2>. That’s a clever way to decouple modifications from the actual structure of the model.

Meng: But I wonder if that decoupling is perfect when you look at real-world deployment constraints, like memory limits or specific hardware accelerators on an MCU thirty-nine <ref:2511.22355#pg2>.

Tom: Let's see how they handle those constraints in the next part. We’re going to talk about what this framework actually does once we look at the summary of AutoTailor.

The paper's summary: Jane: So, this paper explains that adaptive model deployment is a strategy where you generate multiple model variants with different architectures, accuracy levels, and efficiency goals to match all those diverse edge devices thirty-two <ref:2511.22355#pg3>. Tom They start by acknowledging that while SuperNet approaches offer a way to generate many variants efficiently using weight sharing, applying SuperNets in practice usually means users have to manually transform static DNNs into dynamic ones which takes a lot of time and effort <ref:2511.22355#pg2>.

Lu: The paper sets up the problem by pointing out that edge devices have two main issues: first, they exhibit wide hardware diversity, from high-performance NVIDIA Orin boards to commodity smartphones and resource-constrained microcontrollers thirty-nine, and second, performance requirements are varied; some tasks need high-frequency inference while others need energy efficiency twenty twenty-one <ref:2511.22355#pg3,from high-performance NVIDIA Orin boards>.

Meng: So the paper is saying that adaptive model deployment is a promising strategy because it tries to align the model execution with those heterogeneous resources and application needs eight fourteen fifty-two <ref:2511.22355#pg3>. But they highlight that current methods like LegoDNN or AdaptiveNet still have overhead because you have to fine-tune each block individually three <ref:2511.22355#pg1>.

Tom: And that's where AutoTailor steps in. They present their framework as an end-to-end solution. It claims it automatically compiles any user-provided DNN into TailorIR, which is a novel abstraction representing SuperNets through a computation graph-guided compilation process <ref:2511.22355#pg1>.

Jane: That compilation step is what makes it end-to-end; it means the framework takes the static model and automatically transforms it into this dynamic structure, removing the need for manual coding of those modifications <ref:2511.22355#pg1>.

Lu: The methodology hinges on two main techniques to make this happen: first, a computation graph-guided compilation method that automatically replaces computation graph components without manual coding <ref:2511.22355#pg3>, and second, a learning-free predictor construction method for quickly building latency lookup tables or LUTs <ref:2511.22355#pg3>.

Meng: Learning-free predictors sound good on paper, but I’m worried about how fast they actually build those LUTs when you have to deal with millions of potential variants across different devices Meng.

Tom: Well, they claim the learning-free predictor construction method rapidly builds these latency lookup tables without needing costly sampling or training <ref:2511.22355#pg3>. That’s a big win for deployment speed.

Jane: So in short, the summary is that AutoTailor automates the entire process of making a static model adaptive for edge devices by using graph compilation and these learning-free predictors to handle architectural changes and speed prediction <ref:2511.22355#pg1>.

The paper's improvements: Tom: Let’s talk about the actual improvements they claim in this paper. They highlight that AutoTailor is the first framework to provide fully automated, end-to-end adaptive model deployment for edge devices <ref:2511.22355#pg1>. Jane That means they are claiming it solves the entire problem of getting a static model to work across different hardware and needs without requiring manual effort from the user.

Lu: They also propose two specific techniques to cut down on development and deployment overheads when applying SuperNets <ref:2511.22355#pg2>. The first is the computation graph-guided compilation method, which automatically replaces computation graph components without manual coding <ref:2511.22355#pg3>.

Meng: So they claim this reduces development effort by eleven to twenty-seven times and profiling costs by at least eleven times <ref:2511.22355#pg4>. If that’s true, that’s a massive reduction in engineering time and cost when you have many models to test.

Tom: And they also have these learning-free predictors for latency prediction and accuracy prediction <ref:2511.22355#pg4>. They build operator-level LUTs for latency by profiling only unique operators, and use modification sensitivity to approximate SubNet accuracy <ref:2511.22355#pg4>.

Jane: That means they are trying to avoid the expensive process of profiling every single candidate variant just to see how slow or accurate it is before you deploy anything <ref:2511.22355#pg4>. They’re trying to get that guidance without the heavy computational cost.

Lu: They also have an automatic model exploration step where they use a simple configuration file to automatically sample SubNets using a Tuner component for fine-tuning and a genetic algorithm for architecture optimization <ref:2511.22355#pg9>. That automates finding the best combination of modifications and architecture for a given device <ref:2511.22355#pg9>.

Meng: I’m still skeptical about the accuracy prediction part, though. Approximating accuracy by summing sensitivities from modifications seems like a shortcut that might miss complex interactions between different changes Meng.

Tom: It’s an approximation, but they show it works well in testing. And across their evaluations, they claim up to fifteen point six zero percent absolute accuracy improvement and a sixty point zero three percent latency reduction compared to state-of-the-art approaches <ref:2511.22355#pg2>.

Jane: So, the overall message here is that AutoTailor aims to cut down on the engineering effort and profiling costs significantly while still achieving strong performance gains for model adaptation across different devices <ref:2511.22355#pg4>.

Conclusion: Tom: Okay, so we’ve covered the concept, the automated compilation, and those learning-free predictors. So to wrap up on AutoTailor: it’s an end-to-end framework that takes a pre-trained DNN and a short configuration file to automatically construct a high-performance SuperNet for a specific edge device <ref:2511.22355#pg1>.

Jane: It really seems like the implication is that we can move away from the current tedious process where users have to manually handle all the architectural transformations for SuperNets <ref:2511.22355#pg2>. Lu I think what this means for researchers is that they don't need to spend so much time on writing custom Mutator abstractions or figuring out how to wrap operators correctly <ref:2511.22355#pg4>.

Meng: From a production standpoint, if we can reduce the profiling cost by ten times and development effort by twenty times, that makes deploying models to a huge variety of devices much more feasible Meng <ref:2511.22355#pg2>.

Lalam: I am confident in this direction. The paper shows that the graph structural consistency is robust enough for most operators, which means they can enumerate those unique operators easily <ref:2511.22355#pg2>.

Tom: So, we’re looking at a framework that requires only a short configuration file and minutes of profiling to specialize the SuperNet for new devices <ref:2511.22355#pg1>. It really simplifies the deployment pipeline.

Jane: It makes model specialization much faster, which is crucial for real-time responsiveness on those constrained edge platforms thirty-nine <ref:2511.22355#pg2>. So that’s what AutoTailor is doing to help AI systems deploy tailored models without requiring massive manual architectural redesign <ref:2511.22355#pg1>.

Southeast University · Hong Kong University of Science and Technology · Tsinghua University · University of Macau

cs.LG

Submitted: 2025-11-27

Updated: 2026-10-19

Comments: ATC 2026

Code: https://github.com/liu-mengyang/AutoTailor

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 87/100

The gist: The gist On-device machine learning (ML) has become a fundamental component of emerging mobile applications, and AutoTailor presents the first framework to enable automated, end-to-end SuperNet-based

Key concepts

SuperNets: SuperNets are a technique that uses weight sharing to efficiently generate many different model variants from a single base architecture. Instead of retraining every version, they create diverse models by applying small modifications to the original structure, making it easier to adapt models for different hardware needs.
TailorIR: TailorIR is a novel abstraction that represents SuperNets as computation graphs. This allows the framework to decouple model modifications from the underlying DNN structure, enabling automatic replacement of components during compilation without needing manual coding or deep architectural knowledge.
Computation Graph-guided Compilation: This method automatically translates a standard DNN into TailorIR by analyzing its computation graph. It identifies parts of the graph that can be modified based on predefined templates, replacing them automatically. This process removes the need for manual intervention in transforming static models into dynamic, adaptive ones.
Learning-free Predictors: These are methods used to quickly estimate model performance without requiring extensive training data or complex learning algorithms. For latency, they profile only unique operators to build lookup tables (LUTs). For accuracy, they use modification sensitivity to approximate the accuracy of new variants based on how much each change affects the total result.

Terminology

Accuracy — 2× in this paper · explained in 1 episode
Edge Devices — 2× in this paper · explained in 2 episodes
Machine Learning (ML) — 1× in this paper · explained in 2 episodes

Summary

The gist On-device machine learning (ML) has become a fundamental component of emerging mobile applications, and AutoTailor presents the first framework to enable automated, end-to-end SuperNet-based adaptive model deployment for edge devices.

Background and Motivation

Adaptive model deployment is a strategy that generates multiple model variants with different architectures, accuracy, and efficiency to meet the diverse requirements of heterogeneous edge devices (Page 3). Traditional static DNN deployment fails because edge devices exhibit wide hardware diversity and varying performance needs (Page 3). While methods like LegoDNN and AdaptiveNet generate variants through block scaling, they suffer from overhead due to the need for fine-tuning each block, which limits scalability (Page 3). SuperNets offer a promising solution by using weight sharing to generate a large number of model variants efficiently (Page 2). However, applying SuperNets in practice incurs substantial development and deployment costs because users must manually transform static DNNs into dynamic ones, and profiling candidate variants for hardware-aware optimization is time-consuming (Page 2).

AutoTailor Framework Overview

AutoTailor is an end-to-end adaptive model deployment framework that automatically compiles any user-provided DNN into TailorIR, a novel abstraction representing SuperNets through a computation graph-guided compilation process (Page 2). The framework achieves automatic and efficient deployment through two key techniques: (1) a computation graph-guided compilation method that automatically replaces computation graph components without manual coding, and (2) a learning-free predictor construction method that rapidly builds latency lookup tables (LUTs) without costly sampling or training (Page 3).

Key Contributions

The key contributions of AutoTailor are:

To the best of our knowledge, AutoTailor is the first framework to provide fully automated, end-to-end adaptive model deployment for edge devices (Page 2).
We propose a computation graph-guided compilation method and learning-free predictors building methods to reduce development and deployment overheads in applying SuperNets (Page 2).
We extensively evaluate AutoTailor across diverse DNNs and edge devices, demonstrating that it reduces engineering effort by 11–27× and profiling cost by at least 11×, while achieving up to 15.60% absolute accuracy improvement and 60.03% latency reduction compared to state-of-the-art approaches (Page 2).

Technical Design

AutoTailor relies on two fundamental characteristics of SuperNets:

Graph structural consistency: SuperNets generate variants by applying architectural modifications to a base architecture, which preserves the computation graph of the original DNN, enabling transformable modules to be automatically identified and replaced directly from the computation graph (Page 2).
Cross-SubNet architectural similarity: analysis shows a high repetition ratio (96–98%) for most operators, meaning the number of unique operators is significantly small and feasible to enumerate (Page 2).

The system architecture involves several components:

>(4.1) Computation Graph-guided Compilation:

This step automatically compiles the DNN into TailorIR, which represents SuperNets as computation graphs of TailorIR modules, decoupling modifications from structure (Page 4). This process includes an operator parsing step, a block parsing step matching subgraphs with templated blocks, and a stage division to enable higher-level modifications (Page 4).

>(4.2) Learning-free Predictor Building:

AutoTailor proposes learning-free predictors that avoid the overhead of learning-based methods. For latency prediction, it profiles only unique operators to build operator-level LUTs, and for accuracy prediction, it uses modification sensitivity to approximate SubNet accuracy based on the sum of sensitivities (Page 4).

>(4.3) Automatic Model Exploration:

This step automates sampling SubNets from the search space using a simple configuration file specifying modifications and candidate choices (Page 9). It includes SuperNet Fine-tuning, which uses a Tuner component to sample SubNets by applying modifications and fine-tuning them accordingly, and Architecture Optimization, which uses genetic algorithms to search for the best architecture (Page 9).

Evaluation Results

Experimental results demonstrate the effectiveness of AutoTailor across diverse models and devices. For ResNet50, AutoTailor achieved an average accuracy gain of 2.27% and a 10.37% latency reduction over AdaptiveNet (Page 10). On MobileNetV2, the improvements are larger, with 4.12% higher accuracy and 21.48% lower latency (Page 10). For Transformer-style models like ConvNeXt and Vision Transformer, AutoTailor improved accuracy by 5.12% and reduced latency by 29.62% compared to NestDNN (Page 10). The framework consistently delivers higher accuracy under tight latency budgets than existing methods (Page 10).

Conclusion

AutoTailor is an end-to-end adaptive model deployment framework that requires only a short configuration file to automatically construct a high-performance SuperNet from a pre-trained DNN, and it requires only minutes of profiling to specialize the SuperNet for new devices (Page 12). Overall, AutoTailor significantly advances practical model deployment for diverse real-world edge scenarios.

Improvements for AI systems

The AutoTailor framework enables fully automated, end-to-end adaptive model deployment for edge devices, meaning it can automatically compile any user-provided DNN into TailorIR and then fine-tune it to ensure accuracy of all model variants. This allows AI systems to deploy highly specialized models tailored precisely to the heterogeneous capabilities of diverse edge platforms without requiring manual, time-consuming development.
The system can reduce engineering effort by 11–27× and profiling costs by at least 11×, while achieving up to 15.60% absolute accuracy improvement and 60.03% latency reduction compared to state-of-the-art approaches. This capability allows for faster iteration in model adaptation, enabling the rapid specialization of deep learning models for real-time responsiveness or energy efficiency on resource-constrained devices.
The system can automate SuperNet construction by leveraging graph structural consistency to automatically identify transformable modules and their candidate choices without user intervention, eliminating the need for manual annotation. This allows users to define model modifications through a simple configuration file, decoupling them from the DNN program and reducing reliance on deep DNN architecture design expertise.
The system can build learning-free predictors for latency by profiling only unique operators, which involves pruning redundant operator enumerations through analysis of modification dependencies, thus avoiding costly sampling and training. This enables rapid construction of device-specific lookup tables (LUTs) that provide accurate latency guidance without requiring hours of data collection.
The system can build an accuracy predictor using modification sensitivity to approximate SubNet accuracy as the sum of the sensitivities of all modifications it contains, which allows for efficient modification-level prediction without full model profiling. This enables selection of the most accurate SubNet under given constraints by maintaining the relative order of candidates.
The system can perform Model exploration through a Tuner component that samples SubNets and fine-tunes them, and an Optimizer using genetic algorithms to search for the best architecture using learning-free predictors. This allows for automated searching of optimal architectures across different models and devices, delivering state-of-the-art performance under various latency constraints.
The system can extend its prediction capabilities to include energy and memory consumption by aggregating energy like latency and decomposing memory into parameters and activations, allowing for the estimation of total inference cost based on parameter memory and peak activation memory.

Abstract

On-device machine learning (ML) has become a fundamental component of emerging mobile applications. Adaptive model deployment delivers efficient inference for heterogeneous device capabilities and performance requirements through customizing neural architectures. SuperNet-based approaches offer a promising solution by generating numerous model variants from a pre-trained ML model. However, applying SuperNet in existing frameworks requires tedious model-aware development and time-consuming hardware-aware profiling, which limits their practical adoption. We present AutoTailor, the first framework to enable automated, end-to-end SuperNet-based adaptive model deployment for edge devices. Unlike manual SuperNet construction, AutoTailor employs a computation graph-guided compilation approach to automatically transform user-provided ML models into SuperNets. To support efficient specialization, AutoTailor incorporates learning-free latency and accuracy predictors, enabling low-cost yet accurate performance prediction. Our extensive evaluations demonstrate that AutoTailor reduces the lines of code for SuperNet construction by 10.8--34.5 times, reduces predictor profiling time by 2.2--11.5 times, and achieves up to 15.60% absolute accuracy improvement and 60.03% latency reduction compared to state-of-the-art approaches across diverse models and devices. AutoTailor's source code is publicly available at.