An investigation of AI integration in sound designer workflows and experiences

arXiv:2605.27174 · cs.SD, cs.AI, cs.CY · Submitted 2026-05-26 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "An investigation of AI integration in sound designer workflows and experiences".

Jane: The gist: Current AI tools perform adequately in fast-consumption media contexts but lack the narrative sophistication required for high-end sound design (films, immersive experiences etc).

Tom: First, who's behind it and why it matters.

Title and authors: Jane: Moving into the details, we have the title and authors of this paper: "An investigation of AI integration in sound designer workflows and experiences," by Nelly Garcia and Joshua Reiss.

Tom: Yeah, so they’re looking at how professionals are actually using AI in their jobs. The implication is that we need to look beyond just the technical capabilities of the models themselves.

Lu: They did a mixed-methods study, which means they combined an online survey with some semi-structured interviews to get a really broad view of how people feel about AI in this area.

Meng: And the survey reached quite a diverse group, seventy-six participants across twenty-one countries, and it showed that the sample was heavily weighted towards established professionals <ref:2605.27174#pg2>.

Jane: That’s interesting because it tells us that we aren't just talking to beginners here; we’re talking to people who already have significant experience in sound design.

Tom: The paper suggests that the way AI tools function depends entirely on the context—like whether you're doing quick content versus something high-fidelity and long-form.

Lalam: So it sets up this idea that the demands placed on AI tool design are different depending on whether you’re making something for fast consumption or a massive immersive experience.

The paper's summary: Tom: Now, let's look at what they actually found in the summary of "An investigation of AI integration in sound designer workflows and experiences." They’re saying that existing research often focuses too much on technical benchmarks and computational speed.

Jane: They argue that this focus on machine capability alone risks obscuring the actual creative processes designers use to make sound, which is what they call craft-based processes.

Lu: They point out that the demands of spatial audio and real-time interaction—which are really important for immersive work—place very distinct requirements on how AI tools are designed, and current generative models aren't quite meeting those needs yet.

Meng: This suggests that just having a powerful AI isn't enough; the tool has to be built with the specific needs of sound designers in mind.

Lalam: So they’re emphasizing that the requirements for AI tools are shaped by the actual workflow, not just by how fast it can process data.

Tom: They specifically address this gap by examining how participants perceive and use AI across different professional settings and under what conditions these technologies work as collaborative instruments.

Jane: The study looked at current tool usage, perceptions of creativity in AI, where the lines are between human and machine work, and even the ethical positions people take with these tools.

The paper's improvements: Tom: The paper suggests several ways to improve this situation. They emphasize that practitioners prefer tools that offer granular control over fully automated generation.

Lu: They highlight a clear preference for systems that allow for parametric and hybrid workflows, rather than just completely hands-off, fully automated generation.

Meng: This is significant because it shows that designers don't want to lose control; they want the AI to be an extension of their technical skill, not something that distances them from it.

Jane: The paper frames AI’s potential use as manipulating parameters and layering complex textures at speed, augmenting the iterative process while keeping the final creative decision with the designer.

Lalam: So, the suggested improvement is to build tools that let you manipulate specific settings and layer things quickly, but still leave you in charge of what happens next.

Tom: And they also point out a potential path for accessibility because cloud-based generative tooling can lower the barrier to entry for people who don't have access to expensive studio infrastructure or specialized training.

Conclusion: Jane: So, wrapping up the findings of "An investigation of AI integration in sound designer workflows and experiences," the paper suggests that AI’s value isn't measured by what it generates on its own, but how much it extends the practitioner’s capacity for deliberate, narratively intentional creative decisions.

Tom: It seems like they are arguing that the real measure of success for this technology in sound design is how well it supports a designer's specific creative intent.

Lu: I think the paper makes a strong case that we need to move beyond just looking at what the machine can produce and focus on how it lets us make deliberate choices.

Meng: And from an engineering standpoint, it means designing for those granular controls and parameter manipulation is actually where the most useful AI intervention will be found in these workflows.

Lalam: I agree; the paper points toward a future where AI helps speed up the utility tasks while leaving the high-level aesthetic choices firmly in human hands.

Jane: It’s a call for tools that respect the craft and honor the designer's role as a mediator in that creative process.

Tom: Alright, so we’ve covered how this paper looks at current AI integration in sound design workflows and experiences. That gives us a good idea of where the research is heading next.

Queen Mary University of London

cs.SD, cs.AI, cs.CY

Submitted: 2026-05-26

Updated: 2026-05-26

Journal ref: Proc. AVARIG 2026: Audio for Virtual and Augmented Reality and Immersive Games, Audio Engineering Society, Paper 497, June 2026

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 81/100

The gist: The gist: Current AI tools perform adequately in fast-consumption media contexts but lack the narrative sophistication required for high-end sound design (films, immersive experiences etc).

Key concepts

Production Paradigms: These are the two distinct environments where sound design happens: ephemeral/high-consumption media (where speed and cost matter most) and high-fidelity/long-form production (where human designers keep total control). The AI tools used must adapt their function based on which paradigm is being used.
Technical Bottlenecks: These are specific, time-consuming tasks in sound design, such as cleaning up old audio (restoration), separating sounds from recordings (source separation), and organizing large sound libraries. Practitioners seek AI to solve these bottlenecks so they can focus on creative decisions.
Granular Control Preference: Sound designers prefer AI tools that allow them fine-tuned manipulation, such as using parametric synthesis engines or hybrid workflows, instead of fully automated generation. This preference shows a desire for the AI to extend their technical craft rather than replacing it completely.

Summary

The gist: Current AI tools perform adequately in fast-consumption media contexts but lack the narrative sophistication required for high-end sound design (films, immersive experiences etc).

Context

AI adoption in sound design is contingent on the specific network of production and consumption within which the practitioner operates, functioning as a negotiation between consumer expectations and prevailing industry quality standards The data identifies two distinct production paradigms that condition this negotiation: ephemeral/high-consumption media where efficiency and cost-reduction are primary drivers, and high-fidelity/long-form production where the human designer retains central mediating authority

Workflow

The study identified specific technical bottlenecks that disproportionately consume time and creative energy across both production paradigms, specifically identifying a demand for AI intervention in labour-intensive utility tasks such as restoration, source separation, and library management Practitioners showed a clear preference for tools offering granular control, particularly parametric and hybrid workflows over fully automated generation This preference reflects a broader industry demand where sound designers prioritise systems that act as an extension of their technical craft rather than distancing them from it

Potential Use of AI

AI functions as a means to manipulate parameters, layer complex textures, and traverse libraries at speed, augmenting the iterative process while the practitioner retains final curatorial authority over the output Beyond established practitioners, accessibility is identified as a dimension of this potential because cloud-based generative tooling materially lowers the barrier to entry for practitioners from contexts where costly studio infrastructure and institutional training have historically been unavailable

Risks

The augmentative potential outlined above is conditioned by a significant countervailing risk, as the same tools that expand creative possibility simultaneously threaten the foundational skill development upon which that creative capacity depends The primary concern identified across interview transcripts is professional unskilling, where sustained reliance on generative tools progressively atrophies the technical abilities that practitioners develop through practice This risk applies even where the accessibility dimension holds because lowering the barrier to entry through cloud-based tooling does not resolve the unskilling problem if the tools that provide access simultaneously bypass the learning process

Right Use

The ethical legitimacy of AI integration is governed by transparency and provenance, as practitioners' reluctance is positioned by a pervasive absence of legal certainty regarding training data A primary concern identified across interview transcripts is the black box problem, which involves a widespread lack of clarity regarding the datasets used to train commercial generative models Therefore, Right Use is defined not by what the technology is capable of producing, but by the ethical legitimacy of its development and deployment

The central finding is that practitioner resistance to AI is not directed at the technology per se, but at the lack of creative agency within an increasingly automated production pipeline Future work should examine how the recommendations proposed here translate into concrete tool design, and longitudinal studies tracking AI adoption across experience levels would strengthen the empirical foundation of this emerging field. The value of AI in sound design is ultimately measured not by the output it generates autonomously, but by the degree to which it extends the practitioner’s capacity for deliberate, narratively intentional creative decisions.

References

[1] Sonnenschein, D., Sound Design: The Expressive Power of Music, Voice, and Sound Effects in Cinema, Michael Wiese Productions, Studio City, CA, 2001

[2] Whittington, W., Sound Design and Science Fiction, University of Texas Press

[3] Pavan, G., Budney, G., Klinck, H., Glotin, H., Clink, D., and Thomas, J., History of Sound Recording and Analysis Equipment

[4] Millard, A., America on record: a history of recorded sound, Cambridge University Press

[5] Kellogg, E. W., “History of Sound Motion Pictures,” Technological History of Motion Pictures and Television. An Anthology from the Pages of the Journal of the Society of Motion Picture and Television Engineers, pp. 174–220, 1967

[6] Morton, D., Sound recording: The life story of a technology, JHU Press

[7] Pingili, R., “Generative AI unlocking adaptive workflow design,” Journal of Next-Generation Research 5.0, 2025

[8] Menexopoulos, D., Pestana, P., and Reiss, J., “The State of the Art in Procedural Audio,” Journal of the Audio Engineering Society, 71, pp. 825–847, 2023

[9] Oh, S., Kang, M., Moon, H., Choi, K., and Chon, B. S., “A Demand-Driven Perspective on Generative Audio AI,” 2023

[10] Cámara, M., Marcos, F., Bargum, A. R., Erkut, C., Reiss, J., and Blanco, J. L., “Neural Audio Synthesis for Sound Effects: A Scope Review,” IEEE Transactions on Audio, Speech and Language Processing, 34

[11] Katabi, M., “Beyond Human Authorship: Rethinking Copyright Ownership in the Age of Autonomous Artificial Intelligence,” East African Journal of Law and Ethics, 8

[12] Milczarek, E., The evolution of copyright in the age of artificial intelligence: challenges and perspectives, pp. 158–170, 2024

[13] Hutson, J., “The Evolving Role of Copyright Law in the Age of AI-Generated Works,” Journal of Digital Technologies and Law, 2(4), pp. 886–914, 2024

[14] Frey, C. B. and Osborne, M. A., “The future of employment: How susceptible are jobs to computerisation?” Technological Forecasting and Social Change, 114

[15] Luckin, R., Holmes, W., Griffiths, M., and Forcier, L. B., “Intelligence Unleashed: An Argument for AI in Education,” Technical report, Pearson

[16] Susini, P., Houix, O., and Misdariis, N., “Sound design: An applied, experimental framework to study the perception of everyday sounds,” The New Soundtrack, 4

[17] Selfridge, R., Delle Monache, S., and Pauletto, S., “Sound design ideation: a structured and systematic analysis,” in Proceedings of the 19th International Audio Mostly Conference: Explorations in Sonic Cultures, p. 492–501, Association for Computing Machinery

[18] Pinheiro, S., “Revisiting the Canon of Sound Theory (Michel Chion, Audio-Vision: Sound on Screen),” Iluminace, 33

[19] Chion, M., Gorbman, C., and Murch, W., Audio-vision: Sound on Screen, Columbia University Press

[20] Agostinelli, A., Denk, T. I., Borsos, Z., Engel, J., Verzetti, M., Caillon, A., Huang, Q., Jansen, A., Roberts, A., Tagliasacchi, M., Sharifi, M., Zeghidour, N., and Frank C. “MusicLM: Generating Music From Text,” 2023

[21] Liu, H., Chen, Z., Yuan, Y., Mei, X., Liu, X., Mandic, D., Wang, W., and Plumbley M. D.

Improvements for AI systems

Bold header: Assistive Workflow Optimization

This system can be designed to specifically target labour-intensive utility tasks, specifically restoration, source separation, and library management, allowing practitioners to focus on higher-order aesthetic judgment rather than procedural bottlenecks.

Bold header: Context-Aware Generative Control

The AI must differentiate its operational mode based on the production context; for ephemeral media, it should prioritize efficiency and cost-reduction via turn-key tools, while for high-fidelity work, it must adhere to the need for narrative nuance and emotional specificity.

Bold header: Granular Control Preference Implementation

The system's interface should prioritize outputs offering granular control, such as the preference shown by practitioners for parametric synthesis engines with real-time control over fully automated generation.

Abstract

Artificial intelligence is increasingly being integrated into professional audio production workflows, yet a gap persists between the tools developers produce and the requirements of practising sound designers. This paper investigates this gap through a mixed-methods study comprising a survey of 76 practitioners and follow-up semi-structured interviews with 20 industry professionals. Results were analysed using descriptive statistical analysis and thematic analysis to identify patterns across both datasets. Five themes emerged from our analysis: Context, Workflow, Potential, Risks, and Right Use. Our work indicates that current AI tools perform adequately in fast-consumption media contexts but lack the narrative sophistication required for high-end sound design (films, immersive experiences etc). Practitioners demonstrate a preference for assistive, task-specific applications, particularly in audio restoration and library management, over end-to-end generative systems. This work contributes to the on-going discussion on the use of AI and AI-enhanced tools in the creative industries. We report on the current status of the field from the point of view of sound designers and creative audio practitioners, and offer a set of recommendation for sound technologist and developers based on our findings to guide the development of more informed AI tools for sound design.