DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition

The gist: DECKER introduces a domain-invariant keystroke inference framework that combines several components to learn key-discriminative yet keyboard-invariant representations, which improves robustness across devices, users, and environments.

Dataset Introduction

HEAR is a novel dataset designed to study ASCA along three axes: keyboard generalization, noise adaptation, and user bias (Page 1). HEAR contains recordings from "53 participants using 37 different laptop keyboards, collected in three realistic settings: (i) captured via an external microphone, and (ii) captured through the user’s own device microphone over no network channel noise, and (iii) captured while in a network streaming platform over VoIP (Page 1). This enables controlled benchmarks for cross-user, cross-keyboard, and cross-environment evaluations in a wider scope" (Page 1).

DECKER Framework Overview

DECKER is proposed as an ASCA training framework that unifies five key components:

(i) Keyboard Signature Normalization (KSN) to reduce device coloration in the waveform domain (Page 1).

(ii) domain-adversarial disentanglement to suppress keyboard identity (Page 1).

(iii) supervised cross-keyboard contrastive alignment to enforce key consistency across devices (Page 1).

(iv) Acoustic Style Randomization (ASR) to synthesize unseen keyboard responses (Page 1).

(v) LLM-assisted decoding with constrained beam search to reconstruct coherent text or passwords (Page 2).

Acoustic Modeling and Invariance

The framework aims to learn an embedding space satisfying three key aspects: Key discriminability, Keyboard invariance, and cross-keyboard consistency (Page 5). The process involves several steps:

1. Keyboard Signature Normalization (KSN): This module uses a CNN block with a sequence of four 1D CNNs to model the device-specific response as x i = h d i∗ s i, where s i is the latent keystroke impulse and h d i is the device-specific response (Page 5).

2. Acoustic Style Randomization (ASR): This module randomizes acoustic properties using random IIR filters (order 6-12, Q ∈ [0.4, 1.2]), smooth spectral envelope warp (±12% mel shift), and exponential decay perturbation (γ ∈ [0.85, 1.15]) respectively to simulate unseen keyboard acoustics (Page 7).

3. ECAPA-TDNN Encoder: This module is employed to capture finegrained temporal cues and short-duration transients by extracting log-mel spectrograms and combining them with multi-scale 1D temporal convolutions with channel-attentive feature recalibration (Page 5).

4. Domain Adversarial Disentanglement: This uses a Gradient Reversal Layer (GRL) to train the encoder to minimize classification loss while maximizing domain loss, ensuring that the keyboard identity d i should be unpredictable from the embeddings (Page 8).

Sequence-Level Reconstruction

The final stage involves using an LLM for sequence reconstruction:

1. Key Classification: The ECAPA module passes outputs to a softmax activation function to predict the keystrokes (Page 5).

2. LLM Refinement: This stage applies LLM-assisted decoding with constrained beam search to refine predictions, maximizing the joint objective log Ptheta (y t x t) + λLM log PLM (Y) using a beam size of 8–16 (Page 5). This step imposes linguistic and structural priors, correcting substitution, insertion, and deletion errors (Page 5).

Evaluation and Results

Empirical results show that DECKER improves keystroke identification over strong unimodal and multimodal baselines, especially in cross-keyboard and cross-user settings (Page 1). Table 4 shows that DECKER consistently narrows the generalization gap under keyboard shift, substantially outperforming unimodal, multimodal, and adversarial-only baselines (Page 10). Furthermore, LLM post-processing substantially boosts sequence recovery, providing a more realistic upper bound on keystroke leakage (Page 10). The framework is computationally feasible on commodity hardware like an Apple M1 laptop, with total inference latency for a full sentence being in the range of 2.3 ms (Page 11).

Conclusion and Implications

The study concludes that ASCA remains effective under diverse users, devices, and noisy environments, underscoring its practical security risk (Page 1). DECKER demonstrates that "acoustic emanations from keyboards are not only rich enough to permit high-fidelity keystroke inference, but also substantially more transferable across devices than previously reported—provided that domain-specific artifacts are explicitly disentangled (Page 7). The findings suggest that ASCA should not be viewed as just a purely laboratory phenomenon, but rather as a credible real-world threat (Page 12). This work establishes domain-invariant learning as a central requirement for the evaluation and deployment of ASCA systems under realistic threat models" (Page 12).

--- Page 1 ---

DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition Bikrant Bikram Pratap Maurya, Nitin Choudhury, Daksh Agarwal, and Arun Balaji Buduru. In ACM Asia Conference on Computer and Communications Security (ASIA CCS ’26), June 1–5, 2026, Bangalore, India. ACM, New York, NY, USA; 14 pages. https://doi.org/10.1145/3779208.3805984

--- Page 2 ---

ASIA CCS ’26, June 1–5, 2026, Bangalore, India Bikrant Bikram Pratap Maurya, Nitin Choudhury, Daksh Agarwal, and Arun Balaji Buduru. As microphones become pervasive in modern environments, the attack surface for ASCA expands accordingly; turning everyday laptop use into a potential threat to user privacy. Currently, ASCA is particularly concerning as it is passive, covert, and deployable in ordinary social settings; A single device in a common space can silently record keystrokes from multiple users, independent of network boundaries or encryption. Further, typed inputs are rarely random in nature; they follow linguistic or structural patterns that adversaries can exploit; The emergence of large language models (LLMs) amplifies this threat significantly, making recovery of coherent text even from noisy or partially incorrect keystroke predictions plausible. Previous research on keyboard ASCA has mostly been evaluated under narrow and idealized conditions; This includes setups that involve limited demographic variation among participants, low and stable noise levels, fixed device geometry, uniform keyboard switch mechanisms, and consistent typing habits [10, 14, 28]. Recent work, such as RefleXnoop and Heimdall, partially relaxes the geometric assumptions by developing systems which demonstrate that multipath reflections can enable non-line-of-sight (NLoS) keystroke recovery [32]. On the modeling side, ASCA pipelines have progressed from early signal-processing classifiers to modern probabilistic and deep learning approaches; Initial work relied on temporal and spectral features such as FFT coefficients, MFCCs, and power or energy profiles, combined with statistical classifiers [27, 35]. Subsequent systems incorporated hidden Markov models (HMMs) [11] and neural architectures [14, 28] to better capture temporal structure and variability in keystroke acoustics; Recent efforts adopt convolutional and transformer-based models on audio spectrograms [2] and show that LLMs can correct noisy keystroke predictions when linguistic context is available [2]. Despite these advances, most ASCA studies still assume controlled environments and single-device training regimes; They rarely account for the variability of real-world settings, where posture changes, ambient noise, reflective surfaces, and nearby devices introduce substantial acoustic distortion, and where users type on a wide range of laptop keyboards. As a result, the real-world generalizability and effectiveness of current ASCA techniques remain underexplored; Additionally, user biases, such as gender, age group, and hand dominance, play significant roles in ASCA, influencing factors including typing speed, key pressure, and typing errors; These gaps obscure realistic assessments of ASCA feasibility. Importantly, no prior work systematically evaluates ASCA across demographic diversity or examines user awareness of acoustic leakage threats; This highlights the need for a dataset and evaluation framework that spans diverse users, keyboards, and environments under realistic threat models while explicitly enabling analysis of user bias and human factors in acoustic side-channel attacks.

--- Page 3 ---

DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition ASIA CCS ’26, June 1–5, 2026, Bangalore, India Bikrant Bikram Pratap Maurya, Nitin Choudhury, Daksh Agarwal, and Arun Balaji Buduru.

--- Page 4 ---

ASIA CCS ’26, June 1–5, 2026, Bangalore, India Bikrant Bikram Pratap Maurya, Nitin Choudhury, Daksh Agarwal, and Arun Balaji Buduru. Figure 1: Threat scenario. Bob types on a laptop in a public area while Eve, seated at a separate table, passively captures keystroke acoustics from a concealed device; Eve later performs domain-invariant acoustic inference followed by LLM-driven sentence reconstruction; No line-of-sight to the keyboard is required.

1. Body-invariant acoustic encoder: Implement DECKER's four-stage generalization strategy—Keyboard Signature Normalization (KSN), domain-adversarial disentanglement, supervised cross-keyboard contrastive alignment, and Acoustic Style Randomization (ASR)—to produce key-discriminative yet keyboard-invariant embeddings. This allows the system to perform cross-keyboard and cross-user evaluations by suppressing device coloration and simulating unseen keyboard responses.

2. Sequence reconstruction with linguistic context: Integrate an LLM layer using a constrained beam search post-processing step to apply language priors, as shown in Equation (12): maximize the joint objective: Yˆ = arg max Y ∈ C1:T sum∑t=1 log Ptheta (ytxt) + λLM log PLM (Y). This capability allows the system to refine keystroke predictions using linguistic context, leading to a more realistic upper bound on leakage by correcting local errors.

3. Robust segmentation for continuous streams: Develop a joint segmentation–classification architecture to handle continuous-text ASCA pipelines, as suggested in Section 7.2: Future work should explore joint segmentation – classification models, sequence-to-sequence acoustic transformers, and streaming LLM integration. This system can process fully passive, continuous keystroke streams by mitigating the impact of temporal uncertainty introduced by segmentation jitter and spurious trigger events.

4. Multi-modal side-channel fusion: Implement cross-attention mechanisms (e.g., F3 or F5) to fuse acoustic embeddings with vision features to achieve superior performance, as demonstrated in Table 3 where Cross-Attn (A→V) 97.1% outperformed unimodal baselines. This enables inference based on a richer set of cues, moving beyond just acoustic signals.

5. Real-time computational feasibility: Optimize the DECKER pipeline for low-latency execution on commodity hardware, as shown in Table 6 where the total runtime is 2.3 ms and throughput reaches 430 keystrokes per second. This ensures that an attacker can perform key-level inference in real time even on CPU, rather than requiring desktop GPUs.

Acoustic side-channel attacks (ASCA) on keyboards pose a significant security risk, as keystrokes can be inferred from typing acoustics, revealing sensitive information. Prior ASCA studies are limited by small-scale datasets with restricted diversity in users, keyboards, and environments, constraining analysis across devices, microphones, and noise conditions. We introduce HEAR, a dataset designed to study ASCA along three axes: keyboard generalization, noise adaptation, and user bias. HEAR contains recordings from 53 participants using 37 laptop keyboards, collected in three realistic settings: (1) external microphone capture, (2) device microphone capture without network noise, and (3) VoIP-based streaming capture. This enables controlled evaluation across users, keyboards, and environments. On HEAR, we establish an ASCA benchmark spanning conventional features and pre-trained representations from raw audio and spectrograms in unimodal and multimodal settings. We propose DECKER, a domain-invariant keystroke inference framework with four stages: (1) Keyboard Signature Normalization to reduce device coloration, (2) domain-adversarial disentanglement to suppress keyboard identity, (3) supervised cross-keyboard contrastive alignment to enforce key consistency, and (4) Acoustic Style Randomization to synthesize unseen keyboard responses. We further explore sentence-level inference using an LLM-based post-processing layer to refine keystroke sequences via linguistic context. Results on HEAR show DECKER improves keystroke identification over strong baselines, particularly in cross-keyboard and cross-user settings, with further gains from language-model rectification. These findings highlight that ASCA remains effective across diverse users, devices, and noisy environments, underscoring its practical security risk.