Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction

arXiv:2608.23356 · cs.IR, cs.LG, stat.ML · Submitted 2026-08-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction".

Jane: Here's the gist of the paper. The gist: The proposed Hierarchical Exponential–Gaussian Mixture (HEGM) model improves upon existing Exponential–Gaussian Mixture Network (EGMN) models for watch-time prediction by addressing failure modes like variance collapse and component redundancy through a hierarchical skip–watch decomposition, KL-based variance regularization, and structured initialization.

Tom: Problem Significance Watch-time (WT) prediction is a production-critical component of modern short-video recommendation systems where full-screen, auto-playing feeds dominate content consumption The paper addresses the challenge that WT distributions exhibit properties such as near-zero inflation, long tails, and multimodality which standard point estimation objectives cannot capture Distributional modeling estimates the full density pθ(y x) over the continuous WT variable rather than a single point estimate yˆ This distributional perspective is valuable because it can support multiple ranking signals, including expected WT, threshold-crossing probabilities, and uncertainty-aware decisions.

Jane: EGMN Limitations and HEGM Design The original EGMN model is shown to be fragile at industrial scale due to several failure modes observed in a large-scale reproduction study These pathologies include Gaussian variance collapse, component redundancy, and inactive components The proposed HEGM model addresses these issues through several principled modifications.

Tom: Key modifications include: 1.

Title and authors: Tom: So we're looking at this paper today titled "Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction" and it's about how to model watch time using a mixture of exponential and Gaussian components.

Jane: That sounds really technical. What does that actually mean for the short video recommendation systems we use every day?

Tom: Well, the core idea is moving away from just predicting one single number for watch time, like a point estimate, to estimating the entire distribution of possible watch times given some context.

Lu: Exactly. Instead of just giving us a score, we’re aiming for a full density estimation pθ(y x) over that continuous variable y > zero <ref:2608.23356#pg2>.

Jane: So if we can get the distribution, what kinds of things can we do with it? Can we still just pick a single number to rank videos?

Tom: We can still get a point estimate, which is the expected watch time, but that's just one signal. The paper shows that this distributional approach lets us use multiple signals at once, like expected watch time or even the probability of crossing a certain threshold <ref:2608.23356#pg2>.

Meng: From my side, I worry about how much extra work this adds to the serving infrastructure. If we're doing distribution estimation instead of just a single scalar prediction, it might slow things down significantly. What's the actual computational cost here?

Tom: The authors say that for deployment, the overhead is manageable. They estimate adding only one point two two milliseconds to the median latency, which is well below our serving constraint of thirty milliseconds <ref:2608.23356#pg1>. The model parameters are around one point five million, and they use a batch size of sixteen thousand for serving inference in TensorRT format <ref:2608.23356#pg1>.

Jane: That's reassuring to hear about the latency. But what was wrong with the previous model, EGMN? Why did they need this new Hierarchical Exponential-Gaussian Mixture?

Lu: The original EGMN model, as shown in their reproduction study, it had these issues. They found that it was vulnerable to variance collapse, component redundancy, and having inactive components in a large-scale setup <ref:2608.23356#pg1>.

Tom: So the EGMN just broke down under pressure? What specifically were those failure modes we're talking about?

Lu: They observed Gaussian variance collapse, which means some of the Gaussian components start having zero variance, leading to a loss of information <ref:2608.23356#pg1>. Plus, they saw component redundancy and inactive components that weren't contributing anything useful <ref:2608.23356#pg1>.

Title and authors: Meng: And from a production standpoint, having redundant or inactive components means we're wasting model capacity. If you have too many components that don't matter, the model just gets bloated and less efficient at making decisions <ref:2608.23356#pg1>.

Jane: So this new Hierarchical Exponential-Gaussian Mixture, HEGM, is designed to fix those specific problems? How does it tackle variance collapse and redundancy?

Lu: It tackles these issues through a few principled modifications. First, they use a hierarchical skip–watch decomposition with a sigmoid gate pskip(x) based on the hidden representation h <ref:2608.23356#pg1>. This lets them explicitly separate quick-skip behavior from engaged watching <ref:2608.23356#pg1>.

Tom: A skip probability gate sounds really interpretable for business analytics. It gives us a way to see when a user is just skipping versus when they are actually watching <ref:2608.23356#pg1>.

Jane: And what about the training side? How do they make sure the components stay meaningful and don't collapse?

Lu: They use KL-based variance regularization, which I like. It penalizes degenerate components without forcing their means to follow some other constraint <ref:2608.23356#pg1>. This anchors the component variances to duration-conditioned empirical scales, which keeps the structure clearer even when there are many components <ref:2608.23356#pg1>.

Meng: Anchoring the variances to empirical scales sounds like it prevents those degenerate states we saw before. But what about initialization? How do they get the model started correctly?

Tom: They use structured initialization for the Gaussian means and standard deviations. They use deterministic uniform initialization for the means and specific initializations for the variances <ref:2608.23356#pg1>. This stabilizes training in those first epochs, which is crucial because removing it causes a sharp drop in their XAUC from zero point seven one eight eight to zero point six eight four one <ref:2608.23356#pg4>.

Jane: So, structured initialization is pretty critical for getting the training stable right at the beginning? That’s a big detail for someone who isn't reading the math every day.

Lu: It really is. And they also removed two things that they found didn't help performance: removing entropy regularization over mixture weights and removing the forced Gaussian shift by not shifting Gaussian means with the exponential mean <ref:2608.23356#pg1>. That simplifies things without hurting results <ref:2608.23356#pg1>.

Tom: So, we're talking about a model that is more robust, better initialized, and cleaner in its structure than the original EGMN <ref:2608.23356#pg1>. This sounds like a solid engineering improvement on top of the distributional modeling aspect.

Title and authors: Jane: And what kind of real-world results are we seeing? Does this actually beat the previous models on things that matter, like ranking quality?

Tom: Yes, across public benchmarks like KuaiRec and VK-LSVD, HEGM outperforms EGMN and other baselines in both ranking quality and threshold-event prediction <ref:2608.23356#pg1>. It also shows improvements in watch-completion predictions at the sixty seconds threshold, achieving an ROC AUC of zero point eight nine zero three compared to EGMN’s zero point eight three five six <ref:2608.23356#pg4>.

Meng: But we need more than just benchmark numbers. How does this perform when you actually deploy it on real traffic? Does it hold up in a production environment?

Tom: They validated it on a two hundred eighty-two million interactions industrial dataset, and they did a live A/B test on a major short-video platform <ref:2608.23356#pg1>. That test confirmed statistically significant engagement lifts, like plus nine point two six percent session depth <ref:2608.23356#pg1>.

Jane: So, when you look at the big picture for a user who just watches videos, what does this mean for their experience? What does it change for them?

Lu: It means that the model can now provide several signals useful for ranking, not just one point estimate <ref:2608.23356#pg1>. We get expected watch time, absolute threshold probability, percentage completion probability, and predictive uncertainty <ref:2608.23356#pg1>.

Tom: That's the payoff. It means we can support diverse decision criteria without having to retrain the model for every new thing we want to predict <ref:2608.23356#pg1>. The paper is showing that distributional modeling can be accurate, stable, and deployable when the mixture structure lines up with actual user behavior <ref:2608.23356#pg1>.

Jane: It sounds like they've moved past just getting a single number and are now giving us a richer picture of what’s happening with watch time in these videos. So, as we wrap up this discussion on the Hierarchical Exponential-Gaussian Mixture for Watch-Time Distribution Prediction, what's the final word?

Lu: The main point is that HEGM successfully moves beyond scalar point prediction to provide a robust framework for short-video recommendation systems <ref:2608.23356#pg1>. It’s about making the mixture structure align with user behavior <ref:2608.23356#pg1>.

Meng: For deployment, it’s good that the computational overhead is only one point two two milliseconds on median latency <ref:2608.23356#pg1>. That keeps us within our service constraints <ref:2608.23356#pg1>. If it was way slower, we wouldn't deploy it, no matter how good the accuracy was <ref:2608.23356#pg1>.

Lalam: I am confident in this direction because the results hold up across multiple datasets and validation methods <ref:2608.23356#pg1>. The paper demonstrates that distributional watch-time modeling can be accurate, stable, interpretable, and deployable <ref:2608.23356#pg1>.

Title and authors: Tom: It’s a solid piece of work because it tackles the specific failure modes of the older EGMN model directly with concrete fixes like KL regularization and structured initialization <ref:2608.23356#pg1>. The numbers show that these fixes translate to tangible gains in ranking quality <ref:2608.23356#pg4>.

Jane: It’s interesting how they’ve structured the model hierarchy—the skip-watch decomposition—to make the underlying process more understandable <ref:2608.23356#pg1>. That structural alignment with user behavior is a big conceptual step for us to consider moving forward <ref:2608.23356#pg1>.

Lu: And the way they’ve quantified their regularization term, that KL-based variance regularization, it anchors the component variances to duration-conditioned empirical scales <ref:2608.23356#pg1>. That prevents those degenerate states we saw before <ref:2608.23356#pg1>.

Meng: For production teams like me, the stability of that structure is what matters most when we are dealing with high volumes of data and potential anomalies <ref:2608.23356#pg1>. A stable model is easier to maintain than a highly complex one that might break unexpectedly <ref:2608.23356#pg1>.

Lalam: I think the ability to provide multiple ranking signals without retraining is where the real value lies for our systems <ref:2608.23356#pg1>. It supports diverse decision criteria directly from one model <ref:2608.23356#pg1>.

Tom: So, to summarize this Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction, the authors show how fixing initialization and variance collapse issues with KL regularization leads to a model that’s better for production than the EGMN baseline <ref:2608.23356#pg1>.

Jane: It’s a very practical application of distributional modeling where the structure of the mixture is aligned with what we know about how people watch content <ref:2608.23356#pg1>. That’s something to keep an eye on as we build these recommendation systems <ref:2608.23356#pg1>.

Lu: The paper sets a good direction by showing that this type of hierarchical decomposition is a way to explicitly separate the different behavioral regimes in watch time <ref:2608.23356#pg1>. It gives us a clear mechanism to analyze and potentially improve those regimes <ref:2608.23356#pg1>.

Meng: For us, the key is that it's deployable at scale with acceptable latency, which is a major hurdle when we think about more complex models <ref:2608.23356#pg1>. The authors hit that constraint well enough for us to consider this for real-world use <ref:2608.23356#pg1>.

Lalam: So, the main implication is that we can get a more reliable and interpretable model for watch time prediction by using a hierarchical mixture structure <ref:2608.23356#pg1>. It’s a framework that moves us toward richer decision-making signals <ref:2608.23356#pg1>.

The paper's summary: Tom: So to recap, this new Hierarchical Exponential-Gaussian Mixture model moves away from just predicting a single watch time number and instead tries to model the whole distribution of possible watch times given some input <ref:2608.23356#pg1>.

Jane: Right, it’s like instead of saying a video will be watched for three minutes, it’s estimating the range of what time that video might actually get watched across all different users and situations <ref:2608.23356#pg1>.

Tom: Exactly. And they did this by using a mixture of an exponential component for those quick skips near zero and several Gaussian components for the more engaged watching patterns <ref:2608.23356#pg7>.

Lu: The big move here is how they structure it, using that hierarchical skip–watch decomposition with a sigmoid gate pskip(x) <ref:2608.23356#pg1>. That lets them explicitly separate the quick-skip behavior from the actually engaged watching regimes <ref:2608.23356#pg1>.

Meng: From a deployment standpoint, that separation is useful because it means we can use that skip probability to decide whether to show a video or not <ref:2608.23356#pg1>. It gives us an interpretable signal about the user's intent right there in the model structure <ref:2608.23356#pg1>.

Jane: I see. But how do they keep this complex structure from falling apart when you feed it massive amounts of data? That’s where their variance regularization comes in <ref:2608.23356#pg1>.

Tom: They use a KL-based variance regularization term, LKL, which is mean-agnostic <ref:2608.23356#pg1>. It penalizes degenerate components—those components with zero variance that just don't contribute anything—without forcing the means to follow some other weird constraint <ref:2608.23356#pg1>.

Lu: That’s clever because it anchors the component variances to duration-conditioned empirical scales <ref:2608.23356#pg1>. It keeps the model from collapsing into just one thing even when you have a huge number of components <ref:2608.23356#pg1>.

Meng: I’m concerned about that anchoring part, though. If those empirical scales are wrong, the whole structure gets skewed <ref:2608.23356#pg1>. It puts a lot of pressure on those initial data calibrations <ref:2608.23356#pg1>.

Jane: And they did remove things like entropy regularization and forcing the Gaussian shift, which they said didn't actually help performance <ref:2608.23356#pg1>. That simplifies the math significantly without losing predictive power <ref:2608.23356#pg1>.

Tom: And on the results side, they showed this model beats the older EGMN baseline on ranking quality and threshold prediction metrics <ref:2608.23356#pg4>. They even saw a lift in session depth during a live A/B test <ref:2608.23356#pg1>.

Lu: It’s the stability that really matters to me, Tom. Because they showed structured initialization is critical; removing it caused the AUC to drop from zero point seven one eight eight down to zero point six eight four one <ref:2608.23356#pg4>. That proves how sensitive these models are to starting conditions <ref:2608.23356#pg4>.

Meng: So, if we take that sensitivity into account, the structured initialization is not optional for us in production <ref:2608.23356#pg1>. It’s a requirement for stability <ref:2608.23356#pg1>.

Jane: And finally, what this means for the end user is that we can now give them several different types of signals—like expected time, completion probability, and even uncertainty—all from this one model <ref:2608.23356#pg1>.

Tom: It means we have much richer decision-making tools without needing to run separate models for every new metric <ref:2608.23356#pg1>. This framework is showing that distributional modeling can be robust and deployable when it’s built around how people actually behave <ref:2608.23356#pg1>.

Lu: The direction they’re pushing with this hierarchical decomposition is interesting because it gives us a clear way to analyze those different behavioral regimes in watch time <ref:2608.23356#pg1>. We can actually use the skip probability gate to understand those regimes better <ref:2608.23356#pg1>.

Meng: For me, it’s good that the computational cost is only one point two two milliseconds on median latency <ref:2608.23356#pg1>. That keeps us well within our serving constraints <ref:2608.23356#pg1>.

Jane: So, we’re looking at a model that’s more stable, better initialized, and gives us richer signals for users by aligning the mixture structure with real behavior <ref:2608.23356#pg1>. The next thing to watch is how they handle even more complex distributions next.

The paper's improvements: Tom: So we’ve talked about how the HEGM model was built to handle those old EGMN problems like variance collapse and component redundancy <ref:2608.23356#pg1>. Now we’re looking at what they actually *suggest* as improvements for the model itself <ref:2608.23356#pg9>.

Jane: Okay, so what are these new suggestions? Are they just tweaking the knobs, or is there a bigger conceptual shift in how this AI handles watch time <ref:2608.23356#pg1>?

Lu: They suggest keeping the hierarchical skip–watch decomposition because it’s good for interpretability <ref:2608.23356#pg9>. The idea is to keep that explicit separation between quick skips and engaged watching <ref:2608.23356#pg1>.

Meng: So the paper is suggesting we stick with that structure because it helps us figure out *why* a user isn't watching long videos <ref:2608.23356#pg9>. That level of insight is valuable for understanding user behavior at scale <ref:2608.23356#pg1>.

Tom: I think the most important thing they suggest is sticking with the KL-based variance regularization because it keeps the model from getting degenerate <ref:2608.23356#pg9>. It prevents those components from vanishing in training <ref:2608.23356#pg1>.

Jane: That means they want to keep anchoring those variances to duration-conditioned empirical scales, which is what makes the structure more stable <ref:2608.23356#pg1>. It’s like giving the model a physical constraint on how much variance each component can have <ref:2608.23356#pg1>.

Lu: Right, and they also suggested removing the entropy regularization over mixture weights because it didn't actually help performance <ref:2608.23356#pg1>. That’s a good signal that sometimes you should just keep things simpler if the complex stuff isn't adding value <ref:2608.23356#pg1>.

Meng: I agree with that; complexity is often a trap in production <ref:2608.23356#pg9>. Less regularization terms mean fewer parameters to manage and less risk of overfitting noise <ref:2608.23356#pg1>.

Tom: And they mentioned that the removal of the forced Gaussian shift, not moving the means by the exponential mean, is also a suggestion <ref:2608.23356#pg1>. It simplifies things without hurting results <ref:2608.23356#pg1>.

Jane: So what this means for someone just listening to the show is that the core idea isn't about adding more complicated math, but about making sure the structure you have—the hierarchy and those variances—is fundamentally sound and doesn't collapse under heavy traffic <ref:2608.23356#pg1>.

Lu: It’s a direction toward building models where the structural alignment with user behavior is the main feature, rather than just fitting whatever weird mathematical shape works best for a single prediction <ref:2608.23356#pg1>.

Meng: If we can build models that are inherently more stable and interpretable like this, it makes deploying them to real-world traffic much less scary <ref:2608.23356#pg1>. That stability is what I need when I’m trying to ship something reliably <ref:2608.23356#pg1>.

Tom: Exactly. The paper shows that this path leads to a model that is not just accurate on a benchmark, but one that actually performs well when you put it into the wild <ref:2608.23356#pg4>.

Jane: So, what’s the next step then? Are we moving toward testing other component families for watch time prediction or is this distribution modeling framework considered a solid foundation for future work <ref:2608.23356#pg1>?

Conclusion: Tom: So to wrap up, this Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction model fixes key problems in older models like EGMN by adding structured initialization and KL-based variance regularization <ref:2608.23356#pg1>.

Jane: It moves the goal from just guessing a single watch time to understanding the entire range of possibilities, which is much richer data for a recommendation system <ref:2608.23356#pg1>.

Lu: Yeah, and they proved that this distributional approach isn't just theoretical; it actually beats the old models on ranking quality and threshold prediction metrics across benchmarks like KuaiRec <ref:2608.23356#pg1>.

Meng: For us, the fact that they validated this on a two-hundred and eighty-two million interaction dataset is important because it shows it holds up under industrial scale testing <ref:2608.23356#pg1>. It’s not just a nice result on a small test set <ref:2608.23356#pg4>.

Lalam: I’m confident that the reliability of this distributional output is very high; it provides multiple signals like expected time and uncertainty that we can use directly for ranking decisions <ref:2608.23356#pg1>.

Jane: That’s the practical implication, right? We get more decision criteria without having to retrain the whole system every time we want a new signal <ref:2608.23356#pg1>.

Tom: The computational cost is also low, adding only one point two two milliseconds to median latency <ref:2608.23356#pg1>. That makes it feasible for production serving systems <ref:2608.23356#pg1>.

Lu: And the paper does flag that they haven't explored other component families that might better match the actual support of watch time distributions yet <ref:2608.23356#pg1>. That’s their stated limitation for future work <ref:2608.23356#pg1>.

Meng: I agree with Lu; the model is solid, but if you want to push it further, you need to look at those other component families <ref:2608.23356#pg1>. That’s where the next engineering effort should go <ref:2608.23356#pg1>.

Jane: So, while this paper gives us a very robust framework for modeling watch time distributions, the future direction is looking at making those mixture components even more specific to what users actually do <ref:2608.23356#pg1>.

Tom: Exactly. It’s a step forward from just point prediction, and it shows that aligning the model structure with user behavior can lead to stable and accurate results <ref:2608.23356#pg1>.

Lalam: I see this as laying a really strong foundation for any future work on short-video recommendation systems <ref:2608.23356#pg1>.

Jane: It certainly sets a high bar for stability and interpretability in how we model human attention patterns <ref:2608.23356#pg1>.

Sofia Gulevskaia, Mikhail Trapeznikov, Aleksandr Poslavsky, dr.slink@vk.com

AI VK · Lomonosov MSU

cs.IR, cs.LG, stat.ML

Submitted: 2026-08-24

Updated: 2026-10-21

Comments: 16 pages, 8 figures, 7 tables, accepted at IEEE International Conference on Data Mining (ICDM 2026)

Code: https://github.com/rw404/HEGM

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist: The proposed Hierarchical Exponential–Gaussian Mixture (HEGM) model improves upon existing Exponential–Gaussian Mixture Network (EGMN) models for watch-time prediction by addressing

Key concepts

Exponential-Gaussian Mixture (EGMN)
This is the original model that predicts watch time by combining an exponential distribution (for quick skips near zero) with several Gaussian distributions. It reflects how users often either skip quickly or engage in different levels of watching.
Hierarchical Skip-Watch Decomposition
This modification uses a sigmoid gate to explicitly separate user behavior into two regimes: quick skipping and engaged watching. This makes the model more interpretable by clearly defining these distinct behavioral patterns.
KL-based Variance Regularization (LKL)
This technique penalizes components that become degenerate or collapse, preventing them from losing their structure during training. It anchors component variances to empirical scales, ensuring the model maintains a clear multimodal structure even with many components.

Summary

The gist: The proposed Hierarchical Exponential–Gaussian Mixture (HEGM) model improves upon existing Exponential–Gaussian Mixture Network (EGMN) models for watch-time prediction by addressing failure modes like variance collapse and component redundancy through a hierarchical skip–watch decomposition, KL-based variance regularization, and structured initialization.

Problem Significance

Watch-time (WT) prediction is a production-critical component of modern short-video recommendation systems where full-screen, auto-playing feeds dominate content consumption The paper addresses the challenge that WT distributions exhibit properties such as near-zero inflation, long tails, and multimodality which standard point estimation objectives cannot capture Distributional modeling estimates the full density pθ(y x) over the continuous WT variable rather than a single point estimate yˆ This distributional perspective is valuable because it can support multiple ranking signals, including expected WT, threshold-crossing probabilities, and uncertainty-aware decisions.

EGMN Limitations and HEGM Design

The original EGMN model is shown to be fragile at industrial scale due to several failure modes observed in a large-scale reproduction study These pathologies include Gaussian variance collapse, component redundancy, and inactive components The proposed HEGM model addresses these issues through several principled modifications. Key modifications include:

  1. Hierarchical skip–watch decomposition, which uses a sigmoid gate to estimate the skip probability pskip(x) based on the hidden representation h.

  2. Structured initialization for Gaussian means and standard deviations, using deterministic uniform initialization for means and specific initializations for variances.

  3. Removal of the forced Gaussian shift by not shifting Gaussian means by the exponential mean.

  4. Removal of the entropy regularizer over mixture weights, as empirical results showed it did not improve performance.

HEGM Training Objective and Regularization

The final HEGM objective function is defined as LHEGM = LNLL + λregLreg + λKLLKL. The KL-based variance regularization term, LKL, penalizes degenerate components without constraining their means by anchoring component variances to duration-conditioned empirical scales. The penalty is calculated as DKL N µk, σ¯2b(i) = 1/2nK Xn i=1 X K k=1 σ2k,θ(xi) σ¯2b(i) − log σ2k,θ(xi) σ¯2b(i) − 1. This regularization is mean-agnostic and anchors component variances to duration-conditioned empirical scales.

Empirical Superiority and Production Validation

Across public benchmarks (KuaiRec, VK-LSVD) and a 282M interactions industrial dataset, HEGM outperforms EGMN and other baselines in ranking quality and threshold-event prediction. A live A/B test on a major short-video platform confirmed statistically significant engagement lifts, such as +9.26% session depth. The model maintains competitive point-estimation accuracy while substantially improving mixture stability and interpretability. Ablation studies confirm that structured initialization proves critical, as removing it causes a sharp drop in XAUC from 0.7188 to 0.6841. Furthermore, the HEGM+KL variant outperforms unregularized HEGM in watch-completion predictions at the 60 seconds threshold, achieving an ROC AUC of 0.8903 compared to EGMN’s 0.8356.

Computational Efficiency and Deployment

The computational overhead of HEGM is acceptable for production deployment, adding only 1.22 ms to the median latency, which remains far below the 30 ms serving constraint. The model parameters are approximately 1.5 M, and batch size is set to 16,000. The deployment infrastructure uses 48× NVIDIA RTX 6000 Pro GPUs for serving inference in TensorRT format.

Conclusion

This work demonstrates that distributional WT modeling can be accurate, stable, interpretable, and deployable when the mixture structure is aligned with user behavior. HEGM successfully moves beyond scalar point prediction to provide a robust framework for short-video recommendation systems. Future work may explore alternative component families that better match the support of the WT distribution, and probabilistic quality should be evaluated more comprehensively. The code and models are publicly released at https://github.com/rw404/HEGM.

--- Page 1 ---

HIERARCHICAL EXPONENTIAL-GAUSSIAN MIXTURES FOR WATCH-TIME DISTRIBUTION PREDICTION∗ Sofia Gulevskaia AI VK & Lomonosov MSU Moscow, Russia Mikhail Trapeznikov AI VK & Lomonosov MSU Moscow, Russia Aleksandr Poslavsky AI VK dr.slink@vk.com Alexander D’yakonov B AI VK djakonov@mail.ru

--- Page 2 ---

Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction

--- Page 3 ---

Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction

--- Page 4 ---

Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction

--- Page 5 ---

Hierarchical Exponential–Gaussian Mixture (EGMN) models WT using a mixture of an exponential component and several Gaussian components, reflecting the empirical structure of short-video consumption: a large mass of quick skips near zero and multiple engaged-watch regimes corresponding to partial views, completions, and replays The framework is backbone-agnostic, integrates easily into multi-task production architectures, and provides various distributional statistics.

--- Page 6 ---

Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction

--- Page 7 ---

Hierarchical Exponential–Gaussian Mixture (EGMN) models WT using a mixture of one exponential component and several Gaussian components: pθ(y x) = w0(x)fexp(y λ(x)) + X K k=1 wk(x)fgauss(y µk(x), σ2k (x)), where wk(x) ≥ 0 and PK k=0 wk(x) = 1, fexp y λ = λe−λy, y ≥ 0, fgauss y µ, σ2 = 1/√2πσ2 exp -(y − µ)2/2σ2. The exponential component models coarse-grained skewness and quick skips near zero, while the Gaussians capture fine-grained engaged-watch patterns The corresponding expected WT ybθ(x) is available in closed form: Epθ [Y x] = w0(x)λ(x) −1 + X K k=1 wk(x)µk(x).

--- Page 8 ---

Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction

--- Page 9 ---

Hierarchical Exponential–Gaussian Mixture (EGMN) models WT using a mixture of one exponential component and several Gaussian components: pθ(y x) = w0(x)fexp(y λ(x)) + X K k=1 wk(x)fgauss(y µk(x), σ2k (x)), where wk(x) ≥ 0 and PK k=0 wk(x) = 1, fexp y λ = λe−λy, y ≥ 0, fgauss y µ, σ2 = 1/√2πσ2 exp -(y − µ)2/2σ2.

Improvements for AI systems

  1. Hierarchical Skip-Watch Decomposition allows for an explicit separation between quick-skip behavior and engaged watching by using a sigmoid gate pskip(x). This enables a more interpretable separation of behavioral regimes, enhancing model interpretability and downstream business analytics.

  2. KL-based Variance Regularization prevents Gaussian collapse by penalizing degenerate components without constraining their means. This ensures that the model maintains a clearer multimodal structure even when the number of components becomes large, improving generalization in over-parameterized scenarios.

  3. Structured Initialization ensures that Gaussian means are initialized uniformly across the normalized support and standard deviations are initialized appropriately to preserve early separation while avoiding spikes. This stabilizes the first epochs of training, addressing initialization sensitivity and preventing immediate pathological behavior observed in unregularized models.

  4. Removing entropy regularization and forced Gaussian shift prevents model degradation, as both were empirically found not to improve performance or stability. This simplifies the framework while maintaining competitive predictive power across ranking quality and threshold-event prediction metrics.

  5. The HEGM model can provide multiple signals useful for ranking, such as Expected WT (Point Estimate), Absolute Threshold Probability, Percentage Completion Probability, and Predictive Uncertainty. This allows the model to support diverse decision criteria without retraining or auxiliary tasks.

Sources

Related papers