TransientTrack: Advanced Multi-Object Tracking and Classification of Cancer Cells with Transient Fluorescent Signals

arXiv:2512.01885 · cs.CV, q-bio.CB, q-bio.QM · Submitted 2025-12-01 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "TransientTrack: Advanced Multi-Object Tracking and Classification of Cancer Cells with Transient Fluorescent Signals".

Jane: Here's the gist of the paper. The gist: TransientTrack is a deep learning-based framework for cell tracking in multi-channel microscopy video data with transient signals that fluctuates over time following processes such as the circadian rhythm of cells, allowing it to identify key cellular events like mitosis and apoptosis to build complete trajectories including cell lineage information.

Tom: Tracking cells in time-lapse videos is an essential technique for monitoring cell population dynamics at a single-cell level<ref:2512.01885#pg2> Current methods for cell tracking are developed on videos with mostly single, constant signals and do not detect pivotal events such as cell death<ref:2512.01885#pg4> This imbalance limits the ability to evaluate methods in applications such as quantifying responses to chemotherapeutic drugs, where accurate identification of death events is critical<ref:2512.01885#pg5> The paper addresses the lack of apoptosis detection and the challenge of more difficult multi-channel recordings in MOT by introducing TRANSIENTTRACK<ref:2512.01885#pg7> Unlike existing methods, our framework performs matching based solely on spatial information and cell detection network-derived embeddings<ref:2512.01885#pg8> By directly utilizing the detection embeddings without additional processing, our method remains lightweight, computationally efficient, and well-suited for large-scale time-lapse microscopy experiments<ref:2512.01885#pg8>.

Jane: TRANSIENTTRACK follows the tracking-by-detection paradigm<ref:2512.01885#pg10> In the first step, all cells are detected independently across consecutive frames by processing the raw multi-channel images with transient signals The detection component employs Deformable DETR, which consists of a CNN as backbone for feature extraction followed by a Transformer encoder–decoder architecture The decoder output is passed to two prediction heads: a classification head, which distinguishes between alive and dead cells, and a regression head, which estimates bounding boxes for the detected cells.

Tom: Matching Strategy The matching process is carried out sequentially between two consecutive frames During the first stage of matching, we match high-confidence detections of frame t and t + 1 with each other We base our matching on the distance between the centroids of the detections and the similarity of their embeddings To enforce embedding consistency during matching, we introduce a similarity threshold τsim applied to the L1 distance between cell embeddings cemb and cˆemb To resolve conflicts, we retain the candidate that minimizes the following weighted sum: λ · sim(c, cˆ) / τsim + (1 − λ) · d(c, cˆ) / τdst If multiple candidates remain, we assume a mitotic event and mark cell c as complete, initiating new tracks for the daughter cells.

Paper summary: Jane: So to summarize "TransientTrack: Advanced Multi-Object Tracking and Classification of Cancer Cells with Transient Fluorescent Signals," this framework provides a practical way to study cell populations under complex imaging conditions by combining robust tracking with biologically meaningful endpoints like cell division and death.

Lu: The authors highlight both the strengths and current limitations of their proposed method, suggesting that future work should concentrate on improving the cell detection component itself, since better detections lead to more reliable associations and lineage reconstructions.

Tom: I think the title itself is very descriptive: "Advanced Multi-Object Tracking and Classification of Cancer Cells with Transient Fluorescent Signals." It clearly tells you what it handles—tracking cells, classifying cancer cells, dealing with transient signals.

Meng: For us in the production space, the fact that they achieved high linkage and tracking accuracy across different doses is what matters most. It shows the method is robust enough to handle real-world variability in experimental setups.

Lalam: The main implication is that this method establishes a practical and comprehensive framework for studying cell populations when you have complex imaging conditions, giving us access to events like cell death that were previously missed by simpler trackers.

Jane: It moves the field forward by showing how deep learning can be applied to capture these transient biological processes in real-time, which is crucial for quantitative studies of cancer cell dynamics and treatment response.

Conclusion: Tom: So, we just wrapped up looking at TransientTrack, and now we're going to talk about what that whole title means for us.

Jane: Right, so it’s about tracking cancer cells using those tricky transient signals they found in microscopy videos with this new deep learning framework called TransientTrack.

Tom: Yeah, the authors are talking about how they built something that doesn't just track cells; it tries to figure out what's happening biologically while tracking them.

Lu: What really stands out is their approach to identifying those key events, mitosis and apoptosis, which are things you can't easily see with older methods.

Meng: I’m curious if this detection-based embedding matching they used is actually scalable for our kind of deployment at scale, or if it gets bogged down by the complexity of the multi-channel data.

Lalam: Based on what I’ve read across all the literature, their framework seems pretty solid because it doesn't need to quantify every single feature on the cell; it just uses those embeddings directly.

Tom: It’s that direct use of detection embeddings that makes it lightweight, which is a big plus for running these things on standard hardware instead of needing some massive processing farm.

Jane: So, when you look at the authors’ conclusion, they aren't just saying it works; they're pointing out what this actually *does* for biology.

Lu: They are showing how this lets them build complete trajectories including cell lineage information, which is huge because it gives context to the division and death events.

Tom: So, if we think about the bigger picture, this could really help us get a much clearer picture of how a drug actually affects cancer cells at the single-cell level.

Jane: Exactly. It moves us past just seeing 'more dead cells' to understanding *why* they are dying and what lineage information is being lost or preserved during treatment.

Tom: It’s about connecting the dots between the raw video data, these complex deep learning models, and actual biological processes like chemotherapy response.

Jane: We’ve seen how this method performs across different drug doses, which is a concrete result that really shows its utility in understanding treatment resistance mechanisms.

Lu: I think the real implication is that we can now use these tools to characterize subpopulations within a tumor that might be resistant to the drug, something previous methods just couldn't isolate.

Meng: From my side, if this framework can handle high-throughput data with good accuracy without needing a massive amount of manual tuning for every single cell type, then it becomes deployable.

Lalam: I’m quite sure this paper establishes a new baseline for tracking events like cell division and death in noisy, multi-channel imaging environments.

Tom: It’s definitely the framework that brings those specific biological questions—division and death—into the reliable realm of quantitative analysis.

Jane: So, next up, we're going to look at how they actually measured these things with CTC metrics like DET and LNK scores.

Florian Bäurger, Martim Dias Gomes, Nica Gutu, Adrian E. Granada, Noemie Moreau, Katarzyna Bozek

pro1 Institute for Biomedical Informatics, Faculty of Medicine and University Hospital Cologne, University of Cologne, Germany · Center for Molecular Medicine Cologne (CMMC), Faculty of Medicine and University Hospital Cologne, University of Cologne, Germany · Center for Genomic Regulation, Barcelona, Spain · Charité Comprehensive Cancer Center, Charité Universitätsmedizin, Berlin

cs.CV, q-bio.CB, q-bio.QM

Submitted: 2025-12-01

Updated: 2025-12-16

Comments: 13 pages, 7 figures, 2 tables

DOI: 10.1109/TMI.2026.3744441

Code: https://github.com/bozeklab/TransientTrack

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 72/100

The gist: The gist: TransientTrack is a deep learning-based framework for cell tracking in multi-channel microscopy video data with transient signals that fluctuates over time following processes such as the

Key concepts

Transient Signals: These are fluorescent signals in microscopy videos that change or fluctuate over time, often following internal biological processes like the circadian rhythm within cells. Traditional tracking methods struggle with these dynamic signals because they assume constant cell appearance.
Tracking-by-Detection Paradigm: This is the method used to find cells. First, a detection network finds all potential cell locations in every frame. Then, a matching strategy connects these detections across consecutive frames based on spatial location and learned embeddings, effectively building the track.
Cell Detection Embeddings: These are numerical representations generated by a deep learning model (Deformable DETR) for each detected cell. Instead of looking at raw image pixels, the matching process compares these compact embeddings to determine if two detections belong to the same cell and how they relate spatially.
Mitosis and Apoptosis Detection: The framework is designed to specifically look for critical cellular events. It uses a classification head within the detection network to distinguish between living and dead cells, while matching logic identifies cell division (mitosis) and links living cells to newly detected dead cells (apoptosis).

Terminology

Embeddings — 1× in this paper · explained in 2 episodes
Transformer — 1× in this paper · explained in 2 episodes
Unified Framework — 1× in this paper · explained in 6 episodes

Summary

The gist: TransientTrack is a deep learning-based framework for cell tracking in multi-channel microscopy video data with transient signals that fluctuates over time following processes such as the circadian rhythm of cells, allowing it to identify key cellular events like mitosis and apoptosis to build complete trajectories including cell lineage information.

Introduction and Motivation

Tracking cells in time-lapse videos is an essential technique for monitoring cell population dynamics at a single-cell level<ref:2512.01885#pg2> Current methods for cell tracking are developed on videos with mostly single, constant signals and do not detect pivotal events such as cell death<ref:2512.01885#pg4> This imbalance limits the ability to evaluate methods in applications such as quantifying responses to chemotherapeutic drugs, where accurate identification of death events is critical<ref:2512.01885#pg5> The paper addresses the lack of apoptosis detection and the challenge of more difficult multi-channel recordings in MOT by introducing TRANSIENTTRACK<ref:2512.01885#pg7> Unlike existing methods, our framework performs matching based solely on spatial information and cell detection network-derived embeddings<ref:2512.01885#pg8> By directly utilizing the detection embeddings without additional processing, our method remains lightweight, computationally efficient, and well-suited for large-scale time-lapse microscopy experiments<ref:2512.01885#pg8>.

Methodology

TRANSIENTTRACK follows the tracking-by-detection paradigm<ref:2512.01885#pg10> In the first step, all cells are detected independently across consecutive frames by processing the raw multi-channel images with transient signals The detection component employs Deformable DETR, which consists of a CNN as backbone for feature extraction followed by a Transformer encoder–decoder architecture The decoder output is passed to two prediction heads: a classification head, which distinguishes between alive and dead cells, and a regression head, which estimates bounding boxes for the detected cells.

Matching Strategy

The matching process is carried out sequentially between two consecutive frames During the first stage of matching, we match high-confidence detections of frame t and t + 1 with each other We base our matching on the distance between the centroids of the detections and the similarity of their embeddings To enforce embedding consistency during matching, we introduce a similarity threshold τsim applied to the L1 distance between cell embeddings cemb and cˆemb To resolve conflicts, we retain the candidate that minimizes the following weighted sum: λ · sim(c, cˆ) / τsim + (1 − λ) · d(c, cˆ) / τdst If multiple candidates remain, we assume a mitotic event and mark cell c as complete, initiating new tracks for the daughter cells. To detect cell death, we match a living cell c in frame t to a cell c in frame t + 1 classified as dead using the weighted sum equation

Evaluation and Results

We evaluated our approach using official CTC metrics, specifically the Detection Accuracy (DET) metric, the Linkage Accuracy (LNK) metric, and the Tracking Accuracy (TRA) metric TRANSIENTTRACK achieved a DET score of 0.9395 ± 0.0098, an LNK measure of 0.9156 ± 0.0179, and a TRA score of 0.9364 ± 0.0104 The results indicate that our approach achieved consistently high performance across the high-, medium-, and low-dose groups Furthermore, we reconstructed trajectories of 29,000 cells in a large-scale study, detecting nearly 8,800 mitotic events and 2,345 cell deaths.

Biological Insights

The analysis revealed a clear dose-dependent suppressive effect of cisplatin on proliferation (Fig. 4a) Apoptosis rates remained low and stable before treatment, but approximately 40 frames after exposure, apoptosis began to increase across all treated groups This lag shows that the cytotoxic effect of cisplatin is delayed rather than immediate, likely due to the time that takes cells to attempt to correct the DNA-damage induced by the cisplatin We also observed that cell size is not strongly preserved across successive generations in our dataset, suggesting that lineage information alone is insufficient to explain variability in cell size over time The analysis further revealed cell lineages that showed robust cell cycle progression despite the presence of the chemotherapeutic drug, which may represent subpopulations with features of drug resistance.

Conclusion

TRANSIENTTRACK establishes a practical and comprehensive framework for studying cell populations under complex imaging conditions by combining robust tracking with biologically meaningful endpoints Our experimental evaluation highlighted both the strengths and current limitations of the proposed method Future work should focus on improving cell detection, as higher-quality detections directly translate into more reliable associations and lineage reconstructions.

Acknowledgments

Florian B¨urger and Katarzyna Bozek were supported by the North Rhine-Westphalia return program (311-8.03.03.02-147635), Martim Dias Gomes was supported by the BMBF program Junior Group Consortia in Systems Medicine (01ZX1917B), and No´emie Moreau was supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - Project No. 386793560.

References

Adˇzemovi´c, M. Deep Learning-Based Multi-Object Tracking: A Comprehensive Survey from Foundations to State-of-the-Art Aharon, N., Orfaig, R., and Bobrovsky, B.-Z. BoTSORT: Robust associations multi-pedestrian tracking Akram, S. U., Kannala, J., Eklund, L., and Heikkil¨a, J. Cell tracking via proposal generation and selection Amable, L. Cisplatin resistance and opportunities for precision medicine Ben-Haim, T. and Raviv, T. R. Graph neural network for cell tracking in microscopy videos Bergmann, P., Meinhardt, T., and Leal-Taixe, L. Tracking without bells and whistles Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. Simple online and realtime tracking Bochinski, E., Eiselein, V., and Sikora, T. High-speed tracking-by-detection without using image information Chatterjee, R., Ghosh, M., Chowdhury, A. S., and Ray, N. Cell tracking in microscopic video using matching and linking of bipartite graphs Dai, Y., Hu, Z., Zhang, S., and Liu, L. A survey of detection-based video multi-object tracking Fukunaga, K. Introduction to statistical pattern recognition Gallusser, B. and Weigert, M. Trackastra: Transformerbased cell tracking for live-cell microscopy Gatti, L., Supino, R., Perego, P., Pavesi, R., Caserini, C., Carenini, N., Righetti, S. C., Zuco, V., and Zunino, F. Apoptosis and growth arrest induced by platinum compounds in U2-OS cells reflect a specific DNA damage recognition associated with a different p53-mediated response<ref:

Improvements for AI systems

Advanced cell fate determination in cancer therapy can be quantified by identifying mitosis (cell division) and apoptosis (cell death) events within multi-channel microscopy data, building complete trajectories, including cell lineage information.
The improved system can perform robust tracking on videos with transient fluorescent signals that fluctuate over time by leveraging detection embeddings directly for matching, making the framework lightweight, computationally efficient, and well-suited for large-scale experiments.
By integrating a two-stage matching algorithm using all detection boxes and the interpolation of missing tracklets with the Kalman Filter, the system ensures continuous trajectories even under challenging imaging conditions, overcoming limitations of methods that rely solely on spatial information or constant signals.
The system can distinguish between cell states by classifying cells as living or dead, which is crucial for understanding treatment response and resistance mechanisms, as demonstrated by tracking over 28,000 cell trajectories and quantifying proliferation, apoptosis, and lineage behavior under varying cisplatin dosages.
The improved AI system enables single-cell level study of factors determining cell fate across generations and lineages by analyzing cell size inheritability where it can reveal that cell size is not strongly preserved across successive generations in our dataset under treatment.
The system can identify subpopulations exhibiting potential drug resistance by analyzing lineages that showed robust cell cycle progression despite the presence of the chemotherapeutic drug, providing key insights for precision medicine strategies.