Reproduce “PMKLC: Parallel Multi-Knowledge Learning-based Lossless Compression for Large-Scale Genomics Database” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hui Sun, Yanfeng Ding, Liping Yi, Huidong Ma, Gang Wang, Xiaoguang Liu, Cheng Zhong, Wentong Cai
Researchers from Nankai University and Nanyang Technological University developed PMKLC, a parallel multi-knowledge learning-based framework for lossless compression of large-scale genomic databases. It combines an automated multi-knowledge framework with GPU acceleration and multi-GPU parallelism, achieving average throughput improvements up to 1071.043% and robustness improvements up to 94.528% over baselines, while maintaining high compression ratios and efficient memory usage.
Key insights
- Integrating multiple knowledge sources (static public, static private, dynamic models) with an intelligent Model Selector enhances compression ratio and robustness by adapting to various data characteristics.
- GPU-accelerated (s,k)-mer encoding is highly effective for pre-processing, significantly boosting throughput and memory efficiency while extracting data redundancy.
- The Step-wise Model Passing (SMP) strategy is crucial for mitigating the cold-start problem in multi-GPU parallelism, improving early-stage compression efficiency and overall throughput.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.