内存计算环境下基于索引结构的内存优化策略

英昌甜; 王维庆; 于炯; 卞琛; 国冰磊; 祁雷

doi:10.13568/j.cnki.651094.2018.01.003

您当前的位置：

首页 >

文章列表页 >

内存计算环境下基于索引结构的内存优化策略

研究精粹 | 更新时间：2026-01-21

- 内存计算环境下基于索引结构的内存优化策略
- 内存计算环境下基于索引结构的内存优化策略
- 新疆大学学报（自然科学版中英文） 2018年35卷第1期页码：13-21
- 作者机构：
  
  1. 新疆大学电气工程学科博士后科研流动站
  2. 新疆大学软件学院
  3. 新疆大学电气工程学院
  4. 乌鲁木齐职业大学信息工程学院
- 作者简介：
- 基金信息：
  
  新疆维吾尔自治区自然科学基金项目(2017D01A20)
- DOI：10.13568/j.cnki.651094.2018.01.003
  中图分类号： TP333
- 纸质出版：2018
- 稿件说明：
移动端阅览
[1]英昌甜,王维庆,于炯,等.内存计算环境下基于索引结构的内存优化策略[J],2018,35(01):13-21.
[1]英昌甜,王维庆,于炯,等.内存计算环境下基于索引结构的内存优化策略[J],2018,35(01):13-21. DOI： 10.13568/j.cnki.651094.2018.01.003.

DOI：10.13568/j.cnki.651094.2018.01.003.

摘要

由于内存计算能够较好的满足在线数据密集型应用的需求

近年来受到了研究者的广泛关注.内存云存储数据时使用哈希结构来提高写入和恢复效率

然而该结构会降低系统读性能

同时增加系统清理回收内存的开销.为了解决这个问题

提出一种基于索引压缩存储的内存优化策略.在存储时

将内存划分为两部分

哈希存储和排序存储.在系统繁忙时

对于实时写入和更新的数据存储时采用占用空间较多、插入效率较高的哈希存储;在系统空闲时段时

利用基于索引压缩的排序存储算法

将哈希存储转换为占用内存空间较少、查找效率较高的排序存储.实验结果表明

同未进行优化的Tachyon单一哈希结构存储策略相比

该策略能够很好地均衡系统的写入和读取访问的效率.

Abstract

Recently

in-memory computing has been attracting increasing attention because it can satisfy the requirements of the online data intensive application. Hash structure could improve the write efficiency of the storage system

at the same time limit the access efficiency. In order to solve the storage problem

an optimized strategy based on index compact storage was proposed to optimize the storage of data index.Firstly

memory of severs were divided into two parts

hash store and sorted store. When the system is busy

the new writing and updating data are store in the hash store which has high insertion efficiency but occupy much space. When the system is idle

by using sorted store algorithm

convert the hash store into the sorted store which has high access efficiency and occupy little space. The experiment demonstrates that comparing with non-optimization Tachyon

the proposed strategy is the proper trade-off between write and access efficiency.

关键词

Keywords

references

Li M,Tan J,Wang Y,et al.Spark Bench:a spark benchmarking suite characterizing large-scale in-memory data analytics[J].Cluster Computing,2017:1-15.

英昌甜,于炯,鲁亮,刘建矿.基于小文件的内存云存储优化策略[J].计算机应用,2014,11(34):3104-3108.

英昌甜,于炯,卞琛,等.并行计算框架Spark的自动检查点策略[J].东南大学学报(自然科学版),2017,47(2):231-235.

Vamanan B,Sohail H B,Hasan J,et al.Timetrader:Exploiting latency tail to save datacenter energy for online search[C]//Proc of the 48th Int Symp on Microarchitecture.New York,NY:ACM,2015:585-597.

Fan B,Lim H,Andersen D,et al.Small cache,big effect:provable load balancing for randomly partitioned cluster services.[C]//Proceedings of the ACM Symposium on Cloud Computing,Cascais,Portugal,2011.

Wang P,Sun G,Jiang S,et al.An efficient design and implementation of LSM-tree based key-value store on open-channel SSD[C]//Proceedings of ACM European Conference on Computer Systems,Amsterdam,the Netherlands,2014:1-14.

孔兰菊,李庆忠,史玉良,等.面向Saa S应用基于键值对模式的多租户索引研究[J].计算机学报,2010,33(12):2239-2247.

Zaharia M,Chowdhury M,Das T,et al.Resilient distributed datasets:a fault-tolerant abstraction for in-memory cluster computing[C]//Usenix Conference on Networked Systems Design and Implementation.Berkeley,CA,US:USenix,2012:141-146.

Li Haoyuan,Ghodsi A,Zaharia M,et al.Tachyon:Reliable,memory speed storage for cluster computing frameworks[C]//IEEE Conference on SYSTEM-ON-CHIP.Piscataway,New Jersey,US:IEEE,2014:1-15.

Li Haoyuan,Ghodsi A,Zaharia M,et al.Tachyon:Memory throughput I/O for cluster computing frameworks[C]//Proc of 7th Workshop on Large-Scale Distributed Systems and Middleware.New York:ACM,2013:1-6

Zaharia M,Chowdhury M,Franklin M J,et al.Spark:Cluster computing with working sets[C]//Usenix Conference on Hot Topics in Cloud Computing.Berkeley,CA,US:USenix,2010:1765-1773.

浏览量

154

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据