Computerized data has become critical to the survival of an enterprise. Company must have a strategy forrecovering their data should data lose. RAID is a popular mechanism to offer good fault-tolerance. But RAID can n...Computerized data has become critical to the survival of an enterprise. Company must have a strategy forrecovering their data should data lose. RAID is a popular mechanism to offer good fault-tolerance. But RAID can notwork well when one more disks fail. In this paper, we present an efficient Network-based high High-Tolerance Distributed Shared Storage mechanism called FTDSS. FTDSS makes use of disk space of node in Network to build alarge public shared storage space. Users can read/write their file from/to the public storage space from node of net-work Physically File is stored in each node in form of data fragment or XOR verify fragment. Because of redundantXOR fragments, file is available even when two more nodes fail. FTDSS realize distant redundant srorage. At last.this paper use experiment to prove that FTDSS can offer high fault-tolerance and advanced performance.展开更多
随着多媒体数据的爆发式增长,云端数据呈现出大规模多模态混合并存的特性.服务于数据分析的传统存储系统因为缺乏数据的语义管理而面临读取延时超长的挑战.针对图像和文本2种模态数据,在传统存储系统之上提出一种跨模态图文数据内容筛...随着多媒体数据的爆发式增长,云端数据呈现出大规模多模态混合并存的特性.服务于数据分析的传统存储系统因为缺乏数据的语义管理而面临读取延时超长的挑战.针对图像和文本2种模态数据,在传统存储系统之上提出一种跨模态图文数据内容筛选存储机制(cross-modal image and text content sifting storage,CITCSS),用于提供大规模在线相似性内容筛选服务,从存储系统层面缓解数据分析时必须从存储中读出所有数据的读带宽压力.机制分为离线与在线2个阶段.离线阶段中,引入基于自监督的生成对抗式Hash方法,系统利用这种方法生成语义元数据.然后,将元数据注入独立的元数据空间.最后,根据相似性Hash码间汉明距离能够度量语义距离的特点,利用Neo4j图数据库构建Hash元数据图谱,并在语义图谱中建立Hash码与存储路径之间的映射.在线阶段中,用户发送与分析相关的图像或文本,存储系统首先转化数据为Hash码.然后,在筛选半径内通过Hash元数据图谱寻找相似节点,进而找到相似文件的底层存储路径返回筛选数据.实验结果表明,与传统语义存储系统相比,CITCSS在召回率超过98%的性能下,读取延迟相对降低了99.07%~99.77%.展开更多
文摘Computerized data has become critical to the survival of an enterprise. Company must have a strategy forrecovering their data should data lose. RAID is a popular mechanism to offer good fault-tolerance. But RAID can notwork well when one more disks fail. In this paper, we present an efficient Network-based high High-Tolerance Distributed Shared Storage mechanism called FTDSS. FTDSS makes use of disk space of node in Network to build alarge public shared storage space. Users can read/write their file from/to the public storage space from node of net-work Physically File is stored in each node in form of data fragment or XOR verify fragment. Because of redundantXOR fragments, file is available even when two more nodes fail. FTDSS realize distant redundant srorage. At last.this paper use experiment to prove that FTDSS can offer high fault-tolerance and advanced performance.
文摘随着多媒体数据的爆发式增长,云端数据呈现出大规模多模态混合并存的特性.服务于数据分析的传统存储系统因为缺乏数据的语义管理而面临读取延时超长的挑战.针对图像和文本2种模态数据,在传统存储系统之上提出一种跨模态图文数据内容筛选存储机制(cross-modal image and text content sifting storage,CITCSS),用于提供大规模在线相似性内容筛选服务,从存储系统层面缓解数据分析时必须从存储中读出所有数据的读带宽压力.机制分为离线与在线2个阶段.离线阶段中,引入基于自监督的生成对抗式Hash方法,系统利用这种方法生成语义元数据.然后,将元数据注入独立的元数据空间.最后,根据相似性Hash码间汉明距离能够度量语义距离的特点,利用Neo4j图数据库构建Hash元数据图谱,并在语义图谱中建立Hash码与存储路径之间的映射.在线阶段中,用户发送与分析相关的图像或文本,存储系统首先转化数据为Hash码.然后,在筛选半径内通过Hash元数据图谱寻找相似节点,进而找到相似文件的底层存储路径返回筛选数据.实验结果表明,与传统语义存储系统相比,CITCSS在召回率超过98%的性能下,读取延迟相对降低了99.07%~99.77%.
