Data processing of small samples is an important and valuable research problem in the electronic equipment test. Because it is difficult and complex to determine the probability distribution of small samples, it is di...Data processing of small samples is an important and valuable research problem in the electronic equipment test. Because it is difficult and complex to determine the probability distribution of small samples, it is difficult to use the traditional probability theory to process the samples and assess the degree of uncertainty. Using the grey relational theory and the norm theory, the grey distance information approach, which is based on the grey distance information quantity of a sample and the average grey distance information quantity of the samples, is proposed in this article. The definitions of the grey distance information quantity of a sample and the average grey distance information quantity of the samples, with their characteristics and algorithms, are introduced. The correlative problems, including the algorithm of estimated value, the standard deviation, and the acceptance and rejection criteria of the samples and estimated results, are also proposed. Moreover, the information whitening ratio is introduced to select the weight algorithm and to compare the different samples. Several examples are given to demonstrate the application of the proposed approach. The examples show that the proposed approach, which has no demand for the probability distribution of small samples, is feasible and effective.展开更多
针对利用海量数据构建分类模型时训练数据规模大、训练时间长且碳排放量大的问题,提出面向低能耗高性能的分类器两阶段数据选择方法TSDS(Two-Stage Data Selection)。首先,通过修正余弦相似度确定聚类中心,并将样本数据进行基于不相似...针对利用海量数据构建分类模型时训练数据规模大、训练时间长且碳排放量大的问题,提出面向低能耗高性能的分类器两阶段数据选择方法TSDS(Two-Stage Data Selection)。首先,通过修正余弦相似度确定聚类中心,并将样本数据进行基于不相似点的分裂层次聚类;其次,对聚类结果按数据分布自适应抽样以组成高质量的子样本集;最后,利用子样本集在分类模型上训练,在加速训练过程的同时提升模型精度。在Spambase、Bupa和Phoneme等6个数据集上构建支持向量机(SVM)和多层感知机(MLP)分类模型,验证TSDS的性能。实验结果表明在样本数据压缩比达到85.00%的情况下,TSDS能将分类模型准确率提升3~10个百分点,同时加速模型训练,使训练SVM分类器的能耗平均降低93.76%,训练MLP分类器的能耗平均降低75.41%。可见,TSDS在大数据场景的分类任务上既能缩短训练时间和减少能耗,又能提升分类器性能,从而助力实现“双碳”目标。展开更多
文摘Data processing of small samples is an important and valuable research problem in the electronic equipment test. Because it is difficult and complex to determine the probability distribution of small samples, it is difficult to use the traditional probability theory to process the samples and assess the degree of uncertainty. Using the grey relational theory and the norm theory, the grey distance information approach, which is based on the grey distance information quantity of a sample and the average grey distance information quantity of the samples, is proposed in this article. The definitions of the grey distance information quantity of a sample and the average grey distance information quantity of the samples, with their characteristics and algorithms, are introduced. The correlative problems, including the algorithm of estimated value, the standard deviation, and the acceptance and rejection criteria of the samples and estimated results, are also proposed. Moreover, the information whitening ratio is introduced to select the weight algorithm and to compare the different samples. Several examples are given to demonstrate the application of the proposed approach. The examples show that the proposed approach, which has no demand for the probability distribution of small samples, is feasible and effective.
文摘针对利用海量数据构建分类模型时训练数据规模大、训练时间长且碳排放量大的问题,提出面向低能耗高性能的分类器两阶段数据选择方法TSDS(Two-Stage Data Selection)。首先,通过修正余弦相似度确定聚类中心,并将样本数据进行基于不相似点的分裂层次聚类;其次,对聚类结果按数据分布自适应抽样以组成高质量的子样本集;最后,利用子样本集在分类模型上训练,在加速训练过程的同时提升模型精度。在Spambase、Bupa和Phoneme等6个数据集上构建支持向量机(SVM)和多层感知机(MLP)分类模型,验证TSDS的性能。实验结果表明在样本数据压缩比达到85.00%的情况下,TSDS能将分类模型准确率提升3~10个百分点,同时加速模型训练,使训练SVM分类器的能耗平均降低93.76%,训练MLP分类器的能耗平均降低75.41%。可见,TSDS在大数据场景的分类任务上既能缩短训练时间和减少能耗,又能提升分类器性能,从而助力实现“双碳”目标。
