Algorithm for Mining Frequent Patterns in Big Data Based on Hadoop

Aiming at the traditional frequent pattern mining algorithm can not meet the needs of mining in big data environment, a parallel algorithm for efficiently mining frequent patterns in large databases is proposed. Firstly, PrePost algorithm is improved from compressing database, simplifying data representation and using efficient connection and pruning strategy, which improve the efficiency of mining in stand-alone mode. Then, the improved algorithm is migrated to the Hadoop platform and the MapReduce model is used for parallel computing. A load balancing strategy is proposed to ensure the efficient operation of the cluster. Finally, the frequent pattern mining is evaluated using kulczynski metric and unbalance ratio to ensure that the mining pattern has practical value. Experimental results show that this algorithm can effectively mine the frequent patterns in big data sets.