mechmine - data mining searches for the right information
Data Mining is a technique to search for valuable information hidden in data. Most commonly used are statistical methods which detect patterns or properties in the data. One can process huge data-series in a timely manner, which otherwise cannot be analysed manually in reasonable time. Typical methods and algorithms are based on decision-trees, artificial neural networks (ANN), support vector machines (SVM), genetic algorithms, discriminant or rule-based analysis. The data mining process happens in stages, from feature selection, extraction, classification to post-processing of the results.
The quality and performance of any data science technique depends significantly on the pre-processing of the received data. Any information lost during this stage can never again be retrieved. Data pre-processing is still necessary today despite its criticality to the performance, because typical data-series are too big to be fed into most algorithms. This means that the data must be somehow reduced. Traditional methods of pre-processing can be based on the Fast Fourier Transform (FFT) or the Principal Component Analysis (PCA). However, through the parallel application of various methods, paired with intelligent selection criteria, the performance can be improved.
