
Analysis Technology for Large-scale Log Database
Centralized management of large-scale administrative records.
Now possible to construct optimum systems depending on data quantity.
Overview
It is now possible to store large quantities of administrative records (logs) generated by various kinds of information systems, in order to preserve evidence. Mitsubishi Electric has developed technology for integrated management and rapid collection, search and analysis of huge logs in a variety of formats through the high-speed processing of large-scale data.
Technology Point
Using data-driven parallel processing, which can maximize the use of a large number of processors and disks, we have enabled improved performance to meet increases in the quantity of data.
Using blocked transposed data placement that takes into account the characteristic features of storage tasks, large-scale data that exceeds memory storage capacity can be processed stably and rapidly.
Using the sDFA character string matching algorithm, which can collate 100 million characters a second even with complex retrieval conditions, it is now possible to judge the format of, and rapidly process the search of a variety of log formats.
Using the multi-stream RID data compression algorithm, storage capacity can be cut to less than 1/10 and high-speed data searching is possible.




