Interface for using mmaped files with threads. More...
#include "paal/utils/type_functions.hpp"#include "paal/utils/irange.hpp"#include "paal/data_structures/thread_pool.hpp"#include <boost/iostreams/device/mapped_file.hpp>#include <string>#include <vector>#include <thread>Go to the source code of this file.
Classes | |
| class | paal::data_structures::mapped_file |
| data structure that gets new lines for many threads More... | |
Namespaces | |
| paal | |
| global namespace of project. | |
| paal::data_structures | |
| Data Structure namespace. | |
Functions | |
| template<typename Functor > | |
| auto | paal::data_structures::for_each_line (Functor f, std::string const &file_path, unsigned threads_count=std::thread::hardware_concurrency()) |
| for_every_line function provides basic functionality for processing text files quickly and clearly. Thanks to mmap() functionality it doesn't have to seek through file but it loads it to virtual memory instantly and uses only ram cache to do that. Furthermore file is split instantly - thanks to that it can be processed effectively using threads. Downside of using mmap is that this functionality will not work effectively if threads have small jobs to be done comparing reading the line charge. It's supposed to work with O(threads_count) memory usage but remember - RES (resident size) stands for how much memory of this process is loaded in physical memory, so file pages loaded in ram cache are added to that value. More... | |
Interface for using mmaped files with threads.
Definition in file mapped_file.hpp.
1.8.5