Function swh_graph::utils::sort::par_sort_arcs
source · pub fn par_sort_arcs<Item, Iter, F>(
temp_dir: &Path,
batch_size: usize,
iter: Iter,
num_partitions: usize,
f: F
) -> Result<Vec<Map<KMergeIters<impl Iterator<Item = (usize, usize, ())> + Clone + Send + Sync>, impl FnMut((usize, usize, ())) -> (usize, usize) + Clone + Send + Sync>>>where
F: Fn(&mut PartitionedBuffer, Item) -> Result<()> + Send + Sync,
Iter: ParallelIterator<Item = Item>,
Expand description
Given an iterator and a function to insert its items to [BatchIterator
]s, returns an
iterator of pairs.
f
gets as parameters num_partitions
BatchIterator
s; and should place its pair
in such a way that all (src, dst, ())
in partition n
should be lexicographically
lower than those in partition n+1
.
In orther words, f
writes in arbitrary order in each partition, but partitions
should be sorted with respect to each other. This allows merging partitions in
parallel after they are sorted.