Mapper and Reducer

Those functions are mostly used to illustrate mappers and reducers in map/reduce languages.

sparkouille.fctmr.combiner (fctkey1, gen1, fctkey2, gen2, how = “inner”)

Joins (or combines) two generators. The function is written based on two reducers. The function is more efficient if the groups of the second ensemble gen2 are shorter as each of them will be held in memory.

sparkouille.fctmr.ffilter (fct, gen)

Filters out elements from a generator.

sparkouille.fctmr.mapper (fct, gen)

Applies function fct to a generator.

sparkouille.fctmr.reducer (fctkey, gen, asiter = True, sort = True)

Implements a reducer.

sparkouille.fctmr.take (gen, count = 5, skip = 0)

Skips and takes elements from a generator.