module benchhelper.benchmark

Short summary

module pyquickhelper.benchhelper.benchmark

Helpers to benchmark something

source on GitHub

Classes

class	truncated documentation
BenchMark	Class to help benchmarking. You should overwrite method init, bench, end, graphs.

Properties

property	truncated documentation
Appendix	Returns the metrics.
Graphs	Returns images of graphs.
Metadata	Returns the metrics.
Metrics	Returns the metrics.
Name	Returns the name of the benchmark.

Methods

method	truncated documentation
__init__
_convert	Converts a value knowing its column, its type into something readable.
bench	Runs the benchmark. Overwrite this method.
end	Cleans. Overwrites this method.
fLOG	Logs something.
graphs	Builds graphs after the benchmark was run.
init	Initialisation. Overwrite this method.
meta_to_df	Converts meta data into a dataframe
report	Produces a report.
run	Runs the benchmark.
to_df	Converts the metrics into a dataframe.
uncache	overwrite this method to uncache some previous run

Documentation

Helpers to benchmark something

source on GitHub

class pyquickhelper.benchhelper.benchmark.BenchMark(name, clog=None, fLOG=<function noLOG>, path_to_images='.', cache_file=None, pickle_module=None, progressbar=None, **params)

Bases: object

Class to help benchmarking. You should overwrite method init, bench, end, graphs.

source on GitHub

Parameters:

• name – name of the test

• clog – CustomLog or string

• fLOG – logging function

• params – extra parameters

• path_to_images – path to images

• cache_file – cache file

• pickle_module – pickle or dill if you need to serialize functions

• progressbar – relies on tqdm, example tnrange

If cache_file is specified, the class will store the results of the method bench. On a second run, the function load the cache and run modified or new run (in params_list).

source on GitHub

property Appendix

Returns the metrics.

source on GitHub

property Graphs

Returns images of graphs.

source on GitHub

class LocalGraph(func_gen, filename=None, title=None, root=None)

Bases: object

Information about graphs.

source on GitHub

Parameters:

• func_gen – function generating the graph

• filename – filename

• title – title

• root – path should be relative to this one

source on GitHub

__init__(func_gen, filename=None, title=None, root=None)

Parameters:

• func_gen – function generating the graph

• filename – filename

• title – title

• root – path should be relative to this one

source on GitHub

add(name, value)

Adds an attribute.

Parameters:

• name – name of the attribute

• value – value

source on GitHub

plot(ax=None, text=True, **kwargs)

Draws the graph again.

Parameters:

• ax – axis

• text – add text on the graph

• kwargs – additional parameters

Returns:

axis

source on GitHub

to_html()

Renders as HTML.

source on GitHub

to_rst()

Renders as :ekg:`rst`.

source on GitHub

property Metadata

Returns the metrics.

source on GitHub

property Metrics

Returns the metrics.

source on GitHub

property Name

Returns the name of the benchmark.

source on GitHub

__init__(name, clog=None, fLOG=<function noLOG>, path_to_images='.', cache_file=None, pickle_module=None, progressbar=None, **params)

Parameters:

• name – name of the test

• clog – CustomLog or string

• fLOG – logging function

• params – extra parameters

• path_to_images – path to images

• cache_file – cache file

• pickle_module – pickle or dill if you need to serialize functions

• progressbar – relies on tqdm, example tnrange

If cache_file is specified, the class will store the results of the method bench. On a second run, the function load the cache and run modified or new run (in params_list).

source on GitHub

_convert(df, i, col, ty, value)

Converts a value knowing its column, its type into something readable.

Parameters:

• df – dataframe

• i – line index

• col – column name

• ty – type

• value – value to convert

Returns:

value

source on GitHub

bench(**params)

Runs the benchmark. Overwrite this method.

Parameters:

params – parameters

Returns:

metrics as a dictionary, appendix as a dictionary

The results of this method will be cached if a cache_file was specified in the constructor.

source on GitHub

end()

Cleans. Overwrites this method.

source on GitHub

fLOG(*args, **kwargs)

Logs something.

source on GitHub

graphs(path_to_images)

Builds graphs after the benchmark was run.

Parameters:

path_to_images – path to images

Returns:

a list of LocalGraph

Every returned graph must contain a function which creates the graph. The function must accepts two parameters ax and text. Example:

def local_graph(ax=None, text=True, figsize=(5,5)):
    vx = ...
    vy = ...
    btrys = set(df["_btry"])
    ymin = df[vy].min()
    ymax = df[vy].max()
    decy = (ymax - ymin) / 50
    colors = cm.rainbow(numpy.linspace(0, 1, len(btrys)))
    if len(btrys) == 0:
        raise ValueError("The benchmark is empty.")
    if ax is None:
        fig, ax = plt.subplots(1, 1, figsize=figsize)
        ax.grid(True)
    for i, btry in enumerate(sorted(btrys)):
        subset = df[df["_btry"]==btry]
        if subset.shape[0] > 0:
            subset.plot(x=vx, y=vy, kind="scatter", label=btry, ax=ax, color=colors[i])
        if text:
            tx = subset[vx].mean()
            ty = subset[vy].mean()
            ax.text(tx, ty + decy, btry, size='small',
                    color=colors[i], ha='center', va='bottom')
    ax.set_xlabel(vx)
    ax.set_ylabel(vy)
    return ax

source on GitHub

init()

Initialisation. Overwrite this method.

source on GitHub

meta_to_df(convert=False, add_link=False, format='html')

Converts meta data into a dataframe

Parameters:

• convert – if True, calls method _convert on each cell

• add_link – add hyperlink

• format – format for hyperlinks (html or rst)

Returns:

dataframe

source on GitHub

report(css=None, template_html=None, template_rst=None, engine='mako', filecsv=None, filehtml=None, filerst=None, params_html=None, title=None, description=None)

Produces a report.

Parameters:

• css – css (will take the default one if empty)

• template_html – template HTML (mako or jinja2)

• template_rst – template RST (mako or jinja2)

• engine – 'mako’ or ‘jinja2'

• filehtml – report will written in this file if not None

• filecsv – metrics will be written as a flat table

• filerst – metrics will be written as a RST table

• params_html – parameter to send to function pandas.DataFrame.to_html

• title – title (Name if any)

• description – add a description

Returns:

dictionary {format: content}

You can define your own template by looking into the default ones defines in this class (see the bottom of this file). By default, HTML and RST report are generated.

source on GitHub

run(params_list)

Runs the benchmark.

Parameters:

params_list – list of dictionaries

source on GitHub

to_df(convert=False, add_link=False, format='html')

Converts the metrics into a dataframe.

Parameters:

• convert – if True, calls method _convert on each cell

• add_link – add hyperlink

• format – format for hyperlinks (html or rst)

Returns:

dataframe

source on GitHub

uncache(cache)

overwrite this method to uncache some previous run

source on GitHub