Reader

The Reader Class

The reader allows queryring the logs of several experiments and performing operations on the content of these logs (e.g. grouping and aggregation).

class mlxp.reader.Reader(src_dir: str, dst_dir: str | None = None, parser: ~mlxp.parser.Parser = <mlxp.parser.DefaultParser object>, refresh: bool = True)

Bases: object

A class for exploiting the results stored in several runs contained in a same parent directory ‘src_dir’.

Once, created, it is possible to query the database using the method ‘filter’ to get the results matching a specific configuration setting. The result of the query is returned either as a DataFrame object or a pandas dataframe. The queries are processed using a parser inheriting form the abstract class Parser. By default, the parser is DefaultParser. However, the user can provide a custom parser with a custom syntax inheriting from the class Parser.

src_dir: str

The absolute path of the source/parent directory containing logs of the runs. It must contain sub-directories ‘src_dir/log_id’, where log_id is the uniquely assigned id of a run.

dst_dir: str

The destination directory where the database containing the runs is created. By default it is set to the source directory ‘src_dir’. The user can select a different location for the database by setting the variable ‘dst_dir’ of the constructor to a different directory.

filter(query_string: str = '', result_format: str = 'datadict') → DataFrame | DataFrame

Search a query in a database of runs.

Parameters:

• query_string (str (default "")) – a string defining the query constaints.

• result_format (str (default False)) – format of the result (either a pandas dataframe or an object of type DataFrame). By default returns a DataFrame object.

Returns:

The result of a query either as a DataFrame or a pandas dataframe.

Return type:

Union[DataFrame,pd.DataFrame]

Raises:

SyntaxError – if the query string does not follow expected syntax.

property fields: DataFrame

Return all fields of the database except those specific to MLXP, excluding the fields contained in the file ‘mlxp.yaml’.

return: a dataframe of all fields contained in the database rtype: pd.DataFrame

property searchable: DataFrame

Return all fields of the database that are searchable, excluding the fields contained in the file ‘mlxp.yaml’.

return: a dataframe of all fields contained in the database that can be searched using the method filter rtype: pd.DataFrame

The Parser Module

Parser object for querying results stored by the reader.

class mlxp.parser.Parser

Bases: ABC

An abstract class for parsing queries.

Any parser used by the class Reader must inherit from this abstract class.

abstract parse(query: str) → QueryInstance

Parse a query string into a tinydb QueryInstance object.

Parameters:

query (str) – A query in the form of a string

Returns:

A instance of a QueryInstance class representing the query

Return type:

QueryInstance

Raises:

SyntaxError – if the query string does not follow expected syntax.

class mlxp.parser.DefaultParser

Bases: Parser

MLXP’s deafult parser inspired from python’s syntax.

parse(query: str) → QueryInstance

Parse a query string into a tinydb QueryInstance object.