Local Dataset Classes

Here are the classes for local datasets.

class muspy.FolderDataset(root: Union[str, pathlib.Path], convert: bool = False, kind: str = 'json', n_jobs: int = 1, ignore_exceptions: bool = True, use_converted: Optional[bool] = None)[source]

A class of datasets containing files in a folder.

Two modes are available for this dataset. When the on-the-fly mode is enabled, a data sample is converted to a music object on the fly when being indexed. When the on-the-fly mode is disabled, a data sample is loaded from the precomputed converted data.

root

Root directory of the dataset.

Type:str or Path
Parameters:
  • convert (bool, optional) – Whether to convert the dataset to MusPy JSON/YAML files. If False, will check if converted data exists. If so, disable on-the-fly mode. If not, enable on-the-fly mode and warns. Defaults to False.
  • kind ({'json', 'yaml'}, optional) – File format to save the data. Defaults to ‘json’.
  • n_jobs (int, optional) – Maximum number of concurrently running jobs in multiprocessing. If equal to 1, disable multiprocessing. Defaults to 1.
  • ignore_exceptions (bool, optional) – Whether to ignore errors and skip failed conversions. This can be helpful if some of the source files is known to be corrupted. Defaults to True.
  • use_converted (bool, optional) – Force to disable on-the-fly mode and use stored converted data

Important

muspy.FolderDataset.converted_exists() depends solely on a special file named .muspy.success in the folder {root}/_converted/, which serves as an indicator for the existence and integrity of the converted dataset. If the converted dataset is built by muspy.FolderDataset.convert(), the .muspy.success file will be created as well. If the converted dataset is created manually, make sure to create the .muspy.success file in the folder {root}/_converted/ to prevent errors.

Notes

This class is extended from muspy.Dataset. To build a custom dataset based on this class, please refer to muspy.Dataset for the docmentation of the methods __getitem__ and __len__, and the class attribute _info.

In addition, the attribute _extension and method read should be properly set. _extension is the extension to look for when building the dataset. All files with the given extension will be included as source files. read is a callable that takes as inputs a filename of a source file and return the converted Music object.

See also

muspy.Dataset
The base class for all MusPy datasets.
convert(kind: str = 'json', n_jobs: int = 1, ignore_exceptions: bool = True) → FolderDatasetType[source]

Convert and save the Music objects.

The converted files will be named by its index and saved to root/_converted. The original filenames can be found in the filenames attribute. For example, the file at filenames[i] will be converted and saved to {i}.json.

Parameters:
  • kind ({'json', 'yaml'}, optional) – File format to save the data. Defaults to ‘json’.
  • n_jobs (int, optional) – Maximum number of concurrently running jobs in multiprocessing. If equal to 1, disable multiprocessing. Defaults to 1.
  • ignore_exceptions (bool, optional) – Whether to ignore errors and skip failed conversions. This can be helpful if some of the source files is known to be corrupted. Defaults to True.
Returns:
Return type:

Object itself.
converted_dir

Return the path to the root directory of the converted dataset.

converted_exists() → bool[source]

Return True if the saved dataset exists, otherwise False.

exists() → bool[source]

Return True if the dataset exists, otherwise False.

load(filename: Union[str, pathlib.Path]) → muspy.music.Music[source]

Read a file into a Music object.

on_the_fly() → FolderDatasetType[source]

Enable on-the-fly mode and convert the data on the fly.

Returns:
Return type:Object itself.
read(filename: Any) → muspy.music.Music[source]

Read a file into a Music object.

use_converted() → FolderDatasetType[source]

Disable on-the-fly mode and use converted data.

Returns:
Return type:Object itself.
class muspy.MusicDataset(root: Union[str, pathlib.Path], kind: str = 'json')[source]

A local dataset containing MusPy JSON/YAML files in a folder.

root

Root directory of the dataset.

Type:str or Path
kind

File format of the data. Defaults to ‘json’.

Type:{‘json’, ‘yaml’}, optional
class muspy.ABCFolderDataset(root: Union[str, pathlib.Path], convert: bool = False, kind: str = 'json', n_jobs: int = 1, ignore_exceptions: bool = True, use_converted: Optional[bool] = None)[source]

A class of local datasets containing ABC files in a folder.

on_the_fly() → FolderDatasetType[source]

Enable on-the-fly mode and convert the data on the fly.

Returns:
Return type:Object itself.
read(filename: Tuple[str, Tuple[int, int]]) → muspy.music.Music[source]

Read a file into a Music object.