nnsvs.train_util.batch_by_size

nnsvs.train_util.batch_by_size(indices, num_tokens_fn, max_tokens=None, max_sentences=None, required_batch_size_multiple=1)[source]

Yield mini-batches of indices bucketed by size. Batches may contain sequences of different lengths.

Parameters:

  • indices (List[int]) – ordered list of dataset indices

  • num_tokens_fn (callable) – function that returns the number of tokens at a given index

  • max_tokens (int, optional) – max number of tokens in each batch (default: None).

  • max_sentences (int, optional) – max number of sentences in each batch (default: None).

  • required_batch_size_multiple (int, optional) – require batch size to be a multiple of N (default: 1).