MossForCausalLM

class MossForCausalLM(config)

base_model_prefix = 'model'

classmethod pipeline_layers(config)

Get layers of pipeline.

返回:

list

static load_parallel_state_dict(path, config, process_exclusion=False, protocol='file', **kwargs)

Load state_dict from path.

The format of pretrained model should be the same as that of huggingface.

返回:

state_dict. Note that the state_dict should be processed properly to match the current rank.

training: bool

static save_parallel_state_dict(state_dict, path, config, process_exclusion=False, protocol='file')

Save state_dict to path.

The format of saved state dict should be the same as that of huggingface.