deepmd.utils package

Submodules

deepmd.utils.argcheck module

class deepmd.utils.argcheck.ArgsPlugin[source]

Bases: object

Methods

get_all_argument([exclude_hybrid])

Get all arguments.

register(name[, alias, doc])

Register a descriptor argument plugin.

get_all_argument(exclude_hybrid: bool = False) List[Argument][source]

Get all arguments.

Parameters
exclude_hybridbool

exclude hybrid descriptor to prevent circular calls

Returns
List[Argument]

all arguments

register(name: str, alias: Optional[List[str]] = None, doc: str = '') Callable[[], List[Argument]][source]

Register a descriptor argument plugin.

Parameters
namestr

the name of a descriptor

aliasList[str], optional

the list of aliases of this descriptor

Returns
Callable[[], List[Argument]]

the registered descriptor argument method

Examples

>>> some_plugin = ArgsPlugin()
>>> @some_plugin.register("some_descrpt")
    def descrpt_some_descrpt_args():
        return []
deepmd.utils.argcheck.descrpt_dpa2_args()[source]
deepmd.utils.argcheck.descrpt_hybrid_args()[source]
deepmd.utils.argcheck.descrpt_local_frame_args()[source]
deepmd.utils.argcheck.descrpt_se_a_args()[source]
deepmd.utils.argcheck.descrpt_se_a_ebd_v2_args()[source]
deepmd.utils.argcheck.descrpt_se_a_mask_args()[source]
deepmd.utils.argcheck.descrpt_se_a_tpe_args()[source]
deepmd.utils.argcheck.descrpt_se_atten_args()[source]
deepmd.utils.argcheck.descrpt_se_atten_common_args()[source]
deepmd.utils.argcheck.descrpt_se_atten_v2_args()[source]
deepmd.utils.argcheck.descrpt_se_r_args()[source]
deepmd.utils.argcheck.descrpt_se_t_args()[source]
deepmd.utils.argcheck.descrpt_variant_type_args(exclude_hybrid: bool = False) Variant[source]
deepmd.utils.argcheck.fitting_dipole()[source]
deepmd.utils.argcheck.fitting_dos()[source]
deepmd.utils.argcheck.fitting_ener()[source]
deepmd.utils.argcheck.fitting_polar()[source]
deepmd.utils.argcheck.fitting_variant_type_args()[source]
deepmd.utils.argcheck.frozen_model_args() Argument[source]
deepmd.utils.argcheck.gen_args(**kwargs) List[Argument][source]
deepmd.utils.argcheck.gen_doc(*, make_anchor=True, make_link=True, **kwargs)[source]
deepmd.utils.argcheck.gen_json(**kwargs)[source]
deepmd.utils.argcheck.learning_rate_args()[source]
deepmd.utils.argcheck.learning_rate_dict_args()[source]
deepmd.utils.argcheck.learning_rate_exp()[source]
deepmd.utils.argcheck.learning_rate_variant_type_args()[source]
deepmd.utils.argcheck.limit_pref(item)[source]
deepmd.utils.argcheck.linear_ener_model_args() Argument[source]
deepmd.utils.argcheck.list_to_doc(xx)[source]
deepmd.utils.argcheck.loss_args()[source]
deepmd.utils.argcheck.loss_dict_args()[source]
deepmd.utils.argcheck.loss_dos()[source]
deepmd.utils.argcheck.loss_ener()[source]
deepmd.utils.argcheck.loss_ener_spin()[source]
deepmd.utils.argcheck.loss_tensor()[source]
deepmd.utils.argcheck.loss_variant_type_args()[source]
deepmd.utils.argcheck.make_index(keys)[source]
deepmd.utils.argcheck.mixed_precision_args()[source]
deepmd.utils.argcheck.model_args(exclude_hybrid=False)[source]
deepmd.utils.argcheck.model_compression()[source]
deepmd.utils.argcheck.model_compression_type_args()[source]
deepmd.utils.argcheck.modifier_dipole_charge()[source]
deepmd.utils.argcheck.modifier_variant_type_args()[source]
deepmd.utils.argcheck.multi_model_args() Argument[source]
deepmd.utils.argcheck.normalize(data)[source]
deepmd.utils.argcheck.normalize_data_dict(data_dict)[source]
deepmd.utils.argcheck.normalize_fitting_net_dict(fitting_net_dict)[source]
deepmd.utils.argcheck.normalize_fitting_weight(fitting_keys, data_keys, fitting_weight=None)[source]
deepmd.utils.argcheck.normalize_learning_rate_dict(fitting_keys, learning_rate_dict)[source]
deepmd.utils.argcheck.normalize_learning_rate_dict_with_single_learning_rate(fitting_keys, learning_rate)[source]
deepmd.utils.argcheck.normalize_loss_dict(fitting_keys, loss_dict)[source]
deepmd.utils.argcheck.normalize_multi_task(data)[source]
deepmd.utils.argcheck.pairtab_model_args() Argument[source]
deepmd.utils.argcheck.pairwise_dprc() Argument[source]
deepmd.utils.argcheck.spin_args()[source]
deepmd.utils.argcheck.standard_model_args() Argument[source]
deepmd.utils.argcheck.start_pref(item, label=None, abbr=None)[source]
deepmd.utils.argcheck.training_args()[source]
deepmd.utils.argcheck.training_data_args()[source]
deepmd.utils.argcheck.type_embedding_args()[source]
deepmd.utils.argcheck.validation_data_args()[source]

deepmd.utils.argcheck_nvnmd module

deepmd.utils.argcheck_nvnmd.nvnmd_args()[source]

deepmd.utils.batch_size module

class deepmd.utils.batch_size.AutoBatchSize(initial_batch_size: int = 1024, factor: float = 2.0)[source]

Bases: ABC

This class allows DeePMD-kit to automatically decide the maximum batch size that will not cause an OOM error.

Parameters
initial_batch_sizeint, default: 1024

initial batch size (number of total atoms) when DP_INFER_BATCH_SIZE is not set

factorfloat, default: 2.

increased factor

Notes

In some CPU environments, the program may be directly killed when OOM. In this case, by default the batch size will not be increased for CPUs. The environment variable DP_INFER_BATCH_SIZE can be set as the batch size.

In other cases, we assume all OOM error will raise OutOfMemoryError.

Attributes
current_batch_sizeint

current batch size (number of total atoms)

maximum_working_batch_sizeint

maximum working batch size

minimal_not_working_batch_sizeint

minimal not working batch size

Methods

execute(callable, start_index, natoms)

Excuate a method with given batch size.

execute_all(callable, total_size, natoms, ...)

Excuate a method with all given data.

is_gpu_available()

Check if GPU is available.

is_oom_error(e)

Check if the exception is an OOM error.

execute(callable: Callable, start_index: int, natoms: int) Tuple[int, tuple][source]

Excuate a method with given batch size.

Parameters
callableCallable

The method should accept the batch size and start_index as parameters, and returns executed batch size and data.

start_indexint

start index

natomsint

natoms

Returns
int

executed batch size * number of atoms

tuple

result from callable, None if failing to execute

Raises
OutOfMemoryError

OOM when batch size is 1

execute_all(callable: Callable, total_size: int, natoms: int, *args, **kwargs) Tuple[ndarray][source]

Excuate a method with all given data.

Parameters
callableCallable

The method should accept *args and **kwargs as input and return the similiar array.

total_sizeint

Total size

natomsint

The number of atoms

*args

Variable length argument list.

**kwargs

If 2D np.ndarray, assume the first axis is batch; otherwise do nothing.

abstract is_gpu_available() bool[source]

Check if GPU is available.

Returns
bool

True if GPU is available

abstract is_oom_error(e: Exception) bool[source]

Check if the exception is an OOM error.

Parameters
eException

Exception

Returns
bool

True if the exception is an OOM error

deepmd.utils.compat module

Module providing compatibility between 0.x.x and 1.x.x input versions.

deepmd.utils.compat.convert_input_v0_v1(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any][source]

Convert input from v0 format to v1.

Parameters
jdataDict[str, Any]

loaded json/yaml file

warningbool, optional

whether to show deprecation warning, by default True

dumpOptional[Union[str, Path]], optional

whether to dump converted file, by default None

Returns
Dict[str, Any]

converted output

deepmd.utils.compat.convert_input_v1_v2(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any][source]
deepmd.utils.compat.deprecate_numb_test(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any][source]

Deprecate numb_test since v2.1. It has taken no effect since v2.0.

See #1243.

Parameters
jdataDict[str, Any]

loaded json/yaml file

warningbool, optional

whether to show deprecation warning, by default True

dumpOptional[Union[str, Path]], optional

whether to dump converted file, by default None

Returns
Dict[str, Any]

converted output

deepmd.utils.compat.remove_decay_rate(jdata: Dict[str, Any])[source]

Convert decay_rate to stop_lr.

Parameters
jdataDict[str, Any]

input data

deepmd.utils.compat.update_deepmd_input(jdata: Dict[str, Any], warning: bool = True, dump: Optional[Union[str, Path]] = None) Dict[str, Any][source]

deepmd.utils.data module

class deepmd.utils.data.DataRequirementItem(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None, output_natoms_for_type_sel: bool = False)[source]

Bases: object

A class to store the data requirement for data systems.

Parameters
key

The key of the item. The corresponding data is stored in sys_path/set.*/key.npy

ndof

The number of dof

atomic

The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof

must

The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0

high_prec

Load the data and store in float64, otherwise in float32

type_sel

Select certain type of atoms

repeat

The data will be repeated repeat times.

defaultfloat, default=0.

default value of data

dtypenp.dtype, optional

the dtype of data, overwrites high_prec if provided

output_natoms_for_type_selbool, optional

if True and type_sel is True, the atomic dimension will be natoms instead of nsel

Methods

to_dict

to_dict() dict[source]
class deepmd.utils.data.DeepmdData(sys_path: str, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set: bool = False, sort_atoms: bool = True)[source]

Bases: object

Class for a data system.

It loads data from hard disk, and mantains the data as a data_dict

Parameters
sys_path

Path to the data system

set_prefix

Prefix for the directories of different sets

shuffle_test

If the test data are shuffled

type_map

Gives the name of different atom types

optional_type_map

If the type_map.raw in each system is optional

modifier

Data modifier that has the method modify_data

trn_all_set

Use all sets as training dataset. Otherwise, if the number of sets is more than 1, the last set is left for test.

sort_atomsbool

Sort atoms by atom types. Required to enable when the data is directly feeded to descriptors except mixed types.

Methods

add(key, ndof[, atomic, must, high_prec, ...])

Add a data item that to be loaded.

avg(key)

Return the average value of an item.

check_batch_size(batch_size)

Check if the system can get a batch of data with batch_size frames.

check_test_size(test_size)

Check if the system can get a test dataset with test_size frames.

get_atom_type()

Get atom types.

get_batch(batch_size)

Get a batch of data with batch_size frames.

get_data_dict()

Get the data_dict.

get_item_torch(index)

Get a single frame data .

get_natoms()

Get number of atoms.

get_natoms_vec(ntypes)

Get number of atoms and number of atoms in different types.

get_ntypes()

Number of atom types in the system.

get_numb_batch(batch_size, set_idx)

Get the number of batches in a set.

get_numb_set()

Get number of training sets.

get_sys_numb_batch(batch_size)

Get the number of batches in the data system.

get_test([ntests])

Get the test data with ntests frames.

get_type_map()

Get the type map.

reduce(key_out, key_in)

Generate a new item from the reduction of another atom.

reformat_data_torch(data)

Modify the data format for the requirements of Torch backend.

reset_get_batch

add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None, output_natoms_for_type_sel: bool = False)[source]

Add a data item that to be loaded.

Parameters
key

The key of the item. The corresponding data is stored in sys_path/set.*/key.npy

ndof

The number of dof

atomic

The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof

must

The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0

high_prec

Load the data and store in float64, otherwise in float32

type_sel

Select certain type of atoms

repeat

The data will be repeated repeat times.

defaultfloat, default=0.

default value of data

dtypenp.dtype, optional

the dtype of data, overwrites high_prec if provided

output_natoms_for_type_selbool, optional

if True and type_sel is True, the atomic dimension will be natoms instead of nsel

avg(key)[source]

Return the average value of an item.

check_batch_size(batch_size)[source]

Check if the system can get a batch of data with batch_size frames.

check_test_size(test_size)[source]

Check if the system can get a test dataset with test_size frames.

get_atom_type() List[int][source]

Get atom types.

get_batch(batch_size: int) dict[source]

Get a batch of data with batch_size frames. The frames are randomly picked from the data system.

Parameters
batch_size

size of the batch

get_data_dict() dict[source]

Get the data_dict.

get_item_torch(index: int) dict[source]

Get a single frame data . The frame is picked from the data system by index. The index is coded across all the sets.

Parameters
index

index of the frame

get_natoms()[source]

Get number of atoms.

get_natoms_vec(ntypes: int)[source]

Get number of atoms and number of atoms in different types.

Parameters
ntypes

Number of types (may be larger than the actual number of types in the system).

Returns
natoms

natoms[0]: number of local atoms natoms[1]: total number of atoms held by this processor natoms[i]: 2 <= i < Ntypes+2, number of type i atoms

get_ntypes() int[source]

Number of atom types in the system.

get_numb_batch(batch_size: int, set_idx: int) int[source]

Get the number of batches in a set.

get_numb_set() int[source]

Get number of training sets.

get_sys_numb_batch(batch_size: int) int[source]

Get the number of batches in the data system.

get_test(ntests: int = -1) dict[source]

Get the test data with ntests frames.

Parameters
ntests

Size of the test data set. If ntests is -1, all test data will be get.

get_type_map() List[str][source]

Get the type map.

reduce(key_out: str, key_in: str)[source]

Generate a new item from the reduction of another atom.

Parameters
key_out

The name of the reduced item

key_in

The name of the data item to be reduced

reformat_data_torch(data)[source]

Modify the data format for the requirements of Torch backend.

Parameters
data

original data

reset_get_batch()[source]

deepmd.utils.data_system module

class deepmd.utils.data_system.DeepmdDataSystem(systems: List[str], batch_size: int, test_size: int, rcut: Optional[float] = None, set_prefix: str = 'set', shuffle_test: bool = True, type_map: Optional[List[str]] = None, optional_type_map: bool = True, modifier=None, trn_all_set=False, sys_probs=None, auto_prob_style='prob_sys_size', sort_atoms: bool = True)[source]

Bases: object

Class for manipulating many data systems.

It is implemented with the help of DeepmdData

Attributes
default_mesh

Mesh for each system.

Methods

add(key, ndof[, atomic, must, high_prec, ...])

Add a data item that to be loaded.

add_dict(adict)

Add items to the data system by a dict.

get_batch([sys_idx])

Get a batch of data from the data systems.

get_batch_mixed()

Get a batch of data from the data systems in the mixed way.

get_batch_size()

Get the batch size.

get_batch_standard([sys_idx])

Get a batch of data from the data systems in the standard way.

get_nbatches()

Get the total number of batches.

get_nsystems()

Get the number of data systems.

get_ntypes()

Get the number of types.

get_sys(idx)

Get a certain data system.

get_sys_ntest([sys_idx])

Get number of tests for the currently selected system, or one defined by sys_idx.

get_test([sys_idx, n_test])

Get test data from the the data systems.

get_type_map()

Get the type map.

reduce(key_out, key_in)

Generate a new item from the reduction of another atom.

compute_energy_shift

get_data_dict

print_summary

set_sys_probs

add(key: str, ndof: int, atomic: bool = False, must: bool = False, high_prec: bool = False, type_sel: Optional[List[int]] = None, repeat: int = 1, default: float = 0.0, dtype: Optional[dtype] = None, output_natoms_for_type_sel: bool = False)[source]

Add a data item that to be loaded.

Parameters
key

The key of the item. The corresponding data is stored in sys_path/set.*/key.npy

ndof

The number of dof

atomic

The item is an atomic property. If False, the size of the data should be nframes x ndof If True, the size of data should be nframes x natoms x ndof

must

The data file sys_path/set.*/key.npy must exist. If must is False and the data file does not exist, the data_dict[find_key] is set to 0.0

high_prec

Load the data and store in float64, otherwise in float32

type_sel

Select certain type of atoms

repeat

The data will be repeated repeat times.

default, default=0.

Default value of data

dtype

The dtype of data, overwrites high_prec if provided

output_natoms_for_type_selbool

If True and type_sel is True, the atomic dimension will be natoms instead of nsel

add_dict(adict: dict) None[source]

Add items to the data system by a dict. adict should have items like .. code-block:: python.

adict[key] = {

“ndof”: ndof, “atomic”: atomic, “must”: must, “high_prec”: high_prec, “type_sel”: type_sel, “repeat”: repeat,

}

For the explaination of the keys see add

compute_energy_shift(rcond=None, key='energy')[source]
property default_mesh: List[ndarray]

Mesh for each system.

get_batch(sys_idx: Optional[int] = None) dict[source]

Get a batch of data from the data systems.

Parameters
sys_idxint

The index of system from which the batch is get. If sys_idx is not None, sys_probs and auto_prob_style are ignored If sys_idx is None, automatically determine the system according to sys_probs or auto_prob_style, see the following. This option does not work for mixed systems.

Returns
dict

The batch data

get_batch_mixed() dict[source]

Get a batch of data from the data systems in the mixed way.

Returns
dict

The batch data

get_batch_size() int[source]

Get the batch size.

get_batch_standard(sys_idx: Optional[int] = None) dict[source]

Get a batch of data from the data systems in the standard way.

Parameters
sys_idxint

The index of system from which the batch is get. If sys_idx is not None, sys_probs and auto_prob_style are ignored If sys_idx is None, automatically determine the system according to sys_probs or auto_prob_style, see the following.

Returns
dict

The batch data

get_data_dict(ii: int = 0) dict[source]
get_nbatches() int[source]

Get the total number of batches.

get_nsystems() int[source]

Get the number of data systems.

get_ntypes() int[source]

Get the number of types.

get_sys(idx: int) DeepmdData[source]

Get a certain data system.

get_sys_ntest(sys_idx=None)[source]

Get number of tests for the currently selected system, or one defined by sys_idx.

get_test(sys_idx: Optional[int] = None, n_test: int = -1)[source]

Get test data from the the data systems.

Parameters
sys_idx

The test dat of system with index sys_idx will be returned. If is None, the currently selected system will be returned.

n_test

Number of test data. If set to -1 all test data will be get.

get_type_map() List[str][source]

Get the type map.

print_summary(name: str)[source]
reduce(key_out, key_in)[source]

Generate a new item from the reduction of another atom.

Parameters
key_out

The name of the reduced item

key_in

The name of the data item to be reduced

set_sys_probs(sys_probs=None, auto_prob_style: str = 'prob_sys_size')[source]
deepmd.utils.data_system.get_data(jdata: Dict[str, Any], rcut, type_map, modifier, multi_task_mode=False) DeepmdDataSystem[source]

Get the data system.

Parameters
jdata

The json data

rcut

The cut-off radius, not used

type_map

The type map

modifier

The data modifier

multi_task_mode

If in multi task mode

Returns
DeepmdDataSystem

The data system

deepmd.utils.data_system.print_summary(name: str, nsystems: int, system_dirs: List[str], natoms: List[int], batch_size: List[int], nbatches: List[int], sys_probs: List[float], pbc: List[bool])[source]

Print summary of systems.

Parameters
namestr

The name of the system

nsystemsint

The number of systems

system_dirslist of str

The directories of the systems

natomslist of int

The number of atoms

batch_sizelist of int

The batch size

nbatcheslist of int

The number of batches

sys_probslist of float

The probabilities

pbclist of bool

The periodic boundary conditions

deepmd.utils.data_system.prob_sys_size_ext(keywords, nsystems, nbatch)[source]
deepmd.utils.data_system.process_sys_probs(sys_probs, nbatch)[source]
deepmd.utils.data_system.process_systems(systems: Union[str, List[str]]) List[str][source]

Process the user-input systems.

If it is a single directory, search for all the systems in the directory. Check if the systems are valid.

Parameters
systemsstr or list of str

The user-input systems

Returns
list of str

The valid systems

deepmd.utils.econf_embd module

deepmd.utils.env_mat_stat module

class deepmd.utils.env_mat_stat.EnvMatStat[source]

Bases: ABC

A base class to store and calculate the statistics of the environment matrix.

Methods

compute_stats(data)

Compute the statistics of the environment matrix.

get_avg([default])

Get the average of the environment matrix.

get_std([default, protection])

Get the standard deviation of the environment matrix.

iter(data)

Get the iterator of the environment matrix.

load_or_compute_stats(data[, path])

Load the statistics of the environment matrix if it exists, otherwise compute and save it.

load_stats(path)

Load the statistics of the environment matrix.

save_stats(path)

Save the statistics of the environment matrix.

compute_stats(data: List[Dict[str, ndarray]]) None[source]

Compute the statistics of the environment matrix.

Parameters
dataList[Dict[str, np.ndarray]]

The environment matrix.

get_avg(default: float = 0) Dict[str, float][source]

Get the average of the environment matrix.

Parameters
defaultfloat, optional

The default value of the average, by default 0.

Returns
Dict[str, float]

The average of the environment matrix.

get_std(default: float = 0.1, protection: float = 0.01) Dict[str, float][source]

Get the standard deviation of the environment matrix.

Parameters
defaultfloat, optional

The default value of the standard deviation, by default 1e-1.

protectionfloat, optional

The protection value for the standard deviation, by default 1e-2.

Returns
Dict[str, float]

The standard deviation of the environment matrix.

abstract iter(data: List[Dict[str, ndarray]]) Iterator[Dict[str, StatItem]][source]

Get the iterator of the environment matrix.

Parameters
dataList[Dict[str, np.ndarray]]

The environment matrix.

Yields
Dict[str, StatItem]

The statistics of the environment matrix.

load_or_compute_stats(data: List[Dict[str, ndarray]], path: Optional[DPPath] = None) None[source]

Load the statistics of the environment matrix if it exists, otherwise compute and save it.

Parameters
pathDPH5Path

The path to load the statistics of the environment matrix.

dataList[Dict[str, np.ndarray]]

The environment matrix.

load_stats(path: DPPath) None[source]

Load the statistics of the environment matrix.

Parameters
pathDPH5Path

The path to load the statistics of the environment matrix.

save_stats(path: DPPath) None[source]

Save the statistics of the environment matrix.

Parameters
pathDPH5Path

The path to save the statistics of the environment matrix.

class deepmd.utils.env_mat_stat.StatItem(number: int = 0, sum: float = 0, squared_sum: float = 0)[source]

Bases: object

A class to store the statistics of the environment matrix.

Parameters
numberint

The total size of given array.

sumfloat

The sum value of the matrix.

squared_sumfloat

The sum squared value of the matrix.

Methods

compute_avg([default])

Compute the average of the environment matrix.

compute_std([default, protection])

Compute the standard deviation of the environment matrix.

compute_avg(default: float = 0) float[source]

Compute the average of the environment matrix.

Parameters
defaultfloat, optional

The default value of the average, by default 0.

Returns
float

The average of the environment matrix.

compute_std(default: float = 0.1, protection: float = 0.01) float[source]

Compute the standard deviation of the environment matrix.

Parameters
defaultfloat, optional

The default value of the standard deviation, by default 1e-1.

protectionfloat, optional

The protection value for the standard deviation, by default 1e-2.

Returns
float

The standard deviation of the environment matrix.

deepmd.utils.errors module

exception deepmd.utils.errors.OutOfMemoryError[source]

Bases: Exception

This error is caused by out-of-memory (OOM).

deepmd.utils.finetune module

deepmd.utils.finetune.change_energy_bias_lower(data: DeepmdDataSystem, dp: DeepEval, origin_type_map: List[str], full_type_map: List[str], bias_atom_e: ndarray, bias_adjust_mode='change-by-statistic', ntest=10)[source]

Change the energy bias according to the input data and the pretrained model.

Parameters
dataDeepmdDataSystem

The training data.

dpstr

The DeepEval object.

origin_type_maplist

The original type_map in dataset, they are targets to change the energy bias.

full_type_mapstr

The full type_map in pretrained model

bias_atom_enp.ndarray

The old energy bias in the pretrained model.

bias_adjust_modestr

The mode for changing energy bias : [‘change-by-statistic’, ‘set-by-statistic’] ‘change-by-statistic’ : perform predictions on energies of target dataset,

and do least sqaure on the errors to obtain the target shift as bias.

‘set-by-statistic’ : directly use the statistic energy bias in the target dataset.

ntestint

The number of test samples in a system to change the energy bias.

deepmd.utils.hostlist module

deepmd.utils.hostlist.get_host_names() Tuple[str, List[str]][source]

Get host names of all nodes in the cluster.

If mpi4py is not installed or MPI is not used, then the host name of the current node is returned as those of all nodes.

Returns
str

Host name of the current node

List[str]

List of host names of all nodes in the cluster

deepmd.utils.model_stat module

deepmd.utils.model_stat.make_stat_input(data, nbatches, merge_sys=True)[source]

Pack data for statistics.

Parameters
data

The data

nbatchesint

The number of batches

merge_sysbool (True)

Merge system data

Returns
all_stat:

A dictionary of list of list storing data for stat. if merge_sys == False data can be accessed by

all_stat[key][sys_idx][batch_idx][frame_idx]

else merge_sys == True can be accessed by

all_stat[key][batch_idx][frame_idx]

deepmd.utils.model_stat.merge_sys_stat(all_stat)[source]

deepmd.utils.neighbor_stat module

class deepmd.utils.neighbor_stat.NeighborStat(ntypes: int, rcut: float, mixed_type: bool = False)[source]

Bases: ABC

Abstract base class for getting training data information.

It loads data from DeepmdData object, and measures the data info, including neareest nbor distance between atoms, max nbor size of atoms and the output data range of the environment matrix.

Parameters
ntypesint

The num of atom types

rcutfloat

The cut-off radius

mixed_typebool, optional, default=False

Treat all types as a single type.

Methods

get_stat(data)

Get the data statistics of the training data, including nearest nbor distance between atoms, max nbor size of atoms.

iterator(data)

Abstract method for producing data.

get_stat(data: DeepmdDataSystem) Tuple[float, ndarray][source]

Get the data statistics of the training data, including nearest nbor distance between atoms, max nbor size of atoms.

Parameters
data

Class for manipulating many data systems. It is implemented with the help of DeepmdData.

Returns
min_nbor_dist

The nearest distance between neighbor atoms

max_nbor_size

An array with ntypes integers, denotes the actual achieved max sel

abstract iterator(data: DeepmdDataSystem) Iterator[Tuple[ndarray, float, str]][source]

Abstract method for producing data.

Yields
mnnp.ndarray

The maximal number of neighbors

dtfloat

The squared minimal distance between two atoms

jjstr

The directory of the data system

deepmd.utils.out_stat module

Output statistics.

deepmd.utils.out_stat.compute_stats_from_atomic(output: ndarray, atype: ndarray) Tuple[ndarray, ndarray][source]

Compute the output statistics.

Given the output value and the type of atoms, compute the atomic output bais and std.

Parameters
output

The output value, shape is [nframes, nloc, ndim].

atype

The type of atoms, shape is [nframes, nloc].

Returns
np.ndarray

The computed output bias, shape is [ntypes, ndim].

np.ndarray

The computed output std, shape is [ntypes, ndim].

deepmd.utils.out_stat.compute_stats_from_redu(output_redu: ndarray, natoms: ndarray, assigned_bias: Optional[ndarray] = None, rcond: Optional[float] = None) Tuple[ndarray, ndarray][source]

Compute the output statistics.

Given the reduced output value and the number of atoms for each atom, compute the least-squares solution as the atomic output bais and std.

Parameters
output_redu

The reduced output value, shape is [nframes, ndim].

natoms

The number of atoms for each atom, shape is [nframes, ntypes].

assigned_bias

The assigned output bias, shape is [ntypes, ndim]. Set to nan if not assigned.

rcond

Cut-off ratio for small singular values of a.

Returns
np.ndarray

The computed output bias, shape is [ntypes, ndim].

np.ndarray

The computed output std, shape is [ntypes, ndim].

deepmd.utils.pair_tab module

class deepmd.utils.pair_tab.PairTab(filename: str, rcut: Optional[float] = None)[source]

Bases: object

Pairwise tabulated potential.

Parameters
filename

File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.

Methods

get()

Get the serialized table.

reinit(filename[, rcut])

Initialize the tabulated interaction.

deserialize

serialize

classmethod deserialize(data) PairTab[source]
get() Tuple[array, array][source]

Get the serialized table.

reinit(filename: str, rcut: Optional[float] = None) None[source]

Initialize the tabulated interaction.

Parameters
filename

File name for the short-range tabulated potential. The table is a text data file with (N_t + 1) * N_t / 2 + 1 columes. The first colume is the distance between atoms. The second to the last columes are energies for pairs of certain types. For example we have two atom types, 0 and 1. The columes from 2nd to 4th are for 0-0, 0-1 and 1-1 correspondingly.

serialize() dict[source]

deepmd.utils.path module

class deepmd.utils.path.DPH5Path(path: str, mode: str = 'r')[source]

Bases: DPPath

The path class to data system (DeepmdData) for HDF5 files.

Parameters
pathstr

path

modestr, optional

mode, by default “r”

Notes

OS - HDF5 relationship:

directory - Group file - Dataset

Attributes
name

Name of the path.

Methods

glob(pattern)

Search path using the glob pattern.

is_dir()

Check if self is directory.

is_file()

Check if self is file.

load_numpy()

Load NumPy array.

load_txt([dtype])

Load NumPy array from text.

mkdir([parents, exist_ok])

Make directory.

rglob(pattern)

This is like calling DPPath.glob() with **/ added in front of the given relative pattern.

save_numpy(arr)

Save NumPy array.

glob(pattern: str) List[DPPath][source]

Search path using the glob pattern.

Parameters
patternstr

glob pattern

Returns
List[DPPath]

list of paths

is_dir() bool[source]

Check if self is directory.

is_file() bool[source]

Check if self is file.

load_numpy() ndarray[source]

Load NumPy array.

Returns
np.ndarray

loaded NumPy array

load_txt(dtype: Optional[dtype] = None, **kwargs) ndarray[source]

Load NumPy array from text.

Returns
np.ndarray

loaded NumPy array

mkdir(parents: bool = False, exist_ok: bool = False) None[source]

Make directory.

Parameters
parentsbool, optional

If true, any missing parents of this directory are created as well.

exist_okbool, optional

If true, no error will be raised if the target directory already exists.

property name: str

Name of the path.

rglob(pattern: str) List[DPPath][source]

This is like calling DPPath.glob() with **/ added in front of the given relative pattern.

Parameters
patternstr

glob pattern

Returns
List[DPPath]

list of paths

save_numpy(arr: ndarray) None[source]

Save NumPy array.

Parameters
arrnp.ndarray

NumPy array

class deepmd.utils.path.DPOSPath(path: str, mode: str = 'r')[source]

Bases: DPPath

The OS path class to data system (DeepmdData) for real directories.

Parameters
pathstr

path

modestr, optional

mode, by default “r”

Attributes
name

Name of the path.

Methods

glob(pattern)

Search path using the glob pattern.

is_dir()

Check if self is directory.

is_file()

Check if self is file.

load_numpy()

Load NumPy array.

load_txt(**kwargs)

Load NumPy array from text.

mkdir([parents, exist_ok])

Make directory.

rglob(pattern)

This is like calling DPPath.glob() with **/ added in front of the given relative pattern.

save_numpy(arr)

Save NumPy array.

glob(pattern: str) List[DPPath][source]

Search path using the glob pattern.

Parameters
patternstr

glob pattern

Returns
List[DPPath]

list of paths

is_dir() bool[source]

Check if self is directory.

is_file() bool[source]

Check if self is file.

load_numpy() ndarray[source]

Load NumPy array.

Returns
np.ndarray

loaded NumPy array

load_txt(**kwargs) ndarray[source]

Load NumPy array from text.

Returns
np.ndarray

loaded NumPy array

mkdir(parents: bool = False, exist_ok: bool = False) None[source]

Make directory.

Parameters
parentsbool, optional

If true, any missing parents of this directory are created as well.

exist_okbool, optional

If true, no error will be raised if the target directory already exists.

property name: str

Name of the path.

rglob(pattern: str) List[DPPath][source]

This is like calling DPPath.glob() with **/ added in front of the given relative pattern.

Parameters
patternstr

glob pattern

Returns
List[DPPath]

list of paths

save_numpy(arr: ndarray) None[source]

Save NumPy array.

Parameters
arrnp.ndarray

NumPy array

class deepmd.utils.path.DPPath(path: str, mode: str = 'r')[source]

Bases: ABC

The path class to data system (DeepmdData).

Parameters
pathstr

path

modestr, optional

mode, by default “r”

Attributes
name

Name of the path.

Methods

glob(pattern)

Search path using the glob pattern.

is_dir()

Check if self is directory.

is_file()

Check if self is file.

load_numpy()

Load NumPy array.

load_txt(**kwargs)

Load NumPy array from text.

mkdir([parents, exist_ok])

Make directory.

rglob(pattern)

This is like calling DPPath.glob() with **/ added in front of the given relative pattern.

save_numpy(arr)

Save NumPy array.

abstract glob(pattern: str) List[DPPath][source]

Search path using the glob pattern.

Parameters
patternstr

glob pattern

Returns
List[DPPath]

list of paths

abstract is_dir() bool[source]

Check if self is directory.

abstract is_file() bool[source]

Check if self is file.

abstract load_numpy() ndarray[source]

Load NumPy array.

Returns
np.ndarray

loaded NumPy array

abstract load_txt(**kwargs) ndarray[source]

Load NumPy array from text.

Returns
np.ndarray

loaded NumPy array

abstract mkdir(parents: bool = False, exist_ok: bool = False) None[source]

Make directory.

Parameters
parentsbool, optional

If true, any missing parents of this directory are created as well.

exist_okbool, optional

If true, no error will be raised if the target directory already exists.

abstract property name: str

Name of the path.

abstract rglob(pattern: str) List[DPPath][source]

This is like calling DPPath.glob() with **/ added in front of the given relative pattern.

Parameters
patternstr

glob pattern

Returns
List[DPPath]

list of paths

abstract save_numpy(arr: ndarray) None[source]

Save NumPy array.

Parameters
arrnp.ndarray

NumPy array

deepmd.utils.plugin module

Base of plugin systems.

class deepmd.utils.plugin.Plugin[source]

Bases: object

A class to register and restore plugins.

Examples

>>> plugin = Plugin()
>>> @plugin.register("xx")
    def xxx():
        pass
>>> print(plugin.plugins["xx"])
Attributes
pluginsDict[str, object]

plugins

Methods

get_plugin(key)

Visit a plugin by key.

register(key)

Register a plugin.

get_plugin(key) object[source]

Visit a plugin by key.

Parameters
keystr

key of the plugin

Returns
object

the plugin

register(key: str) Callable[[object], object][source]

Register a plugin.

Parameters
keystr

key of the plugin

Returns
Callable[[object], object]

decorator

class deepmd.utils.plugin.PluginVariant(*args, **kwargs)[source]

Bases: object

A class to remove type from input arguments.

class deepmd.utils.plugin.VariantABCMeta(name, bases, namespace, **kwargs)[source]

Bases: VariantMeta, ABCMeta

Methods

__call__(*args, **kwargs)

Remove type and keys that starts with underline.

mro(/)

Return a type's method resolution order.

register(subclass)

Register a virtual subclass of an ABC.

class deepmd.utils.plugin.VariantMeta[source]

Bases: object

Methods

__call__(*args, **kwargs)

Remove type and keys that starts with underline.

deepmd.utils.plugin.make_plugin_registry(name: Optional[str] = None) Type[object][source]

Make a plugin registry.

Parameters
nameOptional[str]

the name of the registry for the error message, e.g. descriptor, backend, etc.

Examples

>>> class BaseClass(make_plugin_registry()):
        pass

deepmd.utils.random module

deepmd.utils.random.choice(a: Union[ndarray, int], size: Optional[Union[int, Tuple[int, ...]]] = None, replace: bool = True, p: Optional[ndarray] = None)[source]

Generates a random sample from a given 1-D array.

Parameters
a1-D array_like or int

If an ndarray, a random sample is generated from its elements. If an int, the random sample is generated as if it were np.arange(a)

sizeint or tuple of ints, optional

Output shape. If the given shape is, e.g., (m, n, k), then m * n * k samples are drawn. Default is None, in which case a single value is returned.

replacebool, optional

Whether the sample is with or without replacement. Default is True, meaning that a value of a can be selected multiple times.

p1-D array_like, optional

The probabilities associated with each entry in a. If not given, the sample assumes a uniform distribution over all entries in a.

Returns
np.ndarray

arrays with results and their shapes

deepmd.utils.random.random(size=None)[source]

Return random floats in the half-open interval [0.0, 1.0).

Parameters
size

Output shape.

Returns
np.ndarray

Arrays with results and their shapes.

deepmd.utils.random.seed(val: Optional[int] = None)[source]

Seed the generator.

Parameters
valint

Seed.

deepmd.utils.random.shuffle(x: ndarray)[source]

Modify a sequence in-place by shuffling its contents.

Parameters
xnp.ndarray

The array or list to be shuffled.

deepmd.utils.spin module

class deepmd.utils.spin.Spin(use_spin: List[bool], virtual_scale: Union[List[float], float])[source]

Bases: object

Class for spin, mainly processes the spin type-related information. Atom types can be split into three kinds: 1. Real types: real atom species, “Fe”, “H”, “O”, etc. 2. Spin types: atom species with spin, as virtual atoms in input, “Fe_spin”, etc. 3. Placeholder types: atom species without spin, as placeholders in input without contribution, also name “H_spin”, “O_spin”, etc. For any types in 2. or 3., the type index is ntypes plus index of its corresponding real type.

Parameters
use_spin: List[bool]

A list of boolean values indicating whether to use atomic spin for each atom type. True for spin and False for not. List of bool values with shape of [ntypes].

virtual_scale: List[float], float

The scaling factor to determine the virtual distance between a virtual atom representing spin and its corresponding real atom for each atom type with spin. This factor is defined as the virtual distance divided by the magnitude of atomic spin for each atom type with spin. The virtual coordinate is defined as the real coordinate plus spin * virtual_scale. List of float values with shape of [ntypes] or [ntypes_spin] or one single float value for all types, only used when use_spin is True for each atom type.

Methods

get_atom_exclude_types([exclude_types])

Return the atom-wise exclusion types for fitting before out_def.

get_atom_exclude_types_placeholder([...])

Return the atom-wise exclusion types for fitting after out_def.

get_ntypes_input()

Returns the number of double real atom types for input placeholder.

get_ntypes_real()

Returns the number of real atom types.

get_ntypes_real_and_spin()

Returns the number of real atom types and types which contain spin.

get_ntypes_spin()

Returns the number of atom types which contain spin.

get_pair_exclude_types([exclude_types])

Return the pair-wise exclusion types for descriptor.

get_spin_mask()

Return the spin mask of shape [ntypes], with spin types being 1, and non-spin types being 0.

get_use_spin()

Returns the list of whether to use spin for each atom type.

get_virtual_scale()

Returns the list of magnitude of atomic spin for each atom type.

get_virtual_scale_mask()

Return the virtual scale mask of shape [ntypes], with spin types being its virtual scale, and non-spin types being 0.

init_atom_exclude_types_placeholder()

Initialize the atom-wise exclusion types for fitting.

init_atom_exclude_types_placeholder_spin()

Initialize the atom-wise exclusion types for fitting.

init_pair_exclude_types_placeholder()

Initialize the pair-wise exclusion types for descriptor.

deserialize

serialize

classmethod deserialize(data: dict) Spin[source]
get_atom_exclude_types(exclude_types=None) List[int][source]

Return the atom-wise exclusion types for fitting before out_def. Both the placeholder types and spin types are excluded.

get_atom_exclude_types_placeholder(exclude_types=None) List[int][source]

Return the atom-wise exclusion types for fitting after out_def. The placeholder types for those without spin are excluded.

get_ntypes_input() int[source]

Returns the number of double real atom types for input placeholder.

get_ntypes_real() int[source]

Returns the number of real atom types.

get_ntypes_real_and_spin() int[source]

Returns the number of real atom types and types which contain spin.

get_ntypes_spin() int[source]

Returns the number of atom types which contain spin.

get_pair_exclude_types(exclude_types=None) List[Tuple[int, int]][source]

Return the pair-wise exclusion types for descriptor. The placeholder types for those without spin are excluded.

get_spin_mask()[source]

Return the spin mask of shape [ntypes], with spin types being 1, and non-spin types being 0.

get_use_spin() List[bool][source]

Returns the list of whether to use spin for each atom type.

get_virtual_scale() ndarray[source]

Returns the list of magnitude of atomic spin for each atom type.

get_virtual_scale_mask()[source]

Return the virtual scale mask of shape [ntypes], with spin types being its virtual scale, and non-spin types being 0.

init_atom_exclude_types_placeholder() None[source]

Initialize the atom-wise exclusion types for fitting. The placeholder types for those without spin are excluded.

init_atom_exclude_types_placeholder_spin() None[source]

Initialize the atom-wise exclusion types for fitting. Both the placeholder types and spin types are excluded.

init_pair_exclude_types_placeholder() None[source]

Initialize the pair-wise exclusion types for descriptor. The placeholder types for those without spin are excluded.

serialize() dict[source]

deepmd.utils.summary module

class deepmd.utils.summary.SummaryPrinter[source]

Bases: ABC

Base summary printer.

Backends should inherit from this class and implement the abstract methods.

Methods

__call__()

Print build and current running cluster configuration summary.

get_backend_info()

Get backend information.

get_compute_device()

Get Compute device.

get_ngpus()

Get the number of GPUs.

is_built_with_cuda()

Check if the backend is built with CUDA.

is_built_with_rocm()

Check if the backend is built with ROCm.

BUILD: ClassVar = {'build variant': 'cpu', 'installed to': '/home/docs/checkouts/readthedocs.org/user_builds/deepmd/conda/latest/lib/python3.9/site-packages/deepmd', 'source': '', 'source brach': 'HEAD', 'source commit': '23f67a1', 'source commit at': '2024-03-28 12:44:51 +0000', 'use float prec': 'double'}
CITATION = ('Please read and cite:', 'Wang, Zhang, Han and E, Comput.Phys.Comm. 228, 178-184 (2018)', 'Zeng et al, J. Chem. Phys., 159, 054801 (2023)', 'See https://deepmd.rtfd.io/credits/ for details.')
WELCOME = (' _____               _____   __  __  _____           _     _  _   ', '|  __ \\             |  __ \\ |  \\/  ||  __ \\         | |   (_)| |  ', '| |  | |  ___   ___ | |__) || \\  / || |  | | ______ | | __ _ | |_ ', '| |  | | / _ \\ / _ \\|  ___/ | |\\/| || |  | ||______|| |/ /| || __|', '| |__| ||  __/|  __/| |     | |  | || |__| |        |   < | || |_ ', '|_____/  \\___| \\___||_|     |_|  |_||_____/         |_|\\_\\|_| \\__|')
get_backend_info() dict[source]

Get backend information.

abstract get_compute_device() str[source]

Get Compute device.

abstract get_ngpus() int[source]

Get the number of GPUs.

abstract is_built_with_cuda() bool[source]

Check if the backend is built with CUDA.

abstract is_built_with_rocm() bool[source]

Check if the backend is built with ROCm.

deepmd.utils.update_sel module

class deepmd.utils.update_sel.BaseUpdateSel[source]

Bases: object

Update the sel field in the descriptor.

Attributes
neighbor_stat

Methods

get_min_nbor_dist

get_nbor_stat

get_rcut

get_sel

get_type_map

hook

parse_auto_sel

parse_auto_sel_ratio

update_one_sel

wrap_up_4

get_min_nbor_dist(jdata, rcut)[source]
get_nbor_stat(jdata, rcut, mixed_type: bool = False)[source]
get_rcut(jdata)[source]
get_sel(jdata, rcut, mixed_type: bool = False)[source]
get_type_map(jdata)[source]
abstract hook(min_nbor_dist, max_nbor_size)[source]
abstract property neighbor_stat: Type[NeighborStat]
parse_auto_sel(sel)[source]
parse_auto_sel_ratio(sel)[source]
update_one_sel(jdata, descriptor, mixed_type: bool = False, rcut_key='rcut', sel_key='sel')[source]
wrap_up_4(xx)[source]

deepmd.utils.version module

deepmd.utils.version.check_version_compatibility(current_version: int, maximum_supported_version: int, minimal_supported_version: int = 1)[source]

Check if the current version is compatible with the supported versions.

Parameters
current_versionint

The current version.

maximum_supported_versionint

The maximum supported version.

minimal_supported_versionint, optional

The minimal supported version. Default is 1.

Raises
ValueError

If the current version is not compatible with the supported versions.

deepmd.utils.weight_avg module

deepmd.utils.weight_avg.weighted_average(errors: List[Dict[str, Tuple[float, float]]]) Dict[source]

Compute wighted average of prediction errors (MAE or RMSE) for model.

Parameters
errorsList[Dict[str, Tuple[float, float]]]

List: the error of systems Dict: the error of quantities, name given by the key str: the name of the quantity, must starts with ‘mae’ or ‘rmse’ Tuple: (error, weight)

Returns
Dict

weighted averages