mlgidMATCH performs peak-to-structure matching of GID patterns.
The package performs crystal phase identification from (generally, multi-phase) GID patterns based on Bragg peak positions and their intensities. To validate a measured pattern, the framework requires a set of candidate crystal structures (generally, the entire crystal database can be supplied).
The framework returns (generally, multiple) set(s) of crystal structures that explain all or most of the measured peaks.
A full description of the matching algorithm process could be found in (here will be the link to the paper)
pip install mlgidmatchFirst, clone the repository:
git clone https://github.com/mlgid-project/mlgidMATCH.gitThen, to install all required modules, navigate to the cloned directory and execute:
cd mlgidMATCH
pip install -e .Before validation, a preprocessing step is required to convert candidate crystal structures into a neural network-friendly format. It is recommended to perform this step in advance (e.g., before the experiment), as the preprocessing may take several minutes.
To preprocess candidate structures, use the mlgidmatch.preprocess.cif_preprocess.CifPattern class. This class prepares
all data required for the neural matching stage and the subsequent peak-to-structure matching.
The class requires a folder containing CIF files, specified by the folder_path argument. If only a subset of CIF files
from the folder should be used, the cifs argument can be provided.
The argument create_all=True enables precomputation of patterns for all unique crystal orientations. This option is
recommended only when the number of candidate structures is small (up to ~1000), as it may otherwise lead to excessive
memory usage.
The class also requires experimental parameters for correct preprocessing. These parameters can be created using the
experiment.ExpParameters class from the pygidsim package, which is available on PyPi.
import warnings
from mlgidmatch.preprocess.cif_preprocess import CifPattern
from pygidsim.experiment import ExpParameters
warnings.filterwarnings("ignore")
# path to the folder with CIF files
folder_path = './cifs/'
# list of CIF files to preprocess (if not provided, all CIFs from the folder will be used)
all_cifs = ['struct1.cif', 'struct2.cif', ...] # optional
params = ExpParameters(q_xy_max=5, q_z_max=5, en=18_000) # experimental parameters
cif_prepr = CifPattern(
params=params,
folder_path=folder_path,
cifs=all_cifs, # optional
create_all=True, # optional, default: False
)For future use, it is recommended to save the preprocessed data using pickle format:
import pickle
with open('./mlgidmatch/data/prepr_cifs.pickle', 'wb') as file:
pickle.dump(cif_prepr, file)To load the preprocessed data later use the following code:
with open('./mlgidmatch/data/prepr_cifs.pickle', 'rb') as file:
cif_prepr = pickle.load(file)To receive only probabilities for the candidate structures from the neural matching stage, use the following example:
from mlgidmatch.matching import Match
match_class = Match(
model_path='./cif_matching/models/ResNet18_best_model.pt',
cif_prepr=cif_prepr,
device='cuda',
)
probabilities = match_class.match_cifs(
peaks=q_2d, # np.ndarray, shape (peaks_num, 2)
q_range=(q_xy_max, q_z_max), # upper limits of q-range
candidates=['struct1.cif', 'struct5.cif'], # candidate structures for the measurement (optional)
)To perform full matching, including neural matching, phase identification and peak-to-structure assignment, use the following example:
from mlgidmatch.matching import Match
match_class = Match(
cif_prepr=cif_prepr,
model_path='./cif_matching/models/ResNet18_newimage_14ch_state99999.pt', # optional
device='cuda', # optional
)
# names of the measurements
measurements = ['meas1', 'meas2', ...]
# Peak positions and intensities (own ArrayLike per measurement)
peak_list = [q_2d_1, q_2d_2, ...]
intensities_real_list = [intens1, intens2, ...]
# Upper limits of the q-range (q_xy, q_z)
q_range_list = [(2.7, 2.7), (3.1, 2.5), ...]
# type of the peaks - 'segments' or 'rings'
peaks_type = 'segments'
# Probability threshold (optional)
threshold = 0.5
# Candidate structures for each measurement (optional)
candidates_list = [
['struct1.cif', 'struct5.cif'],
['struct2.cif', 'struct3.cif', 'struct7.cif'],
...
], # Leave empty to use all structures from cif_prepr.cifs
# Matching process
data_matched = match_class.match_all(
measurements=measurements,
peak_list=peak_list,
intensities_real_list=intensities_real_list,
q_range_list=q_range_list,
threshold=threshold, # optional, default: 0.5
candidates_list=candidates_list, # optional, Leave empty to use all structures from cif_prepr.cifs
peaks_type=peaks_type,
)
# Make user-friendly output by removing duplicated solutions (e.g. [DIP + HATCH] and [HATCH + DIP]),
# description is below in the Output section.
unique_solutions = match_class.unique_solutions(data_matched)After the matching process, data_matched is a dictionary with the following hierarchical structure:
data_matched/
├── <measurement_name>/ # e.g. "meas_1"
│ └── peaks/ # list of peak positions
│ ├── <phase_1_option_id>/ # integer, first phase index (option 1)
│ │ ├── orient # crystal orientation
│ │ ├── probability # phase probability
│ │ ├── indices_real_matched_all # indices of the peaks matched to the structure
│ │ │
│ │ ├── <phase_2_option_id>/... # integer, second phase index (option 1)
│ │ │ └──...
│ │ ├── <phase_2_option_id>/... # integer, second phase index (option 2)
│ │ │ └──...
... ... ...
│ ├── <phase_1_option_id>/ # integer, first phase index (option 2)
│ │ └──...
│ │
│ └── ...
│
└──This output contains complete information about the peak-to-structure matching process. If no valid solutions are found,
the output tree contains only the peaks entry.
An example output is shown below:
data_matched = {
'meas_1': {
'peaks': np.array(
[[0.0310, 0.7514],
[1.7270, 0.9246],
[0.3772, 2.5963],
...]
),
'0': {
'cif': 'DIP.cif',
'orient': np.array([0, 0, 1]),
'probability': 0.985,
'indices_real_matched_all': np.array([...]),
'0': {
'cif': 'HATCH.cif',
'orient': np.array([1, 0, 1]),
'probability': 0.685,
'indices_real_matched_all': np.array([...])
},
'1': {
'cif': 'ZnPc.cif',
'orient': np.array([1, 1, 1]),
'probability': 0.792,
'indices_real_matched_all': np.array([...]),
'0': {
'cif': 'HATCH.cif',
'orient': np.array([1, 0, 1]),
'probability': 0.582,
'indices_real_matched_all': np.array([...])
}
}
},
'1': {
'cif': 'HATCH.cif',
'orient': np.array([1, 0, 1]),
'probability': 0.991,
'indices_real_matched_all': np.array([...]),
'0': {
'cif': 'DIP.cif',
'orient': np.array([0, 0, 1]),
'probability': 0.911,
'indices_real_matched_all': np.array([...])
}
}
},
'meas_2': {
...
}
}This result indicates that the framework found three valid phase combinations:
00: DIP + HATCH010: DIP + ZnPc + HATCH10: HATCH + DIP
Finally, duplicated solutions (e.g. [DIP + HATCH] and [HATCH + DIP]) can be removed using the
unique_solutions() method:
unique_solutions = match_class.unique_solutions(data_matched)An example of the final output where 'meas_1' contains two unique solutions (DIP + HATCH and DIP + ZnPc + HATCH) is shown below:
unique_solutions = {
'meas_1': {
0: [
{
'cif': 'DIP.cif',
'orientation': np.array([0, 0, 1]),
'matched_peaks': np.array([0.985, 0, 0, ..., 0.985, 0]),
},
{
'cif': 'HATCH.cif',
'orientation': np.array([1, 0, 1]),
'matched_peaks': np.array([0, 0.685, 0, ..., 0.685, 0]),
},
],
1: [
{
'cif': 'DIP.cif',
'orientation': np.array([0, 0, 1]),
'matched_peaks': np.array([0.985, 0, 0, ..., 0.985, 0]),
},
{
'cif': 'ZnPc.cif',
'orientation': np.array([1, 1, 1]),
'matched_peaks': np.array([0.792, 0, 0.792, ..., 0.792, 0]),
},
{
'cif': 'HATCH.cif',
'orientation': np.array([1, 0, 1]),
'matched_peaks': np.array([0, 0, 0.582, ..., 0.582, 0.582]),
}
],
},
'meas_2': {[...]}
}