Process

Overview

The process class is an abstract base class (ABC) derived from Pydantic’s BaseModel, representing a computational process applied to a material. It is designed to serve as the foundation for concrete process subclasses (e.g., relaxation, phonon, forcefield workflows) that manage workflows involving one or more calculations. The process stores the basic information of a process performed on the material. This includes: - holding a workflow, a single DFT calculation, or repeats of the same type of calculation - managing submission and progress tracking - integrating with mat_record as a host

from DeWorks.calculations import process 

Attributes

Core Fields

Field	Type	Description
id	str	UUID generated id for the process obj; used primarily as the _id attribute when stored in mongodb
process_type	str	The identifier between the different sub-classes of process() should not be changed or modified by the user
name	str	The name of the process obj. Defaults to process_type or host.name_process_type if host is available

Calculation Fields

Field	Type	Description
calculators	Calculator	Object that defines how the job is run
calculator_type	Literal["LOCAL", "REMOTE", "VASP", "FORCEFIELD", "unset"]	Categorise the calculator
forcefield_type	Literal["CHGNET", "MACE", "MATTERSIM", "CUSTOM", "NA"]	Indicates forcefield backend if applicable
device	Literal["cpu", "cuda"]	compute backend to use cpu or cuda
calculations	dict[str, Calculation]	Stores calculation data per job
submit_obj	Flow | None	Submit Job/Flow to a remote executor

Output Fields

Field	Type	Description
output_collation	dict | None	Collated output metrics
error_collation	dict | None	Collated errors from failed jobs
result	PropertyResolved |None	Field storing for the finalised outcome of a process in a PROPERTY subclass instance; In the process subclasses, this definition can be overwritten with type specified as a single PROPERTY subclass to increase performance; The more generalised form in the base class is recommended for better type safety

Record Control Fields

Field	Type	Description
saved_json	str	None
saving_mode	Literal[1, 2]	1: write to file 2: update host
saving_dir	str	Directory for saving process data
submitted	list[str]	Job Ids from job submissions
host	MatRecord | None	Associated material records
state	Literal["INIT", "MADE", "SUBMITTED", "PART_FAILED", "COMPLETE", "FAILED"]	Status of the process

model_dump(self, args, *kwargs)

Serilaizes the object using field aliases. A manual implementation of serialize_by_alias by default.

Computed Fields

short_name(self)

A short descriptive label based on calculator and process type. Can be overwritten to better reflect characteristics.

Validation and De-Serialisation

default_id(cls, values:dict)

Automatically generates a UUID if _id is not provided.

set_defaults(cls, values)

Enforces PROPERTY subclass-specific defaults for flexibility.

resolve_calculator(self)

Initialises/Identifies the calculator backend (local, remote, VASP, forcefield)

default_result(self)

Returns a default result object based on the PROPERTY subclass.

load(cls, value: str | dict)

Loads a process from JSON file or a dictionary. Can be called by higher-evel records for loading in both dumped and un-dumped cases

# Loading from file
processObj = Process.load(“./path/to/a/process.json”)

# Loading from dictionary
data = {"id": "sampleID", "name": "relaxation", ...}
processObj = Process.load(data)

Utility Functions

set_host(self, host: MatRecord)

Attachest a mat_record host to the process and configures save paths.

processObj.set_host(record)

save(mode = 1 | 2)

Save any progress or modifications to the calculations being run. Called by other functions of a subclass to save changes made by any process.

processObj.save(mode: 1) #Save to local work directory as JSON file
processObj set_host(host: matRecord)
Attach a ‘mat_record’ host to the process and configures save paths
processObj.set_host(record) 
.save(mode: 2) #Used when called internally in a workflow to save to `mat_record.history`

Core Functions

make(material, overwrite = False, **kwargs)

Create Job/Flow objects, updates self.calculations and stores in submit_obj. Must be implemented in sublcasses

processObj.make(overwrite=True) 

submit(cluster, overwrite = Flase)

Submits generated Job/Flow to the cluster

processObj.submit(cluster = user_cluster)

start(submit = True, cluster, overwrite = False, make_kwargs)

Runs make() and submit() unless submission is skipped or LOCAL calculation is used.

processObj.start(cluster = user_cluster, submit = True)

progress(type = “simple”, state, show = True)

Returns job status using JobController. types are simple, states, full_info

process.Obj.progress(type= “simple”) #returns the number of jobs at the specified state
process.Obj.progress(type= “states”) # returns a table listing the state of all the jobs
process.Obj.progress(type= “full_info”) # returns the full list of ‘job_info’ objects retrieved without any processing

Post Processing

rerun(new_cluster, new_job, new_incar)

Reruns any failed jobs submitted by the ‘Process’ object from ‘jobflow_remote” backend

processObj.rerun(new_cluster = new_user_clster)

_reset_values(new_cluster, new_job, new_incar)

Used internally by ‘rerun( )’ method to help reconfigured failed jobs before resumission

fail_assess( )

Inspects failed jobs and calssifies the type of failure, such as timeout, unconverged, etc..

df = processObj.fail_assess( )

_error_check( )

Reruns any failed jobs submitted by the Process object from jobflow_remote backend

processObj.rerun(new_cluster = new_user_clster)

output_collate(fields)

Collates the outputs generated by the calculations. output_collation stores structured successful outputs. error_collation stores any errors or issues encountered during retrieval. Note: subclasses must implement logic to extract outputs

process.output_collate( )

plot(save_path, show = False)

Plot outputs generated by the calculations using matplot.lib

process.plot()

Notes - All subclasses must implement - result( ) - write_property( ) - outpute_collate( ) - plot( ) - submit_obj is intentionally excluded from serialisation to prevent accidental re-submission - jobflow_remote integration is used to provide full execution, monitoring, and resubmission capabilities

Dependencies - pydantic - jobflow - atomate2, - jobflow_remote - monty - matplotlib - pandas - DeWorks

Summary Example Workflow

#Process instantiation
sample=NewProcess(…)
sample.set_host(record)

#Running workflow
sample.start(submit=True, cluster=a_ new_cluster)
sample.progress(type= “full_info)

#Gather and save results
sample.write_property( )
sample.output_collate( )
sample.save( )