🚀 Covalent v0.220.0: Streamlining Workflows Like Never Before
pip install covalent --upgradeWe are thrilled to introduce Covalent v0.220.0! This latest update brings a suite of powerful new features designed to elevate your workflow management and execution capabilities. We’ve been closely monitoring user feedback and have meticulously crafted this release to provide the most sought-after improvements for both new and existing users. Experience the future of Covalent with v0.220.0 today!
In this update, you’ll find the following standout features:
| Feature | Real-world Usage |
|---|---|
| Workflow Re-dispatch Functionality | High-performance computing specialists can now save valuable time and resources by re-executing workflows with updated tasks without rerunning the entire workflow. This allows them to focus on refining simulations and delivering results faster. |
| Workflow Cancellation | Users can now quickly cancel ongoing workflows when errors occur, freeing up resources, saving money, and enabling them to fix issues and restart simulations without wasting computational time. This ensures research stays on track and resources are used efficiently. |
| Workflow Triggers | Machine learning engineers can now automate the initiation of workflows based on specific events or conditions, such as new data arrivals or completion of related workflows, using Covalent’s triggers. This allows them to focus on analyzing results and optimizing models, knowing that workflows will be executed promptly and efficiently whenever needed. |
| Slurm Plugin Upgrade | The Covalent Slurm plugin now exposes many more options for users to customize how jobs are bound to compute resources, and includes support for job profiling as well as tighter integration with environment modules. The recent release further introduces support for two-factor authentication with a remote backend! |
🌟 New Features in v0.220.0
1. Workflow Re-dispatch Functionality
We understand the importance of adaptability in today’s fast-paced development environment. That’s why we’ve added the Covalent re-dispatch function, enabling you to re-execute workflows with ease and precision. This feature allows for:
- Re-executing workflows with new input parameters
- Updating or replacing task definitions
- Optionally reusing previous results for faster execution
This powerful feature lets you rerun your workflow with modified inputs without requiring access to the underlying lattice function – a simple dispatch id is all you need!
To demonstrate the simplicity and power of this feature, take a look at this example:
redispatch_id = ct.redispatch(
dispatch_id,
replace_electrons={"task_2": task_2_redefined},
reuse_previous_results=True
)()
2. Workflow Cancellation
Long-running workflows can be a hassle, especially when you realize they’re executing incorrectly or consuming valuable resources. The new Workflow Cancellation feature provides the ultimate solution, allowing you to:
- Cancel running workflows on-demand
- Free up resources for other tasks
- Maintain better control over your workflow executions
- Save money and compute cycles!
To cancel a running workflow, simply use the following command:
ct.cancel_workflow(dispatch_id)
❕ Note that not all plugins support cancellation yet as plugins need to opt into this feature. Watch out for this support in respective plugins as they roll out.
3. Workflow Triggers
Automation is the key to efficiency, and with the introduction of Workflow Triggers, you can now automate your workflow execution process. This powerful feature empowers you to:
- Automatically execute workflows based on specific events or conditions
- Improve efficiency by reducing manual intervention
- Streamline your workflow management process
Triggers in Covalent allow you to automate repetitive tasks and streamline your workflow. By defining a pre-defined set of steps that run automatically every time a specific event occurs, you can easily integrate triggers into your Covalent workflows. Triggers can be easily attached to a lattice, and once registered, the observe() method of the trigger starts observing for the desired condition to be met in an unblocking manner.
For instance, Covalent offers several types of triggers out of the box, such as DirTrigger, TimeTrigger, and SQLiteTrigger with more to come. These triggers cater to diverse use cases, simplifying the automation of tasks based on a range of conditions.
For example, you can use TimeTrigger to perform a trigger action every 5 seconds:
from covalent.triggers import TimeTrigger
import covalent as ct
time_trigger = TimeTrigger(time_gap=5) # Trigger action every 5 seconds
@ct.lattice(triggers=time_trigger)
def my_workflow():
...
With Covalent triggers, you can automate processes, reducing the risk of human error and ensuring that your pipeline runs smoothly and efficiently.
📚 Full List of Features in v0.220.0
| Features | Summary |
|---|---|
| Workflow Re-dispatch | Easily re-execute workflows with new parameters or updated task definitions. |
| Workflow Cancellation | Cancel running workflows on-demand for better control and resource management. |
| Workflow Triggers | Automate workflow execution based on specific events or conditions. |
We are confident that these new features will significantly enhance your Covalent experience and empower you to manage your workflows more effectively. Your feedback is invaluable to us, and we encourage you to share your thoughts and suggestions for future improvements.
Don’t hesitate to try out Covalent v0.220.0 and harness the power of the enhanced workflow management features! We are dedicated to continually improving our platform to better serve your needs, and we look forward to your feedback on these new additions.
To get started with the new release, simply update your covalent with pip install covalent --upgradeand dive into the comprehensive documentation to learn more about these exciting new features.
As always, our team is here to support you. If you have any questions or need assistance, please reach out to our community forums or join us in Slack.
Thank you for being part of the Covalent community, and we can’t wait to see how you leverage the power of v0.220.0 to create innovative and efficient workflows.
Happy dispatching!