Announcing Covalent v0.209.1
New year, new release! Today, we are excited to announce the latest version of Covalent, v0.209.1. This release brings a host of improvements and new features to make your experience using Covalent even better.
pip install covalentOne of the main goals of this release was to improve the workflow execution logic and bring more organization and visibility to the task and workflow level errors/exceptions during workflow execution. This will make it easier to identify and resolve any issues that may arise while using Covalent.
Apart from these major changes, we are also releasing the following additions/improvements:
- AWS S3 File Transfer between folders
- Executor inheritance in electrons & sublattices
See the full list of new features included in v0.209.1 in the table below:
| Features | Summary |
|---|---|
| Database | Migrations are now run automatically after fresh install/purging of covalent. PostgreSQL DB backend is now supported. |
| AWS S3 File Transfer | Support was added for transferring the contents of folders to and from S3 buckets instead of transferring files one at a time. |
| Executor inheritance | Electrons now inherit the lattice executors if no executors are specified. Sublattices inherit the parent lattice executor. |
| Workflow execution errors | When tasks fail in a workflow, the Lattice Error Card in the UI summarizes the failed tasks |
| Electron execution error messages UI | Electron errors appear in the Electron ErrorCard when one clicks on a failed task in the UI. Electron errors are segregated by type; task runtime errors are stored in stderr while the error attribute of a node is reserved for exceptions raised by Covalent itself. |
| Electron exceptions | When an electron raises an exception during execution, the local and Dask executors now try to recover any already written output. |
| Covalent CLI | When the Covalent server is restarted, it starts with a Dask cluster – or not – depending on the choice before the server was restarted |
✨ Database
When a new Covalent version is installed, and the server is restarted, the database migration is automatic, and the user does not need to run covalent db migrate anymore. In addition, PostgreSQL DB is now supported.
The database feature in v0.209.1 makes it easier to install and use Covalent by streamlining the process of migrating the database.
In previous versions of Covalent, after a new version was installed, the user had to manually run the command covalent db migrate to migrate the database to the new version. However, with the new database feature in v0.209.1, the database migrations are now run automatically after a fresh install or whenever the server is restarted. This eliminates the need for the user to run the migration command manually and makes upgrading to a new version of Covalent smoother and more efficient.
In addition, support for PostgreSQL database backend is now included. This means that users can now use PostgreSQL as the database backend for Covalent in addition to the existing SQLite backend. This gives users more flexibility and options when choosing a database backend for their Covalent installation.
✨ AWS S3 File Transfer Folder Support
In this release, the AWS S3 File Transfer module in Covalent allows transferring files by specifying the initial and destination folders as opposed to transferring files one by one.
✨ Executor inheritance
An electron/sublattice will now inherit the executor assigned to the parent lattice if an executor is not assigned to the child electron/sublattice.
The new executor inheritance feature in v0.209.1 makes it easier to manage your workflows and reduces the amount of code you need to write.
In previous versions of Covalent, if you wanted to run an electron or sublattice, you had to assign an executor to it explicitly. However, with executor inheritance, electrons and sublattices now inherit the executor assigned to the parent lattice if an executor is not assigned to the child electron or sublattice.
This means that you no longer have to manually assign an executor to every electron and sublattice in your workflow. Instead, you can assign an executor to the parent lattice, and all child electrons and sublattices will inherit that executor. This can save you time and makes it easier to manage your workflows, especially if you have many electrons and sublattices.
It also makes it easier to switch between different executors for your workflows. For example, suppose you have a local executor for development and testing and switch to a Dask executor for production. In that case, you can simply change the executor assigned to the parent lattice, and all child electrons and sublattices will automatically use the new executor.
Overall, the executor inheritance feature in this release makes it easier to manage your workflows and reduces the amount of code you need to write.
✨Electron execution error handling
The error corresponding to a failed task now appears on the Electron ErrorCard which can be accessed by clicking on the failed task in the UI. Furthermore, task runtime errors are stored in stderr while the error attribute of a node is reserved for exceptions raised by Covalent itself.
✨Workflow execution errors
In previous versions of Covalent, if a task failed in a workflow, it could be challenging to identify which task had failed and why. The new workflow execution errors feature addresses this issue by providing a clear and concise summary of the failed tasks in the Lattice ErrorCard in the UI.
When a task fails in a workflow, the Lattice ErrorCard in the UI now shows a list of all the failed tasks, along with a brief description of the error. This makes it easy to quickly identify which task(s) have failed and take appropriate action to resolve the issue.
We hope you enjoy these new features and improvements. As always, please let us know if you have any feedback or suggestions for future releases.
🔌 Tested with Plugins
The new version of covalent has been tested to work with the following executor plugins which correspond to the latest stable versions ✅
| Plugin | Version |
|---|---|
| covalent-aws-plugins | 0.13.0 |
| covalent-awsbatch-plugin | 0.26.0 |
| covalent-awslambda-plugin | 0.22.0 |
| covalent-braket-plugin | 0.13.0 |
| covalent-ecs-plugin | 0.23.0 |
| covalent-ec2-plugin | 0.12.0 |
🙋🏽 Contributors
This release would not have been possible without the hard work of the team at Agnostiq and our contributors: @AgnostiqHQ , @mshkanth, @Prasy12, @amalanpsiog, @mpvgithub,@Aravind-Psiog,@ArunPsiog,@Kamalesh-suresh,@RaviPsiog,@Socrates,