High-performance sandboxing for AI-generated scientific code, with LangChain and Covalent

How-To

March 15, 2024

LangChain is a popular open-source framework for creating applications based on AI language models. Following its debut in October of 2022, LangChain has cultivated a still-growing community of over fifty thousands users and over two thousand contributors. It has also grown beyond the core framework during this time, with the introduction of LangServe, a deployment tool for production-scale LLM applications, and more recently LangGraph, which provides a high-level interface for creating LLM agent runtimes based on cyclical graphs.

An extensive library of third-party platform integrations expands LangChain’s interoperability with various AI platforms and related tools. With this blog post, we introduce the latest expansion of this ecosystem: A LangChain interface for Covalent and Covalent Cloud!

This new interface allows LangChain applications to execute workflows on arbitrary compute backends; from high-end CPUs to GPUs like Nvidia H100s—essentially everything that Covalent natively supports.

Topic summary

To demonstrate this new functionality, we’ll create an AI agent that iteratively refines compute-intensive, scientific code by using Covalent to provide an integrated, high-performance sandbox for code execution.

What is Covalent?

Covalent is an open-core Pythonic framework for designing and managing high-compute workflows across cloud and/or on-prem environments. Covalent Cloud is the paid and fully-managed version of its open-source progenitor, covalent.

In a nutshell, Covalent executes local code using remote resources. It lets you access advanced hardware, without the hassle of setting it up yourself. More than this, Covalent is a workflow tool that’s designed to scale and interconnect seamlessly.

Feel free to take a detour to our Quick Start for a more complete introduction. In the remainder of this blog, we’ll discuss two new LangChain components, Covalent and CovalentCloud, which serve as async-compatible wrappers for launching Covalent workflows.

Writing a basic sandbox function

The sandbox function below takes a JSON object with a “code” field and a “dependencies” field. The latter optionally contains a preparatory shell command for running the Python code contained in the former. This is just about the simplest sandbox one can conjure up

Here, sandbox runs the code, captures is outputs, and adds those outputs the original data, before returning the expanded JSON object.

import json
import subprocess
import tempfile
from operator import itemgetter
from typing import Any, Dict, List, Optional, Union

from langchain.agents import AgentExecutor, AgentOutputParser
from langchain.callbacks.base import BaseCallbackHandler
from langchain.chat_models import ChatOpenAI
from langchain.llms import Covalent, CovalentCloud
from langchain.output_parsers.pydantic import PydanticOutputParser
from langchain.prompts import ChatPromptTemplate
from langchain.pydantic_v1 import BaseModel, Field
from langchain.schema import AgentAction, AgentFinish, LLMResult
from langchain.schema.messages import AIMessage
from langchain.schema.runnable import RunnableLambda
from langchain.tools import tool

def sandbox(info_json: str) -> str:
    """
    Covalent will ship this function to a user-specified compute backend.
    """
    info_dict = json.loads(info_json)

    # Write the code to a temp file and run it.
    with tempfile.NamedTemporaryFile(prefix="my_script_", suffix=".py") as f:
        f.write(info_dict["code"].encode())
        f.flush()

        exec_cmd = f"{info_dict.get('dependencies', '')} && python {f.name}".strip(" &")
        proc = subprocess.run(exec_cmd, shell=True, capture_output=True)

	  # Add two new fields to the LLM input.
    output = {"stdout": proc.stdout.decode(), "stderr": proc.stderr.decode()}
    info_dict.update({k: ('...\n' + v[-2000:]) if len(v) > 2000 else v for k, v in output.items()})

    return json.dumps(info_dict)

Inputs for sandbox (i.e., the string info_json) will come from an AI. The outputs, on the other hand, will be used by another AI. Importantly, however, for sandbox to run any useful scientific code, we’ll need a capable environment with the appropriate hardware and software. This is where Covalent comes in.

Using Covalent to access cloud compute

In the next code cell, we use langchain.llms.CovalentCloud to transform sandbox into a LangChain-compatible Covalent workflow. This class exposes the same interface as langchain.llms.Covalent, which in turn targets the open-source version.

import covalent_cloud as cc

# Executor to define resources for running the code.
cloud_exec = cc.CloudExecutor(
    env="my_env-3.9", gpu_type="h100", num_gpus=10, num_cpus=64, memory="32GB"
) 

# Create the LangChain LLM component. 
covalent_sandbox = CovalentCloud(
    func=sandbox,
    executor=cloud_exec,
    covalent_cloud_api_key="MY-API-KEY"
)

Details of the execution environment are codified in the CloudExecutor instance named cloud_exec. This object specifies a custom Python environment (”my_env-3.9”) and requests access to several CPUs and H100 GPUs, together with 32 GB of memory. These resources require no reservation and are provisioned on-demand. If the specified environment does not yet exist in the user’s Covalent Cloud account, then it, too, can be created here straightforwardly by using the cc.create_env utility (see here).

Note that we can pass any regular function (like sandbox), or a Covalent electron or lattice as the func argument. The optional executor argument overrides any existing executors associated with electrons or lattices. For regular functions, as in our case, it creates a minimal one-step, one-executor Covalent workflow from func.

Demonstrating basic usage

We now have a LangChain “LLM” (covalent_sandbox) that, when invoked with a JSON-able object, remotely executes the code inside that object with resources of our choosing; returning an updated JSON object that contains the outputs.

Here’s a toy preview of how that works in LangChain:

from operator import itemgetter
from langchain.schema.runnable import RunnableLambda

covalent_sandbox = CovalentCloud(func=sandbox, executor=cloud_exec)
# covalent_sandbox = Covalent(func=sandbox, executor="awsbatch")

chain = (
    {
        "code": itemgetter("code"),
        "dependencies": itemgetter("dependencies"),
    }
    | RunnableLambda(json.dumps)
    | covalent_sandbox  # sandboxed execution on remote hardware
)

# Invoke the chain.
chain.invoke(
    {
        "code": "print('hello world')",
        "dependencies": "",
    }
)

Output:

'{"code": "print(\'hello world\')", "dependencies": "", "stdout": "hello world\\n", "stderr": ""}'`

We’ll build upon this basic idea to craft our LangChain agent in the sections that follow.

At this point, we’ve created a high-performance cloud compute sandbox without manual infrastructure provisioning, network configuration, or any file transfers. Covalent takes care of all this automatically.

Building an LLM code generator agent for materials science

Toy examples aside, the LangChain agent will consist of three main components:

First-stage LLM
- This LLM is prompted with a code description and optional constraints. Its job is to generate the initial code (and a preparatory shell command), as well as a natural language explanation of the generated code.
The Covalent sandbox
- Safely executes the AI generated code in a user-specified environment and captures the code outputs. These outputs inform the response from the next component.
Second-stage LLM
- This LLM is prompted with the code snippet and AI generated explanation from A., along with the stdout and stderr contents captured by B. This LLM’s job is to re-evaluate the code against the explanation and outputs. If changes are needed, the response will include a non-empty list of AI-generated suggestions.

Here is the agent implementation via the LangChain expression language (LCEL).

agent = (
    {
        "description": itemgetter("description"),
        "user_constraints": lambda x: "\n".join(x.get("user_constraints", ["None"])),
        "ai_suggestions": lambda x: "\n".join(x.get("ai_suggestions", final_parser.suggestions)),
        "chat_history": lambda x: "\n\n".join(x.get("chat_history", final_parser.history)),
    }
    | prompt_1  # agent component (A)
    | llm_1
    | output_parser_1
    | RunnableLambda(lambda x: x.json())  # agent component (B)
    | covalent_sandbox
    | RunnableLambda(json.loads)
    | prompt_2  # agent component (C)
    | llm_2
    | output_parser_2
    | final_parser  # check exit condition; see full code
)

For brevity, we’ve omitted the definitions of several components from this code cell. The output schemas output_schema_{1,2} are standard [PydanticOutputParser](<https://python.langchain.com/docs/modules/model_io/output_parsers/types/pydantic>) instances, whereas the FinalOutputParser inherits from [AgentOutputParser](<https://api.python.langchain.com/en/latest/agents/langchain.agents.agent.AgentOutputParser.html>) to return either an [AgentAction](<https://api.python.langchain.com/en/latest/agents/langchain_core.agents.AgentAction.html>) or an [AgentFinish](<https://api.python.langchain.com/en/latest/agents/langchain_core.agents.AgentFinish.html#langchain_core.agents.AgentFinish>) from its parse method, based on the presence or absence of AI suggestions from the second-stage LLM, i.e. agent component (C).

We used OpenAI’s GPT3.5 for both the first- and second-stage LLM; though it is easy in LangChain to swap in another chat model of your choosing. Please see the full code for complete definitions.

The agent is then wrapped in an AgentExecutor to finalize the iterative code-refiner.

@tool
def measure_suggestion_volume(suggestions: str) -> int:
    """Quantify the volume of suggestions to estimate how problematic the code may be."""
    return len(suggestions)


def generate_code_executor(
    description: str,
    user_constraints: Optional[List[str]] = None,
    max_iterations: int = 4,
) -> Dict[str, Any]:
    """We'll use this function for scope-safe invokation of the agent"""

    final_parser.suggestions = []

    agent_executor = AgentExecutor.from_agent_and_tools(
        agent=agent,
        tools=[measure_suggestion_volume],
        verbose=True,
        max_iterations=max_iterations,
    )

    result = agent_executor.invoke(
        {
            "description": description,
            "user_constraints": user_constraints,
        },
    )
    return result

Prompt engineering

Effective prompt engineering enhances the model’s ability to interpret requests accurately, leading to more reliable and useful applications. It involves understanding the model’s strengths and limitations, and creatively formulating prompts that guide the model towards desired outcomes. This can include specifying context, providing examples, or asking questions in a certain way.

We used OpenAI’s ChatGPT to create the agent in this blog post, so we followed this guide when designing our prompts. The code cells that create the prompts for the first- and second-stage LLMs can be seen below. In both cases, the field {format_instructions} is populated by a LangChain PydanticOutputParser (i.e., output_parser_1 and output_parser_2, from earlier).

This essentially inserts some text that tells the LLM to respond in valid JSON. Please refer to the full code for further context.

Prompt for LLM #1

from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder

SYSTEM_PROMPT_1 = """\
You will receive three things
(1) a description or request for generating code
(2) a list of zero or more user constraints
(3) a list of zero or more suggestions from an AI like yourself
(4) a list of zero or more responses from an AI like yourself

Your job is generate code that adheres to all user constraints while also considering the AI suggestions, which hint at potential issues, and avoiding any issues in previous responses.

{format_instructions}

Remember to respond ONLY in JSON!
"""

USER_PROMPT_1 = """
(1) {description}

(2) Adhere to these constraints:

{user_constraints}

(3) Consider these AI suggestions:

{ai_suggestions}

(4) Consider previous AI responses:

{chat_history}

"""

prompt_1 = ChatPromptTemplate.from_messages(
    [
        ("system", SYSTEM_PROMPT_1),
        ("user", USER_PROMPT_1),
    ]
)
prompt_1.input_variables = [
    "description",
    "user_constraints",
    "ai_suggestions",
    "chat_history",
]

# Output
_code = "the code you have generated, as a string"
_explanation = "a point by point explanation of what your code does under what user constraints"
_dependencies = "an empty string OR any shell command(s) to prepare the execution environment and/or install any dependencies; e.g., `cp my_data /path/to/data/my_data && pip install package==1.2.3`"

class LLMOutput1(BaseModel):
    """Schema for parsing the LLM's output"""
    code: str = Field(description=_code)
    explanation: str = Field(description=_explanation)
    dependencies: str = Field(description=_dependencies)

output_parser_1 = PydanticOutputParser(pydantic_object=LLMOutput1)
prompt_1.partial_variables = {
    "format_instructions": output_parser_1.get_format_instructions,
}

Prompt for LLM #2

SYSTEM_PROMPT_2 = """\
You will receive five things:

  (1) some code
  (2) an explanation of what the code does or is supposed to do
  (3) optional shell commands to prepare the execution environment
  (4) the stdout output of the code
  (5) the stderr output of the code

Your job is to evaluate the code and any shell command(s) against the explanation and outputs.

If the outputs fit the code explanation: return the code as is and do not make any suggestions! Otherwise, include a list of suggestions in your response. Always avoid making suggestions that conflict with explicit constraints.

{format_instructions}

Remember: Respond ONLY in JSON!
"""


USER_PROMPT_2 = """\
This code...
```
{code}
```
is supposed to...

{explanation}

The execution environment is prepared with the following shell command:
```shell
{dependencies}
```
Currently, the code produces the outputs below.

stdout:
```
{stdout}
```
stderr:
```
{stderr}
```
"""

prompt_2 = ChatPromptTemplate.from_messages(
    [
        ("system", SYSTEM_PROMPT_2),
        ("user", USER_PROMPT_2),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ],
)
prompt_2.input_variables = [
    "code",
    "explanation",
    "dependencies",
    "stdout",
    "stderr",
    "agent_scratchpad",
]

Generating high-compute code

See below for a sample generation run using the agent that we’ve prepared.

result = generate_code_executor(
    description=(
        "Use ASE and GPAW to calculate the potential energy of bulk FCC aluminum."
    ),
    user_constraints=[
        "write output files to /tmp",
        "use the command `gpaw install-data /tmp --gpaw --no-register` to install needed data",
        "export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000' inside the dependencies command",
        "repeat the calculation using 100, 200, ..., 500 PWs in the basis set",
    ],
)

print(result["code"])
print("\n", result["dependencies"])

Output:

from ase.build import bulk
from gpaw import GPAW, PW

for ecut in range(100, 600, 100):
    # Define the system
    atoms = bulk('Al', 'fcc', a=4.05)

    # Define the calculator
    calc = GPAW(mode=PW(ecut),
                xc='PBE',
                txt='/tmp/gpaw_output_ecut_'+str(ecut)+'.txt')

    atoms.set_calculator(calc)

    # Calculate potential energy
    potential_energy = atoms.get_potential_energy()
    print('Potential energy for ecut', ecut, 'is', potential_energy, 'eV')

pip install ase gpaw && gpaw install-data /tmp --gpaw --no-register && export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000'

LangChain Output

> Entering new AgentExecutor chain...


--- PROMPT (openai-chat) ---
You will receive three things
(1) a description or request for generating code
(2) a list of zero or more user constraints
(3) a list of zero or more suggestions from an AI like yourself
(4) a list of zero or more responses from an AI like yourself

Your job is generate code that adheres to all user constraints while also considering the AI suggestions, which hint at potential issues, and avoiding any issues in previous responses.

The output should be formatted as a JSON instance that conforms to the JSON schema below.

As an example, for the schema {"properties": {"foo": {"title": "Foo", "description": "a list of strings", "type": "array", "items": {"type": "string"}}}, "required": ["foo"]}
the object {"foo": ["bar", "baz"]} is a well-formatted instance of the schema. The object {"properties": {"foo": ["bar", "baz"]}} is not well-formatted.

Here is the output schema:
```
{"description": "Schema for parsing the LLM's output", "properties": {"code": {"title": "Code", "description": "the code you have generated, as a string", "type": "string"}, "explanation": {"title": "Explanation", "description": "a point by point explanation of what your code does under what user constraints", "type": "string"}, "dependencies": {"title": "Dependencies", "description": "an empty string OR any shell command(s) to prepare the execution environment and/or install any dependencies; e.g., `cp my_data /path/to/data/my_data && pip install package==1.2.3`", "type": "string"}}, "required": ["code", "explanation", "dependencies"]}
```

Remember to respond ONLY in JSON!

Human: 
(1) Use ASE and GPAW to calculate the potential energy of bulk FCC aluminum.

(2) Adhere to these constraints:

write output files to /tmp
use the command `gpaw install-data /tmp --gpaw --no-register` to install needed data
export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000' inside the dependencies command
repeat the calculation using 100, 200, ..., 500 PWs in the basis set

(3) Consider these AI suggestions:



(4) Consider previous AI responses:


--- RESPONSE (openai-chat) ---
{"code": "from ase.build import bulk\nfrom gpaw import GPAW, PW\n\nfor ecut in range(100, 600, 100):\n    # Define the system\n    atoms = bulk('Al', 'fcc', a=4.05)\n\n    # Define the calculator\n    calc = GPAW(mode=PW(ecut),\n                xc='PBE',\n                txt='/tmp/gpaw_output_ecut_'+str(ecut)+'.txt')\n\n    atoms.set_calculator(calc)\n\n    # Calculate potential energy\n    potential_energy = atoms.get_potential_energy()\n    print('Potential energy for ecut', ecut, 'is', potential_energy, 'eV')", "explanation": "The code first imports the necessary modules from ASE and GPAW. Then, for each plane wave cutoff energy from 100 to 500 in steps of 100, it defines a bulk aluminum system using ASE. For each system, it sets up a GPAW calculator with the plane wave mode set to the current cutoff energy, the exchange-correlation functional set to PBE, and the output written to a file in the /tmp directory. It then sets this calculator to the current atoms object and calculates the potential energy, printing the result.", "dependencies": "pip install ase gpaw && gpaw install-data /tmp --gpaw --no-register && export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000'"}


--- PROMPT (covalent-cloud) ---
"from ase.build import bulk\nfrom gpaw import GPAW, PW\n\nfor ecut in range(100, 600, 100):\n    # Define the system\n    atoms = bulk('Al', 'fcc', a=4.05)\n\n    # Define the calculator\n    calc = GPAW(mode=PW(ecut),\n                xc='PBE',\n                txt='/tmp/gpaw_output_ecut_'+str(ecut)+'.txt')\n\n    atoms.set_calculator(calc)\n\n    # Calculate potential energy\n    potential_energy = atoms.get_potential_energy()\n    print('Potential energy for ecut', ecut, 'is', potential_energy, 'eV')", "explanation": "The code first imports the necessary modules from ASE and GPAW. Then, for each plane wave cutoff energy from 100 to 500 in steps of 100, it defines a bulk aluminum system using ASE. For each system, it sets up a GPAW calculator with the plane wave mode set to the current cutoff energy, the exchange-correlation functional set to PBE, and the output written to a file in the /tmp directory. It then sets this calculator to the current atoms object and calculates the potential energy, printing the result.", "dependencies": "pip install ase gpaw && gpaw install-data /tmp --gpaw --no-register && export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000'"}


--- RESPONSE (covalent-cloud) ---
{"code": "from ase.build import bulk\nfrom gpaw import GPAW, PW\n\nfor ecut in range(100, 600, 100):\n    # Define the system\n    atoms = bulk('Al', 'fcc', a=4.05)\n\n    # Define the calculator\n    calc = GPAW(mode=PW(ecut),\n                xc='PBE',\n                txt='/tmp/gpaw_output_ecut_'+str(ecut)+'.txt')\n\n    atoms.set_calculator(calc)\n\n    # Calculate potential energy\n    potential_energy = atoms.get_potential_energy()\n    print('Potential energy for ecut', ecut, 'is', potential_energy, 'eV')", "explanation": "The code first imports the necessary modules from ASE and GPAW. Then, for each plane wave cutoff energy from 100 to 500 in steps of 100, it defines a bulk aluminum system using ASE. For each system, it sets up a GPAW calculator with the plane wave mode set to the current cutoff energy, the exchange-correlation functional set to PBE, and the output written to a file in the /tmp directory. It then sets this calculator to the current atoms object and calculates the potential energy, printing the result.", "dependencies": "pip install ase gpaw && gpaw install-data /tmp --gpaw --no-register && export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000'", "stdout": "...\n.1 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (1.4.5)\nRequirement already satisfied: packaging>=20.0 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (23.2)\nRequirement already satisfied: pillow>=8 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (10.2.0)\nRequirement already satisfied: pyparsing>=2.3.1 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (3.1.1)\nRequirement already satisfied: python-dateutil>=2.7 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (2.8.2)\nRequirement already satisfied: importlib-resources>=3.2.0 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (6.1.1)\nRequirement already satisfied: zipp>=3.1.0 in ./lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib>=3.1.0->ase) (3.17.0)\nRequirement already satisfied: six>=1.5 in ./lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib>=3.1.0->ase) (1.16.0)\nAvailable setups and pseudopotentials\n  [*] https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz\n      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.11271.tar.gz\n      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.9672.tar.gz\n      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.8.7929.tar.gz\n      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.6.6300.tar.gz\n      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.5.3574.tar.gz\n\nSelected gpaw-setups-0.9.20000.tar.gz.  Downloading...\nExtracting tarball into /tmp\nSetups installed into /tmp/gpaw-setups-0.9.20000.\nYou can manually register the setups by adding the\nfollowing line to /home/covalent/.gpaw/rc.py:\n\nsetup_paths.insert(0, '/tmp/gpaw-setups-0.9.20000')\n\nInstallation complete.\nPotential energy for ecut 100 is 1.357932848777831 eV\nPotential energy for ecut 200 is 1.2351350509262142 eV\nPotential energy for ecut 300 is 1.2311083959401117 eV\nPotential energy for ecut 400 is 1.2300249417145923 eV\nPotential energy for ecut 500 is 1.229695457094227 eV\n", "stderr": ""}


--- PROMPT (openai-chat) ---
You will receive five things:

  (1) some code
  (2) an explanation of what the code does or is supposed to do
  (3) optional shell commands to prepare the execution environment
  (4) the stdout output of the code
  (5) the stderr output of the code

Your job is to evaluate the code and any shell command(s) against the explanation and outputs.

If the outputs fit the code explanation: return the code as is and do not make any suggestions! Otherwise, include a list of suggestions in your response. Always avoid making suggestions that conflict with explicit constraints.

The output should be formatted as a JSON instance that conforms to the JSON schema below.

As an example, for the schema {"properties": {"foo": {"title": "Foo", "description": "a list of strings", "type": "array", "items": {"type": "string"}}}, "required": ["foo"]}
the object {"foo": ["bar", "baz"]} is a well-formatted instance of the schema. The object {"properties": {"foo": ["bar", "baz"]}} is not well-formatted.

Here is the output schema:
```
{"description": "Schema for parsing the LLM's output", "properties": {"ai_suggestions": {"title": "Ai Suggestions", "description": "either a list of your suggestions for improving the code or null if you don't have any", "type": "array", "items": {"type": "string"}}, "code": {"title": "Code", "description": "your improved version of the code", "type": "string"}, "dependencies": {"title": "Dependencies", "description": "an empty string OR any shell command(s) to prepare the execution environment and/or install any dependencies; e.g., `cp my_data /path/to/data/my_data && pip install package==1.2.3`", "type": "string"}, "chat_history": {"title": "Chat History", "default": [], "type": "array", "items": {"$ref": "#/definitions/AIMessage"}}}, "required": ["code", "dependencies"], "definitions": {"AIMessage": {"title": "AIMessage", "description": "A Message from an AI.", "type": "object", "properties": {"content": {"title": "Content", "anyOf": [{"type": "string"}, {"type": "array", "items": {"anyOf": [{"type": "string"}, {"type": "object"}]}}]}, "additional_kwargs": {"title": "Additional Kwargs", "type": "object"}, "type": {"title": "Type", "default": "ai", "enum": ["ai"], "type": "string"}, "example": {"title": "Example", "default": false, "type": "boolean"}}, "required": ["content"]}}}
```

Remember to respond ONLY in JSON!

Human: (1) Consider the code

```
from ase.build import bulk
from gpaw import GPAW, PW

for ecut in range(100, 600, 100):
    # Define the system
    atoms = bulk('Al', 'fcc', a=4.05)

    # Define the calculator
    calc = GPAW(mode=PW(ecut),
                xc='PBE',
                txt='/tmp/gpaw_output_ecut_'+str(ecut)+'.txt')

    atoms.set_calculator(calc)

    # Calculate potential energy
    potential_energy = atoms.get_potential_energy()
    print('Potential energy for ecut', ecut, 'is', potential_energy, 'eV')
```

(2) and the following explanation:

The code first imports the necessary modules from ASE and GPAW. Then, for each plane wave cutoff energy from 100 to 500 in steps of 100, it defines a bulk aluminum system using ASE. For each system, it sets up a GPAW calculator with the plane wave mode set to the current cutoff energy, the exchange-correlation functional set to PBE, and the output written to a file in the /tmp directory. It then sets this calculator to the current atoms object and calculates the potential energy, printing the result.

(3) The execution environment is prepared with the following (optional) shell command:

```shell
pip install ase gpaw && gpaw install-data /tmp --gpaw --no-register && export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000'
```

Currently, the code produces these outputs...

(4) stdout:
```
...
.1 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (1.4.5)
Requirement already satisfied: packaging>=20.0 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (23.2)
Requirement already satisfied: pillow>=8 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (10.2.0)
Requirement already satisfied: pyparsing>=2.3.1 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (2.8.2)
Requirement already satisfied: importlib-resources>=3.2.0 in ./lib/python3.9/site-packages (from matplotlib>=3.1.0->ase) (6.1.1)
Requirement already satisfied: zipp>=3.1.0 in ./lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib>=3.1.0->ase) (3.17.0)
Requirement already satisfied: six>=1.5 in ./lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib>=3.1.0->ase) (1.16.0)
Available setups and pseudopotentials
  [*] https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.11271.tar.gz
      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.9672.tar.gz
      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.8.7929.tar.gz
      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.6.6300.tar.gz
      https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.5.3574.tar.gz

Selected gpaw-setups-0.9.20000.tar.gz.  Downloading...
Extracting tarball into /tmp
Setups installed into /tmp/gpaw-setups-0.9.20000.
You can manually register the setups by adding the
following line to /home/covalent/.gpaw/rc.py:

setup_paths.insert(0, '/tmp/gpaw-setups-0.9.20000')

Installation complete.
Potential energy for ecut 100 is 1.357932848777831 eV
Potential energy for ecut 200 is 1.2351350509262142 eV
Potential energy for ecut 300 is 1.2311083959401117 eV
Potential energy for ecut 400 is 1.2300249417145923 eV
Potential energy for ecut 500 is 1.229695457094227 eV

```

(5) stderr:
```

```


--- RESPONSE (openai-chat) ---
{"ai_suggestions": null, "code": "from ase.build import bulk\nfrom gpaw import GPAW, PW\n\nfor ecut in range(100, 600, 100):\n    # Define the system\n    atoms = bulk('Al', 'fcc', a=4.05)\n\n    # Define the calculator\n    calc = GPAW(mode=PW(ecut),\n                xc='PBE',\n                txt='/tmp/gpaw_output_ecut_'+str(ecut)+'.txt')\n\n    atoms.set_calculator(calc)\n\n    # Calculate potential energy\n    potential_energy = atoms.get_potential_energy()\n    print('Potential energy for ecut', ecut, 'is', potential_energy, 'eV')", "dependencies": "pip install ase gpaw && gpaw install-data /tmp --gpaw --no-register && export GPAW_SETUP_PATH='/tmp/gpaw-setups-0.9.20000'", "chat_history": []}

Finished chain.

Conclusion

In this blog post, we used the new Covalent integrations for LangChain to create an iterative code-refining LLM agent endowed with a powerful sandbox for testing AI generated code. Covalent helped us immensely in this endeavour by providing a simple interface for launching cloud workflows. Making use of the fully-managed Covalent Cloud platform, we avoided all of the crucial plumbing that’s typically necessary for smooth and secure cloud-based computing. This allowed us to concentrate at the problem at hand, which proved, not unexpectedly, to be an exercise in effective prompt in engineering.

Our agent performed reasonably well, considering the minimal implementation that we outlined above. It’s worth emphasizing that the use of Covalent for sandboxing in LangChain is just one (rather generic) example among countless possible applications. The Covalent and CovalentCloud interfaces are fully capable of running virtually any function or Covalent workflow, and preserve all the existing features of Covalent, as they are mere wrappers for the underlying platform. One can envision, for example, agent-based automation centred on more specific and complex high-compute workflows (e.g. in computational chemistry, myriad problems in physics, or biological systems simulations), with Covalent providing a tailored cloud backend for executing any related code.

Cookie	Duration	Description
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.