From de524be99e5369e234d91f1e52c0d46dec360e00 Mon Sep 17 00:00:00 2001 From: alexisxy Date: Mon, 29 Apr 2024 18:19:59 -0400 Subject: [PATCH] notes on setup and reset environment --- README.md | 4 ++++ environment_docker/README.md | 15 +++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/README.md b/README.md index d0c37ad..b201071 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,9 @@ action = create_id_based_action(f"click [id]") obs, _, terminated, _, info = env.step(action) ``` ## End-to-end Evaluation +> [!IMPORTANT] +> To ensure the correct evaluation, please setup your own WebArena websites following step 1 and step 2. The demo sites are only for browsing purpose to help you better understand the content. After evaluating the 812 examples, reset the environment to the initial state following the instructions [here](./environment_docker/README.md#environment-reset). + 1. Setup the standalone environment. Please check out [this page](environment_docker/README.md) for details. @@ -110,6 +113,7 @@ python run.py \ ``` This script will run the first example with GPT-3.5 reasoning agent. The trajectory will be saved in `/0.html` + ## Develop Your Prompt-based Agent 1. Define the prompts. We provide two baseline agents whose corresponding prompts are listed [here](./agent/prompts/raw). Each prompt is a dictionary with the following keys: ```python diff --git a/environment_docker/README.md b/environment_docker/README.md index 92d1f1e..93f8886 100644 --- a/environment_docker/README.md +++ b/environment_docker/README.md @@ -3,6 +3,7 @@ This REAME file host the instructions for our Docker images and quick start guid # Table of Content - [Pre-installed Amazon Machine Image](#pre-installed-amazon-machine-image) +- [Environment Reset](#environment-reset) - [Shopping Website (OneStopShop)](#shopping-website--onestopshop-) - [E-commerce Content Management System (CMS)](#e-commerce-content-management-system--cms-) - [Social Forum Website (Reddit)](#social-forum-website--reddit-) @@ -61,6 +62,20 @@ docker exec gitlab gitlab-ctl reconfigure You should be able to access your environment websites now, and stop reading. However, if you are unable to use AWS AMI, read below to set up on your own machine. +## Environment Reset +After evaluating the 812 examples, reset the environment to the initial state +```bash +# stop and remove the images +docker stop shopping_admin forum gitlab shopping +docker remove shopping_admin forum gitlab shopping +# start the images +docker start gitlab +docker start shopping +docker start shopping_admin +docker start forum + +``` + ## Shopping Website (OneStopShop) Download the image tar from the following mirrors: