Skip to main content
Welcome to contribute to the browseruse-bench project! This guide will help you understand how to participate in project development.

​
Development Environment Setup

1

Fork Repository

Fork the browseruse-bench repository on GitHub
2

Clone Code

git clone https://github.com/your-username/browseruse-bench.git
cd browseruse-bench
3

Install Dependencies

# Using uv (Recommended)
uv sync --all-extras

# Or using pip
pip install -e ".[all,dev]"
4

Create Branch

git checkout -b feature/your-feature-name

​
Contribution Types

​
Adding New Agents

  1. Create a new Agent directory in agents/
  2. Implement the Agent interface
  3. Add config.yaml.example and keep config.yaml local
  4. Update documentation
See Adding New Agents for details.

​
Adding New Benchmarks

  1. Create a new Benchmark directory in benchmarks/
  2. Prepare task data and data_info.json
  3. Implement evaluator (optional)
  4. Update documentation
See Custom Benchmark for details.

​
Fixing Bugs

  1. Create an Issue to describe the problem
  2. Submit a PR with the fix
  3. Ensure tests pass

​
Improving Documentation

  1. Modify documentation in docs/ directory
  2. Submit PR

​
Adding New Agents

​
Directory Structure

agents/
└── YourAgent/
    ├── config.yaml.example  # Committed example config
    ├── config.yaml          # Local config (ignored)
    ├── requirements.txt # Dependencies (Optional)
    └── run.py           # Entry Script (Optional)
Use config.yaml.example in the repo and keep config.yaml local for secrets.

​
Implement Interface

New Agents need to implement the following interface: 
class YourAgent:
    def __init__(self, config: dict):
        """Initialize Agent"""
        pass
    
    async def execute_task(self, task: dict) -> dict:
        """
        Execute task
        
        Args:
            task: Task dictionary, containing task_id, task, etc.
            
        Returns:
            Result dictionary, containing action_history, metrics, etc.
        """
        pass

​
Register Agent

Register in browseruse_bench/agents/__init__.py: 
AGENTS = {
    "browser-use": ...,
    "Agent-TARS": ...,
    "YourAgent": {
        "module": "agents.YourAgent.run:YourAgent",
        "config_file": "agents/YourAgent/config.yaml"
    }
}

​
Code Standards

​
Formatting

# Use ruff to format
ruff format .

# Check code style
ruff check .

​
Type Checking

# Use mypy for type checking
mypy browseruse_bench

​
Testing

# Run all tests
pytest tests/

# Run specific test
pytest tests/test_eval.py -v

​
Submitting PR

1

Ensure Tests Pass

pytest tests/
ruff check .
2

Commit Changes

git add .
git commit -m "feat: add your feature description"
3

Push Branch

git push origin feature/your-feature-name
4

Create PR

Create a Pull Request on GitHub and describe your changes

​
Commit Convention

Use Conventional Commits format:
  • feat: New feature
  • fix: Bug fix
  • docs: Documentation update
  • refactor: Code refactoring
  • test: Add test
  • chore: Other changes
Example: 
feat: add WebArena benchmark support
fix: resolve timeout issue in browser-use agent
docs: update quickstart guide