# Bureaucracy Escape Room - OpenEnv Hackathon

A reinforcement learning environment that simulates permit paperwork, where an agent must navigate departments, collect documents and resolve dependencies, built for the OpenEnv Hackathon.

- Role: Sole Developer (RL Environment)
- Date: Mar 2026 – Apr 2026
- Categories: hackathon
- Tech: Python, Docker, OpenAI API, Reinforcement Learning, Hugging Face Spaces
- GitHub: https://github.com/AmulyaJain2004/HackOpenEnv
- Live: https://huggingface.co/spaces/Alokik089/bureaucracy_escape_room

## Overview

The OpenEnv Hackathon asked participants to build real-world environments for training and evaluating agents. "Bureaucracy Escape Room" models a familiar problem: obtaining a permit by moving between departments, gathering the right documents and satisfying requirements that depend on one another.

## System Design

```mermaid
flowchart TD
    Agent["Baseline agent (inference.py)<br/>task-aware planner + gpt-4o-mini fallback"] -->|"RlAction"| Client["RlEnv client<br/>OpenEnv HTTP client"]
    Client --> Server["OpenEnv FastAPI server<br/>up to 8 concurrent environments"]
    Server --> Env["RlEnvironment<br/>reset() · step() · state()"]
    Env --> Tasks[("tasks.py<br/>departments · actions · requires · gives")]
    Env --> Check{"Valid action and<br/>requirements met?"}
    Check -->|"no"| Reject["0 reward + hint<br/>naming the missing items"]
    Check -->|"yes"| Apply["Update inventory, milestones, department<br/>reward − step penalty − loop penalty"]
    Apply --> Win{"Permit issued<br/>or step budget hit?"}
    Win --> Obs["RlObservation<br/>message · inventory · available actions · reward"]
    Reject --> Obs
    Obs --> Client
    Win -->|"episode ends"| Grader["grade_episode()<br/>progress · success · efficiency"]
```

- **Server:** the environment runs behind the OpenEnv HTTP server (FastAPI), packaged in Docker and deployed as a Hugging Face Space, with typed action and observation models shared by client and server.
- **Data-driven tasks:** each task is a dictionary of departments, and each department lists its actions with what they require, what documents they give, which milestone they complete and where they lead. Adding a task means adding data, not code.
- **Step logic:** invalid actions and missing requirements return zero reward with a hint, valid actions update the inventory and apply the step and loop penalties, and an episode ends when the permit is issued or the step budget runs out.
- **Baseline agent:** a task-aware planner picks actions where it can, breaks out of repeated-action loops, and falls back to gpt-4o-mini otherwise. It logs every step in a structured START / STEP / END format.

## Environment Design

- **Tasks:** three deterministic tasks of increasing difficulty. A dog licence has a short linear chain of steps. A business permit adds circular dependencies with a hidden workaround. A construction permit requires coordinating several departments under ordering constraints, with substitute documents.
- **Observations:** the agent sees its current department, feedback from the clerk, collected documents, completed milestones, the actions available in the current state and the step count, with occasional hints.
- **Actions:** the agent chooses from the actions available in its current state and may attach a short natural-language note.
- **Rewards:** dense per-step progress rewards, step penalties for inefficiency, penalties for repeating the same action, and a bonus for a successful permit, all normalised to the range 0 to 1.

## Evaluation

Each episode is graded by a deterministic function that scores milestone progress, final success and step efficiency between 0 and 1. As a baseline, a gpt-4o-mini agent completed all three tasks, scoring 0.68 on the dog licence in 8 steps, 0.72 on the business permit in 15 steps and 0.68 on the construction permit in 21 steps.

## Status

This was a solo hackathon project: I designed and built the environment, reward structure, grader and deployment myself. It is packaged with Docker, includes validation scripts, and is deployed as a Hugging Face Space.

---

Back to the full index: /agent