Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
294 changes: 294 additions & 0 deletions docs/tradeSiteFiltering.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,294 @@
# Path of Exile Trade Site Filtering System

## Overview

The Path of Exile trade site uses a JSON-based query system to filter items. This document explains how the filtering works based on the Path of Building implementation.

## URL Structure

The trade site supports two URL formats:

1. **Query ID format**: `https://www.pathofexile.com/trade/search/{league}/{queryId}`
- Example: `https://www.pathofexile.com/trade/search/Keepers/8rlgmrO8FV`
- The `queryId` (e.g., `8rlgmrO8FV`) is generated by the trade site when you submit a search
- This ID can be used to retrieve the search results without re-submitting the query

2. **Encoded query format**: `https://www.pathofexile.com/trade/search/{league}/?q={encodedQuery}`
- The `q` parameter contains a URL-encoded JSON query object
- This format allows direct sharing of search parameters

## Query JSON Structure

The query is a JSON object with the following structure:

```json
{
"query": {
"filters": {
"type_filters": {
"filters": {
"category": { "option": "armour.chest" },
"rarity": { "option": "nonunique" }
}
},
"misc_filters": {
"disabled": false,
"filters": {
"mirrored": false
}
},
"trade_filters": {
"filters": {
"price": {
"option": "chaos",
"max": 50
}
}
},
"socket_filters": {
"disabled": false,
"filters": {
"sockets": {
"min": 4,
"max": 6
},
"links": {
"min": 4,
"max": 6
}
}
},
"req_filters": {
"disabled": false,
"filters": {
"lvl": {
"max": 80
}
}
}
},
"status": { "option": "available" },
"stats": [
{
"type": "weight",
"value": { "min": 1000 },
"filters": [
{ "id": "stat_id_1", "value": { "weight": 1.5 } },
{ "id": "stat_id_2", "value": { "weight": 2.0 } }
]
}
]
},
"sort": {
"statgroup.0": "desc"
},
"engine": "new"
}
```

## Key Components

### 1. Type Filters
- **category**: Item category (e.g., `armour.chest`, `weapon.bow`, `accessory.ring`)
- **rarity**: Item rarity (`nonunique`, `unique`, etc.)

### 2. Stat Filters
The `stats` array contains stat groups that can be:
- **type: "weight"**: Weighted search where each stat has a weight multiplier
- **type: "and"**: All stats in the group must match
- **type: "count"**: Count of matching stats
- **type: "if"**: Conditional stat matching

Each stat filter has:
- **id**: The trade site's internal stat ID (fetched from `/api/trade/data/stats`)
- **value**: Can contain `min`, `max`, or `weight` depending on the filter type

### 3. Trade Filters
- **price**: Maximum price in specified currency
- **listing_time**: How recently the item was listed

### 4. Socket Filters
- **sockets**: Number of sockets (min/max)
- **links**: Number of linked sockets (min/max)

### 5. Requirement Filters
- **lvl**: Maximum level requirement

### 6. Miscellaneous Filters
- **mirrored**: Whether to include mirrored items
- **corrupted**: Corruption status
- **quality**: Item quality percentage

## How Path of Building Generates Queries

### 1. Stat ID Mapping
Path of Building fetches stat definitions from the trade API:
```lua
-- Fetches from: https://www.pathofexile.com/api/trade/data/stats
local tradeStats = fetchStats()
```

This provides a mapping between stat text (e.g., "+#% to Fire Resistance") and stat IDs used in queries.

### 2. Query Generation Process

1. **Mod Weight Calculation**: For each mod that can appear on the item:
- Creates a test item with that mod
- Calculates the stat difference compared to base item
- Assigns a weight based on configured stat priorities

2. **Query Assembly**:
- Sets item category based on slot type
- Adds stat filters with calculated weights
- Applies user options (price, level, sockets, etc.)

3. **URL Encoding**: The JSON query is URL-encoded using percent encoding:
```lua
function urlEncode(str)
local charToHex = function(c)
return s_format("%%%02X", string.byte(c))
end
return str:gsub("([^%w_%-.~])", charToHex)
end
```

### 3. Example Query Generation

From `TradeQueryGenerator.lua`:

```lua
local queryTable = {
query = {
filters = {
type_filters = {
filters = {
category = { option = "armour.chest" },
rarity = { option = "nonunique" }
}
}
},
status = { option = "available" },
stats = {
{
type = "weight",
value = { min = minWeight },
filters = { }
}
}
},
sort = { ["statgroup.0"] = "desc" },
engine = "new"
}

-- Add stat filters
for _, entry in pairs(self.modWeights) do
table.insert(queryTable.query.stats[1].filters, {
id = entry.tradeModId,
value = { weight = entry.weight }
})
end

local queryJson = dkjson.encode(queryTable)
local url = "https://www.pathofexile.com/trade/search/" .. league .. "/?q=" .. urlEncode(queryJson)
```

## Extracting Query from Query ID

When you have a URL with a query ID (like `8rlgmrO8FV`), Path of Building extracts the query by:

1. **Fetching the HTML page**: `https://www.pathofexile.com/trade/search/{league}/{queryId}`
2. **Extracting JSON from HTML**: The page contains embedded JSON in a specific pattern:
```lua
local dataStr = response.body:match('require%(%["main"%].+ t%((.+)%);}%);}%);')
```
3. **Parsing the state**: The JSON contains a `state` object with the query:
```lua
local data = dkjson.decode(dataStr)
local query = { query = data.state }
```

## Stat Categories

The trade site organizes stats into categories:
- **Explicit** (index 2): Regular item mods
- **Implicit** (index 3): Base item implicits
- **Corrupted** (index 3): Corrupted item mods
- **Scourge** (index 6): Scourge mods
- **Eater** (index 3): Eater of Worlds mods
- **Exarch** (index 3): Searing Exarch mods
- **Synthesis** (index 3): Synthesis mods
- **PassiveNode** (index 2): Passive tree nodes (for cluster jewels)

## Weighted Searches

Path of Building uses weighted searches to find items that improve your build:

1. **Weight Calculation**: Each stat is weighted based on:
- How much it improves your build's key stats (DPS, EHP, etc.)
- User-configured stat priorities
- The stat's value range

2. **Minimum Weight**: The query sets a minimum weight threshold:
```lua
local minWeight = currentStatDiff * 0.5
```

3. **Sorting**: Results are sorted by weighted sum in descending order:
```json
"sort": { "statgroup.0": "desc" }
```

## Item Category Mapping

Path of Building maps item slots to trade site categories:

| Slot | Category |
|------|----------|
| Body Armour | `armour.chest` |
| Helmet | `armour.helmet` |
| Gloves | `armour.gloves` |
| Boots | `armour.boots` |
| Shield | `armour.shield` |
| Amulet | `accessory.amulet` |
| Ring | `accessory.ring` |
| Belt | `accessory.belt` |
| 1H Weapon | `weapon.one` |
| 2H Weapon | `weapon.two` |
| Bow | `weapon.bow` |
| Staff | `weapon.staff` |
| Jewel | `jewel` or `jewel.base` |
| Abyss Jewel | `jewel.abyss` |
| Flask | `flask` |

## API Endpoints

- **Search**: `POST https://www.pathofexile.com/api/trade/search/{realm}/{league}`
- Body: JSON query object
- Returns: `{ id: "queryId", result: ["itemHash1", ...], total: 1234 }`

- **Fetch Items**: `GET https://www.pathofexile.com/api/trade/fetch/{itemHashes}?query={queryId}`
- Returns: Full item details

- **Get Stats**: `GET https://www.pathofexile.com/api/trade/data/stats`
- Returns: List of all available stat filters with IDs

## Example: Decoding a Query ID URL

Given: `https://www.pathofexile.com/trade/search/Keepers/8rlgmrO8FV`

1. Extract league: `Keepers`
2. Extract query ID: `8rlgmrO8FV`
3. Fetch HTML: `GET https://www.pathofexile.com/trade/search/Keepers/8rlgmrO8FV`
4. Extract JSON from HTML using regex pattern
5. Parse `data.state` to get the query object
6. The query object contains all the filters that were used in the original search

## References

- Trade Query Generator: `src/Classes/TradeQueryGenerator.lua`
- Trade Query Requests: `src/Classes/TradeQueryRequests.lua`
- Trade Query: `src/Classes/TradeQuery.lua`
- URL Encoding: `src/Modules/Common.lua` (urlEncode function)



Loading