Call of Duty 2: d3dbsp
The .d3dbsp structure, Call of Duty 2's variant on the well-known BSP format, is rather difficult to decipher. Since it's a binary file you can't simply read it. A hexadecimal editor is the best tool in this case. I am still deciphering it myself, I will post anything I happen to get known of regarding the file format.
Be sure to read here and here before you even try to decipher it. Knowing what the technique is that BSP formatted maps use, could be handy as well. The Call of Duty 2 engine is based on its ancestor, so is the BSP: Call of Duty 1: d3dbsp
Every number ending with an 'h' indicates its a number using the hexadecimal count system. For other numbers one can assume the decimal count system is used.
Numbers are stored in little-endian. The endianness describes the order in which a sequence of bytes are stored in computer memory. Little-endian is an order in which the "little end" (least significant value in the sequence) is stored first.
An useful however not documented feature of the cod2map.exe is the -info parameter. It analyzes the given BSP and prints the name, length and entry count of all lumps to console. One can use it for deductions regarding the lump and entry sizes and also to identify the kind of content.
Header
The file starts with 2 DWORDS: the header 'IBSP' indicating this is a BSP file originally designed by ID software and the version number 4 (4h).
Then there is an array, filled with DWORD pairs, indicating the lump's offset and length respectively. The array ends with 2 empty (zero) DWORDs. Each lump that has size 0 in the list is simply not filled / not used.
Note: there can be a difference of number of lumps per map. If a map has a leak
(no solid skybox or not everything is IN the skybox) the bsp only has 24 lumps (or even 23 if light is not
compiled). The bsp description yet only supports the 37-lumps version (a valid map).
Lumps
A BSP is organized in several sections, called lumps. Each lump stores data with different functionality. It's a data unit containing sub-sections, or better: entries. Every entry is a binary representations of something, a plane custom shot glasses for instance. Such a plane is stored as a normal vector (3 floats) and a distance (1 float). The length of the datatype "float" is 4 byte (DWORD), so the size of one plane entry is 4 DWORDs (4*4 = 16 bytes).
Lump[0] - Materials
former Textures
Texture information is 72 bytes long per texture. The first 64 bytes are used for the texture name. Then we have an DWORD for flags, and another DWORD for content flags.
struct Material
{
char name[64];
unsigned char flags[2][4];
};
Example:
d_beachpebble 0x00b00000 0x00000004 stalingrad_ground_trench 0x00e00000 0x00000001 duhoc_dirt_ground1 0x00600000 0x00000001
Flags:
Surface type:
-------------
- - <error> (asset manager will fail)
0x00000000 0x00000001 <none>
0x01600000 0x00000001 asphalt
0x00100000 0x00000001 bark
0x00200000 0x00000001 brick
0x00300000 0x00000001 carpet
0x00400000 0x00000001 cloth
0x00500000 0x00000001 concrete
0x00600000 0x00000001 dirt
0x00700000 0x00000001 flesh
0x00800000 0x00000002 foliage
0x00900000 0x00000010 glass
0x00a00000 0x00000001 grass
0x00b00000 0x00000001 gravel
0x00c00000 0x00000001 ice
0x00d00000 0x00000001 metal
0x00e00000 0x00000001 mud
0x00f00000 0x00000001 paper
0x01000000 0x00000001 plaster
0x01100000 0x00000001 rock
0x01200000 0x00000001 sand
0x01300000 0x00000001 snow
0x01400000 0x00000020 water
0x01500000 0x00000001 wood
Surface properties:
-------------------
0x00000000 0x00000080 missileClip
0x00000000 0x00002000 bulletClip
0x00000000 0x00010000 playerClip
0x00000000 0x00000200 vehicleClip
0x00000000 0x00020000 aiClip
0x00000000 0x00000400 itemClip
0x00000000 0x00000041 canShootClip
0x00000000 0x00001000 aiSightClip
0x00000001 0x00000001 noFallDamage
0x00002000 0x00000001 noSteps
0x00000010 0x00000001 noImpact
0x00000020 0x00000001 noMarks
0x00000000 0x80000000 noDrop
0x00000002 0x00000001 slick
0x00000008 0x00000001 ladder
0x02000000 0x01000001 mantleOn
0x04000000 0x01000001 mantleOver
0x00000000 0x00000001 noLightmap
0x00020000 0x00000001 noDynamicLight
0x00040000 0x00000001 noCastShadow
0x00000080 0x00000001 noDraw
0x00000000 0x00000001 noFog
0x00000000 0x00000001 drawToggle
0x00000004 0x00000800 sky
0x00000000 0x00000001 radialNormals
0x00000000 0x00000004 nonColliding
0x00004000 0x00000000 nonSolid
0x00000000 0x20000001 transparent \__ (two entries
0x00000000 0x28000001 transparent / added to lump)
0x00000000 0x08000001 detail
0x00000000 0x10000001 structural
0x80000000 0x00000000 portal
0x00000000 0x00000001 occluder
- - origin (no lump entry)
Combinations (examples):
0x00000000 0x00002080 missleClip & bulletClip 0x00000000 0xb0000000 noDrop & transparent & structural 0x00000000 0x90000000 noDrop & structural 0x00000004 0x20000800 transparent & sky \__ (two entries 0x00000004 0x28000800 transparent & sky / added to lump) 0x01400088 0x00000020 water & noFog & noDraw & ladder
These flags can be set in AssMan, nonetheless some Radiant functions influence them as well, e.g.
"Make Weapon Clip" on a wood-textured brush 0x01500000 0x00000001 \__ usual wood flags 0x01500000 0x00002080 / first flag like wood, second flag missleClip & bulletClip Note: the brush lost its collision (less planes in lump 4 etc.)
Lump[1] - Lightmaps
former World Lightning
Contains most of the pre-processed lightning. Not present if compiled without lightning, otherwise a multiple of 4.096 KB.
Without: World geometry is rendered overbright
Lump[2] - Light Grid Hash
former Light Volume
Precalculated (sun)light in the 3d space of the playable area, also called light grid. Each entry is 8 bytes long.
Without: a lot less light influence and a little rainbow colors on entities (e.g. player)
Lump[3] - Light Grid Values
former Model Lightning
Base (sun)light for model entities (ground lighting for xmodels). 24 bytes per entry.
Without: models show up very dark
Lump[4] - Planes
Each plane is defined by 4 DWORDs (16 bytes), 3 for the normal vector and 1 for the offset of the origin. Planes are faces with infinite length, we suspect they have to do with the rendering system.
struct Plane
{
float normal[3];
float distance;
};
Example:
0 | 0 0 1 128 1 | 0 0 1 320 2 | 0 1 0 -384 3 | 1 0 0 384 4 | 0 1 0 -128 5 | 1 0 0 128 6 | 0 0.707107 0.707107 90.5097
Lump[5] - Brushsides
Without: CMod_LoadBrushes: bad side count
struct Brushside
{
union Column1 // first 6 entries indicated by an entry in lump 6 are distances, rest is plane ID's
{
unsigned int plane;
float distance;
} column1;
unsigned int texture;
};
Lump 5 and 6 are used for the collision map, however the second column holds texture IDs. The ID is a reference to lump 0 and can be used to look up the texture flags. These tell the game wether the brush is used for collision detection only, acts as ladder, player clip etc.
Each entry has a length of 8 bytes. Two columns, first either distance or plane id:
float distance - plane parallel to x, y or z axisushort plane_id - slant plane reference to lump 4 (#)
The first six entries per brush (lump 6) are distances defining a bounding box, followed by optional plane IDs for more complex, but still convex brushes. Imagine a wooden cube (bounding box), where you cut off pieces with an axe. Note that the axe's blade is infinite, you can't do partial cuts.
The normal vector's direction represents the plane's facing. The decision, which part will be kept of the volume (brush), is made here. The part the vector points is dropped. If you negate the normal vector, it points to the remaining volume.
Example:
128 20
384 18
-384 17
-128 19
128 15
320 16
13 # 21
128 12
384 14
128 13
384 14
128 11
320 14
10 # 14
12 # 14
|
Order of the bounding box sides:
The minimum or smallest bounding or enclosing box |
Second column is a texture id, texture lump:
0 | dawnville2_wartorn_brick07 0x00200000 0x00000001 1 | dawnville2_wood_boards01 0x01500000 0x00000001 2 | dawnville2_wood_boards01 0x01500000 0x00002080
Lump[6] - Brushes
Without: no collision with brush geometry. Footage (Xfire)
struct Brush
{
unsigned short sides;
unsigned short material_id;
};
Brushside index, two ushorts per entry (2 byte unsigned integer, 4 bytes each row).
- First column:
Amount of brushsides, start id can be calculated (sum_of_previous_amounts - 1)
For end id, add the entry value to the start id (sum_of_previous_amounts - 1 + amount_of_brushsides)
7 15 8 11
- Second column:
Might be texture id, perhaps one of the applied textures as default.
If more than one texture is mapped to the brush, it may get overwritten (brushsides lump texture id).
New theory: the texture flags (properties) are actually referenced to set up the brush characteristic and the engine behaves accordingly.
Lump[7] - TriangleSoups
Without: LoadMap: funny lump size in (*.d3dbsp)
It's like the missing link between the triangles (lump 9) and vertices (lump 8). It refers to both of them defining the texture per triangle, giving the offset in the vertex lump to which the numbers in lump 9 point to (ID's in lump 9 point to vertex as in: offset + ID, offset is given in this lump). Also offset in triangles lump is given.
16 bytes per entry:
struct TriangleSoup
{
unsigned short material_id;
unsigned short draw_order;
unsigned int vertex_offset;
unsigned short vertex_length;
unsigned short triangle_length;
unsigned int triangle_offset;
};
Example:
0 0 0 24 84 0
1 0 24 8 24 84
2 0 32 14 18 108
3 0 46 40 66 126
4 0 86 16 24 192
5 0 102 78 144 216
Lump[8] - Vertices
or DrawVerts
Every vertex entry is 17 DWORDs long (68 bytes). First 3 DWORDs define the position (x y and z). Next 3 DWORDs define the normal vector of the vertex (x y and z) (WHY?!), then a DWORD which defines the colour (BGR) and the opacity (A). Then 2 DWORDs which have to do with texture shifting, and 2 more DWORDs for something yet unknown. At the end we have 6 DWORDs having to do with texture alignment (rotation?).
struct DrawVertex
{
float position[3];
float normal[3];
unsigned char rgba[4];
float uv[2];
float st[2];
float unknown[6];
};
Example:
256 256 -64 0.235702 0.235702 0.942809 0xff80a0c0 1.000000 1.000000 0.023438 0.001953 0.970143 0.000000 -0.242536 -0.057166 0.971825 -0.228665
0 256 0 0.119573 0.119573 0.985599 0xff80a0c0 0.000000 1.000000 0.007813 0.001953 0.992719 0.000117 -0.120451 -0.014518 0.992825 -0.118689
0 0 0 0 0 1 0xff80a0c0 0.000000 0.000000 0.007813 0.017578 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
256 0 0 0.119573 0.119573 0.985599 0xff80a0c0 1.000000 0.000000 0.023438 0.017578 0.992719 0.000117 -0.120451 -0.014518 0.992825 -0.118689
-100 0 0 0 0 1 0xffffffff 0.804688 0.000000 0.001709 0.017578 1.000000 0.000000 0.000000 0.000000 -1.000000 0.000000
0 0 0 0 0 1 0xffffffff 1.000000 0.000000 0.007813 0.017578 1.000000 0.000000 0.000000 0.000000 -1.000000 0.000000
Lump[9] - Triangles
or DrawIndexes
A face is made up of 3 vertexes, giving a triangle. One entry is actually 2 bytes long (ushort), but 3 sequently stored make up one triangle. So 3 entries can be treated as 1.
struct DrawIndex
{
unsigned short vertex[3];
};
Example:
8 9 6 12 13 10 22 23 20 20 21 22 2 19 0 0 5 2 18 3 16 16 17 18 10 14 12 6 15 8 11 7 4 4 1 11 6 7 4 4 5 6 2 3 0 0 1 2
Lump[10] - Cull Groups
32 bytes per entry.
Lump[11] - Cull Group Indexes
4 bytes per entry.
Lump[17] - Portal Verts
12 bytes per entry.
struct PortalVertex
{
float unknown[3];
};
Lump[18] - Occluder
20 bytes per entry. Fine portalling within a cell, Valve Engine example
Lump[19] - Occluder Planes
4 bytes per entry.
Lump[20] - Occluder Edges
4 bytes per entry.
Lump[21] - Occluder Indexes
2 bytes per entry.
Lump[22] - AABB Trees
Without: game crashes with application error
12 bytes per entry. Web links: [1] [2]
struct AABBTree
{
unsigned int unknown[3];
};
Example:
0 1278 8
0 300 6
300 22 2
322 292 5
614 17 2
Lump[23] - Cells
Assumed to be VIS portals, contains a cell in Skybox dimension (first entry). 52 bytes per entry (example needs update!).
struct Cell
{
float unknown[3];
};
Example:
120 -392 120
392 -120 328
0 0 0
0 0 0
Lump[24] - Portals
16 bytes per entry.
struct Portal
{
unsigned int unknown[4];
};
Lump[25] - Nodes
Without: Map has no Nodes
36 bytes per entry.
struct Node
{
int unknown[9];
};
Example:
1 -2 1 -16 -16 -8 8 8 16
5 2 -8 -16 -16 -8 8 8 8
0 3 -7 -8 -16 -8 8 8 8
3 -3 4 -8 -16 0 8 8 8
4 -4 5 -8 -16 0 0 8 8
2 -5 -6 -8 -16 0 0 0 8
Lump[26] - Leafs
Without: Map has no Leafs
36 bytes per entry.
struct Leaf
{
int unknown[9];
};
Example:
0 0 0 0 0 0 0 0 0
-1 -1 0 0 0 1 -1 0 0
-1 -1 0 0 1 1 -1 0 0
-1 -1 0 0 2 1 -1 0 0
0 0 0 0 3 0 0 0 0
-1 -1 0 0 3 1 -1 0 0
-1 -1 0 0 4 1 -1 0 0
The 5th column is possibly an id of lump 27.
Lump[27] - Leaf Brushes
Without: Stuck in loadscreen, game closes after a couple seconds without error.
This lump contains a list of brush IDs (lump 6), there might be an index lump for it. 4 bytes per entry.
struct LeafBrush
{
int unknown;
};
Example:
53
52
51
50
14
1
Lump[29] - Collision Verts
Without: no visible changes
16 bytes per entry.
struct CollisionVertex
{
float unknown[4];
};
Lump[30] - Collision Edges
former Player Terrain Collision
Without: terrain retains collision, but player can't move smoothly on the ground. It's easy to get stuck or float around. Still okay where terrain is very even. Footage (Xfire). 56 bytes per entry.
struct CollisionEdge
{
unsigned int unknown;
float position[3];
float normal[3][3];
float distance;
};
Example:
0 64 128 8 0 0 1 -0.707107 -0.707107 0 0.707107 -0.707107 0 90.5097
0 128 64 8 0 1 0 0 0 -1 -1 0 0 64
0 64 64 8 1 0 0 0 0 -1 0 1 0 64
0 64 128 8 0 1 0 0 0 1 1 0 0 64
0 128 128 8 1 0 0 0 0 1 0 -1 0 64
0 128 128 8 0 0 1 -0.707107 -0.707107 0 0.707107 -0.707107 0 90.5097
Lump[31] - Collision Tris
Without: game crashes with application error
72 bytes per entry.
struct CollisionTriangles
{
float normal[3];
float distance;
float unknown[8];
int id[6];
};
Lump[32] - Collision Borders
Without: no visible changes
28 bytes per entry.
Lump[33] - Collision Parts
Without: game crashes with application error
12 bytes per entry.
Lump[34] - Collision AABBs
Without: game crashes with application error
32 bytes per entry.
Lump[35] - Models
Without: Map with no models
Possibly similar to "rigid groups of world geometry": Quake3 - Model Lump
Seems to contain script_brushmodels, maybe more. 48 bytes per entry.
struct Models
{
float position[6];
int id[6];
};
Example:
-18048 -11904 -896 -2944 6144 5160 0 5748 0 0 0 8260
-112 -112 -48 112 112 48 5748 0 3072 0 8260 1
-11072 -7504 16 -11014 -7382 144 5748 0 3072 0 8261 1
-112 -112 -48 112 112 48 5748 0 3072 0 8262 1
-9208 -6264 0 -9150 -6142 128 5748 0 3072 0 8263 1
-40 -40 0 40 40 56 5748 0 3072 0 8264 1
-17920 -11904 -512 -3072 6144 -384 5748 0 3072 0 8265 1
-17849 -11977 -512 -3001 6071 -384 5748 0 3072 0 8266 1
Current investigation status:
How a script_brushmodel appears in the BSP (lump 37):
{ // ----- worldspawn block start
{
// brush definition inside of the worldspawn entity
// all geometry elements which belongs to a group of lump 35 are stored sequently
}
} // ----- worldspawn block end
{
"classname" "script_brushmodel"
"model" "*1"
"targetname" "auto1"
}
In the source .map it looks different:
// entity 1
{
"targetname" "auto1"
"classname" "script_brushmodel"
// brush 0
{
// ...
}
// brush 1
{
// ...
}
// brush 2
{
// ...
}
}
It requires a transformation to get script_brushmodels back. The missing link between the geometry and the entity is lump 35. model *n is an id and refers to an entry of this lump. It references 3 other lumps in turn.
Example: [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] 0 256 192 112 320 256 0 1 0 0 0 1 16 0 0 152 128 112 1 4 0 0 1 3 208 0 192 320 128 312 5 2 0 0 4 2 1 - left (x) 2 - front (y) 3 - bottom (z) 4 - right (x) 5 - back (y) 6 - top (z) 7 - texture group (start) 8 - texture group (amount) 9 - mesh (start) 10 - mesh (amount) 11 - brush (start) 12 - brush (amount) 1-6 : bounding box 7+8 : texture group id reference (lump 7) 9+10: patch id reference 11+12: brush id reference (lump 6)
Note: No planes in lump 4 for meshes / patches. Additional verts and tris if surface is visible. Mesh related lumps: 29, 30, 31, 33 and 34. One or more of them may contain collision data.
Lump[36] - Visibility
former VIS
Present if VIS (portalling) has been calculated during the compile process. Apparently a single block of data.
Without: /r_singlecell 1 still shows the portals, forced VIS?
Lump[37] - Entities
or EntData
The entities lump is readable and stored (almost) the same way described in the .MAP file structure.
struct Entity
{
char data[];
};
Or, after getting rid of redundant data, one might interpret the entity lump like this:
struct Entity
{
std::map<std::string, std::string> pairs;
};
In other words, the Entities lump consists of a a number of Entity structs. Each entity struct holds a list of string pairs.
Couple entity types in this lump got a "model" key/value pair with a star and a number (*n), causing Radiant to crash and the compiler to ignore the whole block (see lump 35). You also have to remove certain other things til the decompiler supports a reconstruction method for the necessary data:
- model *n
- fx
- script_exploder
- trigger
- gndLt (optional, ignored by Radiant)
Furthermore, you need to recreate the skybox at the moment, as there will likely be leaks in the ground caused from caulk/clip/... textured geometry. It's not visible in game as you know and therefore not stored in the usual lumps for vertices, triangles and so on. But it has collision, and the collision data is either stored in the brush or terrain collmap lumps. Note that many models are clipped using invisible textures as well. If no collmap file is present for a model, create one or it will lack collision after compile. A decompiler would have to reconstruct the collision map to avoid this.
Lump[38] - Paths
SinglePlayer only, connected waypoints and/or paths? Variable in length.
Unidentified lumps
Not seen on any map so far, presumably not in use. Feasible lump number predictions in brackets:
- [12] Shadow Verts
- [13] Shadow Indices
- [14] Shadow Clusters
- [15] Shadow AABB Trees
- [16] Shadow Sources
- [28] Leaf Surfaces
- [39] Lights
Web Links
- Binary Space Partitioning
- BSP Format
- BSP (File Extension)
- BSP for Dummies
- Unofficial Quake3 Map Specs
- Quake Documentation v3.4
- Quake 2 BSP File Format
- Call of Duty 2: Map File Format
- Understanding Vis and Hint Brushes (Portalling)
- Quake BSP Renderer
- Quake Glossary
- Endianness
Made by CoDEmanX and Daevius
