Call of Duty 4: FastFile Format

From COD Modding & Mapping Wiki
Jump to navigation Jump to search

*** Article and investigation in early progress ***

Preface

Be aware, FF files are little endian, so swap those bytes 'round!

The data block types (also called "index identifier") are written in big endian in this article, e.g. 00 00 00 1F is actually 1F 00 00 00 in ff, although ints and floats in the examples are kept little endian!


The global file layout starts with a header. After which you get a string list (model tag names, animation notetracks, maybe more; not always present). Followed by an index list. This list gives information about the rest of the file, which is a concatenation of compiled files (reinterpreted and linked data generated from source files like d3dbsp and stringtables, hereinafter referred to as data blocks).

  • Header
  • String list index
  • String list
  • Data block index
  • Data blocks

Each file is separated by the separation key: 0xFFFFFFFF. But be aware, each file itself can also contain such a sequence which is not meant to be the separation key! So it is necessary to understand each data type in the FF file.

Header

(Note: not up to date and not generalized)

  • Byte 0-3: decompressed fastfile size minus 44 (0x2C)
  • Byte 4-7: about the total size of referenced data, e.g. the required memory for IWI textures if material files are in the ff
  • Byte 8-11: unknown, might be a flag
  • Byte 12-23: unknown
  • Byte 24-27: somehow related to ff size?
  • Byte 28-43: unknown
  • Byte 44-47: equal to number of entries in "1st index" and amount of (model tag/joint/notetrack) strings (times 4 for index length)
  • Byte 48-51: separator? (FF FF FF FF)
  • Byte 52-55: number of records* ("2nd index", times 8 for index length)
  • Byte 56-63: separator? (FF FF FF FF FF FF FF FF)
  • Byte 64-x: string list (xmodelsurfs?) --> not always, depends on the content (required offsets are in the FF header)*

* under re-investigation


Example: smallest possible fastfile (mod.ff)

29 00 00 00  decompressed fastfile size minus 44 
00 00 00 00  
1C 00 00 00  unknown

00 00 00 00 00 00 00 00 00 00 00 00

0D 00 00 00  data size in memory (?)

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

01 00 00 00  one index entry

FF FF FF FF 1F 00 00 00  a rawfile

FF FF FF FF  index end

FF FF FF FF  start separator
00 00 00 00  rawfile length (empty, no errors occured during compilation)
FF FF FF FF  separator
6D 6F 64 00  fastfile name as rawfile name
         00  rawfile string (empty)


Example: mp_backlot_load.ff

Contains:

  • 2x shaders
  • 3x materials
  • (1x fastfile name)


Materials:

  1. $victorybackdrop - mile_high_victory_screen.iwi (699,092 bytes)
  2. $defeatbackdrop - (none)
  3. $levelbriefing - loadscreen_mp_backlot.iwi (524,316 bytes)
-----------  [start of file and header]

71 03 00 00  Decompressed file size - 44 (hex: 371 + 2C = 39D; dec: 881 + 44 = 925)

B8 AA 12 00  Size of both IWI's the materials in FF point to - 56 (2x28, 28 bytes = IWI header)
             0x12AAB8          = 1,223,352
             699,092 + 524,316 = 1,223,408
             difference:                56

A4 00 00 00  Might be a flag

00 00 00 00  Unknown
00 00 00 00  Unknown
00 00 00 00  Unknown
E5 00 00 00  Size of fastfile in memory (that is at least without header and separators, and maybe more)
00 00 00 00  Unknown
00 00 00 00  Unknown
00 00 00 00  Unknown
00 00 00 00  Unknown

-----------  [end of header] (44 bytes), [start of index]

00 00 00 00  Unknown, is sometimes FF FF FF FF

06 00 00 00  Number of records in index (6)

FF FF FF FF  05 00 00 00  1st: shader
FF FF FF FF  05 00 00 00  2nd: shader
FF FF FF FF  04 00 00 00  3rd: material
FF FF FF FF  04 00 00 00  4th: material
FF FF FF FF  04 00 00 00  5th: material
FF FF FF FF  1F 00 00 00  6th: rawfile

FF FF FF FF  end separator (the following FF FF FF FF is the start seperator of the next data block!)

-----------  [end of index]


Index

Starts with an int telling the number of dword couples. Such a couple is composed of 2 ints:

4 byte offset (FF FF FF FF = -1, meaning right after the previous block?)
4 byte identifier (data block type)

Separator (FF FF FF FF) after last couple

Types

  • 00 00 00 01 - physpreset
  • 00 00 00 02 - xanim
  • 00 00 00 03 - xmodel
  • 00 00 00 04 - material
  • 00 00 00 05 - shader / techset
  • 00 00 00 06 - image?
  • 00 00 00 07 - soundalias (+audio data)
  • 00 00 00 0B - map_ents (d3dbsp)
  • 00 00 00 0C - related to col_map_mp (d3dbsp)
  • 00 00 00 0E - related to game_map_mp (d3dbsp)
  • 00 00 00 10 - related to col_map_mp / gfx_map (d3dbsp)
  • 00 00 00 13 - font
  • 00 00 00 14 - menu
  • 00 00 00 16 - localized string asset
  • 00 00 00 17 - physpreset??
  • 00 00 00 19 - fx
  • 00 00 00 1F - raw file
  • 00 00 00 20 - stringtable (comma separated list)


Content

Since there is a FF FF FF FF separator in front of every block and none at the very end (after last 1F-block), we can assume that each block has a start separation and no end separation. Thus, any FF FF FF FFs you find after data blocks are actually the start separators of the next block. This means the index end separator is rather FFFFFFFF than FFFFFFFFFFFFFFFF!


00 00 00 04 (material)

-----------  [block start]

FF FF FF FF  start separation

00 2B 01 01  First 4 bytes seem to be 4 (unsigned) char's.
             1st byte is so far always 0x00 (perhaps it's a short together with the second byte).
             2nd byte so far either 0x2B or 0x00, I think this is a length.
             3rd byte so far 0x01, when 0x00 (all 4) the content is only a number of 0x00's, then the filename (0x00 termination)
             4th byte either 0x00 or 0x01. Can be a length or a bool.
             Then 0x10 bytes of more header.
FF FF FF FF  Separator
00           When fourth byte is 01, there's a 0x00 following?             
             Then there's garbage (0xFF) and some shorts it looks of length in the second byte.
             Then more information...
02 00 0E E0  Often ends with this

-----------  [block end]


00 00 00 05 (shader)

A shader consists of 3 levels. One, the top level, consists of a header (with a name inside). Then the next level is a pack of shaders, it has a header, shaders and at the end a name. Then the bottom level is the shader itself, it has a header (containing a name of the source file and the length of the data) and the raw shader data. 

-----------  [block start]

FF FF FF FF  Start separation

FF FF FF FF  pointer
00 00 00 00  0x18 bytes of 0x00
FF FF FF FF  Or 0x00000000 (if no content further on)
00 00 00 00  0x74 bytes which are often 0x00
?? ?? .. 00  Then techset file name (0x00 termination) if pointer is 0xFFFFFFFF?

-----------  [start of shader pack]

             pointer
             1 dword of some character-length options / flags?
             3 pointers
             1 dword of some character-length sizes
FF FF FF FF  Separator
             0x64 bytes of some short length options / flags? For the first 05 entry? Or if the references above are 0xFFFFFFFF?

-----------  [start of shader]

             pointer, if not FF FF FF FF, no filename is further on
             pointer, often 00 00 00 00
FF FF FF FF  Separator
?? ?? 00 00  2 shorts, first short multiplied by 4 gives the length of the shader, second is a flag?
?? ?? .. 00  String if first pointer is FF FF FF FF, 0x00 terminated (name of shader source file)
             Raw shader file (except the first 4 bytes, which, in the shader file, denotes the length of the file)
FF FF 00 00  which ends with this (always?)

-----------  [end of shader]

             Can either have more shaders (go back to [start of shader]) or shaders have ended

             Variable length of 'stuff' (technique commands?)
41 10 AB A0  which ends with this (always?)
?? ?? .. 00  String, 0x00 terminated (technique file name)

-----------  [end of shader pack]

             Can either have more shader packs (go back to [start of shader pack]) or shader packs have ended

-----------  [block end]

00 00 00 06 (image)

Stores some information about the specified IWI in fastfile. Does not include the actual image data. Length is 52 bytes + IWI name (+1 for string termination). 

-----------  [block start]
             Note: obviously no start separator
03 00 00 00
FE FF FF FF  separator?
00 00 00 00
00 00 00 00

70 55 01 00  IWI size minus 28 (0x1C)
70 55 01 00  same

00 01 00 01
01 00 03 00
FF FF FF FF  separator?

73 70 .. 00  IWI name without file extension (string + termination, variable size)

00 00 00 01
00 01 01 00

44 58 54 35 00 00 00 00  fixed size char[] for type? (here: DXT5....)

-----------  [block end]


00 00 00 16 (localized string asset)

A localized string asset consists of an alias name and a translated string. During compilation, it takes the given alias name (REFERENCE) from *.str file and prepands the *.str file's name in upper-case plus an underscore. Example:


example.str 

VERSION             "1"
CONFIG              "C:\trees\cod3\cod3\bin\StringEd.cfg"
FILENOTES           ""
//--------------------------------

REFERENCE           MARINES_NAME
LANG_ENGLISH        "US Airforce"

//--------------------------------
ENDMARKER


mod.ff 

ÿÿÿÿ
ÿÿÿÿ
EXAMPLE_MARINES_NAME.
US Airforce.


Length is 8 bytes + string sizes (string count = 0-2, depends on the first two ints). 

-----------  [block start]
             Note: no start separator!

FF FF FF FF  either pointer or FFFFFFFF if the string wasn't used before
FF FF FF FF  same. If two pointers are used here, then no strings will follow.
55 53 .. 00  alias name for the localized
53 54 .. 00  actual localized string

-----------  [block end]


00 00 00 1F (rawfile)

A rawfile. Often a rawfile named right after the fastfile's name is implicitly added. It contains the errors thrown when it was compiling / linking. 

-----------  [block start]

FF FF FF FF  Start separation

64 00 00 00  Length of the file's content (example: decimal 100; does not count the string termination!)

FF FF FF FF  Separator

6D 6F 64 00  File name or if it's the last 1F-block FastFile name (example: mod.)
?? ?? .. 00  Plain text content of length above (string, terminated by 0x00)

-----------  [block end OR even end of file]


00 00 00 20 (stringtable)

A compiled stringtable (comma separated list, *.csv). Even values are stored as strings. 

-----------  [block start]

FF FF FF FF  start separation

05 00 00 00  (int) columns
02 00 00 00  (int) rows

FF FF FF FF  separator
?? ?? .. 00  filename (path/*.csv) + string termination (0x00)

-----------  [table cell description]

?? ?? ?? ??  columns * rows integers (5 * 2 * 4 = 40 bytes in total, 5 * 2 = 10 cells)
             FF FF FF FF - if the string wasn't used before and follows after the cell description
             ?? ?? ?? 4? - if an identical string was already used (in another csv). It is some kind of memory offset (pointer). Base address calculation not figured out yet (0x40000072 ?)

-----------  [cell content / strings]

?? ?? .. 00  string + termination (0x00)
             Number of strings = amount of FF FF FF FFs in table cell description

-----------  [block end]


--CoDEmanX 01:45, 8 December 2009 (UTC)

--Daevius 14:24, 5 November 2009 (UTC)




Retrieved from "https://wiki.zeroy.com/index.php?title=Call_of_Duty_4:_FastFile_Format&oldid=7178"