Call of Duty 4: FastFile Format - Revision history

CoDEmanX: -spam

2012-03-10T16:06:30Z

-spam

← Older revision		Revision as of 16:06, 10 March 2012
Line 539:		Line 539:

	-->		-->


	~~[http://cvresumewritingservices.org/ resume writing services]~~

CoDEmanX: /* Types */ --> table

2012-01-03T21:55:53Z

Types: --> table

← Older revision		Revision as of 21:55, 3 January 2012
Line 136:		Line 136:
	=== Types ===		=== Types ===

	* xmodelpieces = 0,		{\|
	* physpreset = 1,		\|-
	* xanim = 2,		! Type name
	* xmodel = 3,		! hex value
	* material = 4,		\|-
	* pixelshader = 5,		\| xmodelpieces
	* techset = 6,		\| 0
	* image = 7,		\|-
	* sndcurve = 8,		\| physpreset
	* loaded_sound = 9,		\| 1
	* col_map_sp = 0x0a,		\|-
	* col_map_mp = 0x0b,		\| xanim
	* com_map = 0x0c,		\| 2
	* game_map_sp = 0x0d,		\|-
	* game_map_mp = 0x0e,		\| xmodel
	* map_ents = 0x0f,		\| 3
	* gfx_map = 0x10,		\|-
	* lightdef = 0x11,		\| material
	* ui_map = 0x12,		\| 4
	* font = 0x13,		\|-
	* menufile = 0x14,		\| pixelshader
	* menu = 0x15,		\| 5
	* localize = 0x16,		\|-
	* weapon = 0x17,		\| techset
	* snddriverglobals = 0x18,		\| 6
	* impactfx = 0x19,		\|-
	* aitype = 0x1a,		\| image
	* mptype = 0x1b,		\| 7
	* character = 0x1c,		\|-
	* xmodelalias = 0x1d,		\| sndcurve
	* rawfile = 0x1f,		\| 8
	* stringtable = 0x20		\|-
			\| loaded_sound
			\| 9
			\|-
			\| col_map_sp
			\| 0x0a
			\|-
			\| col_map_mp
			\| 0x0b
			\|-
			\| com_map
			\| 0x0c
			\|-
			\| game_map_sp
			\| 0x0d
			\|-
			\| game_map_mp
			\| 0x0e
			\|-
			\| map_ents
			\| 0x0f
			\|-
			\| gfx_map
			\| 0x10
			\|-
			\| lightdef
			\| 0x11
			\|-
			\| ui_map
			\| 0x12
			\|-
			\| font
			\| 0x13
			\|-
			\| menufile
			\| 0x14
			\|-
			\| menu
			\| 0x15
			\|-
			\| localize
			\| 0x16
			\|-
			\| weapon
			\| 0x17
			\|-
			\| snddriverglobals
			\| 0x18
			\|-
			\| impactfx
			\| 0x19
			\|-
			\| aitype
			\| 0x1a
			\|-
			\| mptype
			\| 0x1b
			\|-
			\| character
			\| 0x1c
			\|-
			\| xmodelalias
			\| 0x1d
			\|-
			\| rawfile
			\| 0x1f
			\|-
			\| stringtable
			\| 0x20
			\|}

	== Content ==		== Content ==

Randallphillips at 12:04, 24 June 2011

2011-06-24T12:04:31Z

New page

[[Image:Under Construction Small.png‎]] '''*** Article and investigation in early progress ***'''

== Preface ==

Be aware, FF files are little endian, so swap those bytes 'round!

The data block types (also called "index identifier") are written in big endian in this article, e.g. 00 00 00 1F is actually 1F 00 00 00 in ff, although ints and floats in the examples are kept little endian!

The global file layout starts with a header. After which you get a string list (model tag names, animation notetracks, maybe more; not always present). Followed by an index list. This list gives information about the rest of the file, which is a concatenation of compiled files (reinterpreted and linked data generated from source files like d3dbsp and stringtables, hereinafter referred to as data blocks).

* Header
* String list index
* String list
* Data block index
* Data blocks

Each file is separated by the separation key: 0xFFFFFFFF. But be aware, each file itself can also contain such a sequence which is not meant to be the separation key! So it is necessary to understand each data type in the FF file.

Edit: The latest research led to the assumption, that it's no real separation key but an address pointer (which is 0xFFFFFFFF if the data follows or 0x??????4? if the content is located elsewhere).

It seems to be 0xFFFFFFFF replaces pointers used during compilation, leading to the following data block structure in case of some data blocks (at least rawfile):

* Block header (containing one or more 0xFFFFFFFF pointers, size needs to be known to properly read files)
* Block data (one for each 0xFFFFFFFF data, can be anything from a string to raw binary data, possibly a 00 marks the end for strings?)

== Header ==

''(Note: not up to date and not generalized)''

*Byte 0-3: decompressed fastfile size minus 44 (0x2C)
*Byte 4-7: about the total size of referenced data, e.g. the required memory for IWI textures if material files are in the ff
*Byte 8-11: ''unknown'', might be a flag
*Byte 12-23: ''unknown''
*Byte 24-27: somehow related to ff size?
*Byte 28-43: ''unknown''
*Byte 44-47: equal to number of entries in "1st index" and amount of (model tag/joint/notetrack) strings (times 4 for index length)
*Byte 48-51: separator? (FF FF FF FF)
*Byte 52-55: number of records* ("2nd index", times 8 for index length)
*Byte 56-63: separator? (FF FF FF FF FF FF FF FF)
*Byte 64-x: string list (xmodelsurfs?) --> not always, depends on the content (required offsets are in the FF header)*

<nowiki>*</nowiki> under re-investigation 

=== Example: smallest possible fastfile (mod.ff) ===

<pre>
29 00 00 00 decompressed fastfile size minus 44
00 00 00 00
1C 00 00 00 unknown

00 00 00 00 00 00 00 00 00 00 00 00

0D 00 00 00 data size in memory (?)

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

01 00 00 00 one index entry

FF FF FF FF 1F 00 00 00 a rawfile

FF FF FF FF index end

FF FF FF FF start separator
00 00 00 00 rawfile length (empty, no errors occured during compilation)
FF FF FF FF separator
6D 6F 64 00 fastfile name as rawfile name
00 rawfile string (empty)
</pre>

=== Example: mp_backlot_load.ff ===

'''Contains:'''
*2x shaders
*3x materials
*(1x fastfile name)

Materials:
# $victorybackdrop - mile_high_victory_screen.iwi (699,092 bytes)
# $defeatbackdrop - (none)
# $levelbriefing - loadscreen_mp_backlot.iwi (524,316 bytes)

<pre>
----------- [start of file and header]

71 03 00 00 Decompressed file size - 44 (hex: 371 + 2C = 39D; dec: 881 + 44 = 925)

B8 AA 12 00 Size of both IWI's the materials in FF point to - 56 (2x28, 28 bytes = IWI header)
0x12AAB8 = 1,223,352
699,092 + 524,316 = 1,223,408
difference: 56

A4 00 00 00 Might be a flag

00 00 00 00 Unknown
00 00 00 00 Unknown
00 00 00 00 Unknown
E5 00 00 00 Size of fastfile in memory (that is at least without header and separators, and maybe more)
00 00 00 00 Unknown
00 00 00 00 Unknown
00 00 00 00 Unknown
00 00 00 00 Unknown

----------- [end of header] (44 bytes), [start of index]

00 00 00 00 Unknown, is sometimes FF FF FF FF

06 00 00 00 Number of records in index (6)

FF FF FF FF 05 00 00 00 1st: shader
FF FF FF FF 05 00 00 00 2nd: shader
FF FF FF FF 04 00 00 00 3rd: material
FF FF FF FF 04 00 00 00 4th: material
FF FF FF FF 04 00 00 00 5th: material
FF FF FF FF 1F 00 00 00 6th: rawfile

FF FF FF FF end separator (the following FF FF FF FF is the start seperator of the next data block!)

----------- [end of index]
</pre>

== Index ==

Starts with an int telling the number of dword couples. Such a couple is composed of 2 ints:

4 byte offset (FF FF FF FF = -1, meaning right after the previous block?)<br>
4 byte identifier (data block type)

Separator (FF FF FF FF) after last couple

=== Types ===

* xmodelpieces = 0,
* physpreset = 1,
* xanim = 2,
* xmodel = 3,
* material = 4,
* pixelshader = 5,
* techset = 6,
* image = 7,
* sndcurve = 8,
* loaded_sound = 9,
* col_map_sp = 0x0a,
* col_map_mp = 0x0b,
* com_map = 0x0c,
* game_map_sp = 0x0d,
* game_map_mp = 0x0e,
* map_ents = 0x0f,
* gfx_map = 0x10,
* lightdef = 0x11,
* ui_map = 0x12,
* font = 0x13,
* menufile = 0x14,
* menu = 0x15,
* localize = 0x16,
* weapon = 0x17,
* snddriverglobals = 0x18,
* impactfx = 0x19,
* aitype = 0x1a,
* mptype = 0x1b,
* character = 0x1c,
* xmodelalias = 0x1d,
* rawfile = 0x1f,
* stringtable = 0x20

== Content ==

Since there is a FF FF FF FF separator (respectively pointer) in front of every block and none at the very end (after last 1F-block), we can assume that each block has a start separation and no end separation. Thus, any FF FF FF FFs you find after data blocks are actually the start separators of the next block. This means the index end separator is rather FFFFFFFF than FFFFFFFFFFFFFFFF!

=== 00 00 00 04 (material) ===

<pre>
----------- [block start]

00 2B 01 01 First 4 bytes seem to be 4 (unsigned) char's.
1st byte is so far always 0x00 (perhaps it's a short together with the second byte).
2nd byte so far either 0x2B or 0x00, I think this is a length.
3rd byte so far 0x01, when 0x00 (all 4) the content is only a number of 0x00's, then the filename (0x00 termination)
4th byte either 0x00 or 0x01. Can be a length or a bool.
Then 0x10 bytes of more header.
FF FF FF FF Separator
00 When fourth byte is 01, there's a 0x00 following?
Then there's garbage (0xFF) and some shorts it looks of length in the second byte.
Then more information...
02 00 0E E0 Often ends with this

----------- [block end]
</pre>

=== 00 00 00 05 (shader) ===

A shader consists of 3 levels. One, the top level, consists of a header (with a name inside). Then the next level is a pack of shaders, it has a header, shaders and at the end a name. Then the bottom level is the shader itself, it has a header (containing a name of the source file and the length of the data) and the raw shader data.

<pre>
----------- [block start]

36 pointers (0x90 bytes) (bytes 0x19 up to 0x22 say if we get any content at all?)
?? ?? .. 00 Then techset file name (0x00 termination) if pointer is 0xFFFFFFFF?

----------- [start of shader pack]

pointer
1 dword of some character-length options / flags?
3 pointers
1 dword of some character-length sizes
FF FF FF FF Separator
0x64 bytes of some short length options / flags? If the second pointer above is 0xFFFFFFFF

----------- [start of shader]

pointer, if not 0xFFFFFFFF, no filename is further on
pointer, often 00 00 00 00
FF FF FF FF Separator
?? ?? 00 00 2 shorts, first short multiplied by 4 gives the length of the shader, second is a flag?
?? ?? .. 00 String if first pointer is 0xFFFFFFFF, 0x00 terminated (name of shader source file)
Raw shader file (except the first 4 bytes, which, in the shader file, denotes the length of the file)
FF FF 00 00 which ends with this (always?)

----------- [end of shader]

Can either have more shaders (go back to [start of shader]) or shaders have ended

Variable length of 'stuff' (technique commands?)
41 10 AB A0 which ends with this (always?)
?? ?? .. 00 String, 0x00 terminated (technique file name)

----------- [end of shader pack]

Can either have more shader packs (go back to [start of shader pack]) or shader packs have ended

----------- [block end]
</pre>

=== 00 00 00 06 (image) ===

Stores some information about the specified IWI in fastfile. Does not include the actual image data.
Length is 52 bytes + IWI name (+1 for string termination).

<pre>
----------- [block start]

03 00 00 00
FE FF FF FF separator?
00 00 00 00
00 00 00 00

70 55 01 00 IWI size minus 28 (0x1C)
70 55 01 00 same

00 01 00 01
01 00 03 00
FF FF FF FF separator?

73 70 .. 00 IWI name without file extension (string + termination, variable size)

00 00 00 01
00 01 01 00

44 58 54 35 00 00 00 00 fixed size char[] for type? (here: DXT5....)

----------- [block end]
</pre>

=== 00 00 00 13 (font) ===

See also [[Call of Duty: Font System]]. Implicitly adds shaders.

<pre>
----------- [block start] 216 byte + 2 strings
18 00 00 00 int font_size
BF 00 00 00 int number_of_records
FE FF FF FF ?
FE FF FF FF ?
FF FF FF FF separator
66 6F .. 00 string font_name (e.g. fonts/bigFont.)
FF FF FF FF separator
00 2B 01 01 ?
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
FF FF FF FF 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
FF FF 01 00 01 00 03 00
0D 00 00 40 pointer?
FF FF FF FF
00 00 00 00
FF FF FF FF
66 6F .. 00 string font_material (e.g. fonts/gamefonts_pc.)
41 10 AB A0 63 70 EB 00 ?

----------- [image block] 112 bytes + 1 string

FF FF FF FF start separator / pointer
03 00 00 00
FE FF FF FF separator?
00 00 01 00 ?
00 00 00 00
D0 AA 0A 00 IWI size minus 28 (0x1C)
D0 AA 0A 00 same
00 02 00 04
01 00 03 00 ?
FF FF FF FF separator
67 61 .. 00 IWI name
00 00 00 02
00 04 01 00 ?
44 58 54 35
00 00 00 00 compression (here: DXT5....)

----------- [unknown block] 168 bytes

65 91 28 19
02 00 0E E0 ?
FF FF FF FF separator?
00 04 01 01 ?
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
FF FF FF FF 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 00 01 00 03 00 ?
0D 00 00 40 pointer? (same as above)
FF FF FF FF separator?
00 00 00 00

----------- [font block] 48 bytes + 1 string + number_of_records * 24 bytes

FF FF FF FF separator/pointer
66 6F .. 00 font effect name? (e.g. fonts/gamefonts_pc_glow.)
41 10 AB A0
63 70 EB 00
95 06 00 40
25 91 28 19
02 00 0E E0 ?

?? ?? ?? ?? font file entries (24 bytes each)

----------- [block end]
</pre>

=== 00 00 00 16 (localized string asset) ===

A localized string asset consists of an alias name and a translated string. During compilation, it takes the given alias name (REFERENCE) from *.str file and prepands the *.str file's name in upper-case plus an underscore. Example:

'''example.str'''
<pre>VERSION "1"
CONFIG "C:\trees\cod3\cod3\bin\StringEd.cfg"
FILENOTES ""
//--------------------------------

REFERENCE MARINES_NAME
LANG_ENGLISH "US Airforce"

//--------------------------------
ENDMARKER</pre>

'''mod.ff'''
<pre>
ÿÿÿÿ
ÿÿÿÿ
EXAMPLE_MARINES_NAME.
US Airforce.
</pre>

Length is 8 bytes + string sizes (string count = 0-2, depends on the first two ints).

<pre>
----------- [block start]

FF FF FF FF either pointer or FFFFFFFF if the string wasn't used before
FF FF FF FF same. If two pointers are used here, then no strings will follow.
55 53 .. 00 alias name for the localized
53 54 .. 00 actual localized string

----------- [block end]
</pre>

=== 00 00 00 1F (rawfile) ===

A rawfile. Often a rawfile named right after the fastfile's name is implicitly added. It contains the errors thrown when it was compiling / linking.

<pre>
----------- [block start]

64 00 00 00 Length of the file's content (example: decimal 100; does not count the string termination!)

FF FF FF FF Separator

6D 6F 64 00 File name or if it's the last 1F-block FastFile name (example: mod.)
?? ?? .. 00 Plain text content of length above (string, terminated by 0x00)

----------- [block end OR even end of file]
</pre>

=== 00 00 00 20 (stringtable) ===

A compiled stringtable (comma separated list, *.csv). Even values are stored as strings.

<pre>
----------- [block start]

05 00 00 00 (int) columns
02 00 00 00 (int) rows

FF FF FF FF separator
?? ?? .. 00 filename (path/*.csv) + string termination (0x00)

----------- [table cell description]

?? ?? ?? ?? columns * rows integers (5 * 2 * 4 = 40 bytes in total, 5 * 2 = 10 cells)
FF FF FF FF - if the string wasn't used before and follows after the cell description
?? ?? ?? 4? - if an identical string was already used (in another csv). It is some kind of memory offset (pointer). Base address calculation not figured out yet (0x40000072 ?)

----------- [cell content / strings]

?? ?? .. 00 string + termination (0x00)
Number of strings = amount of FF FF FF FFs in table cell description

----------- [block end]
</pre>

--[[User:NTAuthority|NTAuthority]] 10:54, 3 June 2010 (UTC) (some tiny information)

--[[User:CoDEmanX|CoDEmanX]] 01:45, 8 December 2009 (UTC)

--[[User:Daevius|Daevius]] 23:00, 8 December 2009 (UTC)

[[Category:Call of Duty 4]]
[[Category:Technical Reference]]
[[Category:File formats]]



[http://cvresumewritingservices.org/ resume writing services]

CoDEmanX: /* 00 00 00 13 (font) */

2009-12-22T11:18:40Z

00 00 00 13 (font)

← Older revision		Revision as of 11:18, 22 December 2009
Line 252:		Line 252:
	=== 00 00 00 13 (font) ===		=== 00 00 00 13 (font) ===

	~~Implicitly adds shaders.~~ See also [[Call of Duty: Font System]]		See also [[Call of Duty: Font System]]. Implicitly adds shaders.

	<pre>		<pre>
	----------- [block start]		----------- [block start] 216 byte + 2 strings
	18 00 00 00 int font_size		18 00 00 00 int font_size
	BF 00 00 00 int number_of_records		BF 00 00 00 int number_of_records
Line 274:		Line 274:
	41 10 AB A0 63 70 EB 00 ?		41 10 AB A0 63 70 EB 00 ?

	----------- [image block]		----------- [image block] 112 bytes + 1 string

	FF FF FF FF start separator / pointer		FF FF FF FF start separator / pointer
Line 292:		Line 292:
	00 00 00 00 compression (here: DXT5....)		00 00 00 00 compression (here: DXT5....)

	----------- [unknown block]		----------- [unknown block] 168 bytes

	65 91 28 19		65 91 28 19
Line 305:		Line 305:
	00 00 00 00		00 00 00 00

	----------- [font block]		----------- [font block] 48 bytes + 1 string + number_of_records * 24 bytes

	FF FF FF FF separator/pointer		FF FF FF FF separator/pointer

CoDEmanX at 23:17, 10 December 2009

2009-12-10T23:17:26Z

@@ Line 17: / Line 17: @@
 Each file is separated by the separation key: 0xFFFFFFFF. But be aware, each file itself can also contain such a sequence which is not meant to be the separation key! So it is necessary to understand each data type in the FF file.
 == Header ==
@@ Line 150: / Line 153: @@
 == Content ==
-Since there is a FF FF FF FF separator in front of every block and none at the very end (after last 1F-block), we can assume that each block has a start separation and no end separation. Thus, any FF FF FF FFs you find after data blocks are actually the start separators of the next block. This means the index end separator is rather FFFFFFFF than FFFFFFFFFFFFFFFF!
+Since there is a FF FF FF FF separator (respectively pointer) in front of every block and none at the very end (after last 1F-block), we can assume that each block has a start separation and no end separation. Thus, any FF FF FF FFs you find after data blocks are actually the start separators of the next block. This means the index end separator is rather FFFFFFFF than FFFFFFFFFFFFFFFF!
@@ Line 243: / Line 246: @@
 58 54 35 00 00 00 00  fixed size char[] for type? (here: DXT5....)
 -----------  [block end]

Daevius: /* 00 00 00 06 (image) */

2009-12-08T23:15:43Z

00 00 00 06 (image)

← Older revision		Revision as of 23:15, 8 December 2009
Line 224:		Line 224:
	<pre>		<pre>
	----------- [block start]		----------- [block start]
	~~Note: obviously no start separator~~
	03 00 00 00		03 00 00 00
	FE FF FF FF separator?		FE FF FF FF separator?
Line 246:		Line 246:
	----------- [block end]		----------- [block end]
	</pre>		</pre>


	=== 00 00 00 16 (localized string asset) ===		=== 00 00 00 16 (localized string asset) ===

Daevius: /* 00 00 00 16 (localized string asset) */

2009-12-08T23:15:32Z

00 00 00 16 (localized string asset)

← Older revision		Revision as of 23:15, 8 December 2009
Line 279:		Line 279:
	<pre>		<pre>
	----------- [block start]		----------- [block start]
	~~Note: no start separator!~~

	FF FF FF FF either pointer or FFFFFFFF if the string wasn't used before		FF FF FF FF either pointer or FFFFFFFF if the string wasn't used before
Line 288:		Line 287:
	----------- [block end]		----------- [block end]
	</pre>		</pre>


	=== 00 00 00 1F (rawfile) ===		=== 00 00 00 1F (rawfile) ===

Daevius: /* 00 00 00 20 (stringtable) */

2009-12-08T23:15:07Z

00 00 00 20 (stringtable)

← Older revision		Revision as of 23:15, 8 December 2009
Line 313:		Line 313:
	<pre>		<pre>
	----------- [block start]		----------- [block start]

	~~FF FF FF FF start separation~~

	05 00 00 00 (int) columns		05 00 00 00 (int) columns

Daevius: /* 00 00 00 1F (rawfile) */

2009-12-08T23:05:02Z

00 00 00 1F (rawfile)

← Older revision		Revision as of 23:05, 8 December 2009
Line 296:		Line 296:
	<pre>		<pre>
	----------- [block start]		----------- [block start]

	~~FF FF FF FF Start separation~~

	64 00 00 00 Length of the file's content (example: decimal 100; does not count the string termination!)		64 00 00 00 Length of the file's content (example: decimal 100; does not count the string termination!)
Line 308:		Line 306:
	----------- [block end OR even end of file]		----------- [block end OR even end of file]
	</pre>		</pre>


	=== 00 00 00 20 (stringtable) ===		=== 00 00 00 20 (stringtable) ===

Daevius: /* 00 00 00 04 (material) */

2009-12-08T23:04:42Z

00 00 00 04 (material)

← Older revision		Revision as of 23:04, 8 December 2009
Line 157:		Line 157:
	<pre>		<pre>
	----------- [block start]		----------- [block start]

	~~FF FF FF FF start separation~~

	00 2B 01 01 First 4 bytes seem to be 4 (unsigned) char's.		00 2B 01 01 First 4 bytes seem to be 4 (unsigned) char's.
Line 174:		Line 172:
	----------- [block end]		----------- [block end]
	</pre>		</pre>


	=== 00 00 00 05 (shader) ===		=== 00 00 00 05 (shader) ===