Article mystery of the blend

mystery of the blend
By jeroen baker.

https://www.foro3d.com/extras/tutori...-blender/0.jpg

Introduction
Im working on a product that integrates Blender into a render pipeline by using the Blender command line and blendfiles (.blend). The command line is not a problem as it is commonly used, but using blend-files outside Blender is dificult, because it is not that well documented. On the internet, ive only found some clues about it on the Blender architecture pages [ref: http://www.blender.org/development/Arc hitecture/] and these were not suficient. To really understand the file format, y had todo go th rouge Blender source code. In this article i Will describe the blend-file-format with a request todo tool-makers todo support blend-file.

First ill describe how Blender works with blend-files. Youll notice why the blend-file-format is not that well documented, as from Blender perspective this is not neded. We look at the global file-structure of a blend-file (the file-Header and file-blocks). After this is explained, we go deper todo the Core of the blend-file, the dna-structures. They hold the blueprints of the blend-file and the key asset of Understanding blend-files. When that done, we can use these dna-structures todo read información from elsewhere in the blend-file. In this article well be using the default blend-file from Blender 2.48, with the goal todo read the output resolution from the scene. The article is written todo be programming lenguaje independent and ive setup a web-site for support.

Loading and saving in Blender
Loading and saving in Blender is very fast and Blender is known todo have excellent downward and upward compatibility. Tom Rosendal demonstrated that in diciembre 2008 by loading a 1.0 blendfile using Blender 2.48a [ref: http://www.blendernation.com/2008/12/01/blenderdna- RNA-and-backward-compatibility/]. Saving complex scenes in Blender is done within seconds. Blender achieves this by saving data in memory todo disk without any transformations or translations. Blender only adds as file-block-headers todo this data. A file-block-Header contains clues on how todo interpret the data. After the data, all internal Blender structures are estored. These structures Will ACT as blue-prints when Blender loads the file. Blendfiles can be diferent when estored on diferent hardware platforms or Blender releases. There is no efort taken todo make blend-files binary the same. Blender has created the blend-files in this manner since reléase 1.0. Backward and upwards compatibility is not implemented when saving the file, this is done during loading.
When Blender loads a blend-file, the dna-structures are read first. Blender creates a catalog of these dna-structures. Blender uses this catalog together with the data in the file, the internal Blender structures of the Blender reléase youre using and a lot of transformation and translation logic todo implement the backward and upward compatibility. In the source code of Blender there is actually logic which can transform and translate every structure used by a Blender reléase todo the one of the reléase youre using [ref: http://download.blender.org/source/blender- 2.48a, (*.tar), (*.gz).]the more diference between releases the more logic is executed.

The blend-file-format is not well documented, as it does not difer from internally used structures and the file can really explain itself.

Global file-structure.

Let us look at the global file-structure. A blend-file always start with the file-Header followed by file-blocks. The default blend file of Blender 2.48 contains more than 400 of these file-blocks. Each file-block has a fileblock- Header and data. This section explains how the global file-structure can be read.

File-header
The first 12 bytes of every blend-file is the file-Header. The file-Header has información on Blender (versiónnumber) and the PC the blend-file was saved on (pointer-size and endianness). This is required as all data inside the blend-file is ordered in that way, because no translation or transformation is done during saving. The next table describes the información in the file-header.

https://www.foro3d.com/extras/tutori...-blender/1.jpg

Endianness addresses the bien values are ordered in a sequence of bytes [ref: http://en.wikipedia.org/wiki/endiannes Blender supports Little-Endian and big-Endian. In a big Endian ordering, the largest part of the value is placed on the first byte and the lowest part of the value is placed on the last byte. In a Little Endian ordering, largest part of the value is placed on the last byte and the smallest part of the value is placed on the first byte. Example: writing the integer 0x4a3b2c1dh, Will be ordered in big Endian as 0x4vaya, 0x3bh, 0 por 2ch, 0 por 1dh and be ordered in Little Endian as 0 por 1dh, 0 por 2ch, 0x3bh, 0x4ah.

The endianness can be diferent between the blend-file and the PC youre using. When these are diferent, Blender changes it todo the byte ordering of your PC. Nowadays, Little-Endian is the most commonly used. The next Hex-dump describes a file-Header created with Blender 2.48 on Little-Endian hardware with a 32 bits pointer length.

https://www.foro3d.com/extras/tutori...-blender/2.jpg

File-block.

File-blocks contain a file-block-Header and data. The start of a file-block is always aligned at 4 bytes. The fileblock- Header describes the total length of the data, the type of información estored in the file-block, the number of Items of this información and the old memory pointer at the momento the data was written todo disk. Depending on the pointer-size estored in the file-Header, a file-block-Header can be 20 or 24 bytes long. The next table describes how a file-block-Header is structured.

https://www.foro3d.com/extras/tutori...-blender/3.jpg

Code describes diferent types of file-blocks. The code determines with what logic the data must be read. These codes a los allow fast finding of data like library, scenes, object or materiales as they all have a specific code. The size contains the total length of data After the fileblock- Header. After the data a new file-block starts. The last file-block in the file has code endb. The old memory address contains the memory address when the structure was last estored. When loading the file the structures can be placed on diferent memory addresses.

https://www.foro3d.com/extras/tutori...-blender/4.jpg

Blender updates pointers todo these structures todo the new memory addresses. Sdna index contains the index in the DNA structures todo be used when reading this file-block-data. More información about this subject Will be explained in the reading scene información section. Count tells how Many elements of the specific sdna structure can be found in the data. The next section is an example of a file-block-Header. The code SC+0x00h identifies that it is a scene. Size of the data is 1376 bytes (0x05h x 256 + 0 por 60h = 1280 + 96), the old pointer is 0x0a042fa0h and the sdna index is 139 (8 por 16 + 11). The section contains a single scene. Before we can interpreted the data of this file-block we first have todo read the DNA structures in the file. The section structure DNA Will show how todo do that.

Structure dna
Structure DNA is estored in a file-block with code dna1. It can be just before the endb file-block. It contains all internal structures of the Blender reléase the file was created in. The data in this file-block must be interpreted as described in this section. In a blend-file created with Blender 2.48a this section is 43468 bytes long and contains 309 structures. These structures can be described as c-structures. They can hold Fields, arrays and pointers todo other structures, just like a normal cstructure.

https://www.foro3d.com/extras/tutori...-blender/5.jpg

The next section describes how this información is ordered in the data of the dna1 file-block.

https://www.foro3d.com/extras/tutori...-blender/6.jpg

As you can se, the structures are estored in 4 arrays: names, types, lengths and structures. Every structure a los contains an array of Fields. A field is the combination of a type and a name. From this información a catalog of all structures can be constructed. The names are estored as how a c-developer defines them. This means that the name a los defines pointers and arrays. (when a name starts with * it is used as a pointer. When the name contains for example [ it is used as a array of 3 long.) in the types, youll find simple-types (like: integer, char, float), but a los complex-types like scene and metaball. tlen part describes the length of the types. A char is 1 byte, an integer is 4 bytes and a scene is 1376 bytes long.

• [note: while reading the DNA youll Will come across some Strange names like (*doit)(). These are method pointers and Blender updates them todo the correct methods.
  
• [note: the Fields type identifier, length identifier and structure identifier are aligned at 4 bytes.

The DNA structures inside a Blender 2.48 blend-file can be found at http://www.atmind.nl/blender/blendersdna, html. If we understand the DNA part of the file it is now posible todo read información from other parts fileblocks. The next section Will tell us how.

Reading scene información
Let us look at the file-block we have sen earlier. The code is SC+0x00h and the sdna index is 139. The 139th structure in the DNA is a structure of type scene. The associated type (scene) has the length of 1376 bytes. This is exact the same length as the data in the fileblock. We can map the scene-structure on the data of the file-blocks. But before we can do that, we have todo Flatten the scene-structure. The first field in the scene-structure is of type id with the name id. Inside the list of DNA structures there is a structure defined for type id (structure index 17). The first field in this structure has type Void and name *next. Looking in the structure list there is no structure defined for type Void. It is a simple type and therefore the data should be read. *next describes a pointer. The first 4 bytes of the data can be mapped todo id, next. Using this method well map a structure todo its data. If we want todo read a specific field we know at what offset in the data it is located and how much space it takes. The next table shows the output of this flattening process for some parts of the scene-structure. Not all rows are described in the table as there is a lot of información in a scene-structure.

https://www.foro3d.com/extras/tutori...-blender/7.jpg

https://www.foro3d.com/extras/tutori...-blender/8.jpg

We can now read the x and y resolution of the scene. The x-resolution is located on offset 326 of the fileblock- Data and must be read as a short. The y-resolution is located on offset 328 and is a los a short.

• [note: an array of chars can mean 2 things. The field contains readable text or it contains an array of flags (not humanly readable).
  
• [note: a file-block containing a list refers todo the DNA structure and has a count larger than 1. For example vertexes and Faces are estored in this way.

Next steps
The implementation of saving in Blender is easy, but loading is dificult. When implementing loading and saving blend-files in a custom tool the dificulty is the opposite. In a custom tool loading a blend-file is easy, and saving a blend-file is dificult. If you want todo save blendfiles i suggest todo start with Understanding the the global file structure and parsing the DNA section of the file. After this is done it should be easy todo read información from existing blend files like scene data, materiales and meshes. When you fel familiar with this you can start creating blend-libraries using the internal Blender structures of a specific release. If you dont want todo dive into the Blender source code you can find them all at http://www.atmind.nl/blender/blender-sdna.html there is a característica request on supporting an xml based import/export system in Blender. I dont support the request, but it is interesting todo look at how this can be implemented. An xml export can be implemented with Low efort as an xsd can be used as DNA structures and the data can be written into xml [se http://www.atmind.nl/blender/blender-file.zip todo download java example including source cod. Implementing an xml import system uses a lot of memory and CPU. If you really want todo implement it, y expect that the easiest bien is todo convert the xml-file bak todo a normal blendfile and then load it using the current implementation. One real drawbak is that parsing a xml based blend-file uses a lot of memory and CPU and the files can become very large.

At this momento im using this información in an automated render pipeline. The render pipeline is build around a web-server and SVN. When an artista commits a new blend-file in SVN, it is picked up by the webserver and it Will extract resolutions, frames scenes and libraries from the blend-file. This información is matched with the other files in SVN and the blend-file Will be placed in the render pipeline.

Jeroen baker jeroen (Ámsterdam, the netherlands, 33 years old) worked as coder in the demo scene. He is interested in open source and 3d animations. At the momento he is working on products supporting impact Analysis and change management around a fully automated render pipeline. Website: http://www.atmind.nl/blender email: j, baker@atmind, nl.

By jeroen baker. www.blenderart.org.