6.2.3 Structure of UIO files

While opening a file for writing, the file format (``formatted'' or ``unformatted'') and the conversion type (``native'', e.g.`ieee_4'', ...) have to be specified.

The formatted ASCII data representation allows I/O independent of platform or compiler. It is useful for parameter files which can be read and edited by hand, for the direct inspection of data, the transfer between very different systems, or for the import of data which exist e.g. in a table format. From the specified conversion type only the default output format for numbers (e.g.`E13.6'' for 4 byte reals) is determined.

The unformatted binary I/O is much faster and gives usually more compact files with higher accuracy (ideally exact) in the numerical data representation. But - in principle - the file format is machine dependent. Fortunately, the IEEE format turns out to become a quasi-standard among a variety of machines. Most workstations work internally with this format. Some CRAYS which have a different internal data representation allow the hidden transformation between the internal and IEEE format during the I/O process. The UIO routines support this feature of CRAY FORTRAN compilers by means of a module (uio_mac_module), individually designed for (two types of) CRAY machines using certain CRAY specific system calls (CRAY FFIO assign logic). Nevertheless, there is also a machine independent version of this module, written completely in standard Fortran90 but providing less features than the machine-dependent versions.

Besides the format, the conversion type (see table 6.2) has to be specified. The ``native'' conversion type is the internal binary-data representation, which is also standard for unformatted Fortran output. If this representation happens to be conformal with the IEEE standard the conversion type ``ieee_4'' should be used. It gives the same data format, but in the header of the file the term ``convert=ieee_4'' instead of ``convert=native'' describes the data format precisely - in a way also understandable by other machines. On CRAY machines the native format is equal to the conversion type ``crayxmp_8'', but also the conversion types ``ieee_4'', and ``ieee_8'' can be chosen. The last two conversion types correspond to the CRAY internal types ``ieee_32'' and ``ieee_64'', respectively.

On a machine with an internal data representation not within the list in the existing uio_mac_module.F90 file, one could use the ``native'' conversion type only. But it is better to invent an appropriate name for the new data format and to build a proper machine-dependent branch with its own compiler switch. However, it was possible in all cases so far to activate some internal conversion process, which allowed the handling of UIO files in the standard Big-Endian-IEEE format.

conversion type	I	R	D	description
`native`	?	?	?	internal data format on all machines
				(sometimes useful but not recommended)
`ieee_4`	4	4	8	standard IEEE `big_endian` format (recommended)
`ieeele_4`	4	4	8	IEEE `little_endian` format
`ieee_8`	8	8	16	double precision IEEE `big_endian` format
				(on some machines possible)
`crayxmp_8`	8	8	16	CRAY internal data format
`idl`	4	4	8	IDL format (but IDL also supports `ieee_4`)
`xdr`	4	4	8	format possible with IDL
`ieee_4_limit`	4	8	8	standard IEEE format $\rightarrow$ `ieee_4`
`ieee`	?	?	?	IEEE format, unknown length (not recommended)

Some attention has to be paid if weird compiler switches (as e.g.tex2html_verb_mark>971) are used to modify the accuracy and standard memory size of variables.

If an existing file is opened for reading, the file format and conversion type are determined automatically from the file if the conversion type of the data in the file is among the conversion types supported by the compiler. If the file has a conversion type ``native'' but is created on a machine with different internal data representation, the file header might be readable, but an error will probably occur during the reading of a real variable.

6.2.3.2 Data file structure

The UIO routines only handle sequential files. Each file consists of a list of entries. The first entry describes the file format, conversion type, and the machine who is responsible for it. The following entries contain data (scalars and 1D ...4D arrays of type integer, real (single & double precision), complex (single precision), character; tables with columns of type integer, real (single precision), or character), or structuring information (labels).

Each header is a list of at most 20 terms separated by blanks or linefeeds. The first term is the entry type (e.g.tex2html_verb_mark>972, see table 6.3), followed by an identifier. This identifier should follow the standard rules for variables (lowercase letters, numbers, underscore; starting with letter). It is a name as e.g.tex2html_verb_mark>973, v_1. The rest of the terms come in the form ``keyword=value''. See Tab. 6.4 for some pre-defined keywords.

entry type	entry contents
`fileform`	file description (first entry)
`integer`	scalars, 1D ...4D arrays
`real`	scalars, 1D ...4D arrays, single & double precision
`complex`	scalars, 1D ...4D arrays, single precision
`character`	scalars, 1D ...4D arrays
`table`	table with integer, real, character columns
`label`	label entry for file structuring

keyword	description	example	descriptor	info.#35022#>	necessary
b	byte number	4	format		yes
d	dimension	(0:9)	format		yes (arrays)
ds	dimension shift	(1:1)		yes
f	Fortran format	E13.6	format		yes
p	values per line	4	format		yes (arrays)
t	transformation	log10	format
n	name	density		yes
u	unit	g/cm`^`3		yes
date	date	1.1.98		yes
c0...9	comment	Dichte		yes
form	file format	formatted	file		yes (file header)
convert	conversion	ieee_4	file		yes (file header)
version	version	0.1.1997.11.29		yes
system	system	IRIX		yes
machine	machine name	atlas		yes
osrelease	OS release	6.3		yes
osversion	OS version	12161207		yes
hardware	machine hardware	IP32		yes
language	program. language	Fortran90		yes
program	program	uiotst		yes
xyz...	user defined	source		yes

A header line has a maximum length of 80 characters. A continuation line is indicated by & at the end of the line. A header consists of 20 lines at maximum. It can be preceded by empty lines (except for the file header entry). Example:

The entry header is followed by the entry data block. This block is empty for labels and the fileform entry but non-empty otherwise.

In an unformatted file each header line is an individual record containing a string with exactly 80 characters. The following data block (scalar or array) is one single record.

In a formatted file each header line is a string of at most 80 characters delimited by a LINEFEED of whatever the operating system decided to be appropriate as EOL character (sequence). The following data block is written as sequence of lines. The number of items per line is specified by the ``p=?'' keyword in the header.

6.2.3.3 Tables

For a table the entry header is followed by a list of headers for the individual table columns, a single table header line consisting of (abbreviations of) the table entry identifiers and the table itself (see the example in section 6.2.2). The dimension keyword gives the number of columns and rows in the form ``d=(1:columns,1:rows)''.

The Holweger-Müller-Atmosphere is chosen as the following ``real world'' example for a file with a table in UIO format:

fileform uio form=formatted convert=ieee_4 version=0.0.1996.10.29 & date='20-Feb-1997 18:40:45' system=SunOS machine=saturn osrelease=4.1.3 & osversion=3 hardware=sun4m language='IDL 4.0' program='by hand' character description d=(0:3) f=A80 p=1 b=80 d='13-Nov-1996 18:29:48' Holweger-Mueller-Atmosphere, Hartmut Holweger & Edith Mueller (1974) Solar Physics 39, 19-30, table II, empirical solar temperature stratification to fit solar spectral lines and limb darkening character history d=(0:3) f=A80 p=1 b=80 d='13-Nov-1996 18:29:52' Holweger-Mueller-Atmosphere, from 1974 uio-form: 13-Nov-1996 18:29:52 conversion type added: 20-Feb-1997 18:21:01 xi -> vmicro: 20-Feb-1997 18:23:43 real teff f=F6.1 b=4 n='effective temperature' u=K texa='T_ eff' 5780.0 table atmosphere d=(1:7,1:29) f=X b=1 n=Holweger-Mueller-Atmosphere & c0='Hartmut Holweger & Edith Mueller (1974) Solar Physics 39, 19-30, table II' & c1=Teff(Sun)=5780K real tauross f=E9.2 b=4 n='optical depth (Rosseland)' u=1 real tau5000 f=F7.3 b=4 n='optical depth (lambda5000)' u=1 t=log10 real t f=F7.0 b=4 n=temperature u=K real pgas f=F6.3 b=4 n='gas pressure' u=dyn/cm^2 real pel f=F6.3 b=4 n='electron pressure' u=dyn/cm^2 real vmicro f=F4.2 b=4 n=microturbulence u=km/s real q f=F8.5 b=4 n='Hopf function' u=1 c0='q=((T(tau)/Teff)^4)/0.75)-tau' tauross tau5000 t pgas pel vmic q 2.00E-07 -6.539 3900. 0.769 -3.140 0.00 0.27637 2.50E-07 -6.279 3920. 1.171 -2.752 0.00 0.28208 5.00E-07 -5.868 3970. 1.598 -2.342 0.00 0.29675 1.00E-06 -5.588 4030. 1.842 -2.105 0.00 0.31510 2.00E-06 -5.334 4080. 2.042 -1.910 0.00 0.33103 5.00E-06 -5.001 4160. 2.279 -1.674 0.00 0.35776 1.00E-05 -4.747 4210. 2.450 -1.508 0.00 0.37527 2.00E-05 -4.486 4270. 2.618 -1.341 0.00 0.39712 5.00E-05 -4.131 4340. 2.835 -1.128 0.00 0.42377 1.00E-04 -3.856 4400. 2.999 -0.967 0.50 0.44765 2.00E-04 -3.577 4460. 3.162 -0.804 0.50 0.47248 5.00E-04 -3.200 4530. 3.377 -0.596 0.50 0.50256 1.00E-03 -2.912 4590. 3.541 -0.437 0.50 0.52925 2.00E-03 -2.621 4640. 3.704 -0.279 0.50 0.55173 5.00E-03 -2.233 4720. 3.919 -0.070 0.50 0.58792 1.00E-02 -1.939 4800. 4.083 0.094 0.50 0.62415 2.00E-02 -1.645 4900. 4.245 0.266 0.65 0.66867 5.00E-02 -1.256 5080. 4.460 0.504 0.85 0.74558 1.00E-01 -0.961 5260. 4.622 0.705 1.00 0.81447 2.50E-01 -0.571 5560. 4.830 1.002 1.25 0.89163 4.00E-01 -0.371 5850. 4.926 1.251 1.40 0.99911 7.00E-01 -0.133 6260. 5.022 1.613 1.50 1.13453 1.00E+00 0.019 6570. 5.070 1.879 1.60 1.22581 1.50E+00 0.191 6880. 5.114 2.140 1.70 1.17659 2.00E+00 0.312 7160. 5.140 2.363 1.80 1.13964 4.00E+00 0.597 7920. 5.191 2.889 1.80 0.70033 6.00E+00 0.761 8250. 5.213 3.092 1.80 -0.46595 8.00E+00 0.877 8420. 5.229 3.196 1.80 -1.99552 1.00E+01 0.967 8500. 5.242 3.245 1.80 -3.76404

6.2.3.4 Recommendations for standard file structure

The very first entry in an UIO file is always the fileform uio entry, containing information about the file format and conversion type. Afterwards, entries can follow in any order. But it is perhaps a good idea to start the file with three special entries (file_id, description, history) as in

A recommended format for sets of multi-dimensional arrays (e.g. hydrodynamics: x-axis, y-axis, z-axis, density, velocities, energy, ...) is shown in Sect.8.1.