Data format

Midas supports two differents data format so far. A possible new candidate would be the NeXus format, but presently no implementation has been developed.

Midas format

Special formats are used in MIDAS for the event header, banks and when writing to disk or tape. This appendix explains these formats in detail. Each event carries a 16-byte header. The header is generated by the front-end with the bm_compose_event() routine and is used by consumers to distinguish between different events. The header is defined in the EVENT_HEADER structure in midas.h. It has following structure:

Event and bank headers with data block.

The event ID describes the type of event. Usually 1 is used for triggered events, 2 for scaler events, 3 for HV events etc. The trigger mask can be used to describe the sub-type of an event. A trigger event can have different trigger sources like "physics event", "calibration event", "clock event". These trigger sources are usually read in by the front-end in a pattern unit. Consumers can request events with a specific triggering mask. The serial number starts at one and is incremented by the front-end for each event. The time stamp is written by the front-end before an event is read out. It uses the time() function which returns the time in seconds since 1.1.1970 00:00:00 UTC. The data size contains the number of bytes that follows the event header. The data area of the event can contain information in any user format, although only certain formats are supported when events are copied to the ODB or written by the logger in ASCII format. Event headers are always kept in the byte ordering of the local machine. If events are sent over the network between computers with different byte ordering, the event header is swapped automatically, but not the event contents.

[Bank Format] Events in MIDAS format contain "MIDAS banks". A bank is a substructure of an event and can contain only one type of data, either a single value or an array of values. Banks have a name of exactly four characters, which are treated, as a bank ID. Banks in an event consist of a global bank header and an individual bank header for each bank. Following picture shows a MIDAS event containing banks:

The "data size total" is the size in bytes of all bank headers and bank data. Flags are currently not used. The bank header contains four characters as identification, a bank type that is one of the TID_xxx values defined in midas.h, and the data size in bytes. If the byte ordering of the contents of a complete event has to be swapped, the routine bk_swap() can be used.

[Tape Format] Events are written to disk files without any reformatting. For tapes, a fixed block size is used. The block size TAPE_BUFFER_SIZE is defined in midas.h and usually 32kB. Three special events are produced by the system. A begin-of-run (BOR) and end-of-run (EOR) event is produced which contains an ASCII dump of the ODB in its data area. Their IDs is 0x8000 (BOR) and 0x8001 (EOR). A message event (ID 0x8002) is created if Log messages is enabled in the logger channel setting. The message is contained in the data area as an ASCII string. The BOR event has the number MIDAS_MAGIC (0x494d or 'MI') as the trigger mask and the current run number as the serial number. A tape can therefore be identified as a MIDAS formatted tape. The routine tape_copy() in the utility mtape.c is an example of how to read a tape in MIDAS format.

YBOS format

As mentioned earlier the YBOS documentation is available at the following URL address: Ybos site Originally YBOS is a collection of FORTRAN functions which facilitate the manipulation of group of data. It also describes a mode of encoding/storing data in an organized way. YBOS defines specific ways for:

Gathering related data (bank structure).
Gathering banks structure (logical record).
Gathering/Writing/Reading logical record from/to storage device such as disk or tape. (Physical record).

YBOS is organized on a 4-byte alignment structure.

The YBOS library function provides all the tools for manipulation of the above mentioned elements in a independent Operating System like. But the implementation of the YBOS part in Midas does not use any reference to the YBOS library code. Instead only the strict necessary functions have to be re-written in C and incorporated into the Midas package. This has been motivated by the fact that only a sub-set of function is essential to the operation of:

The front-end code: for the composition of the YBOS event (bank structure, logical record).
The data logger: for writing data to storage device (physical record).

This Midas/YBOS implementation restricts the user to a subset of the YBOS package only for the front-end part. It doesn't prevent him/her to use the full YBOS library for stand alone program accessing data file written by Midas.

The YBOS implementation under Midas has the following restrictions:

Single leveled bank structures only (no recursive bank allowed).
Bank structure of the following type: ASCII, BINARY, WORD, DOUBLE WORD, IEEE FLOATING.
No mixed data type bank structure allowed.

Logical Record format (Event Format) In the YBOS terminology a logical record refers to a collection of YBOS bank while in the Midas front-end, it can be referred to as an event. The logical record consists of a logical record length of a 32bit-word size followed by a single or collection of YBOS bank. The logical record length counts the number of double word (32bit word) composing the record without counting itself.

YBOS uses "double word" unit for all length references.

[Bank Format] The YBOS bank is composed of a bank header 5 double long words followed by the data section which has to end on a 4 bytes boundary.

Ybos Event and bank headers with data block.

The bank length parameter corresponds to the size of the data section in double word count + 1. The supported bank type are defined in the ybos.h file see YBOS Bank Types.

[Physical Record (Tape/Disk Format)] The YBOS physical record structure is based on a fixed block size (8190 double words) composed of a physical record header followed by data from logical records.

Ybos Physical record structure with data block.

The Offset is computed with the following rules:

If the logical record fits completely in the space of the physical record, the offset value in the physical record header will be 4.
If the block contains first the left over fragment of the previous event started in the previous block, the offset will be equal to the length of the physical record header + the left over fragment size.
If the logical record extent beyond a full block, the offset will be set to -1.
The mark of the end of file is defined with a logical record length set to -1.

Utilities - Top - Supported hardware

Midas DOC Version 1.9.5 ---- PSI Stefan Ritt ----
Contributions: Pierre-Andre Amaudruz - Sergio Ballestrero - Suzannah Daviel - Doxygen - Peter Green - Qing Gu - Greg Hackman - Gertjan Hofman - Paul Knowles - Rudi Meier - Glenn Moloney - Dave Morris - John M O'Donnell - Konstantin Olchanski - Renee Poutissou - Tamsen Schurman - Andreas Suter - Jan M.Wouters - Piotr Adam Zolnierczuk