Internal features - Top - Data format
The Midas system provides several off-the-shelf programs to control, monitor, debug the data aquisition system. Starting with the main utility (odbedit) which provide access to the Online data base and run control.
The purpose of the Midas Frontend application is to collect data from hardware and transmit this information to a central place where data logging and analysis can be performed. This task is achieved with a) a specific code written by the user describing the sequence of action to acquire the hardware data and b) a framework code handling the data flow control, data transmission and run control operation. From Midas version 1.9.5 a new argument (-i index) has been introduced to facilitate the multiple frontend configuration operation required for the Event Builder Functions.
- Arguments
- [-h ] : help
- [-h hostname ] : host name (see odbedit task)
- [-e exptname ] : experiment name (see odbedit task)
- [-D ] : Become a Daemon.
- [-O ] : Become a Daemon but keep stdout
- [-d ] : Used for debugging.
- [-i index] : Set frontend index (used with mevb task).
odbedit refers to the Online DataBase Editor. This is the main application to interact with the different components of the Midas system.
See ODB Structure for more information.
- Arguments
- [-h ] : help.
- [-h hostname ] :Specifies host to connect to. Must be a valid IP host name. This option supersedes the MIDAS_SERVER_HOST environment variable.
- [-e exptname ] :Specifies the experiment to connect to. This option supersedes the MIDAS_EXPT_NAME environment variable.
- [-c command ] :Perform a single command. Can be used to perform operations in script files.
- [-c @commandFile ] :Perform commands in sequence found in the commandFile.
- [-s size ] : size in byte (for creation). Specify the size of the ODB file to be created when no shared file is present in the experiment directory (default 128KB).
- [-d ODB tree] :Specify the initial entry ODB path to go to.
- Usage ODBedit is the MIDAS run control program. It has a simple command line interface with command line editing similar to the UNIX tcsh shell. Following edit keys are implemented:
- [Backspace] Erase the character left from cursor
- [Delete/Ctrl-D] Erase the character under cursor
- [Ctrl-W/Ctrl-U] Erase the current line
- [Ctrl-K] Erase the line from cursor to end
- [Left arrow/Ctrl-B] Move cursor left
- [Right arrow/Ctrl-F] Move cursor right
- [Home/Ctrl-A] Move cursor to beginning of line
- [End/Ctrl-E] Move cursor to end of line
- [Up arrow/Ctrl-P] Recall previous command
- [Down arrow/Ctrl-N] Recall next command
- [Ctrl-F] Find most recent command which starts with current line
- [Tab/Ctrl-I] Complete directory. The command ls /Sy <tab> yields to ls /System.
- Remarks
- ODBedit treats the hierarchical online database very much like a file system. Most commands are similar to UNIX file commands like ls, cd, chmod, ln etc. The help command displays a short description of all commands.
- From Midas version 1.9.5, the ODB content can be saved into XML format if the file extension is .xml
C:\odbedit
[local:midas:S]/>save odb.xml
[local:midas:S]/>q
more odb.xml
<?xml version="1.0" encoding="ISO-8859-1"?>
<!-- created by ODBEdit on Wed Oct 06 22:48:26 2004 -->
<dir name="root">
<dir name="System">
<dir name="Clients">
<dir name="3880">
<key name="Name" type="STRING" size="32">ebfe01</key>
<key name="Host" type="STRING" size="256">pierre2</key>
<key name="Hardware type" type="INT">42</key>
<key name="Server Port" type="INT">4658</key>
...
[local:midas:Stopped]/>help
Database commands ([] are options, <> are placeholders):
alarm - reset all alarms
cd <dir> - change current directory
chat - enter chat mode
chmod <mode> <key> - change access mode of a key
1=read | 2=write | 4=delete
cleanup - delete hanging clients
copy <src> <dest> - copy a subtree to a new location
create <type> <key> - create a key of a certain type
create <type> <key>[n] - create an array of size [n]
del/rm [-l] [-f] \<key> - delete a key and its subkeys
-l follow links
-f force deletion without asking
exec <key>/<cmd> - execute shell command (stored in key) on server
find <pattern> - find a key with wildcard pattern
help/? [command] - print this help [for a specific command]
hi [analyzer] [id] - tell analyzer to clear histos
ln <source> <linkname> - create a link to <source> key
load <file> - load database from .ODB file at current position
ls/dir [-lhvrp] [<pat>] - show database entries which match pattern
-l detailed info
-h hex format
-v only value
-r show database entries recursively
-p pause between screens
make [analyzer name] - create experim.h
mem - show memeory <b> Usage </b>
mkdir <subdir> - make new <subdir>
move <key> [top/bottom/[n]] - move key to position in keylist
msg [user] <msg> - compose user message
old - display old messages
passwd - change MIDAS password
pause - pause current run
pwd - show current directory
resume - resume current run
rename <old> <new> - rename key
rewind [channel] - rewind tapes in logger
save [-c -s] <file> - save database at current position
in ASCII format
-c as a C structure
-s as a #define'd string
set <key> <value> - set the value of a key
set <key>[i] <value> - set the value of index i
set <key>[*] <value> - set the value of all indices of a key
set <key>[i..j] <value> - set the value of all indices i..j
scl [-w] - show all active clients [with watchdog info]
shutdown <client>/all - shutdown individual or all clients
sor - show open records in current subtree
start [number] [now] [-v] - start a run [with a specific number], [without question]
[-v verbose the transaction to the different clients]
stop [-v] - stop current run
[-v verbose the transaction to the different clients]
trunc <key> <index> - truncate key to [index] values
ver - show MIDAS library version
webpasswd - change WWW password for mhttpd
wait <key> - wait for key to get modified
quit/exit - exit
- Example
>odbedit -c stop
>odbedit
[hostxxx:exptxxx:Running]/> ls /equipment/trigger
mstat is a simple ASCII status display. It presents in a compact form the most valuable information of the current condition of the Midas Acquisition system. The display is composed at the most of 5 sections depending on the current status of the experiment. The section displayed in order from top to bottom refer to:
- Run information.
- Equipment listing and statistics if any frontend is active.
- Logger information and statistics if mlogger is active.
- Lazylogger status if lazylogger is active.
- Client listing.
- Arguments
- [-h ] : help
- [-h hostname ] : host name (see odbedit task)
- [-e exptname ] : experiment name (see odbedit task)
- [-l ] : loop. Forces mstat to remain in a display loop. Enter "!" to terminate the command.
- [-w time ] : refresh rate in second. Specifies the delay in second before refreshing the screen with up to date information. Default: 5 seconds. Has to be used in conjunction with -l switch. Enter "R" to refresh screen on next update.
- Usage
>mstat -l
*-v1.8.0- MIDAS status page -------------------------Mon Apr 3 11:52:52 2000-*
Experiment:chaos Run#:8699 State:Running Run time :00:11:34
Start time:Mon Apr 3 11:41:18 2000
FE Equip. Node Event Taken Event Rate[/s] Data Rate[Kb/s]
B12Y pcch02 67 0.0 0.0
CUM_Scaler vwchaos 23 0.2 0.2
CHV pcch02 68 0.0 0.0
KOS_Scalers vwchaos 330 0.4 0.6
KOS_Trigger vwchaos 434226 652.4 408.3
KOS_File vwchaos 0 0.0 0.0
Target pcch02 66 0.0 0.0
Logger Data dir: /scr0/spring2000 Message File: midas.log
Chan. Active Type Filename Events Taken KBytes Taken
0 Yes Disk run08699.ybs 434206 4.24e+06
Lazy Label Progress File name #files Total
cni-53 100[%] run08696.ybs 15 44.3[%]
Clients: MStatus/koslx0 Logger/koslx0 Lazy_Tape/koslx0
CHV/pcch02 MChart1/umelba ODBEdit/koslx0
CHAOS/vwchaos ecl/koslx0 Speaker/koslx0
MChart/umelba targetFE/pcch02 HV_MONITOR/umelba
SUSIYBOS/koslx0 History/kosal2 MStatus1/dasdevpc
*------------------------------------------------------------------------------*
analyzer is the main online / offline event analysis application. analyzer uses fully the ODB capabilities as all the analyzer parameters are dynamically controllable from the Online Database editor odbedit task.
For more detailed information see MIDAS Analyzer
- Arguments
- -c <filename1> <filename2> Configuration file name(s). May contain a '%05d' to be replaced by the run number. Up to ten files can be specified in one "-c" statement.
- -d Debug flag when starts the analyzer from a debugger. Prevents the system to kill the analyzer when the debugger stops at a breakpoint
- -D Start analyzer as a daemon in the background (UNIX only).
- -e <experiment> MIDAS experiment to connect to. (see odbedit task)
- -f Filter mode. Write original events to output file only if the analyzer accepts them (doesn't return ANA_SKIP).
- -h <hostname> MIDAS host to connect to when running the analyzer online (see odbedit task)
- -i <filename1> <filename2> Input file name. May contain a '%05d' to be replaced by the run number. Up to ten input files can be specified in one "-i" statement.
- -l If set, don't load histos from last histo file when running online.
- -L HBOOK LREC size. Default is 8190.
- -n <count> Analyze only "count" events.
- -n <first> <last> Analyze only events from "first" to "last".
- -n <first> <last> <n> Analyze every n-th event from "first" to "last".
- -o <filename> Output file name. Extension may be .mid (MIDAS binary), .asc (ASCII) or .rz (HBOOK). If the name contains a '%05d', one output file is generated for each run. Use "OFLN" as output file name to creaate a HBOOK shared memory instead of a file.
- -p <param=value> Set individual parameters to a specific value. Overrides any setting in configuration files
- -P <ODB tree> Protect an ODB subtree from being overwritten with the online data when ODB gets loaded from .mid file
- -q Quiet flag. If set, don't display run progress in offline mode.
- -r <range> Range of run numbers to analyzer like "-r 120 125" to analyze runs 120 to 125 (inclusive). The "-r" flag must be used with a '%05d' in the input file name.
- -s <port#> Specify the ROOT server TCP/IP port number (default 9090).
- -v Verbose output.
- -w Produce row-wise N-tuples in outpur .rz file. By default, column-wise N-tuples are used.
- Remarks
- The creation of the experim.h is done through the odbedit> make <analyzer>. In order to include your analyzer section, the ODB /<Analyzer>/Parameters has to be present.
- Usage
>analyzer
>analyzer -D -r 9092
>analyzer -i run00023.mid -o run00023.rz -w
>analyzer -i run%05d.mid -o runall.rz -r 23 75 -w
mlogger is the main application to collect data from the different frontends under certain conditions and store them onto physical device such as disk or tape. It also acts as a history event collector if either the history flags are enabled in the frontend equipment (see The Equipment structure or if the ODB tree /History/Links is defined (See History system). See the ODB /Logger Tree for reference on the tree structure.
- Arguments
- [-h ] : help
- [-e exptname ] : experiment name (see odbedit task)
- [-D ] : start program as a daemon (UNIX only).
- [-s] : Save mode (debugging: protect ODB).
- [-v] : Verbose (not to be used in conjunction with -D).
- Remarks
- The mlogger application requires to have an existing /Equipment/ tree in the ODB!
- As soon as the mlogger starts to run, the history mechanism is enabled.
- The data channels as well as the history logging is rescanned automatically at each "begin of run" transition. In other word, additional channel can be defined while running but effect will take place only at the following begin of run transition.
- The default setting defines a data "Midas" format with a file name of the type "run\%05d.mid". Make sure this is the requested setting for your experiment.
- Once the mlogger is running, you should be able to monitor its state. through the mstat task or through the mhttpd task web browser.
- From version 1.9.5
- mlogger will not run if started remotely (argument -h hostname has been removed).
- The file size limitation (<2GB) has been removed for older OS version.
- mySQL data entry support.
lazylogger is an application which decouples the data aquisition from the data logging mechanism. The need of such application has been dictated by the slow response time of some of the media logging devices (Tape devices). Delay due to tape mounting, retension, reposition implies that the data acquisition has to hold until operation completion. By using mlogger to log data to disk in a first stage and then using lazylogger to copy or move the stored files to the "slow device" we can keep the acquisition running without interruption.
- Multiple lazyloggers can be running comtemporary on the same computer, each one taking care of a particular channel.
- Each lazylogger channels will have a dedicated ODB tree containg its own information.
- All the lazylogger channel will be under the ODB /Lazy/<channel_name>/...
- Each channel tree is composed of three sub-tree Settings, Statistics, List.
Self-explanatory the Settings and Statistics contains the running operation of the channel. While the List- will have a dynamic list of run number which has been sucessfully manipulated by the Lazylogger channel. This list won't exist until the first successful operation of the channel is completed.
- Arguments
- [-h ] : help.
- [-h hostname ] : host name.
- [-e exptname ] : experiment name.
- [-D ] : start program as a daemon.
- [-c channel ] : logging channel. Specify the lazylogger to activate.
- [-z ] : zap statistics. Clear the statistics tree of all the defined lazylogger channels.
- ODB parameters (Settings/)
Settings DIR
Maintain free space(%) INT 1 4 3m 0 RWD 0
Stay behind INT 1 4 3m 0 RWD -3
Alarm Class STRING 1 32 3m 0 RWD
Running condition STRING 1 128 3m 0 RWD ALWAYS
Data dir STRING 1 256 3m 0 RWD /home/midas/online
Data format STRING 1 8 3m 0 RWD MIDAS
Filename format STRING 1 128 3m 0 RWD run%05d.mid
Backup type STRING 1 8 3m 0 RWD Tape
Execute after rewind STRING 1 64 3m 0 RWD
Path STRING 1 128 3m 0 RWD
Capacity (Bytes) FLOAT 1 4 3m 0 RWD 5e+09
List label STRING 1 128 3m 0 RWD
Execute before writing file STRING 1 64 11h 0 RWD lazy_prewrite.csh
Execute after writing file STRING 1 64 11h 0 RWD rundb_addrun.pl
Modulo.Position STRING 1 8 11h 0 RWD 2.1
Tape Data Append BOOL 1 4 11h 0 RWD y
- [Maintain free space] As the Data Logger (mlogger) runs independently from the Lazylogger, the disk contains all the recorded data files. Under this condition, Lazylogger can be instructed to "purge" the data logging device (disk) after successful backup of the data onto the "slow device". The Maintain free space(%) parameter controls (if none zero) the percentage of disk space required to be maintain free.
- The condition for removing a data file is defined as: The data file corresponding to the given run number following the format declared under "Settings/Filename format" IS PRESENT on the "Settings/Data Dir" path. AND The given run number appears anywhere under the "List/" directory of ALL the Lazy channel having the same "Settings/Filename format"as this channel. AND The given run number appears anywhere under the "List/" directory of that channel
- [Stay behind] This parameter defines how many consecutive data files should be kept between the current run and the last lazylogger run.
- Example with "Stay behind = -3" :
- Current acquisition run number 253 -> run00253.mid is being logged by mlogger.
- Files available on the disk corresponding to run #248, #249, #250, #251, #252.
- Lazylogger will start backing up run #250 as soon as the new run 254 starts. -3 "Stay behind = -3" corresponds to 3 file untouched on the disk (#251, #252, #253). The negative sign instructs the lazylogger to always scan the entire "Data Dir" from the oldest to the most recent file sitting on the disk at the "Data Dir" path- for backup. If the "Stay behind" is positive, lazylogger will backup starting from- x behind the current acquisition run number. Run order will be ignored.
- [Alarm Class] Specify the Alarm class to be used in case of triggered alarm.
- [Running condition] Specify the type of condition for which the lazylogger should be actived. By default lazylogger is ALWAYS running. In the case of high data rate acquisition it could be necessary to activate the lazylogger only when the run is either paused, stopped or when some external condition is satisfied such as "Low beam intensity". In this latter case, condition based on a single field of the ODB can be given to establish when the application should be active.
- Example :
odbedit> set "Running condition" WHILE_ACQ_NOT_RUNNING
odbedit> set "Running condition" "/alias/max_rate < 200"
- [Data dir] Specify the Data directory path of the data files. By default if the "/Logger/Data Dir" is present, the pointed value is taken otherwise the current directory where lazylogger has been started is used.
- [Data format] Specify the Data format of the data files. Currently supported formats are: MIDAS and YBOS.
- [Filename format] Specify the file format of the data files. Same format as given for the data logger.
- [Backup type] Specify the "slow device" backup type. Default Tape. Can be Disk or Ftp.
- [Execute after rewind] Specify a script to run after completion of a lazylogger backup set (see below "Capacity (Bytes)").
- [Path] Specify the "slow device" path. Three possible types of Path:
- For Tape : /dev/nst0- (UNIX like).
- For Disk : /data1/myexpt
- For Ftp : host,port,user,password,directory
- [Capacity (Bytes)] Specify the maximum "slow device" capacity in bytes. When this capacity is reached,the lazylogger will close the backup device and clear the "List Label" field to prevent further backup (see below). It will aslo rewind the stream device if possible.
- [List label] Specify a label for a set of backed up files to the "slow device". This label is used only internaly by the lazylogger for creating under the "/List" a new array composed of the backed up runs until the "Capacity" value has been reached. As the backup set is complete, lazylogger will clear this field and therefore prevent any further backup until a none empty label list is entered again. In the other hand the list label will remain under the "/List" key to display all run being backed up until the corresponding files have been removed from the disk.
- [Exec preW file] Permits to run a script before the begining of the lazy job. The arguments passed to the scripts are: input file name , output file name, current block number.
- [Exec postW file] Permits to run a script after the completion of the lazy job. The arguments passed to the scripts are: list label, current job number, source path, file name, file size in MB, current block number.
- [Modulo.Position] This field is for multiple instances of the lazylogger where each instance works on a sub-set of run number. By specifying the Modulo.Position you're telling the current lazy instance how many instances are simultaneously running (3.) and the position of which this instance is assigned to (.1). As an example for 3 lazyloggers running contemporaneously the field assignment should be :
Channel Field Run#
Lazy_1 3.0 21, 24, 27, ...
Lazy_2 3.1 22, 25, 28, ...
Lazy_3 3.2 23, 26, 29, ...
- [Tape Data Append] Enable the spooling of the Tape device to the End_of_Device (EOD) before starting the lazy job. This command is valid only for "Backup Type" Tape. If this flag is not enabled the lazy job starts at the current tape position.
- [Statistics/] ODB tree specifying general information about the status of the current lazylogger channel state.
- [List/] ODB tree, will contain arrays of run number associated with the array name backup-set label. Any run number appearing in any of the arrays is considered to have been backed up.
- Usage lazylogger requires to be setup prior data file can be moved. This setup consists of 4 steps:
- [Step 1] Invoking the lazylogger once for setting up the appropriate ODB tree and exit.
- [Step 2] Edit the newly created ODB tree. Correct the setting field to match your requirement.
> odbedit -e midas
[local:midas:Stopped]/>cd /Lazy/tape/
[local:midas:Stopped]tape>ls
[local:midas:Stopped]tape>ls -lr
Key name Type #Val Size Last Opn Mode Value
---------------------------------------------------------------------------
tape DIR
Settings DIR
Maintain free space(%) INT 1 4 3m 0 RWD 0
Stay behind INT 1 4 3m 0 RWD -3
Alarm Class STRING 1 32 3m 0 RWD
Running condition STRING 1 128 3m 0 RWD ALWAYS
Data dir STRING 1 256 3m 0 RWD /home/midas/online
Data format STRING 1 8 3m 0 RWD MIDAS
Filename format STRING 1 128 3m 0 RWD run%05d.mid
Backup type STRING 1 8 3m 0 RWD Tape
Execute after rewind STRING 1 64 3m 0 RWD
Path STRING 1 128 3m 0 RWD
Capacity (Bytes) FLOAT 1 4 3m 0 RWD 5e+09
List label STRING 1 128 3m 0 RWD
Statistics DIR
Backup file STRING 1 128 3m 0 RWD none
File size [Bytes] FLOAT 1 4 3m 0 RWD 0
KBytes copied FLOAT 1 4 3m 0 RWD 0
Total Bytes copied FLOAT 1 4 3m 0 RWD 0
Copy progress [%] FLOAT 1 4 3m 0 RWD 0
Copy Rate [bytes per s] FLOAT 1 4 3m 0 RWD 0
Backup status [%] FLOAT 1 4 3m 0 RWD 0
Number of Files INT 1 4 3m 0 RWD 0
Current Lazy run INT 1 4 3m 0 RWD 0
[local:midas:Stopped]tape>cd Settings/
[local:midas:Stopped]Settings>set "Data dir" /data
[local:midas:Stopped]Settings>set "Capacity (Bytes)" 15e9
- [Step 3] Start lazylogger in the background
- [Step 4] At this point the lazylogger is running and waiting for the "list label" to be defined before starting the copy procedure. mstat task will display information regarding the status of the lazylogger.
> odbedit -e midas
[local:midas:Stopped]/>cd /Lazy/tape/Settings
[local:midas:Stopped]Settings>set "List label" cni-043
- Remarks
- For every major operation of the lazylogger a message is sent to the Message buffer and will be appended to the default Midas log file (midas.log). These messages are the only mean of finding out What/When/Where/How the lazylogger has operated on a data file. See below a fragment of the midas::log for the chaos experiment. In this case the Maintain free space() field was enabled which produces the cleanup of the data files and the entry in the List tree after copy.
Fri Mar 24 14:40:08 2000 [Lazy_Tape] 8351 (rm:16050ms) /scr0/spring2000/run08351.ybs file REMOVED
Fri Mar 24 14:40:08 2000 [Lazy_Tape] Tape run#8351 entry REMOVED
Fri Mar 24 14:59:55 2000 [Logger] stopping run after having received 1200000 events
Fri Mar 24 14:59:56 2000 [CHAOS] Run 8366 stopped
Fri Mar 24 14:59:56 2000 [Logger] Run #8366 stopped
Fri Mar 24 14:59:57 2000 [SUSIYBOS] saving info in run log
Fri Mar 24 15:00:07 2000 [Logger] starting new run
Fri Mar 24 15:00:07 2000 [CHAOS] Run 8367 started
Fri Mar 24 15:00:07 2000 [Logger] Run #8367 started
Fri Mar 24 15:06:59 2000 [Lazy_Tape] cni-043[15] (cp:410.6s) /dev/nst0/run08365.ybs 864.020MB file NEW
Fri Mar 24 15:07:35 2000 [Lazy_Tape] 8352 (rm:25854ms) /scr0/spring2000/run08352.ybs file REMOVED
Fri Mar 24 15:07:35 2000 [Lazy_Tape] Tape run#8352 entry REMOVED
Fri Mar 24 15:27:09 2000 [Lazy_Tape] 8353 (rm:23693ms) /scr0/spring2000/run08353.ybs file REMOVED
Fri Mar 24 15:27:09 2000 [Lazy_Tape] Tape run#8353 entry REMOVED
Fri Mar 24 15:33:22 2000 [Logger] stopping run after having received 1200000 events
Fri Mar 24 15:33:22 2000 [CHAOS] Run 8367 stopped
Fri Mar 24 15:33:23 2000 [Logger] Run #8367 stopped
Fri Mar 24 15:33:24 2000 [SUSIYBOS] saving info in run log
Fri Mar 24 15:33:33 2000 [Logger] starting new run
Fri Mar 24 15:33:34 2000 [CHAOS] Run 8368 started
Fri Mar 24 15:33:34 2000 [Logger] Run #8368 started
Fri Mar 24 15:40:18 2000 [Lazy_Tape] cni-043[16] (cp:395.4s) /dev/nst0/run08366.ybs 857.677MB file NEW
Fri Mar 24 15:50:15 2000 [Lazy_Tape] 8354 (rm:28867ms) /scr0/spring2000/run08354.ybs file REMOVED
Fri Mar 24 15:50:15 2000 [Lazy_Tape] Tape run#8354 entry REMOVED
...
- Once lazylogger has started a job on a data file, trying to terminate the application will result in producing a log message informing the actual percentage of the backup completed so far. This message will repeat it self until completion of the backup and only then the lazylogger application will terminate.
- If an interruption of the lazylogger is forced (kill...) The state of the backup device is undertermined. Recovery is not possible and the full backup set has to be redone. In order to do this, you need:
- To rewind the backup device.
- Delete the /Lazy/<channel_name>/List/<list label> array.
- Restart the lazylogger with the -z switch which will "zap" the statistics entries.
- In order to facilitate the recovery procedure, lazylogger produces an ODB ASCII file of the lazy channel tree after completion of successful operation. This file (Tape_recover.odb) stored in Data_Dir can be used for ODB as well as lazylogger recovery.
This application allows to "peep" into the data flow in order to display a snap-shot of the event. Its use is particularly powerful during experiment setup. In addition mdump has the capability to operate on data save-set files stored on disk or tape. The main mdump restriction is the fact that it works only for events formatted in banks (i.e.: MIDAS, YBOS bank).
- Arguments for Online
- [-h ] : help for online use.
- [-h hostname ] : Host name.
- [-e exptname ] : Experiment name.
- [-b bank name] : Display event containg only specified bank name.
- [-c compose] : Retrieve and compose file with either Add run# or Not (def:N).
- [-f format] : Data representation (x/d/ascii) def:hex.
- [-g type ] : Sampling mode either Some or All (def:S). >>> in case of -c it is recommented to used -g all.
- [-i id ] : Event Id.
- [-j ] : Display bank header only.
- [-k id ] : Event mask. >>> -i and -k are valid for YBOS ONLY if EVID bank is present in the event
- [-l number ] : Number of consecutive event to display (def:1).
- [-m mode] : Display mode either Bank or Raw (def:B)
- [-p path] : Path for file composition (see -c)
- [-s ] : Data transfer rate diagnositic.
- [-w time] : Insert wait in [sec] between each display.
- [-x filename ] : Input channel. data file name of data device. (def:online)
- [-y ] : Display consistency check only.
- [-z buffer name] : Midas buffer name to attach to (def:SYSTEM)
- Additional Arguments for Offline
- [-x -h ] : help for offline use.
- [-t type ] : Bank format (Midas/Ybos). >>> if -x is a /dev/xxx, -t has to be specified.
- [-r #] : skip record(YBOS) or event(MIDAS) to #.
- [-w what] : Header, Record, Length, Event, Jbank_list (def:E) >>> Header & Record are not supported for MIDAS as it has no physical record structure.
- Usage mdump can operate on either data stream (online) or on save-set data file. Specific help is available for each mode.
Tue> mdump -x run37496.mid | more
------------------------ Event# 0 --------------------------------
------------------------ Event# 1 --------------------------------
Evid:0001- Mask:0100- Serial:1- Time:0x393c299a- Dsize:72/0x48
#banks:2 - Bank list:-SCLRRATE-
Bank:SCLR Length: 24(I*1)/6(I*4)/6(Type) Type:Integer*4
1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
Bank:RATE Length: 24(I*1)/6(I*4)/6(Type) Type:Real*4 (FMT machine dependent)
1-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
------------------------ Event# 2 --------------------------------
Evid:0001- Mask:0004- Serial:1- Time:0x393c299a- Dsize:56/0x38
#banks:2 - Bank list:-MMESMMOD-
Bank:MMES Length: 24(I*1)/6(I*4)/6(Type) Type:Real*4 (FMT machine dependent)
1-> 0x3de35788 0x3d0b0e29 0x00000000 0x00000000 0x3f800000 0x00000000
Bank:MMOD Length: 4(I*1)/1(I*4)/1(Type) Type:Integer*4
1-> 0x00000001
------------------------ Event# 3 --------------------------------
Evid:0001- Mask:0008- Serial:1- Time:0x393c299a- Dsize:48/0x30
#banks:1 - Bank list:-BMES-
Bank:BMES Length: 28(I*1)/7(I*4)/7(Type) Type:Real*4 (FMT machine dependent)
1-> 0x443d7333 0x444cf333 0x44454000 0x4448e000 0x43bca667 0x43ce0000 0x43f98000
------------------------ Event# 4 --------------------------------
Evid:0001- Mask:0010- Serial:1- Time:0x393c299a- Dsize:168/0xa8
#banks:1 - Bank list:-CMES-
Bank:CMES Length: 148(I*1)/37(I*4)/37(Type) Type:Real*4 (FMT machine dependent)
1-> 0x3f2f9fe2 0x3ff77fd6 0x3f173fe6 0x3daeffe2 0x410f83e8 0x40ac07e3 0x3f6ebfd8 0x3c47ffde
9-> 0x3e60ffda 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x3f800000
17-> 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
25-> 0x3f800000 0x3f800000 0x3f800000 0x00000000 0x3f800000 0x00000000 0x3f800000 0x3f800000
33-> 0x3f800000 0x3f800000 0x3f800000 0x3f800000 0x00000000
------------------------ Event# 5 --------------------------------
Evid:0001- Mask:0020- Serial:1- Time:0x393c299a- Dsize:32/0x20
#banks:1 - Bank list:-METR-
Bank:METR Length: 12(I*1)/3(I*4)/3(Type) Type:Real*4 (FMT machine dependent)
1-> 0x00000000 0x39005d87 0x00000000
...
mevb is an event builder application taking several frontends Midas data source and assembles a new overall Midas event.
In the case where overall data collection is handled by multiple physically separated frontends, it could be necessary to assemble these data fragments into a dedicated event. The synchonization of the fragment collection is left to the user which is done usually through specific hardware mechanism. Once the fragments are composed in each frontend, they are sent to the "Event Builder" (eb) where the serial number (pheader->serial_number) of each fragment is compared one event at a time for serial match. In case of match, a new event will be composed with its own event ID and serial number followed by all the expected fragments. The composed event is then sent to the next stage which is usually the data logger (mlogger).
The mhttpd task will present the status of the event builder as an extra equipment with its corresponding statistical information.
- Arguments
- [-h ] : help
- [-h hostname ] : host name
- [-e exptname ] : experiment name
- [-b ] : Buffer name
- [-v ] : Show wheel
- [-d ] : debug messages
- [-D ] : start program as a daemon
- Usage
Thu> mevb -e midas
Program mevb/EBuilder version 2 started
mspeaker, mlxspeaker are utilities which listen to the Midas messages system and pipe these messages to a speech synthesizer application. mspeaker is for the Windows based system and interface to the FirstByte/ProVoice package. The mlxspeaker is for Linux based system and interface to the Festival. In case of use of either package, the speech synthesis system has to be installed prior to the activation of the mspeaker, mlxspeaker.
- Arguments
- [-h ] : help
- [-h hostname ] : host name
- [-e exptname ] : experiment name
- [-t mt_talk_cmd] : Specify the talk alert command (ux only).
- [-u mt_user_cmd] : Specify the user alert command (ux only).
- [-s shut up time]: Specify the min time interval between alert [s] The -t & -u switch require a command equivalent to: '-t play --volume=0.3 file.wav'
- [-D ] : start program as a daemon
mcnaf is an interactive CAMAC tool which allows "direct" access to the CAMAC hardware. This application is operational under either of the two following conditions:
- mcnaf has been built against a particular CAMAC driver (see CAMAC drivers).
- A user frontend code using a valid CAMAC driver is currently active. In this case the frontend acts as a RPC CAMAC server and will handle the CAMAC request. This last option is only available if the frontend code (mfe.c) from the Building Options has included the HAVE_CAMAC pre-compiler flag.
- Arguments
- [-h ] : help
- [-h hostname ] : host name
- [-e exptname ] : experiment name
- [-f frontend name] : Frontend name to connect to.
- [-s RPC server name] : CAMAC RPC server name for remote connection.
- Building application The midas/utils/makefile.mcnaf will build a collection of mcnaf applications which are hardware dependent, see Example below:
- [miocnaf] cnaf application using the declared CAMAC hardware DRIVER (kcs2927 in this case). To be used with dio CAMAC application starter (see dio task).
- [mwecnaf] cnaf application using the WI-E-N-ER PCI/CAMAC interface (see CAMAC drivers). Please contact: midas@triumf.ca for further information.
- [mcnaf] cnaf application using the CAMAC RPC capability of any Midas frontend program having CAMAC access.
- [mdrvcnaf] cnaf application using the Linux CAMAC driver for either kcs2927, kcs2926, dsp004. This application would require to have the proper Linux module loaded in the system first. Please contact mailto:midas@triumf.ca for further information.
Thu> cd /midas/utils
Thu> make -f makefile.mcnaf DRIVER=kcs2927
gcc -O3 -I../include -DOS_LINUX -c -o mcnaf.o mcnaf.c
gcc -O3 -I../include -DOS_LINUX -c -o kcs2927.o ../drivers/bus/kcs2927.c
gcc -O3 -I../include -DOS_LINUX -o miocnaf mcnaf.o kcs2927.o ../linux/lib/libmidas.a -lutil
gcc -O3 -I../include -DOS_LINUX -c -o wecc32.o ../drivers/bus/wecc32.c
gcc -O3 -I../include -DOS_LINUX -o mwecnaf mcnaf.o wecc32.o ../linux/lib/libmidas.a -lutil
gcc -O3 -I../include -DOS_LINUX -c -o camacrpc.o ../drivers/bus/camacrpc.c
gcc -O3 -I../include -DOS_LINUX -o mcnaf mcnaf.o camacrpc.o ../linux/lib/libmidas.a -lutil
gcc -O3 -I../include -DOS_LINUX -c -o camaclx.o ../drivers/bus/camaclx.c
gcc -O3 -I../include -DOS_LINUX -o mdrvcnaf mcnaf.o camaclx.o ../linux/lib/libmidas.a -lutil
rm *.o
- Running application
- Direct CAMAC access: This requires the computer to have the proper CAMAC interface installed and the BASE ADDRESS matching the value defined in the corresponding CAMAC driver. For kcs2926.c, kcs2927.c, dsp004.c, hyt1331.c, the base address (CAMAC_BASE) is set to 0x280.
- RPC CAMAC through frontend: This requires to have a frontend running which will be able to serve the CAMAC RPC request. Any Midas frontend has that capability built-in but it has to have the proper CAMAC driver included in it.
>mcnaf -e <expt> -h <host> -f <fe_name>
Electronic Log utility. Submit full Elog entry to the specified Elog port.
- Arguments
- [-h ] : help
- [-h hostname ] : host name
- [-l exptname or logbook ]
- [-u username password ]
- [-f <attachment> ] : up to 10 files.
- -a <attribute>=<value> : up to 20 attributes. The attribute "Author=..." must at least be present for submission of Elog.
- -m <textfile> | text> Arguments with blanks must be enclosed in quotes. The elog message can either be submitted on the command line or in a file with the -m flag. Multiple attributes and attachments can be supplied.
- Usage By default the attributes are "Author", "Type", "System" and "Subject". The "Author" attribute has to be present in the elog command in order to successfully submit the message. If multiple attributes are required append before "text" field the full specification of the attribute. In case of multiple attachements, only one "-f" is required followed by up to 10 file names.
>melog -h myhost -p 8081 -l myexpt -a author=pierre "Just a elog message"
>melog -h myhost -p 8081 -l myexpt -a author=pierre -f file2attach.txt \
"Just this message with an attachement"
>melog -h myhost -p 8081 -l myexpt -a author=pierre -m file_containing_the_message.txt
>melog -h myhost -p 8081 -l myexpt -a Author=pierre -a Type=routine -a system=general \
-a Subject="my test" "A full Elog message"
- Remarks
History data retriever.
- Arguments
- [-h ] : help
- [-e Event ID] : specify event ID
- [-v Variable Name] : specify variable name for given Event ID
- [-i Index] : index of variables which are arrays
- [-i Index1:Index2] index range of variables which are arrays (max 50)
- [-t Interval] : minimum interval in sec. between two displayed records
- [-h Hours] : display between some hours ago and now
- [-d Days] : display between some days ago and now
- [-f File] : specify history file explicitly
- [-s Start date] : specify start date DDMMYY[.HHMM[SS]]
- [-p End date] : specify end date DDMMYY[.HHMM[SS]]
- [-l] : list available events and variables
- [-b] : display time stamp in decimal format
- [-z] : History directory (def: cwd).
- Usage
- Example
--- All variables of event ID 9 during last hour with at least 5 minutes interval.
> mhist
Available events:
ID 9: Target
ID 5: CHV
ID 6: B12Y
ID 20: System
Select event ID: 9
Available variables:
0: Time
1: Cryostat vacuum
2: Heat Pipe pressure
3: Target pressure
4: Target temperature
5: Shield temperature
6: Diode temperature
Select variable (0..6,-1 for all): -1
How many hours: 1
Interval [sec]: 300
Date Time Cryostat vacuum Heat Pipe pressure Target pressure Target temperature Shield temperature Diode temperature
Jun 19 10:26:23 2000 104444 4.614 23.16 -0.498 22.931 82.163 40
Jun 19 10:31:24 2000 104956 4.602 23.16 -0.498 22.892 82.108 40
Jun 19 10:36:24 2000 105509 4.597 23.099 -0.498 22.892 82.126 40
Jun 19 10:41:33 2000 110021 4.592 23.16 -0.498 22.856 82.08 40
Jun 19 10:46:40 2000 110534 4.597 23.147 -0.498 22.892 82.117 40
Jun 19 10:51:44 2000 111046 4.622 23.172 -0.498 22.907 82.117 40
Jun 19 10:56:47 2000 111558 4.617 23.086 -0.498 22.892 82.117 40
Jun 19 11:01:56 2000 112009 4.624 23.208 -0.498 22.892 82.117 40
Jun 19 11:07:00 2000 112521 4.629 23.172 -0.498 22.896 82.099 40
Jun 19 11:12:05 2000 113034 4.639 23.074 -0.498 22.896 82.117 40
Jun 19 11:17:09 2000 113546 4.644 23.172 -0.498 22.892 82.126 40
Jun 19 11:22:15 2000 114059 4.661 23.16 -0.498 22.888 82.099 40
- Single variable "I-WC1+_Anode" of event 5 every hour over the full April 24/2000.
mhist -e 5 -v "I-WC1+_Anode" -t 3600 -s 240400 -p 250400
Apr 24 00:00:09 2000 160
Apr 24 01:00:12 2000 160
Apr 24 02:00:13 2000 160
Apr 24 03:00:14 2000 160
Apr 24 04:00:21 2000 180
Apr 24 05:00:26 2000 0
Apr 24 06:00:31 2000 160
Apr 24 07:00:37 2000 160
Apr 24 08:00:40 2000 160
Apr 24 09:00:49 2000 160
Apr 24 10:00:52 2000 160
Apr 24 11:01:01 2000 160
Apr 24 12:01:03 2000 160
Apr 24 13:01:03 2000 0
Apr 24 14:01:04 2000 0
Apr 24 15:01:05 2000 -20
Apr 24 16:01:11 2000 0
Apr 24 17:01:14 2000 0
Apr 24 18:01:19 2000 -20
Apr 24 19:01:19 2000 0
Apr 24 20:01:21 2000 0
Apr 24 21:01:23 2000 0
Apr 24 22:01:32 2000 0
Apr 24 23:01:39 2000 0
- Remarks : History data can be retrieved and displayed through the Midas web page (see mhttpd task).
- Example Midas Web History display.
mchart is a periodic data retriever of a specific path in the ODB which can be used in conjunction with a stripchart graphic program.
- In the first of two step procedure, a specific path in the ODB can be scanned for composing a configuration file by extracting all numerical data references file.conf .
- In the second step the mchart will produce at fix time interval a refreshed data file containing the values of the numerical data specified in the configuration file. This file is then available for a stripchart program to be used for chart recording type of graph.
Two possible stripchart available are:
- gstripchart The configuration file generated by mchart is compatible with the GNU stripchart which permits sofisticated data equation manipulation. On the other hand, the data display is not very fency and provides just a basic chart recorder.
- stripchart.tcl file This tcl/tk application written by Gertjan Hofman provides a far better graphical chart recorder display tool, it also permits history save-set display, but the equation scheme is not implemented.
- Arguments
- [-h ] : help
- [-h hostname ] : host name.
- [-e exptname ] : experiment name.
- [-D ] : start program as a daemon.
- [-u time] : data update periodicity (def:5s).
- [-f file] : file name (+.conf: if using existing file).
- [-q ODBpath] : ODB tree path for extraction of the variables.
- [-c ] : ONLY creates the configuration file for later use.
- [-b lower_value] : sets general lower limit for all variables.
- [-t upper_value] : sets general upper limit for all variables.
- [-g ] : spawn the graphical stripchart if available.
- [-gg ] : force the use of gstripchart for graphic.
- [-gh ] : force the use of stripchart (tcl/tk) for graphic.
- Usage : The configuration contains one entry for each variable found under the ODBpath requested. The format is described in the gstripchart documentation.
Once the configuration file has been created, it is possible to apply any valid operation (equation) to the parameters of the file following the gstripchart syntax.
In the case of the use of the stripchart from G.Hofman, only the "filename", "pattern", "maximum", "minimum" fields are used.
When using mchart with -D Argument, it is necessary to have the MCHART_DIR defined in order to allow the daemon to find the location of the configuration and data files (see Environment variables).
chaos:~/chart> more trigger.conf
#Equipment: >/equipment/kos_trigger/statistics
menu: on
slider: on
type: gtk
minor_ticks: 12
major_ticks: 6
chart-interval: 1.000
chart-filter: 0.500
slider-interval: 0.200
slider-filter: 0.200
begin: Events_sent
filename: /home/chaos/chart/trigger
fields: 2
pattern: Events_sent
equation: \$2
color: \$blue
maximum: 1083540.00
minimum: 270885.00
id_char: 1
end: Events_sent
begin: Events_per_sec.
filename: /home/chaos/chart/trigger
fields: 2
pattern: Events_per_sec.
equation: $2
color: \$red
maximum: 1305.56
minimum: 326.39
id_char: 1
end: Events_per_sec.
begin: kBytes_per_sec.
filename: /home/chaos/chart/trigger
fields: 2
pattern: kBytes_per_sec.
equation: $2
color: \$brown
maximum: 898.46
minimum: 224.61
id_char: 1
end: kBytes_per_sec.
A second file (data file) will be updated a fixed interval by the {mchart} utility.
chaos:~/chart> more trigger
Events_sent 6.620470e+05
Events_per_sec. 6.463608e+02
kBytes_per_sec. 4.424778e+02
- Creation with ODBpath being one array and one element of 2 sitting under variables/:
chaos:~/chart> mchart -f chvv -q /equipment/chv/variables/chvv -c
chaos:~/chart> ls -l chvv*
-rw-r--r-- 1 chaos users 474 Apr 18 14:37 chvv
-rw-r--r-- 1 chaos users 4656 Apr 18 14:37 chvv.conf
- Creation with ODBpath of all the sub-keys sittings in variables:
mchart -e myexpt -h myhost -f chv -q /equipment/chv/variables -c
- Creation and running in debug:
chaos:~/chart> mchart -f chv -q /equipment/chv/variables -d
CHVV : size:68
#name:17 #Values:17
CHVI : size:68
- Running a pre-existing conf file (chv.conf) debug:
chaos:~/chart> mchart -f chv.conf -d
CHVV : size:68
#name:17 #Values:17
CHVI : size:68
#name:17 #Values:17
- Running a pre-existing configuration file and spawning gstripchart:
chaos:~/chart> mchart -f chv.conf -gg
spawning graph with gstripchart -g 500x200-200-800 -f /home/chaos/chart/chv.conf ...
- Running a pre-existing configuration file and spawning stripchart, this will work only if Tcl/Tk and bltwish packages are installed and the stripchart.tcl has been installed through the Midas Makefile.
chaos:~/chart> mchart -f chv.conf -gh
spawning graph with stripchart /home/chaos/chart/chv.conf ...
Tape manipulation utility.
- Arguments
- [-h ] : help
- [-h hostname ] : host name
- [-e exptname ] : experiment name
- [-D ] : start program as a daemon
Direct I/O task provider (LINUX).
If no particular Linux driver is installed for the CAMAC access, the dio- program will allow you to access the I/O ports to which the CAMAC interface card is connected to.
- Arguments
- [application name ] : Program name requiring I/O permission.
- Usage
>dio miocnaf
>dio frontend
- Remark
- This "hacking" utility restricts the access to a range of I/O ports from 0x200 to 0x3FF.
- As this mode if I/O access by-passes the driver (if any), concurrent access to the same I/O port may produce unexpected result and in the worst case it will freeze the computer. It is therefore important to ensure to run one and only one dio application to a given port in order to prevent potential hangup problem.
- Interrupt handling, DMA capabilities of the interface will not be accessible under this mode of operation.
Graphical stripchart data display. Operates on mchart task data or on Midas history save-set files. (see also History system).
- Arguments
- [-mhist ] : start stripchart for Midas history data.
- Usage : stripchart <-options> <config-file> -mhist: (look at history file -default) -dmhist: debug mhist -debug: debug stripchart -config_file: see mchart_task
> stripchart.tcl -debug
> stripchart.tcl
- Example gstripchart display with parameters and data pop-up.
stripchart.tcl mhist mode: main window with pull-downs.
stripchart.tcl Online data, running in conjunction with mchart
Root/Midas remote GUI application for root histograms and possible run control under the ROOT. environment.
- Arguments
- [-h ] : help
- [-h hostname ] : host name
- [-e exptname ] : experiment name
- Usage to be written.
- Example
>rmidas midasserver.domain
rmidas display sample. Using the example/experiment/ demo setup.
High Voltage editor, graphical interface to the Slow Control System. Originally for Windows machines, but recently ported on Linux under Qt by Andreas Suter.
- Arguments
- [-h ] : help
- [-h hostname ] : host name
- [-e exptname ] : experiment name
- [-D ] : start program as a daemon
- Usage : To control the high voltage system, the program HVEdit can be used under Windows 95/NT. It can be used to set channels, save and load values from disk and print them. The program can be started several times even on different computers. Since they are all linked to the same ODB arrays, the demand and measured values are consistent among them at any time. HVEdit is started from the command line:
- Example
mserver provides remote access to any midas client. This task usually runs in the background and doesn't need to be modified. In the case where debugging is required, the mserver can be started with the -d flag which will write an entry for each transaction appearing onto the mserver. This log entry contains the time stamp and RPC call request.
- Arguments
- [-h ] : help
- [-s ] : Single process server
- [-t ] : Multi thread server
- [-m ] : Milti process server (default)
- [-d ] : Write debug info to /tmp/mserver.log
- [-D ] : Become a Daemon
Midas DOC Version 1.9.5 ---- PSI Stefan Ritt ----
Contributions: Pierre-Andre Amaudruz - Sergio Ballestrero - Suzannah Daviel -
Doxygen - Peter Green -
Qing Gu - Greg Hackman - Gertjan Hofman - Paul Knowles - Rudi Meier - Glenn
Moloney - Dave Morris - John M O'Donnell - Konstantin Olchanski - Renee Poutissou
- Tamsen Schurman - Andreas Suter - Jan M.Wouters - Piotr Adam Zolnierczuk