Uwar tools: Difference between revisions

From University of Washington - Ubicomp Research Page
Jump to navigationJump to search
mNo edit summary
mNo edit summary
Line 382: Line 382:




== uwar_clean ==  
== uwar-clean ==  
* <tt>msp/src/fileio/uwar-clean.c</tt> (not yet committed to CVS)
* <tt>msp/src/fileio/uwar-clean.c</tt> (not yet committed to CVS)
* [{{SERVER}}/files/uwar/tools_linx86/uwar_clean linux download (x86)]
* [{{SERVER}}/files/uwar/tools_linx86/uwar-clean linux download (x86)]
* [{{SERVER}}/files/uwar/tools_win32/uwar_clean.exe windows download]
* [{{SERVER}}/files/uwar/tools_win32/uwar_clean.exe windows download]



Revision as of 19:40, 23 October 2006

Overview

This document lists the various tools for working with uwar files and related data. An example .uwar file is hosted on this machine that you can use to test with. All outputs on this page are from this file: example.uwar.


central

central is the server for the IPC (Inter Process Communications) package developed by Reid Simmons. IPC provides a simple means of communicating between applications on the same computer and other machines via TCP/IP. Central is used by the msb-server and several other applications to share information. More information about IPC can be found at the projects homepage.

usage

    Typical Usage

      Spawn central and send it to the background
      central -u &
      

      Note, that only 1 copy of central can be run at one time. If you want to run multiple central's (see the -p argument below) or access central running on another machine you will need to export the CENTRALHOST variable: <bash># If you want to use another port export CENTRALHOST=localhost:1234

      1. If you want to connect to another machine

      export CENTRALHOST=othermachine </bash>

    Available Commands

      Various debugging/logging capabilities are also available if necessary. Remember to have a process cat the .log file that central creates.
      The following are central commands:
       help: print this message
       display: display the active and pending messages
       status: display the known modules and their status
       memory: display total memory usage
       close <module>: close a connection to a module
       unlock <resource>: unlock a locked resource
      
      The following command line options can also be used as commands:
      -v: display server version information
      -l<option>: logging onto terminal. Options are:
         m (message traffic)
         s (status of IPC)
         t (time messages are received by central)
         d (data associated with messages)
         i (ignore logging certain internal messages)
         h (handle time summary of incoming messages)
         r (log the reference ID as well as the message name)
         p (log the reference ID of the message’s parent)
         x (no logging)
      
      -l (no options) is equivalent to -lmstdh; the default is –lmsi
      
      -L<option>: logging into file.
        Options are the same as above, with the addition of
         F (don’t flush file after each line)
         n (don’t prompt user for comments)
        The default is –Lx
      
      -f<filename>: filename to use for logging; If not specified, name is automatically generated.
      
      -p<port>: connect to central server on this port number.
      -c: Use direct (not via central) connections when possible
      -I<msgName>: Ignore logging this message (can occur multiple times).
      -I<filename>: File with names of messages to ignore logging
      -s: silent running; don’t print anything to terminal.
      -u: don’t run the user (tty) interface.
      -r: try resending non-completed messages when modules crash and then reconnect.
      

    example output

      central typically does not output anything to the console unless an error occurs.


msb-server

The msb-server is a program which interfaces with the MSB via the serial port and handles logging of uwar files and transmission of the sensor data over IPC and other shared memory mechanisms. msb-server was written by Dirk Hähnel of Intel Research Seattle.

usage

    Typical Usage

      msb-server -ini msb.ini
      

      where msb.ini will typically contain:

      #Desktops should use the USB device
      #SERIAL_DEVICE 			/dev/ttyUSB1
      #the iMote2 uses ttyS0 only
      SERIAL_DEVICE 			/dev/ttyS0
      BAUD_RATE			      921600
      HARDWARE_FLOW_CONTROL		0
      SOFTWARE_FLOW_CONTROL		0
      USE_RTC                       1
      SCHEDULE_INTERRUPT_COUNT      5120
      SCHEDULE_MIC                 8192
      SCHEDULE_ACCELEROMETER        5120
      SCHEDULE_AMB_LIGHT            2560
      SCHEDULE_BAROMETER              71
      SCHEDULE_DALS                   30
      SCHEDULE_HUMIDITY               10
      SCHEDULE_POWER                   0
      MIC_GAIN                        64
      
      LOG_FILE			 test.uwar
      LOG_START			 1
      

    Available Commands

      In addition the msb-server also supports the following switch commands:
      -ipc-publish-data    publish the converted data via IPC
      -ipc-publish-raw     publish the raw sensor data via IPC
      -ipc-publish-stream  stream serial data via ipc
      -shm-publish-stream  stream serial data via shared memory
      -ipc-publish-mic     publish microphone packets
      -sync-rtc
      -show-rtc
      -set-rtc
      -use-rtc
      -log        specify a file to log data to, see -max-size and -max-time for log rotation settings
      -log-type   specify the type of data to log:
        uwar      serial data is stored as packets in a uwar file
        stream    serial data is written directly to a file
      -max-size   maximum size of the uwar file before rotating the log file (in kilobytes)
      -max-time   maximum time between file rotations
      

example output

    Typically the msb-server will output something similar to the following. Where the final recieved stats will continue until the program is terminated or an error occurs.
    ./msb-server -ini ./msb.ini -ipc-publish-data
    # INFO: ********************************
    # INFO: ***********   MSB   ************
    # INFO: ********************************
    # INFO: read ini file ./msb.ini
    # INFO: initialize data structures
    # INFO: allocate data messages
    # INFO: IPC for Mobile Sensing Platform Toolkit (Ver. 1.0.0)
    # INFO: set device:
    # INFO:    port   = /dev/ttyUSB1
    # INFO:    baud   = 921600
    # INFO:    params = 8N1
    # INFO: send reset command
    # INFO: ---------------------
    # INFO: send cease command ... done
    # INFO: send rtc query command to MSB
    # INFO: current msb time = 2065-25-45 45:85:85!
    # INFO: change system time to RTC!
    # INFO: set system time failed!
    # INFO: send calibration query command to MSB
    # INFO: ---- calibration ----
    # INFO: board nr            = b
    # INFO: hardware version nr = 2
    # INFO: major soft version  = 1
    # INFO: minor soft version  = 1
    # INFO: calibration word 1  = 48224
    # INFO: calibration word 2  = 47962
    # INFO: calibration word 3  = 28705
    # INFO: calibration word 4  = 44956
    # INFO: calibration val c1  = 24112
    # INFO: calibration val c2  = 2140
    # INFO: calibration val c3  = 702
    # INFO: calibration val c4  = 448
    # INFO: calibration val c5  = 749
    # INFO: calibration val c6  = 26
    # INFO: send schedule query command to MSB
    # INFO: --- USER REQUEST  ---
    # INFO: ------ schedule -----
    # INFO: interrupt count     = 5120
    # INFO: mic hz              = 8192
    # INFO: accelerometer hz    = 512.0
    # INFO: amb light hz        = 256.0
    # INFO: barometer hz        = 7.1
    # INFO: compass hz          = 0.0
    # INFO: dals hz             = 3.0
    # INFO: humidity hz         = 1.0
    # INFO: power               = 0
    # INFO: microphone gain     = 255
    # INFO: send schedule query command to MSB
    # INFO: --- NEW SETTINGS  ---
    # INFO: ------ schedule -----
    # INFO: interrupt count     = 5120
    # INFO: mic hz              = 8192
    # INFO: accelerometer hz    = 512.0
    # INFO: amb light hz        = 256.0
    # INFO: barometer hz        = 7.1
    # INFO: compass hz          = 0.0
    # INFO: dals hz             = 3.0
    # INFO: humidity hz         = 1.0
    # INFO: power               = 0
    # INFO: microphone gain     = 255
    # INFO: ------ schedule -----
    # INFO: interrupt count     = 5120
    # INFO: mic hz              = 8192
    # INFO: accelerometer hz    = 512.0
    # INFO: amb light hz        = 256.0
    # INFO: barometer hz        = 7.1
    # INFO: compass hz          = 0.0
    # INFO: dals hz             = 3.0
    # INFO: humidity hz         = 1.0
    # INFO: power               = 0
    # INFO: microphone gain     = 255
    # INFO: -------   log   -------
    # INFO: open log-file file test-0002.uwar [test-0002.uwar]
    # INFO: write header to log file ... => 967903080
    done
    # INFO: log-max-size: 4 Mbytes
    # INFO: log-max-time: 5.00 minutes 
    # INFO: ------- start -------
    # INFO: send init command to MSB
    # INFO:  2.40 kbits/sec (0 crc of 9 packets)
    # INFO: 234.95 kbits/sec (0 crc of 169 packets)
    # INFO: 221.89 kbits/sec (0 crc of 321 packets)
    # INFO: unknown packet type 0xb4
    # INFO: 221.89 kbits/sec (0 crc of 473 packets)
    ...
    


uwar_cat

uwar_cat is a program to extract data from uwar files and dump them in a human readable format. The output format is something of the form:

SENSORA      TIMESTAMP DATA ...
SENSORB      TIMESTAMP DATA ...
SENSORC      TIMESTAMP DATA ...

If you want to retrieve data from a specific sensor you can use the grep command.
Windows users: You can download a win32 version of grep from the gnuwin32 webpage. You will also need to grab some additional libraries or you can try this zipped version which should contain all the necessary libraries. Simply extract the zip to your Windows\system32 directory.

usage

    uwar_cat <filename>
    

example output

    uwar_cat example.uwar
    # INFO: open file example.uwar
    # INFO: read UWAR v2 header
    # INFO: read UWAR v2 data
    ACCEL        1160595736.915 -0.028381 0.013855 0.980713
    ACCEL        1160595736.915 -0.027588 0.013733 0.981079
    AMB-LIGHT    1160595736.915 16 9.602142
    DALS         1160595736.915 671.500000 73.500000 219.288132
    ACCEL        1160595736.915 -0.027222 0.014099 0.980042
    ACCEL        1160595736.915 -0.027039 0.013428 0.981628
    AMB-LIGHT    1160595736.915 16 9.661065
    ACCEL        1160595736.915 -0.027893 0.014221 0.981079
    ACCEL        1160595736.915 -0.027100 0.013916 0.981812
    ...
    


uwar-info

uwar_info will display the meta data stored in the uwar file. This information can be useful in determining what streams are inside a uwar file and what settings might be in use, for example you can retrieve the schedule and the calibration data from the sensor boards with this command.

usage

    uwar_info <filename>
    

example output

    uwar-info example.uwar 
    # INFO: open file example.uwar
    # INFO: read UWAR v2 header
    # INFO: all meta  0: id=(726e6462:bdnr) type='i' = 11
    # INFO: all meta  1: id=(72657668:hver) type='i' = 2
    # INFO: all meta  2: id=(616d7773:swma) type='i' = 1
    # INFO: all meta  3: id=(696d7773:swmi) type='i' = 1
    # INFO: all meta  4: id=(31306477:wd01) type='i' = 48224
    # INFO: all meta  5: id=(32306477:wd02) type='i' = 47962
    # INFO: all meta  6: id=(33306477:wd03) type='i' = 28705
    # INFO: all meta  7: id=(34306477:wd04) type='i' = 44956
    # INFO: all meta  8: id=(72746e69:intr) type='i' = 5120
    # INFO: all meta  9: id=(2063696d:mic ) type='i' = 8192
    # INFO: all meta 10: id=(20636361:acc ) type='i' = 5120
    # INFO: all meta 11: id=(20626d61:amb ) type='i' = 2560
    # INFO: all meta 12: id=(20726162:bar ) type='i' = 71
    # INFO: all meta 13: id=(706d6f63:comp) type='i' = 0
    # INFO: all meta 14: id=(736c6164:dals) type='i' = 30
    # INFO: all meta 15: id=(206d7568:hum ) type='i' = 10
    # INFO: all meta 16: id=(72776f70:powr) type='i' = 0
    # INFO: stream 0: id=0 (2063696d:mic ) -> ref(0)
    # INFO: stream 1: id=1 (3262736d:msb2) -> ref(1)
    # INFO: stream 2: id=2 (20676174:tag ) -> ref(2)
    # INFO: stream 3: id=3 (6b636c63:clck) -> ref(3)
    # INFO: stream 4: id=4 (20737067:gps ) -> ref(4)
    # INFO: read UWAR data
    # INFO: ---- calibration ----
    # INFO: board nr            = b (11)
    # INFO: hardware version nr = 2
    # INFO: major soft version  = 1
    # INFO: minor soft version  = 1
    # INFO: calibration word 1  = 48224
    # INFO: calibration word 2  = 47962
    # INFO: calibration word 3  = 28705
    # INFO: calibration word 4  = 44956
    # INFO: calibration val c1  = 24112
    # INFO: calibration val c2  = 2140
    # INFO: calibration val c3  = 702
    # INFO: calibration val c4  = 448
    # INFO: calibration val c5  = 749
    # INFO: calibration val c6  = 26
    # INFO: ------ schedule -----
    # INFO: interrupt count     = 5120
    # INFO: mic hz              = 8192
    # INFO: accelerometer hz    = 512.0
    # INFO: amb light hz        = 256.0
    # INFO: barometer hz        = 7.1
    # INFO: compass hz          = 0.0
    # INFO: dals hz             = 3.0
    # INFO: humidity hz         = 1.0
    # INFO: power               = 0
    # INFO: read UWAR v2 data
    # INFO: read 53899 packets           
    
    # INFO: time from:   11.09.2006 12:42:16.914999
    # INFO: time to:     11.09.2006 12:43:09.532000
    
    INFO: statistics packets
           26953   msb2 (missed 0 packets (0.0000%))
           26946   mic  (missed 6 packets (0.0223%))
           53899   total
    


uwar_msb_dump

uwar_msb_dump is a program to dump the raw serial packets stored inside the uwar file to a single binary output file. This is useful for debugging problems with the MSB itself and analyzing the packets received by the msb-server.

usage

    uwar_msb_dump <input_uwar_filename> <output_binary_file>
    

example output

    uwar_msb_dump example.uwar example.bin
    # INFO: open file example.uwar
    # INFO: Output file example.bin
    # INFO: read UWAR v2 header
    # INFO: read UWAR v2 data
    


uwar-clean

uwar_clean removes corrupted packets from a .uwar file and stores the cleaned stream in a new file. Note, this program is based off of uwar-info so any file uwar-info can process it should be able to handle. It does NOT clean bad data inside a valid uwar packet.

usage

    uwar_clean <input_uwar_filename> <output_cleaned_uwar_filename>
    

    Note: Be careful not to specify the same input/output name as this will immediately erase your file. Remember if you say something like 'uwar_clean myVeryImportantData.uwar ./myVeryImportantData.uwar' you will likely lose your data!

example output

    uwar_clean.exe log-1969-11-31-16.00.dat log-1969-11-31-16.00_clean.dat
    
    # INFO: open file log-1969-11-31-16.00.dat
    # INFO: read UWAR v2 header
    # INFO: all meta  0: id=(726e6462:bdnr) type='i' = 61463
    # INFO: all meta  1: id=(72657668:hver) type='i' = 2
    # INFO: all meta  2: id=(616d7773:swma) type='i' = 1
    # INFO: all meta  3: id=(696d7773:swmi) type='i' = 1
    # INFO: all meta  4: id=(31306477:wd01) type='i' = 39064
    # INFO: all meta  5: id=(32306477:wd02) type='i' = 31260
    # INFO: all meta  6: id=(33306477:wd03) type='i' = 36959
    # INFO: all meta  7: id=(34306477:wd04) type='i' = 42677
    # INFO: all meta  8: id=(72746e69:intr) type='i' = 5120
    # INFO: all meta  9: id=(2063696d:mic ) type='i' = 16384
    # INFO: all meta 10: id=(20636361:acc ) type='i' = 5120
    # INFO: all meta 11: id=(20626d61:amb ) type='i' = 2560
    # INFO: all meta 12: id=(20726162:bar ) type='i' = 71
    # INFO: all meta 13: id=(706d6f63:comp) type='i' = 0
    # INFO: all meta 14: id=(736c6164:dals) type='i' = 30
    # INFO: all meta 15: id=(206d7568:hum ) type='i' = 10
    # INFO: all meta 16: id=(72776f70:powr) type='i' = 0
    # INFO: stream 0: id=0 (2063696d:mic ) -> ref(0)
    # INFO: stream 1: id=1 (3262736d:msb2) -> ref(1)
    # INFO: stream 2: id=2 (20676174:tag ) -> ref(2)
    # INFO: stream 3: id=3 (6b636c63:clck) -> ref(3)
    # INFO: stream 4: id=4 (20737067:gps ) -> ref(4)
    # INFO: read UWAR data
    # INFO: ---- calibration ----
    # INFO: board nr            = f017 (61463)
    # INFO: hardware version nr = 2
    # INFO: major soft version  = 1
    # INFO: minor soft version  = 1
    # INFO: calibration word 1  = 39064
    # INFO: calibration word 2  = 31260
    # INFO: calibration word 3  = 36959
    # INFO: calibration word 4  = 42677
    # INFO: calibration val c1  = 19532
    # INFO: calibration val c2  = 2037
    # INFO: calibration val c3  = 666
    # INFO: calibration val c4  = 577
    # INFO: calibration val c5  = 488
    # INFO: calibration val c6  = 28
    # INFO: ------ schedule -----
    # INFO: interrupt count     = 5120
    # INFO: mic hz              = 16384
    # INFO: accelerometer hz    = 512.0
    # INFO: amb light hz        = 256.0
    # INFO: barometer hz        = 7.1
    # INFO: compass hz          = 0.0
    # INFO: dals hz             = 3.0
    # INFO: humidity hz         = 1.0
    # INFO: power               = 0
    # INFO: read UWAR v2 data
    # INFO: read 463037 packets
    
    # INFO: time from:   31.11.1969 16:00:28.949999
    # INFO: time to:     31.11.1969 16:08:01.166999
    
    INFO: statistics packets
          231555   msb2 (missed 5 packets (0.0022%))
          231478   mic  (missed 81 packets (0.0350%))
               4   corrupt
          463037   total
    

    The cleaned file should now show no corrupt packets:

    uwar_info.exe log-1969-11-31-16.00_clean.dat
    
    # INFO: open file log-1969-11-31-16.00_clean.dat
    # INFO: read UWAR v2 header
    # INFO: all meta  0: id=(726e6462:bdnr) type='i' = 61463
    # INFO: all meta  1: id=(72657668:hver) type='i' = 2
    # INFO: all meta  2: id=(616d7773:swma) type='i' = 1
    # INFO: all meta  3: id=(696d7773:swmi) type='i' = 1
    # INFO: all meta  4: id=(31306477:wd01) type='i' = 39064
    # INFO: all meta  5: id=(32306477:wd02) type='i' = 31260
    # INFO: all meta  6: id=(33306477:wd03) type='i' = 36959
    # INFO: all meta  7: id=(34306477:wd04) type='i' = 42677
    # INFO: all meta  8: id=(72746e69:intr) type='i' = 5120
    # INFO: all meta  9: id=(2063696d:mic ) type='i' = 16384
    # INFO: all meta 10: id=(20636361:acc ) type='i' = 5120
    # INFO: all meta 11: id=(20626d61:amb ) type='i' = 2560
    # INFO: all meta 12: id=(20726162:bar ) type='i' = 71
    # INFO: all meta 13: id=(706d6f63:comp) type='i' = 0
    # INFO: all meta 14: id=(736c6164:dals) type='i' = 30
    # INFO: all meta 15: id=(206d7568:hum ) type='i' = 10
    # INFO: all meta 16: id=(72776f70:powr) type='i' = 0
    # INFO: stream 0: id=0 (2063696d:mic ) -> ref(0)
    # INFO: stream 1: id=1 (3262736d:msb2) -> ref(1)
    # INFO: stream 2: id=2 (20676174:tag ) -> ref(2)
    # INFO: stream 3: id=3 (6b636c63:clck) -> ref(3)
    # INFO: stream 4: id=4 (20737067:gps ) -> ref(4)
    # INFO: read UWAR data
    # INFO: ---- calibration ----
    # INFO: board nr            = f017 (61463)
    # INFO: hardware version nr = 2
    # INFO: major soft version  = 1
    # INFO: minor soft version  = 1
    # INFO: calibration word 1  = 39064
    # INFO: calibration word 2  = 31260
    # INFO: calibration word 3  = 36959
    # INFO: calibration word 4  = 42677
    # INFO: calibration val c1  = 19532
    # INFO: calibration val c2  = 2037
    # INFO: calibration val c3  = 666
    # INFO: calibration val c4  = 577
    # INFO: calibration val c5  = 488
    # INFO: calibration val c6  = 28
    # INFO: ------ schedule -----
    # INFO: interrupt count     = 5120
    # INFO: mic hz              = 16384
    # INFO: accelerometer hz    = 512.0
    # INFO: amb light hz        = 256.0
    # INFO: barometer hz        = 7.1
    # INFO: compass hz          = 0.0
    # INFO: dals hz             = 3.0
    # INFO: humidity hz         = 1.0
    # INFO: power               = 0
    # INFO: read UWAR v2 data
    # INFO: read 463033 packets
    
    # INFO: time from:   31.11.1969 16:00:28.949999
    # INFO: time to:     31.11.1969 16:08:01.167000
    
    INFO: statistics packets
          231555   msb2 (missed 5 packets (0.0022%))
          231478   mic  (missed 81 packets (0.0350%))
          463033   total