VM Data Analysis

Getting started with data analysis

Configuring VirtuaBox

  1. Download the RAY Analysis virtual machine from this link.
  2. Download and install VirtualBox from http://virtualbox.org
  3. Import the virtual machine into VirtualBox (File > Import Appliance; check the box labelled "Initialise MAC addresses") and start it up (select the VM and click "Start"). Your machine will need at least 3 GB of RAM to run the VM smoothly.

Configuring the VM

  1. Click the "Terminal" icon on the desktop to open a shell. The default shell is bash, which is required to run root smoothly; type info bash (and hit enter) to learn more. An alternative shell is fish, which is more user friendly, but has some problems running root; type fish to learn more
  2. You will need to change the password for the ray user and for the superuser. The default password is alpine, and can be changed with the passwd command. You will need to run this as both the normal user and the superuser.1
sudo passwd

Managing files

  1. You will need data files for your analysis. The VM contains example data in the data/testdata folder, or you can download data from ArcGIS. For now, extract these files with the GUI.
  2. In the terminal, type man2 cd, then press CTRL-Z to exit the manual.
  3. Use cd3 to open the folder data on the desktop. Use ls4 to determine the path.
  4. mkdir5 creates new directories, or folders. Each set of data you analyse should be contained in a new sub-folder of ~/Desktop/data.
  5. Your data should have been extracted to ~/Downloads. Use ls and mv6 to move your files to your new folder in ~/Desktop/data. TIP: the asterisk, *, acts as a wildcard, so would match all the files in a directory.
mv ~/Downloads/!downloaded_data!/* ~/Desktop/data/!session_folder!

Setting analysis variables

  1. Open the folder containing your data and use pwd7 to discover its path in the system. Copy this path to the clipboard (keyboard shortcuts or navigation are unsupported).
  2. export creates and defines variables. Rather than repeatedly typing paths to files, we can store the information in a variable for later use. Variables can be defined as plaintext (export var=hello) or as the output of a function (export var=`pwd` - export evaluates the function inside the backticks before defining var).
  3. Set FILEPATH as the current working directory using export and pwd
  4. Change the current directory to the Desktop
  5. List the files on the desktop and use cat8 to read any files of interest, such as start.sh.
  6. In bash, you can run scripts within the current shell, and return their output to the shell. Type . start.sh9 to begin processing the data. Right click and paste the path you copied earlier, and press enter. A file ID should appear, containing the date when the data was captured.
  7. Change the directory to ~/software/data-conversion-toolkit/toolkit-build. List the files, and run Px2Mf-converter in a subshell: ./Px2Mf-converter $FILEPATH $RUNID $FILEPATH.
  8. Go back to your data folder. List the folder contents, then pipe10 the output into grep11 and search for files with the extension .root: ls | grep .root.
  9. To store this file as a variable, you need to store both the name (generated above) and the path. To do this, recall the FILEPATH variable ($FILEPATH) and follow it with the name of the file: export DATAFILE=$FILEPATH/`ls | grep .root`.

Viewing, Filtering and Analysing Frames

  1. We can use the shell to run python scripts as well, with the command python. Open the data-conversion-toolkit directory, and find the file called frame-viewer.py. Run python ~/software/data-conversion-toolkit/toolkit-python/frame-viewer.py $DATAFILE12 to open the viewer.
  2. You can create a pixel mask, so that any stuck or noisy pixels don’t affect your analysis. Open and list the toolkit-python directory, and try to run make-mask.py. The exact command is as follows: python make-mask.py $DATAFILE filter.txt $FILEPATH filter n13
  3. You can apply the filter from the directory toolkit-build. Type ./Mf-filter $DATAFILE $FILEPATH/filter.txt filter, then use the frame-viewer again to see the difference.
  4. You are now ready to analyse the data! Run rate-plotter.py with $DATAFILE and $FILEPATH as arguments14, then look for html files in the data directory. firefox can open these files when it is run from the same directory as them.
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License