Skip to content

Storing and managing large files on EOS

Learning Objectives

  • Open and view files on EOS
  • Understand the basics of XRootD

EOS is a large area of disk space provided by CERN for users. See here for more information.

On lxplus EOS is mounted with a FUSE mount under /eos . If you list the contents of the /eos directory you should see various experiments areas, and the user area:

lhcb user

There are two useful workspaces on EOS: - /eos/user which is your private EOS space, also accessible via CERNBox, and has 1TB of space; - /eos/lhcb which is the LHCb area and contains the following sub-directories: - /eos/lhcb/user which is your user area, where you can store your tuples, and has 2TB of space; - /eos/lhcb/wg which contains the Working Group areas (e.g. for preserving tuples for completed analyses); - /eos/lhcb/grid/ which is the path to the CERN grid site; and many others.

Let's inspect your user area:

$ ls -al /eos/lhcb/user/a/another

On lxplus, files stored on EOS can be open in ROOT the usual way:

$ root /eos/lhcb/user/a/another/myfavouritefile.root

XRootD

An advantage of EOS besides is huge capacity is its accessibility with XRootD protocols. In general the fuse mount of EOS is not very reliable and it is not recommended to use it for moving or accessing large numbers of files. Instead it is preferred to use XRootD or EOS commands. XRootD allows browsing as well as reading and writing to /eos as well as several other user spaces.

In order to authenticate, it is first necessary to generate a kerberos token, this is done by running:

kinit -f <username>@CERN.CH # It is necessary for CERN.CH to be in all-caps

Let's look at an example of an address of a file stored on /eos. This file was used in the lesson Interactively exploring a DST

root://eoslhcb.cern.ch//eos/lhcb/wg/dpa/wp7/Run3SK/exampleDST/b2oc_exclusive_s24c4.dst.

The double slashes are necessary here. This address may be broken into two parts:

The host server : root://eoslhcb.cern.ch/
The file path   : /eos/lhcb/wg/dpa/wp7/Run3SK/exampleDST/b2oc_exclusive_s24c4.dst

Root files may be loaded into root using the full address e.g.

$ root root://eoslhcb.cern.ch//eos/lhcb/user/a/another/myfavouritefile.root

File management may then be done using XRootD commands such as xrdcp and xrdfs. The syntax for these is.

xrdcp <source> <dest> where either source, dest or both may be XRootD addresses or local addresses will copy a file or files.

xrdfs <host> <command> <file> will perform file system commands (e.g. ls, stat, rm, ...) on a remote file.

Statting some files

Try using xrdfs to stat the dst file above!