Volume Tests Sample Clauses

Volume Tests. In these tests we attempt to investigate fie size limitations. In many communities, individual files are becoming larger and larger. Already we are seeing individual sizes of the order of a terabyte, and even the norm is approaching tens of gigabytes for these communities. If a storage system or federation layer is unable to archive these larger file sizes then this has an implication for communities in terms of how they will need to store data within EUDAT. Due to the nature of the clients involved it was difficult to exceed 100GB as an input file for most sites. However, at several sites it is known that files of 1Tb or more are already archived. Within the current test infrastructures it is not possible to replicate this file size. However, there are technologies that interface to mass storage systems which do limit the filesize, such as ‘Safety Deposit Box’ which reads the whole file into memory before sending it to storage and hence is limited by the amount of memory available on the host. We have looked at the transfer time to migrate these files in addition to see what impact file size had on transfer rate. These are shown for ‘native’ protocol Figures 16 and 17, and when using iRODS in Figures 18 and 19. 1 2 5 10 20 50 100 STFC PNSC GPFS) PNSC (NFS) DKRZ UiO RZG STFC PNSC(GPFS) PNSC(NFS) DKRZ UiO RZG From the tests with the native protocol it is clear that transfer rates do not vary significantly with file size, and also the setup at RZG (XFS) and PNSC (with GPFS mounting) performs significantly better than the other storage systems. This may be gown to different configurations at each site, such as some sites using 10GB NIC cards while others are still using 1GB. However, for most sites it is clear that the transfer rate is of the order 80 MB/sec using ‘native’ protocol. 1 2 5 10 20 50 100 STFC PNSC (GPFS) PNSC (NFS) DKRZ UiO RZG 45 40 35 30 25 20 15 10 5