Skip to main content

Using File Storage - High Performance on Baremetal GPU Server (Japan Region)

Overview​

The File Storage - High Performance service is now available for Baremetal infrastructure in the Japan region. To request access, please contact sales or technical support and provide the following information:

  • Required capacity (quota)
  • Subnet to connect

Usage Guide​

A. Using images from FPT AI FACTORY​

If you are using images provided by FPT AI FACTORY, the DDN client is already pre-installed. Follow the steps below:

Step 1: Create the mount directory

mkdir /mnt/hps

Step 2: Perform manual mount command

mount -t lustre 100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps

Step 3: Configure automatic mount on reboot Edit the /etc/fstab file:

## LustreFS ##
100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps lustre flock,defaults 0 0

Explanation of each part

  • "100.69.252.1@o2ib,100.69.252.2@o2ib:...:100.69.252.8@o2ib:/scratch**"**
    • 100.69.252.1, 100.69.252.2 ... 100.69.252.8 are fixed endpoint addresses.
    • /scratch is the filesystem path on the Lustre server.
  • /mnt/hps
    • Mount Point on the client (created in Step 1).

Note: Run the following command to verify the mount configuration has no errors.

mount -a

Step 4: Verify

df -h -t lustre

Expected output:

Filesystem                    Size  Used Avail Use% Mounted on
...:/scratch                 11T   8.0K   11T   1%   /mnt/hps

B. Using a custom Ubuntu image​

If you are using a custom image, you need to build the packages yourself by following these steps:

Step 1: Build the Lustre Client You can skip this if you have already built it for the current kernel.

wget https://s3-sgn10.fptcloud.com/file-storage-resource/exa-client-6.3.2.tar.gz
tar xzvf exa-client-6.3.2.tar.gz
cd exa-client
./exa_client_deploy.py

If your Ubuntu OS is using kernel 5.15.0-130-generic, you can skip the build step and download the pre-built packages below:

wget https://s3-sgn10.fptcloud.com/package-pub/lustre-dev_2.14.0-ddn184-1_amd64.deb
wget https://s3-sgn10.fptcloud.com/package-pub/lustre-client-modules-5.15.0-130-generic_2.14.0-ddn184-1_amd64.deb
wget https://s3-sgn10.fptcloud.com/package-pub/lustre-client-utils_2.14.0-ddn184-1_amd64.deb

Step 2: Install the Lustre Client Install the Lustre Client using the .deb packages built in Step B1 or the pre-built packages.

  • This step should only be performed if a compatible .deb package is available.
  • If no package is available, go back to

Step 1 to build the package before installing.

  • If the kernel does not match the .deb package, errors may occur and you will need to rebuild for the current kernel.
sudo dpkg -i lustre-dev_2.14.0-ddn184-1_amd64.deb \
             lustre-client-modules-5.15.0-130-generic_2.14.0-ddn184-1_amd64.deb \
             lustre-client-utils_2.14.0-ddn184-1_amd64.deb

Step 3: Optimize the configuration Edit /etc/modprobe.d/lustre.conf:

[/etc/modprobe.d/lustre.conf]
options lnet networks=o2ib(bond0)    <==HERE!
options ko2iblnd peer_credits=32 peer_credits_hiw=16 concurrent_sends=64

Edit /etc/lustre_client:

[/etc/lustre_client]

#++++++++++++++++++++++++++++++++++++
IF1=bond0        <==HERE!
IF2=none
IF3=none
IF4=none
IF5=none
IF6=none
IF7=none
IF8=none

Step 4: Create the mountpoint

mkdir /mnt/hps

Step 5: Perform manual mount command

mount -t lustre 100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps

Step 6: Configure automatic mount Edit the /etc/fstab file:

##LustreFS##

100.69.252.1@o2ib,100.69.252.2@o2ib:100.69.252.3@o2ib,100.69.252.4@o2ib:100.69.252.5@o2ib,100.69.252.6@o2ib:100.69.252.7@o2ib,100.69.252.8@o2ib:/scratch /mnt/hps lustre flock,defaults 0 0

Explanation of each part

  • "100.69.252.1@o2ib,100.69.252.2@o2ib:...:100.69.252.8@o2ib:/scratch**"**
    • 100.69.252.1, 100.69.252.2 ... 100.69.252.8 are fixed endpoint addresses.
    • /scratch is the filesystem path on the Lustre server.
  • /mnt/hps
    • Mount Point on the client (created in Step 1).

Note: Run the following command to verify the mount configuration has no errors.

mount -a

Step 4: Verify

df -h -t lustre

C. VM GPU Customer - Ubuntu OS​

If you are using a custom image running Ubuntu, you need to build the packages yourself by following these steps:

1. Build Packages This step builds the Lustre Client package from source in cases where no pre-built package is compatible with the current system kernel.

wget https://s3-sgn10.fptcloud.com/file-storage-resource/exa-client-6.3.2.tar.gz
tar xzvf exa-client-6.3.2.tar.gz
cd exa-client
./exa_client_deploy.py

2. Set up the Lustre Client Add configuration to optimize performance:

[/etc/modprobe.d/lustre.conf]
options lnet networks=tcp(eth1)    <==HERE!
options ko2iblnd peer_credits=32 peer_credits_hiw=16 concurrent_sends=64

[/etc/lustre_client]

#++++++++++++++++++++++++++++++++++++
IF1=eth1        <==HERE!
IF2=none
IF3=none
IF4=none
IF5=none
IF6=none
IF7=none
IF8=none

3. Create the mountpoint on the OS

mkdir /mnt/hps

4. Run the mount command manually.

mount -t lustre 100.69.252.1@tcp,100.69.252.2@tcp:100.69.252.3@tcp,100.69.252.4@tcp:100.69.252.5@tcp,100.69.252.6@tcp:100.69.252.7@tcp,100.69.252.8@tcp:/scratch /mnt/hps

5. Add the Lustre mount entry to /etc/fstab Configure the system to automatically mount the Lustre filesystem by adding an entry to /etc/fstab.

##LustreFS##

100.69.252.1@tcp,100.69.252.2@tcp:100.69.252.3@tcp,100.69.252.4@tcp:100.69.252.5@tcp,100.69.252.6@tcp:100.69.252.7@tcp,100.69.252.8@tcp:/scratch /mnt/hps lustre flock,defaults 0 0

Explanation:

  • "100.69.252.1@tcp,100.69.252.2@tcp:...:100.69.252.8@tcp:/scratch"
    • 100.69.252.1, 100.69.252.2 ... 100.69.252.8 are the fixed DDN endpoints for all customers.
    • /scratch is the filesystem path on the Lustre server.
  • /mnt/hps
    • Mount Point on the client (created in Step 4).

Note: Run the following command to verify the mount configuration has no errors.

mount -a

6. Confirm

df -h -t lustre

Filesystem                                                                       Size  Used Avail Use% Mounted on
100.69.252.1@tcp,100.69.252.2@tcp:100.69.252.3@tcp,100.69.252.4@tcp:100.69.252.5@tcp,100.69.252.6@tcp:100.69.252.7@tcp,100.69.252.8@tcp:/scratch   11T  8.0K   11T   1% /mnt/hps

Last updated on by Dang Tran