Welcome to the forums. Please post in English or French.

You are not logged in.

#1 2018-09-14 15:07:09

will.logie
Member
From: Canberra
Registered: 2017-09-23
Posts: 10

Parallel install of CA 13.4.0 on Ubuntu 16.04 (Linux Mint 18.3)

Generally speaking, I like using prerequisites for my distribution where possible, hence I have used Ubuntu versions of OpenBLAS (0.2.18) and ScaLAPACK (1.8.0).

The procedure I have followed starts with the installation of these and other system packages:

sudo apt-get install gcc g++ gfortran cmake tk bison flex zlib1g-dev \
  python python-dev python-numpy python-qt4 libboost-python-dev \
  liblapack-dev libblas-dev libopenblas-dev libatlas-base-dev  \
  openmpi-bin libopenmpi-dev libscalapack-mpi-dev libblacs-mpi-dev

PART 1:
The question I would firstly like to ask is regarding the compilation of MUMPS 5.1.1. I cannot perform a waf configuration without getting a negative response to "Checking for program using blacs". For some reason, BLACS_PINFO is not compiling/working in that test.

In /opt/aster/public/mumps-5.1.1_mpi with the following environment variables:

export INCLUDES="/opt/aster/public/scotch-6.0.4/include /opt/aster/public/metis-5.1.0/include"
export LIBPATH="/opt/aster/public/scotch-6.0.4/lib /opt/aster/public/metis-5.1.0/lib"

the waf configure/install seems to succeed (no grievous errors) with the command:

./waf configure install --prefix=/opt/aster/public/mumps-5.1.1_mpi --enable-mpi

however I can't help feeling pensive. I have attached a log of terminal from this (waf_mumps.log).

PART 2:
I tried compiling CA in parallel although I was getting the MUMPS waf compile error above - this I managed by commenting the system exit call on line #315 of waftools/mathematics.py in the aster-13.4.0 source directory (while cringing). The waf configuration files are:

waf_seq.py

# coding=utf-8

"""
Configuration for waf using aster-full-13.4.0 prerequisites and:
-- Linux       : Ubuntu 16.04 (Mint 18.3)
-- GNU         : system 5.4.0
-- BLAS        : system 0.2.18 (libopenblas-dev)
"""

import os
import os.path as osp
import sys

def configure(self):
    opts = self.options

    self.env.append_value('LIBPATH', [
        '/opt/aster/public/hdf5-1.8.14/lib',
        '/opt/aster/public/med-3.2.1/lib',
        '/opt/aster/public/mumps-5.1.1/lib',
        '/opt/aster/public/metis-5.1.0/lib',
        '/opt/aster/public/mfront-3.0.0/lib',
        '/opt/aster/public/scotch-6.0.4/lib',
    ])

    self.env.append_value('INCLUDES', [
        '/opt/aster/public/hdf5-1.8.14/include',
        '/opt/aster/public/med-3.2.1/include',
        '/opt/aster/public/mumps-5.1.1/include',
        '/opt/aster/public/mumps-5.1.1/include_seq',
        '/opt/aster/public/metis-5.1.0/include',
        '/opt/aster/public/scotch-6.0.4/include',
        '/opt/aster/public/mfront-3.0.0/include',
    ])

    self.env.append_unique('OPTLIB_FLAGS_HDF5', ['-Wl,--allow-multiple-definition','-ldl'])

    # mfront path
    self.env.TFELHOME = '/opt/aster/public/mfront-3.0.0'

    opts.maths_libs = 'openblas'
    opts.embed_math = True

    opts.enable_med = True
    opts.hdf5_libs  = 'hdf5 hdf5_hl z'
    opts.embed_hdf5 = True
    opts.med_libs  = 'med medC medimport stdc++'
    opts.embed_med  = True

    opts.enable_mumps  = True
    opts.mumps_version = '5.1.1'
    opts.mumps_libs = 'dmumps zmumps smumps cmumps mumps_common pord metis'
    opts.embed_mumps   = True

    opts.enable_petsc = False
    
    opts.enable_mfront = True

    opts.enable_scotch = True
    opts.embed_scotch  = True

    opts.embed_aster    = True
    opts.embed_fermetur = True
    
    # add paths for external programs
    os.environ['METISDIR'] = '/opt/aster/public/metis-5.1.0'
    os.environ['GMSH_BIN_DIR'] = '/opt/aster/public/gmsh-3.0.6-Linux64/bin'
    os.environ['GIBI_DIR'] = '/opt/aster/public/gibi-2000'
    os.environ['HOMARD_ASTER_ROOT_DIR'] = '/opt/aster/public/homard-11.7'

waf_mpi.py

# encoding: utf-8

"""
Configuration for waf using aster-full-13.4.0 prerequisites and:
-- Linux       : Ubuntu 16.04 (Mint 18.3)
-- GNU         : system 5.4.0
-- BLAS        : system 0.2.18 (libopenblas-dev)
-- MPI         : system 1.10.2 (openmpi-bin libopenmpi-dev)
-- ScaLAPACK   : system (liblapack-dev libscalapack-mpi-dev libblacs-mpi-dev libatlas-base-dev libatlas-dev)
-- PETSc       : 3.7.3 (with HYPRE and ML) compiled locally (using system BLAS)
-- MUMPS       : 5.1.1 compiled locally (using CA scotch-6.0.4 and metis-5.1.0)
"""

import waf_seq

def configure(self):
    opts = self.options
    waf_seq.configure(self)

    self.env['LINKFLAGS_MATH'] = ['-lscalapack-openmpi',
                                  '-lblacsF77init-openmpi',
                                  '-lblacs-openmpi',
                                  '-lblacsF77init-openmpi',]

    self.env.prepend_value('LIBPATH', [
        '/opt/aster/public/mumps-5.1.1_mpi/lib',
        '/opt/aster/public/petsc-3.7.3/arch-linux2-c-opt/lib'
    ])

    self.env.prepend_value('INCLUDES', [
        '/opt/aster/public/mumps-5.1.1_mpi/include',
        '/opt/aster/public/petsc-3.7.3/arch-linux2-c-opt/include',
        '/opt/aster/public/petsc-3.7.3/include'
    ])

    self.env.append_value('LIB', ('X11',))

    opts.parallel = True

    opts.enable_mumps  = True
    opts.mumps_version = '5.1.1'
    opts.mumps_libs = 'dmumps zmumps smumps cmumps mumps_common pord metis'
    opts.embed_mumps   = True

    opts.enable_petsc = True
    opts.petsc_libs='ml HYPRE'
    opts.embed_petsc = True

The log from the terminal is also included in the attached archive (waf_asterMPI.log).

If anyone can tell me of their success compiling CA 13.4 on a similar system (Ubuntu 16.04) - including which versions of OpenBLAS, ScaLAPACK, OpenMPI and PETSc you used - I would be immensely grateful.

Many thanks,
Will.

Last edited by will.logie (2018-09-17 12:08:02)


Attachments:
log.tar.gz, Size: 26.71 KiB, Downloads: 2

Solar Thermal Group
Research School of Engineering
Australian National University
http://stg.anu.edu.au/

Offline

#2 2018-09-18 08:06:50

will.logie
Member
From: Canberra
Registered: 2017-09-23
Posts: 10

Re: Parallel install of CA 13.4.0 on Ubuntu 16.04 (Linux Mint 18.3)

I must pay my dues to Tianyi who has done some excellent work (github.com/tianyikillua). The issue of my post above is superseded.

Cheers,
Will.


Solar Thermal Group
Research School of Engineering
Australian National University
http://stg.anu.edu.au/

Offline

#3 2018-09-18 11:26:22

tianyikillua
Member
From: Paris
Registered: 2017-11-06
Posts: 45

Re: Parallel install of CA 13.4.0 on Ubuntu 16.04 (Linux Mint 18.3)

Nice to know that it helps!!

Many compiling tricks can be found within the open-sourced Dockerfile

github.com/tianyikillua/code_aster_on_docker/blob/master/code_aster/Dockerfile

For the blacs problem, the trick is "cp libscalapack.so libblacs.so"

Some patch files are also useful

github.com/tianyikillua/code_aster_on_docker/blob/master/code_aster/data/patch_mumps

Cheeeers,

Tianyi

Offline

Board footer